zoukankan      html  css  js  c++  java
  • kaggle比赛之youtube视频分类示例

    1.训练模型:建bucket,建job,提交运行。

    BUCKET_NAME=gs://${USER}_yt8m_train_bucket_logisticmodel
    # (One Time) Create a storage bucket to store training logs and checkpoints.
    gsutil mb -l us-east1 $BUCKET_NAME
    # Submit the training job.
    JOB_NAME=yt8m_train_LogisticModel$(date +%Y%m%d_%H%M%S); gcloud --verbosity=debug ml-engine jobs 
    submit training $JOB_NAME 
    --package-path=youtube-8m --module-name=youtube-8m.train 
    --staging-bucket=$BUCKET_NAME --region=us-east1 
    --config=youtube-8m/cloudml-gpu.yaml 
    -- --train_data_pattern='gs://youtube8m-ml-us-east1/1/video_level/train/train*.tfrecord' 
    --model=LogisticModel 
    --train_dir=$BUCKET_NAME/yt8m_train_video_level_logistic_model
    
    
    
    BUCKET_NAME=gs://${USER}_yt8m_train_bucket_lstmmodel
    gsutil mb -l us-east1 $BUCKET_NAME
    JOB_NAME=yt8m_train_LstmModel$(date +%Y%m%d_%H%M%S); gcloud --verbosity=debug ml-engine jobs 
    submit training $JOB_NAME 
    --package-path=youtube-8m --module-name=youtube-8m.train 
    --staging-bucket=$BUCKET_NAME --region=us-east1 
    --config=youtube-8m/cloudml-gpu.yaml 
    -- --train_data_pattern='gs://youtube8m-ml-us-east1/1/frame_level/train/train*.tfrecord' 
    --frame_features=True --model=LstmModel --feature_names="rgb" 
    --feature_sizes="1024" --batch_size=128 
    --train_dir=$BUCKET_NAME/yt8m_train_frame_level_lstmModel
    
    
    BUCKET_NAME=gs://${USER}_yt8m_train_bucket_framelevellogisticmodel
    gsutil mb -l us-east1 $BUCKET_NAME
    JOB_NAME=yt8m_train_FrameLevelLogisticModel$(date +%Y%m%d_%H%M%S); gcloud --verbosity=debug ml-engine jobs 
    submit training $JOB_NAME 
    --package-path=youtube-8m --module-name=youtube-8m.train 
    --staging-bucket=$BUCKET_NAME --region=us-east1 
    --config=youtube-8m/cloudml-gpu.yaml 
    -- --train_data_pattern='gs://youtube8m-ml-us-east1/1/frame_level/train/train*.tfrecord' 
    --frame_features=True --model=FrameLevelLogisticModel --feature_names="rgb" 
    --feature_sizes="1024" --batch_size=128 
    --train_dir=$BUCKET_NAME/yt8m_train_video_framelevel_logisticmodel
    
    
    BUCKET_NAME=gs://${USER}_yt8m_train_bucket_dbofmodel
    gsutil mb -l us-east1 $BUCKET_NAME
    JOB_NAME=yt8m_train_DbofModel$(date +%Y%m%d_%H%M%S); gcloud --verbosity=debug ml-engine jobs 
    submit training $JOB_NAME 
    --package-path=youtube-8m --module-name=youtube-8m.train 
    --staging-bucket=$BUCKET_NAME --region=us-east1 
    --config=youtube-8m/cloudml-gpu.yaml 
    -- --train_data_pattern='gs://youtube8m-ml-us-east1/1/frame_level/train/train*.tfrecord' 
    --frame_features=True --model=DbofModel --feature_names="rgb" 
    --feature_sizes="1024" --batch_size=128 
    --train_dir=$BUCKET_NAME/yt8m_train_frame_level_dbofmodel

    2.查看log,训练过程

    点击侧边栏的logging可以查看程序输出。

    tensorboard:https://cloud.google.com/ml-engine/docs/how-tos/getting-started-training-prediction#tensorboard-local

    OUTPUT=$BUCKET_NAME/yt8m_train_video_framelevel_logisticmodel       (就是填入train_dir的内容)
    python -m tensorflow.tensorboard --logdir=$OUTPUT --port=8080

    Select "Preview on port 8080" from the Web Preview menu at the top of the command-line.

    3.在测试集上进行测试:

    JOB_TO_EVAL=yt8m_train_video_level_logistic_model
    JOB_NAME=yt8m_inference_$(date +%Y%m%d_%H%M%S); gcloud --verbosity=debug ml-engine jobs 
    submit training $JOB_NAME 
    --package-path=youtube-8m --module-name=youtube-8m.inference 
    --staging-bucket=$BUCKET_NAME --region=us-east1 
    --config=youtube-8m/cloudml-gpu.yaml 
    -- --input_data_pattern='gs://youtube8m-ml/1/video_level/test/test*.tfrecord' 
    --train_dir=$BUCKET_NAME/${JOB_TO_EVAL} 
    --output_file=$BUCKET_NAME/${JOB_TO_EVAL}/predictions.csv
    
    JOB_NAME=yt8m_dbofmodel_inference_$(date +%Y%m%d_%H%M%S); gcloud --verbosity=debug ml-engine jobs 
    submit training $JOB_NAME 
    --package-path=youtube-8m --module-name=youtube-8m.inference 
    --staging-bucket=$BUCKET_NAME --region=us-east1 
    --config=youtube-8m/cloudml-gpu.yaml 
    -- --input_data_pattern='gs://youtube8m-ml-us-east1/1/frame_level/test/test*.tfrecord' 
    --frame_features=True --model=FrameLevelLogisticModel --feature_names="rgb" 
    --feature_sizes="1024" --batch_size=128 
    --train_dir=$BUCKET_NAME/${JOB_TO_EVAL} 
    --output_file=$BUCKET_NAME/${JOB_TO_EVAL}/predictions.csv
    
    JOB_NAME=yt8m_framelevellogistic_inference_$(date +%Y%m%d_%H%M%S); gcloud --verbosity=debug ml-engine jobs 
    submit training $JOB_NAME 
    --package-path=youtube-8m --module-name=youtube-8m.inference 
    --staging-bucket=$BUCKET_NAME --region=us-east1 
    --config=youtube-8m/cloudml-gpu.yaml 
    -- --input_data_pattern='gs://youtube8m-ml-us-east1/1/frame_level/test/test*.tfrecord' 
    --frame_features=True --model=FrameLevelLogisticModel --feature_names="rgb" 
    --feature_sizes="1024" --batch_size=128 
    --train_dir=$BUCKET_NAME/${JOB_TO_EVAL} 
    --output_file=$BUCKET_NAME/${JOB_TO_EVAL}/predictions.csv
  • 相关阅读:
    在定义SharePoint列表的SPD数据视图的时候需要注意的问题
    如何自定义改变SharePoint 中列表Web部件中所有行某列中的固定值为图片或其它HTML代码
    [C#3] 1扩展方法
    特效编辑器开发手记2——cocos2dx粒子系统的plist文件 深圳
    让人死去活来的cocos2dx安卓开发环境搭建(windows+eclipse+ndk 不用cygwin)【上图】 深圳
    《疾风》开发手记:NxOgre最新版本的搭建20111020 深圳
    Linux 操作系统下CPU多核心的绑定 深圳
    巧用Unix时间戳 深圳
    AS3加载AS2的swf文件报错 深圳
    GLUT函数说明(转载) 深圳
  • 原文地址:https://www.cnblogs.com/huangshiyu13/p/6685639.html
Copyright © 2011-2022 走看看