zoukankan      html  css  js  c++  java
  • caffe:用自己的数据训练网络mnist

    画黑底白字的软件:KolourPaint。

    假设所有“1”的图片放到名字为1的文件夹下。(0-9类似)。。获取每个数字的名称文件后,手动表上标签。然后合成train。txt

    1、获取文件夹内全部图像的名称:

    find ./1 -name '*.png'>1.txt

    //此时的1.txt文件中的图像名称包括路劲信息,要把前面的路径信息去掉。

    $ sudo sed -i "s/./1///g" 1.txt          //(表示转义,所以这里用双引号而不是单引号)

    2、要在1.txt 内的每个名称后面加上标签

    1.txt:

    1101.png  1

    1102.png  1

    .....(如此)

    3、将图片数据转换为lmdb格式的数据

    caffe/examples下建一个文件保存训练用的文件:sd_mnist

    3.1 sd_mnist下创建一个sd_create_lmdb.sh用来转换图片格式:

    sudo vim sd_create_lmdb.sh  ,内容如下:

    #!/usr/bin/env sh
    # Create the imagenet lmdb inputs
    # N.B. set the path to the imagenet train + val data dirs


    EXAMPLE=examples/sd_mnist       (!注意:这是你在examples下创建的目录)
    DATA=data/sd_mnist      (!注意:就是你在data文件夹下新建目录,里面有两个图片集(训练和测试训练集)及上面所说的两个txt)
    TOOLS=build/tools


    TRAIN_DATA_ROOT=data/sd_mnist/train/     (!注意:就是训练图片集路径)
    VAL_DATA_ROOT=data/sd_mnist/test/      (!注意:就是测试图片集路径)


    # Set RESIZE=true to resize the images to 256x256. Leave as false if images have
    # already been resized using another tool.
    RESIZE=true
    if $RESIZE; then
    RESIZE_HEIGHT=28
    RESIZE_WIDTH=28
    else
    RESIZE_HEIGHT=0
    RESIZE_WIDTH=0
    fi


    if [ ! -d "$TRAIN_DATA_ROOT" ]; then
    echo "Error: TRAIN_DATA_ROOT is not a path to a directory: $TRAIN_DATA_ROOT"
    echo "Set the TRAIN_DATA_ROOT variable in create_imagenet.sh to the path"
    "where the ImageNet training data is stored."
    exit 1
    fi


    if [ ! -d "$VAL_DATA_ROOT" ]; then
    echo "Error: VAL_DATA_ROOT is not a path to a directory: $VAL_DATA_ROOT"
    echo "Set the VAL_DATA_ROOT variable in create_imagenet.sh to the path"
    "where the ImageNet validation data is stored."
    exit 1
    fi


    echo "Creating train lmdb..."


    GLOG_logtostderr=1 $TOOLS/convert_imageset
    --resize_height=$RESIZE_HEIGHT
    --resize_width=$RESIZE_WIDTH
    --shuffle
    $TRAIN_DATA_ROOT
    $DATA/train.txt                (!注意路劲)
    $EXAMPLE/mnist_train_lmdb


    echo "Creating test lmdb..."


    GLOG_logtostderr=1 $TOOLS/convert_imageset
    --resize_height=$RESIZE_HEIGHT
    --resize_width=$RESIZE_WIDTH
    --shuffle
    $VAL_DATA_ROOT
    $DATA/test.txt          (!注意路劲)
    $EXAMPLE/mnist_test_lmdb


    echo "Done."

    -----------------------------------------------------------------------

    3.2 运行sh example/sd_mnist/sd_create_lmdb.sh

    如果成功的话,终端返回的信息中,图片是有大小的而不是0kb。并且在examples/sd_mnist下会有两个文件:mnist_train_lmdb,mnist_test_lmdb它们里面都是data.mdb和lock.mdb。

    4、对我们的数据集进行训练:下面的文件都是从caffeexamplesmnist下复制到caffeexamplessd_mnist下来进行修改的。主要是修改路径信息,整个网络保持不变。

    4.1第一个sh文件是train_lenet,sh

    #!/usr/bin/env sh
    set -e

    ./build/tools/caffe train --solver=examples/sd_mnist/lenet_solver.prototxt $@

    4.2、复制lenet_solver.prototxt文件,并修改:

    # The train/test net protocol buffer definition
    net: "examples/sd_mnist/lenet_train_test.prototxt"
    # test_iter specifies how many forward passes the test should carry out.
    # In the case of MNIST, we have test batch size 100 and 100 test iterations,
    # covering the full 10,000 testing images.
    test_iter: 100
    # Carry out testing every 500 training iterations.
    test_interval: 500
    # The base learning rate, momentum and the weight decay of the network.
    base_lr: 0.01
    momentum: 0.9
    weight_decay: 0.0005
    # The learning rate policy
    lr_policy: "inv"
    gamma: 0.0001
    power: 0.75
    # Display every 100 iterations
    display: 100
    # The maximum number of iterations
    max_iter: 10000
    # snapshot intermediate results
    snapshot: 5000
    snapshot_prefix: "examples/sd_mnist/lenet"
    # solver mode: CPU or GPU
    solver_mode: CPU

    4.3、lenet_train_test.prototxt复制从mnist文件夹到当前文件夹下

    修改路径

    name: "LeNet"
    layer {
    name: "mnist"
    type: "Data"
    top: "data"
    top: "label"
    include {
    phase: TRAIN
    }
    transform_param {
    scale: 0.00390625
    }
    data_param {
    source: "examples/sd_mnist/mnist_train_lmdb"
    batch_size: 64
    backend: LMDB
    }
    }
    layer {
    name: "mnist"
    type: "Data"
    top: "data"
    top: "label"
    include {
    phase: TEST
    }
    transform_param {
    scale: 0.00390625
    }
    data_param {
    source: "examples/sd_mnist/mnist_test_lmdb"
    batch_size: 100
    backend: LMDB
    }
    }
    layer {
    name: "conv1"
    type: "Convolution"
    bottom: "data"
    top: "conv1"
    param {
    lr_mult: 1
    }
    param {
    lr_mult: 2
    }
    convolution_param {
    num_output: 20
    kernel_size: 5
    stride: 1
    weight_filler {
    type: "xavier"
    }
    bias_filler {
    type: "constant"
    }
    }
    }
    layer {
    name: "pool1"
    type: "Pooling"
    bottom: "conv1"
    top: "pool1"
    pooling_param {
    pool: MAX
    kernel_size: 2
    stride: 2
    }
    }
    layer {
    name: "conv2"
    type: "Convolution"
    bottom: "pool1"
    top: "conv2"
    param {
    lr_mult: 1
    }
    param {
    lr_mult: 2
    }
    convolution_param {
    num_output: 50
    kernel_size: 5
    stride: 1
    weight_filler {
    type: "xavier"
    }
    bias_filler {
    type: "constant"
    }
    }
    }
    layer {
    name: "pool2"
    type: "Pooling"
    bottom: "conv2"
    top: "pool2"
    pooling_param {
    pool: MAX
    kernel_size: 2
    stride: 2
    }
    }
    layer {
    name: "ip1"
    type: "InnerProduct"
    bottom: "pool2"
    top: "ip1"
    param {
    lr_mult: 1
    }
    param {
    lr_mult: 2
    }
    inner_product_param {
    num_output: 500
    weight_filler {
    type: "xavier"
    }
    bias_filler {
    type: "constant"
    }
    }
    }
    layer {
    name: "relu1"
    type: "ReLU"
    bottom: "ip1"
    top: "ip1"
    }
    layer {
    name: "ip2"
    type: "InnerProduct"
    bottom: "ip1"
    top: "ip2"
    param {
    lr_mult: 1
    }
    param {
    lr_mult: 2
    }
    inner_product_param {
    num_output: 10
    weight_filler {
    type: "xavier"
    }
    bias_filler {
    type: "constant"
    }
    }
    }
    layer {
    name: "accuracy"
    type: "Accuracy"
    bottom: "ip2"
    bottom: "label"
    top: "accuracy"
    include {
    phase: TEST
    }
    }
    layer {
    name: "loss"
    type: "SoftmaxWithLoss"
    bottom: "ip2"
    bottom: "label"
    top: "loss"
    }

    4.4 lenet.prototxt复制从mnist文件夹到当前文件夹下,不用修改

    4.5 运行 sh example/sd_mnist/train_lenet.sh

    没报错,出来accuracy loss这些,说明成功!!

    参考:http://blog.csdn.net/xiaoxiao_huitailang/article/details/51361036

  • 相关阅读:
    如何使用jetty
    windows安装TortoiseGit详细使用教程【基础篇】
    shiro实现APP、web统一登录认证和权限管理
    Eclipse上安装GIT插件EGit及使用
    RPC之——HTTP协议栈
    Spring Boot构建RESTful API与单元测试
    Spring 之注解事务 @Transactional
    浅析Java中的final关键字
    String中intern的方法
    MySQL索引的查看创建和删除
  • 原文地址:https://www.cnblogs.com/beihaidao/p/6123387.html
Copyright © 2011-2022 走看看