caffe笔记之例程学习

学习notebook自带例程Classification with HDF5 data时遇到了一些问题，认真把模型文件看了一遍。

模型定义中有一点比较容易被误解，信号在有向图中是自下而上流动的，并不是自上而下。

层的结构定义如下：

 name:层名称 2 type:层类型 3 top:出口 4 bottom:入口 

Each layer type defines three critical computations: setup, forward, and backward.

• Setup: initialize the layer and its connections once at model initialization.
• Forward: given input from bottom compute the output and send to the top.
• Backward: given the gradient w.r.t. the top output compute the gradient w.r.t. to the input and send to the bottom. A layer with parameters computes the gradient w.r.t. to its parameters and stores it internally.

From: caffe_root/examples/hdf5_classification/train_val.prototxt

#From: caffe_root/examples/hdf5_classification/train_val.prototxt
#计算Logistic函数
name: "LogisticRegressionNet" 
#训练数据层，put in HDF5Data
layer {
  name: "data"
  type: "HDF5Data"
  top: "data"
  top: "label"
  include {
    phase: TRAIN
  }
  hdf5_data_param {
    source: "examples/hdf5_classification/data/train.txt"
    batch_size: 10
  }
}
#测试数据层，put in HDF5Data
layer {
  name: "data"
  type: "HDF5Data"
  top: "data"
  top: "label"
  include {
    phase: TEST
  }
  hdf5_data_param {
    source: "examples/hdf5_classification/data/test.txt"
    batch_size: 10
  }
}
#全连接层
layer {
  name: "fc1"
  type: "InnerProduct"
  bottom: "data"
  top: "fc1"
  param {
    lr_mult: 1        # learning rate multiplier for the filters
    decay_mult: 1    # learning rate multiplier for the biases
  }
  param {
    lr_mult: 2        # weight decay multiplier for the filters
    decay_mult: 0    # weight decay multiplier for the biases
  }
  inner_product_param {
    num_output: 2
    weight_filler {
      type: "gaussian"
      std: 0.01
    }
    bias_filler {
      type: "constant"
      value: 0
    }
  }
}
#loss函数层
layer {
  name: "loss"
  type: "SoftmaxWithLoss"    #计算loss函数的方法
  bottom: "fc1"
  bottom: "label"
  top: "loss"
}
#ACCURACY层
layer {
  name: "accuracy"
  type: "Accuracy"
  bottom: "fc1"
  bottom: "label"
  top: "accuracy"
  include {
    phase: TEST
  }
}

模型详解（实在懒得翻译了）：

1.HD5 Input:

HDF5 Input

    LayerType: HDF5_DATA
    Parameters
        Required
            source: the name of the file to read from
            batch_size

2.Inner Product(全连接层，计算向量内积):

Inner Product

    LayerType: INNER_PRODUCT
    CPU implementation: ./src/caffe/layers/inner_product_layer.cpp
    CUDA GPU implementation: ./src/caffe/layers/inner_product_layer.cu
    Parameters (InnerProductParameter inner_product_param)
        Required
            num_output (c_o): the number of filters
        Strongly recommended
            weight_filler [default type: 'constant' value: 0]
        Optional
            bias_filler [default type: 'constant' value: 0]
            bias_term [default true]: specifies whether to learn and apply a set of additive biases to the filter outputs
    Input
        n * c_i * h_i * w_i
    Output
        n * c_o * 1 * 1

    Sample

    layers {
      name: "fc8"
      type: INNER_PRODUCT
      blobs_lr: 1          # learning rate multiplier for the filters
      blobs_lr: 2          # learning rate multiplier for the biases
      weight_decay: 1      # weight decay multiplier for the filters
      weight_decay: 0      # weight decay multiplier for the biases
      inner_product_param {
        num_output: 1000
        weight_filler {
          type: "gaussian"
          std: 0.01
        }
        bias_filler {
          type: "constant"
          value: 0
        }
      }
      bottom: "fc7"
      top: "fc8"
    }

The INNER_PRODUCT layer (also usually referred to as the fully connected layer) treats the input as a simple vector and produces an output in the form of a single vector (with the blob’s height and width set to 1).

3.Loss(最基本的loss函数):

Loss

In Caffe, as in most of machine learning, learning is driven by a loss function (also known as an error, cost, or objective function). A loss function specifies the goal of learning by mapping parameter settings (i.e., the current network weights) to a scalar value specifying the “badness” of these parameter settings. Hence, the goal of learning is to find a setting of the weights that minimizes the loss function.

The loss in Caffe is computed by the Forward pass of the network. Each layer takes a set of input (bottom) blobs and produces a set of output (top) blobs. Some of these layers’ outputs may be used in the loss function. A typical choice of loss function for one-versus-all classification tasks is the SOFTMAX_LOSS function, used in a network definition as follows, for example:
/***********************************************************************/
layers {
  name: "loss"
  type: SOFTMAX_LOSS
  bottom: "pred"
  bottom: "label"
  top: "loss"
}

In a SOFTMAX_LOSS function, the top blob is a scalar (dimensions 1×1×1×1) which averages the loss (computed from predicted labels pred and actuals labels label) over the entire mini-batch.

4.Accuracy(和loss功能类似，但只是简单做差，并不求和):

Accuracy and Top-k
 
ACCURACY scores the output as the accuracy of output with respect to target – it is not actually a loss and has no backward step.
 Activation / Neuron Layers
 
In general, activation / Neuron layers are element-wise operators, taking one bottom blob and producing one top blob of the same size. In the layers below, we will ignore the input and out sizes as they are identical:
 
    Input
        n * c * h * w
    Output
        n * c * h * w

嗯。这是所有例程中最简单的一个模型。天要亮了，洗把脸去。

References:

1.caffe.berkeleyvision.org/tutorial/forward_backward.html

2.caffe.berkeleyvision.org/tutorial/layers.html

3.nbviewer.ipython.org/github/BVLC/caffe/blob/master/examples/hdf5_classification.ipynb