Linear neuron:
[y=b+sumlimits_i{x_i w_i}
]
Binary threshold neuron:
[z = sumlimits_i{x_i w_i}
]
[y=left{egin{aligned} 1,~~~~~~~zgt heta \ 0,otherwiseend{aligned}
ight.
]
也可以写作：
[z = sumlimits_i{x_i w_i}
]
[y=left{egin{aligned} 1,~~~~~~~zgt 0 \ 0, otherwiseend{aligned}
ight.
]
Rectified linear neuron:
[z=b+sumlimits_i{x_i w_i}
]
[y=left{egin{aligned} 1,~~~~~~~zgt 0 \ 0, otherwiseend{aligned}
ight.
]
Sigmoid neuron:
[z=b+sumlimits_i{x_i w_i}
]
[y=frac{1}{1+e^{-z}}
]
Stochastic binary neuron:
we treat the output of the logistic as the probability of producing (s=1).
[z = sumlimits_i{x_i w_i}
]
[P(s=1)=frac{1}{1+e^{-z}}
]
Note：
- (x): neuron input.
- (w): input weight matrix.
- (b): bias term.
- (y): neuron output .