zoukankan      html  css  js  c++  java
  • 机器学习中使用的神经网络(三)

    神经元的简单模型

    Idealized neurons
    • To model things we have to idealize them (e.g. atoms)
    – Idealization removes complicated details that are not essential for understanding the main principles.
    – It allows us to apply mathematics and to make analogies to other, familiar systems.
    – Once we understand the basic principles, its easy to add complexity to make the model more faithful.
    • It is often worth understanding models that are known to be wrong (but we must not forget that they are wrong!)
    – E.g. neurons that communicate real values rather than discrete spikes of activity.
    Linear neurons
    • These are simple but computationally limited
    – If we can make them learn we may get insight into more complicated neurons

    Binary threshold neurons 二值阈值神经元
    McCulloch-Pitts (1943): influenced Von Neumann.
    – First compute a weighted sum of the inputs.
    – Then send out a fixed size spike of activity if the weighted sum exceeds a threshold.
    – McCulloch and Pitts thought that each spike is like the truth value of a proposition and compute the truth value of another proposition!


    There are two equivalent ways to write the equations for a binary threshold neuron:

    Rectified Linear Neurons
    (sometimes called linear threshold neurons)
    They compute a linear weighted sum of their inputs.
    The output is a non-linear function of the total input


    Sigmoid neurons 这个神经元经常使用
    These give a real-valued output that is a smooth and bounded function of their total input.
    – Typically they use the logistic function
    – They have nice smooth derivatives, the derivatives change continuously  and they're nicely behaved and they make it easy to do learning.



    Stochastic binary neurons 随机二进制神经元

    These use the same equations as logistic units.
    – But they treat the output of the logistic as the probability of producing a spike in a short time window.

    Instead of outputtiing that probability as a real number they actually make a probabilistic decision, and so what they acutally output is either a one or a zero. They're intrisically random. So they're treating the P as the probability of producing a one, not as a real number.

    • We can do a similar trick for rectified(改正的) linear units:
    – The output is treated as the Poisson rate for spikes.

    So the rectified linear unit determines the rate, but intrinsic randomness in the unit determines when the spikes are actually produced.

  • 相关阅读:
    前端知识笔记
    Vue 组件设计
    使用 puppeteer 创建一个自动化导出 PDF 的服务
    解决浏览器缓存导致页面非最新的小技巧
    浏览器文件上传浅淡
    Vue 项目代理设置的优化
    「django2」macos系统下安装及创建工程
    「android」gomobile argument unused during compilation: '-stdlib=libc++'
    「linux」大文本文件中查找指定字符串并删除所在行
    「color」颜色RGB
  • 原文地址:https://www.cnblogs.com/jinee/p/4472585.html
Copyright © 2011-2022 走看看