【第三课】kaggle案例分析三

Evernote Export

比赛题目介绍

• TalkingData是中国最大的第三方移动数据平台，移动设备用户日常的选择和行为用户画像。目前，TalkingData正在寻求每天在中国活跃的5亿移动设备70%以上的行为数据，帮助客户更好的了解与其用户互动。
• TalkingData提供了大约20万用户的数据(全部真实且经过脱敏处理)，按照年龄和性别分成12个组，比如男性22到25岁，女性30到35岁，同时提供了用户行为属性，比如在什么样的时间点出现什么样的地理位置等等，选手通过这些信息去推测用户是分在哪一个性别年龄里面。
• 什么是用户画像
• 用户画像可以理解为就是为用户画标签，标签是不固定的，依照企业业务场景动态添加、删除等，有的标签是永久的，有的标签是可以动态变更的
• 用户画像可以看做是综合性的标签系统，在自身拥有(或从第三方获得)的原始数据中，从多个维度对用户进行产品进行刻画，提取出商业价值潜力的语义信息，是常见的企业级大数据应用
• 对于人常见标签与维度，地域、年龄、性别、文化、职业、收入、生活习惯、消费习惯等
• 对于产品常见刻画维度，产品类别、活跃频率、产品喜好、产品驱动、使用习惯、产品消费等
• 技术栈：无监督学习、半监督学习(技巧性很强)，大多数是有监督学习
• 用户画像的作用
• 精准营销，分析产品潜在客户，针对特定群体利用短信邮件等方式进行营销
• 用户统计，比如中国大学购买书籍人数TOP10，全国分城市指数等
• 数据挖掘，构建智能推荐系统，利用关联规则计算，喜欢红酒的通常喜欢什么运动品牌，利用聚类算法分析，喜欢红酒的人的年龄段分布情况
• 进行效果评估，完善产品运用，提升服务质量，其实这也就是相当于市场调研、用户调研，迅速下定位服务群体，提供高水平的服务
• 对服务或产品进行私人定制，即个性化的服务某类群体甚至每一位用户(这是未来的消费趋势)比如，某公司退出一款5-10岁儿童的玩具，通过用户画像分析，发现某些特征的比重最大，就给产品提供了非常可观的决策依据
• 业务经营分析以及竞争分析，影响企业发展战略

人工神经网络原理

• 知识地图
• 从单层感知器到多层感知器
• 从多层感知器到自编码器
• 从多层感知器到卷积神经网络，再到深度残差网络
• 从多层感知器到递归神经网络，再到LSTM
• 从单层感知器到Hopfield神经网络，再到Bolazmann机和RBM
• 用RBM堆叠成DBN，DBN与多层自编码器结合成DBN-DNN
• 神经网络要素
• 网路结构(全连接、分层、有时滞回路、权值共享、激活函数)
• 运行机制(异步更新、同步更新、前馈)
• 训练算法，训练中使用的trick(mini batch BN drop out等)，损失函数的定义
• 训练数据(数据的预处理，输入和输出的构成等)
• 单层感知器
• 输入节点
• 输出节点
• 权向量
• 偏置因子
• 激活函数
• 学习率
• 单层感知器学习算法
• 单层感知器的局限
  激活函数

sng(w1​x1​+w2​x2​+w3x3​)=0

单层感知器类比于线性分类器

感知器学习的规则

1958年，首先定义了一个具有单层计算单元的神经网络结构，称为感知器
感知器的学习规则规定，学习信号等于神经元期望输出(教师信号)与实际输出之差

r=dj​−oj​

式中，dj​为期望的输出，oj​=f(WjT​X)感知器采用了与阈值转移函数类似的符号转移函数，其表达式为

f(WjT​X)=sgn(WjT​X)={1,−1,​(WjT​≥0)(WjT​<0)​

因此，权值调整公式为：

ΔWj​=±2ηX

感知器学习规则只适用于二进制神经元，初始值可取任意值。
感知器学习规则代表一种有导师学习，由于感知器理论是研究其他神经网络的基础，该规则对于神经网络的有导师学习具有极为重要的意义。

多层前馈神经网络（BP网络）

• 隐藏层与隐藏节点
• 前馈-----每一层的节点仅和下一层节点相连

BP学习算法本质就是梯度下降法

神经网络的两个过程：1.训练过程 2.推断过程

小结：数学背景：梯度下降法(使用误差平方和构造误差函数)
梯度下降法给出权值，通过反向传播误差逐层训练，推断时误差往后走，构造完后给出权值，误差信号通过公式计算梯度，得出梯度对权值进行迭代改变，知道权值不再变化或者次数足够收敛，就结束整个训练过程

在python深度学习中，可以使用keras进行简单的神经网络等设计
keras框架是简单方便的，模块可以作为数据或序列的运算图等，keras中的各个设计环节都是单独的一个模块，所以需要根据自己的需求来设计
keras中文文档
keras默认使用tensorflow，如果使用theano需要修改文件中的路径内的json文件

keras的基本使用

• Sequential 模型构建
• 序贯模型是多个网络层的线性堆叠
• 可以通过向Sequential模型传递一个layer的list来构造该模型

from keras.models import Sequential
from keras.layers import Dense,Activation
model = Sequential([    
Dense(32,units=784),    
Activation('relu'),    
Dense(10),    
Activation('softmax'),])

• 也可以通过.add()方法一个个的将layer加入模型中

model = Sequential()
model.add(Dense(32,input_shape=(784,))
)model.add(Activation('relu'))

• 指定输入数据的shape
• input shape（元祖型数据）
• input dim

model = Sequential()
model.add(Dense(32,input_dim=784))

• input length

model = Sequentail()
model.add(Dense(32,input_shape=784))

• batch_size

评分标准

• 选手算出用户在不同分组上的概率，现实中一个用户只能在一个分组。理想状态下如果能算出这个概率是1，其他是0的话，这个答案就是没有任何损失的。一般来说，提交的答案中，某个用户会有或大或小的概率属于多个组别，这个时候就有概率上的损失，这个损失的高低代表答案的水平。我们优化函数或者说是评估指标就是下面的损失函数

logloss=−N1​i=1∑N​j=1∑M​yij​log(pij​)

max(min(p,1−10−15),10−15)

%23%23%23%20%E6%AF%94%E8%B5%9B%E9%A2%98%E7%9B%AE%E4%BB%8B%E7%BB%8D%0A*%20TalkingData%E6%98%AF%E4%B8%AD%E5%9B%BD%E6%9C%80%E5%A4%A7%E7%9A%84%E7%AC%AC%E4%B8%89%E6%96%B9%E7%A7%BB%E5%8A%A8%E6%95%B0%E6%8D%AE%E5%B9%B3%E5%8F%B0%EF%BC%8C%E7%A7%BB%E5%8A%A8%E8%AE%BE%E5%A4%87%E7%94%A8%E6%88%B7%E6%97%A5%E5%B8%B8%E7%9A%84%E9%80%89%E6%8B%A9%E5%92%8C%E8%A1%8C%E4%B8%BA%E7%94%A8%E6%88%B7%E7%94%BB%E5%83%8F%E3%80%82%E7%9B%AE%E5%89%8D%EF%BC%8CTalkingData%E6%AD%A3%E5%9C%A8%E5%AF%BB%E6%B1%82%E6%AF%8F%E5%A4%A9%E5%9C%A8%E4%B8%AD%E5%9B%BD%E6%B4%BB%E8%B7%83%E7%9A%845%E4%BA%BF%E7%A7%BB%E5%8A%A8%E8%AE%BE%E5%A4%8770%25%E4%BB%A5%E4%B8%8A%E7%9A%84%E8%A1%8C%E4%B8%BA%E6%95%B0%E6%8D%AE%EF%BC%8C%E5%B8%AE%E5%8A%A9%E5%AE%A2%E6%88%B7%E6%9B%B4%E5%A5%BD%E7%9A%84%E4%BA%86%E8%A7%A3%E4%B8%8E%E5%85%B6%E7%94%A8%E6%88%B7%E4%BA%92%E5%8A%A8%E3%80%82%0A*%20TalkingData%E6%8F%90%E4%BE%9B%E4%BA%86%E5%A4%A7%E7%BA%A620%E4%B8%87%E7%94%A8%E6%88%B7%E7%9A%84%E6%95%B0%E6%8D%AE(%E5%85%A8%E9%83%A8%E7%9C%9F%E5%AE%9E%E4%B8%94%E7%BB%8F%E8%BF%87%E8%84%B1%E6%95%8F%E5%A4%84%E7%90%86)%EF%BC%8C%E6%8C%89%E7%85%A7%E5%B9%B4%E9%BE%84%E5%92%8C%E6%80%A7%E5%88%AB%E5%88%86%E6%88%9012%E4%B8%AA%E7%BB%84%EF%BC%8C%E6%AF%94%E5%A6%82%E7%94%B7%E6%80%A722%E5%88%B025%E5%B2%81%EF%BC%8C%E5%A5%B3%E6%80%A730%E5%88%B035%E5%B2%81%EF%BC%8C%E5%90%8C%E6%97%B6%E6%8F%90%E4%BE%9B%E4%BA%86%E7%94%A8%E6%88%B7%E8%A1%8C%E4%B8%BA%E5%B1%9E%E6%80%A7%EF%BC%8C%E6%AF%94%E5%A6%82%E5%9C%A8%E4%BB%80%E4%B9%88%E6%A0%B7%E7%9A%84%E6%97%B6%E9%97%B4%E7%82%B9%E5%87%BA%E7%8E%B0%E4%BB%80%E4%B9%88%E6%A0%B7%E7%9A%84%E5%9C%B0%E7%90%86%E4%BD%8D%E7%BD%AE%E7%AD%89%E7%AD%89%EF%BC%8C%E9%80%89%E6%89%8B%E9%80%9A%E8%BF%87%E8%BF%99%E4%BA%9B%E4%BF%A1%E6%81%AF%E5%8E%BB%E6%8E%A8%E6%B5%8B%E7%94%A8%E6%88%B7%E6%98%AF%E5%88%86%E5%9C%A8%E5%93%AA%E4%B8%80%E4%B8%AA%E6%80%A7%E5%88%AB%E5%B9%B4%E9%BE%84%E9%87%8C%E9%9D%A2%E3%80%82%0A*%20**%E4%BB%80%E4%B9%88%E6%98%AF%E7%94%A8%E6%88%B7%E7%94%BB%E5%83%8F**%0A*%20%E7%94%A8%E6%88%B7%E7%94%BB%E5%83%8F%E5%8F%AF%E4%BB%A5%E7%90%86%E8%A7%A3%E4%B8%BA%E5%B0%B1%E6%98%AF%E4%B8%BA%E7%94%A8%E6%88%B7%E7%94%BB%E6%A0%87%E7%AD%BE%EF%BC%8C%E6%A0%87%E7%AD%BE%E6%98%AF%E4%B8%8D%E5%9B%BA%E5%AE%9A%E7%9A%84%EF%BC%8C%E4%BE%9D%E7%85%A7%E4%BC%81%E4%B8%9A%E4%B8%9A%E5%8A%A1%E5%9C%BA%E6%99%AF%E5%8A%A8%E6%80%81%E6%B7%BB%E5%8A%A0%E3%80%81%E5%88%A0%E9%99%A4%E7%AD%89%EF%BC%8C%E6%9C%89%E7%9A%84%E6%A0%87%E7%AD%BE%E6%98%AF%E6%B0%B8%E4%B9%85%E7%9A%84%EF%BC%8C%E6%9C%89%E7%9A%84%E6%A0%87%E7%AD%BE%E6%98%AF%E5%8F%AF%E4%BB%A5%E5%8A%A8%E6%80%81%E5%8F%98%E6%9B%B4%E7%9A%84%0A*%20%E7%94%A8%E6%88%B7%E7%94%BB%E5%83%8F%E5%8F%AF%E4%BB%A5%E7%9C%8B%E5%81%9A%E6%98%AF%E7%BB%BC%E5%90%88%E6%80%A7%E7%9A%84%E6%A0%87%E7%AD%BE%E7%B3%BB%E7%BB%9F%EF%BC%8C%E5%9C%A8%E8%87%AA%E8%BA%AB%E6%8B%A5%E6%9C%89(%E6%88%96%E4%BB%8E%E7%AC%AC%E4%B8%89%E6%96%B9%E8%8E%B7%E5%BE%97)%E7%9A%84%E5%8E%9F%E5%A7%8B%E6%95%B0%E6%8D%AE%E4%B8%AD%EF%BC%8C%E4%BB%8E%E5%A4%9A%E4%B8%AA%E7%BB%B4%E5%BA%A6%E5%AF%B9%E7%94%A8%E6%88%B7%E8%BF%9B%E8%A1%8C%E4%BA%A7%E5%93%81%E8%BF%9B%E8%A1%8C%E5%88%BB%E7%94%BB%EF%BC%8C%E6%8F%90%E5%8F%96%E5%87%BA%E5%95%86%E4%B8%9A%E4%BB%B7%E5%80%BC%E6%BD%9C%E5%8A%9B%E7%9A%84%E8%AF%AD%E4%B9%89%E4%BF%A1%E6%81%AF%EF%BC%8C%E6%98%AF%E5%B8%B8%E8%A7%81%E7%9A%84%E4%BC%81%E4%B8%9A%E7%BA%A7%E5%A4%A7%E6%95%B0%E6%8D%AE%E5%BA%94%E7%94%A8%0A*%20%E5%AF%B9%E4%BA%8E%E4%BA%BA%E5%B8%B8%E8%A7%81%E6%A0%87%E7%AD%BE%E4%B8%8E%E7%BB%B4%E5%BA%A6%EF%BC%8C%E5%9C%B0%E5%9F%9F%E3%80%81%E5%B9%B4%E9%BE%84%E3%80%81%E6%80%A7%E5%88%AB%E3%80%81%E6%96%87%E5%8C%96%E3%80%81%E8%81%8C%E4%B8%9A%E3%80%81%E6%94%B6%E5%85%A5%E3%80%81%E7%94%9F%E6%B4%BB%E4%B9%A0%E6%83%AF%E3%80%81%E6%B6%88%E8%B4%B9%E4%B9%A0%E6%83%AF%E7%AD%89%0A*%20%E5%AF%B9%E4%BA%8E%E4%BA%A7%E5%93%81%E5%B8%B8%E8%A7%81%E5%88%BB%E7%94%BB%E7%BB%B4%E5%BA%A6%EF%BC%8C%E4%BA%A7%E5%93%81%E7%B1%BB%E5%88%AB%E3%80%81%E6%B4%BB%E8%B7%83%E9%A2%91%E7%8E%87%E3%80%81%E4%BA%A7%E5%93%81%E5%96%9C%E5%A5%BD%E3%80%81%E4%BA%A7%E5%93%81%E9%A9%B1%E5%8A%A8%E3%80%81%E4%BD%BF%E7%94%A8%E4%B9%A0%E6%83%AF%E3%80%81%E4%BA%A7%E5%93%81%E6%B6%88%E8%B4%B9%E7%AD%89%0A*%20%E6%8A%80%E6%9C%AF%E6%A0%88%EF%BC%9A%E6%97%A0%E7%9B%91%E7%9D%A3%E5%AD%A6%E4%B9%A0%E3%80%81%E5%8D%8A%E7%9B%91%E7%9D%A3%E5%AD%A6%E4%B9%A0(%E6%8A%80%E5%B7%A7%E6%80%A7%E5%BE%88%E5%BC%BA)%EF%BC%8C%E5%A4%A7%E5%A4%9A%E6%95%B0%E6%98%AF%E6%9C%89%E7%9B%91%E7%9D%A3%E5%AD%A6%E4%B9%A0%0A*%20**%E7%94%A8%E6%88%B7%E7%94%BB%E5%83%8F%E7%9A%84%E4%BD%9C%E7%94%A8**%0A*%20%E7%B2%BE%E5%87%86%E8%90%A5%E9%94%80%EF%BC%8C%E5%88%86%E6%9E%90%E4%BA%A7%E5%93%81%E6%BD%9C%E5%9C%A8%E5%AE%A2%E6%88%B7%EF%BC%8C%E9%92%88%E5%AF%B9%E7%89%B9%E5%AE%9A%E7%BE%A4%E4%BD%93%E5%88%A9%E7%94%A8%E7%9F%AD%E4%BF%A1%E9%82%AE%E4%BB%B6%E7%AD%89%E6%96%B9%E5%BC%8F%E8%BF%9B%E8%A1%8C%E8%90%A5%E9%94%80%0A*%20%E7%94%A8%E6%88%B7%E7%BB%9F%E8%AE%A1%EF%BC%8C%E6%AF%94%E5%A6%82%E4%B8%AD%E5%9B%BD%E5%A4%A7%E5%AD%A6%E8%B4%AD%E4%B9%B0%E4%B9%A6%E7%B1%8D%E4%BA%BA%E6%95%B0TOP10%EF%BC%8C%E5%85%A8%E5%9B%BD%E5%88%86%E5%9F%8E%E5%B8%82%E6%8C%87%E6%95%B0%E7%AD%89%0A*%20%E6%95%B0%E6%8D%AE%E6%8C%96%E6%8E%98%EF%BC%8C%E6%9E%84%E5%BB%BA%E6%99%BA%E8%83%BD%E6%8E%A8%E8%8D%90%E7%B3%BB%E7%BB%9F%EF%BC%8C%E5%88%A9%E7%94%A8%E5%85%B3%E8%81%94%E8%A7%84%E5%88%99%E8%AE%A1%E7%AE%97%EF%BC%8C%E5%96%9C%E6%AC%A2%E7%BA%A2%E9%85%92%E7%9A%84%E9%80%9A%E5%B8%B8%E5%96%9C%E6%AC%A2%E4%BB%80%E4%B9%88%E8%BF%90%E5%8A%A8%E5%93%81%E7%89%8C%EF%BC%8C%E5%88%A9%E7%94%A8%E8%81%9A%E7%B1%BB%E7%AE%97%E6%B3%95%E5%88%86%E6%9E%90%EF%BC%8C%E5%96%9C%E6%AC%A2%E7%BA%A2%E9%85%92%E7%9A%84%E4%BA%BA%E7%9A%84%E5%B9%B4%E9%BE%84%E6%AE%B5%E5%88%86%E5%B8%83%E6%83%85%E5%86%B5%0A*%20%E8%BF%9B%E8%A1%8C%E6%95%88%E6%9E%9C%E8%AF%84%E4%BC%B0%EF%BC%8C%E5%AE%8C%E5%96%84%E4%BA%A7%E5%93%81%E8%BF%90%E7%94%A8%EF%BC%8C%E6%8F%90%E5%8D%87%E6%9C%8D%E5%8A%A1%E8%B4%A8%E9%87%8F%EF%BC%8C%E5%85%B6%E5%AE%9E%E8%BF%99%E4%B9%9F%E5%B0%B1%E6%98%AF%E7%9B%B8%E5%BD%93%E4%BA%8E%E5%B8%82%E5%9C%BA%E8%B0%83%E7%A0%94%E3%80%81%E7%94%A8%E6%88%B7%E8%B0%83%E7%A0%94%EF%BC%8C%E8%BF%85%E9%80%9F%E4%B8%8B%E5%AE%9A%E4%BD%8D%E6%9C%8D%E5%8A%A1%E7%BE%A4%E4%BD%93%EF%BC%8C%E6%8F%90%E4%BE%9B%E9%AB%98%E6%B0%B4%E5%B9%B3%E7%9A%84%E6%9C%8D%E5%8A%A1%0A*%20%E5%AF%B9%E6%9C%8D%E5%8A%A1%E6%88%96%E4%BA%A7%E5%93%81%E8%BF%9B%E8%A1%8C%E7%A7%81%E4%BA%BA%E5%AE%9A%E5%88%B6%EF%BC%8C%E5%8D%B3%E4%B8%AA%E6%80%A7%E5%8C%96%E7%9A%84%E6%9C%8D%E5%8A%A1%E6%9F%90%E7%B1%BB%E7%BE%A4%E4%BD%93%E7%94%9A%E8%87%B3%E6%AF%8F%E4%B8%80%E4%BD%8D%E7%94%A8%E6%88%B7(%E8%BF%99%E6%98%AF%E6%9C%AA%E6%9D%A5%E7%9A%84%E6%B6%88%E8%B4%B9%E8%B6%8B%E5%8A%BF)%E6%AF%94%E5%A6%82%EF%BC%8C%E6%9F%90%E5%85%AC%E5%8F%B8%E9%80%80%E5%87%BA%E4%B8%80%E6%AC%BE5-10%E5%B2%81%E5%84%BF%E7%AB%A5%E7%9A%84%E7%8E%A9%E5%85%B7%EF%BC%8C%E9%80%9A%E8%BF%87%E7%94%A8%E6%88%B7%E7%94%BB%E5%83%8F%E5%88%86%E6%9E%90%EF%BC%8C%E5%8F%91%E7%8E%B0%E6%9F%90%E4%BA%9B%E7%89%B9%E5%BE%81%E7%9A%84%E6%AF%94%E9%87%8D%E6%9C%80%E5%A4%A7%EF%BC%8C%E5%B0%B1%E7%BB%99%E4%BA%A7%E5%93%81%E6%8F%90%E4%BE%9B%E4%BA%86%E9%9D%9E%E5%B8%B8%E5%8F%AF%E8%A7%82%E7%9A%84%E5%86%B3%E7%AD%96%E4%BE%9D%E6%8D%AE%0A*%20%E4%B8%9A%E5%8A%A1%E7%BB%8F%E8%90%A5%E5%88%86%E6%9E%90%E4%BB%A5%E5%8F%8A%E7%AB%9E%E4%BA%89%E5%88%86%E6%9E%90%EF%BC%8C%E5%BD%B1%E5%93%8D%E4%BC%81%E4%B8%9A%E5%8F%91%E5%B1%95%E6%88%98%E7%95%A5%0A%0A%23%23%23%23%20%E4%BA%BA%E5%B7%A5%E7%A5%9E%E7%BB%8F%E7%BD%91%E7%BB%9C%E5%8E%9F%E7%90%86%0A*%20**%E7%9F%A5%E8%AF%86%E5%9C%B0%E5%9B%BE**%0A*%20%E4%BB%8E%E5%8D%95%E5%B1%82%E6%84%9F%E7%9F%A5%E5%99%A8%E5%88%B0%E5%A4%9A%E5%B1%82%E6%84%9F%E7%9F%A5%E5%99%A8%0A*%20%E4%BB%8E%E5%A4%9A%E5%B1%82%E6%84%9F%E7%9F%A5%E5%99%A8%E5%88%B0%E8%87%AA%E7%BC%96%E7%A0%81%E5%99%A8%0A*%20%E4%BB%8E%E5%A4%9A%E5%B1%82%E6%84%9F%E7%9F%A5%E5%99%A8%E5%88%B0%E5%8D%B7%E7%A7%AF%E7%A5%9E%E7%BB%8F%E7%BD%91%E7%BB%9C%EF%BC%8C%E5%86%8D%E5%88%B0%E6%B7%B1%E5%BA%A6%E6%AE%8B%E5%B7%AE%E7%BD%91%E7%BB%9C%0A*%20%E4%BB%8E%E5%A4%9A%E5%B1%82%E6%84%9F%E7%9F%A5%E5%99%A8%E5%88%B0%E9%80%92%E5%BD%92%E7%A5%9E%E7%BB%8F%E7%BD%91%E7%BB%9C%EF%BC%8C%E5%86%8D%E5%88%B0LSTM%0A*%20%E4%BB%8E%E5%8D%95%E5%B1%82%E6%84%9F%E7%9F%A5%E5%99%A8%E5%88%B0Hopfield%E7%A5%9E%E7%BB%8F%E7%BD%91%E7%BB%9C%EF%BC%8C%E5%86%8D%E5%88%B0Bolazmann%E6%9C%BA%E5%92%8CRBM%0A*%20%E7%94%A8RBM%E5%A0%86%E5%8F%A0%E6%88%90DBN%EF%BC%8CDBN%E4%B8%8E%E5%A4%9A%E5%B1%82%E8%87%AA%E7%BC%96%E7%A0%81%E5%99%A8%E7%BB%93%E5%90%88%E6%88%90DBN-DNN%0A*%20**%E7%A5%9E%E7%BB%8F%E7%BD%91%E7%BB%9C%E8%A6%81%E7%B4%A0**%0A*%20%E7%BD%91%E8%B7%AF%E7%BB%93%E6%9E%84(%E5%85%A8%E8%BF%9E%E6%8E%A5%E3%80%81%E5%88%86%E5%B1%82%E3%80%81%E6%9C%89%E6%97%B6%E6%BB%9E%E5%9B%9E%E8%B7%AF%E3%80%81%E6%9D%83%E5%80%BC%E5%85%B1%E4%BA%AB%E3%80%81%E6%BF%80%E6%B4%BB%E5%87%BD%E6%95%B0)%0A*%20%E8%BF%90%E8%A1%8C%E6%9C%BA%E5%88%B6(%E5%BC%82%E6%AD%A5%E6%9B%B4%E6%96%B0%E3%80%81%E5%90%8C%E6%AD%A5%E6%9B%B4%E6%96%B0%E3%80%81%E5%89%8D%E9%A6%88)%0A*%20%E8%AE%AD%E7%BB%83%E7%AE%97%E6%B3%95%EF%BC%8C%E8%AE%AD%E7%BB%83%E4%B8%AD%E4%BD%BF%E7%94%A8%E7%9A%84trick(mini%20batch%20BN%20drop%20out%E7%AD%89)%EF%BC%8C%E6%8D%9F%E5%A4%B1%E5%87%BD%E6%95%B0%E7%9A%84%E5%AE%9A%E4%B9%89%0A*%20%E8%AE%AD%E7%BB%83%E6%95%B0%E6%8D%AE(%E6%95%B0%E6%8D%AE%E7%9A%84%E9%A2%84%E5%A4%84%E7%90%86%EF%BC%8C%E8%BE%93%E5%85%A5%E5%92%8C%E8%BE%93%E5%87%BA%E7%9A%84%E6%9E%84%E6%88%90%E7%AD%89)%0A*%20**%E5%8D%95%E5%B1%82%E6%84%9F%E7%9F%A5%E5%99%A8**%0A*%20%E8%BE%93%E5%85%A5%E8%8A%82%E7%82%B9%0A*%20%E8%BE%93%E5%87%BA%E8%8A%82%E7%82%B9%0A*%20%E6%9D%83%E5%90%91%E9%87%8F%0A*%20%E5%81%8F%E7%BD%AE%E5%9B%A0%E5%AD%90%0A*%20%E6%BF%80%E6%B4%BB%E5%87%BD%E6%95%B0%0A*%20%E5%AD%A6%E4%B9%A0%E7%8E%87%0A*%20%E5%8D%95%E5%B1%82%E6%84%9F%E7%9F%A5%E5%99%A8%E5%AD%A6%E4%B9%A0%E7%AE%97%E6%B3%95%0A*%20%E5%8D%95%E5%B1%82%E6%84%9F%E7%9F%A5%E5%99%A8%E7%9A%84%E5%B1%80%E9%99%90%0A%E6%BF%80%E6%B4%BB%E5%87%BD%E6%95%B0%0A%24%24sng(w_1x_1%2Bw_2x_2%2Bw3x_3)%3D0%24%24%0A%E5%8D%95%E5%B1%82%E6%84%9F%E7%9F%A5%E5%99%A8%E7%B1%BB%E6%AF%94%E4%BA%8E%E7%BA%BF%E6%80%A7%E5%88%86%E7%B1%BB%E5%99%A8%0A%23%23%23%23%20%E6%84%9F%E7%9F%A5%E5%99%A8%E5%AD%A6%E4%B9%A0%E7%9A%84%E8%A7%84%E5%88%99%0A1958%E5%B9%B4%EF%BC%8C%E9%A6%96%E5%85%88%E5%AE%9A%E4%B9%89%E4%BA%86%E4%B8%80%E4%B8%AA%E5%85%B7%E6%9C%89%E5%8D%95%E5%B1%82%E8%AE%A1%E7%AE%97%E5%8D%95%E5%85%83%E7%9A%84%E7%A5%9E%E7%BB%8F%E7%BD%91%E7%BB%9C%E7%BB%93%E6%9E%84%EF%BC%8C%E7%A7%B0%E4%B8%BA%E6%84%9F%E7%9F%A5%E5%99%A8%0A%E6%84%9F%E7%9F%A5%E5%99%A8%E7%9A%84%E5%AD%A6%E4%B9%A0%E8%A7%84%E5%88%99%E8%A7%84%E5%AE%9A%EF%BC%8C%E5%AD%A6%E4%B9%A0%E4%BF%A1%E5%8F%B7%E7%AD%89%E4%BA%8E%E7%A5%9E%E7%BB%8F%E5%85%83%E6%9C%9F%E6%9C%9B%E8%BE%93%E5%87%BA(%E6%95%99%E5%B8%88%E4%BF%A1%E5%8F%B7)%E4%B8%8E%E5%AE%9E%E9%99%85%E8%BE%93%E5%87%BA%E4%B9%8B%E5%B7%AE%0A%24%24r%3Dd_j-o_j%24%24%0A%E5%BC%8F%E4%B8%AD%EF%BC%8C%24d_j%24%E4%B8%BA%E6%9C%9F%E6%9C%9B%E7%9A%84%E8%BE%93%E5%87%BA%EF%BC%8C%24o_j%3Df(W%5ET_jX)%24%E6%84%9F%E7%9F%A5%E5%99%A8%E9%87%87%E7%94%A8%E4%BA%86%E4%B8%8E%E9%98%88%E5%80%BC%E8%BD%AC%E7%A7%BB%E5%87%BD%E6%95%B0%E7%B1%BB%E4%BC%BC%E7%9A%84%E7%AC%A6%E5%8F%B7%E8%BD%AC%E7%A7%BB%E5%87%BD%E6%95%B0%EF%BC%8C%E5%85%B6%E8%A1%A8%E8%BE%BE%E5%BC%8F%E4%B8%BA%0A%0A%24%24%20f(W%5ET_jX)%3Dsgn(W%5ET_jX)%3D%20%5Cbegin%7Bcases%7D%201%2C%20%26%20%5Ctext%20%7B%24(W%5ET_j%20%5Cgeq%200)%24%7D%20%5C%5C%20-1%2C%20%26%20%5Ctext%7B%24(W%5ET_j%20%3C%200)%24%7D%20%5Cend%7Bcases%7D%20%24%24%0A%0A%E5%9B%A0%E6%AD%A4%EF%BC%8C%E6%9D%83%E5%80%BC%E8%B0%83%E6%95%B4%E5%85%AC%E5%BC%8F%E4%B8%BA%EF%BC%9A%0A%24%24%5CDelta%20W_j%20%3D%5Cpm%20%202%20%5Ceta%20X%24%24%0A%E6%84%9F%E7%9F%A5%E5%99%A8%E5%AD%A6%E4%B9%A0%E8%A7%84%E5%88%99%E5%8F%AA%E9%80%82%E7%94%A8%E4%BA%8E%E4%BA%8C%E8%BF%9B%E5%88%B6%E7%A5%9E%E7%BB%8F%E5%85%83%EF%BC%8C%E5%88%9D%E5%A7%8B%E5%80%BC%E5%8F%AF%E5%8F%96%E4%BB%BB%E6%84%8F%E5%80%BC%E3%80%82%0A%E6%84%9F%E7%9F%A5%E5%99%A8%E5%AD%A6%E4%B9%A0%E8%A7%84%E5%88%99%E4%BB%A3%E8%A1%A8%E4%B8%80%E7%A7%8D%E6%9C%89%E5%AF%BC%E5%B8%88%E5%AD%A6%E4%B9%A0%EF%BC%8C%E7%94%B1%E4%BA%8E%E6%84%9F%E7%9F%A5%E5%99%A8%E7%90%86%E8%AE%BA%E6%98%AF%E7%A0%94%E7%A9%B6%E5%85%B6%E4%BB%96%E7%A5%9E%E7%BB%8F%E7%BD%91%E7%BB%9C%E7%9A%84%E5%9F%BA%E7%A1%80%EF%BC%8C%E8%AF%A5%E8%A7%84%E5%88%99%E5%AF%B9%E4%BA%8E%E7%A5%9E%E7%BB%8F%E7%BD%91%E7%BB%9C%E7%9A%84%E6%9C%89%E5%AF%BC%E5%B8%88%E5%AD%A6%E4%B9%A0%E5%85%B7%E6%9C%89%E6%9E%81%E4%B8%BA%E9%87%8D%E8%A6%81%E7%9A%84%E6%84%8F%E4%B9%89%E3%80%82%0A%0A**%E5%A4%9A%E5%B1%82%E5%89%8D%E9%A6%88%E7%A5%9E%E7%BB%8F%E7%BD%91%E7%BB%9C%EF%BC%88BP%E7%BD%91%E7%BB%9C%EF%BC%89**%0A*%20%E9%9A%90%E8%97%8F%E5%B1%82%E4%B8%8E%E9%9A%90%E8%97%8F%E8%8A%82%E7%82%B9%0A*%20%E5%89%8D%E9%A6%88-----%E6%AF%8F%E4%B8%80%E5%B1%82%E7%9A%84%E8%8A%82%E7%82%B9%E4%BB%85%E5%92%8C%E4%B8%8B%E4%B8%80%E5%B1%82%E8%8A%82%E7%82%B9%E7%9B%B8%E8%BF%9E%0A%0A!%5B88f5c6cf69add3012dd2319faa0f31e1.png%5D(en-resource%3A%2F%2Fdatabase%2F1352%3A1)%0A%0A**BP%E5%AD%A6%E4%B9%A0%E7%AE%97%E6%B3%95**%E6%9C%AC%E8%B4%A8%E5%B0%B1%E6%98%AF%E6%A2%AF%E5%BA%A6%E4%B8%8B%E9%99%8D%E6%B3%95%0A%3E%E7%A5%9E%E7%BB%8F%E7%BD%91%E7%BB%9C%E7%9A%84%E4%B8%A4%E4%B8%AA%E8%BF%87%E7%A8%8B%EF%BC%9A1.%E8%AE%AD%E7%BB%83%E8%BF%87%E7%A8%8B%202.%E6%8E%A8%E6%96%AD%E8%BF%87%E7%A8%8B%0A%0A%3E%E5%B0%8F%E7%BB%93%EF%BC%9A%E6%95%B0%E5%AD%A6%E8%83%8C%E6%99%AF%EF%BC%9A%E6%A2%AF%E5%BA%A6%E4%B8%8B%E9%99%8D%E6%B3%95(%E4%BD%BF%E7%94%A8%E8%AF%AF%E5%B7%AE%E5%B9%B3%E6%96%B9%E5%92%8C%E6%9E%84%E9%80%A0%E8%AF%AF%E5%B7%AE%E5%87%BD%E6%95%B0)%0A%3E%E6%A2%AF%E5%BA%A6%E4%B8%8B%E9%99%8D%E6%B3%95%E7%BB%99%E5%87%BA%E6%9D%83%E5%80%BC%EF%BC%8C%E9%80%9A%E8%BF%87%E5%8F%8D%E5%90%91%E4%BC%A0%E6%92%AD%E8%AF%AF%E5%B7%AE%E9%80%90%E5%B1%82%E8%AE%AD%E7%BB%83%EF%BC%8C%E6%8E%A8%E6%96%AD%E6%97%B6%E8%AF%AF%E5%B7%AE%E5%BE%80%E5%90%8E%E8%B5%B0%EF%BC%8C%E6%9E%84%E9%80%A0%E5%AE%8C%E5%90%8E%E7%BB%99%E5%87%BA%E6%9D%83%E5%80%BC%EF%BC%8C%E8%AF%AF%E5%B7%AE%E4%BF%A1%E5%8F%B7%E9%80%9A%E8%BF%87%E5%85%AC%E5%BC%8F%E8%AE%A1%E7%AE%97%E6%A2%AF%E5%BA%A6%EF%BC%8C%E5%BE%97%E5%87%BA%E6%A2%AF%E5%BA%A6%E5%AF%B9%E6%9D%83%E5%80%BC%E8%BF%9B%E8%A1%8C%E8%BF%AD%E4%BB%A3%E6%94%B9%E5%8F%98%EF%BC%8C%E7%9F%A5%E9%81%93%E6%9D%83%E5%80%BC%E4%B8%8D%E5%86%8D%E5%8F%98%E5%8C%96%E6%88%96%E8%80%85%E6%AC%A1%E6%95%B0%E8%B6%B3%E5%A4%9F%E6%94%B6%E6%95%9B%EF%BC%8C%E5%B0%B1%E7%BB%93%E6%9D%9F%E6%95%B4%E4%B8%AA%E8%AE%AD%E7%BB%83%E8%BF%87%E7%A8%8B%0A%0A%3E%E5%9C%A8python%E6%B7%B1%E5%BA%A6%E5%AD%A6%E4%B9%A0%E4%B8%AD%EF%BC%8C%E5%8F%AF%E4%BB%A5%E4%BD%BF%E7%94%A8keras%E8%BF%9B%E8%A1%8C%E7%AE%80%E5%8D%95%E7%9A%84%E7%A5%9E%E7%BB%8F%E7%BD%91%E7%BB%9C%E7%AD%89%E8%AE%BE%E8%AE%A1%0A%3Ekeras%E6%A1%86%E6%9E%B6%E6%98%AF%E7%AE%80%E5%8D%95%E6%96%B9%E4%BE%BF%E7%9A%84%EF%BC%8C%E6%A8%A1%E5%9D%97%E5%8F%AF%E4%BB%A5%E4%BD%9C%E4%B8%BA%E6%95%B0%E6%8D%AE%E6%88%96%E5%BA%8F%E5%88%97%E7%9A%84%E8%BF%90%E7%AE%97%E5%9B%BE%E7%AD%89%EF%BC%8Ckeras%E4%B8%AD%E7%9A%84%E5%90%84%E4%B8%AA%E8%AE%BE%E8%AE%A1%E7%8E%AF%E8%8A%82%E9%83%BD%E6%98%AF%E5%8D%95%E7%8B%AC%E7%9A%84%E4%B8%80%E4%B8%AA%E6%A8%A1%E5%9D%97%EF%BC%8C%E6%89%80%E4%BB%A5%E9%9C%80%E8%A6%81%E6%A0%B9%E6%8D%AE%E8%87%AA%E5%B7%B1%E7%9A%84%E9%9C%80%E6%B1%82%E6%9D%A5%E8%AE%BE%E8%AE%A1%0A%3E**%5Bkeras%E4%B8%AD%E6%96%87%E6%96%87%E6%A1%A3%5D(http%3A%2F%2Fkeras-cn.readthedocs%2Fen%2Flatest)**%0A%3Ekeras%E9%BB%98%E8%AE%A4%E4%BD%BF%E7%94%A8tensorflow%EF%BC%8C%E5%A6%82%E6%9E%9C%E4%BD%BF%E7%94%A8theano%E9%9C%80%E8%A6%81%E4%BF%AE%E6%94%B9%E6%96%87%E4%BB%B6%E4%B8%AD%E7%9A%84%E8%B7%AF%E5%BE%84%E5%86%85%E7%9A%84json%E6%96%87%E4%BB%B6%0A%0A%23%23%23%23%20keras%E7%9A%84%E5%9F%BA%E6%9C%AC%E4%BD%BF%E7%94%A8%0A*%20Sequential%20%E6%A8%A1%E5%9E%8B%E6%9E%84%E5%BB%BA%0A*%20%E5%BA%8F%E8%B4%AF%E6%A8%A1%E5%9E%8B%E6%98%AF%E5%A4%9A%E4%B8%AA%E7%BD%91%E7%BB%9C%E5%B1%82%E7%9A%84%E7%BA%BF%E6%80%A7%E5%A0%86%E5%8F%A0%0A*%20%E5%8F%AF%E4%BB%A5%E9%80%9A%E8%BF%87%E5%90%91Sequential%E6%A8%A1%E5%9E%8B%E4%BC%A0%E9%80%92%E4%B8%80%E4%B8%AAlayer%E7%9A%84list%E6%9D%A5%E6%9E%84%E9%80%A0%E8%AF%A5%E6%A8%A1%E5%9E%8B%0A%60%60%60python%0Afrom%20keras.models%20import%20Sequential%0Afrom%20keras.layers%20import%20Dense%2CActivation%0Amodel%20%3D%20Sequential(%5B%C2%A0%C2%A0%C2%A0%20%0ADense(32%2Cunits%3D784)%2C%C2%A0%C2%A0%C2%A0%20%0AActivation('relu')%2C%C2%A0%C2%A0%C2%A0%20%0ADense(10)%2C%C2%A0%C2%A0%C2%A0%20%0AActivation('softmax')%2C%5D)%0A%60%60%60%0A%0A*%20%E4%B9%9F%E5%8F%AF%E4%BB%A5%E9%80%9A%E8%BF%87.add()%E6%96%B9%E6%B3%95%E4%B8%80%E4%B8%AA%E4%B8%AA%E7%9A%84%E5%B0%86layer%E5%8A%A0%E5%85%A5%E6%A8%A1%E5%9E%8B%E4%B8%AD%0A%60%60%60python%0Amodel%20%3D%20Sequential()%0Amodel.add(Dense(32%2Cinput_shape%3D(784%2C))%0A)model.add(Activation('relu'))%0A%60%60%60%0A%0A*%20%E6%8C%87%E5%AE%9A%E8%BE%93%E5%85%A5%E6%95%B0%E6%8D%AE%E7%9A%84shape%0A*%20input%20shape%EF%BC%88%E5%85%83%E7%A5%96%E5%9E%8B%E6%95%B0%E6%8D%AE%EF%BC%89%0A*%20input%20dim%20%0A%60%60%60python%0Amodel%20%3D%20Sequential()%0Amodel.add(Dense(32%2Cinput_dim%3D784))%0A%60%60%60%0A*%20input%20length%0A%60%60%60python%0Amodel%20%3D%20Sequentail()%0Amodel.add(Dense(32%2Cinput_shape%3D784))%0A%60%60%60%0A*%20batch_size%0A%0A%23%23%23%23%20%E8%AF%84%E5%88%86%E6%A0%87%E5%87%86%0A*%20%E9%80%89%E6%89%8B%E7%AE%97%E5%87%BA%E7%94%A8%E6%88%B7%E5%9C%A8%E4%B8%8D%E5%90%8C%E5%88%86%E7%BB%84%E4%B8%8A%E7%9A%84%E6%A6%82%E7%8E%87%EF%BC%8C%E7%8E%B0%E5%AE%9E%E4%B8%AD%E4%B8%80%E4%B8%AA%E7%94%A8%E6%88%B7%E5%8F%AA%E8%83%BD%E5%9C%A8%E4%B8%80%E4%B8%AA%E5%88%86%E7%BB%84%E3%80%82%E7%90%86%E6%83%B3%E7%8A%B6%E6%80%81%E4%B8%8B%E5%A6%82%E6%9E%9C%E8%83%BD%E7%AE%97%E5%87%BA%E8%BF%99%E4%B8%AA%E6%A6%82%E7%8E%87%E6%98%AF1%EF%BC%8C%E5%85%B6%E4%BB%96%E6%98%AF0%E7%9A%84%E8%AF%9D%EF%BC%8C%E8%BF%99%E4%B8%AA%E7%AD%94%E6%A1%88%E5%B0%B1%E6%98%AF%E6%B2%A1%E6%9C%89%E4%BB%BB%E4%BD%95%E6%8D%9F%E5%A4%B1%E7%9A%84%E3%80%82%E4%B8%80%E8%88%AC%E6%9D%A5%E8%AF%B4%EF%BC%8C%E6%8F%90%E4%BA%A4%E7%9A%84%E7%AD%94%E6%A1%88%E4%B8%AD%EF%BC%8C%E6%9F%90%E4%B8%AA%E7%94%A8%E6%88%B7%E4%BC%9A%E6%9C%89%E6%88%96%E5%A4%A7%E6%88%96%E5%B0%8F%E7%9A%84%E6%A6%82%E7%8E%87%E5%B1%9E%E4%BA%8E%E5%A4%9A%E4%B8%AA%E7%BB%84%E5%88%AB%EF%BC%8C%E8%BF%99%E4%B8%AA%E6%97%B6%E5%80%99%E5%B0%B1%E6%9C%89%E6%A6%82%E7%8E%87%E4%B8%8A%E7%9A%84%E6%8D%9F%E5%A4%B1%EF%BC%8C%E8%BF%99%E4%B8%AA%E6%8D%9F%E5%A4%B1%E7%9A%84%E9%AB%98%E4%BD%8E%E4%BB%A3%E8%A1%A8%E7%AD%94%E6%A1%88%E7%9A%84%E6%B0%B4%E5%B9%B3%E3%80%82%E6%88%91%E4%BB%AC%E4%BC%98%E5%8C%96%E5%87%BD%E6%95%B0%E6%88%96%E8%80%85%E8%AF%B4%E6%98%AF%E8%AF%84%E4%BC%B0%E6%8C%87%E6%A0%87%E5%B0%B1%E6%98%AF%E4%B8%8B%E9%9D%A2%E7%9A%84%E6%8D%9F%E5%A4%B1%E5%87%BD%E6%95%B0%0A%24%24logloss%3D-%5Cfrac%7B1%7D%7BN%7D%20%5Csum%5EN_%7Bi%3D1%7D%20%5Csum%5EM_%7Bj%3D1%7D%20y_%7Bij%7D%20log(p_%7Bij%7D)%24%24%0A%24%24max(min(p%2C1-10%5E%7B-15%7D)%2C10%5E%7B-15%7D)%24%24%0A%0A!%5Bc430046ff391af67322b0c4d02f2c6a7.png%5D(en-resource%3A%2F%2Fdatabase%2F1354%3A0)%0A%0A%3E%E6%96%B0%E6%80%9D%E8%B7%AF%0A%3E**%E7%89%B9%E5%BE%81**%0A%3E%20*%20Xgboost%3A%E4%B8%BB%E8%A6%81%E4%BD%BF%E7%94%A8count%E7%89%B9%E5%BE%81%E4%B8%8ETF-IDF%E7%89%B9%E5%BE%81%0A%3E%20*%20Keras%EF%BC%9A%E4%B8%BB%E8%A6%81%E4%BD%BF%E7%94%A8bag%20of%20apps%E7%89%B9%E5%BE%81%0A%3E%20**%E4%B8%A4%E6%AE%B5%E9%A2%84%E6%B5%8B**%0A%3E%20%E7%94%9F%E6%88%90%E7%89%B9%E5%BE%81%E9%9B%86%0A%3E%20%E9%A2%84%E6%B5%8B%E7%94%A8%E6%88%B7%E7%9A%84%E6%80%A7%E5%88%AB%E6%A6%82%E7%8E%87(Stage%201)%0A%3E%20%E4%BD%BF%E7%94%A8%E9%A2%84%E6%B5%8B%E7%9A%84%E6%80%A7%E5%88%AB%E4%BD%9C%E4%B8%BA%E9%A2%9D%E5%A4%96%E7%9A%84%E7%89%B9%E5%BE%81%EF%BC%8C%E9%A2%84%E6%B5%8B%E5%90%84%E4%B8%AA%E5%B9%B4%E9%BE%84%E6%AE%B5%E7%9A%84%E6%A6%82%E7%8E%87%EF%BC%8C%E5%85%88%E5%81%87%E8%AE%BE%E7%94%A8%E6%88%B7%E6%98%AF%E5%A5%B3%E6%80%A7%EF%BC%8C%E9%A2%84%E6%B5%8B%E5%85%B6%E5%B1%9E%E4%BA%8E%E5%90%84%E4%B8%AA%E5%B9%B4%E9%BE%84%E6%AE%B5%E7%9A%84%E6%A6%82%E7%8E%87%EF%BC%8C%E5%BE%97%E5%88%B0%24P(A_i%7CF)%24%EF%BC%8C%E5%86%8D%E5%81%87%E8%AE%BE%E7%94%A8%E6%88%B7%E4%B8%BA%E7%94%B7%E6%80%A7%EF%BC%8C%E9%A2%84%E6%B5%8B%E5%85%B6%E5%B1%9E%E4%BA%8E%E5%90%84%E4%B8%AA%E5%B9%B4%E9%BE%84%E6%AE%B5%E7%9A%84%E6%A6%82%E7%8E%87(Stage%201)%0A%3E%20%E6%A0%B9%E6%8D%AE%E6%9D%A1%E4%BB%B6%E6%A6%82%E7%8E%87%E5%85%AC%E5%BC%8F%E8%8E%B7%E5%BE%97%E7%94%A8%E6%88%B7%E7%9A%84%E4%BA%BA%E5%8F%A3%E7%89%B9%E5%BE%81%E5%88%86%E7%BB%84%E6%A6%82%E7%8E%87%0A%3E%20%24P(A_i%2CF)%3DP(A_i%7CF)P(F)%24%20for%20i...6%20and%20%24P(A_i%2CM)%3DP(A_i%7CM)P(M)%24%20for%20i%3D1....6%20%24A_i%24%E4%BB%A3%E8%A1%A8%E8%BF%99%E4%B8%AA%E5%B9%B4%E9%BE%84%E7%9A%846%E4%B8%AA%E5%88%86%E7%BB%84%EF%BC%8C1%20to%206%20%E4%BB%A3%E8%A1%A8%E7%9A%84%E6%98%AFF%E5%A5%B3%E6%80%A7%EF%BC%8CM%E7%94%B7%E6%80%A7