Pytorch-分类器

1.分类问题

二分类

• $f:x ightarrow p(y=1|x)$  
  • $p(y=1|x)$  解释成给定x，求y=1的概率，如果概率>0.5，预测为1，否则预测为0
• minimize MSE

多分类

• $f:x ightarrow p(y|x)$
  • $[p(y=0|x),p(y=1|x),...,p(y=9|x)]$
• $p(y|x)epsilon [0,1]$
• $sum_{i=0}^{9}p(y=i|x)=1$

2.交叉熵

信息熵：描述信息的不确定性。$H(U)=E(-logp_i)=-sum _{i=1}^{n}p_{i}logp_{i}$

描述一个分布，不确定性越高，熵越低。

a=torch.full([4],1/4.)
print(-(a*torch.log2(a)).sum())               #tensor(2.)

b=torch.tensor([0.1,0.1,0.1,0.7])
print(-(b*torch.log2(b)).sum())               #tensor(1.3568)

c=b=torch.tensor([0.001,0.001,0.001,0.999])
print(-(c*torch.log2(c)).sum())               #tensor(0.0313)

2.1交叉熵

$H(p,q)=-sum _{xepsilon X}p(x)logq(x)$

$H(p,q)=H(p)+D_{KL}(p|q)$  （KL散度=交叉熵-信息熵，衡量p，q分布的重叠情况）

• p=q，H(p,q)=H(p)
• 对独热编码来说，H(p)=0

2.2二分类问题的交叉熵

$H(p,q)=-sum _{iepsilon cat,dog}P(i)logQ(i)$  P(i)指i的真实值，Q(i)指i的预测值。

           $=-P(cat)logQ(cat)-P(dog)logQ(dog)$    $P(dog)=(1-P(cat))$

           $=-sum _{i=1}^{n}y_ilog(p_i)+(1-y_i)log(1-p_i)$    $y_i$指i的真实值，$p_i$指i的预测值。

 

import torch
from torch.nn import functional as F

x=torch.randn(1,784)
w=torch.randn(10,784)
logits=x@w.t()                                            #shape=torch.Size([1,10])

print(F.cross_entropy(logits, torch.tensor([3])))         #tensor(77.1405)

pred=F.softmax(logits,dim=1)                              #shape=torch.Size([1,10])
pred_log=torch.log(pred)
print(F.nll_loss(pred_log, torch.tensor([3])))            #tensor(77.1405)

cross_entropy函数=>softmax->log->nll_loss

3.多分类实战

识别手写数据集

import  torch
import  torch.nn as nn
import  torch.nn.functional as F
import  torch.optim as optim
from    torchvision import datasets, transforms

#超参数
batch_size=200
learning_rate=0.01
epochs=10

#获取训练集
train_loader = torch.utils.data.DataLoader(
    datasets.MNIST('../data', train=True, download=True,          #train=True则得到的是训练集
                   transform=transforms.Compose([                 #transform进行数据预处理
                       transforms.ToTensor(),                     #转成Tensor类型的数据
                       transforms.Normalize((0.1307,), (0.3081,)) #进行数据标准化(减去均值除以方差)
                   ])),
    batch_size=batch_size, shuffle=True)                          #按batch_size分出一个batch维度在最前面,shuffle=True打乱顺序



#获取测试集
test_loader = torch.utils.data.DataLoader(
    datasets.MNIST('../data', train=False, transform=transforms.Compose([
        transforms.ToTensor(),
        transforms.Normalize((0.1307,), (0.3081,))
    ])),
    batch_size=batch_size, shuffle=True)

#设定参数w和b
w1, b1 = torch.randn(200, 784, requires_grad=True),
         torch.zeros(200, requires_grad=True)             #w1(out,in)
w2, b2 = torch.randn(200, 200, requires_grad=True),
         torch.zeros(200, requires_grad=True)
w3, b3 = torch.randn(10, 200, requires_grad=True),
         torch.zeros(10, requires_grad=True)

torch.nn.init.kaiming_normal_(w1)
torch.nn.init.kaiming_normal_(w2)
torch.nn.init.kaiming_normal_(w3)


def forward(x):
    x = x@w1.t() + b1
    x = F.relu(x)
    x = x@w2.t() + b2
    x = F.relu(x)
    x = x@w3.t() + b3
    x = F.relu(x)
    return x


#定义sgd优化器,指明优化参数、学习率
optimizer = optim.SGD([w1, b1, w2, b2, w3, b3], lr=learning_rate)
criteon = nn.CrossEntropyLoss()

for epoch in range(epochs):

    for batch_idx, (data, target) in enumerate(train_loader):
        data = data.view(-1, 28*28)          #将二维的图片数据摊平[样本数,784]

        logits = forward(data)               #前向传播
        loss = criteon(logits, target)       #nn.CrossEntropyLoss()自带Softmax

        optimizer.zero_grad()                #梯度信息清空   
        loss.backward()                      #反向传播获取梯度
        optimizer.step()                     #优化器更新

        if batch_idx % 100 == 0:             #每100个batch输出一次信息
            print('Train Epoch: {} [{}/{} ({:.0f}%)]	Loss: {:.6f}'.format(
                epoch, batch_idx * len(data), len(train_loader.dataset),
                       100. * batch_idx / len(train_loader), loss.item()))


    test_loss = 0
    correct = 0                                         #correct记录正确分类的样本数
    for data, target in test_loader:
        data = data.view(-1, 28 * 28)
        logits = forward(data)
        test_loss += criteon(logits, target).item()     #其实就是criteon(logits, target)的值，标量
        
        pred = logits.data.max(dim=1)[1]                #也可以写成pred=logits.argmax(dim=1)
        correct += pred.eq(target.data).sum()

    test_loss /= len(test_loader.dataset)
    print('
Test set: Average loss: {:.4f}, Accuracy: {}/{} ({:.0f}%)
'.format(
        test_loss, correct, len(test_loader.dataset),
        100. * correct / len(test_loader.dataset)))

Train Epoch: 0 [0/60000 (0%)] Loss: 2.669597
Train Epoch: 0 [20000/60000 (33%)] Loss: 0.815616
Train Epoch: 0 [40000/60000 (67%)] Loss: 0.531289

Test set: Average loss: 0.0029, Accuracy: 8207/10000 (82%)

Train Epoch: 1 [0/60000 (0%)] Loss: 0.629100
Train Epoch: 1 [20000/60000 (33%)] Loss: 0.534102
Train Epoch: 1 [40000/60000 (67%)] Loss: 0.372538

Test set: Average loss: 0.0015, Accuracy: 9135/10000 (91%)

Train Epoch: 2 [0/60000 (0%)] Loss: 0.387911
Train Epoch: 2 [20000/60000 (33%)] Loss: 0.285009
Train Epoch: 2 [40000/60000 (67%)] Loss: 0.239334

Test set: Average loss: 0.0012, Accuracy: 9266/10000 (93%)

Train Epoch: 3 [0/60000 (0%)] Loss: 0.190144
Train Epoch: 3 [20000/60000 (33%)] Loss: 0.227097
Train Epoch: 3 [40000/60000 (67%)] Loss: 0.212044

Test set: Average loss: 0.0011, Accuracy: 9342/10000 (93%)

Train Epoch: 4 [0/60000 (0%)] Loss: 0.209257
Train Epoch: 4 [20000/60000 (33%)] Loss: 0.158875
Train Epoch: 4 [40000/60000 (67%)] Loss: 0.163808

Test set: Average loss: 0.0010, Accuracy: 9391/10000 (94%)

Train Epoch: 5 [0/60000 (0%)] Loss: 0.171966
Train Epoch: 5 [20000/60000 (33%)] Loss: 0.155037
Train Epoch: 5 [40000/60000 (67%)] Loss: 0.207052

Test set: Average loss: 0.0010, Accuracy: 9431/10000 (94%)

Train Epoch: 6 [0/60000 (0%)] Loss: 0.253138
Train Epoch: 6 [20000/60000 (33%)] Loss: 0.144874
Train Epoch: 6 [40000/60000 (67%)] Loss: 0.245244

Test set: Average loss: 0.0009, Accuracy: 9441/10000 (94%)

Train Epoch: 7 [0/60000 (0%)] Loss: 0.139446
Train Epoch: 7 [20000/60000 (33%)] Loss: 0.189258
Train Epoch: 7 [40000/60000 (67%)] Loss: 0.136632

Test set: Average loss: 0.0009, Accuracy: 9477/10000 (95%)

Train Epoch: 8 [0/60000 (0%)] Loss: 0.178528
Train Epoch: 8 [20000/60000 (33%)] Loss: 0.141744
Train Epoch: 8 [40000/60000 (67%)] Loss: 0.148902

Test set: Average loss: 0.0008, Accuracy: 9508/10000 (95%)

Train Epoch: 9 [0/60000 (0%)] Loss: 0.074641
Train Epoch: 9 [20000/60000 (33%)] Loss: 0.095056
Train Epoch: 9 [40000/60000 (67%)] Loss: 0.261752

Test set: Average loss: 0.0008, Accuracy: 9514/10000 (95%)