Rethinking Bottleneck Structure for Efficient Mobile Network Design

Rethinking Bottleneck Structure for Efficient Mobile Network Design

一. 论文简介

减少模块提取特征时的丢失，增强模块提取特征的能力

采用类似hourglass的结构，先降采样后升采样（相对于通道）

二. 模块详解

2.1 整体结构介绍

depth-wise（w/ Relu） + points-wise(W/O Relu) + point-wise(W/ Relu) + depth-wise(W/O Relu)

class SandGlassModule(nn.Module):
    def __init__(self, inp, oup, stride, reduction_ratio):
        super(SandGlass, self).__init__()
        assert stride in [1, 2]

        hidden_dim = round(inp // reduction_ratio)
        self.identity = stride == 1 and inp == oup

        self.conv = nn.Sequential(
            # dw + relu
            nn.Conv2d(inp, inp, 3, 1, 1, groups=inp, bias=False),
            nn.BatchNorm2d(inp),
            nn.ReLU6(inplace=True),
            # pw + linear(non-relu)
            nn.Conv2d(inp, hidden_dim, 1, 1, 0, bias=False),
            nn.BatchNorm2d(hidden_dim),
            # pw + relu
            nn.Conv2d(hidden_dim, oup, 1, 1, 0, bias=False),
            nn.BatchNorm2d(oup),
            nn.ReLU6(inplace=True),
            # dw + linear(non-relu)
            nn.Conv2d(oup, oup, 3, stride, 1, groups=oup, bias=False),
            nn.BatchNorm2d(oup),
        )

    def forward(self, x):
        if self.identity:
            return x + self.conv(x)
        else:
            return self.conv(x)