CNN中的注意力机制

通道注意力机制SENet

SENet(Squeeze-and-Excitation Networks,SENet)是ImageNet2017分类比赛的冠军模型,主要思想为对H×W×C的feature map进行处理,得到一个1×1×C的权重,再乘回去得到最终的feature map。具体结构如下图所示:

  • Sequeeze:对H×W×C的feature map进行global average pooling,得到具有全局感受野的feature map(1×1×C)
  • Excitation:通过全连接层Sigmoid进行非线性变换,得到每个通道的权重
  • Scale:H×W×C的feature map以1×1×C的权重,得到最终的feature map(H×W×C)

# SENet的Pytorch实现
class SELayer(nn.Module):
    def __init__(self, channel, reduction=16):
        super(SELayer, self).__init__()
        self.avg_pool = nn.AdaptiveAvgPool2d(1)
        self.fc = nn.Sequential(
            nn.Linear(channel, channel // reduction, bias=False),
            nn.ReLU(inplace=True),
            nn.Linear(channel // reduction, channel, bias=False),
            nn.Sigmoid()
        )

    def forward(self, x):
        b, c, _, _ = x.size()
        y = self.avg_pool(x).view(b, c) # view可改变Tensor的形状,但数据不变
        y = self.fc(y).view(b, c, 1, 1)
        return x * y.expand_as(x) # 通过expand_as将y拓展成和x想同的shape

通道+空间注意力机制CBAM

CBAM(Convolutional Block Attention Module,CBAM)发表于ECCV2018,其同时集成了通道注意力机制和空间注意力机制,具体结构如下图所示:

通道注意力模块

首先采用两种池化方法对feature map进行汇聚,

然后将生成的两种通道注意力向量输入共享的MLP(Multi-Layer Perception)网络,生成两个维数为C×1×1的注意力向量

最后将两种向量相加,载通道sigmoid函数,得到最终的通道注意力向量(C×1×1)

class ChannelAttention(nn.Module):
    def __init__(self, in_planes, rotio=16):
        super(ChannelAttention, self).__init__()
        self.avg_pool = nn.AdaptiveAvgPool2d(1)
        self.max_pool = nn.AdaptiveMaxPool2d(1)

        self.sharedMLP = nn.Sequential(
            nn.Conv2d(in_planes, in_planes // ratio, 1, bias=False), nn.ReLU(),
            nn.Conv2d(in_planes // rotio, in_planes, 1, bias=False))
        self.sigmoid = nn.Sigmoid()

    def forward(self, x):
        avgout = self.sharedMLP(self.avg_pool(x))
        maxout = self.sharedMLP(self.max_pool(x))
        return self.sigmoid(avgout + maxout)

空间注意力模块

首先同时采用两种池化方法,在通道维度对feature map进行汇聚,得到两个维度相同的feature map(1×H×W)

然后把得到的两个feature map拼接在一起,得到一个特殊的feature map

最后利用一个7*7的卷积层对feature map进行卷积,再通过sigmoid函数,得到最终的feature map

class SpatialAttention(nn.Module):
    def __init__(self, kernel_size=7):
        super(SpatialAttention, self).__init__()
        assert kernel_size in (3,7), "kernel size must be 3 or 7"   # 判断kernel的尺寸,不为3,7时输出语句
        padding = 3 if kernel_size == 7 else 1

        self.conv = nn.Conv2d(2,1,kernel_size, padding=padding, bias=False)
        self.sigmoid = nn.Sigmoid()

    def forward(self, x):
        avgout = torch.mean(x, dim=1, keepdim=True) # z在通道维度上进行mean和max操作
        maxout, _ = torch.max(x, dim=1, keepdim=True)
        x = torch.cat([avgout, maxout], dim=1)  # 按通道维度拼接在一起
        x = self.conv(x) # 卷积
        return self.sigmoid(x) # 激活

通道 + 空间注意力

class BasicBlock(nn.Module):
    expansion = 1
    def __init__(self, inplanes, planes, stride=1, downsample=None):
        super(BasicBlock, self).__init__()
        self.conv1 = conv3x3(inplanes, planes, stride)  # 自己定义的函数
        self.bn1 = nn.BatchNorm2d(planes)
        self.relu = nn.ReLU(inplace=True)
        self.conv2 = conv3x3(planes, planes)
        self.bn2 = nn.BatchNorm2d(planes)
        self.ca = ChannelAttention(planes)
        self.sa = SpatialAttention()
        self.downsample = downsample
        self.stride = stride

    def forward(self, x):
        residual = x
        out = self.conv1(x)
        out = self.bn1(out)
        out = self.relu(out)
        out = self.conv2(out)
        out = self.bn2(out)
        out = self.ca(out) * out  # 广播机制
        out = self.sa(out) * out  # 广播机制
        if self.downsample is not None:
            residual = self.downsample(x)
        out += residual
        out = self.relu(out)
        return out

  目录