【科研基础】Parameters和FLOPs计算

作者搜集了网上存在的利用Tensorflow计算FLOPs的代码（如下所示），但是几乎都不能准确计算，因此我们尝试通过人工计算。
CNN 模型所需的计算力（flops）和参数（parameters）数量是怎么计算的？
卷积层计算量(FLOPS)和参数量的计算
MACs和FLOPs

def get_flops_params():
    sess = tf.Session()
    graph = sess.graph
    flops = tf.profiler.profile(graph, options=tf.profiler.ProfileOptionBuilder.float_operation())
    params = tf.profiler.profile(graph, options=tf.profiler.ProfileOptionBuilder.trainable_variables_parameter())
    print('FLOPs: {};    Trainable params: {}'.format(flops.total_float_ops, params.total_parameters))

1.SCNN

1.1.Parameters计算

def SCNN():
    # build the CNN model
    in_shp = [2, 128]
    L = 128  # sample points
    xm_input = Input(in_shp)
    xm = Reshape([128, 2], input_shape=in_shp)(xm_input)
    x1 = Conv1D(128, 16, activation='relu', padding='same', input_shape=[L, 2])(xm)
    x2 = BatchNormalization()(x1)
    x3 = Dropout(0.5)(x2)

    x4 = SeparableConv1D(64, 8, activation='relu', padding='same')(x3)
    x5 = BatchNormalization()(x4)
    x6 = Dropout(0.5)(x5)

    x7 = Flatten()(x6)
    x8 = Dense(10)(x7)
    predicts = Activation('softmax')(x8)
    model = Model(xm_input, predicts)
    model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])
    model.summary()
    return model

输出每一层具体信息:

_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
input_1 (InputLayer)         (None, 2, 128)            0         
_________________________________________________________________
reshape_1 (Reshape)          (None, 128, 2)            0         
_________________________________________________________________
conv1d_1 (Conv1D)            (None, 128, 128)          4224      
_________________________________________________________________
batch_normalization_1 (Batch (None, 128, 128)          512       
_________________________________________________________________
dropout_1 (Dropout)          (None, 128, 128)          0         
_________________________________________________________________
separable_conv1d_1 (Separabl (None, 128, 64)           9280      
_________________________________________________________________
batch_normalization_2 (Batch (None, 128, 64)           256       
_________________________________________________________________
dropout_2 (Dropout)          (None, 128, 64)           0         
_________________________________________________________________
flatten_1 (Flatten)          (None, 8192)              0         
_________________________________________________________________
dense_1 (Dense)              (None, 11)                90123     
_________________________________________________________________
activation_1 (Activation)    (None, 11)                0         
=================================================================
Total params: 104,395
Trainable params: 104,011
Non-trainable params: 384
_________________________________________________________________

• 卷积层：
  以第一个conv1d_1 (Conv1D)为例，
  $$P_{CONV}=(K_h\ast K_w\ast C_{in})\ast C_{out}+C_{out}=(16\ast 2)\ast 128+128=4224$$$$C_{in}=2,C_{out}=128,K_h\ast K_w=16$$

1.2.FLOPs计算

• 卷积层：
  以第一个conv1d_1 (Conv1D)为例（注意：不考虑bias时有-1，有bias时没有-1，这里的计算考虑了bias)
  $$F_{CONV}=(2\ast C_{in}\ast K_h\ast K_w-1）\ast H\ast W\ast C_{out}=(2\ast 2\ast 16-1)\ast 128\ast 128=1,032,192$$
• 可分离卷积，参考：该捋清！卷积、可分离卷积的参数和FLOPs计算！
  $$F_{SepCONV}=2(K_h\ast K_w+C_{out})\ast H\ast W\ast C_{in}=2(8+64)\ast 128\ast 128=2,359,296$$
• 全连接dense层，参考神经网络层的FLOPs计算,
  $$F_{Dense}=2\ast H\ast W\ast C_{in}\ast C_{out}=2\ast 8192\ast 11=180,224$$
  $$F_{CONV}+F_{SepCONV}+F_{Dense}=3,571,712$$

2.PET-CGDNN

def cal1(x):
    y = tf.keras.backend.cos(x)
    return y

def cal2(x):
    y = tf.keras.backend.sin(x)
    return y
def MCLDNN(weights=None,
           input_shape1=[2, 128],
           input_shape2=[128, 1],
           classes=11,
           **kwargs):
    if weights is not None and not (os.path.exists(weights)):
        raise ValueError('The `weights` argument should be either '
                         '`None` (random initialization), '
                         'or the path to the weights file to be loaded.')

    dr = 0.5  # dropout rate (%)
    input = Input(input_shape1 + [1], name='input1')
    input1 = Input(input_shape2, name='input2')
    input2 = Input(input_shape2, name='input3')

    x1 = Flatten()(input)
    x1 = Dense(1, name='fc2')(x1)
    x1 = Activation('linear')(x1)

    cos1 = Lambda(cal1)(x1)
    sin1 = Lambda(cal2)(x1)
    x11 = Multiply()([input1, cos1])
    x12 = Multiply()([input2, sin1])
    x21 = Multiply()([input2, cos1])
    x22 = Multiply()([input1, sin1])
    y1 = Add()([x11, x12])
    y2 = Subtract()([x21, x22])
    y1 = Reshape(target_shape=(128, 1), name='reshape1')(y1)
    y2 = Reshape(target_shape=(128, 1), name='reshape2')(y2)
    x11 = concatenate([y1, y2])
    x3 = Reshape(target_shape=((128, 2, 1)), name='reshape3')(x11)

    # spatial feature
    x3 = Conv2D(75, (8, 2), padding='valid', activation="relu", name="conv1_1", kernel_initializer='glorot_uniform')(
        x3)
    x3 = Conv2D(25, (5, 1), padding='valid', activation="relu", name="conv1_2", kernel_initializer='glorot_uniform')(
        x3)
    # temporal feature
    x4 = Reshape(target_shape=((117, 25)), name='reshape4')(x3)
    x4 = keras.layers.GRU(units=128)(x4)
    # x4 = Flatten()(x4)
    #
    x = Dense(classes, activation='softmax', name='softmax')(x4)

    model = Model(inputs=[input, input1, input2], outputs=x)

输出每一层具体信息:

__________________________________________________________________________________________________
Layer (type)                    Output Shape         Param #     Connected to                     
==================================================================================================
input1 (InputLayer)             (None, 2, 128, 1)    0                                            
__________________________________________________________________________________________________
flatten_1 (Flatten)             (None, 256)          0           input1[0][0]                     
__________________________________________________________________________________________________
fc2 (Dense)                     (None, 1)            257         flatten_1[0][0]                  
__________________________________________________________________________________________________
activation_1 (Activation)       (None, 1)            0           fc2[0][0]                        
__________________________________________________________________________________________________
input2 (InputLayer)             (None, 128, 1)       0                                            
__________________________________________________________________________________________________
lambda_1 (Lambda)               (None, 1)            0           activation_1[0][0]               
__________________________________________________________________________________________________
input3 (InputLayer)             (None, 128, 1)       0                                            
__________________________________________________________________________________________________
lambda_2 (Lambda)               (None, 1)            0           activation_1[0][0]               
__________________________________________________________________________________________________
multiply_1 (Multiply)           (None, 128, 1)       0           input2[0][0]                     
                                                                 lambda_1[0][0]                   
__________________________________________________________________________________________________
multiply_2 (Multiply)           (None, 128, 1)       0           input3[0][0]                     
                                                                 lambda_2[0][0]                   
__________________________________________________________________________________________________
multiply_3 (Multiply)           (None, 128, 1)       0           input3[0][0]                     
                                                                 lambda_1[0][0]                   
__________________________________________________________________________________________________
multiply_4 (Multiply)           (None, 128, 1)       0           input2[0][0]                     
                                                                 lambda_2[0][0]                   
__________________________________________________________________________________________________
add_1 (Add)                     (None, 128, 1)       0           multiply_1[0][0]                 
                                                                 multiply_2[0][0]                 
__________________________________________________________________________________________________
subtract_1 (Subtract)           (None, 128, 1)       0           multiply_3[0][0]                 
                                                                 multiply_4[0][0]                 
__________________________________________________________________________________________________
reshape1 (Reshape)              (None, 128, 1)       0           add_1[0][0]                      
__________________________________________________________________________________________________
reshape2 (Reshape)              (None, 128, 1)       0           subtract_1[0][0]                 
__________________________________________________________________________________________________
concatenate_1 (Concatenate)     (None, 128, 2)       0           reshape1[0][0]                   
                                                                 reshape2[0][0]                   
__________________________________________________________________________________________________
reshape3 (Reshape)              (None, 128, 2, 1)    0           concatenate_1[0][0]              
__________________________________________________________________________________________________
conv1_1 (Conv2D)                (None, 121, 1, 75)   1275        reshape3[0][0]                   
__________________________________________________________________________________________________
conv1_2 (Conv2D)                (None, 117, 1, 25)   9400        conv1_1[0][0]                    
__________________________________________________________________________________________________
reshape4 (Reshape)              (None, 117, 25)      0           conv1_2[0][0]                    
__________________________________________________________________________________________________
gru_1 (GRU)                     (None, 128)          59136       reshape4[0][0]                   
__________________________________________________________________________________________________
softmax (Dense)                 (None, 11)           1419        gru_1[0][0]                      
==================================================================================================
Total params: 71,487
Trainable params: 71,487
Non-trainable params: 0

$$F_{fc2}=2\ast H\ast W\ast C_{in}\ast C_{out}=2\ast 256\ast 1=512$$ $$F_{conv{1}_1}=(2\ast C_{in}\ast K_h\ast K_w-1）\ast H\ast W\ast C_{out}=(2\ast 1\ast 8\ast 2-1)\ast 121\ast 1\ast 75=281,325$$$$F_{conv{1}_2}=(2\ast C_{in}\ast K_h\ast K_w-1）\ast H\ast W\ast C_{out}=(2\ast 75\ast 5\ast 1-1)\ast 117\ast 1\ast 25=2,190,825$$ $$F_{gr{u}_1}=(E+H)\ast H\ast 3\ast 2=(25+128)\ast 128\ast 3\ast 2=117,504$$$$F_{fc2}+F_{conv{1}_1}+F_{conv{1}_2}+F_{gr{u}_1}=2,590,166$$
GRU的计算参考这两篇博客：
1.知乎
2.LSTM
3.GRU

注：
1.我们如果用tensorflow作为backend时候，系统会默认是channel_last。
2.如何计算GRU卷积神经网络CNN中的参数量（parameters）和计算量（FLOPs ）这篇文章里面的结论直接用：FLOPs = paras * H * W * 2
我们以上面的计算结果为例$$F_{fc2}=paras\ast H\ast W\ast 2=257\ast 1\ast 2=514$$
514和512差别不大。
$$F_{conv{1}_1}=paras\ast H\ast W\ast 2=1275\ast 121\ast 2=308,550$$
308,550和281,325差别不大。
$$F_{conv{1}_2}=paras\ast H\ast W\ast 2=9400\ast 117\ast 2=2,199,600$$
2,199,600和2,190,825差别不大。
因此GRU的FLOPs计算公式，我们也尝试用这个公式计算
$$F_{gr{u}_1}=paras\ast H\ast W\ast 2=59136\ast 2=118,272$$
于是得到总的
$$F_{fc2}+F_{conv{1}_1}+F_{conv{1}_2}+F_{gr{u}_1}=514+308,550+2,199,600+118,272=2,626,936$$

3.MCLDNN

def MCLDNN(weights=None,
           input_shape1=[2, 128],
           input_shape2=[128, 1],
           classes=11,
           **kwargs):
    if weights is not None and not (os.path.exists(weights)):
        raise ValueError('The `weights` argument should be either '
                         '`None` (random initialization), '
                         'or the path to the weights file to be loaded.')

    dr = 0.5

    input1 = Input(input_shape1 + [1], name='I/Qchannel')
    input2 = Input(input_shape2, name='Ichannel')
    input3 = Input(input_shape2, name='Qchannel')

    # Part-A: Multi-channel Inputs and Spatial Characteristics Mapping Section
    x1 = Conv2D(50, (2, 8), padding='same', activation="relu", name="Conv1", kernel_initializer="glorot_uniform")(
        input1)
    x2 = Conv1D(50, 8, padding='causal', activation="relu", name="Conv2", kernel_initializer="glorot_uniform")(input2)
    x2_reshape = Reshape([-1, 128, 50])(x2)
    x3 = Conv1D(50, 8, padding='causal', activation="relu", name="Conv3", kernel_initializer="glorot_uniform")(input3)
    x3_reshape = Reshape([-1, 128, 50], name="reshap2")(x3)
    x = concatenate([x2_reshape, x3_reshape], axis=1, name='Concatenate1')
    x = Conv2D(50, (1, 8), padding='same', activation="relu", name="Conv4", kernel_initializer="glorot_uniform")(x)
    x = concatenate([x1, x], name="Concatenate2")
    x = Conv2D(100, (2, 5), padding="valid", activation="relu", name="Conv5", kernel_initializer="glorot_uniform")(x)

    # Part-B: TRemporal Characteristics Extraction Section
    x = Reshape(target_shape=((124, 100)))(x)
    x = CuDNNLSTM(units=128, return_sequences=True, name="LSTM1")(x)
    x = CuDNNLSTM(units=128, name="LSTM2")(x)

    # DNN
    x = Dense(128, activation="selu", name="FC1")(x)
    x = Dropout(dr)(x)
    x = Dense(128, activation="selu", name="FC2")(x)
    x = Dropout(dr)(x)
    x = Dense(classes, activation="softmax", name="Softmax")(x)

    model = Model(inputs=[input1, input2, input3], outputs=x)

    # Load weights.
    if weights is not None:
        model.load_weights(weights)

    return model

__________________________________________________________________________________________________
Layer (type)                    Output Shape         Param #     Connected to                     
==================================================================================================
Ichannel (InputLayer)           (None, 128, 1)       0                                            
__________________________________________________________________________________________________
Qchannel (InputLayer)           (None, 128, 1)       0                                            
__________________________________________________________________________________________________
Conv2 (Conv1D)                  (None, 128, 50)      450         Ichannel[0][0]                   
__________________________________________________________________________________________________
Conv3 (Conv1D)                  (None, 128, 50)      450         Qchannel[0][0]                   
__________________________________________________________________________________________________
reshape_1 (Reshape)             (None, 1, 128, 50)   0           Conv2[0][0]                      
__________________________________________________________________________________________________
reshap2 (Reshape)               (None, 1, 128, 50)   0           Conv3[0][0]                      
__________________________________________________________________________________________________
I/Qchannel (InputLayer)         (None, 2, 128, 1)    0                                            
__________________________________________________________________________________________________
Concatenate1 (Concatenate)      (None, 2, 128, 50)   0           reshape_1[0][0]                  
                                                                 reshap2[0][0]                    
__________________________________________________________________________________________________
Conv1 (Conv2D)                  (None, 2, 128, 50)   850         I/Qchannel[0][0]                 
__________________________________________________________________________________________________
Conv4 (Conv2D)                  (None, 2, 128, 50)   20050       Concatenate1[0][0]               
__________________________________________________________________________________________________
Concatenate2 (Concatenate)      (None, 2, 128, 100)  0           Conv1[0][0]                      
                                                                 Conv4[0][0]                      
__________________________________________________________________________________________________
Conv5 (Conv2D)                  (None, 1, 124, 100)  100100      Concatenate2[0][0]               
__________________________________________________________________________________________________
reshape_2 (Reshape)             (None, 124, 100)     0           Conv5[0][0]                      
__________________________________________________________________________________________________
LSTM1 (CuDNNLSTM)               (None, 124, 128)     117760      reshape_2[0][0]                  
__________________________________________________________________________________________________
LSTM2 (CuDNNLSTM)               (None, 128)          132096      LSTM1[0][0]                      
__________________________________________________________________________________________________
FC1 (Dense)                     (None, 128)          16512       LSTM2[0][0]                      
__________________________________________________________________________________________________
dropout_1 (Dropout)             (None, 128)          0           FC1[0][0]                        
__________________________________________________________________________________________________
FC2 (Dense)                     (None, 128)          16512       dropout_1[0][0]                  
__________________________________________________________________________________________________
dropout_2 (Dropout)             (None, 128)          0           FC2[0][0]                        
__________________________________________________________________________________________________
Softmax (Dense)                 (None, 11)           1419        dropout_2[0][0]                  
==================================================================================================
Total params: 406,199
Trainable params: 406,199
Non-trainable params: 0

4.MCMBNN

def _group_conv2d(x, filters, kernel, stride, groups):
    channel_axis = 1 if K.image_data_format() == 'channels_first' else -1
    in_channels = K.int_shape(x)[channel_axis]

    # number of input channels per group
    nb_ig = in_channels // groups
    # number of output channels per group
    nb_og = filters // groups

    gc_list = []
    # Determine whether the number of filters is divisible by the number of groups
    assert filters % groups == 0

    for i in range(groups):
        if channel_axis == -1:
            x_group = Lambda(lambda z: z[:, :, :,i * nb_ig: (i + 1) * nb_ig])(x)
        else:
            x_group = Lambda(lambda z: z[:, i * nb_ig: (i + 1) * nb_ig, :])(x)
        gc_list.append(Conv2D(filters=nb_og, kernel_size=kernel, strides=stride,
                              padding='same', use_bias=False)(x_group))

    return Concatenate(axis=channel_axis)(gc_list)

def cal1(x):
    y = tf.keras.backend.cos(x)
    return y

def cal2(x):
    y = tf.keras.backend.sin(x)
    return y
def MCLDNN(weights=None,
           input_shape1=[2, 128],
           input_shape2=[128, 1],
           input_shape0=[128, 2],
           input_shape22=[128],
           classes=11,
           **kwargs):
    if weights is not None and not (os.path.exists(weights)):
        raise ValueError('The `weights` argument should be either '
                         '`None` (random initialization), '
                         'or the path to the weights file to be loaded.')

    dr = 0.5

    input1 = Input(input_shape1 + [1], name='I/Qchannel')
    input2 = Input(input_shape2, name='Ichannel')
    input3 = Input(input_shape2, name='Qchannel')
    ##################PET#######################
    # input = Input(input_shape1+[1], name='input11')
    # input11 = Input(input_shape2, name='input22')
    # input22 = Input(input_shape2, name='input33')
    x1f = Flatten()(input1)
    x1f = Dense(1, name='fc2')(x1f)
    x1f = Activation('linear')(x1f)

    cos1= Lambda(cal1)(x1f)
    sin1 = Lambda(cal2)(x1f)
    x11f = Multiply()([input2, cos1])
    x12f = Multiply()([input3, sin1])
    x21f = Multiply()([input3, cos1])
    x22f = Multiply()([input2, sin1])
    y1 = Add()([x11f,x12f])
    y2 = Subtract()([x21f,x22f])
    y1 = Reshape(target_shape=(128, 1), name='reshape1')(y1)
    y2 = Reshape(target_shape=(128, 1), name='reshape2')(y2)
    x11f = concatenate([y1, y2])
    x3f = Reshape(target_shape=((128, 2, 1)), name='reshape3')(x11f)
    ####################################################

    # Part-A: Multi-channel Inputs and Spatial Characteristics Mapping Section
    # x1=Conv2D(50,(2,8),padding='same',activation="relu",name="Conv1",kernel_initializer="glorot_uniform")(input1)
    xm0 = Conv2D(20, (2, 8), padding='same',activation="relu", kernel_initializer='glorot_normal', data_format='channels_last')(input1)
    # xm0 = MaxPooling2D(pool_size=(1,2), strides=(1,2), padding='valid', data_format='channels_last')(xm0)
    xm1 = Conv2D(20, (8, 2), padding='same',activation="relu", kernel_initializer='glorot_normal', data_format='channels_last')(input1)
    # xm1 = MaxPooling2D(pool_size=(1,2), strides=(1,2), padding='valid', data_format='channels_last')(xm1)
    xm2 = Conv2D(10, (1, 1), padding='same',activation="relu", kernel_initializer='glorot_normal', data_format='channels_last')(input1)
    # xm2 = MaxPooling2D(pool_size=(1,2), strides=(1,2), padding='valid', data_format='channels_last')(xm2)
    xm = concatenate([xm0, xm1], axis=3)
    x1 = concatenate([xm, xm2], axis=3)
    x1 = Activation('relu')(x1)
    x1 = Dropout(dr)(x1)

    # x2=Conv1D(50,8,padding='causal',activation="relu",name="Conv2",kernel_initializer="glorot_uniform")(input2)
    xc0 = Conv1D(20, 2, padding='causal',activation="relu", kernel_initializer='glorot_normal', data_format='channels_last')(input2)
    xc1 = Conv1D(20, 4, padding='causal',activation="relu", kernel_initializer='glorot_normal', data_format='channels_last')(input2)
    xc2 = Conv1D(10, 8, padding='causal',activation="relu", kernel_initializer='glorot_normal', data_format='channels_last')(input2)
    xc = concatenate([xc0, xc1], axis=2)
    x2 = concatenate([xc, xc2], axis=2)
    x2 = Activation('relu')(x2)
    x2 = Dropout(dr)(x2)
    x2_reshape = Reshape([-1, 128, 50])(x2)

    # x3=Conv1D(50,8,padding='causal',activation="relu",name="Conv3",kernel_initializer="glorot_uniform")(input3)
    xd0 = Conv1D(20, 2, padding='causal', kernel_initializer='glorot_normal', data_format='channels_last')(input3)
    xd1 = Conv1D(20, 4, padding='causal', kernel_initializer='glorot_normal', data_format='channels_last')(input3)
    xd2 = Conv1D(10, 8, padding='causal', kernel_initializer='glorot_normal', data_format='channels_last')(input3)
    xd = concatenate([xd0, xd1], axis=2)
    xd = concatenate([xd, xd2], axis=2)
    xd = Activation('relu')(xd)
    xd = Dropout(dr)(xd)
    x3_reshape = Reshape([-1, 128, 50], name="reshap2")(xd)

    x = concatenate([x2_reshape, x3_reshape], axis=1, name='Concatenate1')
    xm0 = Conv2D(20, (1, 8), padding='same',activation="relu", kernel_initializer='glorot_normal', data_format='channels_last')(x)
    xm1 = Conv2D(20, (8, 1), padding='same',activation="relu", kernel_initializer='glorot_normal', data_format='channels_last')(x)
    xm2 = Conv2D(10, (1, 1), padding='same',activation="relu", kernel_initializer='glorot_normal', data_format='channels_last')(x)
    xm = concatenate([xm0, xm1], axis=3)
    x = concatenate([xm, xm2], axis=3)
    x = Activation('relu')(x)
    x = Dropout(dr)(x)
    x = concatenate([x1, x], name="Concatenate2")
    # x = _group_conv(x, 100, (2,5), 2)
    x = _group_conv2d(x, filters=50, kernel=(3, 3), stride=(1, 1), groups=2)
    x = Activation('relu')(x)
    x3f = Reshape([2, 128,1])(x3f)
    x = Add()([x3f, x])

    # Part-B: TRemporal Characteristics Extraction Section
    x = Reshape(target_shape=((256, 50)))(x)
    x = CuDNNGRU(units=128)(x)
    x = Dropout(dr)(x)

    # DNN
    x = Dense(classes, activation="softmax", name="Softmax")(x)

    model = Model(inputs=[input1, input2, input3], outputs=x)

    # Load weights.
    if weights is not None:
        model.load_weights(weights)

    return model

__________________________________________________________________________________________________
Layer (type)                    Output Shape         Param #     Connected to                     
==================================================================================================
Ichannel (InputLayer)           (None, 128, 1)       0                                            
__________________________________________________________________________________________________
Qchannel (InputLayer)           (None, 128, 1)       0                                            
__________________________________________________________________________________________________
conv1d_1 (Conv1D)               (None, 128, 20)      60          Ichannel[0][0]                   
__________________________________________________________________________________________________
conv1d_2 (Conv1D)               (None, 128, 20)      100         Ichannel[0][0]                   
__________________________________________________________________________________________________
conv1d_4 (Conv1D)               (None, 128, 20)      60          Qchannel[0][0]                   
__________________________________________________________________________________________________
conv1d_5 (Conv1D)               (None, 128, 20)      100         Qchannel[0][0]                   
__________________________________________________________________________________________________
concatenate_4 (Concatenate)     (None, 128, 40)      0           conv1d_1[0][0]                   
                                                                 conv1d_2[0][0]                   
__________________________________________________________________________________________________
conv1d_3 (Conv1D)               (None, 128, 10)      90          Ichannel[0][0]                   
__________________________________________________________________________________________________
concatenate_6 (Concatenate)     (None, 128, 40)      0           conv1d_4[0][0]                   
                                                                 conv1d_5[0][0]                   
__________________________________________________________________________________________________
conv1d_6 (Conv1D)               (None, 128, 10)      90          Qchannel[0][0]                   
__________________________________________________________________________________________________
concatenate_5 (Concatenate)     (None, 128, 50)      0           concatenate_4[0][0]              
                                                                 conv1d_3[0][0]                   
__________________________________________________________________________________________________
concatenate_7 (Concatenate)     (None, 128, 50)      0           concatenate_6[0][0]              
                                                                 conv1d_6[0][0]                   
__________________________________________________________________________________________________
activation_3 (Activation)       (None, 128, 50)      0           concatenate_5[0][0]              
__________________________________________________________________________________________________
activation_4 (Activation)       (None, 128, 50)      0           concatenate_7[0][0]              
__________________________________________________________________________________________________
dropout_2 (Dropout)             (None, 128, 50)      0           activation_3[0][0]               
__________________________________________________________________________________________________
dropout_3 (Dropout)             (None, 128, 50)      0           activation_4[0][0]               
__________________________________________________________________________________________________
reshape_1 (Reshape)             (None, 1, 128, 50)   0           dropout_2[0][0]                  
__________________________________________________________________________________________________
reshap2 (Reshape)               (None, 1, 128, 50)   0           dropout_3[0][0]                  
__________________________________________________________________________________________________
I/Qchannel (InputLayer)         (None, 2, 128, 1)    0                                            
__________________________________________________________________________________________________
Concatenate1 (Concatenate)      (None, 2, 128, 50)   0           reshape_1[0][0]                  
                                                                 reshap2[0][0]                    
__________________________________________________________________________________________________
flatten_1 (Flatten)             (None, 256)          0           I/Qchannel[0][0]                 
__________________________________________________________________________________________________
conv2d_1 (Conv2D)               (None, 2, 128, 20)   340         I/Qchannel[0][0]                 
__________________________________________________________________________________________________
conv2d_2 (Conv2D)               (None, 2, 128, 20)   340         I/Qchannel[0][0]                 
__________________________________________________________________________________________________
conv2d_4 (Conv2D)               (None, 2, 128, 20)   8020        Concatenate1[0][0]               
__________________________________________________________________________________________________
conv2d_5 (Conv2D)               (None, 2, 128, 20)   8020        Concatenate1[0][0]               
__________________________________________________________________________________________________
fc2 (Dense)                     (None, 1)            257         flatten_1[0][0]                  
__________________________________________________________________________________________________
concatenate_2 (Concatenate)     (None, 2, 128, 40)   0           conv2d_1[0][0]                   
                                                                 conv2d_2[0][0]                   
__________________________________________________________________________________________________
conv2d_3 (Conv2D)               (None, 2, 128, 10)   20          I/Qchannel[0][0]                 
__________________________________________________________________________________________________
concatenate_8 (Concatenate)     (None, 2, 128, 40)   0           conv2d_4[0][0]                   
                                                                 conv2d_5[0][0]                   
__________________________________________________________________________________________________
conv2d_6 (Conv2D)               (None, 2, 128, 10)   510         Concatenate1[0][0]               
__________________________________________________________________________________________________
activation_1 (Activation)       (None, 1)            0           fc2[0][0]                        
__________________________________________________________________________________________________
concatenate_3 (Concatenate)     (None, 2, 128, 50)   0           concatenate_2[0][0]              
                                                                 conv2d_3[0][0]                   
__________________________________________________________________________________________________
concatenate_9 (Concatenate)     (None, 2, 128, 50)   0           concatenate_8[0][0]              
                                                                 conv2d_6[0][0]                   
__________________________________________________________________________________________________
lambda_1 (Lambda)               (None, 1)            0           activation_1[0][0]               
__________________________________________________________________________________________________
lambda_2 (Lambda)               (None, 1)            0           activation_1[0][0]               
__________________________________________________________________________________________________
activation_2 (Activation)       (None, 2, 128, 50)   0           concatenate_3[0][0]              
__________________________________________________________________________________________________
activation_5 (Activation)       (None, 2, 128, 50)   0           concatenate_9[0][0]              
__________________________________________________________________________________________________
multiply_1 (Multiply)           (None, 128, 1)       0           Ichannel[0][0]                   
                                                                 lambda_1[0][0]                   
__________________________________________________________________________________________________
multiply_2 (Multiply)           (None, 128, 1)       0           Qchannel[0][0]                   
                                                                 lambda_2[0][0]                   
__________________________________________________________________________________________________
multiply_3 (Multiply)           (None, 128, 1)       0           Qchannel[0][0]                   
                                                                 lambda_1[0][0]                   
__________________________________________________________________________________________________
multiply_4 (Multiply)           (None, 128, 1)       0           Ichannel[0][0]                   
                                                                 lambda_2[0][0]                   
__________________________________________________________________________________________________
dropout_1 (Dropout)             (None, 2, 128, 50)   0           activation_2[0][0]               
__________________________________________________________________________________________________
dropout_4 (Dropout)             (None, 2, 128, 50)   0           activation_5[0][0]               
__________________________________________________________________________________________________
add_1 (Add)                     (None, 128, 1)       0           multiply_1[0][0]                 
                                                                 multiply_2[0][0]                 
__________________________________________________________________________________________________
subtract_1 (Subtract)           (None, 128, 1)       0           multiply_3[0][0]                 
                                                                 multiply_4[0][0]                 
__________________________________________________________________________________________________
Concatenate2 (Concatenate)      (None, 2, 128, 100)  0           dropout_1[0][0]                  
                                                                 dropout_4[0][0]                  
__________________________________________________________________________________________________
reshape1 (Reshape)              (None, 128, 1)       0           add_1[0][0]                      
__________________________________________________________________________________________________
reshape2 (Reshape)              (None, 128, 1)       0           subtract_1[0][0]                 
__________________________________________________________________________________________________
lambda_3 (Lambda)               (None, 2, 128, 50)   0           Concatenate2[0][0]               
__________________________________________________________________________________________________
lambda_4 (Lambda)               (None, 2, 128, 50)   0           Concatenate2[0][0]               
__________________________________________________________________________________________________
concatenate_1 (Concatenate)     (None, 128, 2)       0           reshape1[0][0]                   
                                                                 reshape2[0][0]                   
__________________________________________________________________________________________________
conv2d_7 (Conv2D)               (None, 2, 128, 25)   11250       lambda_3[0][0]                   
__________________________________________________________________________________________________
conv2d_8 (Conv2D)               (None, 2, 128, 25)   11250       lambda_4[0][0]                   
__________________________________________________________________________________________________
reshape3 (Reshape)              (None, 128, 2, 1)    0           concatenate_1[0][0]              
__________________________________________________________________________________________________
concatenate_10 (Concatenate)    (None, 2, 128, 50)   0           conv2d_7[0][0]                   
                                                                 conv2d_8[0][0]                   
__________________________________________________________________________________________________
reshape_2 (Reshape)             (None, 2, 128, 1)    0           reshape3[0][0]                   
__________________________________________________________________________________________________
activation_6 (Activation)       (None, 2, 128, 50)   0           concatenate_10[0][0]             
__________________________________________________________________________________________________
add_2 (Add)                     (None, 2, 128, 50)   0           reshape_2[0][0]                  
                                                                 activation_6[0][0]               
__________________________________________________________________________________________________
reshape_3 (Reshape)             (None, 256, 50)      0           add_2[0][0]                      
__________________________________________________________________________________________________
cu_dnngru_1 (CuDNNGRU)          (None, 128)          69120       reshape_3[0][0]                  
__________________________________________________________________________________________________
dropout_5 (Dropout)             (None, 128)          0           cu_dnngru_1[0][0]                
__________________________________________________________________________________________________
Softmax (Dense)                 (None, 11)           1419        dropout_5[0][0]                  
==================================================================================================
Total params: 111,046
Trainable params: 111,046
Non-trainable params: 0

5.CNN

def CNN():
    # build the CNN model
    in_shp = [2, 128]
    xm_input = Input(in_shp)
    xm = Reshape(in_shp + [1], input_shape=in_shp)(xm_input)
    xm = Conv2D(128, (3, 3), padding='same', kernel_initializer='glorot_normal', data_format=data_format,
                activation="relu")(xm)
    # xm = BatchNormalization()(xm)
    xm = Dropout(dr)(xm)

    xm = Conv2D(64, (3, 3), padding='same', kernel_initializer='glorot_normal', data_format=data_format,
                activation="relu")(xm)
    # xm = BatchNormalization()(xm)
    xm = Dropout(dr)(xm)

    xm = Flatten()(xm)

    # DNN
    xm = Dense(128, activation="relu", name="FC1")(xm)
    xm = Dropout(dr)(xm)
    xm = Dense(128, activation="relu", name="FC2")(xm)
    xm = Dropout(dr)(xm)
    xm = Dense(11, activation="softmax", name="Softmax")(xm)
    model = Model(inputs=xm_input, outputs=xm)
    model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])
    return model

channels_last
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
input_1 (InputLayer)         (None, 2, 128)            0         
_________________________________________________________________
reshape_1 (Reshape)          (None, 2, 128, 1)         0         
_________________________________________________________________
conv2d_1 (Conv2D)            (None, 2, 128, 128)       1280      
_________________________________________________________________
dropout_1 (Dropout)          (None, 2, 128, 128)       0         
_________________________________________________________________
conv2d_2 (Conv2D)            (None, 2, 128, 64)        73792     
_________________________________________________________________
dropout_2 (Dropout)          (None, 2, 128, 64)        0         
_________________________________________________________________
flatten_1 (Flatten)          (None, 16384)             0         
_________________________________________________________________
FC1 (Dense)                  (None, 128)               2097280   
_________________________________________________________________
dropout_3 (Dropout)          (None, 128)               0         
_________________________________________________________________
FC2 (Dense)                  (None, 128)               16512     
_________________________________________________________________
dropout_4 (Dropout)          (None, 128)               0         
_________________________________________________________________
Softmax (Dense)              (None, 11)                1419      
=================================================================
Total params: 2,190,283
Trainable params: 2,190,283
Non-trainable params: 0

6.CNNLSTM

def MCLDNN():
    # build the CNN model
    in_shp = [2, 128]
    L = 128  # sample points
    xm_input = Input(in_shp)
    xm = Reshape([128, 2], input_shape=in_shp)(xm_input)
    # build the CNN model
    in_shp = [2, 128]
    xm_input = Input(in_shp)  #
    xm = Reshape(in_shp + [1], input_shape=in_shp)(xm_input)
    xm = Conv2D(128, (3, 3), padding='same', kernel_initializer='glorot_normal', data_format=data_format,
                activation="relu")(xm)
    # xm = BatchNormalization()(xm)
    xm = Dropout(dr)(xm)

    xm = Conv2D(64, (3, 3), padding='same', kernel_initializer='glorot_normal', data_format=data_format,
                activation="relu")(xm)
    # xm = BatchNormalization()(xm)
    xm = Dropout(dr)(xm)

    # Part-B: TRemporal Characteristics Extraction Section
    xm = Reshape(target_shape=((256, 64)))(xm)
    xm = CuDNNLSTM(units=128, return_sequences=True, name="LSTM1")(xm)
    xm = CuDNNLSTM(units=128, name="LSTM2")(xm)

    # DNN
    xm = Dense(128, activation="selu", name="FC1")(xm)
    xm = Dropout(dr)(xm)
    xm = Dense(128, activation="selu", name="FC2")(xm)
    xm = Dropout(dr)(xm)
    xm = Dense(11, activation="softmax", name="Softmax")(xm)

    model = Model(inputs=xm_input, outputs=xm)
    model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])
    return model

channels_last
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
input_2 (InputLayer)         (None, 2, 128)            0         
_________________________________________________________________
reshape_2 (Reshape)          (None, 2, 128, 1)         0         
_________________________________________________________________
conv2d_1 (Conv2D)            (None, 2, 128, 128)       1280      
_________________________________________________________________
dropout_1 (Dropout)          (None, 2, 128, 128)       0         
_________________________________________________________________
conv2d_2 (Conv2D)            (None, 2, 128, 64)        73792     
_________________________________________________________________
dropout_2 (Dropout)          (None, 2, 128, 64)        0         
_________________________________________________________________
reshape_3 (Reshape)          (None, 256, 64)           0         
_________________________________________________________________
LSTM1 (CuDNNLSTM)            (None, 256, 128)          99328     
_________________________________________________________________
LSTM2 (CuDNNLSTM)            (None, 128)               132096    
_________________________________________________________________
FC1 (Dense)                  (None, 128)               16512     
_________________________________________________________________
dropout_3 (Dropout)          (None, 128)               0         
_________________________________________________________________
FC2 (Dense)                  (None, 128)               16512     
_________________________________________________________________
dropout_4 (Dropout)          (None, 128)               0         
_________________________________________________________________
Softmax (Dense)              (None, 11)                1419      
=================================================================
Total params: 340,939
Trainable params: 340,939
Non-trainable params: 0

7.MCNET

concat_axis = 3

# pre_block
# pre_B_conv1_size = (1, 3)
# pre_B_conv2_size = (3, 1)
# pre_B_pool_size = (1, 2)
def pre_block(xm, conv1_size, conv2_size, pool_size, mcSeq):
    print('pre_block')
    base = xm
    xm0 = Conv2D(32, conv1_size, padding='same', activation="relu", name=mcSeq + "_pre_block_conv1", kernel_initializer='glorot_normal', data_format='channels_last')(base)
    xm0 = AveragePooling2D(pool_size=pool_size, strides=pool_size, padding='valid', data_format='channels_last')(xm0)
    xm1 = Conv2D(32, conv2_size, padding='same', activation="relu", name=mcSeq + "_pre_block_conv2", kernel_initializer='glorot_normal', data_format='channels_last')(base)
    xm1 = MaxPooling2D(pool_size=pool_size, strides=pool_size, padding='valid', data_format='channels_last')(xm1)
    xm = concatenate([xm0, xm1], axis=concat_axis)
    return xm


# m_block
def m_block(xm, filters_size01, filters_size02, filters_size03, conv0_size, conv1_size, conv2_size, conv3_size, mcSeq):
    print('m-block')
    base = xm
    base_xm = Conv2D(filters_size01, conv0_size, padding='same', activation="relu", name=mcSeq + "_m_block_conv0", kernel_initializer='glorot_normal', data_format='channels_last')(base)
    xm0 = Conv2D(filters_size02, conv1_size, padding='same', activation="relu", name=mcSeq + "_m_block_conv1", kernel_initializer='glorot_normal', data_format='channels_last')(base_xm)
    xm1 = Conv2D(filters_size02, conv2_size, padding='same', activation="relu", name=mcSeq + "_m_block_conv2", kernel_initializer='glorot_normal', data_format='channels_last')(base_xm)
    xm2 = Conv2D(filters_size03, conv3_size, padding='same', activation="relu", name=mcSeq + "_m_block_conv3", kernel_initializer='glorot_normal', data_format='channels_last')(base_xm)
    xm = concatenate([xm0, xm1], axis=concat_axis)
    xm = concatenate([xm, xm2], axis=concat_axis)
    # concat(xm0, xm1, xm2)
    return xm


# m_block_pool
# m_Bp1_conv0_size = (1, 1)
# m_Bp1_conv1_size = (3, 1)
# m_Bp1_conv2_size = (1, 3)
# m_Bp1_conv3_size = (1, 1)
# m_Bp1_pool_size = (1, 2)
def m_block_p(xm, conv0_size, conv1_size, conv2_size, conv3_size, pool_size, mcSeq):
    print('m_block_pool')
    base = xm
    base_xm = Conv2D(32, conv0_size, padding='same', activation="relu", name=mcSeq + "_m_block_p_conv0", kernel_initializer='glorot_normal', data_format='channels_last')(base)
    xm0 = Conv2D(48, conv1_size, padding='same', activation="relu", name=mcSeq + "_pre_block_p_conv1", kernel_initializer='glorot_normal', data_format='channels_last')(base_xm)
    xm0 = MaxPooling2D(pool_size=pool_size, strides=pool_size, padding='valid', data_format='channels_last')(xm0)
    xm1 = Conv2D(48, conv2_size, padding='same', activation="relu", name=mcSeq + "_pre_block_p_conv2", kernel_initializer='glorot_normal', data_format='channels_last')(base_xm)
    xm1 = MaxPooling2D(pool_size=pool_size, strides=pool_size, padding='valid', data_format='channels_last')(xm1)
    xm2 = Conv2D(32, conv3_size, padding='same', activation="relu", name=mcSeq + "_pre_block_p_conv3", kernel_initializer='glorot_normal', data_format='channels_last')(base_xm)
    xm2 = MaxPooling2D(pool_size=pool_size, strides=pool_size, padding='valid', data_format='channels_last')(xm2)
    xm = concatenate([xm0, xm1], axis=concat_axis)
    xm = concatenate([xm, xm2], axis=concat_axis)
    return xm

def MCLDNN():
    in_shp = [2, 128]
    xm_input = Input(in_shp)

    # # Reshape() [1,128,2]
    xm = Reshape([2, 128, 1], input_shape=in_shp)(xm_input)
    xm = Conv2D(64, kernel_size=(3, 7), strides=(1, 1), padding='same', activation="relu", name='conv0', kernel_initializer='glorot_normal', data_format='channels_last')(xm)
    xm = MaxPooling2D(pool_size=(1, 2), strides=(1, 2), padding='valid', data_format='channels_last')(xm)

##########################################################################################################
    mcPreBName = 'mc_net01'
    pre_B_conv1_size = (1, 3)
    pre_B_conv2_size = (3, 1)
    pre_B_pool_size = (1, 2)
    xm = pre_block(xm, pre_B_conv1_size, pre_B_conv2_size, pre_B_pool_size, mcPreBName)

    jumpPool1_size = (1, 2)
    jumpStrides1_size = (1, 2)
    xm = Conv2D(64, kernel_size=(1, 1), strides=(1, 1), padding='same')(xm)
    xm_tmp1 = MaxPooling2D(pool_size=jumpPool1_size, strides=jumpStrides1_size, padding='valid', data_format='channels_last')(xm)
    xm_tmp1 = Reshape([2, 8, 128])(xm_tmp1)
    # pool1----->MaxPooling2D()
    xm = MaxPooling2D(pool_size=(1, 2), strides=(1, 2), padding='valid', data_format='channels_last')(xm)

    # m_Bp1----->m_block_p
    # m_block_p(xm, conv0_size, conv1_size, conv2_size, conv3_size, pool_size, mcSeq)
    mcMBp1Name = 'mc_net02'
    m_Bp1_conv0_size = (1, 1)
    m_Bp1_conv1_size = (3, 1)
    m_Bp1_conv2_size = (1, 3)
    m_Bp1_conv3_size = (1, 1)
    m_Bp1_pool_size = (1, 2)
    xm = m_block_p(xm, m_Bp1_conv0_size, m_Bp1_conv1_size, m_Bp1_conv2_size, m_Bp1_conv3_size, m_Bp1_pool_size, mcMBp1Name)
    # add  M-block 1
    xm = keras.layers.Add()([xm, xm_tmp1])

    xm_tmp2 = xm
    # m_B1----->m_block
    # m_block(xm, conv0_size, conv1_size, conv2_size, conv3_size, mcSeq)
    mcMB1Name = 'mc_net03'
    m_B1_filter_size01 = 32
    m_B1_filter_size02 = 48
    m_B1_filter_size03 = 32
    m_B1_conv0_size = (1, 1)
    m_B1_conv1_size = (1, 3)
    m_B1_conv2_size = (3, 1)
    m_B1_conv3_size = (1, 1)
    xm = m_block(xm, m_B1_filter_size01, m_B1_filter_size02, m_B1_filter_size03, m_B1_conv0_size, m_B1_conv1_size, m_B1_conv2_size, m_B1_conv3_size, mcMB1Name)
    # add  M-block 2
    # xm =concatenate([xm, xm_tmp2], axis=concat_axis)
    xm = keras.layers.Add()([xm, xm_tmp2])

    xm_tmp3 = xm
    # poolJump1----->MaxPooling2D()
    jumpPool2_size = (1, 2)
    jumpStrides2_size = (1, 2)
    xm_tmp3_pool = MaxPooling2D(pool_size=jumpPool2_size, strides=jumpStrides2_size, padding='valid', data_format='channels_last')(xm_tmp3)
    # M-block-p
    mcMBp2Name = 'mc_net04'
    m_Bp2_conv0_size = (1, 1)
    m_Bp2_conv1_size = (1, 3)
    m_Bp2_conv2_size = (3, 1)
    m_Bp2_conv3_size = (1, 1)
    m_Bp2_pool_size = (1, 2)
    xm = m_block_p(xm, m_Bp2_conv1_size, m_Bp2_conv1_size, m_Bp2_conv2_size, m_Bp2_conv3_size, m_Bp2_pool_size, mcMBp2Name)
    # add  M-block 3
    # xm = concatenate([xm, xm_tmp3_pool], axis=concat_axis)
    xm = keras.layers.Add()([xm, xm_tmp3_pool])

    xm_tmp4 = xm
    # M-block
    mcMB2Name = 'mc_net05'
    m_B2_filter_size01 = 32
    m_B2_filter_size02 = 48
    m_B2_filter_size03 = 32
    m_B2_conv0_size = (1, 1)
    m_B2_conv1_size = (1, 3)
    m_B2_conv2_size = (3, 1)
    m_B2_conv3_size = (1, 3)
    xm = m_block(xm, m_B2_filter_size01, m_B2_filter_size02, m_B2_filter_size03, m_B2_conv0_size, m_B2_conv1_size, m_B2_conv2_size, m_B2_conv3_size, mcMB2Name)
    # add  M-block 4
    # xm = concatenate([xm, xm_tmp4], axis=concat_axis)
    xm = keras.layers.Add()([xm, xm_tmp4])
    xm_tmp5 = xm
    xm = concatenate([xm, xm_tmp4], axis=concat_axis)
    # xm = keras.layers.Add()([xm, xm_tmp4])

    ############################################################################################################

    # pool2----->avg-pool----->AveragePooling2D()
    xm = AveragePooling2D(pool_size=(2, 1), strides=(1, 2), padding='valid', data_format='channels_last')(xm)
    xm = BatchNormalization()(xm)
    xm = Flatten()(xm)
    # dense fc
    xm = Dense(11, kernel_initializer='glorot_normal', name="dense3",activation='softmax')(xm)

    # xm = Lambda(squeeze_dim, name='sqe_dim')(xm)
    model = Model(inputs=xm_input, outputs=xm)
    adam = keras.optimizers.Adam(lr=0.0001, beta_1=0.9, beta_2=0.999, epsilon=None, decay=0.0, amsgrad=False)
    model.compile(loss='categorical_crossentropy', optimizer=adam, metrics=['accuracy'])
    model.summary()
    return model

__________________________________________________________________________________________________
Layer (type)                    Output Shape         Param #     Connected to                     
==================================================================================================
input_1 (InputLayer)            (None, 2, 128)       0                                            
__________________________________________________________________________________________________
reshape_1 (Reshape)             (None, 2, 128, 1)    0           input_1[0][0]                    
__________________________________________________________________________________________________
conv0 (Conv2D)                  (None, 2, 128, 64)   1408        reshape_1[0][0]                  
__________________________________________________________________________________________________
max_pooling2d_1 (MaxPooling2D)  (None, 2, 64, 64)    0           conv0[0][0]                      
__________________________________________________________________________________________________
mc_net01_pre_block_conv1 (Conv2 (None, 2, 64, 32)    6176        max_pooling2d_1[0][0]            
__________________________________________________________________________________________________
mc_net01_pre_block_conv2 (Conv2 (None, 2, 64, 32)    6176        max_pooling2d_1[0][0]            
__________________________________________________________________________________________________
average_pooling2d_1 (AveragePoo (None, 2, 32, 32)    0           mc_net01_pre_block_conv1[0][0]   
__________________________________________________________________________________________________
max_pooling2d_2 (MaxPooling2D)  (None, 2, 32, 32)    0           mc_net01_pre_block_conv2[0][0]   
__________________________________________________________________________________________________
concatenate_1 (Concatenate)     (None, 2, 32, 64)    0           average_pooling2d_1[0][0]        
                                                                 max_pooling2d_2[0][0]            
__________________________________________________________________________________________________
conv2d_1 (Conv2D)               (None, 2, 32, 64)    4160        concatenate_1[0][0]              
__________________________________________________________________________________________________
max_pooling2d_4 (MaxPooling2D)  (None, 2, 16, 64)    0           conv2d_1[0][0]                   
__________________________________________________________________________________________________
mc_net02_m_block_p_conv0 (Conv2 (None, 2, 16, 32)    2080        max_pooling2d_4[0][0]            
__________________________________________________________________________________________________
mc_net02_pre_block_p_conv1 (Con (None, 2, 16, 48)    4656        mc_net02_m_block_p_conv0[0][0]   
__________________________________________________________________________________________________
mc_net02_pre_block_p_conv2 (Con (None, 2, 16, 48)    4656        mc_net02_m_block_p_conv0[0][0]   
__________________________________________________________________________________________________
max_pooling2d_5 (MaxPooling2D)  (None, 2, 8, 48)     0           mc_net02_pre_block_p_conv1[0][0] 
__________________________________________________________________________________________________
max_pooling2d_6 (MaxPooling2D)  (None, 2, 8, 48)     0           mc_net02_pre_block_p_conv2[0][0] 
__________________________________________________________________________________________________
mc_net02_pre_block_p_conv3 (Con (None, 2, 16, 32)    1056        mc_net02_m_block_p_conv0[0][0]   
__________________________________________________________________________________________________
concatenate_2 (Concatenate)     (None, 2, 8, 96)     0           max_pooling2d_5[0][0]            
                                                                 max_pooling2d_6[0][0]            
__________________________________________________________________________________________________
max_pooling2d_7 (MaxPooling2D)  (None, 2, 8, 32)     0           mc_net02_pre_block_p_conv3[0][0] 
__________________________________________________________________________________________________
max_pooling2d_3 (MaxPooling2D)  (None, 2, 16, 64)    0           conv2d_1[0][0]                   
__________________________________________________________________________________________________
concatenate_3 (Concatenate)     (None, 2, 8, 128)    0           concatenate_2[0][0]              
                                                                 max_pooling2d_7[0][0]            
__________________________________________________________________________________________________
reshape_2 (Reshape)             (None, 2, 8, 128)    0           max_pooling2d_3[0][0]            
__________________________________________________________________________________________________
add_1 (Add)                     (None, 2, 8, 128)    0           concatenate_3[0][0]              
                                                                 reshape_2[0][0]                  
__________________________________________________________________________________________________
mc_net03_m_block_conv0 (Conv2D) (None, 2, 8, 32)     4128        add_1[0][0]                      
__________________________________________________________________________________________________
mc_net03_m_block_conv1 (Conv2D) (None, 2, 8, 48)     4656        mc_net03_m_block_conv0[0][0]     
__________________________________________________________________________________________________
mc_net03_m_block_conv2 (Conv2D) (None, 2, 8, 48)     4656        mc_net03_m_block_conv0[0][0]     
__________________________________________________________________________________________________
concatenate_4 (Concatenate)     (None, 2, 8, 96)     0           mc_net03_m_block_conv1[0][0]     
                                                                 mc_net03_m_block_conv2[0][0]     
__________________________________________________________________________________________________
mc_net03_m_block_conv3 (Conv2D) (None, 2, 8, 32)     1056        mc_net03_m_block_conv0[0][0]     
__________________________________________________________________________________________________
concatenate_5 (Concatenate)     (None, 2, 8, 128)    0           concatenate_4[0][0]              
                                                                 mc_net03_m_block_conv3[0][0]     
__________________________________________________________________________________________________
add_2 (Add)                     (None, 2, 8, 128)    0           concatenate_5[0][0]              
                                                                 add_1[0][0]                      
__________________________________________________________________________________________________
mc_net04_m_block_p_conv0 (Conv2 (None, 2, 8, 32)     12320       add_2[0][0]                      
__________________________________________________________________________________________________
mc_net04_pre_block_p_conv1 (Con (None, 2, 8, 48)     4656        mc_net04_m_block_p_conv0[0][0]   
__________________________________________________________________________________________________
mc_net04_pre_block_p_conv2 (Con (None, 2, 8, 48)     4656        mc_net04_m_block_p_conv0[0][0]   
__________________________________________________________________________________________________
max_pooling2d_9 (MaxPooling2D)  (None, 2, 4, 48)     0           mc_net04_pre_block_p_conv1[0][0] 
__________________________________________________________________________________________________
max_pooling2d_10 (MaxPooling2D) (None, 2, 4, 48)     0           mc_net04_pre_block_p_conv2[0][0] 
__________________________________________________________________________________________________
mc_net04_pre_block_p_conv3 (Con (None, 2, 8, 32)     1056        mc_net04_m_block_p_conv0[0][0]   
__________________________________________________________________________________________________
concatenate_6 (Concatenate)     (None, 2, 4, 96)     0           max_pooling2d_9[0][0]            
                                                                 max_pooling2d_10[0][0]           
__________________________________________________________________________________________________
max_pooling2d_11 (MaxPooling2D) (None, 2, 4, 32)     0           mc_net04_pre_block_p_conv3[0][0] 
__________________________________________________________________________________________________
concatenate_7 (Concatenate)     (None, 2, 4, 128)    0           concatenate_6[0][0]              
                                                                 max_pooling2d_11[0][0]           
__________________________________________________________________________________________________
max_pooling2d_8 (MaxPooling2D)  (None, 2, 4, 128)    0           add_2[0][0]                      
__________________________________________________________________________________________________
add_3 (Add)                     (None, 2, 4, 128)    0           concatenate_7[0][0]              
                                                                 max_pooling2d_8[0][0]            
__________________________________________________________________________________________________
mc_net05_m_block_conv0 (Conv2D) (None, 2, 4, 32)     4128        add_3[0][0]                      
__________________________________________________________________________________________________
mc_net05_m_block_conv1 (Conv2D) (None, 2, 4, 48)     4656        mc_net05_m_block_conv0[0][0]     
__________________________________________________________________________________________________
mc_net05_m_block_conv2 (Conv2D) (None, 2, 4, 48)     4656        mc_net05_m_block_conv0[0][0]     
__________________________________________________________________________________________________
concatenate_8 (Concatenate)     (None, 2, 4, 96)     0           mc_net05_m_block_conv1[0][0]     
                                                                 mc_net05_m_block_conv2[0][0]     
__________________________________________________________________________________________________
mc_net05_m_block_conv3 (Conv2D) (None, 2, 4, 32)     3104        mc_net05_m_block_conv0[0][0]     
__________________________________________________________________________________________________
concatenate_9 (Concatenate)     (None, 2, 4, 128)    0           concatenate_8[0][0]              
                                                                 mc_net05_m_block_conv3[0][0]     
__________________________________________________________________________________________________
add_4 (Add)                     (None, 2, 4, 128)    0           concatenate_9[0][0]              
                                                                 add_3[0][0]                      
__________________________________________________________________________________________________
concatenate_10 (Concatenate)    (None, 2, 4, 256)    0           add_4[0][0]                      
                                                                 add_3[0][0]                      
__________________________________________________________________________________________________
average_pooling2d_2 (AveragePoo (None, 1, 2, 256)    0           concatenate_10[0][0]             
__________________________________________________________________________________________________
batch_normalization_1 (BatchNor (None, 1, 2, 256)    1024        average_pooling2d_2[0][0]        
__________________________________________________________________________________________________
flatten_1 (Flatten)             (None, 512)          0           batch_normalization_1[0][0]      
__________________________________________________________________________________________________
dense3 (Dense)                  (None, 11)           5643        flatten_1[0][0]                  
==================================================================================================
Total params: 90,763
Trainable params: 90,251
Non-trainable params: 512

8.消融实验

8.1移除多信道

def MCLDNN():
    # build the CNN model
    in_shp = [2, 128]
    xm_input = Input(in_shp)
    xm = Reshape(in_shp + [1], input_shape=in_shp)(xm_input)
    # Part-A: Multi-channel Inputs and Spatial Characteristics Mapping Section
    # x1=Conv2D(50,(2,8),padding='same',activation="relu",name="Conv1",kernel_initializer="glorot_uniform")(input1)
    xm0 = Conv2D(20, (2, 8), padding='same', activation="relu", kernel_initializer='glorot_normal',
                 data_format='channels_last')(xm)
    # xm0 = MaxPooling2D(pool_size=(1,2), strides=(1,2), padding='valid', data_format='channels_last')(xm0)
    xm1 = Conv2D(20, (8, 2), padding='same', activation="relu", kernel_initializer='glorot_normal',
                 data_format='channels_last')(xm)
    # xm1 = MaxPooling2D(pool_size=(1,2), strides=(1,2), padding='valid', data_format='channels_last')(xm1)
    xm2 = Conv2D(10, (1, 1), padding='same', activation="relu", kernel_initializer='glorot_normal',
                 data_format='channels_last')(xm)
    # xm2 = MaxPooling2D(pool_size=(1,2), strides=(1,2), padding='valid', data_format='channels_last')(xm2)
    xmm = concatenate([xm0, xm1], axis=3)
    x1 = concatenate([xmm, xm2], axis=3)
    x1 = Activation('relu')(x1)
    x1 = Dropout(dr)(x1)

    xm0 = Conv2D(20, (1, 8), padding='same', activation="relu", kernel_initializer='glorot_normal',
                 data_format='channels_last')(xm)
    xm1 = Conv2D(20, (8, 1), padding='same', activation="relu", kernel_initializer='glorot_normal',
                 data_format='channels_last')(xm)
    xm2 = Conv2D(10, (1, 1), padding='same', activation="relu", kernel_initializer='glorot_normal',
                 data_format='channels_last')(xm)
    xmm2 = concatenate([xm0, xm1], axis=3)
    x = concatenate([xmm2, xm2], axis=3)
    x = Activation('relu')(x)
    x = Dropout(dr)(x)
    x = concatenate([x1, x], name="Concatenate2")

    x = _group_conv2d(x, filters=50, kernel=(3, 3), stride=(1, 1), groups=2)
    x = Activation('relu')(x)
    # Part-B: TRemporal Characteristics Extraction Section

    x = Reshape(target_shape=((256, 50)))(x)
    x = CuDNNGRU(units=128)(x)
    x = Dropout(dr)(x)

    # DNN
    x = Dense(11, activation="softmax", name="Softmax")(x)

    model = Model(inputs=xm_input, outputs=x)
    model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])
    return model

channels_last
__________________________________________________________________________________________________
Layer (type)                    Output Shape         Param #     Connected to                     
==================================================================================================
input_1 (InputLayer)            (None, 2, 128)       0                                            
__________________________________________________________________________________________________
reshape_1 (Reshape)             (None, 2, 128, 1)    0           input_1[0][0]                    
__________________________________________________________________________________________________
conv2d_1 (Conv2D)               (None, 2, 128, 20)   340         reshape_1[0][0]                  
__________________________________________________________________________________________________
conv2d_2 (Conv2D)               (None, 2, 128, 20)   340         reshape_1[0][0]                  
__________________________________________________________________________________________________
conv2d_4 (Conv2D)               (None, 2, 128, 20)   180         reshape_1[0][0]                  
__________________________________________________________________________________________________
conv2d_5 (Conv2D)               (None, 2, 128, 20)   180         reshape_1[0][0]                  
__________________________________________________________________________________________________
concatenate_1 (Concatenate)     (None, 2, 128, 40)   0           conv2d_1[0][0]                   
                                                                 conv2d_2[0][0]                   
__________________________________________________________________________________________________
conv2d_3 (Conv2D)               (None, 2, 128, 10)   20          reshape_1[0][0]                  
__________________________________________________________________________________________________
concatenate_3 (Concatenate)     (None, 2, 128, 40)   0           conv2d_4[0][0]                   
                                                                 conv2d_5[0][0]                   
__________________________________________________________________________________________________
conv2d_6 (Conv2D)               (None, 2, 128, 10)   20          reshape_1[0][0]                  
__________________________________________________________________________________________________
concatenate_2 (Concatenate)     (None, 2, 128, 50)   0           concatenate_1[0][0]              
                                                                 conv2d_3[0][0]                   
__________________________________________________________________________________________________
concatenate_4 (Concatenate)     (None, 2, 128, 50)   0           concatenate_3[0][0]              
                                                                 conv2d_6[0][0]                   
__________________________________________________________________________________________________
activation_1 (Activation)       (None, 2, 128, 50)   0           concatenate_2[0][0]              
__________________________________________________________________________________________________
activation_2 (Activation)       (None, 2, 128, 50)   0           concatenate_4[0][0]              
__________________________________________________________________________________________________
dropout_1 (Dropout)             (None, 2, 128, 50)   0           activation_1[0][0]               
__________________________________________________________________________________________________
dropout_2 (Dropout)             (None, 2, 128, 50)   0           activation_2[0][0]               
__________________________________________________________________________________________________
Concatenate2 (Concatenate)      (None, 2, 128, 100)  0           dropout_1[0][0]                  
                                                                 dropout_2[0][0]                  
__________________________________________________________________________________________________
lambda_1 (Lambda)               (None, 2, 128, 50)   0           Concatenate2[0][0]               
__________________________________________________________________________________________________
lambda_2 (Lambda)               (None, 2, 128, 50)   0           Concatenate2[0][0]               
__________________________________________________________________________________________________
conv2d_7 (Conv2D)               (None, 2, 128, 25)   11250       lambda_1[0][0]                   
__________________________________________________________________________________________________
conv2d_8 (Conv2D)               (None, 2, 128, 25)   11250       lambda_2[0][0]                   
__________________________________________________________________________________________________
concatenate_5 (Concatenate)     (None, 2, 128, 50)   0           conv2d_7[0][0]                   
                                                                 conv2d_8[0][0]                   
__________________________________________________________________________________________________
activation_3 (Activation)       (None, 2, 128, 50)   0           concatenate_5[0][0]              
__________________________________________________________________________________________________
reshape_2 (Reshape)             (None, 256, 50)      0           activation_3[0][0]               
__________________________________________________________________________________________________
cu_dnngru_1 (CuDNNGRU)          (None, 128)          69120       reshape_2[0][0]                  
__________________________________________________________________________________________________
dropout_3 (Dropout)             (None, 128)          0           cu_dnngru_1[0][0]                
__________________________________________________________________________________________________
Softmax (Dense)                 (None, 11)           1419        dropout_3[0][0]                  
==================================================================================================
Total params: 94,119
Trainable params: 94,119
Non-trainable params: 0

8.2移除PET

channels_last
__________________________________________________________________________________________________
Layer (type)                    Output Shape         Param #     Connected to                     
==================================================================================================
Ichannel (InputLayer)           (None, 128, 1)       0                                            
__________________________________________________________________________________________________
Qchannel (InputLayer)           (None, 128, 1)       0                                            
__________________________________________________________________________________________________
conv1d_1 (Conv1D)               (None, 128, 20)      60          Ichannel[0][0]                   
__________________________________________________________________________________________________
conv1d_2 (Conv1D)               (None, 128, 20)      100         Ichannel[0][0]                   
__________________________________________________________________________________________________
conv1d_4 (Conv1D)               (None, 128, 20)      60          Qchannel[0][0]                   
__________________________________________________________________________________________________
conv1d_5 (Conv1D)               (None, 128, 20)      100         Qchannel[0][0]                   
__________________________________________________________________________________________________
concatenate_3 (Concatenate)     (None, 128, 40)      0           conv1d_1[0][0]                   
                                                                 conv1d_2[0][0]                   
__________________________________________________________________________________________________
conv1d_3 (Conv1D)               (None, 128, 10)      90          Ichannel[0][0]                   
__________________________________________________________________________________________________
concatenate_5 (Concatenate)     (None, 128, 40)      0           conv1d_4[0][0]                   
                                                                 conv1d_5[0][0]                   
__________________________________________________________________________________________________
conv1d_6 (Conv1D)               (None, 128, 10)      90          Qchannel[0][0]                   
__________________________________________________________________________________________________
concatenate_4 (Concatenate)     (None, 128, 50)      0           concatenate_3[0][0]              
                                                                 conv1d_3[0][0]                   
__________________________________________________________________________________________________
concatenate_6 (Concatenate)     (None, 128, 50)      0           concatenate_5[0][0]              
                                                                 conv1d_6[0][0]                   
__________________________________________________________________________________________________
activation_2 (Activation)       (None, 128, 50)      0           concatenate_4[0][0]              
__________________________________________________________________________________________________
activation_3 (Activation)       (None, 128, 50)      0           concatenate_6[0][0]              
__________________________________________________________________________________________________
dropout_2 (Dropout)             (None, 128, 50)      0           activation_2[0][0]               
__________________________________________________________________________________________________
dropout_3 (Dropout)             (None, 128, 50)      0           activation_3[0][0]               
__________________________________________________________________________________________________
reshape_1 (Reshape)             (None, 1, 128, 50)   0           dropout_2[0][0]                  
__________________________________________________________________________________________________
reshap2 (Reshape)               (None, 1, 128, 50)   0           dropout_3[0][0]                  
__________________________________________________________________________________________________
I/Qchannel (InputLayer)         (None, 2, 128, 1)    0                                            
__________________________________________________________________________________________________
Concatenate1 (Concatenate)      (None, 2, 128, 50)   0           reshape_1[0][0]                  
                                                                 reshap2[0][0]                    
__________________________________________________________________________________________________
conv2d_1 (Conv2D)               (None, 2, 128, 20)   340         I/Qchannel[0][0]                 
__________________________________________________________________________________________________
conv2d_2 (Conv2D)               (None, 2, 128, 20)   340         I/Qchannel[0][0]                 
__________________________________________________________________________________________________
conv2d_4 (Conv2D)               (None, 2, 128, 20)   8020        Concatenate1[0][0]               
__________________________________________________________________________________________________
conv2d_5 (Conv2D)               (None, 2, 128, 20)   8020        Concatenate1[0][0]               
__________________________________________________________________________________________________
concatenate_1 (Concatenate)     (None, 2, 128, 40)   0           conv2d_1[0][0]                   
                                                                 conv2d_2[0][0]                   
__________________________________________________________________________________________________
conv2d_3 (Conv2D)               (None, 2, 128, 10)   20          I/Qchannel[0][0]                 
__________________________________________________________________________________________________
concatenate_7 (Concatenate)     (None, 2, 128, 40)   0           conv2d_4[0][0]                   
                                                                 conv2d_5[0][0]                   
__________________________________________________________________________________________________
conv2d_6 (Conv2D)               (None, 2, 128, 10)   510         Concatenate1[0][0]               
__________________________________________________________________________________________________
concatenate_2 (Concatenate)     (None, 2, 128, 50)   0           concatenate_1[0][0]              
                                                                 conv2d_3[0][0]                   
__________________________________________________________________________________________________
concatenate_8 (Concatenate)     (None, 2, 128, 50)   0           concatenate_7[0][0]              
                                                                 conv2d_6[0][0]                   
__________________________________________________________________________________________________
activation_1 (Activation)       (None, 2, 128, 50)   0           concatenate_2[0][0]              
__________________________________________________________________________________________________
activation_4 (Activation)       (None, 2, 128, 50)   0           concatenate_8[0][0]              
__________________________________________________________________________________________________
dropout_1 (Dropout)             (None, 2, 128, 50)   0           activation_1[0][0]               
__________________________________________________________________________________________________
dropout_4 (Dropout)             (None, 2, 128, 50)   0           activation_4[0][0]               
__________________________________________________________________________________________________
Concatenate2 (Concatenate)      (None, 2, 128, 100)  0           dropout_1[0][0]                  
                                                                 dropout_4[0][0]                  
__________________________________________________________________________________________________
lambda_1 (Lambda)               (None, 2, 128, 50)   0           Concatenate2[0][0]               
__________________________________________________________________________________________________
lambda_2 (Lambda)               (None, 2, 128, 50)   0           Concatenate2[0][0]               
__________________________________________________________________________________________________
conv2d_7 (Conv2D)               (None, 2, 128, 25)   11250       lambda_1[0][0]                   
__________________________________________________________________________________________________
conv2d_8 (Conv2D)               (None, 2, 128, 25)   11250       lambda_2[0][0]                   
__________________________________________________________________________________________________
concatenate_9 (Concatenate)     (None, 2, 128, 50)   0           conv2d_7[0][0]                   
                                                                 conv2d_8[0][0]                   
__________________________________________________________________________________________________
activation_5 (Activation)       (None, 2, 128, 50)   0           concatenate_9[0][0]              
__________________________________________________________________________________________________
reshape_2 (Reshape)             (None, 256, 50)      0           activation_5[0][0]               
__________________________________________________________________________________________________
cu_dnngru_1 (CuDNNGRU)          (None, 128)          69120       reshape_2[0][0]                  
__________________________________________________________________________________________________
Softmax (Dense)                 (None, 11)           1419        cu_dnngru_1[0][0]                
==================================================================================================
Total params: 110,789
Trainable params: 110,789
Non-trainable params: 0

8.3替换分组卷积

channels_last
__________________________________________________________________________________________________
Layer (type)                    Output Shape         Param #     Connected to                     
==================================================================================================
Ichannel (InputLayer)           (None, 128, 1)       0                                            
__________________________________________________________________________________________________
Qchannel (InputLayer)           (None, 128, 1)       0                                            
__________________________________________________________________________________________________
conv1d_1 (Conv1D)               (None, 128, 20)      60          Ichannel[0][0]                   
__________________________________________________________________________________________________
conv1d_2 (Conv1D)               (None, 128, 20)      100         Ichannel[0][0]                   
__________________________________________________________________________________________________
conv1d_4 (Conv1D)               (None, 128, 20)      60          Qchannel[0][0]                   
__________________________________________________________________________________________________
conv1d_5 (Conv1D)               (None, 128, 20)      100         Qchannel[0][0]                   
__________________________________________________________________________________________________
concatenate_4 (Concatenate)     (None, 128, 40)      0           conv1d_1[0][0]                   
                                                                 conv1d_2[0][0]                   
__________________________________________________________________________________________________
conv1d_3 (Conv1D)               (None, 128, 10)      90          Ichannel[0][0]                   
__________________________________________________________________________________________________
concatenate_6 (Concatenate)     (None, 128, 40)      0           conv1d_4[0][0]                   
                                                                 conv1d_5[0][0]                   
__________________________________________________________________________________________________
conv1d_6 (Conv1D)               (None, 128, 10)      90          Qchannel[0][0]                   
__________________________________________________________________________________________________
concatenate_5 (Concatenate)     (None, 128, 50)      0           concatenate_4[0][0]              
                                                                 conv1d_3[0][0]                   
__________________________________________________________________________________________________
concatenate_7 (Concatenate)     (None, 128, 50)      0           concatenate_6[0][0]              
                                                                 conv1d_6[0][0]                   
__________________________________________________________________________________________________
I/Qchannel (InputLayer)         (None, 2, 128, 1)    0                                            
__________________________________________________________________________________________________
activation_3 (Activation)       (None, 128, 50)      0           concatenate_5[0][0]              
__________________________________________________________________________________________________
activation_4 (Activation)       (None, 128, 50)      0           concatenate_7[0][0]              
__________________________________________________________________________________________________
flatten_1 (Flatten)             (None, 256)          0           I/Qchannel[0][0]                 
__________________________________________________________________________________________________
dropout_2 (Dropout)             (None, 128, 50)      0           activation_3[0][0]               
__________________________________________________________________________________________________
dropout_3 (Dropout)             (None, 128, 50)      0           activation_4[0][0]               
__________________________________________________________________________________________________
fc2 (Dense)                     (None, 1)            257         flatten_1[0][0]                  
__________________________________________________________________________________________________
reshape_1 (Reshape)             (None, 1, 128, 50)   0           dropout_2[0][0]                  
__________________________________________________________________________________________________
reshap2 (Reshape)               (None, 1, 128, 50)   0           dropout_3[0][0]                  
__________________________________________________________________________________________________
activation_1 (Activation)       (None, 1)            0           fc2[0][0]                        
__________________________________________________________________________________________________
Concatenate1 (Concatenate)      (None, 2, 128, 50)   0           reshape_1[0][0]                  
                                                                 reshap2[0][0]                    
__________________________________________________________________________________________________
lambda_1 (Lambda)               (None, 1)            0           activation_1[0][0]               
__________________________________________________________________________________________________
lambda_2 (Lambda)               (None, 1)            0           activation_1[0][0]               
__________________________________________________________________________________________________
conv2d_1 (Conv2D)               (None, 2, 128, 20)   340         I/Qchannel[0][0]                 
__________________________________________________________________________________________________
conv2d_2 (Conv2D)               (None, 2, 128, 20)   340         I/Qchannel[0][0]                 
__________________________________________________________________________________________________
conv2d_4 (Conv2D)               (None, 2, 128, 20)   8020        Concatenate1[0][0]               
__________________________________________________________________________________________________
conv2d_5 (Conv2D)               (None, 2, 128, 20)   8020        Concatenate1[0][0]               
__________________________________________________________________________________________________
multiply_1 (Multiply)           (None, 128, 1)       0           Ichannel[0][0]                   
                                                                 lambda_1[0][0]                   
__________________________________________________________________________________________________
multiply_2 (Multiply)           (None, 128, 1)       0           Qchannel[0][0]                   
                                                                 lambda_2[0][0]                   
__________________________________________________________________________________________________
multiply_3 (Multiply)           (None, 128, 1)       0           Qchannel[0][0]                   
                                                                 lambda_1[0][0]                   
__________________________________________________________________________________________________
multiply_4 (Multiply)           (None, 128, 1)       0           Ichannel[0][0]                   
                                                                 lambda_2[0][0]                   
__________________________________________________________________________________________________
concatenate_2 (Concatenate)     (None, 2, 128, 40)   0           conv2d_1[0][0]                   
                                                                 conv2d_2[0][0]                   
__________________________________________________________________________________________________
conv2d_3 (Conv2D)               (None, 2, 128, 10)   20          I/Qchannel[0][0]                 
__________________________________________________________________________________________________
concatenate_8 (Concatenate)     (None, 2, 128, 40)   0           conv2d_4[0][0]                   
                                                                 conv2d_5[0][0]                   
__________________________________________________________________________________________________
conv2d_6 (Conv2D)               (None, 2, 128, 10)   510         Concatenate1[0][0]               
__________________________________________________________________________________________________
add_1 (Add)                     (None, 128, 1)       0           multiply_1[0][0]                 
                                                                 multiply_2[0][0]                 
__________________________________________________________________________________________________
subtract_1 (Subtract)           (None, 128, 1)       0           multiply_3[0][0]                 
                                                                 multiply_4[0][0]                 
__________________________________________________________________________________________________
concatenate_3 (Concatenate)     (None, 2, 128, 50)   0           concatenate_2[0][0]              
                                                                 conv2d_3[0][0]                   
__________________________________________________________________________________________________
concatenate_9 (Concatenate)     (None, 2, 128, 50)   0           concatenate_8[0][0]              
                                                                 conv2d_6[0][0]                   
__________________________________________________________________________________________________
reshape1 (Reshape)              (None, 128, 1)       0           add_1[0][0]                      
__________________________________________________________________________________________________
reshape2 (Reshape)              (None, 128, 1)       0           subtract_1[0][0]                 
__________________________________________________________________________________________________
activation_2 (Activation)       (None, 2, 128, 50)   0           concatenate_3[0][0]              
__________________________________________________________________________________________________
activation_5 (Activation)       (None, 2, 128, 50)   0           concatenate_9[0][0]              
__________________________________________________________________________________________________
concatenate_1 (Concatenate)     (None, 128, 2)       0           reshape1[0][0]                   
                                                                 reshape2[0][0]                   
__________________________________________________________________________________________________
dropout_1 (Dropout)             (None, 2, 128, 50)   0           activation_2[0][0]               
__________________________________________________________________________________________________
dropout_4 (Dropout)             (None, 2, 128, 50)   0           activation_5[0][0]               
__________________________________________________________________________________________________
reshape3 (Reshape)              (None, 128, 2, 1)    0           concatenate_1[0][0]              
__________________________________________________________________________________________________
Concatenate2 (Concatenate)      (None, 2, 128, 100)  0           dropout_1[0][0]                  
                                                                 dropout_4[0][0]                  
__________________________________________________________________________________________________
reshape_2 (Reshape)             (None, 2, 128, 1)    0           reshape3[0][0]                   
__________________________________________________________________________________________________
conv2d_7 (Conv2D)               (None, 2, 128, 50)   45050       Concatenate2[0][0]               
__________________________________________________________________________________________________
add_2 (Add)                     (None, 2, 128, 50)   0           reshape_2[0][0]                  
                                                                 conv2d_7[0][0]                   
__________________________________________________________________________________________________
reshape_3 (Reshape)             (None, 256, 50)      0           add_2[0][0]                      
__________________________________________________________________________________________________
cu_dnngru_1 (CuDNNGRU)          (None, 128)          69120       reshape_3[0][0]                  
__________________________________________________________________________________________________
Softmax (Dense)                 (None, 11)           1419        cu_dnngru_1[0][0]                
==================================================================================================
Total params: 133,596
Trainable params: 133,596
Non-trainable params: 0

8.4替换LSTM

channels_last
__________________________________________________________________________________________________
Layer (type)                    Output Shape         Param #     Connected to                     
==================================================================================================
Ichannel (InputLayer)           (None, 128, 1)       0                                            
__________________________________________________________________________________________________
Qchannel (InputLayer)           (None, 128, 1)       0                                            
__________________________________________________________________________________________________
conv1d_1 (Conv1D)               (None, 128, 20)      60          Ichannel[0][0]                   
__________________________________________________________________________________________________
conv1d_2 (Conv1D)               (None, 128, 20)      100         Ichannel[0][0]                   
__________________________________________________________________________________________________
conv1d_4 (Conv1D)               (None, 128, 20)      60          Qchannel[0][0]                   
__________________________________________________________________________________________________
conv1d_5 (Conv1D)               (None, 128, 20)      100         Qchannel[0][0]                   
__________________________________________________________________________________________________
concatenate_4 (Concatenate)     (None, 128, 40)      0           conv1d_1[0][0]                   
                                                                 conv1d_2[0][0]                   
__________________________________________________________________________________________________
conv1d_3 (Conv1D)               (None, 128, 10)      90          Ichannel[0][0]                   
__________________________________________________________________________________________________
concatenate_6 (Concatenate)     (None, 128, 40)      0           conv1d_4[0][0]                   
                                                                 conv1d_5[0][0]                   
__________________________________________________________________________________________________
conv1d_6 (Conv1D)               (None, 128, 10)      90          Qchannel[0][0]                   
__________________________________________________________________________________________________
concatenate_5 (Concatenate)     (None, 128, 50)      0           concatenate_4[0][0]              
                                                                 conv1d_3[0][0]                   
__________________________________________________________________________________________________
concatenate_7 (Concatenate)     (None, 128, 50)      0           concatenate_6[0][0]              
                                                                 conv1d_6[0][0]                   
__________________________________________________________________________________________________
activation_3 (Activation)       (None, 128, 50)      0           concatenate_5[0][0]              
__________________________________________________________________________________________________
activation_4 (Activation)       (None, 128, 50)      0           concatenate_7[0][0]              
__________________________________________________________________________________________________
dropout_2 (Dropout)             (None, 128, 50)      0           activation_3[0][0]               
__________________________________________________________________________________________________
dropout_3 (Dropout)             (None, 128, 50)      0           activation_4[0][0]               
__________________________________________________________________________________________________
reshape_1 (Reshape)             (None, 1, 128, 50)   0           dropout_2[0][0]                  
__________________________________________________________________________________________________
reshap2 (Reshape)               (None, 1, 128, 50)   0           dropout_3[0][0]                  
__________________________________________________________________________________________________
I/Qchannel (InputLayer)         (None, 2, 128, 1)    0                                            
__________________________________________________________________________________________________
Concatenate1 (Concatenate)      (None, 2, 128, 50)   0           reshape_1[0][0]                  
                                                                 reshap2[0][0]                    
__________________________________________________________________________________________________
flatten_1 (Flatten)             (None, 256)          0           I/Qchannel[0][0]                 
__________________________________________________________________________________________________
conv2d_1 (Conv2D)               (None, 2, 128, 20)   340         I/Qchannel[0][0]                 
__________________________________________________________________________________________________
conv2d_2 (Conv2D)               (None, 2, 128, 20)   340         I/Qchannel[0][0]                 
__________________________________________________________________________________________________
conv2d_4 (Conv2D)               (None, 2, 128, 20)   8020        Concatenate1[0][0]               
__________________________________________________________________________________________________
conv2d_5 (Conv2D)               (None, 2, 128, 20)   8020        Concatenate1[0][0]               
__________________________________________________________________________________________________
fc2 (Dense)                     (None, 1)            257         flatten_1[0][0]                  
__________________________________________________________________________________________________
concatenate_2 (Concatenate)     (None, 2, 128, 40)   0           conv2d_1[0][0]                   
                                                                 conv2d_2[0][0]                   
__________________________________________________________________________________________________
conv2d_3 (Conv2D)               (None, 2, 128, 10)   20          I/Qchannel[0][0]                 
__________________________________________________________________________________________________
concatenate_8 (Concatenate)     (None, 2, 128, 40)   0           conv2d_4[0][0]                   
                                                                 conv2d_5[0][0]                   
__________________________________________________________________________________________________
conv2d_6 (Conv2D)               (None, 2, 128, 10)   510         Concatenate1[0][0]               
__________________________________________________________________________________________________
activation_1 (Activation)       (None, 1)            0           fc2[0][0]                        
__________________________________________________________________________________________________
concatenate_3 (Concatenate)     (None, 2, 128, 50)   0           concatenate_2[0][0]              
                                                                 conv2d_3[0][0]                   
__________________________________________________________________________________________________
concatenate_9 (Concatenate)     (None, 2, 128, 50)   0           concatenate_8[0][0]              
                                                                 conv2d_6[0][0]                   
__________________________________________________________________________________________________
lambda_1 (Lambda)               (None, 1)            0           activation_1[0][0]               
__________________________________________________________________________________________________
lambda_2 (Lambda)               (None, 1)            0           activation_1[0][0]               
__________________________________________________________________________________________________
activation_2 (Activation)       (None, 2, 128, 50)   0           concatenate_3[0][0]              
__________________________________________________________________________________________________
activation_5 (Activation)       (None, 2, 128, 50)   0           concatenate_9[0][0]              
__________________________________________________________________________________________________
multiply_1 (Multiply)           (None, 128, 1)       0           Ichannel[0][0]                   
                                                                 lambda_1[0][0]                   
__________________________________________________________________________________________________
multiply_2 (Multiply)           (None, 128, 1)       0           Qchannel[0][0]                   
                                                                 lambda_2[0][0]                   
__________________________________________________________________________________________________
multiply_3 (Multiply)           (None, 128, 1)       0           Qchannel[0][0]                   
                                                                 lambda_1[0][0]                   
__________________________________________________________________________________________________
multiply_4 (Multiply)           (None, 128, 1)       0           Ichannel[0][0]                   
                                                                 lambda_2[0][0]                   
__________________________________________________________________________________________________
dropout_1 (Dropout)             (None, 2, 128, 50)   0           activation_2[0][0]               
__________________________________________________________________________________________________
dropout_4 (Dropout)             (None, 2, 128, 50)   0           activation_5[0][0]               
__________________________________________________________________________________________________
add_1 (Add)                     (None, 128, 1)       0           multiply_1[0][0]                 
                                                                 multiply_2[0][0]                 
__________________________________________________________________________________________________
subtract_1 (Subtract)           (None, 128, 1)       0           multiply_3[0][0]                 
                                                                 multiply_4[0][0]                 
__________________________________________________________________________________________________
Concatenate2 (Concatenate)      (None, 2, 128, 100)  0           dropout_1[0][0]                  
                                                                 dropout_4[0][0]                  
__________________________________________________________________________________________________
reshape1 (Reshape)              (None, 128, 1)       0           add_1[0][0]                      
__________________________________________________________________________________________________
reshape2 (Reshape)              (None, 128, 1)       0           subtract_1[0][0]                 
__________________________________________________________________________________________________
lambda_3 (Lambda)               (None, 2, 128, 50)   0           Concatenate2[0][0]               
__________________________________________________________________________________________________
lambda_4 (Lambda)               (None, 2, 128, 50)   0           Concatenate2[0][0]               
__________________________________________________________________________________________________
concatenate_1 (Concatenate)     (None, 128, 2)       0           reshape1[0][0]                   
                                                                 reshape2[0][0]                   
__________________________________________________________________________________________________
conv2d_7 (Conv2D)               (None, 2, 128, 25)   11250       lambda_3[0][0]                   
__________________________________________________________________________________________________
conv2d_8 (Conv2D)               (None, 2, 128, 25)   11250       lambda_4[0][0]                   
__________________________________________________________________________________________________
reshape3 (Reshape)              (None, 128, 2, 1)    0           concatenate_1[0][0]              
__________________________________________________________________________________________________
concatenate_10 (Concatenate)    (None, 2, 128, 50)   0           conv2d_7[0][0]                   
                                                                 conv2d_8[0][0]                   
__________________________________________________________________________________________________
reshape_2 (Reshape)             (None, 2, 128, 1)    0           reshape3[0][0]                   
__________________________________________________________________________________________________
activation_6 (Activation)       (None, 2, 128, 50)   0           concatenate_10[0][0]             
__________________________________________________________________________________________________
add_2 (Add)                     (None, 2, 128, 50)   0           reshape_2[0][0]                  
                                                                 activation_6[0][0]               
__________________________________________________________________________________________________
reshape_3 (Reshape)             (None, 256, 50)      0           add_2[0][0]                      
__________________________________________________________________________________________________
LSTM1 (CuDNNLSTM)               (None, 128)          92160       reshape_3[0][0]                  
__________________________________________________________________________________________________
dropout_5 (Dropout)             (None, 128)          0           LSTM1[0][0]                      
__________________________________________________________________________________________________
Softmax (Dense)                 (None, 11)           1419        dropout_5[0][0]                  
==================================================================================================
Total params: 134,086
Trainable params: 134,086
Non-trainable params: 0

8.4替换非对称卷积

 # Part-A: Multi-channel Inputs and Spatial Characteristics Mapping Section
    # x1=Conv2D(50,(2,8),padding='same',activation="relu",name="Conv1",kernel_initializer="glorot_uniform")(input1)
    xm0 = Conv2D(20, (8, 8), padding='same',activation="relu", kernel_initializer='glorot_normal', data_format='channels_last')(input1)
    # xm0 = MaxPooling2D(pool_size=(1,2), strides=(1,2), padding='valid', data_format='channels_last')(xm0)
    xm1 = Conv2D(20, (8, 8), padding='same',activation="relu", kernel_initializer='glorot_normal', data_format='channels_last')(input1)
    # xm1 = MaxPooling2D(pool_size=(1,2), strides=(1,2), padding='valid', data_format='channels_last')(xm1)
    xm2 = Conv2D(10, (1, 1), padding='same',activation="relu", kernel_initializer='glorot_normal', data_format='channels_last')(input1)
    # xm2 = MaxPooling2D(pool_size=(1,2), strides=(1,2), padding='valid', data_format='channels_last')(xm2)
    xm = concatenate([xm0, xm1], axis=3)
    x1 = concatenate([xm, xm2], axis=3)
    x1 = Activation('relu')(x1)
    x1 = Dropout(dr)(x1)

    # x2=Conv1D(50,8,padding='causal',activation="relu",name="Conv2",kernel_initializer="glorot_uniform")(input2)
    xc0 = Conv1D(20, 2, padding='causal',activation="relu", kernel_initializer='glorot_normal', data_format='channels_last')(input2)
    xc1 = Conv1D(20, 4, padding='causal',activation="relu", kernel_initializer='glorot_normal', data_format='channels_last')(input2)
    xc2 = Conv1D(10, 8, padding='causal',activation="relu", kernel_initializer='glorot_normal', data_format='channels_last')(input2)
    xc = concatenate([xc0, xc1], axis=2)
    x2 = concatenate([xc, xc2], axis=2)
    x2 = Activation('relu')(x2)
    x2 = Dropout(dr)(x2)
    x2_reshape = Reshape([-1, 128, 50])(x2)

channels_last
__________________________________________________________________________________________________
Layer (type)                    Output Shape         Param #     Connected to                     
==================================================================================================
Ichannel (InputLayer)           (None, 128, 1)       0                                            
__________________________________________________________________________________________________
Qchannel (InputLayer)           (None, 128, 1)       0                                            
__________________________________________________________________________________________________
conv1d_1 (Conv1D)               (None, 128, 20)      60          Ichannel[0][0]                   
__________________________________________________________________________________________________
conv1d_2 (Conv1D)               (None, 128, 20)      100         Ichannel[0][0]                   
__________________________________________________________________________________________________
conv1d_4 (Conv1D)               (None, 128, 20)      60          Qchannel[0][0]                   
__________________________________________________________________________________________________
conv1d_5 (Conv1D)               (None, 128, 20)      100         Qchannel[0][0]                   
__________________________________________________________________________________________________
concatenate_4 (Concatenate)     (None, 128, 40)      0           conv1d_1[0][0]                   
                                                                 conv1d_2[0][0]                   
__________________________________________________________________________________________________
conv1d_3 (Conv1D)               (None, 128, 10)      90          Ichannel[0][0]                   
__________________________________________________________________________________________________
concatenate_6 (Concatenate)     (None, 128, 40)      0           conv1d_4[0][0]                   
                                                                 conv1d_5[0][0]                   
__________________________________________________________________________________________________
conv1d_6 (Conv1D)               (None, 128, 10)      90          Qchannel[0][0]                   
__________________________________________________________________________________________________
concatenate_5 (Concatenate)     (None, 128, 50)      0           concatenate_4[0][0]              
                                                                 conv1d_3[0][0]                   
__________________________________________________________________________________________________
concatenate_7 (Concatenate)     (None, 128, 50)      0           concatenate_6[0][0]              
                                                                 conv1d_6[0][0]                   
__________________________________________________________________________________________________
activation_3 (Activation)       (None, 128, 50)      0           concatenate_5[0][0]              
__________________________________________________________________________________________________
activation_4 (Activation)       (None, 128, 50)      0           concatenate_7[0][0]              
__________________________________________________________________________________________________
dropout_2 (Dropout)             (None, 128, 50)      0           activation_3[0][0]               
__________________________________________________________________________________________________
dropout_3 (Dropout)             (None, 128, 50)      0           activation_4[0][0]               
__________________________________________________________________________________________________
reshape_1 (Reshape)             (None, 1, 128, 50)   0           dropout_2[0][0]                  
__________________________________________________________________________________________________
reshap2 (Reshape)               (None, 1, 128, 50)   0           dropout_3[0][0]                  
__________________________________________________________________________________________________
I/Qchannel (InputLayer)         (None, 2, 128, 1)    0                                            
__________________________________________________________________________________________________
Concatenate1 (Concatenate)      (None, 2, 128, 50)   0           reshape_1[0][0]                  
                                                                 reshap2[0][0]                    
__________________________________________________________________________________________________
flatten_1 (Flatten)             (None, 256)          0           I/Qchannel[0][0]                 
__________________________________________________________________________________________________
conv2d_1 (Conv2D)               (None, 2, 128, 20)   1300        I/Qchannel[0][0]                 
__________________________________________________________________________________________________
conv2d_2 (Conv2D)               (None, 2, 128, 20)   1300        I/Qchannel[0][0]                 
__________________________________________________________________________________________________
conv2d_4 (Conv2D)               (None, 2, 128, 20)   8020        Concatenate1[0][0]               
__________________________________________________________________________________________________
conv2d_5 (Conv2D)               (None, 2, 128, 20)   8020        Concatenate1[0][0]               
__________________________________________________________________________________________________
fc2 (Dense)                     (None, 1)            257         flatten_1[0][0]                  
__________________________________________________________________________________________________
concatenate_2 (Concatenate)     (None, 2, 128, 40)   0           conv2d_1[0][0]                   
                                                                 conv2d_2[0][0]                   
__________________________________________________________________________________________________
conv2d_3 (Conv2D)               (None, 2, 128, 10)   20          I/Qchannel[0][0]                 
__________________________________________________________________________________________________
concatenate_8 (Concatenate)     (None, 2, 128, 40)   0           conv2d_4[0][0]                   
                                                                 conv2d_5[0][0]                   
__________________________________________________________________________________________________
conv2d_6 (Conv2D)               (None, 2, 128, 10)   510         Concatenate1[0][0]               
__________________________________________________________________________________________________
activation_1 (Activation)       (None, 1)            0           fc2[0][0]                        
__________________________________________________________________________________________________
concatenate_3 (Concatenate)     (None, 2, 128, 50)   0           concatenate_2[0][0]              
                                                                 conv2d_3[0][0]                   
__________________________________________________________________________________________________
concatenate_9 (Concatenate)     (None, 2, 128, 50)   0           concatenate_8[0][0]              
                                                                 conv2d_6[0][0]                   
__________________________________________________________________________________________________
lambda_1 (Lambda)               (None, 1)            0           activation_1[0][0]               
__________________________________________________________________________________________________
lambda_2 (Lambda)               (None, 1)            0           activation_1[0][0]               
__________________________________________________________________________________________________
activation_2 (Activation)       (None, 2, 128, 50)   0           concatenate_3[0][0]              
__________________________________________________________________________________________________
activation_5 (Activation)       (None, 2, 128, 50)   0           concatenate_9[0][0]              
__________________________________________________________________________________________________
multiply_1 (Multiply)           (None, 128, 1)       0           Ichannel[0][0]                   
                                                                 lambda_1[0][0]                   
__________________________________________________________________________________________________
multiply_2 (Multiply)           (None, 128, 1)       0           Qchannel[0][0]                   
                                                                 lambda_2[0][0]                   
__________________________________________________________________________________________________
multiply_3 (Multiply)           (None, 128, 1)       0           Qchannel[0][0]                   
                                                                 lambda_1[0][0]                   
__________________________________________________________________________________________________
multiply_4 (Multiply)           (None, 128, 1)       0           Ichannel[0][0]                   
                                                                 lambda_2[0][0]                   
__________________________________________________________________________________________________
dropout_1 (Dropout)             (None, 2, 128, 50)   0           activation_2[0][0]               
__________________________________________________________________________________________________
dropout_4 (Dropout)             (None, 2, 128, 50)   0           activation_5[0][0]               
__________________________________________________________________________________________________
add_1 (Add)                     (None, 128, 1)       0           multiply_1[0][0]                 
                                                                 multiply_2[0][0]                 
__________________________________________________________________________________________________
subtract_1 (Subtract)           (None, 128, 1)       0           multiply_3[0][0]                 
                                                                 multiply_4[0][0]                 
__________________________________________________________________________________________________
Concatenate2 (Concatenate)      (None, 2, 128, 100)  0           dropout_1[0][0]                  
                                                                 dropout_4[0][0]                  
__________________________________________________________________________________________________
reshape1 (Reshape)              (None, 128, 1)       0           add_1[0][0]                      
__________________________________________________________________________________________________
reshape2 (Reshape)              (None, 128, 1)       0           subtract_1[0][0]                 
__________________________________________________________________________________________________
lambda_3 (Lambda)               (None, 2, 128, 50)   0           Concatenate2[0][0]               
__________________________________________________________________________________________________
lambda_4 (Lambda)               (None, 2, 128, 50)   0           Concatenate2[0][0]               
__________________________________________________________________________________________________
concatenate_1 (Concatenate)     (None, 128, 2)       0           reshape1[0][0]                   
                                                                 reshape2[0][0]                   
__________________________________________________________________________________________________
conv2d_7 (Conv2D)               (None, 2, 128, 25)   11250       lambda_3[0][0]                   
__________________________________________________________________________________________________
conv2d_8 (Conv2D)               (None, 2, 128, 25)   11250       lambda_4[0][0]                   
__________________________________________________________________________________________________
reshape3 (Reshape)              (None, 128, 2, 1)    0           concatenate_1[0][0]              
__________________________________________________________________________________________________
concatenate_10 (Concatenate)    (None, 2, 128, 50)   0           conv2d_7[0][0]                   
                                                                 conv2d_8[0][0]                   
__________________________________________________________________________________________________
reshape_2 (Reshape)             (None, 2, 128, 1)    0           reshape3[0][0]                   
__________________________________________________________________________________________________
activation_6 (Activation)       (None, 2, 128, 50)   0           concatenate_10[0][0]             
__________________________________________________________________________________________________
add_2 (Add)                     (None, 2, 128, 50)   0           reshape_2[0][0]                  
                                                                 activation_6[0][0]               
__________________________________________________________________________________________________
reshape_3 (Reshape)             (None, 256, 50)      0           add_2[0][0]                      
__________________________________________________________________________________________________
cu_dnngru_1 (CuDNNGRU)          (None, 128)          69120       reshape_3[0][0]                  
__________________________________________________________________________________________________
dropout_5 (Dropout)             (None, 128)          0           cu_dnngru_1[0][0]                
__________________________________________________________________________________________________
Softmax (Dense)                 (None, 11)           1419        dropout_5[0][0]                  
==================================================================================================
Total params: 112,966
Trainable params: 112,966
Non-trainable params: 0
__________________________