stream_kws_cnn

最新推荐文章于 2023-06-02 00:35:27 发布

Lebhoryi

最新推荐文章于 2023-06-02 00:35:27 发布

阅读量1.4k

点赞数

分类专栏： kws 文章标签： stream kws tensorflow2

本文链接：https://blog.csdn.net/weixin_37598106/article/details/106801481

版权

kws 专栏收录该内容

5 篇文章 1 订阅

订阅专栏

原创: Lebhoryi@rt-thread.com
时间: 2020/06/17
code: google-research/kws_streaming/
paper: Streaming keyword spotting on mobile devices
关于stream kws 代码上的一些探索

0x00 parser 超参传入

base_parser.py —> base_parser()

parser = base_parser.base_parser()

python3 train/model_train_eval.py \
--data_dir ../data \
--lr_schedule linear \
--background_volume 0.1 \
--l2_weight_decay 0.0 \
--background_frequency 0.8 \
--split_data 1 \
--time_shift_ms 100.0 \
--how_many_training_steps 10000,5000 \
--learning_rate 0.0001,0.00005 \
--batch_size 100 \
--train_dir ./train_model/cnn/ \
--feature_type 'mfcc_tf' \
--start_checkpoint '' \
--resample 0.15 \
--train 1 \
--window_size_ms 40 \
--window_stride_ms 20 \
--dct_num_features 20 \
--preprocess mfcc \
--feature_type mfcc_tf \
--mel_lower_edge_hertz 20.0 \
--mel_upper_edge_hertz 7000 \
--fft_magnitude_squared 0 \
--mel_num_bins 40 \
cnn \
--cnn_filters 64,64,64,64,128,64,128 \
--cnn_kernel_size '(3,3),(5,3),(5,3),(5,3),(5,2),(5,1),(3,1)' \
--cnn_dilation_rate '(1,1),(1,1),(1,1),(1,1),(1,1),(1,1),(1,1)'

cnn 模型相关参数传入
cnn.py —> model_parameters()

parser_cnn = subparsers.add_parser('cnn')
cnn.model_parameters(parser_cnn)

继续更新参数

model_flags.py —> update_flags(FLAGS)

flags = model_flags.update_flags(FLAGS)

label_count = 14
desired_samples = 16000
window_size_samples = 640
window_stride_samples = 320
spectrogram_length = 49
fingerprint_width = 40
fft_magnitude_squared = False
summaries_dir = os.path.join(flags.train_dir, ‘logs/’)

0x01 数据预处理

audio_processor = input_data.AudioProcessor(flags)

不多赘述，大神解释的很清楚了: 使用Tensorflow识别语音关键词

只不过该项目在之前的基础上新增了raw和micro 的特征处理

打印flags日志：

logging.info(flags)

0x02 model 此处选择 CNN

空洞卷积核计算方式： $k_{new} = (k - 1) * d + 1$

2.1 输入( 此处 flags.preprocess = mfcc)

train:
- TRAINING
inference:
- STREAM_INTERNAL_STATE_INFERENCE
- STREAM_EXTERNAL_STATE_INFERENCE
- NON_STREAM_INFERENCE

# cnn.py
input_audio = tf.keras.layers.Input(
        shape=modes.get_input_data_shape(flags, modes.Modes.TRAINING),
        batch_size=flags.batch_size)

non-streaming and training

shape = (flags.spectrogram_length, flags.dct_num_features,)

shape = (49, 20)

streaming
```
 shape = (1, flags.dct_num_features,)
```
- shape = (1, 20)

输入分两种：

当flags.preprocess为 raw 时，net = input_audio
当flags.preprocess为 mfcc or micro 时,

# it is a self contained model, user need to feed raw audio only
net = speech_features.SpeechFeatures(
        speech_features.SpeechFeatures.get_params(flags))(
            net)

2.2 输入增维

为了增加chanel

# input shape = （100，49， 20， 1）
net = tf.keras.backend.expand_dims(net)

2.3 构建网络

for + zip 简直就是神来之笔

for filters, kernel_size, activation, dilation_rate, strides in zip(
      parse(flags.cnn_filters), parse(flags.cnn_kernel_size),
      parse(flags.cnn_act), parse(flags.cnn_dilation_rate),
      parse(flags.cnn_strides)):
    net = Stream(
        cell=tf.keras.layers.Conv2D(
            filters=filters,
            kernel_size=kernel_size,
            activation=activation,
            dilation_rate=dilation_rate,
            strides=strides))(
                net)

2.3.1 stream()函数

灵魂拷问：

为什么conv要加stream()
为什么flattern 要加stream()
为什么stream() 有效果
为什么使用 build()

回答：

为了产生self.state_shape 这一个变量。
在训练（卷积）的过程生成每一层的self.state_shape变量，在推理时候，如果是Modes.STREAM_EXTERNAL_STATE_INFERENCE会生成一个新的self.input_state的输入，如果是Modes.STREAM_INTERNAL_STATE_INFERENCE的时候会生成一个不可训练的权重文件

当定义网络时不知道网络的维度时可以重写build()函数，用获得的shape构建网络

__init__()

- super().\_\_init\_\_(**kwargs)

- self.cell = tf.keras.layers.Conv2D(filters=filters, kernel_size=kernel_size,
             activation=activation, dilation_rate=dilation_rate, strides=strides)

- self.mode = Modes.TRAINING

- self.pad_time_dim = False

- self.state_shape = None

- self.ring_buffer_size_in_time_dim = None

- strides = (1,1)

- dilation_rate = (1,1)

- kernel_size = (3,3)

- self.ring_buffer_size_in_time_dim = 3  # 时间维度上的核大小

 # conv
 self.ring_buffer_size_in_time_dim = dilation_rate[0] * (kernel_size[0] - 1) + 1
 # Flattern 
 self.ring_buffer_size_in_time_dim = None

build() ：

build()是在__init__()调用之后被调用

  self.state_shape = [
            self.inference_batch_size, self.ring_buffer_size_in_time_dim
        ] + input_shape.as_list()[2:]

self.state_shape = [1, 3, 20, 1]

  # Modes.STREAM_INTERNAL_STATE_INFERENCE
  # 增加一个变量， state 变为 weight
  self.states = self.add_weight(name='states', shape=self.state_shape, trainable=False,
      initializer=tf.zeros_initializer)

call() 前向传播

训练的时候，no padding，call()如下：

self.cell(inputs)

# 即 tf.keras.layers.Conv2D()(inputs)

推理的时候，假定mode = STREAM_INTERNAL_STATE_INFERENCE

输入的第一维必须是1，移除第一帧，补上最后一帧，作为网络的输入

  def _streaming_internal_state(self, inputs):
    # The time dimenstion always has to equal 1 in streaming mode.
    if inputs.shape[1] != 1:
      raise ValueError('inputs.shape[1]: %d must be 1 ' % inputs.shape[1])

    # remove latest row [batch_size, (memory_size-1), feature_dim, channel]
    memory = self.states[:, 1:self.ring_buffer_size_in_time_dim, :]

    # add new row [batch_size, memory_size, feature_dim, channel]
    memory = tf.keras.backend.concatenate([memory, inputs], 1)

    assign_states = self.states.assign(memory)

    with tf.control_dependencies([assign_states]):
      return self.cell(memory)

2.3.2 网络结构模型 CNN

logging.info(model.summary())

Model: "model"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   

=================================================================
input_1 (InputLayer)         [(100, 49, 20)]           0         
_________________________________________________________________
tf_op_layer_ExpandDims (Tens [(100, 49, 20, 1)]        0         
_________________________________________________________________
stream (Stream)                (100, 47, 18, 64)         640       
_________________________________________________________________
stream_1 (Stream)            (100, 43, 16, 64)         61504     
_________________________________________________________________
stream_2 (Stream)            (100, 35, 14, 64)         61504     
_________________________________________________________________
stream_3 (Stream)            (100, 31, 12, 64)         61504     
_________________________________________________________________
stream_4 (Stream)            (100, 23, 11, 128)        82048     
_________________________________________________________________
stream_5 (Stream)            (100, 19, 11, 64)         41024     
_________________________________________________________________
stream_6 (Stream)            (100, 1, 11, 128)         82048     
_________________________________________________________________
stream_7 (Stream)            (100, 1408)        	  	    0         
_________________________________________________________________
dropout (Dropout)            (100, 1408)     		          0         
_________________________________________________________________
dense (Dense)                     (100, 128)                   180352    
_________________________________________________________________
dense_1 (Dense)                (100, 256)                    33024     
_________________________________________________________________
dense_2 (Dense)                (100, 14)                       3598      

=================================================================
Total params: 607,246
Trainable params: 607,246
Non-trainable params: 0

2.3.3 保存网络结构模型和参数到 txt

这个函数的代码可以抄袭，很有借鉴意义

utils.save_model_summary(model, flags.train_dir)

def save_model_summary(model, path, file_name='model_summary.txt'):
  """Saves model topology/summary in text format.

  Args:
    model: Keras model
    path: path where to store model summary
    file_name: model summary file name
  """
  with open(os.path.join(path, file_name), 'wt') as fd:
    stringlist = []
    # print_fn 默认是 print ，传入的是函数
    model.summary(print_fn=lambda x: stringlist.append(x))  # pylint: disable=unnecessary-lambda
    model_summary = '\n'.join(stringlist)
    fd.write(model_summary)

  # save model and data flags
  with open(os.path.join(flags.train_dir, 'flags.txt'), 'wt') as f:
    # stream 流向f，默认是屏幕
    pprint.pprint(flags, stream=f)

0x03 训练

训练的过程一直是 non-stream，不过每一层会产生self.state_shape 这个值

3.1 优化器等参数配置

# 设定loss
loss = tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True)
# optimizer = tf.keras.optimizers.Adam(epsilon=flags.optimizer_epsilon)

# 设定优化器
if flags.optimizer == 'adam':
  optimizer = tf.keras.optimizers.Adam(epsilon=flags.optimizer_epsilon)
elif flags.optimizer == 'momentum':
  optimizer = tf.keras.optimizers.SGD(momentum=0.9)
else:
  raise ValueError('Unsupported optimizer:%s' % flags.optimizer)

model.compile(optimizer=optimizer, loss=loss, metrics=['accuracy'])

# 指定一个文件用来保存图
train_writer = tf.summary.FileWriter(flags.summaries_dir + '/train',
                                     sess.graph)
validation_writer = tf.summary.FileWriter(flags.summaries_dir + '/validation')

# 初始化所有变量
sess.run(tf.global_variables_initializer())

start_step = 1

logging.info('Training from step: %d ', start_step)

# Save graph.pbtxt.
# 保存图, 但不保存variable
tf.train.write_graph(sess.graph_def, flags.train_dir, 'graph.pbtxt')

# Save list of words.
# 保存标签
with tf.io.gfile.GFile(os.path.join(flags.train_dir, 'labels.txt'), 'w') as f:
  f.write('\n'.join(audio_processor.words_list))

best_accuracy = 0.0

# prepare parameters for exp learning rate decay
training_steps_max = np.sum(training_steps_list)
lr_init = learning_rates_list[0]
# exp_rate 与后面的各个阶段的学习率有关
exp_rate = -np.log(learning_rates_list[-1] / lr_init)/training_steps_max

3.2 开始训练

获取输入、学习率设定

训练方式：

result = model.train_on_batch(train_fingerprints, train_ground_truth)

3.3 训练结束

会生成如下文件

./speech_commands_train
├── accuracy_last.txt  # 训练结束后的测试集的准确率
├── best_weights.data-00000-of-00002  # 最高准确率的权重保存
├── best_weights.data-00001-of-00002  # 最高准确率的权重保存
├── best_weights.index  # 最高准确率的权重保存
├── flags.txt  # flags 参数保存
├── graph.pbtxt  # 网络结构图保存，The model architecture, and training configuration
├── labels.txt  # 标签保存
├── last_weights.data-00000-of-00002  # 最后一次的权重保存？？
├── last_weights.data-00001-of-00002  # 最后一次的权重保存？？
├── last_weights.index  # 最后一次的权重保存？？
├── logs  # tensorlfow 可视化工具
│   ├── train
│   │   ├── events.out.tfevents.1591857342.ubuntu
│   │   ├── events.out.tfevents.1591857494.ubuntu
│   │   ├── events.out.tfevents.1591870751.RT-AI
│   │   └── events.out.tfevents.1591875621.ubuntu
│   └── validation
│       ├── events.out.tfevents.1591857855.ubuntu
│       └── events.out.tfevents.1591870970.RT-AI
├── model_summary.txt  # 模型结构及参数保存
└── train  # 每经过400次 step 保存一个权重文件，name: acc+weights+steps
    ├── 0weights_400.data-00000-of-00002
    ├── 0weights_400.data-00001-of-00002
    ├── 0weights_400.index
    ├── ...
    ├── 9027weights_15000.data-00000-of-00002
    ├── 9027weights_15000.data-00001-of-00002
    ├── 9027weights_15000.index
    └── checkpoint

查看权重详细数值（tensorflow1)

查看了之后存在一个问题，所有的权重文件数值都是一样的，文件的大小也是一样的，不知道原因

from tensorflow.python import pywrap_tensorflow

checkpoint_path = 'your saved weight file path'
# Read data from checkpoint file
reader = pywrap_tensorflow.NewCheckpointReader(checkpoint_path)
var_to_shape_map = reader.get_variable_to_shape_map()
# Print tensor name and values
for key in var_to_shape_map:
    print("tensor_name: ", key)
    print(reader.get_tensor(key))

0x04 保存model

4.1 保存non-stream model

# convert to SavedModel
test.convert_model_saved(flags, 'non_stream', Modes.NON_STREAM_INFERENCE)

# 在convert_model_saved() 函数中使用了这么一行代码
# convert_model_saved(flags, folder, mode, weights_name='best_weights')
utils.model_to_saved(model, flags, path_model, mode)

model_to_saved() 的功能是保存网络结构到txt 和保存网络结构图，权重文件保存在variables下面

./non_stream
├── assets  # 未知
├── model_summary.txt
├── saved_model.pb
└── variables
    ├── variables.data-00000-of-00002
    ├── variables.data-00001-of-00002
    └── variables.index

4.2 保存 stream model

test.convert_model_saved(flags, 'stream_state_internal',
                         Modes.STREAM_INTERNAL_STATE_INFERENCE)

同样调用utils.model_to_saved()这个函数，不同的是mode = Modes.STREAM_INTERNAL_STATE_INFERENCE，

在保存之前，多了这么一步，这一步完成之后才是保存网络结构到txt 和保存网络结构图:

# convert non streaming Keras model to Keras streaming model, internal state
model = to_streaming_inference(model_non_stream, flags, mode)

上面的函数的功能是将之前的训练好的模型转换成 streaming model

怎么转换呢？继续深入源码

发现其中有两个比较重要的函数：

model = _set_mode(model, mode)
new_model = _clone_model(model, input_tensors)

第一个函数是将model.layers 中的所有mode 从TRAINING 改为STREAM_INTERNAL_STATE_INFERENCE；

第二个函数可就厉害了，功能是克隆网络结构和configs，

写的代码，调用的tensorflow 的API在市面上没有任何的流通资料，一头抓瞎，心态裂开，源码如下：

def _clone_model(model, input_tensors):
  """Clone model with configs, except of weights."""
  new_input_layers = {}  # Cache for created layers.
  # pylint: disable=protected-access
  if input_tensors is not None:
    # Make sure that all input tensors come from a Keras layer.
    input_tensors = tf.nest.flatten(input_tensors)
    for i, input_tensor in enumerate(input_tensors):
      if not tf.keras.backend.is_keras_tensor(input_tensor):
        raise ValueError('Expected keras tensor but get', input_tensor)
      
      # 新旧Input 保存成一个字典 dict[旧] = 新
      original_input_layer = model._input_layers[i]
      newly_created_input_layer = input_tensor._keras_history.layer
      new_input_layers[original_input_layer] = newly_created_input_layer

  # 将原来的第一层 Input层改为现在的Input层
  # shape [1, 25, 10] --> [1, 1, 10]
  # 如果不是第一层，就copy 各层的config
  model_config, created_layers = models._clone_layers_and_model_config(
      model, new_input_layers, models._clone_layer)
  # pylint: enable=protected-access

  # Reconstruct model from the config, using the cloned layers.
  input_tensors, output_tensors, created_layers = (
      functional.reconstruct_from_config(
          model_config, created_layers=created_layers))

  new_model = tf.keras.Model(input_tensors, output_tensors, name=model.name)
  return new_model

此处引发了关于tf版本的一个问题，需要tf_nightly==2.3.0.dev20200515 # was validated on tf_nightly-2.3.0.dev20200515-cp36-cp36m-manylinux2010_x86_64.whl ，未完待续

4.3 最终的文件目录

⇒  tree ./train_model/cnn
./train_model/cnn
├── accuracy_last.txt
├── best_weights.data-00000-of-00002
├── best_weights.data-00001-of-00002
├── best_weights.index
├── flags.json
├── flags.txt
├── graph.pbtxt
├── labels.txt
├── last_weights.data-00000-of-00002
├── last_weights.data-00001-of-00002
├── last_weights.index
├── logs
│   ├── train
│   │   ├── events.out.tfevents.1591857342.ubuntu
│   │   ├── events.out.tfevents.1591857494.ubuntu
│   │   ├── events.out.tfevents.1591870751.RT-AI
│   │   └── events.out.tfevents.1591875621.ubuntu
│   └── validation
│       ├── events.out.tfevents.1591857855.ubuntu
│       └── events.out.tfevents.1591870970.RT-AI
├── model_summary.txt
├── non_stream
│   ├── assets
│   ├── model_summary.txt
│   ├── saved_model.pb
│   └── variables
│       ├── variables.data-00000-of-00002
│       ├── variables.data-00001-of-00002
│       └── variables.index
├── quantize_opt_for_size_tflite_non_stream
│   ├── model_summary.txt
│   ├── non_stream.tflite
│   └── tflite_non_stream_model_accuracy.txt
├── quantize_opt_for_size_tflite_stream_state_external
│   ├── model_summary.txt
│   ├── stream_state_external.tflite
│   ├── tflite_stream_state_external_model_accuracy_reset0.txt
│   └── tflite_stream_state_external_model_accuracy_reset1.txt
├── stream_state_internal
│   ├── assets
│   ├── model_summary.txt
│   ├── saved_model.pb
│   └── variables
│       ├── variables.data-00000-of-00002
│       ├── variables.data-00001-of-00002
│       └── variables.index
├── tf
│   ├── model_summary_non_stream.png
│   ├── model_summary_non_stream.txt
│   ├── model_summary_stream_state_external.png
│   ├── model_summary_stream_state_external.txt
│   ├── model_summary_stream_state_internal.png
│   ├── model_summary_stream_state_internal.txt
│   ├── stream_state_external_model_accuracy_sub_set_reset0.txt
│   ├── stream_state_external_model_accuracy_sub_set_reset1.txt
│   ├── tf_non_stream_model_accuracy.txt
│   ├── tf_non_stream_model_sampling_stream_accuracy.txt
│   └── tf_stream_state_internal_model_accuracy_sub_set.txt
├── tflite_non_stream
│   ├── model_summary.txt
│   ├── non_stream.tflite
│   └── tflite_non_stream_model_accuracy.txt
├── tflite_stream_state_external
│   ├── model_summary.txt
│   ├── stream_state_external.tflite
│   ├── tflite_stream_state_external_model_accuracy_reset0.txt
│   └── tflite_stream_state_external_model_accuracy_reset1.txt
└── train
    ├── 0weights_400.data-00000-of-00002
    ├── 0weights_400.data-00001-of-00002
    ├── 0weights_400.index
    ├── ...
    ├── 9027weights_15000.data-00000-of-00002
    ├── 9027weights_15000.data-00001-of-00002
    ├── 9027weights_15000.index
    └── checkpoint

0x05 Other 问题集收录

1 stride_ms 40 & dct_num_features 10 报错

    ValueError: Negative dimension size caused by subtracting 5 from 4 for '{{node stream_4/conv2d_4/Conv2D}} = Conv2D[T=DT_FLOAT, 
data_format="NHWC", dilations=[1, 1, 1, 1], explicit_paddings=[], padding="VALID", strides=[1, 1, 1, 1], use_cudnn_on_gpu=true]
(stream_4/conv2d_4/SpaceToBatchND, stream_4/conv2d_4/Conv2D/ReadVariableOp)' with input shapes: [200,4,2,64], [5,2,64,128]

第五层卷积核过大，重新设置参数：

--cnn_kernel_size (3,3),(5,3),(5,3),(5,3),(5,2),(5,1),(3,1) \
--cnn_dilation_rate (1,1),(1,1),(1,1),(1,1),(1,1),(1,1),(1,1) \

出现新的报错：

    raise ValueError('malformed node or string: ' + repr(node))
ValueError: malformed node or string: <_ast.Name object at 0x7f8ca813a630>

原因：

--units2 128,256 \
--act2 linear,relu

参数传入有问题，删除即可

class MyLayer(layers.Layer):
    def __init__(self, input_dim=32, unit=32):
        super(MyLayer, self).__init__()
        self.weight = self.add_weight(shape=(input_dim, unit),
                                     initializer=keras.initializers.RandomNormal(),
                                     trainable=True)
        self.bias = self.add_weight(shape=(unit,),
                                   initializer=keras.initializers.Zeros(),
                                   trainable=True)
    
    def call(self, inputs):
        return tf.matmul(inputs, self.weight) + self.bias

2. 最开始训练时传入参数有个坑

末尾需要补上’\’，否则保存的权重文件会出现在神奇的位置

# 保存训练的权重
model.save_weights(flags.train_dir + 'train/' +
                         str(int(best_accuracy * 10000)) + 'weights_' +
                         str(training_step))

# 保存每次验证的最优的权重
model.save_weights(flags.train_dir + 'best_weights')

# 保存最后一次测试的权重
model.save_weights(flags.train_dir + 'last_weights')

3. 模型训练好之后，non-stream to streaming

WARNING: failed to convert to SavedModel: module 'tensorflow.python.keras.engine.node' has no attribute '_CONSTANT_VALUE'

在 /home/lebhoryi/anaconda3/envs/tf2/lib/python3.6/site-packages/tensorflow/python/keras/engine/node.py中，少了一行定义，增加在第25行

_CONSTANT_VALUE = '_CONSTANT_VALUE'

下面的解决思想产生了点小偏差，花了一天多一点的时间，最后在下班之前发现是这个原因：

/home/lebhoryi/anaconda3/envs/tf2/lib/python3.6/site-packages/tensorflow_core/python/keras/engine/node.py

版本不匹配，包括上面的_CONSTANT_VALUE 值的问题，战术绕头，有点懵逼。

在github master分支上的node.py 有output、_CONSTANT_VALUE两个属性的，但是在tf2.1 和 2.2 中的node.py 就不存在，所以会报错

======== update 2020/06/17 版本问题 =========

作者给出的意见，更新到版本 tf_nightly==2.3.0.dev20200515 # was validated on tf_nightly-2.3.0.dev20200515-cp36-cp36m-manylinux2010_x86_64.whl，

问题看上去解决了

未完待续…

（一些迷了路的想法）

后续继续引发问题：
WARNING: failed to convert to SavedModel: 'Node' object has no attribute 'outputs'
        
# 上述问题代码出错在这一行：
(model_config, created_layers) = models._clone_layers_and_model_config(
      model, new_input_layers, models._clone_layer)
问题原因在于在clone的过程中，丢失了每一层的output这个参数：

正常是这样的：
# 往下面找，发现问题在这
Layer stream has no inbound nodes.
在调用output 属性之前，有一个叫做inbound_nodes 的东西，这个不存在则导致input & output 不存在

这些inbound nodes是什么？
A Node describes the connectivity between two layers. Each time a layer is connected to some new input, a node is added to layer._inbound_nodes. Each time the output of a layer is used by another layer, a node is added to layer._outbound_nodes.

4. 在模型训练好之后转换成`tflite`的过程中

tensorflow.python.framework.errors_impl.InvalidArgumentError: Matrix size-incompatible: In[0]: [100,128], In[1]: [1408,128]
	 [[{{node dense/BiasAdd}}]]

这个是传入的参数不对，训练时的参数dct是20，测试的时候传入10，因此报错

新的问题：[在tf版本更新之后，该问题小时了]

I0615 15:00:39.572662 140162058372864 model_train_eval.py:164] run TF non streaming model accuracy evaluation
I0615 15:01:34.501143 140162058372864 test.py:80] TF Final test accuracy on non stream model = 89.95% (N=5600)
I0615 15:02:19.639499 140162058372864 test.py:80] TF Final test accuracy on non stream model = 89.18% (N=5600)
...
I0615 15:02:20.397487 140162058372864 utils.py:216] None
W0615 15:02:20.405573 140162058372864 stream.py:103] There is no need to use Stream on time dim with size 1
W0615 15:02:20.410688 140162058372864 test.py:582] FAILED to convert to mode NON_STREAM_INFERENCE, tflite: 'Node' object has no attribute 'outputs'
Traceback (most recent call last):
  File "train/model_train_eval.py", line 331, in <module>
    tf.app.run(main=main, argv=[sys.argv[0]] + unparsed)
  File "/home/lebhoryi/anaconda3/envs/tf2/lib/python3.6/site-packages/tensorflow/python/platform/app.py", line 40, in run
    _run(main=main, argv=argv, flags_parser=_parse_flags_tolerate_undef)
  File "/home/lebhoryi/anaconda3/envs/tf2/lib/python3.6/site-packages/absl/app.py", line 299, in run
    _run_main(main, args)
  File "/home/lebhoryi/anaconda3/envs/tf2/lib/python3.6/site-packages/absl/app.py", line 250, in _run_main
    sys.exit(main(argv))
  File "train/model_train_eval.py", line 202, in main
    test.tflite_non_stream_model_accuracy(flags, folder_name, file_name)
  File "/home/lebhoryi/WakeUp-Xiaorui/kws_streaming/train/test.py", line 498, in tflite_non_stream_model_accuracy
    model_path=os.path.join(path, tflite_model_name))
  File "/home/lebhoryi/anaconda3/envs/tf2/lib/python3.6/site-packages/tensorflow/lite/python/interpreter.py", line 204, in __init__
    model_path, self._custom_op_registerers))
ValueError: Mmap of '/tmp/speech_commands_train/tflite_non_stream/non_stream.tflite' failed.

5. 在non-stream to streaming 过程中

在运行这一行代码的时候：

  (model_config, created_layers) = models._clone_layers_and_model_config(
      model, new_input_layers, models._clone_layer)

警告如下：

There is no need to use Stream on time dim with size 1

这是因为在传入参数的时候，在streaming_7 层时，传入的self.stae_shape = [1, 1, 11, 128],

if isinstance(self.cell, tf.keras.layers.Flatten):
      # effective kernel size in time dimension
      if self.state_shape:
        self.ring_buffer_size_in_time_dim = self.state_shape[1]
        
if self.ring_buffer_size_in_time_dim == 1:
      logging.warn('There is no need to use Stream on time dim with size 1')

Lebhoryi

关注

0
点赞
踩
4

收藏

觉得还不错? 一键收藏
3
评论
stream_kws_cnn

文章目录0x00 parser 超参传入0x01 数据预处理0x02 model 此处选择 CNN2.1 输入( 此处 flags.preprocess = mfcc)2.2 输入增维2.3 构建网络2.3.1 stream()函数2.3.2 网络结构模型 CNN2.3.3 保存网络结构模型和参数到 txt0x03 训练3.1 优化器等参数配置3.2 开始训练3.3 训练结束0x04 保存model4.1 保存non-stream model4.2 保存 stream model4.3 最终的文件目录0x0
复制链接

扫一扫