今天使用keras分批训练model时使用model.fit_generator方法报错:ValueError: No data provided for “dense_1_input”. Need data for each key in: [‘dense_1_input’]
最后通过分析keras源码,找到了解决方案:将用于产生batch的generator的返回值的格式写成一个字典,长这样:({‘dense_1_input’: data}, {‘dense_3’: target})
大家看上面的tuple,我解释一下:这个generator的返回值是一个tuple,这个tuple应该由两个字典组成,第一个字典的键为第一层的名字,第二个字典的键为最后一层的名字,比如,我的神经网络结构是:一个输入全连接层+dropout0.5+一个隐藏全连接层+dropout0.5+一个输出全连接层。查看源码,我们就知道了第一层名字是dense_1_input,最后一层名字是dense_3。
但获取每个层的名字其实不用查看源码,直接print(model.summary)就可以查看了。很简单。
修改了键之后,编译可以通过并可以正常训练了。
我的keras版本是2.1.2。
补充一下,keras源码中的fit_generator的注释,给出了一个格式:
generator: A generator.
The output of the generator must be either
- a tuple (inputs, targets)
- a tuple (inputs, targets, sample_weights).
All arrays should contain the same number of samples.
The generator is expected to loop over its data
indefinitely. An epoch finishes when `steps_per_epoch`
batches have been seen by the model.
并给出了一个generator的例子:
# Example
```python
def generate_arrays_from_file(path):
while 1:
f = open(path)
for line in f:
# create Numpy arrays of input data
# and labels, from each line in the file
x, y = process_line(line)
yield (x, y)
f.close()
model.fit_generator(generate_arrays_from_file('/my_file.txt'),
steps_per_epoch=1000, epochs=10)
```
我的程序里面,这个generator的样子是:
def make_batch(self, batch_size, validation=False):
for times in range(self.ks_cfg['ks_train_epoch']):
if len(self.language_tgt) != len(self.language_src):
raise Exception('LengthError:target length and source length is not same.', str(len(self.language_src))+'!='+str(len(self.language_tgt)))
for index in range(0, len(self.language_src), batch_size):
batch_sentence_list = self.language_src[index: index+batch_size]
sentences_vec_list = self.sen2vec(sen_list=batch_sentence_list)
target_list = self.language_tgt[index: index+batch_size]
target_list = self.tar2one_hot(target_list)
# target_list = [[0, 1] for _ in range(len(sentences_vec_list))]
yield ({'dense_1_input': np.array(sentences_vec_list)}, {'dense_3': np.array(target_list)})
祝大家好运!