问题描述:建立好model之后,用model.fit()函数进行训练,发现超出显存容量(一共有12G)
问题分析:fit()函数训练时,将全部训练集载入显存之后,才开始分批训练。显然很容易就超出了12G
解决办法:用fit_generator函数进行训练
fit_generator函数将训练集分批载入显存,但需要自定义其第一个参数——generator函数,从而分批将训练集送入显存
def data_generator(data, targets, batch_size):
batches = (len(data) + batch_size - 1)//batch_size
while(True):
for i in range(batches):
X = data[i*batch_size : (i+1)*batch_size]
Y = targets[i*batch_size : (i+1)*batch_size]
yield (X, Y)
调用fit_generator时的方法
model.fit_generator(generator = data_generator(X_train, Y_train, batch_size),
steps_per_epoch = (len(data) + batch_size - 1) // batch_size,
epochs = num_epochs,
verbose = 1,
callbacks = callbacks,
validation_data = (X_val, Y_val)
)
参考链接:https://zhuanlan.zhihu.com/p/23250782
http://blog.csdn.net/sinat_26917383/article/details/74922230