java调用caffe_如何强制caffe读取所有训练数据？

我正在使用caffe和STILL输入有问题 .

这是我的solver.prototxt：

train_net: "auto_train.prototxt"

test_net: "auto_test.prototxt"

test_iter: 800

test_interval: 20

base_lr: 0.01

momentum: 0.9

weight_decay: 0.0005

lr_policy: "inv"

gamma: 0.0001

power: 0.75

display: 100

max_iter: 10000

snapshot: 5000

snapshot_prefix: "sed"

solver_mode: GPU

这是正在运行的python脚本：

import os

PROJECT_HOME = '/home/romulus/code/project/'

CAFFE_HOME = '/home/romulus/code/caffe/'

os.chdir(PROJECT_HOME)

import sys

sys.path.insert(0, CAFFE_HOME + 'caffe/python')

import caffe, h5py

from pylab import *

from caffe import layers as L, params as P

OUTPUT_DIM = 8

def net(db, batch_size):

n = caffe.NetSpec()

n.data, n.label = L.Data(batch_size=batch_size, backend=P.Data.LEVELDB, source=db,

transform_param=dict(scale=1./255), ntop=2)

n.ip1 = L.InnerProduct(n.data, num_output=500, weight_filler=dict(type='xavier'))

n.relu1 = L.ReLU(n.ip1, in_place=True)

n.ip2 = L.InnerProduct(n.relu1, num_output=500, weight_filler=dict(type='xavier'))

n.relu2 = L.ReLU(n.ip2, in_place=True)

n.ip3 = L.InnerProduct(n.relu2, num_output=OUTPUT_DIM, weight_filler=dict(type='xavier'))

n.loss = L.SoftmaxWithLoss(n.ip3, n.label)

return n.to_proto()

with open('/home/romulus/code/project/auto_train.prototxt', 'w') as f:

f.write(str(net('/home/romulus/code/project/traindb', 64)))

with open('/home/romulus/code/project/auto_test.prototxt', 'w') as f:

f.write(str(net('/home/romulus/code/project/testdb', 100)))

caffe.set_device(0)

caffe.set_mode_gpu()

solver = caffe.SGDSolver(PROJECT_HOME + 'auto_solver.prototxt')

solver.net.forward() # train net

solver.test_nets[0].forward() # test net (there can be more than one)

niter = 500

test_interval = 15

train_loss = zeros(niter)

test_acc = zeros(int(np.ceil(niter * 1.0 / test_interval)))

output = zeros((niter, 8, OUTPUT_DIM))

for it in range(niter):

solver.step(1) # SGD by Caffe

train_loss[it] = solver.net.blobs['loss'].data

solver.test_nets[0].forward(start='ip1')

output[it] = solver.test_nets[0].blobs['ip3'].data[:8]

if it % test_interval == 0:

print 'Iteration', it, 'testing...'

correct = 0

for test_it in range(1):

solver.test_nets[0].forward()

correct += sum(solver.test_nets[0].blobs['ip3'].data.argmax(1)

== solver.test_nets[0].blobs['label'].data)

test_acc[it // test_interval] = correct * 1.0 / len(data)

_, ax1 = subplots()

ax2 = ax1.twinx()

ax1.plot(arange(niter), train_loss)

ax2.plot(test_interval * arange(len(test_acc)), test_acc, 'r')

ax1.set_xlabel('iteration')

ax1.set_ylabel('train loss')

ax2.set_ylabel('test accuracy')

_.savefig('converge.png')

手动生成数据，每个数据为1x256向量，所有相同的比例值为8 * label value . 这意味着，标签3的数据是[24,24,24,24,24 ......，24,24] . 我有8个标签和80000个数据 .

我的问题是，如果我将数据放入带有 0,1,2,3,4,5,6,7,8,0,1,2,3,4,5... 等标签顺序的leveldb，那么caffe可以很好地训练网络 . 但如果按照 0,0,...,0,0,1,1,1,...,1,1,2,2,... 命令，caffe就无法学习 . 如果我将solver.prototxt中的 test_iter 减少为100，则caffe将始终说输出标签为0 .

似乎caffe不会读取所有的训练数据，而只会读取前面的内容 . 但除了培训批次之外，我找不到任何描述它的内容 .

事实上，如果我将训练批量增加到80000，那么caffe会再次训练 . 虽然它很慢而且不是所谓的批次 .

有人可以帮忙吗？谢谢！