API 接口
首先
import pycaffe
前向后向操作,处理IO,可视化网络,模型求解,所有的模型数据和参数,这些内容都已经有了接口,因此可以读写.
caffe.Net is the central interface for loading, configuring, and running models.
caffe.Classifier and caffe.Detector provide convenience interfaces for common tasks.
caffe.SGDSolver exposes the solving interface.
caffe.io handles input / output with preprocessing and protocol buffers.
caffe.draw visualizes network architectures.
Caffe blobs are exposed as numpy ndarrays for ease-of-use and efficiency.
文档的介绍还比较少,因此可以仔细阅读代码
python文件夹下有几个朋友文件和几个cpp文件,
drwxrwxr-x 5 bobin bobin 4096 5月 17 15:41 ./
drwxrwxr-x 3 bobin bobin 4096 4月 17 2016 ../
-rw-rw-r-- 1 bobin bobin 14208 4月 17 2016 _caffe.cpp
-rwxrwxr-x 1 bobin bobin 1485231 5月 17 15:41 _caffe.so*
-rw-rw-r-- 1 bobin bobin 3501 4月 17 2016 classifier.py
-rw-rw-r-- 1 bobin bobin 6721 4月 17 2016 coord_map.py
-rw-rw-r-- 1 bobin bobin 8562 4月 17 2016 detector.py
-rw-rw-r-- 1 bobin bobin 7604 4月 17 2016 draw.py
drwxrwxr-x 2 bobin bobin 4096 4月 17 2016 imagenet/
-rw-rw-r-- 1 bobin bobin 417 4月 17 2016 __init__.py
-rw-rw-r-- 1 bobin bobin 12695 4月 17 2016 io.py
-rw-rw-r-- 1 bobin bobin 8025 4月 17 2016 net_spec.py
drwxrwxr-x 2 bobin bobin 4096 5月 17 15:41 proto/
-rw-rw-r-- 1 bobin bobin 11001 4月 17 2016 pycaffe.py
drwxrwxr-x 2 bobin bobin 4096 4月 17 2016 test/
_caffe.cpp中的定义了几个类和函数,使用Python包装起来,包括set_mode_cpu,set_mode_gpu,
类Net,函数_forward,_backward,reshape
类Blob,类Layer类Sovler和SGDSolver等.
Classifier.py定义了类Classifier,包括函数predict和
io.py定义了load_image,resize_image等函数
draw.py定义了draw_net的函数,用于可视化使用的网络
caffe/python/caffe/_caffe.cpp:
Exports Blob, Layer, Net, and Solver classes
caffe/python/caffe/pycaffe.py
Adds extra methods to Net class
构造网络
from caffe import layers as L
from caffe import params as P
def lenet(lmdb, batch_size):
n = caffe.NetSpec()
n.data, n.label = L.Data(batch_size=batch_size, backend=P.Data.LMDB, source=lmdb,
transform_param=dict(scale=1./255), ntop=2)
n.conv1 = L.Convolution(n.data, kernel_size=5, num_output=20, weight_filler=dict(type='xavier'))
n.pool1 = L.Pooling(n.conv1, kernel_size=2, stride=2, pool=P.Pooling.MAX)
n.conv2 = L.Convolution(n.pool1, kernel_size=5, num_output=50, weight_filler=dict(type='xavier'))
n.pool2 = L.Pooling(n.conv2, kernel_size=2, stride=2, pool=P.Pooling.MAX)
n.ip1 = L.InnerProduct(n.pool2, num_output=500, weight_filler=dict(type='xavier'))
n.relu1 = L.ReLU(n.ip1, in_place=True)
n.ip2 = L.InnerProduct(n.relu1, num_output=10, weight_filler=dict(type='xavier'))
n.loss = L.SoftmaxWithLoss(n.ip2, n.label)
return n.to_proto()
with open('examples/mnist/lenet_auto_train.prototxt', 'w') as f:
f.write(str(lenet('examples/mnist/mnist_train_lmdb', 64)))
with open('examples/mnist/lenet_auto_test.prototxt', 'w') as f:
f.write(str(lenet('examples/mnist/mnist_test_lmdb', 100)))
- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
- 10
- 11
- 12
- 13
- 14
- 15
- 16
- 17
- 18
- 19
- 20
- 21
- 22
- 23
这里是首先用python定义网络结构,然后输出到prototxt文件,这样可以用来训练和测试.
训练网络
solver = caffe.get_solver('models/bvlc_reference_caffenet/solver.prototxt')
solver.net.forward()
solver.test_nets[0].forward()
solver.net.backward()
solver.solve()
accuracy = 0
batch_size = solver.test_nets[0].blobs['data'].num
test_iters = int(len(Xt) / batch_size)
for i in range(test_iters):
solver.test_nets[0].forward()
accuracy += solver.test_nets[0].blobs['accuracy'].data
accuracy /= test_iters
print("Accuracy: {:.3f}".format(accuracy))
- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
- 10
- 11
- 12
- 13
- 14
- 15
- 16
- 17
这里首先加载网络参数和训练数据,训练数据的源在网络的底层指定.
应用网络
net = caffe.Net('conv.prototxt', caffe.TEST)
#load the model
net = caffe.Net('models/bvlc_reference_caffenet/deploy.prototxt',
'models/bvlc_reference_caffenet/bvlc_reference_caffenet.caffemodel',
caffe.TEST)
# load input and configure preprocessing
transformer = caffe.io.Transformer({'data': net.blobs['data'].data.shape})
transformer.set_mean('data', np.load('python/caffe/imagenet/ilsvrc_2012_mean.npy').mean(1).mean(1))
transformer.set_transpose('data', (2,0,1))
transformer.set_channel_swap('data', (2,1,0))
transformer.set_raw_scale('data', 255.0)
#note we can change the batch size on-the-fly
#since we classify only one image, we change batch size from 10 to 1
net.blobs['data'].reshape(1,3,227,227)
#load the image in the data layer
im = caffe.io.load_image('examples/images/cat.jpg')
net.blobs['data'].data[...] = transformer.preprocess('data', im)
#compute
out = net.forward()
# other possibility : out = net.forward_all(data=np.asarray([transformer.preprocess('data', im)]))
#predicted predicted class
print out['prob'].argmax()
#print predicted labels
labels = np.loadtxt("data/ilsvrc12/synset_words.txt", str, delimiter='\t')
top_k = net.blobs['prob'].data[0].flatten().argsort()[-1:-6:-1]
print labels[top_k]
- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
- 10
- 11
- 12
- 13
- 14
- 15
- 16
- 17
- 18
- 19
- 20
- 21
- 22
- 23
- 24
- 25
- 26
- 27
- 28
- 29
- 30
- 31
- 32
- 33
这里使用了Net,也可以使用Classifier,那么可以使用函数predict,net对应的forward函数.
可视化网络
python python/draw_net.py models/bvlc_reference_caffenet/deploy.prototxt caffenet.png
open caffenet.png
refer
- https://christopher5106.github.io/deep/learning/2015/09/04/Deep-learning-tutorial-on-Caffe-Technology.html
- http://adilmoujahid.com/posts/2016/06/introduction-deep-learning-python-caffe/
- https://nbviewer.jupyter.org/github/joyofdata/joyofdata-articles/blob/master/deeplearning-with-caffe/Neural-Networks-with-Caffe-on-the-GPU.ipynb