linux编译tiny cnn,wyh/xilinx-tiny-cnn

tiny-cnn: A header only, dependency-free deep learning framework in C++11

Xilinx changes from original tiny-cnn:

added batchnorm layer (currently feedforward only, no training)

support for offloaded layer

interleave layer

binarized layers

Linux/Mac OS

Windows

tiny-cnn is a C++11 implementation of deep learning. It is suitable for deep learning on limited computational resource, embedded systems and IoT devices.

see Wiki Pages for more info.

Features

fast, without GPU

with TBB threading and SSE/AVX vectorization

98.8% accuracy on MNIST in 13 minutes training (@Core i7-3520M)

header only

Just include tiny_cnn.h and write your model in c++. There is nothing to install.

small dependency & simple implementation

Comparison with other libraries

Prerequisites

Nothing(Optional:TBB,OpenMP)

BLAS,Boost,protobuf,glog,gflags,hdf5, (Optional:CUDA,OpenCV,lmdb,leveldb etc)

Numpy,Scipy,BLAS,(optional:nose,Sphinx,CUDA etc)

numpy,six,protobuf,(optional:CUDA,Bazel)

Modeling By

C++ code

Config File

Python Code

Python Code

GPU Support

No

Yes

Yes

Yes

Installing

Unnecessary

Necessary

Necessary

Necessary

Windows Support

Yes

No*

Yes

No*

Pre-Trained Model

Yes(via caffe-converter)

Yes

No*

No*

*unofficial version is available

Supported networks

layer-types

fully-connected layer

convolutional layer

average pooling layer

max-pooling layer

contrast normalization layer

dropout layer

linear operation layer

activation functions

tanh

sigmoid

softmax

rectified linear(relu)

leaky relu

identity

exponential linear units(elu)

loss functions

cross-entropy

mean-squared-error

optimization algorithm

stochastic gradient descent (with/without L2 normalization and momentum)

stochastic gradient levenberg marquardt

adagrad

rmsprop

adam

Dependencies

Minimum requirements

Nothing.All you need is a C++11 compiler.

Requirements to build sample/test programs

Build

tiny-cnn is header-ony, so there's nothing to build. If you want to execute sample program or unit tests, you need to install cmake and type the following commands:

cmake .

Then open .sln file in visual studio and build(on windows/msvc), or type make command(on linux/mac/windows-mingw).

Some cmake options are available:

options

description

default

additional requirements to use

USE_TBB

Use Intel TBB for parallelization

OFF*

USE_OMP

Use OpenMP for parallelization

OFF*

USE_SSE

Use Intel SSE instruction set

ON

Intel CPU which supports SSE

USE_AVX

Use Intel AVX instruction set

ON

Intel CPU which supports AVX

BUILD_TESTS

Build unist tests

OFF

-**

BUILD_EXAMPLES

Build example projects

ON

-

*tiny-cnn use c++11 standard library for parallelization by default

**to build tests, type git submodule update --init before build

For example, type the following commands if you want to use intel TBB and build tests:

cmake -DUSE_TBB=ON -DBUILD_EXAMPLES=ON .

Customize configurations

You can edit include/config.h to customize default behavior.

Examples

construct convolutional neural networks

#include "tiny_cnn/tiny_cnn.h"

using namespace tiny_cnn;

using namespace tiny_cnn::activation;

void construct_cnn() {

using namespace tiny_cnn;

// specify loss-function and optimization-algorithm

network net;

//network net;

// add layers

net << convolutional_layer(32, 32, 5, 1, 6) // 32x32in, conv5x5, 1-6 f-maps

<< average_pooling_layer(28, 28, 6, 2) // 28x28in, 6 f-maps, pool2x2

<< fully_connected_layer(14 * 14 * 6, 120)

<< fully_connected_layer(120, 10);

assert(net.in_dim() == 32 * 32);

assert(net.out_dim() == 10);

// load MNIST dataset

std::vector train_labels;

std::vector train_images;

parse_mnist_labels("train-labels.idx1-ubyte", &train_labels);

parse_mnist_images("train-images.idx3-ubyte", &train_images);

// train (50-epoch, 30-minibatch)

net.train(train_images, train_labels, 30, 50);

// save

std::ofstream ofs("weights");

ofs << net;

// load

// std::ifstream ifs("weights");

// ifs >> net;

}

construct multi-layer perceptron(mlp)

#include "tiny_cnn/tiny_cnn.h"

using namespace tiny_cnn;

using namespace tiny_cnn::activation;

void construct_mlp() {

network net;

net << fully_connected_layer(32 * 32, 300)

<< fully_connected_layer(300, 10);

assert(net.in_dim() == 32 * 32);

assert(net.out_dim() == 10);

}

another way to construct mlp

#include "tiny_cnn/tiny_cnn.h"

using namespace tiny_cnn;

using namespace tiny_cnn::activation;

void construct_mlp() {

auto mynet = make_mlp({ 32 * 32, 300, 10 });

assert(mynet.in_dim() == 32 * 32);

assert(mynet.out_dim() == 10);

}

more sample, read examples/main.cpp or MNIST example page.

References

[2] Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner, Gradient-based learning applied to document recognition.

Proceedings of the IEEE, 86, 2278-2324.

other useful reference lists:

License

The BSD 3-Clause License

Mailing list

google group for questions and discussions: