imagenet download and train

reference :

imagenet下载及训练 - 仰望高端玩家的小清新 - 博客园

1. download dataset

1.1 Download Torrent.

validation dataset http://academictorrents.com/download/5d6d0df7ed81efd49ca99ea4737e0ae5e3a5f2e5.torrent
train dataset http://academictorrents.com/download/a306397ccf9c2ead27155983c254227c0fd938e2.torrent

1.2 Then download it by thunder.

1.3 Md5sum Check

# After the download, I got two tar files. 
# For preventing file damage, please use md5 to verify.
# Val Verify, expect："29b22e2961454d5413ddabcf34fc5622"
md5sum ILSVRC2012_img_val.tar
# Train Verify, expect：1d675b47d978889d74fa0da5fadfb00e
md5sum ILSVRC2012_img_train.tar

2 Tar -> Prepare -> Tar

val:

wget https://files-cdn.cnblogs.com/files/luruiyuan/valprep.sh
mkdir val && tar -xvf ILSVRC2012_img_val.tar -C val 
mv valprep.sh val && cd val && bash valprep.sh && cd ../

train:

mkdir train && tar -xvf ILSVRC2012_img_train.tar -C train
for x in `ls train/*tar`; do fn=train/`basename $x .tar`; mkdir $fn; tar -xvf $x -C $fn; rm -f $fn.tar; done

3 Statistic

val:

# Expect 51001 = 50000 + 1000folder + 1 shell(valprec.sh)
cnt=0; for f in `ls`; do cd $f; cur=`ls|wc -l`; cnt=`expr $cnt + $cur`; cd ..; done; echo $cnt

train:

# Expect 1281167
cnt=0; for f in `ls`; do cd $f; cur=`ls|wc -l`; cnt=`expr $cnt + $cur`; cd ..; done; echo $cnt