前言
Pytorch 1.8.3 ,python 3.8,CUDA 11.1.1

1 制作缩小版ava数据集
1.1 工具安装
先安装zip、unzip、ffmpeg这三个东西
apt-get update
apt-get install zip
apt-get install unzip
conda install x264 ffmpeg -c conda-forge -y
1.2 准备标注文件
提前下载好:https://research.google.com/ava/download/ava_v2.2.zip
然后上传文件至:AI平台的 /user-data/AVAFile
mkdir -p /user-data/AVAFile/ava /user-data/AVAFile/ava/{frames,frame_lists,annotations}
unzip -d /user-data/AVAFile/ava/annotations /user-data/AVAFile/ava_v2.2.zip
然后提前下载(train.csv 375.5M,val.csv103.1M):
上传到:/user-data/AVAFile/ava/frame_lists
train.csv:https://dl.fbaipublicfiles.com/video-long-term-feature-banks/data/ava/frame_lists/train.csv
val.csv:https://dl.fbaipublicfiles.com/video-long-term-feature-banks/data/ava/frame_lists/val.csv
最后提前下载:
上传到:/user-data/AVAFile/ava/annotations
ava_train_predicted_boxes.csv:https://dl.fbaipublicfiles.com/video-long-term-feature-banks/data/ava/annotations/ava_train_predicted_boxes.csv
ava_val_predicted_boxes.csv:https://dl.fbaipublicfiles.com/video-long-term-feature-banks/data/ava/annotations/ava_val_predicted_boxes.csv
ava_test_predicted_boxes.csv:https://dl.fbaipublicfiles.com/video-long-term-feature-banks/data/ava/annotations/ava_test_predicted_boxes.csv
ava
|_ frames
|_ frame_lists
| |_ train.csv
| |_ val.csv
|_ annotations
|_ [official AVA annotation files]
|_ ava_train_predicted_boxes.csv
|_ ava_val_predicted_boxes.csv

1.3 视频下载
我们只要中ava数据集中的两个:053oq2xB3oU 、Ytga8ciKWJc
我们先创建相关文件夹:
mkdir -p /user-data/AVAFile/avaMin /user-data/AVAFile/avaMin/{frames,frame_lists,annotations,videos,videos_15min}
依旧是提前下载后再上传到/user-data/AVAFile/avaMin/videos:
053oq2xB3oU.mkv:https://s3.amazonaws.com/ava-dataset/trainval/053oq2xB3oU.mkv
Ytga8ciKWJc.mkv:https://s3.amazonaws.com/ava-dataset/trainval/Ytga8ciKWJc.mkv
1.4 视频裁剪
然后在/user-data/AVAFile/avaMin/创建cutVideos.sh:
cd /user-data/AVAFile/avaMin
touch cutVideos.sh
在 cutVideos.sh输入:
IN_DATA_DIR="/user-data/AVAFile/avaMin/videos"
OUT_DATA_DIR="/user-data/AVAFile/avaMin/videos_15min"
if [[ ! -d "${OUT_DATA_DIR}" ]]; then
echo "${OUT_DATA_DIR} doesn't exist. Creating it.";
mkdir -p ${OUT_DATA_DIR}
fi
for video in $(ls -A1 -U ${IN_DATA_DIR}/*)
do
out_name="${OUT_DATA_DIR}/${video##*/}"
if [ ! -f "${out_name}" ]; then
ffmpeg -ss 900 -t 901 -i "${video}" "${out_name}"
fi
done
然后执行
cd /user-data/AVAFile/avaMin
bash cutVideos.sh
1.5 视频抽帧
cd /user-data/AVAFile/avaMin
touch extractFrames.sh
extractFrames.sh
IN_DATA_DIR="/user-data/AVAFile/avaMin/videos_15min"
OUT_DATA_DIR="/user-data/AVAFile/avaMin/frames"
if [[ ! -d "${OUT_DATA_DIR}" ]]; then
echo "${OUT_DATA_DIR} doesn't exist. Creating it.";
mkdir -p ${OUT_DATA_DIR}
fi
for video in $(ls -A1 -U ${IN_DATA_DIR}/*)
do
video_name=${video##*/}
if [[ $video_name = *".webm" ]]; then
video_name=${video_name::-5}
else
video_name=${video_name::-4}
fi
out_video_dir=${OUT_DATA_DIR}/${video_name}/
mkdir -p "${out_video_dir}"
out_name="${out_video_dir}/${video_name}_%06d.jpg"
ffmpeg -i "${video}" -r 30 -q:v 1 "${out_name}"
done
cd /user-data/AVAFile/avaMin
bash extractFrames.sh
1.6 ava_train
cd /user-data/AVAFile/avaMin
touch ava_train.py
内容如下:
import csv
# videos存放从 ava_train_v2.2.csv 截取对应视频标注的 视频名字
videos = ["053oq2xB3oU", "Ytga8ciKWJc"]
# minCsv 用以存放缩小版的ava数据集的标注内容
minCsv = []
with open('/user-data/AVAFile/ava/annotations/ava_train_v2.2.csv', 'r') as db01:
reader = csv.reader(db01)
for row in reader:
for video in videos:
if video in row:
minCsv.append(row)
with open('/user-data/AVAFile/avaMin/annotations/ava_train_v2.2.csv',"w") as csvfile:
writer = csv.writer(csvfile)
writer.writerows(minCsv)
# 只是做个测试,让训练集和测试集相等
with open('/user-data/AVAFile/avaMin/annotations/ava_val_v2.2.csv',"w") as csvfile:
writer = csv.writer(csvfile)
writer.writerows(minCsv)
执行:
cd /user-data/AVAFile/avaMin
python ava_train.py
1.7 excluded_timestamps
cd /user-data/AVAFile/avaMin/annotations
touch ava_train_excluded_timestamps_v2.2.csv
touch ava_val_excluded_timestamps_v2.2.csv
两者都写入
Ytga8ciKWJc,1633
1.8 predicted_boxes
cd /user-data/AVAFile/avaMin
touch ava_train_predicted_boxes.py
import csv
# videos存放从 ava_train_v2.2.csv 截取对应视频标注的 视频名字
videos = ["053oq2xB3oU", "Ytga8ciKWJc"]
# minCsv 用以存放缩小版的ava数据集的标注内容
minCsv = []
with open('/user-data/AVAFile/ava/annotations/ava_train_predicted_boxes.csv', 'r') as db01:
reader = csv.reader(db01)
for row in reader:
for video in videos:
if video in row:
minCsv.append(row)
with open('/user-data/AVAFile/avaMin/annotations/ava_train_predicted_boxes.csv',"w") as csvfile:
writer = csv.writer(csvfile)
writer.writerows(minCsv)
##################
# minCsv 用以存放缩小版的ava数据集的标注内容
minCsv = []
with open('/user-data/AVAFile/ava/annotations/ava_train_predicted_boxes.csv', 'r') as db01:
reader = csv.reader(db01)
for row in reader:
for video in videos:
if video in row:
row[6] = ''
minCsv.append(row)
with open('/user-data/AVAFile/avaMin/annotations/ava_test_predicted_boxes.csv',"w") as csvfile:
writer = csv.writer(csvfile)
writer.writerows(minCsv)
with open('/user-data/AVAFile/avaMin/annotations/ava_val_predicted_boxes.csv',"w") as csvfile:
writer = csv.writer(csvfile)
writer.writerows(minCsv)
执行
cd /user-data/AVAFile/avaMin
python ava_train_predicted_boxes.py
1.9 ava_test_v2.2
cd /user-data/AVAFile/avaMin/annotations
touch ava_test_v2.2.txt
写入
053oq2xB3oU
Ytga8ciKWJc
1.10 frame_lists_train
cd /user-data/AVAFile/avaMin/
touch frame_lists_train.py
import csv
# videos存放从 ava_train_v2.2.csv 截取对应视频标注的 视频名字
videos = ["053oq2xB3oU", "Ytga8ciKWJc"]
# minCsv 用以存放缩小版的ava数据集的标注内容
minCsv = []
minCsv.append(['original_vido_id' 'video_id' 'frame_id' 'path' 'labels'])
with open('/user-data/AVAFile/ava/frame_lists/train.csv', 'r') as db01:
reader = csv.reader(db01)
for row in reader:
for video in videos:
if video in row[0]:
minCsv.append(row)
with open('/user-data/AVAFile/avaMin/frame_lists/train.csv',"w") as csvfile:
writer = csv.writer(csvfile)
writer.writerows(minCsv)
with open('/user-data/AVAFile/avaMin/frame_lists/val.csv',"w") as csvfile:
writer = csv.writer(csvfile)
writer.writerows(minCsv)
cd /user-data/AVAFile/avaMin/
python frame_lists_train.py
1.11 其他文件
cp /user-data/AVAFile/ava/annotations/{ava_action_list_v2.2_for_activitynet_2019.pbtxt,ava_action_list_v2.2.pbtxt,ava_included_timestamps_v2.2.txt} /user-data/AVAFile/avaMin/annotations
1.12 最后文件结构
avaMin的目录结构
avaMin
|_ annotations
| |_ ava_action_list_v2.2_for_activitynet_2019.pbtxt ava_test_v2.2.txt
| |_ ava_val_excluded_timestamps_v2.2.csv
| |_ ava_action_list_v2.2.pbtxt
| |_ ava_train_excluded_timestamps_v2.2.csv
| |_ ava_val_predicted_boxes.csv
| |_ ava_included_timestamps_v2.2.txt
| |_ ava_train_predicted_boxes.csv
| |_ ava_val_v2.2.csv
| |_ ava_test_predicted_boxes.csv
| |_ ava_train_v2.2.csv
|_ frame_lists
| |_ train.csv
| |_ val.csv
|_ frames
| |_ 053oq2xB3oU
| | |_ 053oq2xB3oU_000001.jpg
| | |_ 053oq2xB3oU_000002.jpg
| | |_ ...
| |_ Ytga8ciKWJc
| |_ Ytga8ciKWJc_000001.jpg
| |_ Ytga8ciKWJc_000002.jpg
| |_ ...
|_ videos
| |_ 053oq2xB3oU.mkv
| |_ Ytga8ciKWJc.mkv
|_ videos_15min
|_ 053oq2xB3oU.mkv
|_ Ytga8ciKWJc.mkv
2 训练
安装 facebookresearch SlowFast参考:三分钟快速安装 facebookresearch SlowFast
2.1 未使用预训练模型训练
在/home/slowfast/configs/AVA/下的SLOWFAST_32x2_R50_SHORT.yaml,覆盖下面的内容
我改动的地方就是数据集指向的目录
TRAIN:
ENABLE: True
DATASET: ava
BATCH_SIZE: 64
EVAL_PERIOD: 5
CHECKPOINT_PERIOD: 1
AUTO_RESUME: True
# CHECKPOINT_FILE_PATH: path to the pretrain checkpoint file.
CHECKPOINT_TYPE: caffe2
DATA:
NUM_FRAMES: 32
SAMPLING_RATE: 2
TRAIN_JITTER_SCALES: [256, 320]
TRAIN_CROP_SIZE: 224
TEST_CROP_SIZE: 224
INPUT_CHANNEL_NUM: [3, 3]
PATH_TO_DATA_DIR: "/user-data/AVAFile/avaMin"
DETECTION:
ENABLE: True
ALIGNED: True
AVA:
DETECTION_SCORE_THRESH: 0.8
TRAIN_PREDICT_BOX_LISTS: [
"ava_train_v2.2.csv",
"ava_train_predicted_boxes.csv",
]
TEST_PREDICT_BOX_LISTS: ["ava_val_predicted_boxes.csv"]
ANNOTATION_DIR: '/user-data/AVAFile/avaMin/annotations/'
FRAME_DIR: '/user-data/AVAFile/avaMin/frames'
FRAME_LIST_DIR: '/user-data/AVAFile/avaMin/frame_lists/'
SLOWFAST:
ALPHA: 4
BETA_INV: 8
FUSION_CONV_CHANNEL_RATIO: 2
FUSION_KERNEL_SZ: 7
RESNET:
ZERO_INIT_FINAL_BN: True
WIDTH_PER_GROUP: 64
NUM_GROUPS: 1
DEPTH: 50
TRANS_FUNC: bottleneck_transform
STRIDE_1X1: False
NUM_BLOCK_TEMP_KERNEL: [[3, 3], [4, 4], [6, 6], [3, 3]]
SPATIAL_DILATIONS: [[1, 1], [1, 1], [1, 1], [2, 2]]
SPATIAL_STRIDES: [[1, 1], [2, 2], [2, 2], [1, 1]]
NONLOCAL:
LOCATION: [[[], []], [[], []], [[], []], [[], []]]
GROUP: [[1, 1], [1, 1], [1, 1], [1, 1]]
INSTANTIATION: dot_product
POOL: [[[1, 2, 2], [1, 2, 2]], [[1, 2, 2], [1, 2, 2]], [[1, 2, 2], [1, 2, 2]], [[1, 2, 2], [1, 2, 2]]]
BN:
USE_PRECISE_STATS: False
NUM_BATCHES_PRECISE: 200
SOLVER:
BASE_LR: 0.1
LR_POLICY: steps_with_relative_lrs
STEPS: [0, 10, 15, 20]
LRS: [1, 0.1, 0.01, 0.001]
MAX_EPOCH: 20
MOMENTUM: 0.9
WEIGHT_DECAY: 1e-7
WARMUP_EPOCHS: 5.0
WARMUP_START_LR: 0.000125
OPTIMIZING_METHOD: sgd
MODEL:
NUM_CLASSES: 80
ARCH: slowfast
MODEL_NAME: SlowFast
LOSS_FUNC: bce
DROPOUT_RATE: 0.5
HEAD_ACT: sigmoid
TEST:
ENABLE: True
DATASET: ava
BATCH_SIZE: 8
DATA_LOADER:
NUM_WORKERS: 2
PIN_MEMORY: True
NUM_GPUS: 8
NUM_SHARDS: 1
RNG_SEED: 0
OUTPUT_DIR: .
其中的一个问题
/home/slowfast/slowfast/datasets/ava_dataset.py中
做了修改,原因是视频名字前面不知道为什么多了个#,下面的修改就是删除这个#
boxes_and_labels = [
boxes_and_labels[self._video_idx_to_name[i].split('"')[1]]
for i in range(len(self._image_paths))
]
执行训练
cd /home/slowfast
python tools/run_net.py --cfg /home/slowfast/configs/AVA/SLOWFAST_32x2_R50_SHORT.yaml NUM_GPUS 1 TRAIN.BATCH_SIZE 4
训练过程:
(branch2): BottleneckTransform(
(a): Conv3d(128, 32, kernel_size=[3, 1, 1], stride=[1, 1, 1], padding=[1, 0, 0], bias=False)
(a_bn): BatchNorm3d(32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(a_relu): ReLU(inplace=True)
(b): Conv3d(32, 32, kernel_size=[1, 3, 3], stride=[1, 1, 1], padding=[0, 1, 1], dilation=[1, 1, 1], bias=False)
(b_bn): BatchNorm3d(32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(b_relu): ReLU(inplace=True)
(c): Conv3d(32, 128, kernel_size=[1, 1, 1], stride=[1, 1, 1], padding=[0, 0, 0], bias=False)
(c_bn): BatchNorm3d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
(relu): ReLU(inplace=True)
)
(pathway1_res3): ResBlock(
(branch2): BottleneckTransform(
(a): Conv3d(128, 32, kernel_size=[3, 1, 1], stride=[1, 1, 1], padding=[1, 0, 0], bias=False)
(a_bn): BatchNorm3d(32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(a_relu): ReLU(inplace=True)
(b): Conv3d(32, 32, kernel_size=[1, 3, 3], stride=[1, 1, 1], padding=[0, 1, 1], dilation=[1, 1, 1], bias=False)
(b_bn): BatchNorm3d(32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(b_relu): ReLU(inplace=True)
(c): Conv3d(32, 128, kernel_size=[1, 1, 1], stride=[1, 1, 1], padding=[0, 0, 0], bias=False)
(c_bn): BatchNorm3d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
(relu): ReLU(inplace=True)
)
(pathway1_res4): ResBlock(
(branch2): BottleneckTransform(
(a): Conv3d(128, 32, kernel_size=[3, 1, 1], stride=[1, 1, 1], padding=[1, 0, 0], bias=False)
(a_bn): BatchNorm3d(32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(a_relu): ReLU(inplace=True)
(b): Conv3d(32, 32, kernel_size=[1, 3, 3], stride=[1, 1, 1], padding=[0, 1, 1], dilation=[1, 1, 1], bias=False)
(b_bn): BatchNorm3d(32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(b_relu): ReLU(inplace=True)
(c): Conv3d(32, 128, kernel_size=[1, 1, 1], stride=[1, 1, 1], padding=[0, 0, 0], bias=False)
(c_bn): BatchNorm3d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
(relu): ReLU(inplace=True)
)
(pathway1_res5): ResBlock(
(branch2): BottleneckTransform(
(a): Conv3d(128, 32, kernel_size=[3, 1, 1], stride=[1, 1, 1], padding=[1, 0, 0], bias=False)
(a_bn): BatchNorm3d(32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(a_relu): ReLU(inplace=True)
(b): Conv3d(32, 32, kernel_size=[1, 3, 3], stride=[1, 1, 1], padding=[0, 1, 1], dilation=[1, 1, 1], bias=False)
(b_bn): BatchNorm3d(32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(b_relu): ReLU(inplace=True)
(c): Conv3d(32, 128, kernel_size=[1, 1, 1], stride=[1, 1, 1], padding=[0, 0, 0], bias=False)
(c_bn): BatchNorm3d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
(relu): ReLU(inplace=True)
)
)
(s4_fuse): FuseFastToSlow(
(conv_f2s): Conv3d(128, 256, kernel_size=[7, 1, 1], stride=[4, 1, 1], padding=[3, 0, 0], bias=False)
(bn): BatchNorm3d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu): ReLU(inplace=True)
)
(s5): ResStage(
(pathway0_res0): ResBlock(
(branch1): Conv3d(1280, 2048, kernel_size=(1, 1, 1), stride=[1, 1, 1], bias=False)
(branch1_bn): BatchNorm3d(2048, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(branch2): BottleneckTransform(
(a): Conv3d(1280, 512, kernel_size=[3, 1, 1], stride=[1, 1, 1], padding=[1, 0, 0], bias=False)
(a_bn): BatchNorm3d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(a_relu): ReLU(inplace=True)
(b): Conv3d(512, 512, kernel_size=[1, 3, 3], stride=[1, 1, 1], padding=[0, 2, 2], dilation=[1, 2, 2], bias=False)
(b_bn): BatchNorm3d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(b_relu): ReLU(inplace=True)
(c): Conv3d(512, 2048, kernel_size=[1, 1, 1], stride=[1, 1, 1], padding=[0, 0, 0], bias=False)
(c_bn): BatchNorm3d(2048, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
(relu): ReLU(inplace=True)
)
(pathway0_res1): ResBlock(
(branch2): BottleneckTransform(
(a): Conv3d(2048, 512, kernel_size=[3, 1, 1], stride=[1, 1, 1], padding=[1, 0, 0], bias=False)
(a_bn): BatchNorm3d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(a_relu): ReLU(inplace=True)
(b): Conv3d(512, 512, kernel_size=[1, 3, 3], stride=[1, 1, 1], padding=[0, 2, 2], dilation=[1, 2, 2], bias=False)
(b_bn): BatchNorm3d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(b_relu): ReLU(inplace=True)
(c): Conv3d(512, 2048, kernel_size=[1, 1, 1], stride=[1, 1, 1], padding=[0, 0, 0], bias=False)
(c_bn): BatchNorm3d(2048, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
(relu): ReLU(inplace=True)
)
(pathway0_res2): ResBlock(
(branch2): BottleneckTransform(
(a): Conv3d(2048, 512, kernel_size=[3, 1, 1], stride=[1, 1, 1], padding=[1, 0, 0], bias=False)
(a_bn): BatchNorm3d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(a_relu): ReLU(inplace=True)
(b): Conv3d(512, 512, kernel_size=[1, 3, 3], stride=[1, 1, 1], padding=[0, 2, 2], dilation=[1, 2, 2], bias=False)
(b_bn): BatchNorm3d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(b_relu): ReLU(inplace=True)
(c): Conv3d(512, 2048, kernel_size=[1, 1, 1], stride=[1, 1, 1], padding=[0, 0, 0], bias=False)
(c_bn): BatchNorm3d(2048, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
(relu): ReLU(inplace=True)
)
(pathway1_res0): ResBlock(
(branch1): Conv3d(128, 256, kernel_size=(1, 1, 1), stride=[1, 1, 1], bias=False)
(branch1_bn): BatchNorm3d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(branch2): BottleneckTransform(
(a): Conv3d(128, 64, kernel_size=[3, 1, 1], stride=[1, 1, 1], padding=[1, 0, 0], bias=False)
(a_bn): BatchNorm3d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(a_relu): ReLU(inplace=True)
(b): Conv3d(64, 64, kernel_size=[1, 3, 3], stride=[1, 1, 1], padding=[0, 2, 2], dilation=[1, 2, 2], bias=False)
(b_bn): BatchNorm3d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(b_relu): ReLU(inplace=True)
(c): Conv3d(64, 256, kernel_size=[1, 1, 1], stride=[1, 1, 1], padding=[0, 0, 0], bias=False)
(c_bn): BatchNorm3d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
(relu): ReLU(inplace=True)
)
(pathway1_res1): ResBlock(
(branch2): BottleneckTransform(
(a): Conv3d(256, 64, kernel_size=[3, 1, 1], stride=[1, 1, 1], padding=[1, 0, 0], bias=False)
(a_bn): BatchNorm3d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(a_relu): ReLU(inplace=True)
(b): Conv3d(64, 64, kernel_size=[1, 3, 3], stride=[1, 1, 1], padding=[0, 2, 2], dilation=[1, 2, 2], bias=False)
(b_bn): BatchNorm3d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(b_relu): ReLU(inplace=True)
(c): Conv3d(64, 256, kernel_size=[1, 1, 1], stride=[1, 1, 1], padding=[0, 0, 0], bias=False)
(c_bn): BatchNorm3d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
(relu): ReLU(inplace=True)
)
(pathway1_res2): ResBlock(
(branch2): BottleneckTransform(
(a): Conv3d(256, 64, kernel_size=[3, 1, 1], stride=[1, 1, 1], padding=[1, 0, 0], bias=False)
(a_bn): BatchNorm3d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(a_relu): ReLU(inplace=True)
(b): Conv3d(64, 64, kernel_size=[1, 3, 3], stride=[1, 1, 1], padding=[0, 2, 2], dilation=[1, 2, 2], bias=False)
(b_bn): BatchNorm3d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(b_relu): ReLU(inplace=True)
(c): Conv3d(64, 256, kernel_size=[1, 1, 1], stride=[1, 1, 1], padding=[0, 0, 0], bias=False)
(c_bn): BatchNorm3d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
(relu): ReLU(inplace=True)
)
)
(head): ResNetRoIHead(
(s0_tpool): AvgPool3d(kernel_size=[8, 1, 1], stride=1, padding=0)
(s0_roi): ROIAlign(output_size=[7, 7], spatial_scale=0.0625, sampling_ratio=0, aligned=True)
(s0_spool): MaxPool2d(kernel_size=[7, 7], stride=1, padding=0, dilation=1, ceil_mode=False)
(s1_tpool): AvgPool3d(kernel_size=[32, 1, 1], stride=1, padding=0)
(s1_roi): ROIAlign(output_size=[7, 7], spatial_scale=0.0625, sampling_ratio=0, aligned=True)
(s1_spool): MaxPool2d(kernel_size=[7, 7], stride=1, padding=0, dilation=1, ceil_mode=False)
(dropout): Dropout(p=0.5, inplace=False)
(projection): Linear(in_features=2304, out_features=80, bias=True)
(act): Sigmoid()
)
)
[05/16 00:21:23][INFO] misc.py: 184: Params: 33,828,888
[05/16 00:21:23][INFO] misc.py: 185: Mem: 0.12698125839233398 MB
[05/16 00:21:24][WARNING] jit_analysis.py: 499: Unsupported operator aten::max_pool3d encountered 4 time(s)
[05/16 00:21:24][WARNING] jit_analysis.py: 499: Unsupported operator aten::add encountered 32 time(s)
[05/16 00:21:24][WARNING] jit_analysis.py: 499: Unsupported operator aten::avg_pool3d encountered 2 time(s)
[05/16 00:21:24][WARNING] jit_analysis.py: 499: Unsupported operator torchvision::roi_align encountered 2 time(s)
[05/16 00:21:24][WARNING] jit_analysis.py: 499: Unsupported operator aten::max_pool2d encountered 2 time(s)
[05/16 00:21:24][WARNING] jit_analysis.py: 499: Unsupported operator aten::sigmoid encountered 1 time(s)
[05/16 00:21:24][INFO] misc.py: 186: Flops: 74.49147392 G
[05/16 00:21:25][WARNING] jit_analysis.py: 499: Unsupported operator aten::batch_norm encountered 110 time(s)
[05/16 00:21:25][WARNING] jit_analysis.py: 499: Unsupported operator aten::max_pool3d encountered 4 time(s)
[05/16 00:21:25][WARNING] jit_analysis.py: 499: Unsupported operator aten::add encountered 32 time(s)
[05/16 00:21:25][WARNING] jit_analysis.py: 499: Unsupported operator aten::avg_pool3d encountered 2 time(s)
[05/16 00:21:25][WARNING] jit_analysis.py: 499: Unsupported operator torchvision::roi_align encountered 2 time(s)
[05/16 00:21:25][WARNING] jit_analysis.py: 499: Unsupported operator aten::max_pool2d encountered 2 time(s)
[05/16 00:21:25][WARNING] jit_analysis.py: 499: Unsupported operator aten::sigmoid encountered 1 time(s)
[05/16 00:21:25][INFO] misc.py: 191: Activations: 155.54568 M
[05/16 00:21:25][INFO] misc.py: 196: nvidia-smi
Mon May 16 00:21:25 2022
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 510.47.03 Driver Version: 510.47.03 CUDA Version: 11.6 |
|-------------------------------+----------------------+----------------------+
| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|===============================+======================+======================|
| 0 NVIDIA GeForce ... On | 00000000:8A:00.0 Off | N/A |
| 48% 31C P2 89W / 320W | 3006MiB / 10240MiB | 0% Default |
| | | N/A |
+-------------------------------+----------------------+----------------------+
+-----------------------------------------------------------------------------+
| Processes: |
| GPU GI CI PID Type Process name GPU Memory |
| ID ID Usage |
|=============================================================================|
| 0 N/A N/A 784711 C 3003MiB |
+-----------------------------------------------------------------------------+
bn 220, non bn 112, zero 0 no grad 0
[05/16 00:21:25][INFO] ava_helper.py: 61: Finished loading image paths from: /user-data/AVAFile/avaMin/frame_lists/train.csv
[05/16 00:21:25][INFO] ava_helper.py: 106: Finished loading annotations from: /user-data/AVAFile/avaMin/annotations/ava_train_v2.2.csv, /user-data/AVAFile/avaMin/annotations/ava_train_v2.2.csv, /user-data/AVAFile/avaMin/annotations/ava_train_predicted_boxes.csv
[05/16 00:21:25][INFO] ava_helper.py: 109: Detection threshold: 0.8
[05/16 00:21:25][INFO] ava_helper.py: 110: Number of unique boxes: 7205
[05/16 00:21:25][INFO] ava_helper.py: 111: Number of annotations: 25769
[05/16 00:21:25][INFO] ava_helper.py: 157: 1746 keyframes used.
[05/16 00:21:25][INFO] ava_dataset.py: 96: === AVA dataset summary ===
[05/16 00:21:25][INFO] ava_dataset.py: 97: Split: train
[05/16 00:21:25][INFO] ava_dataset.py: 98: Number of videos: 2
[05/16 00:21:25][INFO] ava_dataset.py: 102: Number of frames: 54060
[05/16 00:21:25][INFO] ava_dataset.py: 103: Number of key frames: 1746
[05/16 00:21:25][INFO] ava_dataset.py: 104: Number of boxes: 7205.
[05/16 00:21:25][INFO] ava_helper.py: 61: Finished loading image paths from: /user-data/AVAFile/avaMin/frame_lists/val.csv
[05/16 00:21:25][INFO] ava_helper.py: 106: Finished loading annotations from: /user-data/AVAFile/avaMin/annotations/ava_val_predicted_boxes.csv
[05/16 00:21:25][INFO] ava_helper.py: 109: Detection threshold: 0.8
[05/16 00:21:25][INFO] ava_helper.py: 110: Number of unique boxes: 921
[05/16 00:21:25][INFO] ava_helper.py: 111: Number of annotations: 1942
[05/16 00:21:25][INFO] ava_helper.py: 157: 437 keyframes used.
[05/16 00:21:25][INFO] ava_dataset.py: 96: === AVA dataset summary ===
[05/16 00:21:25][INFO] ava_dataset.py: 97: Split: val
[05/16 00:21:25][INFO] ava_dataset.py: 98: Number of videos: 2
[05/16 00:21:25][INFO] ava_dataset.py: 102: Number of frames: 54060
[05/16 00:21:25][INFO] ava_dataset.py: 103: Number of key frames: 437
[05/16 00:21:25][INFO] ava_dataset.py: 104: Number of boxes: 921.
[05/16 00:21:26][INFO] ava_helper.py: 61: Finished loading image paths from: /user-data/AVAFile/avaMin/frame_lists/train.csv
[05/16 00:21:26][INFO] ava_helper.py: 61: Finished loading image paths from: /user-data/AVAFile/avaMin/frame_lists/val.csv
[05/16 00:21:26][INFO] train_net.py: 454: Start epoch: 1
[05/16 00:21:31][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "1", "cur_iter": "10", "dt": 0.39153, "dt_data": 0.00812, "dt_net": 0.38342, "eta": "0:02:47", "loss": 0.74648, "lr": 0.00054, "mode": "train"}
[05/16 00:21:35][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "1", "cur_iter": "20", "dt": 0.39193, "dt_data": 0.00822, "dt_net": 0.38371, "eta": "0:02:43", "loss": 0.52791, "lr": 0.00100, "mode": "train"}
[05/16 00:21:38][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "1", "cur_iter": "30", "dt": 0.39409, "dt_data": 0.00815, "dt_net": 0.38594, "eta": "0:02:40", "loss": 0.34277, "lr": 0.00145, "mode": "train"}
[05/16 00:21:42][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "1", "cur_iter": "40", "dt": 0.39437, "dt_data": 0.00826, "dt_net": 0.38611, "eta": "0:02:36", "loss": 0.20848, "lr": 0.00191, "mode": "train"}
[05/16 00:21:46][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "1", "cur_iter": "50", "dt": 0.39334, "dt_data": 0.00810, "dt_net": 0.38524, "eta": "0:02:32", "loss": 0.19194, "lr": 0.00237, "mode": "train"}
[05/16 00:21:50][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "1", "cur_iter": "60", "dt": 0.39185, "dt_data": 0.00815, "dt_net": 0.38370, "eta": "0:02:27", "loss": 0.21235, "lr": 0.00283, "mode": "train"}
[05/16 00:21:54][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "1", "cur_iter": "70", "dt": 0.39272, "dt_data": 0.00819, "dt_net": 0.38453, "eta": "0:02:24", "loss": 0.14129, "lr": 0.00329, "mode": "train"}
[05/16 00:21:58][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "1", "cur_iter": "80", "dt": 0.39764, "dt_data": 0.00823, "dt_net": 0.38940, "eta": "0:02:21", "loss": 0.14686, "lr": 0.00374, "mode": "train"}
[05/16 00:22:02][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "1", "cur_iter": "90", "dt": 0.39202, "dt_data": 0.00816, "dt_net": 0.38386, "eta": "0:02:16", "loss": 0.09108, "lr": 0.00420, "mode": "train"}
[05/16 00:22:06][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "1", "cur_iter": "100", "dt": 0.39299, "dt_data": 0.00811, "dt_net": 0.38488, "eta": "0:02:12", "loss": 0.10285, "lr": 0.00466, "mode": "train"}
[05/16 00:22:10][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "1", "cur_iter": "110", "dt": 0.39235, "dt_data": 0.00811, "dt_net": 0.38423, "eta": "0:02:08", "loss": 0.16798, "lr": 0.00512, "mode": "train"}
[05/16 00:22:14][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "1", "cur_iter": "120", "dt": 0.39329, "dt_data": 0.00811, "dt_net": 0.38517, "eta": "0:02:04", "loss": 0.12485, "lr": 0.00558, "mode": "train"}
[05/16 00:22:18][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "1", "cur_iter": "130", "dt": 0.39428, "dt_data": 0.00811, "dt_net": 0.38616, "eta": "0:02:01", "loss": 0.15235, "lr": 0.00604, "mode": "train"}
[05/16 00:22:22][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "1", "cur_iter": "140", "dt": 0.39547, "dt_data": 0.00823, "dt_net": 0.38724, "eta": "0:01:57", "loss": 0.09438, "lr": 0.00649, "mode": "train"}
[05/16 00:22:26][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "1", "cur_iter": "150", "dt": 0.39538, "dt_data": 0.00810, "dt_net": 0.38728, "eta": "0:01:53", "loss": 0.11352, "lr": 0.00695, "mode": "train"}
[05/16 00:22:30][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "1", "cur_iter": "160", "dt": 0.39346, "dt_data": 0.00810, "dt_net": 0.38537, "eta": "0:01:48", "loss": 0.11081, "lr": 0.00741, "mode": "train"}
[05/16 00:22:34][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "1", "cur_iter": "170", "dt": 0.39401, "dt_data": 0.00811, "dt_net": 0.38590, "eta": "0:01:45", "loss": 0.07131, "lr": 0.00787, "mode": "train"}
[05/16 00:22:38][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "1", "cur_iter": "180", "dt": 0.39547, "dt_data": 0.00811, "dt_net": 0.38736, "eta": "0:01:41", "loss": 0.14390, "lr": 0.00833, "mode": "train"}
[05/16 00:22:42][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "1", "cur_iter": "190", "dt": 0.39579, "dt_data": 0.00810, "dt_net": 0.38769, "eta": "0:01:37", "loss": 0.12767, "lr": 0.00878, "mode": "train"}
[05/16 00:22:46][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "1", "cur_iter": "200", "dt": 0.39618, "dt_data": 0.00822, "dt_net": 0.38796, "eta": "0:01:33", "loss": 0.12724, "lr": 0.00924, "mode": "train"}
[05/16 00:22:50][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "1", "cur_iter": "210", "dt": 0.39533, "dt_data": 0.00810, "dt_net": 0.38723, "eta": "0:01:29", "loss": 0.07577, "lr": 0.00970, "mode": "train"}
[05/16 00:22:54][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "1", "cur_iter": "220", "dt": 0.39673, "dt_data": 0.00815, "dt_net": 0.38858, "eta": "0:01:26", "loss": 0.09678, "lr": 0.01016, "mode": "train"}
[05/16 00:22:58][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "1", "cur_iter": "230", "dt": 0.39577, "dt_data": 0.00811, "dt_net": 0.38766, "eta": "0:01:21", "loss": 0.08184, "lr": 0.01062, "mode": "train"}
[05/16 00:23:02][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "1", "cur_iter": "240", "dt": 0.39573, "dt_data": 0.00810, "dt_net": 0.38763, "eta": "0:01:17", "loss": 0.12548, "lr": 0.01107, "mode": "train"}
[05/16 00:23:06][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "1", "cur_iter": "250", "dt": 0.39658, "dt_data": 0.00815, "dt_net": 0.38843, "eta": "0:01:14", "loss": 0.11235, "lr": 0.01153, "mode": "train"}
[05/16 00:23:09][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "1", "cur_iter": "260", "dt": 0.39656, "dt_data": 0.00810, "dt_net": 0.38846, "eta": "0:01:10", "loss": 0.15453, "lr": 0.01199, "mode": "train"}
[05/16 00:23:13][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "1", "cur_iter": "270", "dt": 0.39363, "dt_data": 0.00809, "dt_net": 0.38554, "eta": "0:01:05", "loss": 0.12279, "lr": 0.01245, "mode": "train"}
[05/16 00:23:17][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "1", "cur_iter": "280", "dt": 0.39433, "dt_data": 0.00815, "dt_net": 0.38617, "eta": "0:01:01", "loss": 0.10631, "lr": 0.01291, "mode": "train"}
[05/16 00:23:21][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "1", "cur_iter": "290", "dt": 0.39533, "dt_data": 0.00822, "dt_net": 0.38711, "eta": "0:00:58", "loss": 0.11734, "lr": 0.01337, "mode": "train"}
[05/16 00:23:25][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "1", "cur_iter": "300", "dt": 0.39581, "dt_data": 0.00810, "dt_net": 0.38771, "eta": "0:00:54", "loss": 0.11052, "lr": 0.01382, "mode": "train"}
[05/16 00:23:29][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "1", "cur_iter": "310", "dt": 0.39588, "dt_data": 0.00809, "dt_net": 0.38779, "eta": "0:00:50", "loss": 0.14302, "lr": 0.01428, "mode": "train"}
[05/16 00:23:33][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "1", "cur_iter": "320", "dt": 0.39433, "dt_data": 0.00810, "dt_net": 0.38624, "eta": "0:00:46", "loss": 0.12586, "lr": 0.01474, "mode": "train"}
[05/16 00:23:37][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "1", "cur_iter": "330", "dt": 0.39494, "dt_data": 0.00810, "dt_net": 0.38684, "eta": "0:00:42", "loss": 0.07875, "lr": 0.01520, "mode": "train"}
[05/16 00:23:41][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "1", "cur_iter": "340", "dt": 0.39580, "dt_data": 0.00810, "dt_net": 0.38770, "eta": "0:00:38", "loss": 0.13401, "lr": 0.01566, "mode": "train"}
[05/16 00:23:45][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "1", "cur_iter": "350", "dt": 0.39564, "dt_data": 0.00814, "dt_net": 0.38750, "eta": "0:00:34", "loss": 0.11493, "lr": 0.01611, "mode": "train"}
[05/16 00:23:49][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "1", "cur_iter": "360", "dt": 0.39680, "dt_data": 0.00808, "dt_net": 0.38872, "eta": "0:00:30", "loss": 0.11246, "lr": 0.01657, "mode": "train"}
[05/16 00:23:53][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "1", "cur_iter": "370", "dt": 0.39631, "dt_data": 0.00813, "dt_net": 0.38818, "eta": "0:00:26", "loss": 0.15141, "lr": 0.01703, "mode": "train"}
[05/16 00:23:57][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "1", "cur_iter": "380", "dt": 0.39749, "dt_data": 0.00813, "dt_net": 0.38936, "eta": "0:00:22", "loss": 0.11167, "lr": 0.01749, "mode": "train"}
[05/16 00:24:01][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "1", "cur_iter": "390", "dt": 0.39705, "dt_data": 0.00814, "dt_net": 0.38891, "eta": "0:00:18", "loss": 0.11514, "lr": 0.01795, "mode": "train"}
[05/16 00:24:05][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "1", "cur_iter": "400", "dt": 0.39534, "dt_data": 0.00810, "dt_net": 0.38724, "eta": "0:00:14", "loss": 0.11790, "lr": 0.01840, "mode": "train"}
[05/16 00:24:09][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "1", "cur_iter": "410", "dt": 0.39594, "dt_data": 0.00809, "dt_net": 0.38785, "eta": "0:00:10", "loss": 0.08583, "lr": 0.01886, "mode": "train"}
[05/16 00:24:13][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "1", "cur_iter": "420", "dt": 0.39695, "dt_data": 0.00813, "dt_net": 0.38881, "eta": "0:00:06", "loss": 0.07596, "lr": 0.01932, "mode": "train"}
[05/16 00:24:17][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "1", "cur_iter": "430", "dt": 0.39553, "dt_data": 0.00815, "dt_net": 0.38737, "eta": "0:00:02", "loss": 0.09653, "lr": 0.01978, "mode": "train"}
[05/16 00:24:19][INFO] train_net.py: 491: Epoch 0 takes 173.53s. Epochs from 0 to 0 take 173.53s in average and 173.53s in median.
[05/16 00:24:19][INFO] train_net.py: 497: For epoch 0, each iteraction takes 0.40s in average. From epoch 0 to 0, each iteraction takes 0.40s in average.
[05/16 00:24:24][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "2", "cur_iter": "10", "dt": 0.39712, "dt_data": 0.00805, "dt_net": 0.38907, "eta": "0:02:49", "loss": 0.09964, "lr": 0.02051, "mode": "train"}
[05/16 00:24:28][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "2", "cur_iter": "20", "dt": 0.39923, "dt_data": 0.00809, "dt_net": 0.39115, "eta": "0:02:46", "loss": 0.12694, "lr": 0.02097, "mode": "train"}
[05/16 00:24:32][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "2", "cur_iter": "30", "dt": 0.40214, "dt_data": 0.00805, "dt_net": 0.39410, "eta": "0:02:43", "loss": 0.07189, "lr": 0.02143, "mode": "train"}
[05/16 00:24:36][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "2", "cur_iter": "40", "dt": 0.39849, "dt_data": 0.00819, "dt_net": 0.39030, "eta": "0:02:38", "loss": 0.11717, "lr": 0.02189, "mode": "train"}
[05/16 00:24:40][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "2", "cur_iter": "50", "dt": 0.39507, "dt_data": 0.00804, "dt_net": 0.38703, "eta": "0:02:32", "loss": 0.08270, "lr": 0.02234, "mode": "train"}
[05/16 00:24:44][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "2", "cur_iter": "60", "dt": 0.39679, "dt_data": 0.00820, "dt_net": 0.38859, "eta": "0:02:29", "loss": 0.09391, "lr": 0.02280, "mode": "train"}
[05/16 00:24:48][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "2", "cur_iter": "70", "dt": 0.39608, "dt_data": 0.00803, "dt_net": 0.38805, "eta": "0:02:25", "loss": 0.10680, "lr": 0.02326, "mode": "train"}
[05/16 00:24:52][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "2", "cur_iter": "80", "dt": 0.39712, "dt_data": 0.00803, "dt_net": 0.38909, "eta": "0:02:21", "loss": 0.09237, "lr": 0.02372, "mode": "train"}
[05/16 00:24:56][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "2", "cur_iter": "90", "dt": 0.39626, "dt_data": 0.00815, "dt_net": 0.38810, "eta": "0:02:17", "loss": 0.09713, "lr": 0.02418, "mode": "train"}
[05/16 00:25:00][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "2", "cur_iter": "100", "dt": 0.39707, "dt_data": 0.00814, "dt_net": 0.38893, "eta": "0:02:13", "loss": 0.12110, "lr": 0.02464, "mode": "train"}
[05/16 00:25:04][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "2", "cur_iter": "110", "dt": 0.39747, "dt_data": 0.00819, "dt_net": 0.38927, "eta": "0:02:09", "loss": 0.13248, "lr": 0.02509, "mode": "train"}
[05/16 00:25:08][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "2", "cur_iter": "120", "dt": 0.39843, "dt_data": 0.00819, "dt_net": 0.39024, "eta": "0:02:06", "loss": 0.12487, "lr": 0.02555, "mode": "train"}
[05/16 00:25:12][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "2", "cur_iter": "130", "dt": 0.39713, "dt_data": 0.00813, "dt_net": 0.38900, "eta": "0:02:01", "loss": 0.11716, "lr": 0.02601, "mode": "train"}
[05/16 00:25:16][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "2", "cur_iter": "140", "dt": 0.39746, "dt_data": 0.00817, "dt_net": 0.38929, "eta": "0:01:58", "loss": 0.16843, "lr": 0.02647, "mode": "train"}
[05/16 00:25:20][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "2", "cur_iter": "150", "dt": 0.39757, "dt_data": 0.00814, "dt_net": 0.38943, "eta": "0:01:54", "loss": 0.07765, "lr": 0.02693, "mode": "train"}
[05/16 00:25:24][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "2", "cur_iter": "160", "dt": 0.39767, "dt_data": 0.00818, "dt_net": 0.38948, "eta": "0:01:50", "loss": 0.14891, "lr": 0.02738, "mode": "train"}
[05/16 00:25:28][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "2", "cur_iter": "170", "dt": 0.39872, "dt_data": 0.00814, "dt_net": 0.39058, "eta": "0:01:46", "loss": 0.14942, "lr": 0.02784, "mode": "train"}
[05/16 00:25:32][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "2", "cur_iter": "180", "dt": 0.39874, "dt_data": 0.00817, "dt_net": 0.39058, "eta": "0:01:42", "loss": 0.12630, "lr": 0.02830, "mode": "train"}
[05/16 00:25:36][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "2", "cur_iter": "190", "dt": 0.39759, "dt_data": 0.00816, "dt_net": 0.38942, "eta": "0:01:38", "loss": 0.07504, "lr": 0.02876, "mode": "train"}
[05/16 00:25:40][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "2", "cur_iter": "200", "dt": 0.39749, "dt_data": 0.00821, "dt_net": 0.38929, "eta": "0:01:34", "loss": 0.07854, "lr": 0.02922, "mode": "train"}
[05/16 00:25:44][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "2", "cur_iter": "210", "dt": 0.39806, "dt_data": 0.00820, "dt_net": 0.38985, "eta": "0:01:30", "loss": 0.12322, "lr": 0.02968, "mode": "train"}
[05/16 00:25:48][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "2", "cur_iter": "220", "dt": 0.39627, "dt_data": 0.00813, "dt_net": 0.38814, "eta": "0:01:25", "loss": 0.08739, "lr": 0.03013, "mode": "train"}
[05/16 00:25:52][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "2", "cur_iter": "230", "dt": 0.39605, "dt_data": 0.00814, "dt_net": 0.38790, "eta": "0:01:21", "loss": 0.16364, "lr": 0.03059, "mode": "train"}
[05/16 00:25:56][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "2", "cur_iter": "240", "dt": 0.39714, "dt_data": 0.00813, "dt_net": 0.38900, "eta": "0:01:18", "loss": 0.09062, "lr": 0.03105, "mode": "train"}
[05/16 00:26:00][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "2", "cur_iter": "250", "dt": 0.39858, "dt_data": 0.00815, "dt_net": 0.39043, "eta": "0:01:14", "loss": 0.09697, "lr": 0.03151, "mode": "train"}
[05/16 00:26:04][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "2", "cur_iter": "260", "dt": 0.39635, "dt_data": 0.00816, "dt_net": 0.38819, "eta": "0:01:10", "loss": 0.08331, "lr": 0.03197, "mode": "train"}
[05/16 00:26:08][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "2", "cur_iter": "270", "dt": 0.39697, "dt_data": 0.00823, "dt_net": 0.38874, "eta": "0:01:06", "loss": 0.08481, "lr": 0.03242, "mode": "train"}
[05/16 00:26:12][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "2", "cur_iter": "280", "dt": 0.39521, "dt_data": 0.00815, "dt_net": 0.38706, "eta": "0:01:02", "loss": 0.12140, "lr": 0.03288, "mode": "train"}
[05/16 00:26:16][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "2", "cur_iter": "290", "dt": 0.39819, "dt_data": 0.00820, "dt_net": 0.38999, "eta": "0:00:58", "loss": 0.07843, "lr": 0.03334, "mode": "train"}
[05/16 00:26:20][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "2", "cur_iter": "300", "dt": 0.39762, "dt_data": 0.00816, "dt_net": 0.38945, "eta": "0:00:54", "loss": 0.08493, "lr": 0.03380, "mode": "train"}
[05/16 00:26:24][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "2", "cur_iter": "310", "dt": 0.39843, "dt_data": 0.00813, "dt_net": 0.39030, "eta": "0:00:50", "loss": 0.09250, "lr": 0.03426, "mode": "train"}
[05/16 00:26:28][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "2", "cur_iter": "320", "dt": 0.39543, "dt_data": 0.00813, "dt_net": 0.38730, "eta": "0:00:46", "loss": 0.07979, "lr": 0.03471, "mode": "train"}
[05/16 00:26:32][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "2", "cur_iter": "330", "dt": 0.39710, "dt_data": 0.00820, "dt_net": 0.38890, "eta": "0:00:42", "loss": 0.11884, "lr": 0.03517, "mode": "train"}
[05/16 00:26:36][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "2", "cur_iter": "340", "dt": 0.39709, "dt_data": 0.00820, "dt_net": 0.38889, "eta": "0:00:38", "loss": 0.07862, "lr": 0.03563, "mode": "train"}
[05/16 00:26:40][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "2", "cur_iter": "350", "dt": 0.39581, "dt_data": 0.00822, "dt_net": 0.38758, "eta": "0:00:34", "loss": 0.11715, "lr": 0.03609, "mode": "train"}
[05/16 00:26:44][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "2", "cur_iter": "360", "dt": 0.39827, "dt_data": 0.00820, "dt_net": 0.39006, "eta": "0:00:30", "loss": 0.09300, "lr": 0.03655, "mode": "train"}
[05/16 00:26:47][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "2", "cur_iter": "370", "dt": 0.39671, "dt_data": 0.00812, "dt_net": 0.38859, "eta": "0:00:26", "loss": 0.17362, "lr": 0.03701, "mode": "train"}
[05/16 00:26:51][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "2", "cur_iter": "380", "dt": 0.39889, "dt_data": 0.00812, "dt_net": 0.39076, "eta": "0:00:22", "loss": 0.10611, "lr": 0.03746, "mode": "train"}
[05/16 00:26:55][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "2", "cur_iter": "390", "dt": 0.39750, "dt_data": 0.00823, "dt_net": 0.38928, "eta": "0:00:18", "loss": 0.07992, "lr": 0.03792, "mode": "train"}
[05/16 00:26:59][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "2", "cur_iter": "400", "dt": 0.39857, "dt_data": 0.00819, "dt_net": 0.39038, "eta": "0:00:14", "loss": 0.08014, "lr": 0.03838, "mode": "train"}
[05/16 00:27:03][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "2", "cur_iter": "410", "dt": 0.39539, "dt_data": 0.00813, "dt_net": 0.38726, "eta": "0:00:10", "loss": 0.09780, "lr": 0.03884, "mode": "train"}
[05/16 00:27:07][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "2", "cur_iter": "420", "dt": 0.39709, "dt_data": 0.00814, "dt_net": 0.38895, "eta": "0:00:06", "loss": 0.10291, "lr": 0.03930, "mode": "train"}
[05/16 00:27:11][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "2", "cur_iter": "430", "dt": 0.39884, "dt_data": 0.00815, "dt_net": 0.39069, "eta": "0:00:02", "loss": 0.12154, "lr": 0.03975, "mode": "train"}
[05/16 00:27:14][INFO] train_net.py: 491: Epoch 1 takes 174.13s. Epochs from 0 to 1 take 173.83s in average and 173.83s in median.
[05/16 00:27:14][INFO] train_net.py: 497: For epoch 1, each iteraction takes 0.40s in average. From epoch 0 to 1, each iteraction takes 0.40s in average.
[05/16 00:27:19][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "3", "cur_iter": "10", "dt": 0.39797, "dt_data": 0.00810, "dt_net": 0.38987, "eta": "0:02:49", "loss": 0.08309, "lr": 0.04049, "mode": "train"}
[05/16 00:27:23][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "3", "cur_iter": "20", "dt": 0.39776, "dt_data": 0.00815, "dt_net": 0.38961, "eta": "0:02:45", "loss": 0.11614, "lr": 0.04095, "mode": "train"}
[05/16 00:27:27][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "3", "cur_iter": "30", "dt": 0.39864, "dt_data": 0.00812, "dt_net": 0.39052, "eta": "0:02:42", "loss": 0.10514, "lr": 0.04140, "mode": "train"}
[05/16 00:27:31][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "3", "cur_iter": "40", "dt": 0.39827, "dt_data": 0.00815, "dt_net": 0.39013, "eta": "0:02:38", "loss": 0.09968, "lr": 0.04186, "mode": "train"}
[05/16 00:27:35][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "3", "cur_iter": "50", "dt": 0.39724, "dt_data": 0.00810, "dt_net": 0.38915, "eta": "0:02:33", "loss": 0.08940, "lr": 0.04232, "mode": "train"}
[05/16 00:27:39][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "3", "cur_iter": "60", "dt": 0.39786, "dt_data": 0.00815, "dt_net": 0.38971, "eta": "0:02:29", "loss": 0.11506, "lr": 0.04278, "mode": "train"}
[05/16 00:27:43][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "3", "cur_iter": "70", "dt": 0.39734, "dt_data": 0.00810, "dt_net": 0.38924, "eta": "0:02:25", "loss": 0.13698, "lr": 0.04324, "mode": "train"}
[05/16 00:27:47][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "3", "cur_iter": "80", "dt": 0.39838, "dt_data": 0.00808, "dt_net": 0.39031, "eta": "0:02:22", "loss": 0.09112, "lr": 0.04369, "mode": "train"}
[05/16 00:27:51][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "3", "cur_iter": "90", "dt": 0.39762, "dt_data": 0.00811, "dt_net": 0.38951, "eta": "0:02:17", "loss": 0.11766, "lr": 0.04415, "mode": "train"}
[05/16 00:27:55][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "3", "cur_iter": "100", "dt": 0.39709, "dt_data": 0.00809, "dt_net": 0.38899, "eta": "0:02:13", "loss": 0.12587, "lr": 0.04461, "mode": "train"}
[05/16 00:27:59][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "3", "cur_iter": "110", "dt": 0.39724, "dt_data": 0.00809, "dt_net": 0.38915, "eta": "0:02:09", "loss": 0.13900, "lr": 0.04507, "mode": "train"}
[05/16 00:28:03][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "3", "cur_iter": "120", "dt": 0.39652, "dt_data": 0.00807, "dt_net": 0.38845, "eta": "0:02:05", "loss": 0.10009, "lr": 0.04553, "mode": "train"}
[05/16 00:28:07][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "3", "cur_iter": "130", "dt": 0.39664, "dt_data": 0.00810, "dt_net": 0.38854, "eta": "0:02:01", "loss": 0.11472, "lr": 0.04599, "mode": "train"}
[05/16 00:28:11][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "3", "cur_iter": "140", "dt": 0.39599, "dt_data": 0.00803, "dt_net": 0.38796, "eta": "0:01:57", "loss": 0.11197, "lr": 0.04644, "mode": "train"}
[05/16 00:28:15][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "3", "cur_iter": "150", "dt": 0.39751, "dt_data": 0.00812, "dt_net": 0.38940, "eta": "0:01:54", "loss": 0.14627, "lr": 0.04690, "mode": "train"}
[05/16 00:28:19][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "3", "cur_iter": "160", "dt": 0.39712, "dt_data": 0.00810, "dt_net": 0.38902, "eta": "0:01:50", "loss": 0.11409, "lr": 0.04736, "mode": "train"}
[05/16 00:28:23][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "3", "cur_iter": "170", "dt": 0.39702, "dt_data": 0.00810, "dt_net": 0.38891, "eta": "0:01:46", "loss": 0.11508, "lr": 0.04782, "mode": "train"}
[05/16 00:28:27][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "3", "cur_iter": "180", "dt": 0.39524, "dt_data": 0.00807, "dt_net": 0.38717, "eta": "0:01:41", "loss": 0.06777, "lr": 0.04828, "mode": "train"}
[05/16 00:28:31][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "3", "cur_iter": "190", "dt": 0.39819, "dt_data": 0.00820, "dt_net": 0.38999, "eta": "0:01:38", "loss": 0.09786, "lr": 0.04873, "mode": "train"}
[05/16 00:28:35][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "3", "cur_iter": "200", "dt": 0.39770, "dt_data": 0.00811, "dt_net": 0.38959, "eta": "0:01:34", "loss": 0.10516, "lr": 0.04919, "mode": "train"}
[05/16 00:28:39][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "3", "cur_iter": "210", "dt": 0.39706, "dt_data": 0.00812, "dt_net": 0.38894, "eta": "0:01:30", "loss": 0.12169, "lr": 0.04965, "mode": "train"}
[05/16 00:28:43][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "3", "cur_iter": "220", "dt": 0.39512, "dt_data": 0.00809, "dt_net": 0.38702, "eta": "0:01:25", "loss": 0.08650, "lr": 0.05011, "mode": "train"}
[05/16 00:28:47][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "3", "cur_iter": "230", "dt": 0.39722, "dt_data": 0.00808, "dt_net": 0.38914, "eta": "0:01:22", "loss": 0.10509, "lr": 0.05057, "mode": "train"}
[05/16 00:28:51][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "3", "cur_iter": "240", "dt": 0.39718, "dt_data": 0.00808, "dt_net": 0.38910, "eta": "0:01:18", "loss": 0.10580, "lr": 0.05102, "mode": "train"}
[05/16 00:28:54][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "3", "cur_iter": "250", "dt": 0.39767, "dt_data": 0.00812, "dt_net": 0.38956, "eta": "0:01:14", "loss": 0.10936, "lr": 0.05148, "mode": "train"}
[05/16 00:28:58][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "3", "cur_iter": "260", "dt": 0.39592, "dt_data": 0.00815, "dt_net": 0.38777, "eta": "0:01:10", "loss": 0.15853, "lr": 0.05194, "mode": "train"}
[05/16 00:29:02][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "3", "cur_iter": "270", "dt": 0.39546, "dt_data": 0.00813, "dt_net": 0.38733, "eta": "0:01:06", "loss": 0.09646, "lr": 0.05240, "mode": "train"}
[05/16 00:29:06][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "3", "cur_iter": "280", "dt": 0.39665, "dt_data": 0.00814, "dt_net": 0.38850, "eta": "0:01:02", "loss": 0.11539, "lr": 0.05286, "mode": "train"}
[05/16 00:29:10][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "3", "cur_iter": "290", "dt": 0.39781, "dt_data": 0.00806, "dt_net": 0.38975, "eta": "0:00:58", "loss": 0.10197, "lr": 0.05332, "mode": "train"}
[05/16 00:29:14][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "3", "cur_iter": "300", "dt": 0.39849, "dt_data": 0.00814, "dt_net": 0.39035, "eta": "0:00:54", "loss": 0.12025, "lr": 0.05377, "mode": "train"}
[05/16 00:29:18][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "3", "cur_iter": "310", "dt": 0.39516, "dt_data": 0.00810, "dt_net": 0.38706, "eta": "0:00:50", "loss": 0.11292, "lr": 0.05423, "mode": "train"}
[05/16 00:29:22][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "3", "cur_iter": "320", "dt": 0.39741, "dt_data": 0.00803, "dt_net": 0.38939, "eta": "0:00:46", "loss": 0.08930, "lr": 0.05469, "mode": "train"}
[05/16 00:29:26][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "3", "cur_iter": "330", "dt": 0.39679, "dt_data": 0.00809, "dt_net": 0.38869, "eta": "0:00:42", "loss": 0.06916, "lr": 0.05515, "mode": "train"}
[05/16 00:29:30][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "3", "cur_iter": "340", "dt": 0.39735, "dt_data": 0.00809, "dt_net": 0.38926, "eta": "0:00:38", "loss": 0.14280, "lr": 0.05561, "mode": "train"}
[05/16 00:29:34][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "3", "cur_iter": "350", "dt": 0.39632, "dt_data": 0.00814, "dt_net": 0.38818, "eta": "0:00:34", "loss": 0.10199, "lr": 0.05606, "mode": "train"}
[05/16 00:29:38][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "3", "cur_iter": "360", "dt": 0.39697, "dt_data": 0.00806, "dt_net": 0.38890, "eta": "0:00:30", "loss": 0.14648, "lr": 0.05652, "mode": "train"}
[05/16 00:29:42][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "3", "cur_iter": "370", "dt": 0.39634, "dt_data": 0.00807, "dt_net": 0.38826, "eta": "0:00:26", "loss": 0.07169, "lr": 0.05698, "mode": "train"}
[05/16 00:29:46][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "3", "cur_iter": "380", "dt": 0.39735, "dt_data": 0.00809, "dt_net": 0.38925, "eta": "0:00:22", "loss": 0.09093, "lr": 0.05744, "mode": "train"}
[05/16 00:29:50][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "3", "cur_iter": "390", "dt": 0.39671, "dt_data": 0.00810, "dt_net": 0.38861, "eta": "0:00:18", "loss": 0.10092, "lr": 0.05790, "mode": "train"}
[05/16 00:29:54][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "3", "cur_iter": "400", "dt": 0.39562, "dt_data": 0.00809, "dt_net": 0.38753, "eta": "0:00:14", "loss": 0.07657, "lr": 0.05835, "mode": "train"}
[05/16 00:29:58][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "3", "cur_iter": "410", "dt": 0.39679, "dt_data": 0.00810, "dt_net": 0.38869, "eta": "0:00:10", "loss": 0.13483, "lr": 0.05881, "mode": "train"}
[05/16 00:30:02][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "3", "cur_iter": "420", "dt": 0.39754, "dt_data": 0.00808, "dt_net": 0.38946, "eta": "0:00:06", "loss": 0.12561, "lr": 0.05927, "mode": "train"}
[05/16 00:30:06][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "3", "cur_iter": "430", "dt": 0.39652, "dt_data": 0.00809, "dt_net": 0.38843, "eta": "0:00:02", "loss": 0.11574, "lr": 0.05973, "mode": "train"}
[05/16 00:30:08][INFO] train_net.py: 491: Epoch 2 takes 174.20s. Epochs from 0 to 2 take 173.95s in average and 174.13s in median.
[05/16 00:30:08][INFO] train_net.py: 497: For epoch 2, each iteraction takes 0.40s in average. From epoch 0 to 2, each iteraction takes 0.40s in average.
[05/16 00:30:14][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "4", "cur_iter": "10", "dt": 0.39956, "dt_data": 0.00820, "dt_net": 0.39137, "eta": "0:02:50", "loss": 0.10272, "lr": 0.06046, "mode": "train"}
[05/16 00:30:18][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "4", "cur_iter": "20", "dt": 0.39677, "dt_data": 0.00815, "dt_net": 0.38862, "eta": "0:02:45", "loss": 0.07508, "lr": 0.06092, "mode": "train"}
[05/16 00:30:22][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "4", "cur_iter": "30", "dt": 0.39977, "dt_data": 0.00817, "dt_net": 0.39160, "eta": "0:02:42", "loss": 0.09632, "lr": 0.06138, "mode": "train"}
[05/16 00:30:26][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "4", "cur_iter": "40", "dt": 0.39728, "dt_data": 0.00814, "dt_net": 0.38914, "eta": "0:02:37", "loss": 0.12252, "lr": 0.06184, "mode": "train"}
[05/16 00:30:30][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "4", "cur_iter": "50", "dt": 0.39952, "dt_data": 0.00819, "dt_net": 0.39133, "eta": "0:02:34", "loss": 0.09488, "lr": 0.06229, "mode": "train"}
[05/16 00:30:34][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "4", "cur_iter": "60", "dt": 0.40080, "dt_data": 0.00807, "dt_net": 0.39273, "eta": "0:02:31", "loss": 0.11153, "lr": 0.06275, "mode": "train"}
[05/16 00:30:38][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "4", "cur_iter": "70", "dt": 0.39734, "dt_data": 0.00840, "dt_net": 0.38893, "eta": "0:02:25", "loss": 0.11775, "lr": 0.06321, "mode": "train"}
[05/16 00:30:42][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "4", "cur_iter": "80", "dt": 0.39746, "dt_data": 0.00804, "dt_net": 0.38943, "eta": "0:02:21", "loss": 0.10195, "lr": 0.06367, "mode": "train"}
[05/16 00:30:46][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "4", "cur_iter": "90", "dt": 0.39863, "dt_data": 0.00804, "dt_net": 0.39060, "eta": "0:02:18", "loss": 0.13062, "lr": 0.06413, "mode": "train"}
[05/16 00:30:50][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "4", "cur_iter": "100", "dt": 0.39760, "dt_data": 0.00805, "dt_net": 0.38954, "eta": "0:02:13", "loss": 0.09182, "lr": 0.06459, "mode": "train"}
[05/16 00:30:53][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "4", "cur_iter": "110", "dt": 0.39898, "dt_data": 0.00804, "dt_net": 0.39094, "eta": "0:02:10", "loss": 0.10372, "lr": 0.06504, "mode": "train"}
[05/16 00:30:57][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "4", "cur_iter": "120", "dt": 0.39546, "dt_data": 0.00805, "dt_net": 0.38740, "eta": "0:02:05", "loss": 0.09011, "lr": 0.06550, "mode": "train"}
[05/16 00:31:01][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "4", "cur_iter": "130", "dt": 0.39766, "dt_data": 0.00803, "dt_net": 0.38962, "eta": "0:02:02", "loss": 0.08374, "lr": 0.06596, "mode": "train"}
[05/16 00:31:06][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "4", "cur_iter": "140", "dt": 0.58670, "dt_data": 0.20149, "dt_net": 0.38521, "eta": "0:02:54", "loss": 0.12493, "lr": 0.06642, "mode": "train"}
[05/16 00:31:10][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "4", "cur_iter": "150", "dt": 0.39543, "dt_data": 0.00812, "dt_net": 0.38731, "eta": "0:01:53", "loss": 0.09308, "lr": 0.06688, "mode": "train"}
[05/16 00:31:14][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "4", "cur_iter": "160", "dt": 0.39609, "dt_data": 0.00804, "dt_net": 0.38805, "eta": "0:01:49", "loss": 0.12191, "lr": 0.06733, "mode": "train"}
[05/16 00:31:18][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "4", "cur_iter": "170", "dt": 0.39700, "dt_data": 0.00803, "dt_net": 0.38896, "eta": "0:01:45", "loss": 0.15425, "lr": 0.06779, "mode": "train"}
[05/16 00:31:21][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "4", "cur_iter": "180", "dt": 0.39557, "dt_data": 0.00812, "dt_net": 0.38745, "eta": "0:01:41", "loss": 0.10861, "lr": 0.06825, "mode": "train"}
[05/16 00:31:25][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "4", "cur_iter": "190", "dt": 0.39515, "dt_data": 0.00808, "dt_net": 0.38707, "eta": "0:01:37", "loss": 0.10790, "lr": 0.06871, "mode": "train"}
[05/16 00:31:29][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "4", "cur_iter": "200", "dt": 0.39713, "dt_data": 0.00803, "dt_net": 0.38910, "eta": "0:01:34", "loss": 0.12322, "lr": 0.06917, "mode": "train"}
[05/16 00:31:33][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "4", "cur_iter": "210", "dt": 0.39710, "dt_data": 0.00807, "dt_net": 0.38902, "eta": "0:01:30", "loss": 0.07913, "lr": 0.06963, "mode": "train"}
[05/16 00:31:37][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "4", "cur_iter": "220", "dt": 0.39653, "dt_data": 0.00808, "dt_net": 0.38845, "eta": "0:01:26", "loss": 0.10420, "lr": 0.07008, "mode": "train"}
[05/16 00:31:41][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "4", "cur_iter": "230", "dt": 0.39517, "dt_data": 0.00812, "dt_net": 0.38704, "eta": "0:01:21", "loss": 0.06824, "lr": 0.07054, "mode": "train"}
[05/16 00:31:45][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "4", "cur_iter": "240", "dt": 0.39647, "dt_data": 0.00808, "dt_net": 0.38839, "eta": "0:01:18", "loss": 0.08961, "lr": 0.07100, "mode": "train"}
[05/16 00:31:49][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "4", "cur_iter": "250", "dt": 0.39739, "dt_data": 0.00809, "dt_net": 0.38930, "eta": "0:01:14", "loss": 0.09768, "lr": 0.07146, "mode": "train"}
[05/16 00:31:53][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "4", "cur_iter": "260", "dt": 0.39830, "dt_data": 0.00819, "dt_net": 0.39010, "eta": "0:01:10", "loss": 0.11066, "lr": 0.07192, "mode": "train"}
[05/16 00:31:57][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "4", "cur_iter": "270", "dt": 0.39621, "dt_data": 0.00802, "dt_net": 0.38818, "eta": "0:01:06", "loss": 0.12221, "lr": 0.07237, "mode": "train"}
[05/16 00:32:01][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "4", "cur_iter": "280", "dt": 0.39837, "dt_data": 0.00809, "dt_net": 0.39028, "eta": "0:01:02", "loss": 0.10468, "lr": 0.07283, "mode": "train"}
[05/16 00:32:05][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "4", "cur_iter": "290", "dt": 0.39640, "dt_data": 0.00803, "dt_net": 0.38836, "eta": "0:00:58", "loss": 0.13450, "lr": 0.07329, "mode": "train"}
[05/16 00:32:11][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "4", "cur_iter": "300", "dt": 0.39723, "dt_data": 0.00815, "dt_net": 0.38908, "eta": "0:00:54", "loss": 0.08742, "lr": 0.07375, "mode": "train"}
[05/16 00:32:15][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "4", "cur_iter": "310", "dt": 0.39565, "dt_data": 0.00810, "dt_net": 0.38755, "eta": "0:00:50", "loss": 0.10374, "lr": 0.07421, "mode": "train"}
[05/16 00:32:19][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "4", "cur_iter": "320", "dt": 0.39598, "dt_data": 0.00812, "dt_net": 0.38786, "eta": "0:00:46", "loss": 0.11984, "lr": 0.07466, "mode": "train"}
[05/16 00:32:23][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "4", "cur_iter": "330", "dt": 0.39631, "dt_data": 0.00812, "dt_net": 0.38819, "eta": "0:00:42", "loss": 0.08739, "lr": 0.07512, "mode": "train"}
[05/16 00:32:27][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "4", "cur_iter": "340", "dt": 0.39774, "dt_data": 0.00809, "dt_net": 0.38964, "eta": "0:00:38", "loss": 0.06971, "lr": 0.07558, "mode": "train"}
[05/16 00:32:31][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "4", "cur_iter": "350", "dt": 0.39620, "dt_data": 0.00810, "dt_net": 0.38810, "eta": "0:00:34", "loss": 0.08905, "lr": 0.07604, "mode": "train"}
[05/16 00:32:35][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "4", "cur_iter": "360", "dt": 0.39570, "dt_data": 0.00809, "dt_net": 0.38761, "eta": "0:00:30", "loss": 0.11079, "lr": 0.07650, "mode": "train"}
[05/16 00:32:39][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "4", "cur_iter": "370", "dt": 0.39640, "dt_data": 0.00810, "dt_net": 0.38830, "eta": "0:00:26", "loss": 0.10096, "lr": 0.07696, "mode": "train"}
[05/16 00:32:43][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "4", "cur_iter": "380", "dt": 0.39778, "dt_data": 0.00807, "dt_net": 0.38971, "eta": "0:00:22", "loss": 0.07510, "lr": 0.07741, "mode": "train"}
[05/16 00:32:47][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "4", "cur_iter": "390", "dt": 0.39529, "dt_data": 0.00808, "dt_net": 0.38721, "eta": "0:00:18", "loss": 0.09666, "lr": 0.07787, "mode": "train"}
[05/16 00:32:51][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "4", "cur_iter": "400", "dt": 0.39663, "dt_data": 0.00810, "dt_net": 0.38853, "eta": "0:00:14", "loss": 0.11165, "lr": 0.07833, "mode": "train"}
[05/16 00:32:55][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "4", "cur_iter": "410", "dt": 0.39682, "dt_data": 0.00809, "dt_net": 0.38872, "eta": "0:00:10", "loss": 0.08037, "lr": 0.07879, "mode": "train"}
[05/16 00:32:59][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "4", "cur_iter": "420", "dt": 0.39674, "dt_data": 0.00807, "dt_net": 0.38867, "eta": "0:00:06", "loss": 0.07973, "lr": 0.07925, "mode": "train"}
[05/16 00:33:03][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "4", "cur_iter": "430", "dt": 0.39553, "dt_data": 0.00814, "dt_net": 0.38739, "eta": "0:00:02", "loss": 0.11905, "lr": 0.07970, "mode": "train"}
[05/16 00:33:05][INFO] train_net.py: 491: Epoch 3 takes 176.29s. Epochs from 0 to 3 take 174.54s in average and 174.16s in median.
[05/16 00:33:05][INFO] train_net.py: 497: For epoch 3, each iteraction takes 0.40s in average. From epoch 0 to 3, each iteraction takes 0.40s in average.
[05/16 00:33:10][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "5", "cur_iter": "10", "dt": 0.39646, "dt_data": 0.00817, "dt_net": 0.38829, "eta": "0:02:49", "loss": 0.10148, "lr": 0.08044, "mode": "train"}
[05/16 00:33:14][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "5", "cur_iter": "20", "dt": 0.39668, "dt_data": 0.00812, "dt_net": 0.38856, "eta": "0:02:45", "loss": 0.10195, "lr": 0.08090, "mode": "train"}
[05/16 00:33:18][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "5", "cur_iter": "30", "dt": 0.39557, "dt_data": 0.00817, "dt_net": 0.38740, "eta": "0:02:40", "loss": 0.06299, "lr": 0.08135, "mode": "train"}
[05/16 00:33:22][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "5", "cur_iter": "40", "dt": 0.39667, "dt_data": 0.00808, "dt_net": 0.38858, "eta": "0:02:37", "loss": 0.10996, "lr": 0.08181, "mode": "train"}
[05/16 00:33:26][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "5", "cur_iter": "50", "dt": 0.39675, "dt_data": 0.00811, "dt_net": 0.38864, "eta": "0:02:33", "loss": 0.10396, "lr": 0.08227, "mode": "train"}
[05/16 00:33:30][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "5", "cur_iter": "60", "dt": 0.39704, "dt_data": 0.00808, "dt_net": 0.38896, "eta": "0:02:29", "loss": 0.11109, "lr": 0.08273, "mode": "train"}
[05/16 00:33:34][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "5", "cur_iter": "70", "dt": 0.39792, "dt_data": 0.00811, "dt_net": 0.38981, "eta": "0:02:26", "loss": 0.08589, "lr": 0.08319, "mode": "train"}
[05/16 00:33:38][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "5", "cur_iter": "80", "dt": 0.39659, "dt_data": 0.00818, "dt_net": 0.38841, "eta": "0:02:21", "loss": 0.11683, "lr": 0.08364, "mode": "train"}
[05/16 00:33:42][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "5", "cur_iter": "90", "dt": 0.39549, "dt_data": 0.00811, "dt_net": 0.38738, "eta": "0:02:17", "loss": 0.07872, "lr": 0.08410, "mode": "train"}
[05/16 00:33:46][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "5", "cur_iter": "100", "dt": 0.39730, "dt_data": 0.00811, "dt_net": 0.38918, "eta": "0:02:13", "loss": 0.10409, "lr": 0.08456, "mode": "train"}
[05/16 00:33:50][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "5", "cur_iter": "110", "dt": 0.39777, "dt_data": 0.00809, "dt_net": 0.38967, "eta": "0:02:10", "loss": 0.08403, "lr": 0.08502, "mode": "train"}
[05/16 00:33:54][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "5", "cur_iter": "120", "dt": 0.39560, "dt_data": 0.00818, "dt_net": 0.38743, "eta": "0:02:05", "loss": 0.08303, "lr": 0.08548, "mode": "train"}
[05/16 00:33:58][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "5", "cur_iter": "130", "dt": 0.39666, "dt_data": 0.00817, "dt_net": 0.38849, "eta": "0:02:01", "loss": 0.08470, "lr": 0.08594, "mode": "train"}
[05/16 00:34:02][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "5", "cur_iter": "140", "dt": 0.39722, "dt_data": 0.00816, "dt_net": 0.38907, "eta": "0:01:57", "loss": 0.13735, "lr": 0.08639, "mode": "train"}
[05/16 00:34:06][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "5", "cur_iter": "150", "dt": 0.39739, "dt_data": 0.00810, "dt_net": 0.38928, "eta": "0:01:54", "loss": 0.08189, "lr": 0.08685, "mode": "train"}
[05/16 00:34:10][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "5", "cur_iter": "160", "dt": 0.39612, "dt_data": 0.00818, "dt_net": 0.38795, "eta": "0:01:49", "loss": 0.11368, "lr": 0.08731, "mode": "train"}
[05/16 00:34:14][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "5", "cur_iter": "170", "dt": 0.39746, "dt_data": 0.00808, "dt_net": 0.38938, "eta": "0:01:46", "loss": 0.07423, "lr": 0.08777, "mode": "train"}
[05/16 00:34:18][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "5", "cur_iter": "180", "dt": 0.39738, "dt_data": 0.00818, "dt_net": 0.38921, "eta": "0:01:42", "loss": 0.10633, "lr": 0.08823, "mode": "train"}
[05/16 00:34:22][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "5", "cur_iter": "190", "dt": 0.39794, "dt_data": 0.00811, "dt_net": 0.38983, "eta": "0:01:38", "loss": 0.08888, "lr": 0.08868, "mode": "train"}
[05/16 00:34:26][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "5", "cur_iter": "200", "dt": 0.39736, "dt_data": 0.00811, "dt_net": 0.38925, "eta": "0:01:34", "loss": 0.08132, "lr": 0.08914, "mode": "train"}
[05/16 00:34:30][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "5", "cur_iter": "210", "dt": 0.39639, "dt_data": 0.00811, "dt_net": 0.38828, "eta": "0:01:29", "loss": 0.08584, "lr": 0.08960, "mode": "train"}
[05/16 00:34:34][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "5", "cur_iter": "220", "dt": 0.39598, "dt_data": 0.00817, "dt_net": 0.38781, "eta": "0:01:25", "loss": 0.14319, "lr": 0.09006, "mode": "train"}
[05/16 00:34:38][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "5", "cur_iter": "230", "dt": 0.39680, "dt_data": 0.00810, "dt_net": 0.38869, "eta": "0:01:22", "loss": 0.10337, "lr": 0.09052, "mode": "train"}
[05/16 00:34:42][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "5", "cur_iter": "240", "dt": 0.39768, "dt_data": 0.00812, "dt_net": 0.38956, "eta": "0:01:18", "loss": 0.08912, "lr": 0.09097, "mode": "train"}
[05/16 00:34:46][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "5", "cur_iter": "250", "dt": 0.39599, "dt_data": 0.00804, "dt_net": 0.38795, "eta": "0:01:14", "loss": 0.08767, "lr": 0.09143, "mode": "train"}
[05/16 00:34:50][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "5", "cur_iter": "260", "dt": 0.39527, "dt_data": 0.00817, "dt_net": 0.38710, "eta": "0:01:09", "loss": 0.07438, "lr": 0.09189, "mode": "train"}
[05/16 00:34:53][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "5", "cur_iter": "270", "dt": 0.39757, "dt_data": 0.00823, "dt_net": 0.38934, "eta": "0:01:06", "loss": 0.09909, "lr": 0.09235, "mode": "train"}
[05/16 00:34:57][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "5", "cur_iter": "280", "dt": 0.39721, "dt_data": 0.00809, "dt_net": 0.38911, "eta": "0:01:02", "loss": 0.11864, "lr": 0.09281, "mode": "train"}
[05/16 00:35:01][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "5", "cur_iter": "290", "dt": 0.39741, "dt_data": 0.00824, "dt_net": 0.38917, "eta": "0:00:58", "loss": 0.14784, "lr": 0.09327, "mode": "train"}
[05/16 00:35:05][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "5", "cur_iter": "300", "dt": 0.39649, "dt_data": 0.00812, "dt_net": 0.38838, "eta": "0:00:54", "loss": 0.09656, "lr": 0.09372, "mode": "train"}
[05/16 00:35:09][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "5", "cur_iter": "310", "dt": 0.39771, "dt_data": 0.00810, "dt_net": 0.38960, "eta": "0:00:50", "loss": 0.09099, "lr": 0.09418, "mode": "train"}
[05/16 00:35:13][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "5", "cur_iter": "320", "dt": 0.39765, "dt_data": 0.00811, "dt_net": 0.38954, "eta": "0:00:46", "loss": 0.08675, "lr": 0.09464, "mode": "train"}
[05/16 00:35:17][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "5", "cur_iter": "330", "dt": 0.39713, "dt_data": 0.00808, "dt_net": 0.38905, "eta": "0:00:42", "loss": 0.10311, "lr": 0.09510, "mode": "train"}
[05/16 00:35:21][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "5", "cur_iter": "340", "dt": 0.39616, "dt_data": 0.00809, "dt_net": 0.38807, "eta": "0:00:38", "loss": 0.06808, "lr": 0.09556, "mode": "train"}
[05/16 00:35:25][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "5", "cur_iter": "350", "dt": 0.39671, "dt_data": 0.00811, "dt_net": 0.38860, "eta": "0:00:34", "loss": 0.12094, "lr": 0.09601, "mode": "train"}
[05/16 00:35:29][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "5", "cur_iter": "360", "dt": 0.39634, "dt_data": 0.00811, "dt_net": 0.38823, "eta": "0:00:30", "loss": 0.08168, "lr": 0.09647, "mode": "train"}
[05/16 00:35:33][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "5", "cur_iter": "370", "dt": 0.39733, "dt_data": 0.00818, "dt_net": 0.38915, "eta": "0:00:26", "loss": 0.10809, "lr": 0.09693, "mode": "train"}
[05/16 00:35:37][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "5", "cur_iter": "380", "dt": 0.39764, "dt_data": 0.00804, "dt_net": 0.38960, "eta": "0:00:22", "loss": 0.13042, "lr": 0.09739, "mode": "train"}
[05/16 00:35:41][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "5", "cur_iter": "390", "dt": 0.39557, "dt_data": 0.00811, "dt_net": 0.38746, "eta": "0:00:18", "loss": 0.08457, "lr": 0.09785, "mode": "train"}
[05/16 00:35:45][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "5", "cur_iter": "400", "dt": 0.39733, "dt_data": 0.00811, "dt_net": 0.38922, "eta": "0:00:14", "loss": 0.12795, "lr": 0.09830, "mode": "train"}
[05/16 00:35:49][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "5", "cur_iter": "410", "dt": 0.39898, "dt_data": 0.00812, "dt_net": 0.39086, "eta": "0:00:10", "loss": 0.11749, "lr": 0.09876, "mode": "train"}
[05/16 00:35:53][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "5", "cur_iter": "420", "dt": 0.39702, "dt_data": 0.00820, "dt_net": 0.38882, "eta": "0:00:06", "loss": 0.07859, "lr": 0.09922, "mode": "train"}
[05/16 00:35:57][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "5", "cur_iter": "430", "dt": 0.39702, "dt_data": 0.00809, "dt_net": 0.38892, "eta": "0:00:02", "loss": 0.11105, "lr": 0.09968, "mode": "train"}
[05/16 00:35:59][INFO] train_net.py: 491: Epoch 4 takes 173.99s. Epochs from 0 to 4 take 174.43s in average and 174.13s in median.
[05/16 00:35:59][INFO] train_net.py: 497: For epoch 4, each iteraction takes 0.40s in average. From epoch 0 to 4, each iteraction takes 0.40s in average.
[05/16 00:36:03][INFO] logging.py: 99: json_stats: {"_type": "val_iter", "cur_epoch": "5", "cur_iter": "10", "dt": 0.11995, "dt_data": 0.00813, "dt_net": 0.11181, "eta": "0:00:12", "mode": "val"}
[05/16 00:36:05][INFO] logging.py: 99: json_stats: {"_type": "val_iter", "cur_epoch": "5", "cur_iter": "20", "dt": 0.11944, "dt_data": 0.00817, "dt_net": 0.11126, "eta": "0:00:10", "mode": "val"}
[05/16 00:36:08][INFO] logging.py: 99: json_stats: {"_type": "val_iter", "cur_epoch": "5", "cur_iter": "30", "dt": 0.11860, "dt_data": 0.00812, "dt_net": 0.11048, "eta": "0:00:09", "mode": "val"}
[05/16 00:36:11][INFO] logging.py: 99: json_stats: {"_type": "val_iter", "cur_epoch": "5", "cur_iter": "40", "dt": 0.14865, "dt_data": 0.03941, "dt_net": 0.10925, "eta": "0:00:10", "mode": "val"}
[05/16 00:36:13][INFO] logging.py: 99: json_stats: {"_type": "val_iter", "cur_epoch": "5", "cur_iter": "50", "dt": 0.14821, "dt_data": 0.03907, "dt_net": 0.10914, "eta": "0:00:09", "mode": "val"}
[05/16 00:36:16][INFO] logging.py: 99: json_stats: {"_type": "val_iter", "cur_epoch": "5", "cur_iter": "60", "dt": 0.30677, "dt_data": 0.19802, "dt_net": 0.10876, "eta": "0:00:15", "mode": "val"}
[05/16 00:36:20][INFO] logging.py: 99: json_stats: {"_type": "val_iter", "cur_epoch": "5", "cur_iter": "70", "dt": 0.31258, "dt_data": 0.20368, "dt_net": 0.10890, "eta": "0:00:12", "mode": "val"}
[05/16 00:36:23][INFO] logging.py: 99: json_stats: {"_type": "val_iter", "cur_epoch": "5", "cur_iter": "80", "dt": 0.31344, "dt_data": 0.20469, "dt_net": 0.10875, "eta": "0:00:09", "mode": "val"}
[05/16 00:36:26][INFO] logging.py: 99: json_stats: {"_type": "val_iter", "cur_epoch": "5", "cur_iter": "90", "dt": 0.35997, "dt_data": 0.25210, "dt_net": 0.10787, "eta": "0:00:07", "mode": "val"}
[05/16 00:36:30][INFO] logging.py: 99: json_stats: {"_type": "val_iter", "cur_epoch": "5", "cur_iter": "100", "dt": 0.28989, "dt_data": 0.18165, "dt_net": 0.10824, "eta": "0:00:03", "mode": "val"}
[05/16 00:36:33][INFO] logging.py: 99: json_stats: {"_type": "val_iter", "cur_epoch": "5", "cur_iter": "110", "dt": 0.03289, "dt_data": 0.00212, "dt_net": 0.03077, "eta": "0:00:00", "mode": "val"}
[05/16 00:36:33][INFO] ava_eval_helper.py: 160: Evaluating with 415 unique GT frames.
[05/16 00:36:33][INFO] ava_eval_helper.py: 161: Evaluating with 437 unique detection frames
[05/16 00:36:33][INFO] ava_eval_helper.py: 303: AVA results wrote to detections_latest.csv
[05/16 00:36:33][INFO] ava_eval_helper.py: 304: took 0 seconds.
[05/16 00:36:33][INFO] ava_eval_helper.py: 303: AVA results wrote to groundtruth_latest.csv
[05/16 00:36:33][INFO] ava_eval_helper.py: 304: took 0 seconds.
[05/16 00:36:37][INFO] object_detection_evaluation.py: 775: The following classes have no ground truth examples: [ 2 3 4 7 9 13 15 16 18 19 20 21 23 24 25 28 30 31 32 33 34 35 37 39
40 41 42 43 44 45 47 49 50 51 52 53 55 56 58 60 62 64 67 68 70 71 73 75
77]
{ 'PascalBoxes_PerformanceByCategory/AP@0.5IOU/answer phone': nan,
'PascalBoxes_PerformanceByCategory/AP@0.5IOU/bend/bow (at the waist)': 0.0,
'PascalBoxes_PerformanceByCategory/AP@0.5IOU/carry/hold (an object)': 0.0,
'PascalBoxes_PerformanceByCategory/AP@0.5IOU/climb (e.g., a mountain)': nan,
'PascalBoxes_PerformanceByCategory/AP@0.5IOU/close (e.g., a door, a box)': 0.0,
'PascalBoxes_PerformanceByCategory/AP@0.5IOU/crouch/kneel': nan,
'PascalBoxes_PerformanceByCategory/AP@0.5IOU/cut': nan,
'PascalBoxes_PerformanceByCategory/AP@0.5IOU/dance': nan,
'PascalBoxes_PerformanceByCategory/AP@0.5IOU/dress/put on clothing': 0.0,
'PascalBoxes_PerformanceByCategory/AP@0.5IOU/drink': 0.0,
'PascalBoxes_PerformanceByCategory/AP@0.5IOU/drive (e.g., a car, a truck)': nan,
'PascalBoxes_PerformanceByCategory/AP@0.5IOU/eat': 0.0,
'PascalBoxes_PerformanceByCategory/AP@0.5IOU/enter': nan,
'PascalBoxes_PerformanceByCategory/AP@0.5IOU/fall down': 0.0,
'PascalBoxes_PerformanceByCategory/AP@0.5IOU/fight/hit (a person)': nan,
'PascalBoxes_PerformanceByCategory/AP@0.5IOU/get up': 0.0,
'PascalBoxes_PerformanceByCategory/AP@0.5IOU/give/serve (an object) to (a person)': 0.0,
'PascalBoxes_PerformanceByCategory/AP@0.5IOU/grab (a person)': 0.0,
'PascalBoxes_PerformanceByCategory/AP@0.5IOU/hand clap': nan,
'PascalBoxes_PerformanceByCategory/AP@0.5IOU/hand shake': nan,
'PascalBoxes_PerformanceByCategory/AP@0.5IOU/hand wave': 0.0,
'PascalBoxes_PerformanceByCategory/AP@0.5IOU/hit (an object)': nan,
'PascalBoxes_PerformanceByCategory/AP@0.5IOU/hug (a person)': nan,
'PascalBoxes_PerformanceByCategory/AP@0.5IOU/jump/leap': nan,
'PascalBoxes_PerformanceByCategory/AP@0.5IOU/kiss (a person)': 0.0,
'PascalBoxes_PerformanceByCategory/AP@0.5IOU/lie/sleep': 0.0,
'PascalBoxes_PerformanceByCategory/AP@0.5IOU/lift (a person)': nan,
'PascalBoxes_PerformanceByCategory/AP@0.5IOU/lift/pick up': 0.0,
'PascalBoxes_PerformanceByCategory/AP@0.5IOU/listen (e.g., to music)': nan,
'PascalBoxes_PerformanceByCategory/AP@0.5IOU/listen to (a person)': 0.0,
'PascalBoxes_PerformanceByCategory/AP@0.5IOU/martial art': nan,
'PascalBoxes_PerformanceByCategory/AP@0.5IOU/open (e.g., a window, a car door)': 0.0,
'PascalBoxes_PerformanceByCategory/AP@0.5IOU/play musical instrument': nan,
'PascalBoxes_PerformanceByCategory/AP@0.5IOU/point to (an object)': nan,
'PascalBoxes_PerformanceByCategory/AP@0.5IOU/pull (an object)': nan,
'PascalBoxes_PerformanceByCategory/AP@0.5IOU/push (an object)': 0.0,
'PascalBoxes_PerformanceByCategory/AP@0.5IOU/push (another person)': 0.0,
'PascalBoxes_PerformanceByCategory/AP@0.5IOU/put down': nan,
'PascalBoxes_PerformanceByCategory/AP@0.5IOU/read': 0.0,
'PascalBoxes_PerformanceByCategory/AP@0.5IOU/ride (e.g., a bike, a car, a horse)': nan,
'PascalBoxes_PerformanceByCategory/AP@0.5IOU/run/jog': 0.0,
'PascalBoxes_PerformanceByCategory/AP@0.5IOU/sail boat': nan,
'PascalBoxes_PerformanceByCategory/AP@0.5IOU/shoot': nan,
'PascalBoxes_PerformanceByCategory/AP@0.5IOU/sing to (e.g., self, a person, a group)': nan,
'PascalBoxes_PerformanceByCategory/AP@0.5IOU/sit': 0.0,
'PascalBoxes_PerformanceByCategory/AP@0.5IOU/smoke': 0.0,
'PascalBoxes_PerformanceByCategory/AP@0.5IOU/stand': 0.0,
'PascalBoxes_PerformanceByCategory/AP@0.5IOU/swim': nan,
'PascalBoxes_PerformanceByCategory/AP@0.5IOU/take (an object) from (a person)': 0.0,
'PascalBoxes_PerformanceByCategory/AP@0.5IOU/take a photo': nan,
'PascalBoxes_PerformanceByCategory/AP@0.5IOU/talk to (e.g., self, a person, a group)': 0.0,
'PascalBoxes_PerformanceByCategory/AP@0.5IOU/text on/look at a cellphone': 0.0,
'PascalBoxes_PerformanceByCategory/AP@0.5IOU/throw': nan,
'PascalBoxes_PerformanceByCategory/AP@0.5IOU/touch (an object)': 0.0,
'PascalBoxes_PerformanceByCategory/AP@0.5IOU/turn (e.g., a screwdriver)': nan,
'PascalBoxes_PerformanceByCategory/AP@0.5IOU/walk': 0.0,
'PascalBoxes_PerformanceByCategory/AP@0.5IOU/watch (a person)': 0.0,
'PascalBoxes_PerformanceByCategory/AP@0.5IOU/watch (e.g., TV)': 0.0,
'PascalBoxes_PerformanceByCategory/AP@0.5IOU/work on a computer': nan,
'PascalBoxes_PerformanceByCategory/AP@0.5IOU/write': 0.0,
'PascalBoxes_Precision/mAP@0.5IOU': 0.0}
[05/16 00:36:37][INFO] ava_eval_helper.py: 170: AVA eval done in 4.012340 seconds.
[05/16 00:36:37][INFO] logging.py: 99: json_stats: {"RAM": "49.37/376.07G", "_type": "val_epoch", "cur_epoch": "5", "gpu_mem": "5.81G", "map": 0.00000, "mode": "val"}
[05/16 00:36:42][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "6", "cur_iter": "10", "dt": 0.39480, "dt_data": 0.00808, "dt_net": 0.38672, "eta": "0:02:48", "loss": 0.12900, "lr": 0.10000, "mode": "train"}
[05/16 00:36:46][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "6", "cur_iter": "20", "dt": 0.39650, "dt_data": 0.00805, "dt_net": 0.38845, "eta": "0:02:45", "loss": 0.10416, "lr": 0.10000, "mode": "train"}
[05/16 00:36:50][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "6", "cur_iter": "30", "dt": 0.39404, "dt_data": 0.00802, "dt_net": 0.38602, "eta": "0:02:40", "loss": 0.12096, "lr": 0.10000, "mode": "train"}
[05/16 00:36:54][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "6", "cur_iter": "40", "dt": 0.39421, "dt_data": 0.00806, "dt_net": 0.38615, "eta": "0:02:36", "loss": 0.10860, "lr": 0.10000, "mode": "train"}
[05/16 00:36:57][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "6", "cur_iter": "50", "dt": 0.39492, "dt_data": 0.00803, "dt_net": 0.38689, "eta": "0:02:32", "loss": 0.09868, "lr": 0.10000, "mode": "train"}
[05/16 00:37:01][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "6", "cur_iter": "60", "dt": 0.39533, "dt_data": 0.00808, "dt_net": 0.38724, "eta": "0:02:29", "loss": 0.10757, "lr": 0.10000, "mode": "train"}
[05/16 00:37:05][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "6", "cur_iter": "70", "dt": 0.39500, "dt_data": 0.00802, "dt_net": 0.38697, "eta": "0:02:24", "loss": 0.11419, "lr": 0.10000, "mode": "train"}
[05/16 00:37:09][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "6", "cur_iter": "80", "dt": 0.39641, "dt_data": 0.00802, "dt_net": 0.38838, "eta": "0:02:21", "loss": 0.09259, "lr": 0.10000, "mode": "train"}
[05/16 00:37:13][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "6", "cur_iter": "90", "dt": 0.39457, "dt_data": 0.00802, "dt_net": 0.38654, "eta": "0:02:16", "loss": 0.10866, "lr": 0.10000, "mode": "train"}
[05/16 00:37:17][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "6", "cur_iter": "100", "dt": 0.39533, "dt_data": 0.00802, "dt_net": 0.38730, "eta": "0:02:13", "loss": 0.07291, "lr": 0.10000, "mode": "train"}
[05/16 00:37:21][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "6", "cur_iter": "110", "dt": 0.39678, "dt_data": 0.00803, "dt_net": 0.38875, "eta": "0:02:09", "loss": 0.07068, "lr": 0.10000, "mode": "train"}
[05/16 00:37:25][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "6", "cur_iter": "120", "dt": 0.39629, "dt_data": 0.00802, "dt_net": 0.38826, "eta": "0:02:05", "loss": 0.10138, "lr": 0.10000, "mode": "train"}
[05/16 00:37:29][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "6", "cur_iter": "130", "dt": 0.39769, "dt_data": 0.00803, "dt_net": 0.38966, "eta": "0:02:02", "loss": 0.09431, "lr": 0.10000, "mode": "train"}
[05/16 00:37:33][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "6", "cur_iter": "140", "dt": 0.39572, "dt_data": 0.00803, "dt_net": 0.38769, "eta": "0:01:57", "loss": 0.08334, "lr": 0.10000, "mode": "train"}
[05/16 00:37:37][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "6", "cur_iter": "150", "dt": 0.39644, "dt_data": 0.00803, "dt_net": 0.38841, "eta": "0:01:53", "loss": 0.07918, "lr": 0.10000, "mode": "train"}
[05/16 00:37:41][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "6", "cur_iter": "160", "dt": 0.39684, "dt_data": 0.00802, "dt_net": 0.38882, "eta": "0:01:49", "loss": 0.09865, "lr": 0.10000, "mode": "train"}
[05/16 00:37:45][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "6", "cur_iter": "170", "dt": 0.39729, "dt_data": 0.00802, "dt_net": 0.38926, "eta": "0:01:46", "loss": 0.10687, "lr": 0.10000, "mode": "train"}
[05/16 00:37:49][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "6", "cur_iter": "180", "dt": 0.39560, "dt_data": 0.00802, "dt_net": 0.38757, "eta": "0:01:41", "loss": 0.07422, "lr": 0.10000, "mode": "train"}
[05/16 00:37:53][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "6", "cur_iter": "190", "dt": 0.39576, "dt_data": 0.00803, "dt_net": 0.38773, "eta": "0:01:37", "loss": 0.08613, "lr": 0.10000, "mode": "train"}
[05/16 00:37:57][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "6", "cur_iter": "200", "dt": 0.39667, "dt_data": 0.00803, "dt_net": 0.38864, "eta": "0:01:34", "loss": 0.08887, "lr": 0.10000, "mode": "train"}
[05/16 00:38:01][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "6", "cur_iter": "210", "dt": 0.39790, "dt_data": 0.00806, "dt_net": 0.38984, "eta": "0:01:30", "loss": 0.11063, "lr": 0.10000, "mode": "train"}
[05/16 00:38:05][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "6", "cur_iter": "220", "dt": 0.39529, "dt_data": 0.00802, "dt_net": 0.38727, "eta": "0:01:25", "loss": 0.07113, "lr": 0.10000, "mode": "train"}
[05/16 00:38:09][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "6", "cur_iter": "230", "dt": 0.39626, "dt_data": 0.00802, "dt_net": 0.38823, "eta": "0:01:22", "loss": 0.08611, "lr": 0.10000, "mode": "train"}
[05/16 00:38:13][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "6", "cur_iter": "240", "dt": 0.39668, "dt_data": 0.00802, "dt_net": 0.38866, "eta": "0:01:18", "loss": 0.07747, "lr": 0.10000, "mode": "train"}
[05/16 00:38:17][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "6", "cur_iter": "250", "dt": 0.39726, "dt_data": 0.00803, "dt_net": 0.38923, "eta": "0:01:14", "loss": 0.13155, "lr": 0.10000, "mode": "train"}
[05/16 00:38:21][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "6", "cur_iter": "260", "dt": 0.39712, "dt_data": 0.00803, "dt_net": 0.38908, "eta": "0:01:10", "loss": 0.05857, "lr": 0.10000, "mode": "train"}
[05/16 00:38:25][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "6", "cur_iter": "270", "dt": 0.39598, "dt_data": 0.00803, "dt_net": 0.38794, "eta": "0:01:06", "loss": 0.11002, "lr": 0.10000, "mode": "train"}
[05/16 00:38:29][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "6", "cur_iter": "280", "dt": 0.39639, "dt_data": 0.00808, "dt_net": 0.38831, "eta": "0:01:02", "loss": 0.07633, "lr": 0.10000, "mode": "train"}
[05/16 00:38:33][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "6", "cur_iter": "290", "dt": 0.39740, "dt_data": 0.00803, "dt_net": 0.38938, "eta": "0:00:58", "loss": 0.12798, "lr": 0.10000, "mode": "train"}
[05/16 00:38:37][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "6", "cur_iter": "300", "dt": 0.39638, "dt_data": 0.00813, "dt_net": 0.38824, "eta": "0:00:54", "loss": 0.10813, "lr": 0.10000, "mode": "train"}
[05/16 00:38:41][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "6", "cur_iter": "310", "dt": 0.39623, "dt_data": 0.00808, "dt_net": 0.38815, "eta": "0:00:50", "loss": 0.10295, "lr": 0.10000, "mode": "train"}
[05/16 00:38:44][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "6", "cur_iter": "320", "dt": 0.39782, "dt_data": 0.00813, "dt_net": 0.38969, "eta": "0:00:46", "loss": 0.06992, "lr": 0.10000, "mode": "train"}
[05/16 00:38:48][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "6", "cur_iter": "330", "dt": 0.39630, "dt_data": 0.00803, "dt_net": 0.38828, "eta": "0:00:42", "loss": 0.11295, "lr": 0.10000, "mode": "train"}
[05/16 00:38:52][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "6", "cur_iter": "340", "dt": 0.39548, "dt_data": 0.00802, "dt_net": 0.38746, "eta": "0:00:38", "loss": 0.09360, "lr": 0.10000, "mode": "train"}
[05/16 00:38:56][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "6", "cur_iter": "350", "dt": 0.39625, "dt_data": 0.00802, "dt_net": 0.38823, "eta": "0:00:34", "loss": 0.09578, "lr": 0.10000, "mode": "train"}
[05/16 00:39:00][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "6", "cur_iter": "360", "dt": 0.39630, "dt_data": 0.00803, "dt_net": 0.38827, "eta": "0:00:30", "loss": 0.08124, "lr": 0.10000, "mode": "train"}
[05/16 00:39:04][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "6", "cur_iter": "370", "dt": 0.39794, "dt_data": 0.00802, "dt_net": 0.38992, "eta": "0:00:26", "loss": 0.06729, "lr": 0.10000, "mode": "train"}
[05/16 00:39:08][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "6", "cur_iter": "380", "dt": 0.39614, "dt_data": 0.00802, "dt_net": 0.38811, "eta": "0:00:22", "loss": 0.08930, "lr": 0.10000, "mode": "train"}
[05/16 00:39:12][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "6", "cur_iter": "390", "dt": 0.39683, "dt_data": 0.00803, "dt_net": 0.38880, "eta": "0:00:18", "loss": 0.10758, "lr": 0.10000, "mode": "train"}
[05/16 00:39:16][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "6", "cur_iter": "400", "dt": 0.39638, "dt_data": 0.00802, "dt_net": 0.38835, "eta": "0:00:14", "loss": 0.09902, "lr": 0.10000, "mode": "train"}
[05/16 00:39:20][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "6", "cur_iter": "410", "dt": 0.39545, "dt_data": 0.00802, "dt_net": 0.38743, "eta": "0:00:10", "loss": 0.09438, "lr": 0.10000, "mode": "train"}
[05/16 00:39:24][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "6", "cur_iter": "420", "dt": 0.39706, "dt_data": 0.00812, "dt_net": 0.38894, "eta": "0:00:06", "loss": 0.09288, "lr": 0.10000, "mode": "train"}
[05/16 00:39:28][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "6", "cur_iter": "430", "dt": 0.39612, "dt_data": 0.00807, "dt_net": 0.38806, "eta": "0:00:02", "loss": 0.12834, "lr": 0.10000, "mode": "train"}
[05/16 00:39:31][INFO] train_net.py: 491: Epoch 5 takes 173.60s. Epochs from 0 to 5 take 174.29s in average and 174.06s in median.
[05/16 00:39:31][INFO] train_net.py: 497: For epoch 5, each iteraction takes 0.40s in average. From epoch 0 to 5, each iteraction takes 0.40s in average.
[05/16 00:39:36][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "7", "cur_iter": "10", "dt": 0.39733, "dt_data": 0.00803, "dt_net": 0.38930, "eta": "0:02:49", "loss": 0.05542, "lr": 0.10000, "mode": "train"}
[05/16 00:39:40][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "7", "cur_iter": "20", "dt": 0.39734, "dt_data": 0.00803, "dt_net": 0.38931, "eta": "0:02:45", "loss": 0.08145, "lr": 0.10000, "mode": "train"}
[05/16 00:39:44][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "7", "cur_iter": "30", "dt": 0.39637, "dt_data": 0.00805, "dt_net": 0.38832, "eta": "0:02:41", "loss": 0.09504, "lr": 0.10000, "mode": "train"}
[05/16 00:39:48][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "7", "cur_iter": "40", "dt": 0.39626, "dt_data": 0.00810, "dt_net": 0.38816, "eta": "0:02:37", "loss": 0.08873, "lr": 0.10000, "mode": "train"}
[05/16 00:39:52][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "7", "cur_iter": "50", "dt": 0.39863, "dt_data": 0.00810, "dt_net": 0.39053, "eta": "0:02:34", "loss": 0.07512, "lr": 0.10000, "mode": "train"}
[05/16 00:39:56][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "7", "cur_iter": "60", "dt": 0.39619, "dt_data": 0.00810, "dt_net": 0.38808, "eta": "0:02:29", "loss": 0.08291, "lr": 0.10000, "mode": "train"}
[05/16 00:40:00][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "7", "cur_iter": "70", "dt": 0.39627, "dt_data": 0.00810, "dt_net": 0.38817, "eta": "0:02:25", "loss": 0.06470, "lr": 0.10000, "mode": "train"}
[05/16 00:40:04][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "7", "cur_iter": "80", "dt": 0.39689, "dt_data": 0.00809, "dt_net": 0.38880, "eta": "0:02:21", "loss": 0.10360, "lr": 0.10000, "mode": "train"}
[05/16 00:40:07][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "7", "cur_iter": "90", "dt": 0.39816, "dt_data": 0.00814, "dt_net": 0.39001, "eta": "0:02:18", "loss": 0.10583, "lr": 0.10000, "mode": "train"}
[05/16 00:40:11][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "7", "cur_iter": "100", "dt": 0.39785, "dt_data": 0.00811, "dt_net": 0.38974, "eta": "0:02:14", "loss": 0.09276, "lr": 0.10000, "mode": "train"}
[05/16 00:40:15][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "7", "cur_iter": "110", "dt": 0.39894, "dt_data": 0.00807, "dt_net": 0.39087, "eta": "0:02:10", "loss": 0.08729, "lr": 0.10000, "mode": "train"}
[05/16 00:40:19][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "7", "cur_iter": "120", "dt": 0.39583, "dt_data": 0.00810, "dt_net": 0.38773, "eta": "0:02:05", "loss": 0.06414, "lr": 0.10000, "mode": "train"}
[05/16 00:40:23][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "7", "cur_iter": "130", "dt": 0.39668, "dt_data": 0.00809, "dt_net": 0.38859, "eta": "0:02:01", "loss": 0.06993, "lr": 0.10000, "mode": "train"}
[05/16 00:40:27][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "7", "cur_iter": "140", "dt": 0.39730, "dt_data": 0.00810, "dt_net": 0.38920, "eta": "0:01:57", "loss": 0.07839, "lr": 0.10000, "mode": "train"}
[05/16 00:40:31][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "7", "cur_iter": "150", "dt": 0.39709, "dt_data": 0.00807, "dt_net": 0.38902, "eta": "0:01:53", "loss": 0.07639, "lr": 0.10000, "mode": "train"}
[05/16 00:40:35][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "7", "cur_iter": "160", "dt": 0.39719, "dt_data": 0.00808, "dt_net": 0.38911, "eta": "0:01:50", "loss": 0.10338, "lr": 0.10000, "mode": "train"}
[05/16 00:40:39][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "7", "cur_iter": "170", "dt": 0.39694, "dt_data": 0.00810, "dt_net": 0.38884, "eta": "0:01:45", "loss": 0.05840, "lr": 0.10000, "mode": "train"}
[05/16 00:40:43][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "7", "cur_iter": "180", "dt": 0.39759, "dt_data": 0.00811, "dt_net": 0.38948, "eta": "0:01:42", "loss": 0.06863, "lr": 0.10000, "mode": "train"}
[05/16 00:40:47][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "7", "cur_iter": "190", "dt": 0.39815, "dt_data": 0.00811, "dt_net": 0.39005, "eta": "0:01:38", "loss": 0.10182, "lr": 0.10000, "mode": "train"}
[05/16 00:40:51][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "7", "cur_iter": "200", "dt": 0.39591, "dt_data": 0.00808, "dt_net": 0.38783, "eta": "0:01:33", "loss": 0.07161, "lr": 0.10000, "mode": "train"}
[05/16 00:40:55][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "7", "cur_iter": "210", "dt": 0.39657, "dt_data": 0.00809, "dt_net": 0.38848, "eta": "0:01:30", "loss": 0.10596, "lr": 0.10000, "mode": "train"}
[05/16 00:40:59][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "7", "cur_iter": "220", "dt": 0.39607, "dt_data": 0.00809, "dt_net": 0.38798, "eta": "0:01:25", "loss": 0.07911, "lr": 0.10000, "mode": "train"}
[05/16 00:41:03][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "7", "cur_iter": "230", "dt": 0.39731, "dt_data": 0.00811, "dt_net": 0.38919, "eta": "0:01:22", "loss": 0.08162, "lr": 0.10000, "mode": "train"}
[05/16 00:41:07][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "7", "cur_iter": "240", "dt": 0.39646, "dt_data": 0.00808, "dt_net": 0.38838, "eta": "0:01:18", "loss": 0.07139, "lr": 0.10000, "mode": "train"}
[05/16 00:41:11][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "7", "cur_iter": "250", "dt": 0.39540, "dt_data": 0.00813, "dt_net": 0.38727, "eta": "0:01:13", "loss": 0.11300, "lr": 0.10000, "mode": "train"}
[05/16 00:41:15][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "7", "cur_iter": "260", "dt": 0.39682, "dt_data": 0.00810, "dt_net": 0.38872, "eta": "0:01:10", "loss": 0.06662, "lr": 0.10000, "mode": "train"}
[05/16 00:41:19][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "7", "cur_iter": "270", "dt": 0.39876, "dt_data": 0.00810, "dt_net": 0.39066, "eta": "0:01:06", "loss": 0.12189, "lr": 0.10000, "mode": "train"}
[05/16 00:41:23][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "7", "cur_iter": "280", "dt": 0.39712, "dt_data": 0.00810, "dt_net": 0.38902, "eta": "0:01:02", "loss": 0.06148, "lr": 0.10000, "mode": "train"}
[05/16 00:41:27][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "7", "cur_iter": "290", "dt": 0.39579, "dt_data": 0.00814, "dt_net": 0.38765, "eta": "0:00:58", "loss": 0.09144, "lr": 0.10000, "mode": "train"}
[05/16 00:41:31][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "7", "cur_iter": "300", "dt": 0.39786, "dt_data": 0.00810, "dt_net": 0.38976, "eta": "0:00:54", "loss": 0.09549, "lr": 0.10000, "mode": "train"}
[05/16 00:41:35][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "7", "cur_iter": "310", "dt": 0.39718, "dt_data": 0.00808, "dt_net": 0.38911, "eta": "0:00:50", "loss": 0.10214, "lr": 0.10000, "mode": "train"}
[05/16 00:41:39][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "7", "cur_iter": "320", "dt": 0.39739, "dt_data": 0.00809, "dt_net": 0.38930, "eta": "0:00:46", "loss": 0.08289, "lr": 0.10000, "mode": "train"}
[05/16 00:41:43][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "7", "cur_iter": "330", "dt": 0.39918, "dt_data": 0.00813, "dt_net": 0.39105, "eta": "0:00:42", "loss": 0.06562, "lr": 0.10000, "mode": "train"}
[05/16 00:41:47][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "7", "cur_iter": "340", "dt": 0.39562, "dt_data": 0.00804, "dt_net": 0.38758, "eta": "0:00:38", "loss": 0.09046, "lr": 0.10000, "mode": "train"}
[05/16 00:41:51][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "7", "cur_iter": "350", "dt": 0.39628, "dt_data": 0.00808, "dt_net": 0.38820, "eta": "0:00:34", "loss": 0.10354, "lr": 0.10000, "mode": "train"}
[05/16 00:41:55][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "7", "cur_iter": "360", "dt": 0.39794, "dt_data": 0.00809, "dt_net": 0.38985, "eta": "0:00:30", "loss": 0.07110, "lr": 0.10000, "mode": "train"}
[05/16 00:41:59][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "7", "cur_iter": "370", "dt": 0.39820, "dt_data": 0.00809, "dt_net": 0.39010, "eta": "0:00:26", "loss": 0.07688, "lr": 0.10000, "mode": "train"}
[05/16 00:42:03][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "7", "cur_iter": "380", "dt": 0.39702, "dt_data": 0.00810, "dt_net": 0.38892, "eta": "0:00:22", "loss": 0.07073, "lr": 0.10000, "mode": "train"}
[05/16 00:42:07][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "7", "cur_iter": "390", "dt": 0.39666, "dt_data": 0.00809, "dt_net": 0.38857, "eta": "0:00:18", "loss": 0.06006, "lr": 0.10000, "mode": "train"}
[05/16 00:42:11][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "7", "cur_iter": "400", "dt": 0.39766, "dt_data": 0.00807, "dt_net": 0.38958, "eta": "0:00:14", "loss": 0.10336, "lr": 0.10000, "mode": "train"}
[05/16 00:42:15][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "7", "cur_iter": "410", "dt": 0.39770, "dt_data": 0.00810, "dt_net": 0.38960, "eta": "0:00:10", "loss": 0.08177, "lr": 0.10000, "mode": "train"}
[05/16 00:42:19][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "7", "cur_iter": "420", "dt": 0.39570, "dt_data": 0.00808, "dt_net": 0.38761, "eta": "0:00:06", "loss": 0.06845, "lr": 0.10000, "mode": "train"}
[05/16 00:42:22][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "7", "cur_iter": "430", "dt": 0.39628, "dt_data": 0.00810, "dt_net": 0.38818, "eta": "0:00:02", "loss": 0.10503, "lr": 0.10000, "mode": "train"}
[05/16 00:42:25][INFO] train_net.py: 491: Epoch 6 takes 174.00s. Epochs from 0 to 6 take 174.25s in average and 174.00s in median.
[05/16 00:42:25][INFO] train_net.py: 497: For epoch 6, each iteraction takes 0.40s in average. From epoch 0 to 6, each iteraction takes 0.40s in average.
[05/16 00:42:30][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "8", "cur_iter": "10", "dt": 0.39752, "dt_data": 0.00810, "dt_net": 0.38941, "eta": "0:02:49", "loss": 0.11148, "lr": 0.10000, "mode": "train"}
[05/16 00:42:34][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "8", "cur_iter": "20", "dt": 0.39731, "dt_data": 0.00811, "dt_net": 0.38920, "eta": "0:02:45", "loss": 0.07810, "lr": 0.10000, "mode": "train"}
[05/16 00:42:38][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "8", "cur_iter": "30", "dt": 0.39726, "dt_data": 0.00806, "dt_net": 0.38919, "eta": "0:02:41", "loss": 0.09568, "lr": 0.10000, "mode": "train"}
[05/16 00:42:42][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "8", "cur_iter": "40", "dt": 0.39567, "dt_data": 0.00809, "dt_net": 0.38758, "eta": "0:02:37", "loss": 0.10002, "lr": 0.10000, "mode": "train"}
[05/16 00:42:46][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "8", "cur_iter": "50", "dt": 0.39705, "dt_data": 0.00809, "dt_net": 0.38896, "eta": "0:02:33", "loss": 0.07997, "lr": 0.10000, "mode": "train"}
[05/16 00:42:50][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "8", "cur_iter": "60", "dt": 0.39696, "dt_data": 0.00808, "dt_net": 0.38887, "eta": "0:02:29", "loss": 0.08590, "lr": 0.10000, "mode": "train"}
[05/16 00:42:54][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "8", "cur_iter": "70", "dt": 0.39877, "dt_data": 0.00808, "dt_net": 0.39069, "eta": "0:02:26", "loss": 0.09332, "lr": 0.10000, "mode": "train"}
[05/16 00:42:58][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "8", "cur_iter": "80", "dt": 0.39574, "dt_data": 0.00806, "dt_net": 0.38767, "eta": "0:02:21", "loss": 0.08480, "lr": 0.10000, "mode": "train"}
[05/16 00:43:02][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "8", "cur_iter": "90", "dt": 0.39745, "dt_data": 0.00814, "dt_net": 0.38931, "eta": "0:02:17", "loss": 0.09758, "lr": 0.10000, "mode": "train"}
[05/16 00:43:06][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "8", "cur_iter": "100", "dt": 0.39786, "dt_data": 0.00806, "dt_net": 0.38980, "eta": "0:02:14", "loss": 0.09326, "lr": 0.10000, "mode": "train"}
[05/16 00:43:10][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "8", "cur_iter": "110", "dt": 0.39693, "dt_data": 0.00813, "dt_net": 0.38880, "eta": "0:02:09", "loss": 0.06707, "lr": 0.10000, "mode": "train"}
[05/16 00:43:14][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "8", "cur_iter": "120", "dt": 0.39716, "dt_data": 0.00812, "dt_net": 0.38904, "eta": "0:02:05", "loss": 0.07063, "lr": 0.10000, "mode": "train"}
[05/16 00:43:18][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "8", "cur_iter": "130", "dt": 0.39585, "dt_data": 0.00809, "dt_net": 0.38776, "eta": "0:02:01", "loss": 0.08474, "lr": 0.10000, "mode": "train"}
[05/16 00:43:22][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "8", "cur_iter": "140", "dt": 0.39736, "dt_data": 0.00806, "dt_net": 0.38930, "eta": "0:01:58", "loss": 0.07582, "lr": 0.10000, "mode": "train"}
[05/16 00:43:26][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "8", "cur_iter": "150", "dt": 0.39762, "dt_data": 0.00814, "dt_net": 0.38948, "eta": "0:01:54", "loss": 0.06391, "lr": 0.10000, "mode": "train"}
[05/16 00:43:30][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "8", "cur_iter": "160", "dt": 0.39581, "dt_data": 0.00813, "dt_net": 0.38767, "eta": "0:01:49", "loss": 0.09297, "lr": 0.10000, "mode": "train"}
[05/16 00:43:34][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "8", "cur_iter": "170", "dt": 0.39667, "dt_data": 0.00809, "dt_net": 0.38858, "eta": "0:01:45", "loss": 0.08782, "lr": 0.10000, "mode": "train"}
[05/16 00:43:38][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "8", "cur_iter": "180", "dt": 0.39645, "dt_data": 0.00806, "dt_net": 0.38839, "eta": "0:01:41", "loss": 0.08393, "lr": 0.10000, "mode": "train"}
[05/16 00:43:42][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "8", "cur_iter": "190", "dt": 0.39781, "dt_data": 0.00809, "dt_net": 0.38972, "eta": "0:01:38", "loss": 0.06571, "lr": 0.10000, "mode": "train"}
[05/16 00:43:46][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "8", "cur_iter": "200", "dt": 0.39623, "dt_data": 0.00806, "dt_net": 0.38817, "eta": "0:01:33", "loss": 0.09078, "lr": 0.10000, "mode": "train"}
[05/16 00:43:50][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "8", "cur_iter": "210", "dt": 0.39784, "dt_data": 0.00808, "dt_net": 0.38975, "eta": "0:01:30", "loss": 0.10442, "lr": 0.10000, "mode": "train"}
[05/16 00:43:54][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "8", "cur_iter": "220", "dt": 0.39783, "dt_data": 0.00809, "dt_net": 0.38974, "eta": "0:01:26", "loss": 0.10400, "lr": 0.10000, "mode": "train"}
[05/16 00:43:58][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "8", "cur_iter": "230", "dt": 0.39708, "dt_data": 0.00806, "dt_net": 0.38901, "eta": "0:01:22", "loss": 0.10211, "lr": 0.10000, "mode": "train"}
[05/16 00:44:01][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "8", "cur_iter": "240", "dt": 0.39729, "dt_data": 0.00809, "dt_net": 0.38919, "eta": "0:01:18", "loss": 0.07454, "lr": 0.10000, "mode": "train"}
[05/16 00:44:05][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "8", "cur_iter": "250", "dt": 0.39669, "dt_data": 0.00808, "dt_net": 0.38860, "eta": "0:01:14", "loss": 0.08429, "lr": 0.10000, "mode": "train"}
[05/16 00:44:09][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "8", "cur_iter": "260", "dt": 0.39891, "dt_data": 0.00810, "dt_net": 0.39081, "eta": "0:01:10", "loss": 0.06451, "lr": 0.10000, "mode": "train"}
[05/16 00:44:13][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "8", "cur_iter": "270", "dt": 0.39748, "dt_data": 0.00809, "dt_net": 0.38939, "eta": "0:01:06", "loss": 0.06942, "lr": 0.10000, "mode": "train"}
[05/16 00:44:17][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "8", "cur_iter": "280", "dt": 0.39869, "dt_data": 0.00809, "dt_net": 0.39059, "eta": "0:01:02", "loss": 0.07384, "lr": 0.10000, "mode": "train"}
[05/16 00:44:21][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "8", "cur_iter": "290", "dt": 0.39756, "dt_data": 0.00808, "dt_net": 0.38947, "eta": "0:00:58", "loss": 0.09228, "lr": 0.10000, "mode": "train"}
[05/16 00:44:25][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "8", "cur_iter": "300", "dt": 0.39553, "dt_data": 0.00816, "dt_net": 0.38737, "eta": "0:00:54", "loss": 0.08333, "lr": 0.10000, "mode": "train"}
[05/16 00:44:29][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "8", "cur_iter": "310", "dt": 0.39681, "dt_data": 0.00808, "dt_net": 0.38873, "eta": "0:00:50", "loss": 0.06485, "lr": 0.10000, "mode": "train"}
[05/16 00:44:33][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "8", "cur_iter": "320", "dt": 0.39729, "dt_data": 0.00816, "dt_net": 0.38913, "eta": "0:00:46", "loss": 0.08500, "lr": 0.10000, "mode": "train"}
[05/16 00:44:37][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "8", "cur_iter": "330", "dt": 0.39732, "dt_data": 0.00807, "dt_net": 0.38924, "eta": "0:00:42", "loss": 0.08151, "lr": 0.10000, "mode": "train"}
[05/16 00:44:41][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "8", "cur_iter": "340", "dt": 0.39781, "dt_data": 0.00809, "dt_net": 0.38971, "eta": "0:00:38", "loss": 0.08254, "lr": 0.10000, "mode": "train"}
[05/16 00:44:45][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "8", "cur_iter": "350", "dt": 0.39512, "dt_data": 0.00806, "dt_net": 0.38705, "eta": "0:00:34", "loss": 0.09577, "lr": 0.10000, "mode": "train"}
[05/16 00:44:49][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "8", "cur_iter": "360", "dt": 0.39709, "dt_data": 0.00808, "dt_net": 0.38902, "eta": "0:00:30", "loss": 0.08782, "lr": 0.10000, "mode": "train"}
[05/16 00:44:53][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "8", "cur_iter": "370", "dt": 0.39803, "dt_data": 0.00809, "dt_net": 0.38994, "eta": "0:00:26", "loss": 0.08364, "lr": 0.10000, "mode": "train"}
[05/16 00:44:57][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "8", "cur_iter": "380", "dt": 0.39718, "dt_data": 0.00809, "dt_net": 0.38910, "eta": "0:00:22", "loss": 0.09814, "lr": 0.10000, "mode": "train"}
[05/16 00:45:01][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "8", "cur_iter": "390", "dt": 0.39676, "dt_data": 0.00807, "dt_net": 0.38869, "eta": "0:00:18", "loss": 0.08543, "lr": 0.10000, "mode": "train"}
[05/16 00:45:05][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "8", "cur_iter": "400", "dt": 0.39671, "dt_data": 0.00809, "dt_net": 0.38862, "eta": "0:00:14", "loss": 0.07936, "lr": 0.10000, "mode": "train"}
[05/16 00:45:09][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "8", "cur_iter": "410", "dt": 0.39768, "dt_data": 0.00812, "dt_net": 0.38956, "eta": "0:00:10", "loss": 0.06991, "lr": 0.10000, "mode": "train"}
[05/16 00:45:13][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "8", "cur_iter": "420", "dt": 0.39833, "dt_data": 0.00811, "dt_net": 0.39022, "eta": "0:00:06", "loss": 0.08326, "lr": 0.10000, "mode": "train"}
[05/16 00:45:17][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "8", "cur_iter": "430", "dt": 0.39583, "dt_data": 0.00811, "dt_net": 0.38771, "eta": "0:00:02", "loss": 0.09978, "lr": 0.10000, "mode": "train"}
[05/16 00:45:19][INFO] train_net.py: 491: Epoch 7 takes 174.03s. Epochs from 0 to 7 take 174.22s in average and 174.01s in median.
[05/16 00:45:19][INFO] train_net.py: 497: For epoch 7, each iteraction takes 0.40s in average. From epoch 0 to 7, each iteraction takes 0.40s in average.
[05/16 00:45:25][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "9", "cur_iter": "10", "dt": 0.39646, "dt_data": 0.00809, "dt_net": 0.38838, "eta": "0:02:49", "loss": 0.09244, "lr": 0.10000, "mode": "train"}
[05/16 00:45:29][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "9", "cur_iter": "20", "dt": 0.39657, "dt_data": 0.00809, "dt_net": 0.38848, "eta": "0:02:45", "loss": 0.09451, "lr": 0.10000, "mode": "train"}
[05/16 00:45:33][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "9", "cur_iter": "30", "dt": 0.39826, "dt_data": 0.00809, "dt_net": 0.39017, "eta": "0:02:42", "loss": 0.07407, "lr": 0.10000, "mode": "train"}
[05/16 00:45:36][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "9", "cur_iter": "40", "dt": 0.39599, "dt_data": 0.00813, "dt_net": 0.38785, "eta": "0:02:37", "loss": 0.08329, "lr": 0.10000, "mode": "train"}
[05/16 00:45:40][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "9", "cur_iter": "50", "dt": 0.39741, "dt_data": 0.00808, "dt_net": 0.38933, "eta": "0:02:33", "loss": 0.06601, "lr": 0.10000, "mode": "train"}
[05/16 00:45:44][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "9", "cur_iter": "60", "dt": 0.39676, "dt_data": 0.00808, "dt_net": 0.38867, "eta": "0:02:29", "loss": 0.10517, "lr": 0.10000, "mode": "train"}
[05/16 00:45:48][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "9", "cur_iter": "70", "dt": 0.39729, "dt_data": 0.00819, "dt_net": 0.38909, "eta": "0:02:25", "loss": 0.07568, "lr": 0.10000, "mode": "train"}
[05/16 00:45:52][INFO] logging.py: 99: json_stats: {"_type": "train_iter", "cur_epoch": "9", "cur_iter": "80", "dt": 0.39602, "dt_data": 0.00813, "dt_net": 0.38788, "eta": "0:02:21", "loss": 0.09305, "lr": 0.10000, "mode": "train"}
1429

被折叠的 条评论
为什么被折叠?



