Monodepth2 加入IMU数据

为了将IMU传感器数据融合到网络之中，需要让IMU和图片同步输入到训练网络中

1.通过pykitti读取IMU数据

在kitti_dataset.py 中KITTIRAWDateset()中通过pykitti utils.load_oxts_packet_ande_pose()获取oxts文件

    def load_oxts(self,folder,frame_index, do_flip):
        """Load OXTS data from file."""

        oxts_filename = os.path.join(
            self.data_path, folder, "oxts/data/{:010d}.txt".format(int(frame_index)))
        print(oxts_filename)
        oxts_filenames = [oxts_filename]
        #oxts = utils.load_oxts_packets_and_poses(oxts_filenames)
        R,t = utils.load_oxts_packets_and_RT(oxts_filenames)

        return  R,t

2.生成R，T，转成4X4的矩阵

在mono_dataset.py中，将生成的R，T 转成4X4矩阵，并tensor化

        R,t = self.load_oxts(folder,frame_index, do_flip)
        # pose_RT = load_oxts1[0].T_w_imu
        # R,T = load_oxts1[0].packet
        R = R.reshape(3, 3)
        t = t.reshape(3, 1)
        T_w_imu =np.vstack((np.hstack([R, t]), [0, 0, 0, 1]))

        #inputs["pose"] = np.expand_dims(T_w_imu,0)
        TW = np.expand_dims(T_w_imu,0)
        inputs["pose"] = torch.from_numpy(T_w_imu.astype(np.float32))

R =
[[-0.60702688 -0.79438127 0.0218349 ]
[ 0.79425304 -0.60556876 0.04948326]
[-0.02608604 0.04738011 0.99853625]]
t =
[[6.19561019e+05]
[4.11578006e+06]
[2.07307388e+02]]

Batchsize=2转换成tensor形式

tensor([[[-3.6927e-01, 9.2916e-01, -1.7130e-02, 6.1858e+05],
[-9.2930e-01, -3.6934e-01, -4.0503e-04, 4.1165e+06],
[-6.7029e-03, 1.5769e-02, 9.9985e-01, 1.1287e+02],
[ 0.0000e+00, 0.0000e+00, 0.0000e+00, 1.0000e+00]],
[[ 2.1411e-01, 9.7661e-01, -1.9779e-02, 6.1959e+05],
[-9.6635e-01, 2.0882e-01, -1.5020e-01, 4.1158e+06],
[-1.4256e-01, 5.1274e-02, 9.8846e-01, 2.1405e+02],
[ 0.0000e+00, 0.0000e+00, 0.0000e+00, 1.0000e+00]]],
device=‘cuda:0’)