TFN--utils.py

最新推荐文章于 2024-09-15 22:31:42 发布

好好学习ya

最新推荐文章于 2024-09-15 22:31:42 发布

阅读量397

点赞数

文章标签： python 深度学习 pytorch

本文链接：https://blog.csdn.net/include1121/article/details/121561202

版权

本文档详细介绍了在深度学习项目中广泛使用的PyTorch实用工具文件utils.py。涵盖关键功能，如数据预处理、损失计算和模型保存。

摘要由CSDN通过智能技术生成

utils.py

'''
This script does all the data preprocessing.
You'll need to install CMU-Multimodal DataSDK 
(https://github.com/A2Zadeh/CMU-MultimodalDataSDK) to use this script.
There's a packaged (and more up-to-date) version
of the utils below at https://github.com/Justin1904/tetheras-utils.
Preprocessing multimodal data is really tiring...
'''
from __future__ import print_function
import mmdata
import numpy as np
from torch.utils.data import Dataset

def pad(data, max_len):
    """Pads data without time stamps"""
    data = remove_timestamps(data)
    n_rows = data.shape[0]
    dim = data.shape[1]
    if max_len >= n_rows:
        diff = max_len - n_rows
        padding = np.zeros((diff, dim))
        padded = np.concatenate((padding, data))
        return padded
    else:
        return data[-max_len:]

def remove_timestamps(segment_data):
    """Removes the start and end time stamps in the Multimodal Data SDK"""
    return np.array([feature[2] for feature in segment_data])

class ProcessedDataset(Dataset):
    """The class object for processed data, pipelined from CMU-MultimodalDataSDK through MultimodalDataset"""
    def __init__(self, audio, visual, text, labels):
        self.audio = audio
        self.visual = visual
        self.text = text
        self.labels = labels

    def __len__(self):
        """Checks the number of data points are the same across different modalities, and return length"""
        assert self.audio.shape[1] == self.visual.shape[1] and self.visual.shape[1] == self.text.shape[1] and self.text.shape[1] == self.labels.shape[0]
        return self.audio.shape[1]

    def __getitem__(self, idx):
        """Returns the target element by index"""
        return [self.audio[:, idx, :], self.visual[:, idx, :], self