目录
3.将训练数据和测试数据转化为tensor形式,为后面训练做准备,因为训练数据在pytorch要求一般为tensor格式
4.将标签转化为float型(此处是因为做的是回归任务),如果想要做分类任务的话需要将标签转化为long型!
5.将已经转化为tensor格式的(train_datas, train_labels)和(test_datas, test_labels)转化为Data.DataLoader可以使用的格式
1.从txt文档中读取训练数据和测试数据
fp = open('E:\\桌面\\test_new_3\\data\\train.txt', 'r', encoding='utf-8')
string = fp.read() # string是一行字符串,该字符串包含文件所有内容
fp.close()
row_list = string.splitlines() # splitlines默认参数是‘\n’
data_list = [[float(i) for i in row.strip().split(" ")] for row in row_list]
shuffle(data_list) ##打乱数据
data_train = np.array(data_list) # data=data.to_numpy()
fp = open('E:\\桌面\\test_new_3\\data\\test.txt', 'r', encoding='utf-8')
string = fp.read() # string是一行字符串,该字符串包含文件所有内容
fp.close()
row_list = string.splitlines() # splitlines默认参数是‘\n’
data_list = [[float(i) for i in row.strip().split(" ")] for row in row_list]
shuffle(data_list) ##打乱数据
data_test = np.array(data_list) # data=data.to_numpy()
2.从txt文件中读取标签数据
fp = open('E:\\桌面\\test_new_3\\label\\1004.txt', 'r', encoding='utf-8')
line = fp.readline()
l = line.split(",")
label1 = [[float(i)] for i in l]
label1 = np.array(label1)
fp = open('E:\\桌面\\test_new_3\\label\\355.txt', 'r', encoding='utf-8')
line = fp.readline()
l = line.split(",")
label2 = [[float(i)] for i in l]
label2 = np.array(label2)
# label1 = np.loadtxt('./284.txt', dtype=np.float32, delimiter=' ')
# print("原始红外矩阵大小为:", label1)
# label2 = np.loadtxt('./71.txt', dtype=np.float32, delimiter=' ')
# print("原始红外矩阵大小为:", label2)
train_datas = data_train
train_labels = label1
test_datas = data_test
test_labels = label2
3.将训练数据和测试数据转化为tensor形式,为后面训练做准备,因为训练数据在pytorch要求一般为tensor格式
##这几步的作用就是将数据转化为tensor的格式,为后面训练做准备(因为训练的时候数据要是tensor格式)
##同时,train_datas = train_datas / 1.0 ,这里的作用就是将其强制转化为浮点类型
train_datas = train_datas / 1.0
train_datas = torch.from_numpy(train_datas).float()
test_datas = test_datas / 1.0
test_datas = torch.from_numpy(test_datas).float()
4.将标签转化为float型(此处是因为做的是回归任务),如果想要做分类任务的话需要将标签转化为long型!
train_labels = torch.from_numpy(train_labels).float()
test_labels = torch.from_numpy(test_labels).float()
5.将已经转化为tensor格式的(train_datas, train_labels)和(test_datas, test_labels)转化为Data.DataLoader可以使用的格式
torch_dataset_train = Data.TensorDataset(train_datas, train_labels)
torch_dataset_test = Data.TensorDataset(test_datas, test_labels)
print(train_datas.size()) # (60000, 28, 28)
print(train_labels.size())
print(test_datas.size())
print(test_labels.size())
6.利用Data.DataLoader来对数据集进行迭代
train_loader = Data.DataLoader(dataset=torch_dataset_train, batch_size=BATCH_SIZE, shuffle=True)
7.训练流程
a = []
for epoch in range(EPOCH):
for step, (b_x, b_y) in enumerate(train_loader): # train_loader的用法!
# print(b_x.size())
b_x = b_x.view(-1, 3)
# print(b_x.size())
output = mlp(b_x) # logistic output
loss = loss_func(output, b_y) # cross entropy loss
optimizer.zero_grad() # clear gradients for this training step
loss.backward() # backpropagation, compute gradients
optimizer.step() # apply gradients
if step % 50 == 0:
test_output = mlp(test_x.view(-1, 3))
accuracy = (((abs(test_output.detach().numpy()-test_y.numpy()) < 0.1)).sum())/len(test_y.numpy())
print('Epoch: ', round(epoch), '| train loss: %.4f' % loss.data.numpy(), '| test accuracy:%.2f' % accuracy)
a.append(accuracy)