Eclipse's Blog
0%

学习小土堆pytorch教程记录-pytorch基础用法

发表于 更新于 分类于

继续学了，注意要多看官方文档，看源码，看模块的输入和输出

1. python文件、python控制台和jupter的不同

运行同一块代码：

1
2
3
4
5
print("Start")
a = "hello world"
b = "2024"
c = a + b
print(c)

python文件是以整体代码为一块运行的
python控制台是以每一行为一块运行的（也能以任意行运行）
jupyter是以任意行为块运行的
后两者，当某一块出错时，修改后前面的块并不需要再次运行。

运行方式 优点 缺点
python文件 传播方便、适用大型项目 需要从头运行
python控制台 显示每个变量属性 不利于代码阅读和修改
jupyter 利于代码阅读和修改 环境需要配置

2. pytorch加载数据

使用Dataset加载处理图像数据

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
from torch.utils.data import Dataset
from PIL import Image
import os

# help(Dataset)

# 继承dataset类
class MyData(Dataset):
    def __init__(self, root_dir, label_dir): 
        self.root_dir = root_dir
        self.label_dir = label_dir
        self.path = os.path.join(self.root_dir, self.label_dir)
        self.img_list = os.listdir(self.path)
        

    def __getitem__(self, idx):
        img_name = self.img_list[idx]
        img_item_path = os.path.join(self.path, img_name)
        img = Image.open(img_item_path)
        label = self.label_dir
        return img, label
    
    def __len__(self):
        return len(self.img_list)
    
# 实例化对象
root_dir = "dataset/train"
ants_label_dir = "ants"
bees_label_dir = "bees"
ants_dataset = MyData(root_dir, ants_label_dir)
bees_dataset = MyData(root_dir, bees_label_dir)

img, label = ants_dataset[1]
# img.show()

train_data = ants_dataset + bees_dataset
print(len(train_data))

img, label = train_data[125]
img.show()

3. Tensorboard的使用

安装tensorboard
conda install tensorboard
作用：画网格结构图，精确率、学习率、损失曲线

add_scalar示例：

1
2
3
4
5
6
7
8
from torch.utils.tensorboard import SummaryWriter

writer = SummaryWriter("logs")

for i in range(100):
    writer.add_scalar("y=2x", i * 2, i)

writer.close()

打开tensorboard查看：
(pytorch) D:\learning\pytorch> tensorboard –logdir=logs
指定端口打开：
(pytorch) D:\learning\pytorch>tensorboard –logdir=logs –port=6007

add_image示例：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
from torch.utils.tensorboard import SummaryWriter
import numpy as np
from PIL import Image

writer = SummaryWriter("logs")

# for i in range(100):
#     writer.add_scalar("y=2x", i * 3, i)

# writer.close()

img_path = "dataset/train/bees_image/16838648_415acd9e3f.jpg"
img = Image.open(img_path)
print(type(img))
numpy_img = np.array(img)
print(numpy_img.shape)
writer.add_image("test", numpy_img, 2, dataformats="HWC")

writer.close()

3. Transform的使用

transform.ToTensor示例：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
from torch.utils.tensorboard import SummaryWriter 
from torchvision import transforms
from PIL import Image


img_path = "dataset/train/ants_image/0013035.jpg"
img = Image.open(img_path)

writer = SummaryWriter("logs")


# 如何使用transforms
# tensor_img = transforms.ToTensor()(img)
tensor_trans = transforms.ToTensor()
tensor_img = tensor_trans(img)
print(tensor_img)

writer.add_image("Tensor_img", tensor_img)

writer.close()

4. 常见的transforms

Normalize的使用

作用是将输入图像的像素值缩放到一个标准化的范围。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
from torch.utils.tensorboard import SummaryWriter 
from PIL import Image
from torchvision import transforms

img_path = "dataset/train/ants_image/0013035.jpg"
img = Image.open(img_path)

writer = SummaryWriter("logs")
tensor_trans = transforms.ToTensor()
tensor_img = tensor_trans(img)

print(tensor_img[0][0][0])
# Normalize使用
trans_norm = transforms.Normalize([0.5, 0.5, 0.5], [0.5, 0.5, 0.5])
img_norms =trans_norm(tensor_img)
print(img_norms[0][0][0])
writer.add_image("Normalize_img", img_norms)

writer.close()

Resize的使用

1
2
3
4
5
6
7
8
9
# Resize使用
print(img.size)
trans_resize = transforms.Resize((512, 512))
# img PIL -> Resize -> img_resize PIL
img_resize = trans_resize(img)
# img_resize PIL -> ToTensor -> img_resize tensor
img_resize = tensor_trans(img_resize)
print(img_resize.size)
writer.add_image("Resize_img", img_resize)

Compose的使用

作用时将多个图像变换操作组合在一起，使得它们可以依次对图像进行处理。前一个操作的输出作为后一个操作的输入。

1
2
3
4
5
6
# Compose -> Resize -> ToTensor
trans_resize_2 = transforms.Resize(512)
# PIL -> PIL -> Tensor
trans_compose = transforms.Compose([trans_resize_2, tensor_trans])
img_resize_2 = trans_compose(img)
writer.add_image("Resize_img_2", img_resize_2)

RandomCrop使用

作用是从图像中随机裁剪出一个指定大小的区域

1
2
3
4
5
6
7
# RandomCrop使用
trans_crop = transforms.RandomCrop(256)
trans_compose_2 = transforms.Compose([trans_crop, tensor_trans])

for i in range(10):
    img_crop = trans_compose_2(img)
    writer.add_image("RandomCrop_img", img_crop, i)

5. 使用torchvision提供的数据集

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
from torch.utils.tensorboard import SummaryWriter
import torchvision

dataset_transform = torchvision.transforms.Compose([
    torchvision.transforms.ToTensor()
])
writer = SummaryWriter("logs")

train_set = torchvision.datasets.CIFAR10(root="./dataset", train=True, transform=dataset_transform, download=True)
test_set = torchvision.datasets.CIFAR10(root="./dataset", train=False, transform=dataset_transform, download=True)

# print(train_set[0])
# print(train_set.classes)
# img, target = train_set[0]
# print(img)
# print(target)
# img.show()

print(test_set[1])
for i in range(10):
    img, target = test_set[i]
    writer.add_image("test_set", img, i)

writer.close()

6. DataLoader的使用

用于从 Dataset中批量加载数据，支持批量处理、打乱和并行加载等功能。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
from torch.utils.tensorboard import SummaryWriter
from torch.utils.data import DataLoader
import torchvision

# 准备的测试数据集
test_set = torchvision.datasets.CIFAR10(root="./dataset", train=False, transform=torchvision.transforms.ToTensor(), download=True)
# drop_last=True: 如果最后一个batch的数据量小于batch_size，就把这个batch丢掉
# shuffle=True: 每个epoch都打乱数据
test_loader = DataLoader(test_set, batch_size=64, shuffle=True, num_workers=0, drop_last=False)

writer = SummaryWriter("logs")

# 测试数据中的第一张图片和target
img, target = test_set[0]
print(img.shape)
print(target)

for epoch in range(2):
    step = 0
    for data in test_loader:
        imgs, targets = data
        # print(imgs.shape)
        # print(targets)
        writer.add_images("Epoch:{}".format(epoch), imgs, step)
        step += 1

writer.close()