Part0:前言

連續幾天睡眠不足，今天好像到一個極限要早睡了~實在是每天鐵人賽發文完，都要好好滑手機紓壓一下，弄得自己睡眠不足(咎由自取XDD 今天下班後實在太睏了，決定今天盡快發文完成任務，早早收工!

Part1:今日目標

1.認識模型學習流程
2.任務應用:LeNet模型如何預測手寫辨識資料集

Part2:內容

1.模型學習流程

圖片來源

流程圖如上，步驟說明如下:
Step1:根據輸入資料輸出預測結果，此為前向傳播(forward pass)。
Step2:定義模型，透過損失函數(loss function)計算預測值和實際值的誤差(損失，loss)。
Step3:模型計算參數改變時對誤差的變化量，即計算誤差對模型參數的梯度(gradient)，該方法是使用連鎖律(chain rule)由後往前對每一神經層函數求導數，因此過程又為反向傳播(backpropagation, backward pass)。
Step4:模型參數會往讓誤差變小的方向前進，進行參數更新。
以上流程會重複進行，直到預測誤差達到可接受範圍。

總結:

圖片來源

前向傳播(forward pass): 模型運算&計算損失
反向傳播(backward pass): 計算梯度&更新參數

2.任務應用:LeNet模型如何預測手寫辨識資料集

將引用網路資源對該任務模型學習流程作解釋說明。

Step1:載入資料(Load the data)&匯入函式庫(Import the libraries)
資料集介紹: MNIST手寫數字資料集，每個圖片大小為28x28，訓練集有6萬張圖片，測試集則為1萬張，圖片為黑白(通道數為1)，共有10類數字(數字0~9)。
圖片來源

# Load in relevant libraries, and alias where appropriate
import torch
import torch.nn as nn
import torchvision
import torchvision.transforms as transforms

# Define relevant variables for the ML task
batch_size = 64  # 每個batch有64張圖片
num_classes = 10  # 圖片共分成10種類別
learning_rate = 0.001  # 學習率
num_epochs = 10  #　訓練總共要跑的回合數，一回合(epoch)即將所有訓練數據都掃過一遍

# Device will determine whether to run the training on GPU or CPU.
device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')

#Loading the dataset and preprocessing
train_dataset = torchvision.datasets.MNIST(root = './data',
       train = True,  # 訓練集
       transform = transforms.Compose([
          transforms.Resize((32,32)),  # 將圖片從原先大小28x28改成LeNet可以接受的輸入大小32x32
          transforms.ToTensor(),  # 轉換成tensor並且將像素範圍(range)從[0, 255]改到[0,1]
          transforms.Normalize(mean = (0.1307,), std = (0.3081,))]),  # 對資料按照通道進行標準化
       download = True)  #　若已經下載過就不再下載


test_dataset = torchvision.datasets.MNIST(root = './data',
      train = False,  # 測試集
      transform = transforms.Compose([  # 與訓練集做相同的處理
          transforms.Resize((32,32)),
          transforms.ToTensor(),
          transforms.Normalize(mean = (0.1325,), std = (0.3105,))]),
      download=True)

# 打包成batch
train_loader = torch.utils.data.DataLoader(dataset = train_dataset,
                                           batch_size = batch_size,
                                           shuffle = True)


test_loader = torch.utils.data.DataLoader(dataset = test_dataset,
                                           batch_size = batch_size,
                                           shuffle = True)

Step2:定義模型LeNet&設定超參數
(1)LeNet模型
詳細說明可參考昨日的鐵人賽文章:D12-Lenet模型_模型架構學習1st。

#Defining the convolutional neural network
class LeNet5(nn.Module):
    def __init__(self, num_classes):
        super(ConvNeuralNet, self).__init__()
        self.layer1 = nn.Sequential(
            nn.Conv2d(1, 6, kernel_size=5, stride=1, padding=0),
            nn.BatchNorm2d(6),
            nn.ReLU(),
            nn.MaxPool2d(kernel_size = 2, stride = 2))
        self.layer2 = nn.Sequential(
            nn.Conv2d(6, 16, kernel_size=5, stride=1, padding=0),
            nn.BatchNorm2d(16),
            nn.ReLU(),
            nn.MaxPool2d(kernel_size = 2, stride = 2))
        self.fc = nn.Linear(400, 120)
        self.relu = nn.ReLU()
        self.fc1 = nn.Linear(120, 84)
        self.relu1 = nn.ReLU()
        self.fc2 = nn.Linear(84, num_classes)
        
    def forward(self, x):
        out = self.layer1(x)
        out = self.layer2(out)
        out = out.reshape(out.size(0), -1)
        out = self.fc(out)
        out = self.relu(out)
        out = self.fc1(out)
        out = self.relu1(out)
        out = self.fc2(out)
        return out

(2)設定超參數(Hyperparameters): 損失函數(loss function)、優化器(optimizer)

model = LeNet5(num_classes).to(device)

#Setting the loss function
cost = nn.CrossEntropyLoss()  # 交叉墒損失函數，適用多分類任務的損失函數

#Setting the optimizer with the model parameters and learning rate
optimizer = torch.optim.Adam(model.parameters(), lr=learning_rate)  # Adam優化器

#this is defined to print how many steps are remaining when training
total_step = len(train_loader)

Step3:進行模型訓練(Training)

total_step = len(train_loader)
for epoch in range(num_epochs):  # 總共進行共num_epochs個回合的訓練
    for i, (images, labels) in enumerate(train_loader):  
        images = images.to(device)  # 將tensor移動到GPU或CPU上訓練
        labels = labels.to(device)
        
        # 前向傳播(Forward pass)
        outputs = model(images)
        loss = cost(outputs, labels)
        	
        # 反向傳播(Backward pass) and optimize
        optimizer.zero_grad()
        loss.backward()
        optimizer.step()
        		
        if (i+1) % 400 == 0:
        print ('Epoch [{}/{}], Step [{}/{}], Loss: {:.4f}'.format(epoch+1, num_epochs, i+1, total_step, loss.item()))

Step4:模型測試(Testing)

# In test phase, we don't need to compute gradients (for memory efficiency)
with torch.no_grad():
    correct = 0
    total = 0
    for images, labels in test_loader:
        images = images.to(device)
        labels = labels.to(device)
        outputs = model(images)
        _, predicted = torch.max(outputs.data, 1)
        total += labels.size(0)  # 總共預測的圖片張數
        correct += (predicted == labels).sum().item()  # 統計預測正確的圖片張數

    print('Accuracy of the network on the 10000 test images: {} %'.format(100 * correct / total))  # 模型在測試資料的預測準確率(accuracy)
	 

Part3:專案進度

完成LeNet模型認識與Pytorch code在MNIST資料集進行訓練和預測的流程。

Part4:下一步

明後兩天預計會對Pytorch的nn.Moule子類別做相關介紹和比較一些常見的影像分類模型，例如:ALXnet、ResNet等。

參考:
Writing LeNet5 from Scratch in PyTorch

心得小語:
YA~今天特別早完成，但也的確內容不像前幾天豐富，但疲勞時總要休息充足好體力，明天又是活龍一條(? 祝閱讀到這篇文的大家一切安好，明天見囉~~
今日工時: 50mins*2

休息一會，經過休養生息的田地五殼豐收 – 奧維德(詩人)
Take rest; a field that has rested gives a bountiful crop. — Ovid, Poet