DAY 29
0
Big Data

## 今日目標

• 了解 DCGAN
• 使用 MNIST 資料集嘗試訓練 DCGAN

Ipython Notebook 好讀完整版

## Introduction

Deep Convolutional Generative Adversarial Network 簡稱 DCGAN 中文稱作深度卷積對抗式生成網路，原始的論文連結．這是基於 GAN 上所做的工作．

(圖片出處論文)

## Implementation

### Parameters

``````batch_size = 256
g_dim = 100
``````

### Build helper functions

``````def conv2d(x, W):
return tf.nn.conv2d(x, W, strides=[1, 2, 2, 1], padding = 'SAME')

def deconv2d(x, W, output_shape):
return tf.nn.conv2d_transpose(x, W, output_shape, strides = [1, 2, 2, 1], padding = 'SAME')
``````

### Build convolutional discriminator and generator

#### Discriminator

1. 原始輸入維度 None x 28 x 28 x 1
2. 經過第一個 5 x 5 convolution filter 後輸出維度為 None x 14 x 14 x 32
3. 經過第二個 5 x 5 convolution filter 後輸出維度為 None x 7 x 7 x 64
4. 經過一個 fully connected 層變成維度 None x 1 的判斷機率輸出

#### Generator

1. 原始輸入取樣向量維度 None x 100
2. 經過第一個 fully connected 層以及 reshape 變成維度 None x 4 x 4 x 64
3. 經過第一個 5 x 5 deconvolution filter 後輸出維度為 None x 7 x 7 x 32
4. 經過第二個 5 x 5 deconvolution filter 後輸出維度為 None x 14 x 14 x 16
5. 經過第三個 5 x 5 deconvolution filter 後輸出維度為 None x 28 x 28 x 1
``````x_d = tf.placeholder(tf.float32, shape = [None, 784])
x_g = tf.placeholder(tf.float32, shape = [None, g_dim])
weights = {
"w_d1" : weight_variable([5, 5, 1, 32], "w_d1"),
"w_d2" : weight_variable([5, 5, 32, 64], "w_d2"),
"w_d3" : weight_variable([7 * 7 * 64, 1], "w_d3"),

"w_g1" : weight_variable([g_dim, 4 * 4 * 64], "w_g1"),
"w_g2" : weight_variable([5, 5, 32, 64], "w_g2"),
"w_g3" : weight_variable([5, 5, 16, 32], "w_g3"),
"w_g4" : weight_variable([5, 5, 1, 16], "w_g4")
}

biases = {
"b_d1" : bias_variable([32], "b_d1"),
"b_d2" : bias_variable([64], "b_d2"),
"b_d3" : bias_variable([1], "b_d3"),
"b_g1" : bias_variable([4 * 4 * 64], "b_g1"),
"b_g2" : bias_variable([32], "b_g2"),
"b_g3" : bias_variable([16], "b_g3"),
"b_g4" : bias_variable([1], "b_g4"),
}

var_d = [weights["w_d1"], weights["w_d2"], weights["w_d3"], biases["b_d1"], biases["b_d2"], biases["b_d3"]]
var_g = [weights["w_g1"], weights["w_g2"],weights["w_g3"], weights["w_g4"], biases["b_g1"], biases["b_g2"],biases["b_g3"], biases["b_g4"]]

``````
``````def generator(z):
#100 x 1
#-1 x 4*4*128
h_g1_reshape = tf.reshape(h_g1, [-1, 4, 4, 64])

output_shape_g2 = tf.stack([tf.shape(z)[0], 7, 7, 32])
h_g2 = tf.nn.relu(tf.add(deconv2d(h_g1_reshape, weights["w_g2"], output_shape_g2), biases["b_g2"]))

output_shape_g3 = tf.stack([tf.shape(z)[0], 14, 14, 16])
h_g3 = tf.nn.relu(tf.add(deconv2d(h_g2, weights["w_g3"], output_shape_g3), biases["b_g3"]))

output_shape_g4 = tf.stack([tf.shape(z)[0], 28, 28, 1])
h_g4 = tf.nn.tanh(tf.add(deconv2d(h_g3, weights["w_g4"], output_shape_g4), biases["b_g4"]))

return h_g4

def discriminator(x):
x_reshape = tf.reshape(x, [-1, 28, 28, 1])
# 28 x 28 x 1
# 14 x 14 x 32
# 7 x 7 x 64
h_d2_reshape = tf.reshape(h_d2, [-1, 7 * 7 * 64])
return h_d3
``````

### Build cost function

``````def sample_Z(m, n):
return np.random.uniform(-1., 1., size=[m, n])

g_sample = generator(x_g)
d_real= discriminator(x_d)
d_fake = discriminator(g_sample)

d_loss = -tf.reduce_mean(tf.log(d_real) + tf.log(1. - d_fake))
g_loss = -tf.reduce_mean(tf.log(d_fake))
``````
``````# 只更新 discriminator
# 只更新 generator parameters
``````

### Training

``````step 0, discriminator loss 2.41063  generator loss 0.29276
step 100, discriminator loss 0.85784  generator loss 1.39115
step 200, discriminator loss 0.44820  generator loss 2.21131
step 300, discriminator loss 0.84276  generator loss 1.69659
step 400, discriminator loss 0.72615  generator loss 1.95878
step 500, discriminator loss 0.71053  generator loss 1.77434
step 600, discriminator loss 0.71514  generator loss 1.84341
step 700, discriminator loss 0.51880  generator loss 1.86431
step 800, discriminator loss 0.26334  generator loss 2.29687
step 900, discriminator loss 0.22319  generator loss 2.62868
step 1000, discriminator loss 0.18192  generator loss 2.64233
step 2000, discriminator loss 0.24685  generator loss 2.95130
step 3000, discriminator loss 0.05564  generator loss 4.25037
step 4000, discriminator loss 0.04217  generator loss 4.71265
step 5000, discriminator loss 0.22856  generator loss 3.57268
step 6000, discriminator loss 0.12961  generator loss 4.49799
step 7000, discriminator loss 0.04830  generator loss 5.12374
step 8000, discriminator loss 0.01379  generator loss 6.38079
step 9000, discriminator loss 0.00626  generator loss 5.92065
step 10000, discriminator loss 0.05310  generator loss 6.42939
step 11000, discriminator loss 0.01101  generator loss 7.24146
step 12000, discriminator loss 0.00744  generator loss 6.53067
step 13000, discriminator loss 0.01497  generator loss 8.72534
step 14000, discriminator loss 0.05123  generator loss 5.25377
step 15000, discriminator loss 0.00633  generator loss 6.64243
step 16000, discriminator loss 0.00066  generator loss 7.72938
step 17000, discriminator loss 0.00209  generator loss 7.22791
step 18000, discriminator loss 0.00046  generator loss 7.68677
step 19000, discriminator loss 0.00222  generator loss 6.73453
step 20000, discriminator loss 0.00914  generator loss 5.61193
``````

1. 只出現了數字 1，沒有其他的數字，怪哉．
2. 到了 step 18000 突然生成式網路就壞掉了，然後又好了．

## 小結

### 問題

• 尋找生成式網路崩壞還有只出現某一數字的原因．
• 調整生成式網路的層數
• 調整生成式網路各層的 channels

## 學習資源連結

tensorflow 學習筆記30