iT邦幫忙

2022 iThome 鐵人賽

DAY 11
0
Software Development

SQL rookie 之天天魯一下系列 第 11

Day 11 - 插話:利用SampleDB seeding 資料(3)

  • 分享至 

  • xImage
  •  

哈囉,大家好!

愉快的週末就要來做些廢廢的事情,如練習語法?

接續前兩日的研究,目前被翻爛的書知道了能用JSONPlaceholder來seeding sample data,也知道可選用適合的HTTP client parse JSON API,昨天也順利地seeding 了Post data。

今天則是想極致偷懶,讓seeding task 能自動根據attributes 去抓API response 相對的value,而前置作業是要先乖乖將各個Model 的attributes 定義好。根據Guide 下方列的資訊,目前有:
/posts/1/comments
/albums/1/photos
/users/1/albums
/users/1/todos
/users/1/posts

簡單畫一下其關聯圖:
JSONPlaceholder ER圖

好了後就開始來處理tables 的migration 囉:

  create_table "albums", force: :cascade do |t|
    t.string "userId"
    t.text "title"
    t.datetime "created_at", precision: 6, null: false
    t.datetime "updated_at", precision: 6, null: false
  end

  create_table "comments", force: :cascade do |t|
    t.integer "postId"
    t.string "name"
    t.text "email"
    t.text "body"
    t.datetime "created_at", precision: 6, null: false
    t.datetime "updated_at", precision: 6, null: false
  end

  create_table "photos", force: :cascade do |t|
    t.integer "albumId"
    t.text "title"
    t.text "url"
    t.text "thumbnailUrl"
    t.datetime "created_at", precision: 6, null: false
    t.datetime "updated_at", precision: 6, null: false
  end

  create_table "posts", force: :cascade do |t|
    t.integer "userId"
    t.text "title"
    t.text "body"
    t.datetime "created_at", precision: 6, null: false
    t.datetime "updated_at", precision: 6, null: false
  end

  create_table "todos", force: :cascade do |t|
    t.integer "userId"
    t.text "title"
    t.boolean "completed"
    t.datetime "created_at", precision: 6, null: false
    t.datetime "updated_at", precision: 6, null: false
  end

  create_table "users", force: :cascade do |t|
    t.datetime "created_at", precision: 6, null: false
    t.datetime "updated_at", precision: 6, null: false
    t.string "name"
    t.string "username"
    t.text "email"
    t.text "phone"
    t.text "website"
  end

接著我們一樣先建立一個task 來一步步修改:

# 根據昨天的經驗,我們能先寫出一個基本版:

# Seeding post
url = 'https://jsonplaceholder.typicode.com/posts'
post_data = HTTParty.get(url).parsed_response

post_data.each do |post|
  Post.create(userId: post["userId"], title: post["title"], body: post["body"])
end
puts "Seeding post completed"

# Seeding comment
url = 'https://jsonplaceholder.typicode.com/comments'
comment_data = HTTParty.get(url).parsed_response

comment_data.each do |post|
  Comment.create(postId: comment["postId"], name: comment["name"], email: comment["email"], body: comment["body"])
end
puts "Seeding comment completed"

但由以上的兩個blocks 來看,如果我們要seeding 這六個Model 的data,就會有六個類似的blocks,且每個欄位都需要手動輸入column name 和response key,這離懶人高尚的境界相去甚遠...

因此,我想先將需要的table name、Model name、url 尾綴整理一下:

models = [ 'post' ] # 之後會擴充加入其他Models

# 1. 先處理Model name 的轉換
models.map do |model_name|
  model_name = model_name.humanize # 能把首字大寫,但因為本身還是String,所以需要利用const_get 方法來轉換為Model name
  model_name.class
end

> String < Object  
 
models.map do |model_name|
  model_name = Class.const_get(model_name.humanize)
  model_name.class
end

> Class < Module

# 2. 再來處理url 的尾綴
url = "https://jsonplaceholder.typicode.com/#{model_name}s"
> "https://jsonplaceholder.typicode.com/posts"

接著我們要處理建立資料表時會自動帶入的欄位:[ "id", "created_at", "updated_at" ]
這裡可以用attributes.keys這方法來檢視:

models = [ 'post' ] # 懶人的最終目標是只想改這行 XD

models.map do |model_name|
  model_class_name = Class.const_get(model_name.humanize)
  model_attributes = model_class_name.new.attributes.keys
end

>>
        [0] "id",
        [1] "userId",
        [2] "title",
        [3] "body",
        [4] "created_at",
        [5] "updated_at"

# 來刪掉那些多餘的欄位
models = [ 'post' ]
models.map do |model_name|
  model_class_name = Class.const_get(model_name.humanize)
  model_attributes = model_class_name.new.attributes.keys - [ "id", "created_at", "updated_at" ]
end

>>
        [0] "userId",
        [1] "title",
        [2] "body"

再來則是要處理API response 內多餘的keys & values,如users 因帶有address、company 等我暫時沒用到的keys,也就是User.attributes 沒有,若整包當成Model.create 的params 則會噴錯。

先給大家看看沒處理時的狀況:

models = [ 'user' ]
models.map do |model_name|
  model_class_name = Class.const_get(model_name.humanize)
  model_attributes = model_class_name.new.attributes.keys - [ "id", "created_at", "updated_at" ]
  url = "https://jsonplaceholder.typicode.com/#{model_name}s"
  response = HTTParty.get(url).parsed_response

  response.each do |res|
    model_class_name.create(res) # 整個response 丟進去
  end
end

Traceback (most recent call last):
        5: from (irb):66
        4: from (irb):66:in `map'
        3: from (irb):72:in `block in irb_binding'
        2: from (irb):72:in `each'
        1: from (irb):73:in `block (2 levels) in irb_binding'
ActiveModel::UnknownAttributeError (unknown attribute 'address' for User.)

讓我們再度利用上面model_attributes 的attributes 吧!並搭配ruby Hash 去掉keys 的方法except:

except(*keys) public
Returns a hash that includes everything except given keys.

models = [ 'user' ]
models.map do |model_name|
  model_class_name = Class.const_get(model_name.humanize)
  model_attributes = model_class_name.new.attributes.keys - [ "id", "created_at", "updated_at" ]
  url = "https://jsonplaceholder.typicode.com/#{model_name}s"
  response = HTTParty.get(url).parsed_response

  response.each do |res|
    keys_to_delete = res.keys - model_attributes
    res = res.except(*keys_to_delete) # I'm here
    model_class_name.create(res)
  end
end

最後再加上fail 的驗證和puts seeding 的處理狀況,咱們就完成啦!

Apidock: const_get
Apidock: except(for hash)

# 偷懶完成品
models = [ 'post', 'comment', 'album', 'photo', 'todo', 'user' ]
models.map do |model_name|

  model_class_name = Class.const_get(model_name.humanize)
  model_attributes = model_class_name.new.attributes.keys - [ "id", "created_at", "updated_at" ]

  url = "https://jsonplaceholder.typicode.com/#{model_name}s"
  response = HTTParty.get(url).parsed_response

  response.each do |res|
    keys_to_delete = res.keys - model_attributes
    res = res.except(*keys_to_delete)
    model_class_name.create(res)
  end

  fail unless model_class_name.count == response.count

  puts "Seeding #{model_name}s completed."
end

有沒有很懶呢?XD 希望大家都能成為怕麻煩的極致懶人~~!


上一篇
Day 10 - 插話:利用SampleDB seeding 資料(2)
下一篇
Day 12 - 利用SampleDB seeding 資料(4)
系列文
SQL rookie 之天天魯一下30
圖片
  直播研討會
圖片
{{ item.channelVendor }} {{ item.webinarstarted }} |
{{ formatDate(item.duration) }}
直播中

尚未有邦友留言

立即登入留言