mongodb

前一天介紹 json 格式，所以今天介紹常常拿來存 json 格式的 mongodb．mongodb 是一種 Document database，至於詳細介紹有興趣的可以在 google 看看．這邊使用使用 docker 方式測試 Mongo DB．並使用 go 來對 mongodb 做操作

docker pull mongo

建立一個 docker-compose.yml，使用 docker-compose 的方式啟動 container．

cd /Users/daniel/mongo-docker
touch docker-compose.yml

docker-compose.yml 內容(/Users/daniel/mongo-docker) :

version: '3.1'

services:

  mongo:
    image: mongo
    restart: always
    ports:
      - 27017:27017
    environment:
      MONGO_INITDB_ROOT_USERNAME: root
      MONGO_INITDB_ROOT_PASSWORD: 123456
    volumes:
      - /Users/daniel/mongo-docker/data


  mongo-express:
    image: mongo-express
    restart: always
    ports:
      - 8081:8081
    environment:
      ME_CONFIG_MONGODB_ADMINUSERNAME: root
      ME_CONFIG_MONGODB_ADMINPASSWORD: 123456

執行 docker-compose up，啟動 mongodb container

> docker-compose up

使用 go 的 mongo-go-driver

參考 mongo-go-driver

下載 gopkg.in/mgo.v2 套件

go get gopkg.in/mgo.v2

新增及查詢 Mongo

package main

import (
  "fmt"
  "log"

  "gopkg.in/mgo.v2"
  "gopkg.in/mgo.v2/bson"
)

type User struct {
  Name string "bson:`name`"
  Age  int    "bson:`age`"
}

func main() {
  var mydb = getDB()
  mydb.Login("root", "123456")
  c := mydb.C("student")
  err := c.Insert(&User{Name: "Daniel", Age: 30})
  if err != nil {
    panic(err)
  }
  fmt.Println(err) // <nil>

  result := User{}
  err = c.Find(bson.M{"name": "Daniel"}).One(&result)
  if err != nil {
    log.Fatal(err)
  }

  fmt.Println("Age:", result.Age) // Age: 30
}

func getDB() *mgo.Database {
  session, err := mgo.Dial("127.0.0.1:27017")
  if err != nil {
    panic(err)
  }

  session.SetMode(mgo.Monotonic, true)
  db := session.DB("admin") //root user is created in the admin authentication database and given the role of root.
  return db
}

執行程式

> go run example1.go
Age: 30

進入 container

docker exec -it mongo-docker_mongo_1 bash

連進 mongo shell

mongo --host 127.0.0.1:27017 -u root -p 123456 --authenticationDatabase admin

查看剛剛執行完程式的資料是否存在

> show dbs
admin   0.000GB
config  0.000GB
local   0.000GB

> db.collection.count()
0
> db
admin
> show collections
student
> db.student.find()
{ "_id" : ObjectId("5bd314fbdac1db3ff5d54eef"), "name" : "Daniel", "age" : 30 }

read file to mongo

做個測試檔案

> cat 20181011_report
96,2288266
240,3501
144,29
192,77
193,5131
241,317
...

讀取測試檔案資料並補上 timestamp 寫入 mongodb

package main

import (
	"bufio"
	"fmt"
	"log"
	"os"
	strcv "strconv"
	"strings"
	"time"

	"gopkg.in/mgo.v2"
)

type LabelCount struct {
	LabelIndex int64     "bson:`labelIndex`"
	CreatedAt  time.Time "bson:`createdAt`"
	Count      int64     "bson:`count`"
	UpdatedAt  time.Time "bson:`updatedAt`"
}

func main() {
	fmt.Println(time.Now().Format("2006-01-02 15:04:05"))
	file, err := os.Open("/Volumes/Transcend/test/20181011_report")
	if err != nil {
		fmt.Println("error")
	}

	var mydb = getDB()
	mydb.Login("root", "123456")
	c := mydb.C("LabelCount_TEST")
	scanner := bufio.NewScanner(file)
	for scanner.Scan() {
		lineData := strings.Split(scanner.Text(), ",")
		labelIndex, err1 := strcv.ParseInt(lineData[0], 10, 64)
		if err1 != nil {
			log.Fatal(err1)
		}
		count, err2 := strcv.ParseInt(lineData[1], 10, 64)
		if err2 != nil {
			log.Fatal(err2)
		}
		err := c.Insert(&LabelCount{LabelIndex: labelIndex, CreatedAt: time.Now(), Count: count, UpdatedAt: time.Now()})
		if err != nil {
			log.Fatal(err)
		}
	}

	if err := scanner.Err(); err != nil {
		log.Fatal(err)
	}

}
func getDB() *mgo.Database {
	session, err := mgo.Dial("127.0.0.1:27017")
	if err != nil {
		panic(err)
	}

	session.SetMode(mgo.Monotonic, true)
	db := session.DB("admin") //root user is created in the admin authentication database and given the role of root.
	return db
}

執行測試

> go run example1.go
2018-10-26 21:25:59

查詢 mongodb 寫入結果

> db.LabelCount_TEST.find()
{ "_id" : ObjectId("5bbec45eb082c29547bcadff"), "labelindex" : NumberLong(96), "createdat" : ISODate("2018-10-11T03:32:46.216Z"), "count" : NumberLong(2288266), "updatedat" : ISODate("2018-10-11T03:32:46.216Z") }
{ "_id" : ObjectId("5bbec45eb082c29547bcae01"), "labelindex" : NumberLong(240), "createdat" : ISODate("2018-10-11T03:32:46.228Z"), "count" : NumberLong(3501), "updatedat" : ISODate("2018-10-11T03:32:46.228Z") }
{ "_id" : ObjectId("5bbec45eb082c29547bcae03"), "labelindex" : NumberLong(144), "createdat" : ISODate("2018-10-11T03:32:46.229Z"), "count" : NumberLong(29), "updatedat" : ISODate("2018-10-11T03:32:46.229Z") }
{ "_id" : ObjectId("5bbec45eb082c29547bcae05"), "labelindex" : NumberLong(192), "createdat" : ISODate("2018-10-11T03:32:46.229Z"), "count" : NumberLong(77), "updatedat" : ISODate("2018-10-11T03:32:46.229Z") }
...

go 的日期時間 format 是使用格式是 "2006-01-02 15:04:05" 類似 java 的 "yyyy-MM-dd hh:mm:ss"

t := time.Now()
fmt.Println(t.Format("2006-01-02 15:04:05")) // 2018-10-11 12:11:13

參考資料
go-date-time-format
MongoDb