iT邦幫忙

第 12 屆 iT 邦幫忙鐵人賽

DAY 15
0
Software Development

基於付費公有雲與開源機房自建私有雲之雲端應用服務測試兼叢集與機房託管服務實戰之勇者崎嶇波折且劍還掉在路上的試煉之路系列 第 15

Day 15 Kubernetes 耐久戰技大考驗 - Dbench on Kubernetes 安裝基礎操作篇

Day 15 Kubernetes 耐久戰技大考驗 - Dbench on Kubernetes 安裝基礎操作篇

本日重點與方向 (TAG): kubernetes、k8s、PV、persistent volume、PVC、persistent volume claim、Pod use PVC、GlusterFS、Heketi、Rancher、Rancher Longhorn、Longhorn、SC、StorageClass、dbench、dbench on kubernetes
今天將會介紹使用 Bare Metal 進行 Kubernetes 環境中持久化儲存的軟體功能對於掛載的 PV/PVC 速度測試,主要會是以 Dbench on Kubernetes 這個開源專案進行驗證測試,並對於先前組建的 kubernetes 功能進行整合,如果你中途裝一裝有一些異常與失敗的話,基本上就是參考安裝筆記最下面的重建儲存叢集,還有搭配一下前幾天的重建 kubernetes 的做法,GlusterFS 跟 Longhorn 的配置上基本上就是參考前面幾天的搭建方式去搞,搞到有 Storage Class 之後就可以去做測試,實際行為就是開 PVC 掛到 dbench 的測試 Job 之上去等結果,因為 CSI 對於 Node 直接去存取有一下速度跟延遲的差異,所以本次也會介紹用同一套系統去搞測速,也就參考一下拿來比對驗證的基礎的,後續當然還有一些進階一點的去看官方網站 吧 (然後官方的 Docker Image 壞掉了) ,詳細的操作就看我隨便寫寫亂唬一通加減弄吧。

本次使用設備資訊

Network Switch

  • 數量: 1
  • 型號: D-Link 1210-28 (L2 Switch)

Bare Metal

Master Node

  • 數量: 1
  • Ubuntu: 16.04 / 18.04
  • Docker Version: 19.03
  • CPU: E3-1230_V3 ^ 1
  • RAM: 16GB
  • OS_Disk: 120 GB (SSD)
  • Data_Disk: 500 GB (HDD)
  • Network: 1Gbps

Worker Node

  • 數量: 3
  • Ubuntu: 16.04 / 18.04
  • Docker Version: 19.03
  • CPU: E3-1230_v3
  • RAM: 8 GB
  • OS_Disk: 120 GB (SSD)
  • Data_Disk: 250 GB (HDD)
  • Network: 1Gbps

dbench on kubernetes

https://github.com/leeliu/dbench

下載 github 上面的專案

git clone https://github.com/leeliu/dbench

切入 dbench 之下進行編譯 Docker Container Image

docker image build -t dbench .
docker images

Container 匯出與匯入

docker save -o <export-tar-filename> <local-image-name>
docker load -i <export-tar-filename>

設定測試的 yaml 檔案

  • PVC:
    • spec.storageClassName: 呼叫生成 PVC 的 storage class (longhorn/glusterfs.........)
    • spec.resources.requests.storage: 切分出來測試的空間,需要比測試的大小來的大一些
  • Job:
    • spec.template.spec.containers.image: 改成剛剛 build 出來的 Container Image
    • spec.template.spec.containers.imagePullPolicy: 改成 Never,使用本地端的 Container Image
    • spec.template.spec.containers.env
      • name:FIO_SIZE
      • value: 1G (看一下需求的測試大小)
dbench.yaml
-----
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: dbench-pv-claim
spec:
  storageClassName: ssd
  # storageClassName: gp2
  # storageClassName: local-storage
  # storageClassName: ibmc-block-bronze
  # storageClassName: ibmc-block-silver
  # storageClassName: ibmc-block-gold
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 10Gi
---
apiVersion: batch/v1
kind: Job
metadata:
  name: dbench
spec:
  template:
    spec:
      containers:
      - name: dbench
        image: dbench:latest
        imagePullPolicy: Always
        env:
          - name: DBENCH_MOUNTPOINT
            value: /data
          # - name: DBENCH_QUICK
          #   value: "yes"
           - name: FIO_SIZE
             value: 1G
          # - name: FIO_OFFSET_INCREMENT
          #   value: 256M
          # - name: FIO_DIRECT
          #   value: "0"
        volumeMounts:
        - name: dbench-pv
          mountPath: /data
      restartPolicy: Never
      volumes:
      - name: dbench-pv
        persistentVolumeClaim:
          claimName: dbench-pv-claim
  backoffLimit: 4
  • 改的參考給個模板

部署 dbench 測試的 yaml 進行測速

kubectl apply -f dbench.yaml

  • 獲得執行狀態
kubectl logs <pod-name> -f 

獲得檢測後測試速度的成果報表

這邊需要先讓 dbench 的 job 狀態變成 complete,後面才會是完整的資料。

kubectl logs <pod-name>
  • 成果報表
    • Random Read/Write IOPS: 隨機讀寫 IOPS
    • Random Read/Write BW : 隨機讀寫速度
    • Average Latency (usec) Read/Write: 平均寫入延遲
    • Sequential Read/Write: 循序寫入速度
    • Mixed Random Read/Write IOPS: 混和讀寫 IOPS
root@sdn-k8s-b2-1-2:~# kubectl logs dbench-vfn94
Working dir: /data

Testing Read IOPS...
read_iops: (g=0): rw=randread, bs=4096B-4096B,4096B-4096B,4096B-4096B, ioengine=libaio, iodepth=64
fio-2.17-45-g06cb
Starting 1 process
read_iops: Laying out IO file(s) (1 file(s) / 1024MiB)

read_iops: (groupid=0, jobs=1): err= 0: pid=17: Wed Aug 26 13:35:22 2020
   read: IOPS=454, BW=1834KiB/s (1878kB/s)(27.5MiB/15321msec)
  cpu          : usr=0.52%, sys=1.12%, ctx=7035, majf=0, minf=1
  IO depths    : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.2%, 32=0.5%, >=64=114.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.1%, >=64=0.0%
     issued rwt: total=6961,0,0, short=0,0,0, dropped=0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=64

Run status group 0 (all jobs):
   READ: bw=1834KiB/s (1878kB/s), 1834KiB/s-1834KiB/s (1878kB/s-1878kB/s), io=27.5MiB (28.8MB), run=15321-15321msec

Disk stats (read/write):
  sdf: ios=7972/36, merge=2/11, ticks=1097004/6156, in_queue=1106928, util=99.41%


Testing Write IOPS...
write_iops: (g=0): rw=randwrite, bs=4096B-4096B,4096B-4096B,4096B-4096B, ioengine=libaio, iodepth=64
fio-2.17-45-g06cb
Starting 1 process

write_iops: (groupid=0, jobs=1): err= 0: pid=33: Wed Aug 26 13:35:40 2020
  write: IOPS=130, BW=537KiB/s (550kB/s)(8304KiB/15472msec)
  cpu          : usr=0.10%, sys=0.35%, ctx=1988, majf=0, minf=1
  IO depths    : 1=0.1%, 2=0.1%, 4=0.2%, 8=0.4%, 16=0.8%, 32=1.6%, >=64=111.2%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.1%, >=64=0.0%
     issued rwt: total=0,2013,0, short=0,0,0, dropped=0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=64

Run status group 0 (all jobs):
  WRITE: bw=537KiB/s (550kB/s), 537KiB/s-537KiB/s (550kB/s-550kB/s), io=8304KiB (8503kB), run=15472-15472msec

Disk stats (read/write):
  sdf: ios=0/2281, merge=0/8, ticks=0/1058200, in_queue=1078804, util=99.42%


Testing Read Bandwidth...
read_bw: (g=0): rw=randread, bs=128KiB-128KiB,128KiB-128KiB,128KiB-128KiB, ioengine=libaio, iodepth=64
fio-2.17-45-g06cb
Starting 1 process

read_bw: (groupid=0, jobs=1): err= 0: pid=49: Wed Aug 26 13:35:58 2020
   read: IOPS=366, BW=46.3MiB/s (48.6MB/s)(714MiB/15428msec)
  cpu          : usr=0.30%, sys=1.06%, ctx=5537, majf=0, minf=1
  IO depths    : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.3%, 32=0.6%, >=64=114.5%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.1%, >=64=0.0%
     issued rwt: total=5647,0,0, short=0,0,0, dropped=0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=64

Run status group 0 (all jobs):
   READ: bw=46.3MiB/s (48.6MB/s), 46.3MiB/s-46.3MiB/s (48.6MB/s-48.6MB/s), io=714MiB (748MB), run=15428-15428msec

Disk stats (read/write):
  sdf: ios=6279/33, merge=217/4, ticks=1060284/7008, in_queue=1074188, util=99.44%


Testing Write Bandwidth...
write_bw: (g=0): rw=randwrite, bs=128KiB-128KiB,128KiB-128KiB,128KiB-128KiB, ioengine=libaio, iodepth=64
fio-2.17-45-g06cb
Starting 1 process

write_bw: (groupid=0, jobs=1): err= 0: pid=65: Wed Aug 26 13:36:16 2020
  write: IOPS=101, BW=13.2MiB/s (13.9MB/s)(206MiB/15593msec)
  cpu          : usr=0.35%, sys=0.29%, ctx=1577, majf=0, minf=1
  IO depths    : 1=0.1%, 2=0.1%, 4=0.3%, 8=0.5%, 16=1.0%, 32=2.0%, >=64=113.1%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=99.9%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.1%, >=64=0.0%
     issued rwt: total=0,1583,0, short=0,0,0, dropped=0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=64

Run status group 0 (all jobs):
  WRITE: bw=13.2MiB/s (13.9MB/s), 13.2MiB/s-13.2MiB/s (13.9MB/s-13.9MB/s), io=206MiB (216MB), run=15593-15593msec

Disk stats (read/write):
  sdf: ios=0/1848, merge=0/25, ticks=0/1093816, in_queue=1100900, util=99.39%


Testing Read Latency...
read_latency: (g=0): rw=randread, bs=4096B-4096B,4096B-4096B,4096B-4096B, ioengine=libaio, iodepth=4
fio-2.17-45-g06cb
Starting 1 process

read_latency: (groupid=0, jobs=1): err= 0: pid=81: Wed Aug 26 13:36:33 2020
   read: IOPS=273, BW=1096KiB/s (1123kB/s)(16.8MiB/15008msec)
    slat (usec): min=6, max=187, avg=30.03, stdev=10.10
    clat (msec): min=1, max=364, avg=14.57, stdev=19.30
     lat (msec): min=1, max=364, avg=14.60, stdev=19.30
    clat percentiles (msec):
     |  1.00th=[    4],  5.00th=[    5], 10.00th=[    5], 20.00th=[    7],
     | 30.00th=[    8], 40.00th=[    9], 50.00th=[   10], 60.00th=[   11],
     | 70.00th=[   13], 80.00th=[   16], 90.00th=[   29], 95.00th=[   48],
     | 99.00th=[   81], 99.50th=[  135], 99.90th=[  200], 99.95th=[  330],
     | 99.99th=[  363]
    lat (msec) : 2=0.39%, 4=4.04%, 10=46.80%, 20=35.56%, 50=8.83%
    lat (msec) : 100=3.82%, 250=0.56%, 500=0.07%
  cpu          : usr=0.43%, sys=1.28%, ctx=4062, majf=0, minf=2
  IO depths    : 1=0.1%, 2=0.1%, 4=113.4%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwt: total=4111,0,0, short=0,0,0, dropped=0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=4

Run status group 0 (all jobs):
   READ: bw=1096KiB/s (1123kB/s), 1096KiB/s-1096KiB/s (1123kB/s-1123kB/s), io=16.8MiB (16.9MB), run=15008-15008msec

Disk stats (read/write):
  sdf: ios=4589/28, merge=0/3, ticks=66876/2260, in_queue=69152, util=99.42%


Testing Write Latency...
write_latency: (g=0): rw=randwrite, bs=4096B-4096B,4096B-4096B,4096B-4096B, ioengine=libaio, iodepth=4
fio-2.17-45-g06cb
Starting 1 process

write_latency: (groupid=0, jobs=1): err= 0: pid=97: Wed Aug 26 13:36:51 2020
  write: IOPS=145, BW=582KiB/s (596kB/s)(8760KiB/15048msec)
    slat (usec): min=5, max=86749, avg=71.26, stdev=1740.32
    clat (msec): min=1, max=466, avg=27.40, stdev=57.28
     lat (msec): min=1, max=466, avg=27.48, stdev=57.30
    clat percentiles (usec):
     |  1.00th=[ 1448],  5.00th=[ 1768], 10.00th=[ 1912], 20.00th=[ 2192],
     | 30.00th=[ 3152], 40.00th=[ 5664], 50.00th=[13632], 60.00th=[15040],
     | 70.00th=[16768], 80.00th=[20096], 90.00th=[64256], 95.00th=[134144],
     | 99.00th=[309248], 99.50th=[342016], 99.90th=[464896], 99.95th=[464896],
     | 99.99th=[464896]
    lat (msec) : 2=13.44%, 4=22.95%, 10=4.66%, 20=38.73%, 50=8.73%
    lat (msec) : 100=4.34%, 250=5.21%, 500=2.06%
  cpu          : usr=0.19%, sys=0.64%, ctx=2196, majf=0, minf=3
  IO depths    : 1=0.1%, 2=0.1%, 4=115.6%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwt: total=0,2187,0, short=0,0,0, dropped=0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=4

Run status group 0 (all jobs):
  WRITE: bw=582KiB/s (596kB/s), 582KiB/s-582KiB/s (596kB/s-596kB/s), io=8760KiB (8970kB), run=15048-15048msec

Disk stats (read/write):
  sdf: ios=0/2535, merge=0/5, ticks=0/67680, in_queue=68148, util=99.42%


Testing Read Sequential Speed...
read_seq: (g=0): rw=read, bs=1024KiB-1024KiB,1024KiB-1024KiB,1024KiB-1024KiB, ioengine=libaio, iodepth=16
...
fio-2.17-45-g06cb
Starting 4 threads
read_seq: Laying out IO file(s) (1 file(s) / 2524MiB)

read_seq: (groupid=0, jobs=1): err= 0: pid=113: Wed Aug 26 13:39:03 2020
   read: IOPS=30, BW=31.1MiB/s (33.5MB/s)(515MiB/16131msec)
  cpu          : usr=0.14%, sys=0.18%, ctx=517, majf=0, minf=0
  IO depths    : 1=0.2%, 2=0.4%, 4=0.8%, 8=1.6%, 16=111.8%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=99.8%, 8=0.0%, 16=0.2%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwt: total=500,0,0, short=0,0,0, dropped=0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16
read_seq: (groupid=0, jobs=1): err= 0: pid=114: Wed Aug 26 13:39:03 2020
   read: IOPS=32, BW=33.4MiB/s (34.7MB/s)(565MiB/17102msec)
  cpu          : usr=0.03%, sys=0.30%, ctx=573, majf=0, minf=0
  IO depths    : 1=0.2%, 2=0.4%, 4=0.7%, 8=1.5%, 16=113.3%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=99.8%, 8=0.0%, 16=0.2%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwt: total=550,0,0, short=0,0,0, dropped=0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16
read_seq: (groupid=0, jobs=1): err= 0: pid=115: Wed Aug 26 13:39:03 2020
   read: IOPS=32, BW=33.3MiB/s (34.9MB/s)(578MiB/17390msec)
  cpu          : usr=0.02%, sys=0.31%, ctx=581, majf=0, minf=0
  IO depths    : 1=0.2%, 2=0.4%, 4=0.7%, 8=1.4%, 16=110.7%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=99.8%, 8=0.0%, 16=0.2%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwt: total=563,0,0, short=0,0,0, dropped=0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16
read_seq: (groupid=0, jobs=1): err= 0: pid=116: Wed Aug 26 13:39:03 2020
   read: IOPS=32, BW=32.2MiB/s (34.6MB/s)(571MiB/17313msec)
  cpu          : usr=0.03%, sys=0.28%, ctx=574, majf=0, minf=0
  IO depths    : 1=0.2%, 2=0.4%, 4=0.7%, 8=1.4%, 16=112.1%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=99.8%, 8=0.0%, 16=0.2%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwt: total=556,0,0, short=0,0,0, dropped=0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16

Run status group 0 (all jobs):
   READ: bw=128MiB/s (134MB/s), 31.1MiB/s-33.3MiB/s (33.5MB/s-34.9MB/s), io=2229MiB (2337MB), run=16131-17390msec

Disk stats (read/write):
  sdf: ios=2475/2, merge=0/1, ticks=1188976/124, in_queue=1193752, util=99.48%


Testing Write Sequential Speed...
write_seq: (g=0): rw=write, bs=1024KiB-1024KiB,1024KiB-1024KiB,1024KiB-1024KiB, ioengine=libaio, iodepth=16
...
fio-2.17-45-g06cb
Starting 4 threads

write_seq: (groupid=0, jobs=1): err= 0: pid=132: Wed Aug 26 13:39:25 2020
  write: IOPS=8, BW=9252KiB/s (9474kB/s)(171MiB/18927msec)
  cpu          : usr=0.13%, sys=0.02%, ctx=163, majf=0, minf=0
  IO depths    : 1=0.6%, 2=1.3%, 4=2.6%, 8=5.1%, 16=112.2%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=99.4%, 8=0.0%, 16=0.6%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwt: total=0,156,0, short=0,0,0, dropped=0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16
write_seq: (groupid=0, jobs=1): err= 0: pid=133: Wed Aug 26 13:39:25 2020
  write: IOPS=8, BW=9425KiB/s (9651kB/s)(170MiB/18471msec)
  cpu          : usr=0.10%, sys=0.05%, ctx=153, majf=0, minf=0
  IO depths    : 1=0.6%, 2=1.3%, 4=2.6%, 8=5.2%, 16=112.9%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=99.4%, 8=0.0%, 16=0.6%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwt: total=0,155,0, short=0,0,0, dropped=0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16
write_seq: (groupid=0, jobs=1): err= 0: pid=134: Wed Aug 26 13:39:25 2020
  write: IOPS=8, BW=9303KiB/s (9526kB/s)(170MiB/18712msec)
  cpu          : usr=0.03%, sys=0.12%, ctx=168, majf=0, minf=0
  IO depths    : 1=0.6%, 2=1.3%, 4=2.6%, 8=5.2%, 16=112.9%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=99.4%, 8=0.0%, 16=0.6%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwt: total=0,155,0, short=0,0,0, dropped=0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16
write_seq: (groupid=0, jobs=1): err= 0: pid=135: Wed Aug 26 13:39:25 2020
  write: IOPS=9, BW=9.85MiB/s (10.3MB/s)(179MiB/18222msec)
  cpu          : usr=0.09%, sys=0.08%, ctx=178, majf=0, minf=0
  IO depths    : 1=0.6%, 2=1.2%, 4=2.4%, 8=4.9%, 16=106.7%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=99.4%, 8=0.0%, 16=0.6%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwt: total=0,164,0, short=0,0,0, dropped=0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16

Run status group 0 (all jobs):
  WRITE: bw=36.5MiB/s (38.3MB/s), 9252KiB/s-9.85MiB/s (9474kB/s-10.3MB/s), io=690MiB (724MB), run=18222-18927msec

Disk stats (read/write):
  sdf: ios=0/763, merge=0/4, ticks=0/1281344, in_queue=1290208, util=99.46%


Testing Read/Write Mixed...
rw_mix: (g=0): rw=randrw, bs=4096B-4096B,4096B-4096B,4096B-4096B, ioengine=libaio, iodepth=64
fio-2.17-45-g06cb
Starting 1 process

rw_mix: (groupid=0, jobs=1): err= 0: pid=151: Wed Aug 26 13:39:42 2020
   read: IOPS=182, BW=741KiB/s (759kB/s)(11.4MiB/15257msec)
  write: IOPS=61, BW=251KiB/s (257kB/s)(3824KiB/15257msec)
  cpu          : usr=0.17%, sys=0.85%, ctx=3690, majf=0, minf=1
  IO depths    : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.2%, 16=0.4%, 32=0.9%, >=64=115.3%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.1%, >=64=0.0%
     issued rwt: total=2784,935,0, short=0,0,0, dropped=0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=64

Run status group 0 (all jobs):
   READ: bw=741KiB/s (759kB/s), 741KiB/s-741KiB/s (759kB/s-759kB/s), io=11.4MiB (11.6MB), run=15257-15257msec
  WRITE: bw=251KiB/s (257kB/s), 251KiB/s-251KiB/s (257kB/s-257kB/s), io=3824KiB (3916kB), run=15257-15257msec

Disk stats (read/write):
  sdf: ios=3269/1086, merge=0/3, ticks=757452/312804, in_queue=1070464, util=99.43%


All tests complete.

==================
= Dbench Summary =
==================
Random Read/Write IOPS: 454/130. BW: 46.3MiB/s / 13.2MiB/s
Average Latency (usec) Read/Write: 14.60/27.48
Sequential Read/Write: 128MiB/s / 36.5MiB/s
Mixed Random Read/Write IOPS: 182/61

主機端使用 dbench 測試硬碟速度

安裝輔助套件

sudo apt-get install fio -y

下載專案 && 切入專案

git clone https://github.com/leeliu/dbench
cd dbench

設定 dbench 對應的 ENV

  • DBENCH_MOUNTPOINT: 測試的掛載路徑
  • FIO_SIZE: 測試的數據大小
export DBENCH_MOUNTPOINT=<longhorn-mount-point>
export FIO_SIZE=1G

執行測試

bash docker-entrypoint.sh fio

執行結果

  • 成果報表
    • Random Read/Write IOPS: 隨機讀寫 IOPS
    • Random Read/Write BW : 隨機讀寫速度
    • Average Latency (usec) Read/Write: 平均寫入延遲
    • Sequential Read/Write: 循序寫入速度
    • Mixed Random Read/Write IOPS: 混和讀寫 IOPS
Working dir: /var/lib/kubelet/pods/9b4b0615-0fa0-4e03-bb45-8bd4793d1460/volumes/kubernetes.io~csi/pvc-5fb58e77-b8e0-40b4-b0b9-0ca21b5f0cfd/mount

Testing Read IOPS...
read_iops: (g=0): rw=randread, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=64
fio-3.1
Starting 1 process
read_iops: Laying out IO file (1 file / 1024MiB)

read_iops: (groupid=0, jobs=1): err= 0: pid=28049: Thu Aug 27 06:27:53 2020
   read: IOPS=489, BW=1974KiB/s (2022kB/s)(29.3MiB/15188msec)
   bw (  KiB/s): min=  881, max= 2650, per=100.00%, avg=1975.13, stdev=434.51, samples=30
   iops        : min=  220, max=  662, avg=493.53, stdev=108.72, samples=30
  cpu          : usr=0.55%, sys=1.04%, ctx=7486, majf=0, minf=1
  IO depths    : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.2%, 32=0.4%, >=64=115.4%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.1%, >=64=0.0%
     issued rwt: total=7433,0,0, short=0,0,0, dropped=0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=64

Run status group 0 (all jobs):
   READ: bw=1974KiB/s (2022kB/s), 1974KiB/s-1974KiB/s (2022kB/s-2022kB/s), io=29.3MiB (30.7MB), run=15188-15188msec

Disk stats (read/write):
  sdf: ios=8534/2, merge=2/1, ticks=1084272/296, in_queue=1088096, util=99.52%


Testing Write IOPS...
write_iops: (g=0): rw=randwrite, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=64
fio-3.1
Starting 1 process

write_iops: (groupid=0, jobs=1): err= 0: pid=28751: Thu Aug 27 06:28:12 2020
  write: IOPS=160, BW=657KiB/s (673kB/s)(10.0MiB/15587msec)
   bw (  KiB/s): min=   24, max= 1036, per=98.32%, avg=645.00, stdev=260.93, samples=31
   iops        : min=    6, max=  259, avg=161.13, stdev=65.22, samples=31
  cpu          : usr=0.16%, sys=0.45%, ctx=2468, majf=0, minf=1
  IO depths    : 1=0.1%, 2=0.1%, 4=0.2%, 8=0.3%, 16=0.6%, 32=1.3%, >=64=115.3%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.1%, >=64=0.0%
     issued rwt: total=0,2497,0, short=0,0,0, dropped=0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=64

Run status group 0 (all jobs):
  WRITE: bw=657KiB/s (673kB/s), 657KiB/s-657KiB/s (673kB/s-673kB/s), io=10.0MiB (10.5MB), run=15587-15587msec

Disk stats (read/write):
  sdf: ios=0/2912, merge=0/12, ticks=0/1099444, in_queue=1115492, util=99.48%


Testing Read Bandwidth...
read_bw: (g=0): rw=randread, bs=(R) 128KiB-128KiB, (W) 128KiB-128KiB, (T) 128KiB-128KiB, ioengine=libaio, iodepth=64
fio-3.1
Starting 1 process

read_bw: (groupid=0, jobs=1): err= 0: pid=29562: Thu Aug 27 06:28:29 2020
   read: IOPS=369, BW=46.6MiB/s (48.9MB/s)(709MiB/15192msec)
   bw (  KiB/s): min=21290, max=61440, per=100.00%, avg=47849.57, stdev=10613.62, samples=30
   iops        : min=  166, max=  480, avg=373.33, stdev=82.82, samples=30
  cpu          : usr=0.41%, sys=0.99%, ctx=5465, majf=0, minf=1
  IO depths    : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.3%, 32=0.6%, >=64=116.4%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.1%, >=64=0.0%
     issued rwt: total=5606,0,0, short=0,0,0, dropped=0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=64

Run status group 0 (all jobs):
   READ: bw=46.6MiB/s (48.9MB/s), 46.6MiB/s-46.6MiB/s (48.9MB/s-48.9MB/s), io=709MiB (743MB), run=15192-15192msec

Disk stats (read/write):
  sdf: ios=6312/2, merge=210/1, ticks=1049796/624, in_queue=1054940, util=99.49%


Testing Write Bandwidth...
write_bw: (g=0): rw=randwrite, bs=(R) 128KiB-128KiB, (W) 128KiB-128KiB, (T) 128KiB-128KiB, ioengine=libaio, iodepth=64
fio-3.1
Starting 1 process

write_bw: (groupid=0, jobs=1): err= 0: pid=30240: Thu Aug 27 06:28:48 2020
  write: IOPS=85, BW=11.2MiB/s (11.7MB/s)(181MiB/16145msec)
   bw (  KiB/s): min=  514, max=21076, per=99.81%, avg=11441.77, stdev=6446.33, samples=31
   iops        : min=    4, max=  164, avg=89.23, stdev=50.25, samples=31
  cpu          : usr=0.21%, sys=0.25%, ctx=1341, majf=0, minf=1
  IO depths    : 1=0.1%, 2=0.1%, 4=0.3%, 8=0.6%, 16=1.2%, 32=2.3%, >=64=115.6%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=99.9%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.1%, >=64=0.0%
     issued rwt: total=0,1383,0, short=0,0,0, dropped=0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=64

Run status group 0 (all jobs):
  WRITE: bw=11.2MiB/s (11.7MB/s), 11.2MiB/s-11.2MiB/s (11.7MB/s-11.7MB/s), io=181MiB (190MB), run=16145-16145msec

Disk stats (read/write):
  sdf: ios=0/1626, merge=0/22, ticks=0/1097396, in_queue=1116564, util=99.49%


Testing Read Latency...
read_latency: (g=0): rw=randread, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=4
fio-3.1
Starting 1 process

read_latency: (groupid=0, jobs=1): err= 0: pid=30982: Thu Aug 27 06:29:05 2020
   read: IOPS=326, BW=1309KiB/s (1340kB/s)(19.2MiB/15017msec)
    slat (nsec): min=5982, max=74702, avg=28906.93, stdev=9359.55
    clat (usec): min=1019, max=181096, avg=12214.17, stdev=11075.50
     lat (usec): min=1032, max=181118, avg=12243.78, stdev=11075.77
    clat percentiles (msec):
     |  1.00th=[    4],  5.00th=[    5], 10.00th=[    5], 20.00th=[    7],
     | 30.00th=[    8], 40.00th=[    9], 50.00th=[   10], 60.00th=[   11],
     | 70.00th=[   12], 80.00th=[   15], 90.00th=[   20], 95.00th=[   34],
     | 99.00th=[   57], 99.50th=[   67], 99.90th=[  128], 99.95th=[  144],
     | 99.99th=[  182]
   bw (  KiB/s): min=  681, max= 1608, per=100.00%, avg=1311.60, stdev=244.47, samples=30
   iops        : min=  170, max=  402, avg=327.63, stdev=61.06, samples=30
  lat (msec)   : 2=0.18%, 4=4.46%, 10=49.27%, 20=36.44%, 50=7.70%
  lat (msec)   : 100=1.89%, 250=0.12%
  cpu          : usr=0.49%, sys=1.45%, ctx=4858, majf=0, minf=2
  IO depths    : 1=0.1%, 2=0.1%, 4=113.5%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwt: total=4910,0,0, short=0,0,0, dropped=0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=4

Run status group 0 (all jobs):
   READ: bw=1309KiB/s (1340kB/s), 1309KiB/s-1309KiB/s (1340kB/s-1340kB/s), io=19.2MiB (20.1MB), run=15017-15017msec

Disk stats (read/write):
  sdf: ios=5574/2, merge=0/1, ticks=67768/156, in_queue=67968, util=99.50%


Testing Write Latency...
write_latency: (g=0): rw=randwrite, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=4
fio-3.1
Starting 1 process

write_latency: (groupid=0, jobs=1): err= 0: pid=31719: Thu Aug 27 06:29:23 2020
  write: IOPS=138, BW=554KiB/s (567kB/s)(8360KiB/15097msec)
    slat (usec): min=7, max=141539, avg=95.76, stdev=3097.66
    clat (usec): min=1044, max=531746, avg=29081.00, stdev=66573.51
     lat (usec): min=1055, max=531789, avg=29177.24, stdev=66668.41
    clat percentiles (usec):
     |  1.00th=[  1385],  5.00th=[  1713], 10.00th=[  1860], 20.00th=[  2040],
     | 30.00th=[  2311], 40.00th=[  3130], 50.00th=[  5014], 60.00th=[ 13960],
     | 70.00th=[ 15664], 80.00th=[ 18744], 90.00th=[ 79168], 95.00th=[187696],
     | 99.00th=[312476], 99.50th=[396362], 99.90th=[517997], 99.95th=[522191],
     | 99.99th=[530580]
   bw (  KiB/s): min=   48, max= 1090, per=100.00%, avg=557.00, stdev=269.00, samples=30
   iops        : min=   12, max=  272, avg=139.07, stdev=67.16, samples=30
  lat (msec)   : 2=17.49%, 4=27.89%, 10=6.47%, 20=29.52%, 50=7.04%
  lat (msec)   : 100=3.07%, 250=5.80%, 500=2.68%, 750=0.19%
  cpu          : usr=0.23%, sys=0.46%, ctx=2079, majf=0, minf=2
  IO depths    : 1=0.1%, 2=0.1%, 4=115.3%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwt: total=0,2087,0, short=0,0,0, dropped=0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=4

Run status group 0 (all jobs):
  WRITE: bw=554KiB/s (567kB/s), 554KiB/s-554KiB/s (567kB/s-567kB/s), io=8360KiB (8561kB), run=15097-15097msec

Disk stats (read/write):
  sdf: ios=0/2411, merge=0/3, ticks=0/68024, in_queue=68832, util=99.46%


Testing Read Sequential Speed...
read_seq: (g=0): rw=read, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=libaio, iodepth=16
...
fio-3.1
Starting 4 threads
read_seq: Laying out IO file (1 file / 2524MiB)

read_seq: (groupid=0, jobs=1): err= 0: pid=2860: Thu Aug 27 06:30:48 2020
   read: IOPS=31, BW=31.0MiB/s (33.5MB/s)(520MiB/16269msec)
   bw (  KiB/s): min=20480, max=51200, per=24.36%, avg=32043.75, stdev=8164.98, samples=32
   iops        : min=   20, max=   50, avg=31.19, stdev= 7.97, samples=32
  cpu          : usr=0.01%, sys=0.30%, ctx=520, majf=0, minf=0
  IO depths    : 1=0.2%, 2=0.4%, 4=0.8%, 8=1.6%, 16=110.7%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=99.8%, 8=0.0%, 16=0.2%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwt: total=505,0,0, short=0,0,0, dropped=0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16
read_seq: (groupid=0, jobs=1): err= 0: pid=2861: Thu Aug 27 06:30:48 2020
   read: IOPS=33, BW=34.2MiB/s (35.8MB/s)(559MiB/16350msec)
   bw (  KiB/s): min=16384, max=53461, per=25.39%, avg=33399.53, stdev=7905.27, samples=32
   iops        : min=   16, max=   52, avg=32.56, stdev= 7.69, samples=32
  cpu          : usr=0.07%, sys=0.26%, ctx=559, majf=0, minf=0
  IO depths    : 1=0.2%, 2=0.4%, 4=0.7%, 8=1.5%, 16=114.5%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=99.8%, 8=0.0%, 16=0.2%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwt: total=544,0,0, short=0,0,0, dropped=0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16
read_seq: (groupid=0, jobs=1): err= 0: pid=2862: Thu Aug 27 06:30:48 2020
   read: IOPS=30, BW=31.9MiB/s (33.5MB/s)(506MiB/15842msec)
   bw (  KiB/s): min= 8192, max=55296, per=24.66%, avg=32437.68, stdev=9129.84, samples=31
   iops        : min=    8, max=   54, avg=31.68, stdev= 8.92, samples=31
  cpu          : usr=0.08%, sys=0.23%, ctx=507, majf=0, minf=0
  IO depths    : 1=0.2%, 2=0.4%, 4=0.8%, 8=1.6%, 16=113.8%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=99.8%, 8=0.0%, 16=0.2%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwt: total=491,0,0, short=0,0,0, dropped=0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16
read_seq: (groupid=0, jobs=1): err= 0: pid=2863: Thu Aug 27 06:30:48 2020
   read: IOPS=31, BW=31.0MiB/s (33.5MB/s)(515MiB/16109msec)
   bw (  KiB/s): min= 6144, max=53248, per=24.33%, avg=31998.31, stdev=9200.49, samples=32
   iops        : min=    6, max=   52, avg=31.22, stdev= 9.00, samples=32
  cpu          : usr=0.02%, sys=0.30%, ctx=515, majf=0, minf=0
  IO depths    : 1=0.2%, 2=0.4%, 4=0.8%, 8=1.6%, 16=111.8%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=99.8%, 8=0.0%, 16=0.2%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwt: total=500,0,0, short=0,0,0, dropped=0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16

Run status group 0 (all jobs):
   READ: bw=128MiB/s (135MB/s), 31.9MiB/s-34.2MiB/s (33.5MB/s-35.8MB/s), io=2100MiB (2202MB), run=15842-16350msec

Disk stats (read/write):
  sdf: ios=2343/2, merge=0/1, ticks=1148652/236, in_queue=1150296, util=99.55%


Testing Write Sequential Speed...
write_seq: (g=0): rw=write, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=libaio, iodepth=16
...
fio-3.1
Starting 4 threads

write_seq: (groupid=0, jobs=1): err= 0: pid=3608: Thu Aug 27 06:31:10 2020
  write: IOPS=8, BW=9414KiB/s (9640kB/s)(171MiB/18601msec)
   bw (  KiB/s): min= 2048, max=18432, per=31.35%, avg=11664.44, stdev=6213.34, samples=27
   iops        : min=    2, max=   18, avg=11.37, stdev= 6.10, samples=27
  cpu          : usr=0.04%, sys=0.09%, ctx=165, majf=0, minf=0
  IO depths    : 1=0.6%, 2=1.3%, 4=2.6%, 8=5.1%, 16=112.2%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=99.4%, 8=0.0%, 16=0.6%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwt: total=0,156,0, short=0,0,0, dropped=0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16
write_seq: (groupid=0, jobs=1): err= 0: pid=3609: Thu Aug 27 06:31:10 2020
  write: IOPS=8, BW=9767KiB/s (10.0MB/s)(171MiB/17929msec)
   bw (  KiB/s): min= 2043, max=24576, per=32.98%, avg=12273.23, stdev=7244.50, samples=26
   iops        : min=    1, max=   24, avg=11.85, stdev= 7.24, samples=26
  cpu          : usr=0.06%, sys=0.07%, ctx=163, majf=0, minf=0
  IO depths    : 1=0.6%, 2=1.3%, 4=2.6%, 8=5.1%, 16=112.2%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=99.4%, 8=0.0%, 16=0.6%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwt: total=0,156,0, short=0,0,0, dropped=0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16
write_seq: (groupid=0, jobs=1): err= 0: pid=3610: Thu Aug 27 06:31:10 2020
  write: IOPS=8, BW=9421KiB/s (9647kB/s)(170MiB/18478msec)
   bw (  KiB/s): min= 2048, max=24625, per=34.18%, avg=12718.60, stdev=6825.36, samples=25
   iops        : min=    2, max=   24, avg=12.40, stdev= 6.66, samples=25
  cpu          : usr=0.06%, sys=0.06%, ctx=161, majf=0, minf=0
  IO depths    : 1=0.6%, 2=1.3%, 4=2.6%, 8=5.2%, 16=112.9%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=99.4%, 8=0.0%, 16=0.6%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwt: total=0,155,0, short=0,0,0, dropped=0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16
write_seq: (groupid=0, jobs=1): err= 0: pid=3611: Thu Aug 27 06:31:10 2020
  write: IOPS=8, BW=9317KiB/s (9540kB/s)(171MiB/18795msec)
   bw (  KiB/s): min= 2048, max=22528, per=37.32%, avg=13887.74, stdev=7037.74, samples=23
   iops        : min=    2, max=   22, avg=13.48, stdev= 6.83, samples=23
  cpu          : usr=0.04%, sys=0.07%, ctx=157, majf=0, minf=0
  IO depths    : 1=0.6%, 2=1.3%, 4=2.6%, 8=5.1%, 16=112.2%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=99.4%, 8=0.0%, 16=0.6%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwt: total=0,156,0, short=0,0,0, dropped=0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16

Run status group 0 (all jobs):
  WRITE: bw=36.3MiB/s (38.1MB/s), 9317KiB/s-9767KiB/s (9540kB/s-10.0MB/s), io=683MiB (716MB), run=17929-18795msec

Disk stats (read/write):
  sdf: ios=0/758, merge=0/4, ticks=0/1262528, in_queue=1278760, util=99.55%


Testing Read/Write Mixed...
rw_mix: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=64
fio-3.1
Starting 1 process

rw_mix: (groupid=0, jobs=1): err= 0: pid=4586: Thu Aug 27 06:31:28 2020
   read: IOPS=179, BW=729KiB/s (747kB/s)(11.0MiB/15502msec)
   bw (  KiB/s): min=  128, max= 1330, per=100.00%, avg=743.43, stdev=313.07, samples=30
   iops        : min=   32, max=  332, avg=185.63, stdev=78.20, samples=30
  write: IOPS=59, BW=244KiB/s (250kB/s)(3788KiB/15502msec)
   bw (  KiB/s): min=   72, max=  433, per=100.00%, avg=247.30, stdev=103.05, samples=30
   iops        : min=   18, max=  108, avg=61.80, stdev=25.74, samples=30
  cpu          : usr=0.27%, sys=0.67%, ctx=3626, majf=0, minf=1
  IO depths    : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.2%, 16=0.4%, 32=0.9%, >=64=112.1%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.1%, >=64=0.0%
     issued rwt: total=2785,926,0, short=0,0,0, dropped=0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=64

Run status group 0 (all jobs):
   READ: bw=729KiB/s (747kB/s), 729KiB/s-729KiB/s (747kB/s-747kB/s), io=11.0MiB (11.6MB), run=15502-15502msec
  WRITE: bw=244KiB/s (250kB/s), 244KiB/s-244KiB/s (250kB/s-250kB/s), io=3788KiB (3879kB), run=15502-15502msec

Disk stats (read/write):
  sdf: ios=3168/1057, merge=2/3, ticks=740884/327188, in_queue=1068460, util=99.48%


All tests complete.

==================
= Dbench Summary =
==================
Random Read/Write IOPS: 489/160. BW: 46.6MiB/s / 11.2MiB/s
Average Latency (usec) Read/Write: 12243.78/29177.24
Sequential Read/Write: 128MiB/s / 36.3MiB/s
Mixed Random Read/Write IOPS: 179/59

上一篇
Day 14 Kubernetes 持久戰 - GlusterFS + Heketi 安裝基礎操作篇
下一篇
Day 16 Kubernetes 異地同步開戰 - Istio Service Mesh 跨叢集安裝基礎操作篇
系列文
基於付費公有雲與開源機房自建私有雲之雲端應用服務測試兼叢集與機房託管服務實戰之勇者崎嶇波折且劍還掉在路上的試煉之路30

尚未有邦友留言

立即登入留言