iT邦幫忙

DAY 19
0

蠻可愛的資料庫與資料處理系列 第 19

CSVKit 4

最近很流行且通用的資料格式 json, csvkit也提供了csv轉換到json
的tool,叫 csvjson.
來看以下的範例:

$ csvcut -c county,item_name data.csv | csvgrep -c county -m "GREELEY" | csvjson --indent 4

[
    {
        "county": "GREELEY", 
        "item_name": "RIFLE,7.62 MILLIMETER"
    }, 
    {
        "county": "GREELEY", 
        "item_name": "RIFLE,7.62 MILLIMETER"
    }, 
    {
        "county": "GREELEY", 
        "item_name": "RIFLE,7.62 MILLIMETER"
    }
]

--indent 4 是縮排4格.

cvsjson 也能直接對csv檔案操作.

例如直接對csv檔依照指定欄位進行處理

$ csvjson --indent 4 --key fips acs2012_5yr_population.csv | head -n13
{
    "31001": {
        "fips": "31001", 
        "name": "Adams County, NE", 
        "total_population": "31299", 
        "margin_of_error": "0"
    }, 
    "31003": {
        "fips": "31003", 
        "name": "Antelope County, NE", 
        "total_population": "6655", 
        "margin_of_error": "0"
    }, 

csvkit也提供了轉換分隔符號的csvformat

原本內容是用,當分隔

$ head -n3 data.csv
state,county,fips,nsn,item_name,quantity,ui,acquisition_cost,total_cost,ship_date,federal_supply_category,federal_supply_category_name,federal_supply_class,federal_supply_class_name
NE,ADAMS,31001,1005-00-589-1271,"RIFLE,7.62 MILLIMETER",1,Each,138,138,2008-07-11,10,WEAPONS,1005,"Guns, through 30 mm"
NE,ADAMS,31001,1005-00-589-1271,"RIFLE,7.62 MILLIMETER",1,Each,138,138,2008-07-11,10,WEAPONS,1005,"Guns, through 30 mm"

轉換為 |

$ csvformat -D \| data.csv | head -n3
state|county|fips|nsn|item_name|quantity|ui|acquisition_cost|total_cost|ship_date|federal_supply_category|federal_supply_category_name|federal_supply_class|federal_supply_class_name
NE|ADAMS|31001|1005-00-589-1271|RIFLE,7.62 MILLIMETER|1|Each|138|138|2008-07-11|10|WEAPONS|1005|Guns, through 30 mm
NE|ADAMS|31001|1005-00-589-1271|RIFLE,7.62 MILLIMETER|1|Each|138|138|2008-07-11|10|WEAPONS|1005|Guns, through 30 mm

轉換為tab

$ csvformat -T data.csv | head -n3
state	county	fips	nsn	item_name	quantity	ui	acquisition_cost	total_cost	ship_date	federal_supply_category	federal_supply_category_name	federal_supply_class	federal_supply_class_name
NE	ADAMS	31001	1005-00-589-1271	RIFLE,7.62 MILLIMETER	1	Each	138	138	2008-07-11	10	WEAPONS	1005	Guns, through 30 mm
NE	ADAMS	31001	1005-00-589-1271	RIFLE,7.62 MILLIMETER	1	Each	138	138	2008-07-11	10	WEAPONS	1005	Guns, through 30 mm

轉換為"

$ csvformat -U 1 data.csv | head -n3
"state","county","fips","nsn","item_name","quantity","ui","acquisition_cost","total_cost","ship_date","federal_supply_category","federal_supply_category_name","federal_supply_class","federal_supply_class_name"
"NE","ADAMS","31001","1005-00-589-1271","RIFLE,7.62 MILLIMETER","1","Each","138","138","2008-07-11","10","WEAPONS","1005","Guns, through 30 mm"
"NE","ADAMS","31001","1005-00-589-1271","RIFLE,7.62 MILLIMETER","1","Each","138","138","2008-07-11","10","WEAPONS","1005","Guns, through 30 mm"

上一篇
CSVKit 3
下一篇
從網路上擷取資料以及指令組合運用
系列文
蠻可愛的資料庫與資料處理30
圖片
  直播研討會
圖片
{{ item.channelVendor }} {{ item.webinarstarted }} |
{{ formatDate(item.duration) }}
直播中

尚未有邦友留言

立即登入留言