csvkit https://csvkit.readthedocs.io/en/1.0.2/
csvkit is a suite of command-line tools for converting to and working with CSV, the king of tabular file formats.
It is inspired by pdftk, gdal and the original csvcut tool by Joe Germuska and Aaron Bycoffe.
If you need to do more complex data analysis than csvkit can handle, use agate.
Important links:
Repository: https://github.com/wireservice/csvkit
Issues: https://github.com/wireservice/csvkit/issues
Documentation: http://csvkit.rtfd.org/
Schemas: https://github.com/wireservice/ffs
Buildbot: https://travis-ci.org/wireservice/csvkit
Why csvkit?
Because it makes your life easier.
Convert Excel to CSV:
in2csv data.xls > data.csv
Convert JSON to CSV:
in2csv data.json > data.csv
Print column names:
csvcut -n data.csv
Select a subset of columns:
csvcut -c column_a,column_c data.csv > new.csv
Reorder columns:
csvcut -c column_c,column_a data.csv > new.csv
Find rows with matching cells:
csvgrep -c phone_number -r “555-555-\d{4}” data.csv > new.csv
Convert to JSON:
csvjson data.csv > data.json
Generate summary statistics:
csvstat data.csv
Query with SQL:
csvsql –query “select name from data where age > 30” data.csv > new.csv
Import into PostgreSQL:
csvsql –db postgresql:///database –insert data.csv
Extract data from PostgreSQL:
sql2csv –db postgresql:///database –query “select * from data” > new.csv
And much more…