/943637

#csvkit
▻https://csvkit.readthedocs.org

Convert Excel to CSV:
in2csv data.xls > data.csv

Convert JSON to CSV:
in2csv data.json > data.csv

Print column names:
csvcut -n data.csv

Select a subset of columns:
csvcut -c column_a,column_c data.csv > new.csv

Reorder columns:
csvcut -c column_c,column_a data.csv > new.csv

Find rows with matching ells:
csvgrep -c phone_number -r 555-555-\d{4}" data.csv > matching.csv

Convert to JSON:
csvjson data.csv > data.json

Generate summary statistics:
csvstat data.csv

Query with SQL:
csvsql --query "select name from data where age > 30" data.csv > old_folks.csv

Import into PostgreSQL:
csvsql --db postgresql:///database --insert data.csv

Extract data from PostgreSQL::
sql2csv --db postgresql:///database --query "select * from data" > extract.csv

pour la plupart de ces trucs j’utilise #q mais @lazuly préfère #csvkit alors…

#q - du #SQL en ligne de commande
►https://github.com/harelba/q

The following command will provide the total size for each date. Notice that the size is in c5.

 q "SELECT c6,SUM(c5) AS size FROM exampledatafile GROUP BY c6"

#outil #data via @archiloque

7 command-line tools for data science | Jeroen Janssens
▻http://jeroenjanssens.com/2013/09/19/seven-command-line-tools-for-data-science.html

1. #jq - sed for JSON
▻https://stedolan.github.io/jq

2. #json2csv - convert JSON to CSV

3. #csvkit - suite of utilities for converting to and working with CSV

4. #scrape - HTML extraction using XPath or CSS selectors

5. #xml2json - convert XML to #JSON

6. #sample - when you’re in debug mode

7. #Rio - making #R part of the pipeline

(PS : je recommande de renommer sample en #samply, car sample existe déjà)

#outils #data