I recently wrote a script to dump data into CSV files. The CSV files work well for using in other scripts, but they are a bit difficult to read in order to verify that the data looks good. Sure, I could transfer the files to my local system and open them up in OpenOffice Spreadsheet or a similar program, but I want to do quick checks of the generated data and constantly copying the data and opening it up again in a program would just slow me down. Fortunately, there is a better way.
Using a combination of the
less commands that are available from most *nix shells, the CSV data can be rendered into a nice table and quickly navigated. Here is an example:
[chris@host data]$ cat file.csv | sed -e 's/,,/, ,/g' | column -s, -t | less -#5 -N -S 1 number_of_tests execution_time min execution_time max execution_time avg execution_time std_dev peak_memory_usage min peak_memory_usage max peak_memory_usage avg peak_memory_usage std_dev real_memory_usage min 2 449 0.2421700954 0.2522599697 0.24422667392717 0.0013405194115834 22062656 22067696 22062951.732739 552.24028841091 22282240 3 416 0.2449610233 0.2619900703 0.24721734340337 0.0015257664849685 21295528 21300888 21295541.019231 262.48728836508 21495808 4 446 0.2286360264 0.2422661781 0.23043336515404 0.001174508347353 20895976 20900800 20895987.03139 228.20177111936 20971520 5 428 0.1955471039 0.2902140617 0.1981168762521 0.0046106433816399 18045464 18048784 18045487.271028 276.98063531264 18087936 6 436 0.2208828926 0.2558329105 0.22297720351353 0.0021463518368546 18717960 18723192 18718020.238532 557.06025400191 18874368
The output is easy to navigate with the cursor keys and is perfect for quickly verifying the generated data.
To use for your files, simply replace
file.csv in the above example with your file’s name. The
-#5 determines how many columns to scroll when using the left and right arrow keys. You can increase or decrease this as needed to make navigating easier.
For those interested, I’ve tested this on Debian-based (Ubuntu, Mint, etc) and Redhat-based (CentOS) systems, and it works on all of them.
Since I’m tying together various command line tools that have no concept of the CSV format, there are some contents that this does not handle well. For example, all commas will be treated as separating out columns, even if the comma is escaped by quotes or a backslash.
I updated the command to fix a problem with handling empty entries, such as “…,data,,data,…”. The sed command takes care of changing those empty values to a space.