I recently wrote a script to dump data into CSV files. The CSV files work well for using in other scripts, but they are a bit difficult to read in order to verify that the data looks good. Sure, I could transfer the files to my local system and open them up in OpenOffice Spreadsheet or a similar program, but I want to do quick checks of the generated data and constantly copying the data and opening it up again in a program would just slow me down. Fortunately, there is a better way.

Using a combination of the cat, column, and less commands that are available from most *nix shells, the CSV data can be rendered into a nice table and quickly navigated. Here is an example:

[chris@host data]$ cat file.csv | sed -e 's/,,/, ,/g' | column -s, -t | less -#5 -N -S
      1 number_of_tests  execution_time min  execution_time max  execution_time avg  execution_time std_dev  peak_memory_usage min  peak_memory_usage max  peak_memory_usage avg  peak_memory_usage std_dev  real_memory_usage min
      2 449              0.2421700954        0.2522599697        0.24422667392717    0.0013405194115834      22062656               22067696               22062951.732739        552.24028841091            22282240
      3 416              0.2449610233        0.2619900703        0.24721734340337    0.0015257664849685      21295528               21300888               21295541.019231        262.48728836508            21495808
      4 446              0.2286360264        0.2422661781        0.23043336515404    0.001174508347353       20895976               20900800               20895987.03139         228.20177111936            20971520
      5 428              0.1955471039        0.2902140617        0.1981168762521     0.0046106433816399      18045464               18048784               18045487.271028        276.98063531264            18087936
      6 436              0.2208828926        0.2558329105        0.22297720351353    0.0021463518368546      18717960               18723192               18718020.238532        557.06025400191            18874368

The output is easy to navigate with the cursor keys and is perfect for quickly verifying the generated data.

To use for your files, simply replace file.csv in the above example with your file’s name. The -#5 determines how many columns to scroll when using the left and right arrow keys. You can increase or decrease this as needed to make navigating easier.

For those interested, I’ve tested this on Debian-based (Ubuntu, Mint, etc) and Redhat-based (CentOS) systems, and it works on all of them.

Since I’m tying together various command line tools that have no concept of the CSV format, there are some contents that this does not handle well. For example, all commas will be treated as separating out columns, even if the comma is escaped by quotes or a backslash.

I updated the command to fix a problem with handling empty entries, such as “…,data,,data,…”. The sed command takes care of changing those empty values to a space.

Did I help you?