Parsing data

Data feeds come in machine-type formats, like a CSV or XML.  Which is fine for storing data but not particularly useful for human readability.  Which is where data visualization, or information visualization comes in.  But there’s an intermediate step: parsing the feed.

Lacking critical parts for my Tweet-A-Watt (which have since arrived, thank you Adafruit), I instead used data from my PowerTap.  Which I have plenty of (most rides back to June 2007), and is eminently parseable and simple to visualize.

The Code

Modifying the code provided by the instructors, I made a simple parser and chart generator in Processing.  Man this was so much easier than doing it in Java, in 1999, when I had no idea what I was doing.

// Data parser for CSV

void setup() {
// frameRate(24);  //probably not necessary

//variables for display
int x = 0;
int y = height;
int time = 0;
int watts = 0;

void parsePTCSV() {
String[] lines = loadStrings(“PTData.csv”);    // Load file into an array of Strings (each line is a String)
for(int i = 0; i < lines.length; i++) {        // For each line,up to the length of the line
String line = lines[i];            // read the line into a string
String chunks[] = line.split(“,”);    // Split the line into an array of Strings, using “,” to separate (each field is a String)

if(i > 1) {                // Ignore the first two rows
String time = chunks[0].trim();    // chunk the fields into named Strings, trim edges
String torque = chunks[1].trim();
String MPH = chunks[2].trim();
String watts = chunks[3].trim();
String miles = chunks[4].trim();
String cadence = chunks[5].trim();
String HR = chunks[6].trim(); //Heart Rate
String ID = chunks[7].trim(); //split ID
String alt = chunks[8].trim(); //altitude

//data for graph
int x = i; // set x equal to i.  moves one space to the right each cycle.
int y = height;
float power = Float.valueOf(watts.trim()).floatValue();  // converts a string into a floating point value.
//display data as a graph  – because variables are private, they reside in here.
if (power > 200) {
stroke(255,0,0); //red
else {
stroke(0,0,255); //blue
line(x, y, x, y-power); // draw line

} //end if

// Print the output to verify
// println(“time: ” + time + “, power: ” + watts);    // simple way to print line
println(String.format(“time: %s, torque: %s, MPH: %s, watts: %s, miles: %s, cadence: %s, HR: %s, ID: %s, Alt: %s”,  // formatting line %s for string (in order below)
time, torque, MPH, watts, miles, cadence, HR, ID, alt)); // data line (variables in order
}//end for loop
}//end setup (run once)


what you end up with from there is a nice graph like this:

blue columns: power levels below 200W
red columns: power levels above 200W

Data File: