Jones

There's a radio show about business and the economy called Marketplace. They feature clips of songs between stories that are often surprisingly groovy. Or perhaps I have aged to the point where I think music via a business show is groovy. Either way, I have been doing some noodling with mapping music using artist similarity data and wondered what the map for Marketplace would look like. Then I wondered what this map would look like when it was painted with economic data for the day it was broadcast. Would you be able to see what kinds of music they played on good days vs. bad days. You might expect there to be no correlation, but there are some areas that are more black than red and vice versa. Maybe classic rock is indicative of a positive business climate?

Here's the resulting sketch!

In the end, I was able to produce a live visual representation of data from three sources in about three weeks of after hours noodling using all open source software.

How It's Made

There's a Groovy script that takes information from Marketplace's nicely formatted summary page and put's it in a MySQL database. It then looks for artist similarity data from Last.FM and adds that to the db. It then fetches any new data about the Dow Jones Industrial Average from Yahoo Finance and stuffs that into another table.

These MySQL tables are exposed as XML pages through some fairly simple Zope Page Templates on this web server.

The Processing sketch then requests this data via the urls for the XML pages (see notes below) and plots out the resulting data. Processing is an extension of Java that was designed to make it easy for non-programmer types to do computer graphics. It's an easy language to learn and use and has a friendly and vibrant community around it.

I used the NetBeans IDE 6.5 for all the editing. It's very nice for both Groovy and Java and even makes a decent html editor.

The Code

The Groovy script source is here. It uses the Last.FM bindings for Java by JRoar to fetch artist data. Groovy is like processing in that is is an extension of Java. It's Java's answer to the Python and Ruby crowd and I'm coming to see it as the best of both worlds. It's an expressive language with database access, regular expressions, and XML baked right in and it's easy to use with Java so there are libraries galore.

It feeds a MySQL database on this server. I've made some notes on the SQL design.

The NetBeans project is available as jones.zip. It uses Processing to handle the display, and the JUNG graphing library 2.0 b1 to calculate and layout the artist graph. I had done some previous graphing sketches using a physics library to handle springs. Using JUNG made for much simpler code and much better performance. I had a tough time with the JUNG docs untill I read this tutorial. JUNG has a display capability, but I'm using Processing instead.

The Server

The sketch gets its data served up by this Plone server as RESTful XML pages. These are Zope page templates that are fed from the MySQL database. If I were doing this from scratch, I might have made them using Groovy and Grails, but this was probably the simplest piece of the puzzle.

1. djia.xml
2. graph_data.xml
   
3. playlist.xml
   

Also See

Contact

AtomBoy [at] swcp [dot] com - Andrew P. E. Collins

I'm ACOIL on Last.fm, AtomBoy on Linked in and AtomBoy on Twine.

Jones by Andrew P. E. Collins is licensed under a Creative Commons Attribution-Share Alike 3.0 United States License.