10 Mar 09 What I’ve been doing

The person at the next desk to the right (my wife) just pointed out that I haven’t updated my blog in quite a while.  Among other things, I’ve been, uh, struggling a bit with Twitter social graph analysis.  The new social graph API calls make it easy to get friend and follower relationships, but they return Twitter IDs, not screen names.  Getting screen names requires a lot of API calls and parsing of JSON or XML… and so far, that’s pretty slow.  I seem to be able to get a few hundred names per minute, which might sound like a lot, but the Twitter social graph is huge.  We have Twitter’s openness to blame.

The fact that for the most part, anybody can follow anybody, along with all the auto-followbacks, makes for a very densely connected graph.  For example, I follow somewhere over 300 people.  They follow several hundred thousand.  The graph that shows the follower relationships of me and all the people I follow has about 1.4 million edges.  That’s a lot to manipulate.  I’ve experimented with removing the people who follow more than 500 or more than 1,000 people, which reduces the graph size considerably, but it is still challenging to analyze on a desktop system.

I think my next blog post will focus on the social graph, why it matters and what direction Twitter might take it.  Feel free to ping me with your thoughts; I’m eager to hear them.

I should mention that even as I’m doing all this, I’m still looking for the right “real” job.  I’m focusing mainly on product management related to social media and analytics.

Tags: social graph, social network, twitter