18 Jul 09 Developers unite – throw off the yoke of Twitter centralization and publish your tweetstreams!
“At the end of the day, Twitter is a prototype.” That’s a comment on Dave Winer’s blog by Chuck Shotton, who created one of the first web servers, long before most people had even heard of the Internet. Chuck’s main point is that Twitter is a good idea, but it should be implemented as a distributed system, not a centralized one.
Dead on, Chuck. I’m not in any way faulting Twitter by agreeing with Chuck. There are good reasons that they are succeeding where others have failed at microblogging. It is good that they are demonstrating the broad appeal and usefulness of this kind of communications. The problem, as Chuck nailed it, is that they are centralized. Compare this to blogging, which was designed from the start to be decentralized. There are dozens of blogging platforms that you an run locally, on a rented host or at a site dedicated to hosting blogs. Choices, choices, choices. But if you want to tweet, there’s only one way to do it – Twitter.
One reason Twitter succeeded where others failed is that it has a good API and is extremely open when it comes to sharing data. The default, unlike most other social media companies, is that all of your data is open to everyone, except for direct messages. That’s fairly radical and perhaps more than anything else, has inspired developers to create many, many Twitter applications.
I caught the bug myself, attracted by the volume of data that is easily available. I threw together TwURLed News, not with the idea of building a company around it, but because I wanted to see how well something like it would work. It wasn’t very hard to built, has a back end that requires a BSD machine worth maybe $1,000 and the front end runs on a very low-cost hosting provider. Amazing.
Still, I can’t believe this is the future of microblogging. Instead of running applications that use the Twitter API on our desktops, it seems much more likely that we will end up running something like the Twitter API ourselves, which talks peer-to-peer instead of client-server.
Consider how Twitter and Google have opposing information flow. The Google model is that people publish information on web servers, then Google’s robots gather the data. To access Google, you use a standard web client. In the Twitter world, nothing gets published until and unless it is pushed to Twitter’s servers and a lot of the people who read Twitter-published information do so using custom clients. I guess you can rationalize this by arguing that Twitter is getting its users to do all the work that Google’s robots would otherwise do, but that’s a terrible idea. As Chuck pointed out, it doesn’t scale.
Consider also how different Twitter’s data flow is from blogging. When you post a blog entry, you’re usually also publishing it as an RSS feed. Outfits like Technorati (and Google, of course) send robots out to read those feeds and make them available via the web or newsreaders. People call Twitter microblogging, but instead of encouraging people to tweet locally and make the tweetstream available to anybody who wants to retrieve it from your site, as with RSS, Twitter says no, you have to send your tweets to Twitter and then they become available to the public. The pain of that centralization is already hurting Twitter, as developers complain about being unable to get even a single user’s entire tweet history, about being unable to search more than a few weeks’ data and other limitations.
So, here’s a thought. How about if every Twitter application developer throws off the yoke of centralization and adds local (or hosted, via XML-RPC) RSS publishing as an option? This is relatively simple for desktop apps – it could use the same mechanisms as RSS. It could actually be an RSS feed tagged as a tweetstream, so that anything that reads it will know that no entry will be more than 140 characters, expect hashtags, “@” screen names, etc. Phone apps could use a proxy to do the same while continuing to publish the tweetstream on Twitter.
Imagine the services that could bloom if everybody’s tweetstream were available without haing to rely exclusively on Twitter and its limited resources? In no time at all, we’d see comprehensive indexing and other value-added services.
So, why not? I’m not suggesting anyone abandon Twitter, I’m just saying that microblogging will take off much faster if Twitter developers realize that they don’t have to depend only on Twitter to publish their tweets.
