URLs in real-time media – blogs, microblogs, etc. – can be the foundation of near-real-time search relevance ranking.  This is simple in principle.  When a lot of people (or an intelligently selected set of people) suddenly start citing links to a given page or site, there is obviously something of interest there.  That is the kind of analysis that Google’s PageRank and similar algorithms use.  Each cite is a signifcant “vote” for the page.  When they occur in real-time media (v. static web pages), it provides a relevance metric that Google and its competitors aren’t touching yet.  The big search engines are not yet doing much in this domain.  Nobody has emerged as a leader in doing it comprehensively. 

In other words, as many people have pointed out, URLs in blogs, microblogs, etc., are the foundation of something like PageRank in near realtime.  But they become increasingly useless as they are uniquely shortened.

It would be interesting to figure out a motivation for the URL shorteners to offer comprehensive resolution to their APIs (given a long URL, return ALL shortened versions).  I can’t quite see why they would do that, but perhaps developers with a vision for real-time search (which probably is more correctly called monitoring rather than search) could lobby the shorteners to add that capability… or even create a shared repository as a community project that would also solve the linkrot problem.

One more thought… Twitter and others who do the shortened themselves have all of this data for the URLs they shorten.  As large customers of the shortening services, they have more influence than the individual developers, so perhaps that’s where the lobbying effort should focus.