The Long Tail (of|and) Open Source Software

I've been wanting to write this post for a while now, so it's going to be long, and it's probably going to take a while to finally get it out the door.

I'm a big fan of The Long Tail - both the book and the blog. Chris Anderson has had a couple of posts dealing with the long tail of software - one about JotSpot, and one about AppExchange. I agree that both of these companies/products are very much centered on Long Tail software.

However, I've want to try to make the point that open-source software is very Long Tail-ish, and has actually been so long before these newcomers were even a twinkle in their respective entrepreneurs' eyes. I also want to show how open-source can be a better (maybe the best) platform for Long Tail software. (Though I think both JotSpot and AppExchange are Very Good platforms.)

One thing I'd like to use in support of my opinion is some actual data...

A nice long tail graph showing the top 100 SourceForge projects, and their total # of downloads on the y-axis. I think this is a crude, yet reasonably accurate, way to show demand for open-source software. Though this graph shows a tail, it is only the top 100, which is actually only the head. I'm using it here to show the SF equivalent to Amazon's "Harry Potter phenomenon" - a couple of super-hits that dominate even the small head of lesser hits. At SF, we call it the "eMule effect."



Here I've chopped off the top 2 super-hit projects, and I've expanded the observation window to 2000 projects at once. (I can't go much higher than a 2k window due to technical limitations on the workstation machine I used to create these graphs, and this window size still demonstrates what I want to demonstrate) But notice how the expanded window size, though still small relative to the whole data set, displays a more pronounced tail shape.




Beginning where the previous graph stopped, this one is showing projects ranked 2000-4000. So, while moving down into the tail region of SF projects, the tail-ish nature of the data is starting to smooth out to a more linear demand curve.








Here's the end of the tail - all the way out at project rank ~62,000. This is, of course, the essence of The Long Tail - that there is at least SOME amount of demand for even the most niche or obscure products. (I should note, however, that SF.net has 130,000 registered projects. So nearly 50% of the projects have no downloads - most likely because those projects have not released any files ... yet?)





So, I think the above data and graphs are sufficient to demonstrate the Long Tail nature of open-source software (assumming SF.net projects are a valid representation of the larger open-source body of software). But why would open-source be a better platform for Long Tail software? To answer that, I'd like to use some of Chris's own ideas.

For the purposes of this argument, I'm calling the "best" platform whichever platform is most aligned with the 3 forces which add economic and cultural significance of The Long Tail. Because, as these forces work, they maximize the value of all Long Tail consumers.

1. Democratization of the tools of production. This is an easy win for open-source over JotSpot or AppExchange. In both cases, the tools of production are held in the hands of single companies. JotSpot or AppExchange tools are tied to their server, their tag language, etc. For open-source software, nearly every tool you could ever need is totally free. The tail can be lengthened much faster this way. I doubt JotSpot or AppExchange will have 100k+ custom applications within 10 years. (Though they don't need to for their more-limited purposes.)

2. Democratizing distribution. Another easy win for open-source. All JotSpot or AppExchange products are married to that distribution channel. And in those cases, it isn't simply "the internet" - it's specifically those application domains on the internet. If you want more access to those niche products, you're locked into the JotSpot or Salesforce.com distribution channel. Not so with open-source, which can be found in many places, and isn't locked into any of them. I wonder how far one would get trying to take their AppExchange software product solo before Salesforce.com sued?

3. Connecting Supply and Demand. The only force in which open-source is not a clear winner, and therefore, IMO, the most pressing need facing the open-source community. However, I would point out that neither JotSpot nor AppExchange have demonstrated an ability to perform this well, given the tiny sizes of their "tails" - JotSpot with only 8 add-on applications, and AppExchange with only 400.

So, trite but true: 2 out of 3 isn't bad.

And I think it should be publicized how open-source software has already created and sustained a Long Tail of software for many many years now.