Link: Talk description and video

2 years and a half ago, SourceForge was all PHP except for one little Python service. SourceForge was originally written in 1999, when Python wasn't so great at the web. After 10 years though, it all started to atrophy.

So SourceForge decided to do a little experiment and assigned 2 people to create FossFor.us. They chose a web framework, thinking that Python was good and impressed with the Django documentation: there was no real learning curve. Couchdb was a bit slow, though.

The experiment was a success (although the site is now gone, due to managerial reasons if I understood correctly) and so they were then given 6 weeks to redo the download page/files of SourceForge. It had to be dynamic, to offer you a download relevant to your platform: they use the user agent string to figure out the operating system, and borrowed code from setuptools to figure out which release is the latest.

Their admins loved Apache so they went with mod_wsgi, which worked out well.

The whole system worked fine on a laptop handling all the traffic from SourceForge: that was the only load testing! And unfortunately when the system went live it took about 8 seconds to load a page. There was no CPU or memory usage. It turned out they saturated their gigabit Ethernet card, by loading all the releases, which some projects have a lot of (e.g. JBoss). Memcached turned out to be slower than MongoDB, because of pickling, the 4 megabytes objects and CPU.

Finally they updated the list to not include all releases and the project was deemed a success! Thus everything shall become Python. They are developing their new platform openly, Allura. They use a FUSE filesystem in Python to control permissions.

From now on, they have an internal mandate that everything should be written using Python unless there is a good reason not to. And now SourceForge can compete again! As well as explore new directions.