While developing web applications using Zope3, my Python web framework of choice, I always have the same issue: where should I store the UGC data? I usually used an NFS filesystem, using a Zope utility to manage the storage and retrieval of the files.
The most obvious disadvantage of this technique is that the NFS server is a single point of failure: if it disappears, there is no way to access the files from the application servers.
Another problem with this implementation is that all the UGC data files have to be served by the application servers, using resources which could be used to serve more clients instead of transmitting a static file over the network.
MogileFS is an application-level distributed filesystem which solves these problems, ensuring data integrity and redundancy, written by Danga. The only problem is that it is written in Perl, it works only with MySQL and requires WebDAV servers for the storage.
To solve my problem, I decided to borrow some of the ideas behind MogileFS and to develop my own distributed application-level filesystem. KoboldFS is written in Python, uses a PostgreSQL database to store the filesystem status and it is released under the GPL license.
I deployed it a few days ago on one of our production clusters, and it reduced the load of the application servers allowing us to serve the UGC files directly from nginx. You can find the source code along with a rough description here.
