Open source is awesome

We have decided to open-source a number of our core technologies so that the community at large can build more cool tech on top of what we’ve got already. Getting parts our codebase ready to release will take a little time yet, but check out the Roadmap.

The source will be freely available under the BSD license, which gives you total freedom with what you do with it. We’d appreciate it if you could submit patches to the code so we can work together to improve it, but we won’t force you to.

We can’t wait to see what you do with it!

Repository

Check out our public repository at Google Code.

The project is called HCFS or Hybrid Cluster File System. That’s a bit grand, because really it’s just filesystem change detection plus incremental ZFS snaphots, combined with a group messaging protocol over which we run a distributed algorithm to maintain the ”redundancy invariant” to keep data replicated in the face of server failure and to keep nodes up-to-date.

Check it out!

Roadmap

• October 2010
• Set up mailing list, version control repository and forum for people to discuss.
• Release core “streaming updates” filesystem technology based on filesystem monitoring combined with ZFS.

• December 2010
• Release multi-SSH session terminal application for controlling many servers simultaneously.
• Continue releasing updates to our core open-source offerings.

Some ideas

We’ve already got some ideas about what could be done with our technology:

• A “storage cluster” which can replace expensive NAS hardware for large organisations with inexpensive servers having local storage. It could load-balance users’ home directories (with either NFS or Samba) in the same way that the web cluster load-balances websites.