Tip of the Trade: Dragonfly BSD and the Hammer Filesystem

Dragonfly BSD v2.0 has recently been released. It's a fork from FreeBSD, and it's ultimate goal is to provide native clustering support in the kernel. Having spent the past five years rewriting various kernel subsystems to set things up for this aim, the team is now onto the second project phase. The current aims are to create a cache management framework for various filesystem and virtual machine aspects and to set up security-conscious resource sharing and resource control across clusters.

In addition to significant kernel and driver changes, the 2.0 release features the release of the Hammer filesystem.

Recent Tips » Clusters with Condor » Fruity Nagios » QEMU Read All Tips of the Trade

Hammer is a fine-grained snapshot filesystem designed for large disks/media. Its sweetspot begins at 500GB, and it can handle up to 1 Exabyte of data. Hammer runs a snapshot every every time the system syncs its mounts, in effect every 30-60 seconds. This history is retained and can easily be sifted through.

Hammer mounts instantly (no fsck needed on crash recovery) and offers queueless mirroring. It also has no inode limitations; after a previous experience of having to recreate an entire filesystem to get over the inode issues caused by a user with millions of very small files, this is enough by itself to interest me!

More information available here.

Hammer is currently in beta, so it's not a good idea for production systems yet, but it's expected to be production-ready later this year. If I were building a cluster I'd be tempted to experiment with both Dragonfly and Hammer.