MooseFS 4: What Changed From Version 3
If you’ve been running MooseFS 3 for a while and start looking at MooseFS 4, the first question is usually just what’s actually different. The version bump makes it sound like a big deal, and in this case it is – most of the real changes landed in one release, 4.0.0, with a few more following not long after.
Here’s what that release and the ones after it brought. A new way to store chunks, a smarter Archive state, and a few things worth knowing if you’re mixing MooseFS 3 and 4 during an upgrade.
A second way to store a chunk
For all of MooseFS 3, there was really one way to store a chunk, full copies. Files get split into chunks of up to 64 MiB, and each chunk gets duplicated across Chunkservers – usually three times. That’s 300% of the data’s size on disk, just to survive two Chunkservers dying at once.
4.0.0 added Erasure Coding as an alternative. Rather than keeping three full copies, MooseFS splits a chunk into data parts and calculates parity from them, so any part can be lost and rebuilt from what’s left. The version that shipped in 4.0.0 used 8 data parts plus 2 parity parts – same protection as three copies against two simultaneous Chunkserver failures, but only 25% overhead instead of 200%. A second variant came later, in 4.26.0, with 4 data parts instead of 8 – more disk overhead, but it needs fewer Chunkservers to work.
Which format a chunk actually uses comes down to its Storage Class, and Erasure Coding is only available for the ARCHIVE and TRASH states – not CREATE or KEEP. A chunk still being written stays in copies no matter what.
Archive got smarter
Storage Classes aren’t new in 4.0.0 – MooseFS 3 already replaced the old goal setting (just a copy count) with named Storage Classes that could do more, things like labels and placement modes (strict, standard, loose). What changed after 4.0.0 was specifically how much control you had over the Archive state.
In MooseFS 3, a file moved into Archive after a set number of days since its ctime, and once it was archived, it stayed that way until someone manually cleared the flag. 4.2.0 opened that up quite a bit. You could trigger the same transition off atime or mtime too, the delay could be set in hours instead of days, and there was now a reversible mode that un-archives a file (all its chunks) once it stops meeting the condition. That release also added a fast mode that archives a file on the next maintenance pass, skipping the delay entirely. It’s not a coincidence this happened alongside the Erasure Coding work – Erasure Coding only applies to Archive and Trash, so making Archive more flexible directly widens where you can actually use it.
What upgrading from 3 to 4 actually involves
A MooseFS 3 Client still works fine once the Master, Chunkservers, and everything else has moved to MooseFS 4, with one catch – it can’t read Erasure-Coded files. So if you’re mid-upgrade, or you just need to keep serving MooseFS 3 Clients for a while, keep them on a Storage Class that stays in copies and save Erasure Coding for classes they’ll never touch. Trying to read or write an Erasure-Coded file from a MooseFS 3 Client just errors out on the client side – it won’t corrupt anything.
The command-line tools also changed shape. MooseFS 3 bundled most admin commands into one mfstools binary. In MooseFS 4 those are split into separate tools, like mfsquota and mfssclass. Jumping straight from a MooseFS 3 release to a current MooseFS 4 one – Master, Chunkservers, Clients, all at once – is a supported path. Stop the old Master, back up metadata.mfs and the changelog.*.mfs files, and the new Master reads that metadata without issue.
Further reading
What a Storage Class defines for each state, and how a chunk moves between them, is covered in Storage Class and Storage states. The mechanics of Erasure Coding itself, including which Chunkserver counts each format needs, are in Erasure Coding. What became of the old goal setting is in Goal (deprecated). Every command MooseFS installs today, grouped by task, is listed in Command line tools.