Blog · 2018-08-27 · MooseFS Team

10 MooseFS Best Practices to Maximize Performance

From replication goals to hardware selection, these ten tips cover the configuration choices that matter most — and the common mistakes that silently degrade a cluster.

Circuit board representing infrastructure best practices

This article addresses the most common technical setup questions and provides ten best practices to get the most out of your MooseFS cluster.

1. The Minimum Goal Set to 2

A goal is a number of copies of files’ chunks distributed among Chunkservers. Setting goal to 2 protects against drive or server failures through automatic replication. A goal of 1 risks permanent data loss if a drive fails.

Set the default minimum goal for all users on the MooseFS mount:

mfssetgoal -r 2 /mnt/mfs

Edit /etc/mfs/mfsexports.cfg:

* / rw,alldirs,mingoal=2,maproot=0:0

Reload the Master Server to apply:

mfsmaster reload
service moosefs-master reload
service moosefs-pro-master reload
kill -HUP `pidof mfsmaster`

For instances 1 PiB or larger, we recommend a goal of 3 due to a higher statistical probability of concurrent failures.

2. Enough Space for Metadata Dumps

The Master Server stores metadata snapshots and changelogs in /var/lib/mfs. Use this formula to calculate the minimum required space:

SPACE = RAM × (BACK_META_KEEP_PREVIOUS + 2) + 1 × (BACK_LOGS + 1) GiB

With defaults (BACK_LOGS=50, BACK_META_KEEP_PREVIOUS=1): RAM × 3 + 51 GiB

Example: a Master Server with 128 GiB RAM requires at least 435 GiB on /var/lib/mfs.

3. RAID 1 or RAID 1+0 for Storing Metadata

We recommend setting up a dedicated RAID 1 or RAID 1+0 array for storing metadata dumps and changelogs. Mount it at /var/lib/mfs, sized according to the formula above. Avoid network storage such as SANs or NFS shares for this path.

4. Virtual Machines and MooseFS

Running MooseFS Master Servers on virtual machines is not recommended for high-performance systems. Hypervisor overhead and I/O scheduling interference can introduce latency that degrades metadata operations across the entire cluster.

5. JBOD and XFS for Chunkservers

Use a JBOD configuration with each disk formatted as XFS and mounted individually (e.g. /mnt/chunk01, /mnt/chunk02, …). Add the mount points to /etc/mfs/mfshdd.cfg.

Benefits over RAID:

  • MooseFS can detect and mark individual disk failures independently, isolating the problem and triggering targeted replication.
  • Replication time stays manageable – a single 2 TiB drive failure is repaired in 20–60 minutes, versus 12–18 hours for a failed 36 TiB RAID set, dramatically reducing the data-at-risk window.

6. Network Bandwidth

A minimum of 1 Gbps networking is recommended. 10 Gbps provides substantially better performance for clusters under load. Use LACP bonding between switches for redundancy and aggregate throughput.

7. overcommit_memory on Master Servers (Linux only)

If you see “fork error” messages in the Master Server log, enable memory overcommit:

echo 1 > /proc/sys/vm/overcommit_memory

To make the setting permanent, add the following line to /etc/sysctl.conf:

vm.overcommit_memory=1

8. Disabled updatedb Feature (Linux only)

updatedb is not recommended for network distributed filesystems. Add fuse.mfs to the PRUNEFS variable in /etc/updatedb.conf:

PRUNEFS="NFS nfs nfs4 rpc_pipefs afs binfmt_misc proc smbfs autofs iso9660
ncpfs coda devpts ftpfs devfs mfs shfs sysfs cifs lustre tmpfs usbfs udf
fuse.glusterfs fuse.sshfs fuse.mfs curlftpfs ecryptfs fusesmb devtmpfs"

9. Up-to-date Operating System

Use current OS versions (Linux, FreeBSD, macOS) to ensure FUSE and kernel feature compatibility with MooseFS. Outdated kernels can exhibit FUSE bugs or missing capabilities that affect stability.

10. Hardware Recommendation

Master Servers are largely single-threaded for metadata operations, so prioritize high clock speed over core count. Disable hyper-threading on Master Server machines. Example processors that work well:

  • Intel Xeon E3-1280 v6 @ 3.90 GHz
  • Intel Xeon E3-1281 v3 @ 3.70 GHz
  • Intel Xeon E3-1276 v3 @ 3.60 GHz
  • Intel Xeon E5-1630 v3 @ 3.70 GHz
  • Intel Xeon E5-1620 v2 @ 3.70 GHz

Chunkservers benefit from modern multi-core processors. The minimum HA configuration is 2 Master Servers and 3 Chunkservers.