What Are the MooseFS Network Requirements
MooseFS runs over standard TCP/IP, but parallel chunk transfers mean your fabric fills up faster than you might expect — even 8 moderate chunkservers can saturate a 40 Gbit/s client link. Here are the concrete recommendations: jumbo frames, minimum 1 Gbit/s, LACP across two switches, and why you must exclude FUSE mounts from updatedb.
Networking requirements
MooseFS requires a TCP/IP stack as a networking protocol and the underlying network is not important. MooseFS has been successfully tested using Ethernet and InfiniBand (IP-over-IB).
Ethernet
It is recommended to set up jumbo frames (MTU=9000). With a larger number of Chunkservers, network switches should be connected either through optical fiber or using aggregated links.
It is recommended to use at least 1 Gbps networks, although MooseFS performs better with faster networks. Configurations with many clients and Chunkservers (even when the speed of disks are not taken into consideration) may easily saturate 10 Gbps and faster networks as communication is performed in many-to-many mode. For instance, even 8 Chunkservers of moderate performance are able to saturate a client’s endpoint 40 Gbps network.
Network Bandwidth
We recommend having at least 1 Gbps network. Of course, MooseFS will perform better in a 10 Gbps network (in our tests we saturated the 10 Gbps network).
We recommend setting LACP between two switches and connecting each machine to both of them to enable redundancy of your network connection.
LACP
To enable redundancy of the network connections (no-SPoF) it is recommended to use two switches, setting LACP between them and connecting each machine to both of them.
Disabled updatedb feature (Linux only)
Updatedb is part of mlocate — an indexing system that keeps a database listing all the files on your server. This database is used by the locate command to do searches.
Updatedb is not recommended for network distributed filesystems.
To disable the updatedb feature for MooseFS, add fuse.mfs to the variable PRUNEFS in /etc/updatedb.conf (it should look similar to this):
PRUNEFS="NFS nfs nfs4 rpc_pipefs afs binfmt_misc proc smbfs autofs iso9660 ncpfs coda devpts ftpfs devfs mfs shfs sysfs cifs lustre tmpfs usbfs udf fuse.glusterfs fuse.sshfs fuse.mfs curlftpfs ecryptfs fusesmb devtmpfs"Download pdf
Now you know what the MooseFS network requirements are. If you want to download the PDF document with even more information, download MooseFS Hardware Guide.