mfsstatsdump

NAME

mfsstatsdump - dump usage data from master stats file in csv or png format

SYNOPSIS

mfsstatsdump -f fields [-i stats_file] [-r range] [-s separator] [-p png_filename [-x chart_width] [-y chart_height]] [-hHd]

DESCRIPTION

mfsstatsdump dumps usage data from master stats file. These are the same data sets that are used to make CGI charts. Output can be either in text format (csv) or in graphical format (png). That means -p, -x and -y options are mutually exclusive with -h, -H and -d options.

MooseFS master server saves the stats file in the DATA_PATH directory - see mfsmaster.cfg(5)

OPTIONS

-f: field names separated by ',' (see FIELDS section); at least one field is mandatory; if an asterisk (*) is used, all fields are dumped
-i: name of mfs binary stats file (default: default_data_path/stats.mfs)
-r: range type: 0 - short range (minute) 1 - medium range (6 minutes) 2 - long range (30 minutes) 3 - very long range (day)
-s: specify column separator (default: tabulator)
-p: optional PNG filename (field name will be added before .png; YYYY -> YYYY_FIELD.png, YYYY.png -> YYYY_FIELD.png)
-x: PNG chart width (default: 1600)
-y: PNG chart height (should be divisible by 20 - if not then it will be adjusted; default:320)
-h: "human-readable" numbers using base 2 prefixes (IEC 60027)
-H: "human-readable" numbers using base 10 prefixes (SI)
-d: show timestamps in YYYY-MM-DD HH:MM format

FIELDS

ucpu: User cpu usage
scpu: System cpu usage
cpu: Total cpu usage (in csv sum of scpu and ucpu, in png shows both in different colors)
delete: Number of chunk deletions
replicate: Number of chunk replications
statfs: Number of statfs operations
getattr: Number of getattr operations
setattr: Number of setattr operations
lookup: Number of lookup operations
mkdir: Number of mkdir operations
rmdir: Number of rmdir operations
symlink: Number of symlink operations
readlink: Number of readlink operations
mknod: Number of mknod operations
unlink: Number of unlink operations
rename: Number of rename operations
link: Number of link operations
readdir: Number of readdir operations
open: Number of open operations
read: Number of read operations
write: Number of write operations
prcvd: Received packets
psent: Sent packets
brcvd: Received bytes
bsent: Sent bytes
memoryrss: Resident memory usage
memoryvirt: Virtual memory usage
mem: Total memory usage (in csv is the same as memoryvirt, in png shows both memoryrss and memoryvirt using two colors)
usedspace: Used raw disk space
totalspace: Total raw disk space
space: Total raw disk space (in csv the same as totalspace, in png shows both usedspace and totalspace using two colors)
readchunk: Number of chunk_read operations
writechunk: Number of chunk_write operations
create: Number of chunk creation attempts
change: Number of chunk internal operation attempts
delete_ok: Number of successful chunk deletions
delete_err: Number of unsuccessful chunk deletions
delete_stat: Sum of succesful and unsuccessful chunk deletions (in png shows both delete_ok and delete_err using two colors)
replicate_ok: Number of successful chunk replications
replicate_err: Number of unsuccessful chunk replications
replicate_stat: Sum of succesful and unsuccessful chunk replications (in png shows both replicate_ok and replicate_err using two colors)
create_ok: Number of successful chunk creations
create_err: Number of unsuccessful chunk creations
create_stat: Sum of succesful and unsuccessful chunk creations (in png shows both create_ok and create_err using two colors)
change_ok: Number of successful chunk internal operations
change_err: Number of unsuccessful chunk internal operations
change_stat: Sum of succesful and unsuccessfulchunk internal operations (in png shows both change_ok and change_err using two colors)
split_ok: Number of successful chunk split operations
split_err: Number of unsuccessful chunk split operations
split_stat: Sum of succesful and unsuccessful chunk split operations (in png shows both split_ok and split_err using two colors)
fileobjects: Number of file objects
metaobjects: Number of non-file objects (directories,symlinks,etc.)
objects: Number of all objects (in png shows both fileobjects and metaobjects using two colors)
chunksec8: Total number of chunks stored in EC8 format
chunksec4: Total number of chunks stored in EC4 format
chunkscopy: Total number of chunks stored in COPY format
chunks: Total number of chunks stored in any format (in png shows chunksec8, chunksec4 and chunkscopy using three colors)
chregdanger: Number of endangered chunks (mark for removal excluded)
chregunder: Number of undergoal chunks (mark for removal excluded)
regunder: Number of endangered and undergoal chunks (mark for removal excluded, in png shows chregdanger and chregunder using two colors)
challdanger: Number of endangered chunks (mark for removal included)
challunder: Number of undergoal chunks (mark for removal included)
allunder: Number of endangered and undergoal chunks (mark for removal included, in png shows challdanger and challunder using two colors)
bytesread: Average number of bytes read per second by all clients
byteswrite: Average number of bytes written per second by all clients
fsync: Number of fsync operations
lock: Number of lock operations
snapshot: Number of snapshot operations
truncate: Number of truncate operations
getxattr: Number of getxattr operations
setxattr: Number of setxattr operations
getfacl: Number of getfacl operations
setfacl: Number of setfacl operations
fcreate: Number of file create operations
meta: Number of extra metadata operations (sclass,trashretention,eattr etc.)
delay: (pro only) Master max delay in seconds
servers: Number of all registered chunk servers (both connected and disconnected)
dservers: Number of all disconnected chunk servers
mdservers: Number of disconnected chunk servers that are in maintenance mode
cservers: Number of all registered chunk servers (in csv equals to servers, in png shows servers, dservers and mdservers using three different colors)
udiff: Difference in space usage percent between the most and least used chunk server

OUTPUT

Text output - text (csv) file where first column is a timestamp (or date if -d option was specified), the following column are what was specified with the -f options, formatted according to used options (none, -s and -h or -H)

Graphical output - one png file with a chart is created for every field specified with the -f option, according to the naming scheme declared with the -p option. Size may be adjusted with -x and -y options.

NOTES

Data sets delete and delete_stat in a properly working system should be roughly the same, but not always identical (in their CSV form), this stems from the fact that the delete set is calculated using the number of requests sent by the master, and the delete_stat set is calculated using the number of replies received from chunk servers, so local fluctuations may be observed (i.e. one minute has more sent requests - higher value in the delete set and the next minute has more answers - higher value in delete_stat set). But if one were to calculate the sum of differences between the two sets over time, this sum should never grow over a certain number.

However, if there is some kind of problem (with network communication and/or chunk server process(es)) and some chunk delete requests are stalled/go missing, then delete values may be constantly higher than the delete_stat values and the sum of differences between the two sets will be steadily growing over time.

Same goes for replicate(_stat), create(_stat), change(_stat) and split(_stat) data sets.

REPORTING BUGS

Report bugs to bugs@moosefs.com

COPYRIGHT

This file is part of MooseFS.

MooseFS is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, version 2 (only).

MooseFS is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.

You should have received a copy of the GNU General Public License along with MooseFS; if not, write to the Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02111-1301, USA or visit http://www.gnu.org/licenses/gpl-2.0.html