Getting started with rrdtools

RRD stands for Round Robin Database, and it is what it sounds like. There are a fixed number of records in the database and once the last record has been written in the database the next update goes to the first record, and around and around it goes. In this way, your databases will never grow out of control. The only downside to this is that, obviously, you've got to know how much data you'll want to look at historically ahead of time so that when you generate your database you have enough data. You may want a days worth of info, or even months.

When creating an RRD database we'll need to specify a couple things, namely one or more Data Sources and one or more Round Robin Archives. The data source (DS) defines what type of data is accepted and some boundaries on what constitutes good data. The round robin archives (RRA) can almost be thought of as views, it defines the different ways we can store and retrieve data.

Step 1: A glance at the structure of RRD database 

    [gpadmin@mdw bin]# rrdtool create temptrax.rrd \
    --start N --step 300 \
    DS:probe1-temp:GAUGE:600:55:95 \
    DS:probe2-temp:GAUGE:600:55:95 \
    DS:probe3-temp:GAUGE:600:55:95 \
    DS:probe4-temp:GAUGE:600:55:95 \
    RRA:MIN:0.5:12:1440 \
    RRA:MAX:0.5:12:1440 \
    RRA:AVERAGE:0.5:1:1440

--start specify the time at which the database starts, measured in seconds since the Epoch,

--step specify the interval between database updates.

Data Source Fields: DS:DS-Name:DST:HeartBeat:Min:Max

• DS Defines a Data Source Field.
• DS-Name The name of this Data Source.
• DST Defines the Data Source Type. Can be GAUGE, COUNTER, DERIVE or ABSOLUTE.
• HeartBeat Defines the minimum heartbeat, the maximum number of seconds that can go by before a DS value is considered unknown.
• Min The minimum acceptable value. Values less than this number are considered unknown. This is optional. Specify "U" (unknown) to not set a min
• Max The maximum acceptable value. Values exceeding this number are considered unknown. This is optional. Specify "U" (unknown) to not set a max

Round Robin Archives: RRA:CF:XFF:Steps:Rows

• RRA Defines a Round Robin Archive.
• CF Consolidation Function. Can be AVERAGE, MIN, MAX, or LAST.
• XFF Defines XFiles Factor, the number of data points that can be anally probed by martians before RRD gives a crap.
• Steps Defines how many Primary Data Points (PDPs) are consolidated using the Consolidation Function (CF) to create the stored value.
• Rows Defines the number of Rows (records) stored in this RRA.

Step 2: Retrieve database system info. 

[gpadmin@mdw bin]$ ./rrdtool info /var/lib/ganglia/rrds/HAWQ_Perf/mdw/mem_free.rrd
filename = "/var/lib/ganglia/rrds/HAWQ_Perf/mdw/mem_free.rrd"
rrd_version = "0003"
step = 15
last_update = 1369204894
header_size = 1000
ds[sum].index = 0
ds[sum].type = "GAUGE"
ds[sum].minimal_heartbeat = 120
ds[sum].min = NaN
ds[sum].max = NaN
ds[sum].last_ds = "49857084"
ds[sum].value = 1.9942833600e+08
ds[sum].unknown_sec = 0
rra[0].cf = "AVERAGE"
rra[0].rows = 5856
rra[0].cur_row = 3824
rra[0].pdp_per_row = 1
rra[0].xff = 5.0000000000e-01
rra[0].cdp_prep[0].value = NaN
rra[0].cdp_prep[0].unknown_datapoints = 0
rra[1].cf = "AVERAGE"
rra[1].rows = 20160
rra[1].cur_row = 19411
rra[1].pdp_per_row = 4
rra[1].xff = 5.0000000000e-01
rra[1].cdp_prep[0].value = 9.9712597333e+07
rra[1].cdp_prep[0].unknown_datapoints = 0
rra[2].cf = "AVERAGE"
rra[2].rows = 52704
rra[2].cur_row = 40546
rra[2].pdp_per_row = 40
rra[2].xff = 5.0000000000e-01
rra[2].cdp_prep[0].value = 2.9911903867e+08
rra[2].cdp_prep[0].unknown_datapoints = 0

Ganglia use 3 RRAs to store data, each for 1, 14, 366 days.
The maximum time of storing data for each RRA = step * pdp_per_row * rows

Step 3: Retrieve the first/last update time.

    [gpadmin@mdw bin]$ ./rrdtool first /var/lib/ganglia/rrds/HAWQ_Perf/mdw/mem_total.rrd | xargs -i date -d '1970-01-01 {} sec utc'
    Mon May 20 21:57:15 PDT 2013
    [gpadmin@mdw bin]$ ./rrdtool last /var/lib/ganglia/rrds/HAWQ_Perf/mdw/mem_total.rrd | xargs -i date -d '1970-01-01 {} sec utc'
    Tue May 21 22:18:14 PDT 2013
    [gpadmin@mdw bin]$ date
    Tue May 21 22:18:29 PDT 2013

Step 4: Retrieve the data.

rrdtool fetch filename CF [--resolution|-r resolution] [--start|-s start] [--end|-e end]
rrd use 'resolution' to decide which rra to retrive.
resolution = pdp_per_row * step

[gpadmin@mdw bin]$./rrdtool fetch /var/lib/ganglia/rrds/HAWQ_Perf/mdw/mem_total.rrd AVERAGE -r 15 

Step 5: Graphing the data.

rrdtool graph filename [option ...] [data definition ...] [data calculation ...] [variable definition ...] [graph element ...] [print element ...]
data definition DEF:<vname>=<rrdfile>:<ds-name>:<CF>[:step=<step>][:start=<time>][:end=<time>][:reduce=<CF>]
data calculation VDEF:vname=RPN expression
variable definition CDEF:vname=RPN expression

[gpadmin@mdw bin]$ ./rrdtool graph /home/gpadmin/2.jpg --start now-12000 DEF:value1=/var/lib/ganglia/rrds/HAWQ_Perf/mdw/swap_total.rrd:sum:AVERAGE AREA:value1
481x141