Bandwidth monitoring

I recently posted on Facebook and G+ about how the TWC buy-out by Spectrum has done a great deal of good for me – I’m actually getting all of the bandwidth that I pay for, and them some. I can see the improvement on my bandwidth graphs, which are generated by home-grown (i.e. messy and hackish) scripts. Some people asked for the sources of those scripts, and I promised I’d put something together, so here it is.

The first component is the speedtest_cli.py script, which is a command-line interface version of speedtest.net. I’m using version 0.3.4, but I see no reason why newer versions wouldn’t work just as well. I don’t modify that script in any way, so I’m not going to post it here.

The next component is the sampler script. This is a purely home-grown script, which I never intended to be used anywhere other than my network, so I won’t guarantee it will work anywhere else. Here it is:


#!/usr/bin/python

import sys
import os
import subprocess
import rrdtool
import datetime
import urllib
import tweepy
from token import *

curdate = datetime.datetime.now()
datestr = curdate.strftime("%Y%m%d%H")
resfile = 'speedtest-results-'+datestr+'.png'

readings = subprocess.check_output(["/usr/bin/python", "/root/speedtest_cli.py", "--simple", "--share", "--secure"])

ping = readings.split('\n')[0].split()[1]
download = readings.split('\n')[1].split()[1]
upload = readings.split('\n')[2].split()[1]
image = readings.split('\n')[3].split()[2]

rrdtool.update('pingtime.rrd', 'N:'+ping)
rrdtool.update('downloadspeed.rrd', 'N:'+download)
rrdtool.update('uploadspeed.rrd', 'N:'+upload)
urllib.urlretrieve(image, resfile)

auth = tweepy.OAuthHandler(consumer_token, consumer_secret)
auth.set_access_token(access_token, access_token_secret)
api = tweepy.API(auth)

msg = 'Download speed %s, upload speed %s, ping time %s' % (download, upload, ping)
api.update_status(msg)

#msg = "@jwbernin: speed problem: %s is only %s"

#if ( float(download) < 240.0 ):
# api.update_status("I'm paying for 300 Mbit down, why am I only getting %s Mbit?" % download)

#if ( float(upload) < 16.0 ):
# api.update_status("I'm paying for 20 Mbit up, why am I only getting %s Mbit?" % upload)

print ping
print download
print upload
print image

Some things to note about this script… First, it uses the tweepy module to post the results of each test to Twitter. The authentication information is in a separate file, “token.py”, that I will not be posting here. That file contains only four variable strings, and those variable strings are used only for authentication of the tweepy agent. Next, it also imports the rrdtool module, and uses RRDTool to record data. I’ll leave the creation of the RRD’s as an exercise for the reader, since it’s a fairly simple process.

The script prints out the upload and download speed, the measured latency, and the URL for the results image, all of which get sent to email since I run this through cron. It also saves the image file in a directory on my firewall – which reminds me that I need to go clean things up. Excuse me a bit while I take care of that…

Okay, I’m back now. So I’ve sampled my bandwidth every hour and recorded it into RRDs. Now, how to display it? I do that with PHP. First, I have a basic page with the alst 24 hours of data for upload, download, and latency.

This is a bit longer, so here we go:

<?php

$rrdDir = '/net/gateway/usr/local/stats/';
$imageDir = '/var/www/html/netspeedGraphs/';

$graphsAvailable = array (
'downloadspeed'=> array ('Download speed', 'MBps', 'MBps'),
'uploadspeed'=> array ('Upload speed', 'MBps', 'MBps'),
'pingtime'=> array ('Ping time', 'ms', 'ms')
);

function callError($errorString) {
print ("Content-Type: text/plain");
print ("\n\n");
printf ("Error message: %s", $errorString);
die();
}

$basicOptions = array (
'-w', '700',
'-h', '150',
'--start', '-86400',
'--end', 'now',
);

foreach ( array_keys($graphsAvailable) as $graph ) {
$options = $basicOptions;
$options[] = "--title";
$options[] = $graphsAvailable[$graph][0];
$options[] = "--vertical-label";
$options[] = $graphsAvailable[$graph][1];
$options[] = sprintf ("DEF:%s=%s:%s:AVERAGE", $graphsAvailable[$graph][2], $rrdDir.$graph.".rrd", $graphsAvailable[$graph][2]);
if ( $graphsAvailable[$graph][0] == "Download speed" ) {
$options[] = sprintf ("HRULE:300#00FF00:Max");
$options[] = sprintf ("HRULE:240#0000FF:80 pct of max");
$options[] = sprintf ("HRULE:200#FF00FF:Min guaranteed");
}
if ( $graphsAvailable[$graph][0] == "Upload speed" ) {
$options[] = sprintf ("HRULE:16#0000FF:80 pct of max");
$options[] = sprintf ("HRULE:20#00FF00:Max");
}
$options[] = sprintf ("LINE1:%s#FF0000", $graphsAvailable[$graph][2]);
$options[] = sprintf ("PRINT:%s:LAST:Cur\: %%5.2lf", $graphsAvailable[$graph][2]);

$tmpname = tempnam("/tmp", "env");
$ret = rrd_graph($tmpname, $options);
if ( ! $ret ) {
echo "<b>Graph error: </b>".rrd_error()."\n";
}
$destname = sprintf ("%s%s.png", $imageDir, $graph);
rename ($tmpname, $destname);
}

?>
<html>
<head>
<title>Network Speeds - Main</title>
<meta http-equiv="refresh" content="300">
</head>
<body>
<center>
<font size="+2"><b>John's Home Network Speeds</b></font><br/>
<a href="specific.php?sensorname=downloadspeed"><img src="netspeedGraphs/downloadspeed.png" border=0 /></a><br/>
<a href="specific.php?sensorname=uploadspeed"><img src="netspeedGraphs/uploadspeed.png" border=0 /></a><br/>
<a href="specific.php?sensorname=pingtime"><img src="netspeedGraphs/pingtime.png" border=0 /></a><br/>
<hr/>
</center>
</body>
</html>

You’ll notice the references to another PHP file, “specific.php” – this is another homegrown script that displays the past day, week, month, quarter, half-year, and year graphs for the selected dataset (upload speed, download speed, latency). That file:


<?php

$rrdDir = '/net/gateway/usr/local/stats/';
$imageDir = '/var/www/html/netspeedGraphs/';

$graphsAvailable = array (
'downloadspeed'=> array ('Download speed', 'bps', 'MBps'),
'uploadspeed'=> array ('Upload speed', 'bps', 'MBps'),
'pingtime'=> array ('Ping time', 'ms', 'ms')
);

$graphPeriods = array(
'day' => '-26hours',
'week' => '-8days',
'month' => '-32days',
'quarter' => '-3months',
'half-year' => '-6months',
'year' => '-1year'
);

$theSensor = $_GET['sensorname'];

function callError($errorString) {
print ("Content-Type: text/plain");
print ("\n\n");
printf ("Error message: %s", $errorString);
die();
}

if ( ! array_key_exists($theSensor, $graphsAvailable) ) {
callError("Invalid sensor name specified.");
die(0);
}

$basicOptions = array (
'-w', '700',
'-h', '150',
'--end', 'now',
);

foreach ( array_keys($graphPeriods) as $graphWindow ) {
$options = $basicOptions;
$options[] = '--start';
$options[] = $graphPeriods[$graphWindow];
$options[] = "--title";
$options[] = $graphsAvailable[$theSensor][0];
$options[] = "--vertical-label";
$options[] = $graphsAvailable[$theSensor][1];
$options[] = sprintf ("DEF:%s=%s:%s:AVERAGE", $graphsAvailable[$theSensor][2], $rrdDir.$theSensor.".rrd", $graphsAvailable[$theSensor][2]);
$options[] = sprintf ("LINE1:%s#FF0000", $graphsAvailable[$theSensor][2]);
$options[] = sprintf ("PRINT:%s:LAST:Cur\: %%5.2lf", $graphsAvailable[$theSensor][2]);
if ( $graphsAvailable[$theSensor][0] == "Download speed" ) {
$options[] = sprintf ("HRULE:300#00FF00:Max");
$options[] = sprintf ("HRULE:240#0000FF:80 pct of max");
$options[] = sprintf ("HRULE:200#FF00FF:Min guaranteed");

}
if ( $graphsAvailable[$theSensor][0] == "Upload speed" ) {
$options[] = sprintf ("HRULE:16#0000FF:80 pct of max");
$options[] = sprintf ("HRULE:20#00FF00:Max");
}

error_log(implode($options));
$tmpname = tempnam("/tmp", "env");
rrd_graph($tmpname, $options);
$destname = sprintf ("%s%s-%s.png", $imageDir, $theSensor, $graphWindow);
rename ($tmpname, $destname);
}

?>
<html>
<head>
<title>Environmental Sensor - <?php echo $graphsAvailable[$theSensor][0]; ?> </title>
<meta http-equiv="refresh" content="300">
</head>
<body>
<center>
Sensor: <?php echo $graphsAvailable[$theSensor][0]; ?><br/>
<?php
foreach ( array_keys($graphPeriods) as $graphWindow ) {
printf ("Previous %s
\n", $graphWindow);
printf ("<img src=\"netspeedGraphs/%s-%s.png\"><br/><hr/>\n", $theSensor, $graphWindow);
}
?>
</center>
</body>
</html>

That’s about it. The sampler is run on the very first device connected to the cable modem – in my case, the firewall – and running it anywhere behind that first device, or having other devices directly connected to the cable modem, will probably give you bad data. Feel free to use it, though there are no guarantees any of it will actually work for you.

Comments are closed.