We automatically generate a wide range of graphs which chart response time, load averages, and other such statistics from our web publishing servers.
The following is an example of the statistics that are generated, and a little bit of fun analysis Plus there are pretty graphs to view, which is always a plus!
Typical Loads
Most of them typically look like these, which are from our departmental web server cluster:
A fairly normal usage chart showing increased load on the servers during business hours and gradually tapering off as the evening continues on. There are a few spikes (16h and 22h), but otherwise there’s nothing too remarkable about this.
And here’s a weekly graph which better shows the daily cycle of load averages:
There’s also something occurring on the depts cluster: around once a day, one of them suddenly spikes way beyond average in the evening. Depts01 did this at 22h in my daily graph, and you can see that all of them do it in the weekly chart.
Ovid
Supporting the departmental web servers is Ovid, a server that mainly hosts MySQL servers. Ovid’s charts are a little more interesting.
Here’s a daily view:
First off, look at that load average! Luckily this is a very robust piece of hardware! Notice how the load average increases during the day, and creates a “shelf” from about 8h to 11h during the busiest time of traffic. But there’s something unusual occurring here: very definite sine curve-like patterns that seem to be occurring 4 times an hour. These are brief spikes caused by our 1000 or so “Is my server on?” scripts that users have set up with cron tabs and run at 00,15,30,45 past the hour.
Something else is going on: the very low (for Ovid) load average at around 02h-04h. I don’t have a reasonable explanation for this other than “nobody visits our web pages after midnight.”
Here’s a weekly view of Ovid:
A reasonable weekly view. We see those constant cronjob spikes, and the morning “shelf” as well as that very low load average immediately after midnight.
Students
Students have their own web servers and database server: students and vergil, respectively.
The daily usage graph from the students clusters is shown below:
It also shows the typical usage plateau during the day, along with our depression at 00h-02h. The low load average for students03 is because it was taken out of the rotation while some work was done on it.
Taking a look at the student MySQL server, vergil, we see something that is a little more remarkable:
Notice it? There are three depressions. The 00h-02h depression is present, as can be expected, but there are two other ones before this time period. Your guess is as good as mine.
Another thing to note is that the 00,15,30,45 minute spikes are present here as well, but are much less noticeable. Students generally stop configuring their databases after it is operational, so they don’t usually bother to set up the cronjob. This server starter script isn’t as important, as they usually aren’t running mission-critical sites.
Here’s a weekly chart for vergil:
Those depressions also appear to occur every day at around the same times. Your guess is as good as mine.
Finally, here’s a new view, a monthly chart:
Looks like someone was running CPU-intensive scripts over the winter break.
That’s all I have. Happy Computing!
[Crossposted from my staff website]
Nikky's Notes 