# host_history.pl 0.0.1 released

Discussion in 'Host History' started by Ian.H, Jun 19, 2004.

Hi all..

I've hacked up a small Perl script to generate some activity graphs for my server. Here's an example the 2 graphs it produces:

Small:

Large:

I wrote some scripts to read these graphs so I could display them on my server but I happen to run Apache on the same box as my LFS server runs so I have that "advantage". For others, you would probably have to implement some kind of automagic FTP upload or similar.

The script runs standalone.. but designed to run from a cronjob every minute (the processing time is very short on my 350Mhz PII (maybe 3 seconds) from start to finish).

Please read the comments in the script for how to configure the small section at the top and the pre-req's. Also please note that I haven't enough free time to explain how to install each of the 3 pre-reqs if you don't have them already. The process isn't difficult, but please see the relevant sites / accompanying README files for those instructions =)

This could probably be coded better.. but it was a quick 10 min jobby =)

The script:
Code:
#!/usr/bin/perl
###########################################################################
#            host_history.pl
#
#  15/06/2004 14:18
#
#  Copyright (c) 2004,  Ian.H <ian.h@dsrc.digiserv.net>
#  Web    : <http://digiserv.net/>
#         : <http://dsrc.digiserv.net/>
#
###########################################################################
#
# A simple script that will use the host{port}.txt file to generate an
# overall log of your server activity and then generate 2 graphs (small
# and large ) representing 24 hours usage for drivers count.
#
###########################################################################
#
# NOTE: This was coded to run on a *nix platform but it should also run
#       on a windoze box.
#
###########################################################################
#
# PRE-REQUISITES:
# ---------------
#
# Perl5 ( <http://www.perl.com/> | <http://www.activestate.com/> )
# GD1 or GD2 libs ( <http://www.boutell.com/gd/> )
# tail ( windoze version at: <http://unxutils.sourceforge.net/> )
#
#
# Please DO NOT ask how to install these if you do not currently have
# them, see their accompanying README files for installation information.
#
###########################################################################
#
# CONFIGURATION:
# --------------
#
# Change the details below in the 'my %CONF = (...)' section to match
# your requirements / filesystem. PLEASE NOTE: the *_file values MUST BE
# fully-qualified paths / filename. The output_file and hourly_log_file
# will be created upon execution. The host_operator should be the name of
# your server connection that displays in the 'connections list' in LFS.
#
# The graph entries are the fully-qualified path / filename of where the
# graphs should be created
#
# The tail entry should be a fully-qualified path/ filename to your
# tail or tail.exe binary
#
###########################################################################

use strict;
use warnings;
use GD;

my %CONF = (

host_file               => '/path/to/LFS/host63392.txt',
output_file             => '/path/to/host_history.log',
hourly_log_file         => '/path/to/host_history_hour.log',
host_operator           => 'B.O.F.H',
large_graph_output_file => '/path/to/driver_history_large.png',
small_graph_output_file => '/path/to/driver_history_small.png',
tail                    => '/path/to/tail(.exe)',
);

my @drivers = ();
my $track = ''; my$cars = '';

my $time = time; open(HOST, "<$CONF{host_file}");
while (<HOST>) {
if (/^trackcfg=(.+)$/s) { ($track = $1); } if (/^cars=(.+)$/s) {
($cars =$1);
}

push @drivers, /^conn=(.+)$/s; }$track =~ s/[\n|\r|[\r\n]$//g;$cars =~ s/[\n|\r|[\r\n]$//g; close(HOST); open(LOG, ">>$CONF{output_file}");
print LOG "$time|$track|$cars|"; foreach (@drivers) { s/[\n|\r|[\r\n]$//g;
print LOG "$_\x00" unless$_ eq $CONF{host_operator} or$_ eq '';
}
print LOG "\n";
close(LOG);

system("$CONF{tail} -n 1440$CONF{output_file} > $CONF{hourly_log_file}"); create_small_driver_history_graph(); create_large_driver_history_graph(); sub create_small_driver_history_graph { my @data = (); my @hourly_log_contents = (); open(LOG, "<$CONF{hourly_log_file}");
@hourly_log_contents = <LOG>;
close(LOG);

foreach (@hourly_log_contents) {
/^[0-9]+\|[A-Z1-9]+\|[A-Z46]+\|(.*)$/s; my$driver_list = $1; my @drivers = split("\x00",$driver_list);
pop @drivers;
push @data, scalar @drivers;
}

my $width = 1440; my$height = 80;
my $out_width = 170; my$out_height = 95;
my $graph = new GD::Image($width, $height); my$out_graph = new GD::Image($out_width,$out_height);

my $white =$graph->colorAllocate(255, 255, 255);
my $black =$graph->colorAllocate(0, 0, 0);
my $light_blue =$graph->colorAllocate(134, 155, 191);
my $background =$graph->colorAllocate(214, 224, 235);
my $int_lines =$graph->colorAllocate(184, 194, 205);
my $int_lines_light =$graph->colorAllocate(204, 214, 225);

my $out_white =$out_graph->colorAllocate(255, 255, 255);
my $out_black =$out_graph->colorAllocate(0, 0, 0);
$out_graph->filledRectangle(0, 0,$width, $height,$out_white);

$out_graph->string(gdTinyFont, 162, 60, '2',$out_black);
$out_graph->string(gdTinyFont, 162, 40, '4',$out_black);
$out_graph->string(gdTinyFont, 162, 20, '6',$out_black);
$out_graph->string(gdTinyFont, 162, 1, '8',$out_black);
$out_graph->string(gdTinyFont, 128, 82, get_time() . '>',$out_black);
$out_graph->string(gdTinyFont, 1, 82,$CONF{server_name}, $out_black);$graph->filledRectangle(0, 0, $width,$height, $background);$graph->line(0, 70, $width, 70,$int_lines_light);
$graph->line(0, 60,$width, 60, $int_lines);$graph->line(0, 50, $width, 50,$int_lines_light);
$graph->line(0, 40,$width, 40, $int_lines);$graph->line(0, 30, $width, 30,$int_lines_light);
$graph->line(0, 20,$width, 20, $int_lines);$graph->line(0, 10, $width, 10,$int_lines_light);

my $i = 0; foreach (@data) {$graph->line($i,$height - ($_ * 10),$i, $height,$light_blue);
$i++; }$out_graph->copyResized($graph, 0, 0, 0, 0, 160, 80,$width, $height);$out_graph->setAntiAliased($light_blue);$out_graph->interlaced('true');

open(GRAPH, ">$CONF{small_graph_output_file}"); binmode GRAPH; print GRAPH$out_graph->png;
close(GRAPH);

undef $graph; undef$out_graph;
}

sub create_large_driver_history_graph {
my @data = ();
my @hourly_log_contents = ();

open(LOG, "<$CONF{hourly_log_file}"); @hourly_log_contents = <LOG>; close(LOG); foreach (@hourly_log_contents) { /^[0-9]+\|[A-Z1-9]+\|[A-Z46]+\|(.*)$/s;
my $driver_list =$1;
my @drivers = split("\x00", $driver_list); pop @drivers; push @data, scalar @drivers; } my$width = 1440;
my $height = 160; my$out_width = 350;
my $out_height = 200; my$graph = new GD::Image($width,$height);
my $out_graph = new GD::Image($out_width, $out_height); my$white = $graph->colorAllocate(255, 255, 255); my$black = $graph->colorAllocate(0, 0, 0); my$light_blue = $graph->colorAllocate(134, 155, 191); my$background = $graph->colorAllocate(214, 224, 235); my$int_lines = $graph->colorAllocate(184, 194, 205); my$int_lines_light = $graph->colorAllocate(204, 214, 225); my$out_white = $out_graph->colorAllocate(255, 255, 255); my$out_black = $out_graph->colorAllocate(0, 0, 0);$out_graph->filledRectangle(0, 0, $width,$height, $out_white);$out_graph->string(gdSmallFont, 1, 5, 'Server activity over the past 24 hours', $out_black);$out_graph->string(gdSmallFont, 342, 140, '2', $out_black);$out_graph->string(gdSmallFont, 342, 100, '4', $out_black);$out_graph->string(gdSmallFont, 342, 60, '6', $out_black);$out_graph->string(gdSmallFont, 342, 20, '8', $out_black);$out_graph->string(gdSmallFont, 300, 182, get_time() . '>', $out_black);$out_graph->string(gdTinyFont, 1, 82, $CONF{server_name},$out_black);

$graph->filledRectangle(0, 0,$width, $height,$background);

$graph->line(0, 140,$width, 140, $int_lines_light);$graph->line(0, 120, $width, 120,$int_lines);
$graph->line(0, 100,$width, 100, $int_lines_light);$graph->line(0, 80, $width, 80,$int_lines);
$graph->line(0, 60,$width, 60, $int_lines_light);$graph->line(0, 40, $width, 40,$int_lines);
$graph->line(0, 20,$width, 20, $int_lines_light); my$i = 0;
foreach (@data) {
$graph->line($i, $height - ($_ * 20), $i,$height, $light_blue);$i++;
}

$out_graph->copyResized($graph, 0, 20, 0, 0, 340, 160, $width,$height);
$out_graph->setAntiAliased($light_blue);

$out_graph->interlaced('true'); open(GRAPH, ">$CONF{large_graph_output_file}");
binmode GRAPH;
print GRAPH $out_graph->png; close(GRAPH); undef$graph;

Regards,

Ian

File size:
16.5 KB
Views:
315
6. ### franky500New Member

Hey Ian.

Not sure if you would still be reading this area but meh.

I have grabbed the Host_History.pl 0.0.2 release that you done a long time ago,

I have installed Perl etc (as far as im awear.. correctly).

When i run the hosthistory.pl file i get the HostHist.log, hourly.log and the 2 graphs created, The Log files both contain entries as expected, however the graph simply puts up the graph without any line data on it. It shows numbers, Time and server name on the axis, but just does not actually draw a line of the actual number of racers.

The server at the time had 30 People on it and i ran the script for 5 minutes as a test.

I know this is all very old but its still exactly what i need so if you have any ideas that would be appreciated.

Regards,
Dean

Hi Dean..

Could you email me a couple of the 'host#.txt' files (preferably with some drivers connected etc)? I don't have an LFS server to test this myself, but if you can send something over to me, I'll run the script agianst that and see if I can find out why it's not working and fix it if I can

There maybe a change in the host#.txt file format since I originally coded it causing the script to fail.. but I'll have a gander.

ian.h@dsrc.digiserv.net should reach me

Regards,

Ian

8. ### franky500New Member

Cheers Ian,

Just e-mailed some to you

Regards,
Dean

Hi Dean..

Got a meeting with a client in a bit, but will have a look at this once I'm back home :thumb3d:

Regards,

Ian

10. ### franky500New Member

No Worries

Enjoy the meeting

Regards,
Dean

Hi Dean..

Meeting went well... got another contract signed

Anyways... worked a bit on the script and have found a much more efficient way of producing them now (the script here used to get the list of driver names, split them into an array, count the number of array elements to get the number of drivers and then use that info to plot the graph. It seems that now, a new 'guests=#' line has been added to the host#.txt file meaning I can just grab that number)

I have this working for both graphs now and just need to finish off the small graph. I had to make adjustments as the current script would only display up to 8 drivers (IIRC, this was the max I had my server set to way back in the day). It now has 0-50 as the y-axis as I believe the grid sizes now can be 48?

Once I have the scaling sorted for the small graph sorted, I'll send you over a copy to test (should be done tomorrow as today's a bit busy)

Regards,

Ian

12. ### franky500New Member

Fantastic stuff Ian.

Just got back from a family BBQ.

Quick question whilst im here. Is there a way for it to pick up the server name at the same time ( i believe that is also in the file). Rather than having to set it in the .pl file?.

Regards,
Dean

13. ### franky500New Member

And just FYI, i'm not planning on using the small graph (not that im aware of anyway!) So if it makes it easier for you to ignore that section, please do so

Regards,
Dean

Hi Dean..

Apologies for the delay.

I'd already changed the code to pick up the server name from the host file

Just sent you an email with an updated version. Let me know how it goes. If all's well, I'll re-publish the code on clue-by-4.org

Cheers..

Ian

15. ### franky500New Member

Hi Ian,

All received and sounds fantastic. Still got that 1 issue though.

Currently populated server, updating every minute at the moment:

http://81.19.209.205/History/serverhistory_37445/driver_history_large.png

Still not drawing the line's like the old one did in the day. This could easily be a config problem with the perl, but i have never used perl before on windows so if you have any pointers (if you think it might be that) that would be appreciated.

Using the command line:

Code:
hosthist.pl -hostfile="C:\path\to\folder\host37445.txt" -log-dir="C:\path\to\folder\log_37445" -chart-dir="C:\path\to\folder\serverhistory_37445"

Last edited: Jun 19, 2010

Hmm.. the cmdline data looks fine.

Can you check the 2 log files Dean to see if there's anything in them?

I've just run it again withe files you sent to me and although it takes 5 mins to initially see anything drawn in the graph (this is due to the graph being drawn fullsize (1440px wide) and then shrunk to an acceptable size for output), it "works for me™".

If there is data in the log files, can you email them over to me please? I'll force some tests to read those without editing them and see what I can generate here. I don't think it's a Perl config issue as the main arse-ache to getting this all working is the GD library.. and you have that sorted else it wouldn't create any image at all

Cheers..

Ian

17. ### franky500New Member

1 log has something in it, but the hourly_log does not.

http://81.19.209.205/History/serverhistory_37445/log_37445.zip

Seems like a strange one then if its not the Perl / GD Setup!.

Appreciate the help Ian.

Regards,
Dean

P.S i should actually note that with that command line the script still runs and creates everything (obviously) but does come up with a standard windows "this is not a valid executable or program" rubbish it kicks out.

Ahah.. that kinda helps (to narrow things down at least).

Do you have the 'tail.exe' file Dean?

Seeing as the global log file is being generated, the script is parsing the host#.txt file properly.. but after that, it calls the tail.exe file to generate the hourly log file. This part seems to be the bit that it's falling over on. If you do have the tail.exe file, have you edited the host_history.pl file %CONF() hash to point to the full path to tail.exe? (I suggest dropping tail into your 'C:\Windows' dir and editing the script to make tail.exe point to 'C:\Windows\tail.exe'.

Regards,

Ian

19. ### franky500New Member

I am such an idiot!!.

Yes i did have tail.exe but it was in a big windows utilities package.. which was in Program files.. which of course meant there were a good few spaces in the path!!!.

Just grabbed the tail.exe from the previous page and chucked that in c:\windows as suggested and that has instantly solved it!! Nice one.

Now that its working i have only one question, (although perhaps after the initial 24 hours it will be as expected)

The graph looks like it is drawing from the left, should it not draw from the right (current time)?

Again, Night one and thanks Ian.

Regards,
Dean

Glad that it was an easy fix. Actually, glad you had the issue in the first place as it gave me a chance to rewrite some of the code in a much better way and to make it more configurable

You're right, it does write the data from the left to right. It's right (and will look as epxetced once 24 hours has passed). Basically, the number of drivers gets fed into an array.. that's then looped through and the lines are drawn, so once the hourly log file has a complete hours worth of entries.. the "now" data will always be the last entry of the file, and will draw the line against the right edge.

I think I did this as the "easy way out", heh.. but I'll have a look to see what's involved in reversing the drawing routine (keeping the current time against the right edge, but making the empty space until full on the left instead).

Cheers..

Ian