lifestyle of a dying network admin

Weather Station

2008-08-13T12:59:00.003+10:00

Installed a Fine Offset Electronics WH1081 Weather Station on my roof the other day. Purchased the device from ebay for $85 and $25 delivered.

The station consists of the following sensors:

Thermo-hydro transmitter
Wind speed
Wind direction
Rain gauge

The console is a touch screen LCD panel. Apart from the LCD being difficult to read due to being too light (needs a contrast setting) it works reasonably well. The best part about it is that it has a USB connection for plugging into a PC.

To go with the USB conneciton the station comes with a software package called 'EasyWeather' which is functional and maintains a log with various graphs showing historical data. I wasn't too fussed on it though since it did things like give false readings and has really bad memory leaks.

I tried Cumulus, which is another weather station application that can upload results to a FTP site for Internet access. However it doesn't recognise the WH1081 natively and instead relies upon EasyWeather to gather statistics - not good for the above reasons of instablity and memory leaks.

Currently I am using the Linux console version of Weather Display. I like it because its a no fuss application with no frills and supports uploading results to Weather Underground natively. However it doesn't support 64bit at all, even when using IA32 libraries. This made me install 32bit Ubuntu (CLI only) within a VirtualBox - had to use the closed source version for the USB support. This is on my Mythbuntu 8.04 HTPC too, so the weather console sits on top of the TV in the loungeroom which worked out well.

You can see my weather data on Weather Underground here.

Network upgrade

2008-04-05T12:10:00.002+10:00

And the upgrades continue.

The network here is expanding something chronic so I needed something that could push the vlans harder. It's basically replacing the local Mikrotik/WRAP1-1 Router and Asus GigaX2024 L2 switch with a single Asus GigaX3112 L3 switch. It certainly tidied up the rack by removing two switches and a stack of patch leads.

Now I have two Asus 2024 L2 switches to stick in Admissions and the Crocosium. This will give me a gigabit vlan trunk to the locations and allow me to create some more subnets to reduce some of this needless traffic off the main networks. I'm trying to have a L2 managed switch on the end of every fibre link to get some flexibility into the network and get things into this decade...

I'm not all that impressed with the Asus network kit so far. It's okay for the price but it's buggy as hell and the 3112 tends to crash due to kernel panics and reboot due to buffer overflows or memory errors. I'm hoping future firmware updates will come and fix things. Not that I had much more luck with Netgear and Linksys stuff. There's a reason why Cisco can charge so much.

The campus wireless network is almost completed. Awaiting a cable run from the Machinery shed to the Conference center to install the AP on the roof there. Also need to install the 12th AP at the Tiger Temple to complete the 'ring of coverage' - full wireless coverage of the safari shuttle track and nearby walkways and buildings. Will put off any more expansion until the Hotel is built.

Terminating fibres

2008-02-09T15:05:00.000+10:00

Been busy terminating a few new fibre runs throughout the Zoo. This particular run goes from the Taj (Crocosium) through to the Dingo Diner and then onto the Snake Shed and Compound. 12 cores to the Diner, 6 cores each to the shed and compound. Single mode glass fibre is used throughout however we may start using Multi-Mode within buildings.

Terminating fiber isn't that difficult these days. Basically you just need cutters and strippers, cleaning wicks, guillotine and fusion splicer. We hire the fusion splicer and guillotine and purchase pre-made tails with ST connectors from AFC. Ends up being around $50 an end instead of the usual $100 an end for a contractor.

Pics:

An A/V Perspective

2008-01-20T09:32:00.000+10:00

Here's a picture of the A/V crew operating the two remote PTZ cameras and 'house' audio gear during the Veronica's gig at Australia Zoo.

The panel between them is the Panasonic Visual Mixing desk. The smaller monitors are the cameras and the screen on the top is what's currently showing on the big screen (a rather large Panasonic LED screen outside).

Asus Eee PC 4G (701)

2008-01-14T21:02:00.000+10:00

Today I purchased a Asus Eee PC 4G from the local Myers department store (Sunshine Plaza).

The only colours they had were black and white. I didn't want to be mistaken for jumping on the iWagon so I got the black one. It happens to look identical to my work laptop, a black LG LS70 Express, so I have a matching pair of portables :/

So far I've played around with the default OS - Xandros with the K Desktop Environment installed along with a custom Asus application launcher. I have also installed a live version of eeexubuntu onto a USB stick (Myers threw in a 2GB Toshiba USB stick for nothing, valued at $28AUD) and it ran fine in its 'live' mode. I will get a 2/4GB Secure Digital card to install xubuntu onto and then set up a dual boot with Xandros. That way I can use Xubuntu for my work and Xandros to demo it to the uninitiated.

I'm not going to go into details about its performance etc - plenty of reviewers out there already demonstrating that and I agree with their findings. I can confirm that the build quality is good and that it will probably handle rough treatment a lot better than other notebooks - or most 'non-ruggidised' portable electronic device for that matter.

I'm looking forward to setting this up as a my ultra-portable network tool. Testing wireless will be made a lot easier, also cheaper than those dedicated PDA based wifi test units. I can also do my usual router configuration using a USB serial adapter and its 10/100 network adapter.

I will consider the purchase of a long life battery when they become available - they apparently add a lot of extra bulk to the package but it would be worth it for extended trips or major network problems.

A Wikipedia article describes the various details of the Eee PC range can be found here.

Asus Eee PC size compared with Sony Ericsson P1i

Home wireless

2008-01-13T22:40:00.000+10:00

Today I finally set up my little public access wireless network at home. I've had a Mikrotik RB133 with RB52 and Senao radios laying around in an outdoor box for sometime now. Today I fixed up the roof cable run and installed it on my existing mast with Cisco 21dBi solid dish.

I used a homebrew 8dBi collinear omni-directional antenna for the public side (Senao 200mW/23dBm) radio - that puts output at around 30dBi in a 360°H/25°V area. I've set up a rudimentary hotspot service on the interface - I will set it up with PayPal etc shortly so the neighbors can gain ad-hoc internet access and not feel guilty for it. Not everyone wants to pay for a permanent connection unbelievable huh.

I attached the dish to the RB52 radio which is adjustable - so I'll set that down to give no more than the regulated 36dBi. I'm hoping to establish a Point-to-Point link to a friend in Buderim. Failing that maybe a repeater link via Kiel Mountain which sits in between. Mainly to share links and to muck around with routing etc.

I'll go for a cruise around the local area and see how much coverage my little 8dBi is providing - might even be able to pick it up at Kiel Mt with the laptop radio...

Progress

2008-01-11T21:16:00.000+10:00

Wireless Network

How things move slower the lager the enterprise. Might have to start pushing to get things done faster and more efficiently in future.

I installed the last two Access Points in from the first batch of six we received all those months ago. I guess a fairly large hold up was the flaky firmware of the Symbol WS5100 I mentioned earlier. It's going very nicely now and I can't wait to start tweaking it and including it into a radius/ldap domain.

One AP was installed in the main warehouse and another under the Taj for coverage of the main hallways. Both work quite well using the supplied 6dBi collinear antennas.

Today a 9dBi collinear from the RFShop arrived with two made up cables to suit the AP300. Another three access points will be ordered shortly to provide coverage of the Foodcourt - using the new 9dBi, open area between the warehouse and Taj - using existing 10dBi yagi and indoors are at the Croc's Lair shop - AP with integrated antennas. That will be the coverage are for now with the possibility of a few more sites later on.

Telstra IP WAN

Almost 6mths in the making and the completion date is within sight. The last week I have sorted out the issue Mooloolaba Travel were having with running their VPN to Galileo running over the new Telstra WAN. It simply wasn't connecting. Discovered that the 1-to-1 NAT addressing that was also used for IPSec and PPTP traffic did not like traffic coming from the direction of Mooloolaba. So they changed it to a Many-to-1 NAT and things are looking good apart from frequent dropouts of the IPSec VPN which I believe are due to the 'keepalives' being blocked - so I turned them off and I will see what happens.

WhaleOne still hasn't been changed over. I had a 3hr crack at it today with all the information I had scrounged from Telstra and the 'net in general. Turns out that their NextG account hasn't yet been set up for use with the IP WAN. What happens is that the APN Name changes from 'telstra.internet' to 'telstra.corp' and this dictates the gateway used for the connection. Before this can occur the SIM card/Account needs to be set up so that the calls are allowed through or something along those lines anyway - the request goes to the Telstra Mobile team. Hopefully it will be fixed up by Tuesday next week.

Myth Frontend

Ages ago I talked about building a mythbox out of a Epia M10000. Well last weekend I finally did it. Epia support is quite good with linux these days, it was easy to set up Mythbuntu on the box and have it connect to the existing Soltek Qbic 3401 Mythbox. It runs okay apart from stutter after a channel change for about 5 minutes - I think more RAM will fix that.

I might upgrade my PC and use the old bits to build a better backend someday.

Shoveling Data

2007-12-01T13:20:00.000+10:00

Because the 'zoo keeps many operations such as Graphic Design and Marketing in-house it generates a considerable amount of data on a day-to-day basis. It's a challenge to keep all this centralised and backed up. What I have done to achieve ample storage with basic redundancy is use a 'front end' NAS (Network Attached Storage) combined with a 'back end' NAS located elsewhere from the front end serving as the primary backup/archive.

The users access the front end NAS directly and generally work from its shares. This will change in future as I intend to access it as a iSCSI mount on a server. This NAS is a standard box housing JBOD and runs OpenFiler - you have probably read about it here earlier. The performance of this NAS is fairly ordinary but since its the network that presents the bottleneck its not something to be concerned about at this stage.

The back end NAS is a purpose built NAS from a company called Thecus, the N5200. It houses up to 5 SATA disks and supports RAIDs 0 thru to 10. I've set up this particular one with 5 x 750GB disks with RAID5. This provides enough space to backup the front end NAS at maximum capacity - about 2.5TB total.

I'm currently backing up the front end NAS via rsync to the Thecus. I had to find the rsync 'module' to install on the Thecus first as it doesn't support it by default however it wasn't a difficult process.

I will consider a Thecus 1U4500 NAS to go with a future Novell OES2 server - mounting it as an iSCSI volume for localised e-mail/data archiving. This will probably use another 5200 for backups.

Overall this provides us with a sizable storage pool at a very reasonable cost. I would like to implement a proper SAN however our needs aren't that great at this stage and a single form of redundancy appears to be acceptable to management. I'll always plan for the upgrade though.

Changed Service Providers once again

2007-11-29T21:33:00.000+10:00

Changed from Internode to Comcen - mainly so I can access the Australian PIPE peering network unmetered again. There's simply too much useful content to be had. I'm on the Comcen 10+10GB 8000/386kbit/sec plan now.

Also because I was using Internode's NodePhone service I changed VoIP providers too. So I went with MyNetFone. No problems thus far - quality is fine and the price is good.

The transition from Internode to the latter services took no more than two days. I cancelled the accounts on Monday noon and was changed over and functioning by Thursday noon.

OpenDNS Media Release

2007-11-29T18:12:00.000+10:00

Thought I'd link this here:

Australia Zoo Conserves Bandwidth, Enjoys 100 Percent Network Uptime with OpenDNS

A new era of IT focused policy

2007-11-25T19:27:00.000+10:00

Firstly I am excited at Australia's change of government, especially with fellow Queenslanders Kevin Rudd as PM and Wayne Swan as Treasurer. I'm sure they will do an excellent job of managing the country and will prove that QLD can produce what it takes. I found that this article gave words to my thoughts.

Now one of the most interesting topics of this election that I found was that Information Technology became a key policy area for all parties. Sure in the last few elections it was there but it was always one of those background things that parties flaunted so they could look 'modern.' However this time IT was front and center.

Labor brings with it the proposal to roll out a national broadband network that will provide Fibre to the Neighborhood (FTTN) to 98% of the population, and provide broadband services to the remaining 2% via wireless and other alternatives. More information about the policy is available on the ALP website here.

As part of my usual blogging I will be covering the progression of this new broadband network. To provide an industry perspective by offering my opinions.

For starters I believe the catalyst for this initiative is the selling of Telstra without splitting it up. In my opinion it was wrong to sell Telstra as a single entity. The government should have split Telstra's retail and wholesale operations first and kept the infrastructure at the very least. This would have provided exactly what this new ALP policy is going to provide - a state owned communications infrastructure that is accessible to anyone for cost price plus future investments.

If you want a example of a similar exercise you need not look further than Queensland where the state government have maintained ownership of the electricity network and generators while privatising the retail operations.

So what's done is done and the government won't get Telstra back. So this leaves the task of producing an alternative communications infrastructure in the hope of removing Telstra's monopoly particularly in rural areas where the private sector can not easily move into.

Going into detail; a key term used by the ALP in this policy is "in partnership with the private sector" as this is crucial for the success of this project. There are many organisations that are interested in the removal of Telstra's monopoly - reducing wholesale rates for network access suits anyone who is trying to compete in a tight market. With this support the ALP gain the additional funds necessary to build out the network nation wide - it has been said that it could cost an extra $20 billion on top of the initial $4.7 billion (I'll find the source of that claim...).

However I believe the policy is lacking detail with regard to Australia's international connections. Our links to other continents are already reaching their peak capacity and to increase the available bandwidth throughout the country will only make matters worse. These links are certainly not cheap and to simply rely on supply and demand to upgrade them will shift the monopoly from the access and distribution back on to the core supply. Its akin to building a million homes and running power to them all and not bother building more power plants until the demand, thus price, is through the roof.

Interesting times ahead.

Links:
Broadband: an election issue (Whirlpool, 23rd November 2007)
Federal Labor Broadband Policy (ALP Website, 21st March 2007)

Posting profile pic here so I can link it :/

Now we're getting somewhere

2007-11-24T09:30:00.000+10:00

It has been a hectic few weeks for me. Major projects for the period include the preparations for Steve Irwin Day, the roll out of the new Telstra Next IP managed WAN and the slowly but surely deployment of a campus wireless network.

I'm also feeding in various nifty services into the Zoo network - such as a OpenFire Jabber server, Twiki wiki, One or Zero Helpdesk and a few other things. Although I'm going to hold off on their deployment to users until I have a LDAP directory of some sort in place for all these things to authenticate against.

I updated the firmware on my Sony Ericsson P1i too - the difference in performance and stability is night and day - and it was pretty good to begin with! The Opera browser and unified messaging apps have been improved quite a bit. I'm actually encoding TopGear episodes to 3gp format on my MythTV box and watching them during my lunch breaks on it - I didn't think I'd be using it like that. nb: I could just watch the xvid/dvix encoded eps but they're a tad large...

Steve Irwin Day
Steve Irwin Day went well as far as the web servers went - they handled a doubling of traffic without a hitch. I will be sad to see the replication servers go, they've done their job well and I'm kinda proud of them. I'm starting an upgrade of massive proportions of the two main web servers this week - hopefully once I'm done they'll be more than capable of handling the load without the need for replicas. More on that later.

Telstra NextIP
I've cut everything over to the zoo's shiny new Telstra Next IP WAN (to use their marketing spin). Speeds are good, response times are awesome. I'm also using Telstra's Proxy Caches too as they're very snappy and are used by many - plus there's a discount on data used through them apparently.

As part of the WAN, each SHDSL connected site has a managed Cisco 1801 router and SHDSL TA, some yumcha device. The NextG connection is as per usual, but when it is connected it has a L2TP into the WAN and thus has access to all the same routes as the other sites. I've set up Mikrotik routers at each site including the NextG connection - the Zoo has 2 x Yawarra WRAP1-2s in a rack enclosure still, Mooloolaba has a Yawarra WRAP1-1 with wireless and WhaleOne has a Mikrotik RB133 in an indoor enclosure.

Campus Wireless
Not too much progress - the Admissions indoors area now as its own AP and the Taj (crocosium buidling) offices have coverage too. I'm waiting for a sparky to run new cabling to key points so I can locate a few more APs in good coverage areas. Also waiting on a $1k order for various bits and pieces from the RFShop so I can start making up tails and prepare the splitters for installation. Also getting some antennas from them that are cheap and appear to have excellent performance - looking forward to trying them out.

Symbol/Motorola WS5100

2007-11-06T07:37:00.000+10:00

If you use these Wireless switches and are still running pre-3.0 firmware, UPDATE! Huge changes made and I suspect it's Motorola weaving its magic. I had all sorts of issues running with Spectralink VoWiFi sets and coverage - updated firmware to 3.0.2.0 and everything is happy now.

Other benefits of the firmware is that the CLI now mimics Cisco's IOS in many ways and the Java/Web interface is greatly improved - information is readily available and the controls make sense...

It's changed my view on this kit I was almost about to turf it in exchange for some Cisco gear or even Mikrotik (but I didn't really want to configure each AP individually).

It's been more than a month

2007-10-23T19:23:00.000+10:00

So I guess I should post an update.

Ubuntu Gutsy Gibbon
I've updated the laptop from Fesity Fawn to Gutsy Gibbon the other day. Apart from fiddling with the sound card settings and installing the XGL version of XServer with the ATi RADEON restricted drivers it all went okay... Do you still want to use Linux? But really, I'm actually noticing a improvement in performance from Feisty - hard to put a finger on it but it just seems more "responsive". Maybe all that talk about improving the Linux kernel for the desktop has paid off and they've included various tweaks to better handle the scheduling?

The 3D desktop is nicely done - just a good sprinkling of bling to improve the overall functionality of the interface, not just a bunch of unstable power sapping eye candy like what Beryl throws up. Also it seems to be more friendly with this ATi Mobility X600 as beryl used to crash, although that could be all sorts of UE causing that - re-enforces the case that its better if such things are included in the distro.

So you can say that I'm happy with the direction of Ubuntu and if you're going to use it, upgrade to Gutsy.

Telstra Next IP WAN
All the ground work has been done and apparently this is the week for the physical rollout of the new service. Given all the problems I have been having with the wireless link into the Zoo it will be a breath of fresh air to have a decent high capacity link to use. I still have some reservations about it being presented as a "BDSL" (Business Digital Subscriber Line, Telstra's term) service though - it will be using a dedicated copper service from the exchange (not a RIM) and I think it might also be using two pairs - sounds very much like a G.SHDSL service which isn't bad at all.

I think they will be using Cisco 18xx series routers. Meaning that all these 857 and 877 routers will soon become redundant. I would like to use them elsewhere but much of the other locations the Zoo is concerned with are too remote for a DSL service. Maybe I can give them to staff who require a reliable VPN back into the Zoo WAN.

So I guess I should be devising a service change-over plan. Such things used to be second nature to me when working for AccessPlus but I have to think about it since my mind isn't buried in networks 90% of the time. I'm sort of split between networks, web servery things and desktop. I'll make an effort to get something together shortly, I guess I am still waiting on IP information from Telstra anyway.

Also once the WAN is in place I will re-establish my trusty webserver.

Novell Work Group Suite - Small Business Edition
I'm pretty keen on seeding Novell in this environment. It's just crying out for some decent administration and services.

The Small Business Workgroup suite is basically Novell OES2 with Groupwise. E-mail is a great introduction for a 'green fields' environment so I'm hoping management are impressed by it and won't be afraid to continue our foray into Novell. I'm hoping the new OES2 and Groupwise lives up to the hype, at least the cost is reasonable.

Facebook
Even though I have a blog I caved in and created a facebook account. I can see why it's addictive to many people - it's pretty bloody easy to find others on it and all of those widgets that seem to spread throughout profiles like some kind of nasty virus.

I predict two things to happen - someone will be assaulted/murdered as a direct result of having all their information in one easy to reach location and/or some kind of worm weaves its way through it, data-mining all the way and people start finding credit accounts opening up in their name. Call me a pessimist.

Election'07
Federal elections are fun aren't they, seeing what policies will be bandied about and then abolished as soon as they are elected. At least we get to see the pollies get their arse kicked on more than one occasion. As far as my political persuasion goes; I'm center-left which puts me on top of the Democrats, somewhat close to the Greens with Labor as the primary. That as close to a preference you'll get from me.

Here's a few blogs and things that I use to follow the election:

I'll try and keep this updated more often.

Posting from a P1i

2007-09-05T17:16:00.000+10:00

Well, it'll take some getting used to but it isn't that hard.

Recording RouterOS's IP Accounting Data

2007-08-25T11:53:00.000+10:00

There are a number of ways to gather data from a Mikrotik RouterOS based router. The easiest would be it's 'Accounting Web Access' feature where you can go to http://routeros_addr/accounting/ip.cgi and view a list of ip pairs similar to a basic netflow output.

Using this feature I wrote the below perl scripts to collect the data into a DB file. To keep things reasonable I set it record the data per the hour, meaning my smallest unit of measurement is hourly. While I could have simply used a MySQL database to dump the data into, I wanted to maintain a level of portability and simplicity - it sucks having to install/configure/run a fully fledged RDBMS just to view some basic data usage statistics.

The first script is used to gather the data from the MT router and store it into the db_file, the second script uses GD::Graph to produce bar charts using the data stored in the db_file. I'll be writing more scripts that dumps the contents of the db_file into a .xls spreadsheet for manual reports - handy for tracking down heavy users and to use as evidence if there are any ISP account discrepancies.

Apologies for the untidy code and the lack of formatting. Blogger doesn't provide any 'code markup' function and I cbf'd looking for alternatives. I'll fix it up when I can.

Example graph output (graph.pl 8 hours):

gather.pl:


#!/usr/bin/perl -w

use strict;
use LWP::Simple;
use MLDBM 'DB_File';
use Time::Local;

my $arg0 = $ARGV[0];
my $arg1 = $ARGV[1];

my $ip_accounting_url="http://<routeros ip>/accounting/ip.cgi";
my $accounting_mldbm_data_db = "~/accounting_data.mldbm";

tie my %h, 'MLDBM', $accounting_mldbm_data_db or die $!;

my ($timestamp) = &time_stamp();
my $epoch = time();
# print "\n Epoch set to: $epoch\n";

&gather_ip_accounting($ip_accounting_url);

sub gather_ip_accounting {
  my $url = $_[0];
  my ($src, $dst, $bytes, $packets, $src_usr, $dst_usr);

  foreach my $line (split(/\n/, get($url))) {
      ($src, $dst, $bytes, $packets, $src_usr, $dst_usr) = split(" ", $line);

      if ($dst && $dst =~ /(192\.168\.)|(10\.2\.)|(172\.16\.)/){
          my $h_dst = $h{$dst . "_" . $timestamp};
          $h_dst->{dst} = $dst;
         # $h_dst->{src} = $src;
          $h_dst->{bytes} += $bytes;
          $h_dst->{packets} += $packets;
         # $h_dst->{src_usr} = $src_usr;
          $h_dst->{dst_usr} = $dst_usr;
   $h_dst->{epoch} = $epoch;
          $h{$dst . "_" . $timestamp} = $h_dst;
      }
  }
}

# use Data::Dumper;
# print Dumper(%h);

untie %h;

sub time_stamp {
my ($d_t);
my ($sec,$min,$hour,$mday,$mon,$year,$wday,$yday,$isdst) = localtime(time);

      $year += 1900;
      $mon++;
      $d_t = sprintf("%4d-%2.2d-%2.2d %2.2d:00:00",$year,$mon,$mday,$hour,$min,$sec);
      return($d_t);
}

graph.pl:


#!/usr/bin/perl -w

use strict;
use LWP::Simple;
use MLDBM 'DB_File';
use Time::Local;
use GD::Graph::bars;

my ($num_values, $period_type);
if ($ARGV[0] && $ARGV[1]) {
if ($ARGV[0] =~ /\d+/) {
 $num_values = $ARGV[0];
}
else {
 print "\nIncorrect value supplied for number of units\n";
 exit;
}

if ($ARGV[1] =~ /(hours)|(days)|(months)/) {
 $period_type = $ARGV[1];
}
else {
 print "\nIncorrect value supplied for type of units\n";
 exit;
}
}
else {
print "\nUsage: period units\nPeriod: The number of values\nUnits: Hours, Days, Months\n\n";
exit;
}
print "\n Gathering $num_values $period_type worth of data from db!\n";

my $epoch = time();

my $accounting_mldbm_data_db = "~/accounting_data.mldbm";
my $graph_image_file = "~/accounting_data_" . $num_values . "_" . $period_type . "_" . $epoch . ".png";

tie my %h, 'MLDBM', $accounting_mldbm_data_db or die $!;



#else {
# &print_period_summary($arg0, $arg1);
#}

my($graphvalues, @graphvalues_tmp);
my $period_total = 0;
my $i = 0;
   while ($i <= $num_values) {
# print "$i\n";
       @graphvalues_tmp = &print_total($i, $period_type);
my $data = $graphvalues_tmp[0];
my $epoch = $graphvalues_tmp[1];
my $HMS = &epoch_to_MDHMS($epoch);
push @{$graphvalues->[0]}, $HMS;
push @{$graphvalues->[1]}, $data;
$period_total = $period_total + $data;
$i++;
   }

my $graph = GD::Graph::bars->new(85*$num_values, 300);
$graph->set(
   x_label     => "$period_type (latest towards the left) Period Total: $period_total",
   y_label     => 'Mbytes',
   title       => "Total Mbytes (Over $num_values $period_type)",
   transparent => '0',
   show_values => '1',
   bar_spacing => '2',
) or warn $graph->error;

my $image = $graph->plot($graphvalues) or die $graph->error;

open(IMG, ">$graph_image_file") or die $!;
binmode IMG;
print IMG $image->png;

# use Data::Dumper;
# print Dumper($graphvalues);

untie %h;

sub print_total {
my $h_total=0;
my ($h_row, $h_column, $h_bytes, $h_dst);

my ($num, $period) = @_;
my ($epoch_start, $epoch_end) = &epoch_period($num, $period);

for my $h_row ( keys %h ) {
 if ($h{$h_row}{epoch} >= $epoch_start && $h{$h_row}{epoch} <= $epoch_end) {           $h_bytes = $h{$h_row}{bytes};    $h_dst = $h{$h_row}{dst};           $h_total = $h_total + $h_bytes;   }     }   my $formatted_total = sprintf("%.3f", $h_total/1024/1024);   return($formatted_total, $epoch_start); }  sub epoch_period {  my ($past_count, $period) = @_;  my ($epoch_period_start, $epoch_period_end);   my ($sec,$min,$hour,$mday,$mon,$year,$wday,$yday,$isdst) = localtime(time);  my ($start_hour, $end_hour);   if ($period eq "hours") {   $start_hour = $hour-$past_count;   $end_hour = $hour-$past_count;  }  elsif ($period eq "days") {   $mday = $mday-$past_count;   $start_hour = '00';   $end_hour = '23';    }  elsif ($period eq "months") {   $mon = $mon-$past_count;   # $mday = '00';   # $hour = '00';  }   $epoch_period_start = timelocal(00,00,$start_hour,$mday,$mon,$year);  print "Start: $epoch_period_start\n";  $epoch_period_end = timelocal(59,59,$end_hour,$mday,$mon,$year);  print "End: $epoch_period_end\n";    # print "SUB EPOCH_PERIOD: $epoch_period_start, $epoch_period_end\n";  return($epoch_period_start, $epoch_period_end); }   sub epoch_to_MDHMS {  my $epoch = $_[0];   my ($sec, $min, $hour, $mday, $mon) = (localtime($epoch))[0,1,2,3,4];   my $mdhms = $mon+1 . "-" . $mday . " " . sprintf("%02d", $hour) . ":" . sprintf("%02d", $min) . ":" . sprintf("%02d", $sec);  return($mdhms); }

A simple .forward vacation enable/disable script

2007-08-11T16:14:00.000+10:00

I got a little tired of manually enabling/disabling peoples vacation AutoReply. So I decided to knock out a simple bash script that does the enable/disable part, leaving me to simply make sure the actual response message was updated and just AT the script for whenever they wanted to leave/come back.

I used to move .forward to dotforward and back when enabling/disabling - so if you're wondering why I'm referencing files called 'dotforward' it's for backwards compatibility - plus I like the idea of setting up dotforward if the user doesn't have any .forward yet and leaving the rest up to the script.

#!/bin/bash

DATE=`date`
TMP_DATE=`date +%Y%m%d`
EMAIL_SUBJECT="AutoReply Status"
EMAIL_FROM="blah@blah.com"

USER="$1"
FORWARD="/home/$USER/.forward"
DOTFORWARD="/home/$USER/dotforward"
VACATION=$(which vacation)

if [ -z "$1" ]; then
echo "usage: $0 username"
exit
fi

echo "Doing the .forward thing with user: $USER"

if ! [ -e $FORWARD ]; then
echo "No .forward found, is there a dotforward?"
if [ -e $DOTFORWARD ]; then
echo "Found $DOTFORWARD, moving it to $FORWARD"
EMAIL_BODY="Hello $USER, I have enabled your AutoReply E-Mail as of $DATE"
mv $DOTFORWARD $FORWARD
echo "Moved $DOTFORWARD to $FORWARD"
fi
else
echo "Hmm, there's already a $FORWARD, I'll just add or remove the vacation reference..."
if [ -e $FORWARD ]; then
if grep "vacation" $FORWARD
then echo "Oooh I found a vacation reference in here! Let's DELETE it buwahaha"
sed -e "s!\"|$VACATION $USER\"!!g" $FORWARD > /tmp/$USER_forward-$TMP_DATE
mv /tmp/$USER_forward-$TMP_DATE $FORWARD
EMAIL_BODY="Hello $USER, I have disabled your AutoReply E-Mail as of $DATE"
else
echo "Didn't find any vacation reference, I'm adding one"
if ! grep "\\$USER," $FORWARD; then
echo "\\$USER," >> $FORWARD
fi
echo " \"|$VACATION $USER\"" >> $FORWARD
EMAIL_BODY="Hello $USER, I have enabled your AutoReply E-Mail as of $DATE"
fi
fi
fi

echo "$FORWARD now looks like:"
echo `cat $FORWARD`

echo "$EMAIL_BODY" | mail -s "$SUBJECT" $USER

Mikrotik RouterOS Firewall Script

2007-08-08T11:49:00.000+10:00

The following will hunt through the firewall filter list and enable/disable all rules whose comment is "Drop_Toggle". Usefull if you want to toggle particular sets of filters periodically etc.

# Enable Drop Rules
:global list ""; :foreach i in [/ip firewall filter find] \
do={:if ([:find [/ip firewall filter get $i comment] "Drop_Toggle"]=0) \
do={/ip firewall filter set $i disabled=no} };

# Disable Drop Rules
:global list ""; :foreach i in [/ip firewall filter find] \
do={:if ([:find [/ip firewall filter get $i comment] "Drop_Toggle"]=0) \
do={/ip firewall filter set $i disabled=yes}};

Mirroring a Plesk vhost script

2007-08-06T10:48:00.000+10:00

The following script will mirror a vhost from a Plesk managed server. It is up to you to modify the Apache vhost configuration includes (usually there's one created by Plesk in /etc/httpd/conf.d or the like).

#!/bin/bash
# RSYNC/SED script to mirror a Plesk host
# 20070801 Ben Johns

# Requirements:
# SSH Pub/Priv keys shared on both hosts
# sshkeygen -t dsa -b 1024 -f `whoami`-`hostname` (NO PASSPHRASE!)
# copy the resultant .pub file to the remote host and append it too
# the RSYNC_USER's .ssh/authorized_keys file.
# RSYNC Version >2.6.3
# HTTPD.INCLUDE needs to be manually configured to suit the config
# of the local host. Ie copy the relevant sections from the remote hosts
# plesk httpd conf to this host. Usually done somewhere in /etc/httpd or /etc/apache.

# REM_HOST: The remote host to mirror
# RSYNC_USER: The user account on the remote host that has permission
# to copy the intended files.
# RSYNC_OPTS: Parameters to use with the rsync command
# SSH_KEY: The private DSA key to use for SSH authentication
# RSYNC_VHOST_SRC_PATH: Path to the source virtual host files on the remote host
# RSYNC_VHOST_SRC_DIR: Directory of the source virtual host files on the remote host
# RSYNC_VHOST_DST_PATH: Path to the destination on the local host
# SED_VHOST_MOD_FILE: Location of the SED parameters to modify VHOST config files

REM_HOST="web.server.com"
RSYNC_USER="rsync"
RSYNC_OPTS="-avz --perms -q --deleteduring"
SSH_KEY="/var/www/rsync_ssh_key"
RSYNC_VHOST_SRC_PATH="/home/httpd/vhosts/"
RSYNC_VHOST_SRC_DIR="vhost_directory"
RSYNC_VHOST_DST_PATH="/var/www/vhosts/"
SED_VHOST_MOD_FILE="/var/www/vhost_include.sed"

rsync $RSYNC_OPTS \
-e "ssh -i $SSH_KEY -l $RSYNC_USER" \ $RSYNC_USER@$REM_HOST:$RSYNC_VHOST_SRC_PATH$RSYNC_VHOST_SRC_DIR $RSYNC_VHOST_DST_PATH
rsync $RSYNC_OPTS --include "*/" --include "*.include" --exclude "*" \
-e "ssh -i $SSH_KEY -l $RSYNC_USER" \
$RSYNC_USER@$REM_HOST:$RSYNC_VHOST_SRC_PATH$RSYNC_VHOST_SRC_DIR $RSYNC_VHOST_DST_PATH

for file in $RSYNC_VHOST_DST_PATH$RSYNC_VHOST_SRC_DIR/conf/httpd.include ; do
sed -f $SED_VHOST_MOD_FILE "$file" > tmp_file
mv tmp_file "$file"
echo "Modified $file"
done

chmod ug+rwx R $RSYNC_VHOST_DST_PATH$RSYNC_VHOST_SRC_DIR

apache2ctl graceful

Planning for high traffic

2007-07-29T16:32:00.000+10:00

The most worrying project on my list at this time is working out how to achieve as much grunt as possible to withstand the estimated traffic from the first Steve Irwin day tribute website since his passing.

Currently the Zoo has a single primary server hosting all sites and a secondary web server sharing the load on a few sites. It's not the perfect model by far and I intend on tidying it up as follows.

What I plan to do is install MySQL 5.x on the secondary server and ready it for replication. Then I will dump the contents from the existing MySQL 4.x databases and point all the websites at it. Once I'm satisfied that it's functioning as expected (I'm in the process of testing this on the bench). Then I will upgrade the MySQL 4.x to 5.x on the primary server and begin multi-master replication with the other. This completes stage one of the preparations.

With the databases in place and functioning, I will work out how to replicate all vhosts between the two servers. Plesk, the web control panel operating on both servers, makes this more complex than it really needs to be since I will also need to create the client/domain accounts for each of the replicated sites. When I have worked out what is required I will script the synchronization as much as possible and hopefully end up with near 100% automation. I may need to tap into Plesk's API to do this. This will complete stage two.

With replication of both the databases and general structures taking place between the two existing servers I can now introduce more servers and the greater complexity they will bring.

I will be working towards four application servers just for static content/scripts and a single localised database server. I intend to just have raw boxes running either Redhat or FreeBSD, no fancy control panels getting in the way. This will allow me to script everything with simplicity and provide a basic configuration to each server. I will replicate the data from the first of the existing servers to one of the new application servers and from there to each of the remaining three. This is so I can keep the amount of public traffic between the servers to a minimum. I will introduce the new database server into the multi-master replication loop and point the four new application servers at it. This completes stage three.

Once I am satisfied that each application server is connecting to the database server over their private network and that the database server is successfully replicating the databases from the existing two servers I will set up the load balancer to include the four new application servers. We may need to cut over to a new load balancer since the existing one may not support this many servers.

This setup will provide me with six front end servers and three database servers - with a bit of sharing of resources here and there. The following diagram shows what I intend on achieving.

Relevant links:
ONLamp Advanced MySQL Replication Techniques
MySQL 5.0 Manual - Replication
Rsync
SWSoft Plesk - Upgrading MySQL

These updates are getting scarce...

2007-07-21T15:46:00.000+10:00

It's not like many people come here for the regular updates anyway.

Network Storage adventures
To get some kind of data redundancy going here at the Zoo I've been playing around with different NAS based operating systems.

The first I tried was FreeNAS - using FreeBSD as its base operating system it provides a modified 'Monowall' administration web interface that allows all sorts of functions for manging storage, services, shares and users. I used it for about 2 months and it failed me when I upgraded the system with 2 more 500GB SATA disks plugged into a Silicon Image 2 port PCI SATA controller. FreeNAS is fairly simple to get up and going - the hardest part was working out the confusing workflow of setting up the physical disks, RAID and partitions.

Given the failure (due to some buggy Sil driver in BSD) I decided to try out OpenFiler instead. I knew that Linux had better support for the Sil chipset - although I probably wouldn't base my whole NAS system around support for cheap controller card in future. OpenFiler is based upon rPath which is a embedded Linux based OS. It's unique package management system, Conary, makes updates quite simple - it's similar to Debian's apt-get.

The OpenFiler installation was rudimentary as far as Linux installations go - it's almost identical to RedHat/Fedora. However it threw up a bunch of obscure and useless error messages while trying to work out the existing partitions - a bit of googling later revealed that it probably didn't understand the GEOM volumes left over from FreeNAS. Once I let it re-initialize the disks it was all fine.

The biggest gripe I had with the whole installation of OpenFiler was that initially I thought that it required a network Directory service to operate. Painful as the only resemblance of a Directory I had was the AD running on some SBS2003 R2 server. So I tried in vein to get that running, no success. I did manage to get somewhat farther using a NT domain function but that only showed groups, not users. It wasn't until I dug through the OpenFiler forums that I found the latest version included it's own OpenLDAP directory service.

So I tried updating it using it's web interface - no go, it would appear to download and install various components but what appeared to be a few key components had to be installed 'in the background'. After waiting sometime it didn't appear it was doing anything of a sort. So I took to it's CLI and worked out how to use the conary package manager and ended up with: #conary updateall --replace-files . Which worked.

Once the system was updated I was presented with a few extra LDAP specific configuration options. Ticked and typed in the appropriate things and I was in business with local directory user/group authentication. I created the LVM PVs/VGs and Volumes and formatted the resultant 2.4TB partition using Ext3.

I set up the SMB and FTP services and shared it out to the appropriate groups. The extra 'host/network' based access control took me off guard and once I worked out how that side of things should work I had workstations backing up to the server using SyncBack in no time.

So far out of one File Server and two departments I have used up 425GB. That's from about 12 workstations total. Now another 10 or so departments and 115 workstations. In future I'll look at its iSCSI and HA support.

IPSec encrypted GRE tunnel, MTU settings suitable for PPPoE/A link:
crypto isakmp policy 10
encr 3des
authentication pre-share
group 2
crypto isakmp key shared_key address 1.2.3.4 no-xauth
!
!
crypto ipsec transform-set transform_name ah-sha-hmac esp-3des esp-sha-hmac
mode transport
!
crypto map map_name 10 ipsec-isakmp
set peer 1.2.3.4
set transform-set transform_name
match address 101
!
!
!
interface Tunnel0
ip address 192.168.254.1 255.255.255.252
ip mtu 1500
keepalive 10 3
tunnel source Dialer0
tunnel destination 1.2.3.4
tunnel key 12345
!
access-list 101 permit gre any any
!
interface Vlan1
ip tcp adjust-mss 1400
crypto map map_name

What have I been doing?

2007-06-24T15:37:00.000+10:00

Been a while since I last posted so time for a update post!

Revamping the old network
The network at the Zoo was a miserable mess - it took me a while to audit what was in place and to devise a topology that would best address the current and future needs of the Zoo. Now the Zoo has a VLAN'd network consisting of dedicated Administration, Point-of-Service and VoIP subnets, OSPF routing at the core, a DMZ, traffic policing and shaping capabilities and VPN (PPTP/L2TP and IPSec) capabilities.

I achieved all this by using two rackmounted WRAP1-2's from Yawarra and a cheap Asus GigaX 2024 switch. I loaded Mikrotik RouterOS 2.9.42 onto the two WRAP1-2's and set up 802.1q VLANs on the switch. The VLANs are routed on the first WRAP1-2 which then connects onto the DMZ where the other WRAP1-2 and Cisco 857/877 routers exist with OSPF routing throughout. The second WRAP1-2 holds up the 1Mbit Unisky wireless connection (PPPoE, over wireless... yuk) and hopefully a substantial fibre based service from someone, such as a 2Mbit E1/G.703 service.

I used pairs of Cisco 8xx series routers to hold up VPN links between the Zoo and its newly opened Mooloolaba retail store. A pair of 857's hold up a general Point-of-Service/LAN traffic IPSec tunnel. In addition to that a pair of 877's hold up a VoIP/Video IPSec tunnel with QoS. The two DSLs are 8Mbit/384Kbit links supplied by Bigpond. Having dedicated 'pairs' of routers/DSL for VPN connectivity is overkill but it's still cheaper than a single fibre service.

These changes provided the framework for the following additions to the network infrastructure.

A new phone system
The Zoo's old Siemens key system was well and truely past its time and was needing upgrades which proved to be exorbitantly costly to do. A new Alcatel OmniPCX PBX was selected and installed by company called Nexon Asia Pacific. Along with the digital and analog extensions a number of VoIP extensions are provided including wireless VoIP sets. Best practice says to establish a dedicated subnet for the PBX/VoIP services to reside within so as to isolate it from the general traffic of the other networks. Having VLAN capability is useful as I can locate the phones nearly anywhere and still keep them within the VoIP subnet. However while the phones support VLANs, they don't want to communicate with the Asus switch.

First it was wireless and whales, now its wireless and... um... elephants?
I will soon have a Zoo wide wireless network built up of Symbol WS5100 and AP300s. These were provided by Barcode Dynamics in addition to inventory/asset tracking equipment. The WS5100 is useful in that it can map VLANs to WLANs - allowing me to simply create wireless extensions of the existing networks with no physical modifications. However security becomes a concern with the absence of a router/firewall - the WS5100 addresses this by supporting WPA1/2, 802.1x and firewall policies. I will also limit transit between the networks and wireless infrastructure via the routers.

To start with the wireless will be used for mobile VoIP. Since the Alcatel mobile sets are basically Spectralink reference designs I can simply apply the pre-configured Spectralink QoS policy on the WS5100 to that WLAN so that it grants expedited access to the wireless bandwidth to VoIP traffic. In the future we will also implement mobile Point-of-Service terminals, either PDA style units or small form factor PCs. There's also the possibility that the roaming photographers could also use the coverage to upload their digital photo's in real-time to the on-site photography lab.

I've just finished setting up a outdoor enclosure for one of the AP300's. It's a pity that the AP300 doesn't have an outdoor variant. The supplied enclosures were just bare boxes, luckily they came with the backing board. However I had to make up the pole brackets myself using some angle brackets, u-bolts and pop-rivets.

Mobile VPN over Telstra's NextG
For the newly launched Whale One vessel the Zoo has established a NextG mobile data service. To connect the boat to this service a ruggedised NextG modem/router was installed on the boat with a 7dBi collinear antenna. The router comes with a PPTP VPN client so I have set this to establish a VPN back to the Zoo. This allows the two Point-of-Service terminals to communicate back to the Zoo's POS services for EFT transactions and accounting/stock control. Under testing we managed to maintain a connection out to 10km to sea and sustain an average data rate of 1.5Mbit/sec. I have yet to test the link with the POS systems running.

Marinanet is coming to Coffs Harbor

2007-05-20T20:37:00.000+10:00

In the not so distant future, the beautiful Coffs Harbor marina will become the southern most Marinanet location. It will also be one of the first Marinas to use the new Hotspot system I developed.

This would probably be one of my favorite Marina locations - particularly the most scenic.

No specific installation date set but it is expected to be in the short term.

Attending to the web servers

2007-05-13T17:25:00.000+10:00

Web servers are like the wilder beasts of the Internet. I imagine them to be out in the open grass plains happily grazing in the sun. I also can imagine tigers, cheetahs and other predators lurking around the fringes looking for the few that aren't paying attention or have been wounded and thus are falling behind the herd.

The predators are the numerous script kiddies (skiddies) and crackers out there that either trying it on, seeking to find yet another server to host their warez, or they're building a massive, high bandwidth botnet in which to strike down those who oppose them.

When I looked at my new herd of servers (okay, 3 isn't really a herd...) I saw a neglected bunch that needed some tender loving care. So all this week I have been looking at what makes them tick and for what purposes they serve. In the process I've been cutting the fat, optimizing and tightening things up. There's still a ways to go but things are already looking better, especially after upgrading the link from 10Mbit to 100. I hope whoever owned those domains I disabled doesn't get too pissed.

Things that need to happen:

Consolidate servers into a single rack and connect them via a private LAN
Adjust load balancing to go between all three servers
Establish a managed firewall
Upgrade OS and packages on each

I won't feel comfortable until all that is in place.

Amazing what a bit of tweaking does

2007-05-11T15:31:00.000+10:00

The big long stretch is where the poor MySQL daemon just couldn't cope with the demand. So a few tweaks to how it caches etc later and you see the boost. DB Server load dropped from 45.0 down to 0.40 too.