Yesterday morning Optus completed a scheduled upgrade of a core router in Brisbane. This is where our Sunshine Coast Frame Relay link terminates. Essentially a MPLS service. The upgrade entailed replacing the router with a larger chassis for scalability and hence the layer 1 and 2 configs would have changed as a result -- the probable cause of our issues.
After this upgrade we discovered that any sort of traffic that utilised larger packet sizes stopped working. This included traffic too and fro our database services and the Brisbane based webservices and any kind of secure connection such as online banking and eBay (noo!).
We circumvented the database woes by rerouting outgoing packets from the database server via our Telstra Frame link, creating a circular route. This is because the connection between the two would be established but the reply traffic was too large and the connection would time out. So now replies go out via Telstra and it all works fine -- this saves us having to change source addressing etc.
We're now waiting on the Optus network monkies to do their thing. As much as we love Optus, it just proves that you can't sit with any single provider and having a backup link with another is very worth while.
Now I have to start planning for the new installation at Brisbane today and try and resolve configuration issues with Colubris CN3200 that's going into La Vida on Tuesday.
The Dockside Marinanet upgrade went fine yesterday. We now have our own DSL with static IPs and the MT Router is happily running with its new config and software.
Looked at a few cars last night. There's a Nissan Navara Dualcab 2.4l ute for $14,000, only has 76,000 on the clock too. Real men drive utes.