RackCorp Managed Hosting
Dashboard View:   Timezone:   Datacenter:

Recent Maintenance / Outage Events

11/01/2018 23:28CLOSED
 Packetloss to some destinations [AU (Sydney GlobalSwitch) - Public Internet]
 11/01/2018 23:28 - We are currently investigating packetloss in this location
 11/01/2018 23:37 - We have dropped an upstream provider who was delivering us erroneous routes and raised the incident with them. Network services should now be back to normal with complete reachability
03/01/2018 03:37CLOSED
 Packetloss issue Sydney [AU (Sydney GlobalSwitch) - Public Internet]
 03/01/2018 03:50 - Issue was related to several servers of a customer being hacked at the same time and used for DDoS attacks. They have now been disabled. Apologies for any service disruption.
14/11/2017 02:12CLOSED
 Packetloss [AU (Sydney Equinix) - Public Internet]
 14/11/2017 02:12 - We're currently investigating some packetloss over our Megaport link to Equinix SYD1
 14/11/2017 02:24 - We believe the link is once again stable
27/09/2017 01:07CLOSED
 Loss of network [New Zealand - Public Internet]
 27/09/2017 01:07 - We are investigating a loss of network that started a few minutes ago.
 27/09/2017 01:10 - It looks like an upstream provider in one of the NZ sites has lost all network connectivity. We'll await them to restore the network and will provide any updates as we hear them.
 27/09/2017 01:13 - Services are back up.
22/09/2017 04:36CLOSED
 vmh56 storage issue [AU (Sydney GlobalSwitch) - Virtual Hosts]
 22/09/2017 04:36 - We have had to restart a few VM's (All VMs that were >1TB storage and a couple of ones that had not been rebooted for a long time) that were attached to this storage node. Other ones were live migrated onto alternate infrastructure. We will be migrating these last few VMs over the next hour from smallest to largest. There should be no impact.
17/09/2017 15:20CLOSED
 Nz-sh-vmh2 restarted [New Zealand - Dedicated Servers]
 17/09/2017 15:20 - We have experienced an unexpected reboot of this host. Vms should be back online, please place a ticket if you are not back online
12/09/2017 12:45CLOSED
 Power outage [US (Fremont) - Public Internet]
 12/09/2017 12:45 - We have booted up all available devices after a brief complete power outage.

We are running on alternate routing avoiding this datacenter for any global transit or cdn services still at this time until we are confident of the cause.
 19/09/2017 04:13 - Routing has returned to normal
02/09/2017 20:31CLOSED
 Equinix POP Network Packetloss [AU (Sydney GlobalSwitch) - Public Internet]
 02/09/2017 20:31 - We have identified an issue starting at 20:11 AEST with our networking at this datacenter. We have implemented temporary routes to get services operating and have deactivated equinix peering until we get it back to normal.
23/08/2017 01:39CLOSED
 vmh55 storage [AU (Sydney GlobalSwitch) - Virtual Hosts]
 23/08/2017 01:39 - We're aware that storage performance is seriously degraded on this host. We are working on resolving it in a non intrusive manner.
 23/08/2017 05:44 - Performance is increasing as data is being shifted onto non-affected resources. We do not have an ETA at this stage other than to suggest observing your application performance to determine when your particular services will have been completely moved away from the affected resources.
 23/08/2017 09:36 - Migration has completed. please note that these hosts are running without SSD backing for speed, but theres 96 SATA drives so it should be somewhat okay performance.

We will contact each customer individually to arrange a good timeslot for migration to alternate storage as we'll retire vmh55 after this incident.
22/08/2017 18:16CLOSED
 Some prefixes being redirected overseas [AU (Sydney GlobalSwitch) - Public Internet]
 22/08/2017 18:16 - We have noticed some IP prefixes were being routed via long paths internationally. We have fixed this by way of a service restart and are investigating the incident further
15/07/2017 00:14CLOSED
 Network loss [AU (Sydney Equinix) - Public Internet]
 15/07/2017 00:14 - We're currently investigating the primary link connecting our GlobalSwitch SYD and Equinix SYD1 DCs being offline
 15/07/2017 00:25 - We've requested Equinix investigate the loss of link.
 15/07/2017 01:13 - Equinix has repaired the crossconnect and inter DC primary link operational again.
06/07/2017 10:56CLOSED
 Periodic Packetloss [AU (Sydney GlobalSwitch) - Public Internet]
 06/07/2017 10:56 - We are currently investigating some recurring periodic packetloss on the network
 06/07/2017 11:21 - We have isolated the issue to one of three devices and are troubleshooting further
 06/07/2017 12:13 - We've determined the failing device and are working on failing away from it without impact
 06/07/2017 12:52 - Device has been failed out of the routing path. Things should now be back to normal. If you're still noticing any issues, please raise a support ticket.
19/06/2017 23:21CLOSED
 Some services unreachable [Germany (Databurg) - Public Internet]
 19/06/2017 23:21 - We are currently investigating a handful of servers offline
 19/06/2017 23:51 - A tech is onsite and investigating
 20/06/2017 00:29 - Services have been restored after some faulty cabling was replaced
15/06/2017 01:47CLOSED
 Network Packetloss US [AU (Sydney GlobalSwitch) - Public Internet]
 15/06/2017 01:47 - We temporarily experienced some packetloss due to a large DDoS attack. We have applied some filters to reduce the attack profile. services should be back to normal
14/06/2017 08:36CLOSED
 Packetloss to some networks for 2 minutes [AU (Sydney GlobalSwitch) - Public Internet]
 14/06/2017 08:36 - We are aware of an outage to several networks between 8:23am and 8:25am. We are currently investigating
 14/06/2017 08:47 - We're still seeing high levels of international packetloss. We're working on routing around the upstream issue right now
 14/06/2017 10:19 - A large ddos attack has been blocked and issue resolved.
18/05/2017 13:31CLOSED
 Datacentee coolin failure [Germany (Databurg) - Public Internet]
 18/05/2017 13:31 - There has been a datacenter cooling failure in Interxion FRA10 which has progressively become worse. This has now reached a point where core routers have been forced to be shut down. We will advise as more information comes to hand.
 18/05/2017 13:41 - Datacenter engineers are onsite and are attempting to fix the problem.
 18/05/2017 14:30 - We understand this is a long outage. It is affecting a large number of services, not just RackCorp. We can only wait for the DC to restore cooling.
 18/05/2017 14:49 - One core router has been powered back up restoring a lot of connectivity.
 18/05/2017 16:25 - Cooling system is running again. As soon as temperature is back to an acceptable number, techs will turn on any equipment that was powered off
 18/05/2017 17:05 - Routers are now all back on. Any customer servers that were powered down are being powered back on now
18/04/2017 09:38CLOSED
 Network Packetloss [AU (Sydney GlobalSwitch) - Public Internet]
 18/04/2017 09:38 - We're aware of some packetloss to/from some networks a short time ago. We're currently investigating.
 18/04/2017 09:56 - Between 09:25 and 09:30 AEST today, one of our upstreams in sydney (Vocus) experienced an issue affecting several of their Sydney customers whereby our routes temporarily flapped up/down which caused inbound network issues in Sydney for many of our customers as various ISP network traffic reconverged.

Vocus is still investigating the cause. At this stage we have turned up preferencing on our other network providers in Sydney to limit any impact should it happen again.
13/02/2017 15:58CLOSED
 DNS resolution difficulties for some domains [AU (Sydney GlobalSwitch) - Public Internet]
 13/02/2017 15:58 - We are currently investigating this
 13/02/2017 16:24 - We understand we aren't the only network in Australia that had significant problems reaching nameservers for .au

At this time we believe the issue was related to servers running the .au level domain space and/or a networking issue with those that was well outside of our network
10/02/2017 09:22CLOSED
 Network unavailable from some locations [AU (Sydney Equinix) - Public Internet]
 10/02/2017 09:22 - We are currently investigating the network being unreachable for services in one of our racks in equinix sydney
 10/02/2017 13:03 - Issue is caused by a bad fibre that we're having replaced at this time.
 10/02/2017 13:13 - The existing cable is up and down, customers may notice services resume and then go back to a non-connectivity state while the new fibre is being laid.
 10/02/2017 14:45 - Services should now all be restored. If you're having any problems please raise a support ticket with us
09/02/2017 07:52CLOSED
 Packetloss to some limited networks [AU (Sydney GlobalSwitch) - Public Internet]
 09/02/2017 07:52 - We saw a few blips of ~50% packetloss between our Sydney POP and a major Sydney ISP as well as to a NZ ISP between 07:12 and 07:38 AEST. We're unsure exactly where the fault for this was but suspect it was with that major AU ISP, we're continuing to monitor things.