Category

Datacentre

Sep 29
Love0

Post-Mortem on Tuesdays BGP Hijack

By CloudSingularity, Datacentre One Comment

What happened?

Today at 10:46 AM PDT our monitoring system detected a hijack of two of our six IPv4 prefixes. This would have affected some traffic to/from networks who are not yet RPKI filtering invalids. (hint: for those who have no idea what we are talking about, some great resources are listed at the bottom of this page)

Shortly after the automated alert, we started our investigation and determined that Telstra Wholesale Australia (AS1221) had hijacked the two prefixes. Only customers who have IPs assigned from those two prefixes (as in above screenshot) would have been affected. Read More

Nov 16
Love0

New Datacentre Announcement

By Datacentre, News, Updates No Comments

As some of you may know from our earlier emails, the time has finally come to officially announce the launch of our brand new datacentre! A bit of background first, it has been a long time coming for us to move into a new space, being sold out of space in our two current datacentres shortly after opening them. The new datacentre will be slightly different than how we have done things for our last two datacentres, but for the better! Read More

Oct 14
Love0

Post-mortem on YVR1 power incidents

By Datacentre No Comments

We would like to share some details on the power outage incident that occurred on October 2nd and October 7th, and what we are doing to prevent this from happening in the future.

What happened

On October 2nd, our Burnaby Datacentre (YVR1) lost our primary power at 9:40 AM PDT. There was a grid power outage, that lasted approximately 10 seconds that occurred at this time. Our primary UPS, failed to transfer the datacentre load to itself, and instead faulted and turned off the load power completely.  Power was fully restored at 11:09 AM PDT by our on-site technician by manually instructing the UPS to turn back on the load power. Remaining services and customer servers were sequentially restored, and we had technicians on-site for the remainder of the day restoring any remaining services.
Read More

Apr 13
Love0

Infrastructure Upgrades Roadmap

By CloudSingularity, Datacentre No Comments

We are exited to share our roadmap for new upgrades across our entire infrastructure. These will take place over the next 3 to 4 weeks. These upgrades will help us deliver new solutions and make existing ones faster and more resilient. Check back to this post later for updates.

Phase 1 (planned mid April, 2017) – Completed

  • replacement of switch modules in our BladeCenter Chassis to dedicated pass-thru modules
  • upgrade of our current Cisco out-of-band management switch to Juniper EX series

Phase 2 (planned end of April, 2017) – In-Progress 60% complete

  • upgrade of existing Juniper EX series core switches to Juniper QFX5100 core switches across our YVR1 datacentre and Harbour Centre point of presence (PoP)

Phase 3 (planned  June, 2017)

  • upgrade of Fortinet FortiGate appliance with new higher performance 800 series FortiGate’s

Phase 4 (planned July, 2017)

  • upgrade to Juniper vMX core routing

Existing customers rest assured we will do everything we can to minimize the impact on any existing services which may be affected.

This post will be continuously updated as we progress through the roadmap. For current maintenance notifications please visit and subscribe to our status page located at https://status.cloudsingularity.net

Aug 17
Love1

Post-Mortem on Tuesday Evenings Network Incident

By CloudSingularity, Datacentre No Comments

We would like to share some details on the incident that occurred yesterday evening at our Vancouver Datacentre YVR1.

At approximately 21:00 PST on Monday, August 15th there was a brief 4 minute spike in traffic that caused some packet loss.

On Tuesday, August 16th between 16:30 and 18:00 PST we detected intermittent packet loss until around 18:00 where there was a complete loss in traffic.

Traffic GraphWe have concluded both incidents were related to a Distributed Denial of Service (DDoS) attack. Tuesday’s attack was a larger scale DDoS attack at over 3Gbps/300Kpps. The attack saturated our port which resulted in dropped traffic. We worked with our upstream providers to mitigate the attack by null-routing the IP being attacked. At around 18:24 after the null-route, all traffic resumed to normal.

We understand how critical it is to our customers that the network is up 100% of the time. This was unfortunately something out of our control, as is the nature of DDoS attacks. We strive to minimize disruptions and outages regardless of the source of the issue; and promise to remain transparent on these kind of attacks and incidents of any nature. Finally, as we continuously make improvements to the network we will eventually be able to absorb these type of attacks without incident.

Jul 01
Love0

We are opening our second datacentre!

By Datacentre, News No Comments

It’s official, we are expanding with a second datacentre (dubbed, YVR2) to open early next year! The facility will be located in Coquitlam, BC, a great DR location from our current facility. This facility is a little smaller than our current datacentre in Burnaby but we will be offering a wide range of services from it, including:

  • 1U, 2U, 1/8 rack, 1/4 rack, 1/2 rack and full private rack colocation
  • bare metal dedicated servers
  • our full line up of cloud solutions
  • meeting room, A/V conferencing, work space and kitchen amenities

The best part is we will have our own dedicated fibre connection between both sites, you can expect sub 1ms latency between the sites…imagine the possibilities!

As a special promotion, any 1/8 rack or larger colocation plans can split their space across either facility! For example on a 1/8 rack plan, which is 6U’s:

  • 2x 2U servers in YVR1, Burnaby
  • 2x 1U servers in YVR2, Coquitlam

Wait! There’s more, you have the option to get a dedicated private layer 2 VLAN connecting equipment at both sites together. This means communication between both sites won’t have to go over the public internet.