Skip to content

api.yubico.com unscheduled downtime

2017-04-24

One of the YubiCloud endpoints, api.yubico.com, was down between 2017-04-23 08:19:36 and 2017-04-23 10:37:25 UTC.

We are investigating what caused this issue and will update this blog with more information once we have it.

All other YubiCloud endpoints (i.e. api2, api3, api4 and api5) were operational during this time window.

Old clients which are only using the single endpoint api.yubico.com would appear to have been down. The only workaround at this time is to upgrade your client to the latest versions, which would simultaneously use all the available endpoints.

api.yubico.com unscheduled downtime

2017-03-31

One of the YubiCloud endpoints, api.yubico.com, was down between 2017-03-30 22:48:00 and 2017-03-30 23:20:00 UTC.

The problem was caused by the physical host at Rackspace, where the virtual machine for this endpoints resides.

All other YubiCloud endpoints (i.e. api2, api3, api4 and api5) were operational during this time window.

upload.yubico.com unscheduled downtime

2017-01-11

The physical host where our upload.yubico.com VPS was hosted developed an issue, and as a result, Rackspace migrated us to a different physical host.

Service was disrupted between 2017-01-11 07:00:00 and 2017-01-11 09:00:00 UTC.

Issues with api last weekend

2016-12-06

One of the YubiCloud endpoints, api.yubico.com, experienced some unscheduled downtime due to issues with our hosting provider.

Our VPS had to be migrated to a different physical host, due to hardware issues.

The downtime started at 2016-12-03 00:47:00 and ended at 2016-12-03 01:17:00 UTC.

All other api machines (i.e. api2, api3, api4 and api5) were still available at all times. For properly configured clients, no noticeable downtime should have been experienced.

Issues with api2 last Saturday

2016-11-28

Last Saturday, Nov 26th 2016, api2 experienced some unscheduled downtime due to issues with our hosting provider.

There were three instances of downtime as shown below (all times are UTC):

2016-11-26 17:50:00 down
2016-11-26 18:22:00 up

2016-11-26 18:36:00 down
2016-11-26 19:28:00 up

2016-11-26 19:33:00 down
2016-11-26 21:18:00 up

All other api machines (i.e. api, api3, api4 and api5) were still available at all times. For properly configured clients, no noticeable downtime should have been experienced.

Using Nagios exclusively to monitor the YubiCloud

2016-09-08

Nagios is a very important part of our infrastructure. We rely on it to provide us with extensive and detailed checks on all our machines and services. We don’t just monitor if a service is up or down, but also a lot of other metrics which allow us to quickly understand the origin of a problem and hopefully fix it before the external service is effected.

Up until recently we also used Pingdom in conjunction with our Nagios infrastructure. Pingdom made monitoring an external service easy enough, while providing us with graphs we could display here on this blog.

Unfortunately, for various reasons, we have come to the decision to stop using Pingdom and rely solely on Nagios instead.

For the time being this means that we cannot, as transparently as before, display uptime statistics for our api*.yubico.com endpoints. We are investigating alternative ways in how we can render this data from the Nagios monitoring system.

Below are uptime statistics for each of our YubiCloud endpoint for the past year.

api.yubico.com
api2.yubico.com
api3.yubico.com
api4.yubico.com
api5.yubico.com

You may have noticed that api2 and api5 have considerably higher downtime than other endpoints. These two machines are currently hosted with Linode, which went through several DDOS attacks between December 25th 2015 and January 5th 2016. We are considering to move one of these machines to another hosting provider.

The rest of the outages were caused by restarts to the machines, due to updates and security patches. Please note that as a whole, the YubiCloud still has had 100% uptime since its inception.

YubiCloud no longer supports DH cipher suites

2016-09-01

Following up on our previous post, the YubiCloud, as of today no longer supports DH cipher suites. The change was made at exactly 2016-09-01 08:00:00 UTC.

The TLS configuration across all five api machines now is:

ssl_protocols  TLSv1 TLSv1.1 TLSv1.2;
ssl_ciphers    "EECDH+AES128:RSA+AES128:EECDH+AES256:RSA+AES256:EECDH+3DES:RSA+3DES";
ssl_prefer_server_ciphers  on;

api 1, 2, 3 & 5 have already been using such configuration for a couple of weeks. Today, the change was made to api 4, bringing all api machines to a standardized TLS configuration.

testssl.sh results for api4 before and after the change was made are available. Note that the files are plain-text, but with a .doc extension to workaround a WordPress limitation.