10 pointsby alcazar2 hours ago6 comments

cuu5082 hours ago
There was no UDP connectivity from any of our servers for ~10 minutes, but TCP was working fine, I could SSH into them.
Even Hetzner's Robot interface was down.
Edit: https://status.hetzner.com/incident/5e901309-0873-42af-90c5-...
- Benderan hour ago
  Very interesting. That would suggest to me that all traffic is passing through a stateful device vs. direct connectivity from a router. Routers can give simple bandwidth metrics. A stateful device or mirrored ports to a netflow analyzer would be required to create detailed connection logs though mirrored ports wont break state such as UDP virtual tables being full. Are they doing detailed logging of traffic?
- alcazar2 hours ago
  > We are currently experiencing a fault with UDP forwarding.
  Exactly right.
  From our logs, the main outage went from 21:01 to 21:10 UTC.
  EDIT: Also https://status.hetzner.com/incident/affecfea-fd04-41a1-a680-... (Fault report on Cloud API and Hetzner Console)
alcazar2 hours ago
It seems to be back up. I can access https://console.hetzner.com/ now, and our VPS are back online.
madjam0022 hours ago
Yeah first time in 6 years being with them that I've had an outage of this scale. Cloud VMs and Dedicated servers all down / intermittent
eknkc2 hours ago
Everything went down but seems to have recovered now.
Grosvenor2 hours ago
They sent out an email about raising their prices earlier today. Perhaps it was everyone running for the doors.
zenon_paradox2 hours ago
[dead]