these internet issues today man 🙃
didn't expect the problems on our highest traffic day ever to have nothing to do with our code but rather:
- cut fiber cable by one of our ISP
- networking issues from our bare metal provider
- cloudflare DNS issues
didn't expect the problems on our highest traffic day ever to have nothing to do with our code but rather:
- cut fiber cable by one of our ISP
- networking issues from our bare metal provider
- cloudflare DNS issues
Comments
including a link to the PDS admins discord 👍
Thank you for helping me learn.
One of those freak things they scare monger about against running your own server hardware that barely ever happen in practice and then also right when things are crazy busy.
Thanks for the hard work y’all do.
But I'm also probably on the other side of the world, so unlikely to be related (unless, a butterflies wings flapped...)
though sometimes that's a bad cable to one of your DNS components.
there's some places we could have better redundancy/faster failover so we're not totally blameless but still wish we'd gotten to work through these issues on literally any other day lol
Sorry, mate, to ruin your party, but this is your everyday from now on...
Maybe look into Fastly? Bonus: they don't support nazi-types of sites like Cloudflare does.
Obviously, only after you get over this hump.
Stay strong, Bluesky engineers!
Whereas, everyone has stories about Cloudflare. Plus nazis. :)