Who knows, it's entirely possible that I'll dislike everything else out there and return to Kubernetes. In which case declarative infrastructure needs more work. - ThreadSky

icyphox.sh • 15 days ago

Who knows, it's entirely possible that I'll dislike everything else out there and return to Kubernetes. In which case declarative infrastructure needs more work.

Comments

zicklag.katharos.group•12 days ago

I absolutely believe that declarative infrastructure needs more work, and that k8s is generally exhausting and overcomplicated.

I've wanted to make an alternative for a while, but don't have the time. 😅

icyphox.sh•12 days ago

Agreed! I'd really like to see an evolution of K8s without all the Google/cloud/Java-but-in-Go cruft it's accumulated over the years. I really like the "abstraction of compute" model that K8s provides—where you can just toss your workload at it and it'll run it wherever. I'll noodle on this a bit...

icyphox.sh•12 days ago

Something in between K8s and Nix (but not in the Nix lang, maybe something sane like Lua) would be ideal.

zicklag.katharos.group•12 days ago

One of the biggest goals for me was being lightweight and having a masterless architecture so you could trivially add nodes to the cluster and the cluster would manage itself.

The CRDT / eventually-consistent stuff I'm working on seems like it might help and replace RAFT.

zicklag.katharos.group•12 days ago

Still might need small, strongly consistent pieces on top of that, though.

I have a hunch that a lot of the cluster configuration could be safely eventually consistent so that you get partition resilient self-management.

icyphox.sh•12 days ago

Oh wow, I actually dig that a lot. And, not sure if you've come across this already, but the folks building Garage S3 have replaced Raft with CRDTs: https://garagehq.deuxfleurs.fr/blog/2023-11-thoughts-on-leaderless-consensus/

zicklag.katharos.group•12 days ago

Ooh, that's really interesting, thanks for sharing! I added to it to my notes so I don't forget about it.

We actually had that same issue Garage is describing in https://weird.one, where we were trying to build on top of an eventually-consistent system, but needed name reservation to be strongly consistent

snepops.gay•14 days ago

You can always deploy k8s on nixos if you feel the need to fall back. The World your oyster once you're working declaratively.

icyphox.sh•14 days ago

Hah, I do run K8s (well K3s) on NixOS at home. I mostly don't mind it for now since it's been pretty fire and forget (+ my investment in 3 physical nodes). But yeah, indeed, I might fallback onto this in the future.

Comments

Posting Rules

Reply