At first look, it feels like oxide is equivalent to proxmox or some virtualization tool, may be it uses qemu stuff underneath.
Just curious.
The reason I am asking this is that, we have lot onprem scenarios in our business. We are tightly coupled with k8s, to solve this we started building an internal project that is kubernetes API compatible [1] but runs containerd or WASM or our platform natively.
Just curious how oxides work in this scenario
FYI I've got `karpenter-provider-oxide` on my bingo card...
My colleague demo'd Karpeneter internally. We haven't committed releasing it yet but we're discussing it.
[0] https://github.com/kubernetes/cloud-provider/blob/master/clo...
RE Karpenter, it always seemed like a natural fit for Oxide, even more so than some of the currently implementors. And now you have the expertise in the team, I'd be really interested in the reason if you don't go down that path.
Autoscaling is a requested feature, and Karpenter fits that shape naturally. The nuance is to decide where something like the Cluster API provider ends and Karpenter begins since there's a bit of overlap in concerns. Specifically, both want to manage Kubernetes nodes but for different reasons. We're discussing it though. We have a Kubernetes watercooler meeting today where we'll likely discuss the comments in this post!
I’m not an oxide customer, just a fan. I have lots of ideas around how something like this should look, being disappointed with the complexity and also shortcomings of products on the market for this stuff, both in the cloud and on prem.
There's an opportunity to more tightly integrate at the network layer but we'll want to get our load balancer released first.
On AWS EKS Fargate, each pod runs in its own dedicated VM. With Oxide each k8s node is a VM, so you still need something like Talos.
On networking it looks like it is getting closer. Where you can have external subnet give pod routed IPs without overlay. But the gap, as marked by the article, is also the load balancing.
It would be nice if Kubernetes were a native feature out-of-the-box. Also integrated within the existing user/access control.
I don't necessarily want to match the public cloud experience if there's an opportunity for Oxide to exceed the public cloud experience. Eliminating the overlay is a good example of this. We have customers using external subnets to eliminate the overlay but we haven't integrated that into our controllers yet.
That's what we prototyped before local disk was released and before we started disk hot-plug work.
> Could you attach a single large volume, and do path-based provisioning on that?
Possibly. We'd still want disk hot-plug first. Otherwise, customers would have to create their cluster in a certain shape before using PVCs.
[0] https://rfd.shared.oxide.computer/rfd/0001
[1] https://oxide-and-friends.transistor.fm/episodes/rfds-the-ba...