ixxie has joined #nixos-kubernetes
ixxie has quit [Ping timeout: 250 seconds]
<johanot> arianvp: yeah we still use kube-router in production, but we're actually considering getting rid of it :D it's not that it isn't fine. it just doesn't scale for our cluster. We are currently at 24 nodes with around 70 pods on each. We tried solving the scalability issues with nftables, and the nftables firewall implementation is overall more sane and declarative, but unfortunately we are
<johanot> seeing nftables rule-commit delays of up to 30 seconds and a lot of cpu consumption while rules are flushing to the kernel. When reconciling every 5 seconds or so, this is bad :) I'm afraid we have to leave kube-router soon and do something custom. But at the time we chose kube-router (only 8 node cluster), it was definitely the right choice.
<arianvp> Why not Calico?
<johanot> a couple of years ago, I think kube-router seemed like an easier choice for us, being kubernetes newbies and all. But also, I personally prefer being able to use kubernetes native resources over CRD's.