johanot has joined #nixos-kubernetes
johanot has quit [Remote host closed the connection]
johanot has joined #nixos-kubernetes
ixxie has joined #nixos-kubernetes
<ixxie> yo
<ixxie> johanot: tried to build your branch and got https://gist.github.com/ixxie/65ea21e7158a3c657372c1b8742e4d6f
<ixxie> of course I might have made a mistake porting that configuration
<johanot> ixxie: hmm.. looks strange. I'm gonna have to look at that later, will be off for 2-3 hrs now.
<ixxie> thanks johanot
<johanot> "signal: killed" .. OOM? :P
<srhb> Yeah that looks like OOM
johanot has quit [Quit: leaving]
<ixxie> srhb: what is OOM?
<srhb> ixxie: Out of memory
<srhb> ixxie: The build failed to allocate enough memory while doing something, so it barfed :)
<ixxie> aah
<ixxie> srhb: I was actually thinking to move the build process from my laptop to the master or something
<ixxie> but I guess a dedicate deployer may be wiser if the master runs stuff
<ixxie> srhb: any tips on how to manage the build without scaling up the build machine?
<srhb> More swap?
<ixxie> hmmm
<ixxie> gonna dig up an option for that
<ixxie> srhb: are you sure Nix can use swap?
<srhb> ixxie: Yup. It's completely transparent to Nix.
<srhb> It just appears as "moar rams"
<ixxie> so any clue to which setting to tweak?
<srhb> ixxie: On a NixOS machine?
<ixxie> yeah
<ixxie> oh
<ixxie> wait
<ixxie> I remember now I have a whole volume for swap on this machine
<ixxie> I just need to specify in swapDevices
<ixxie> that should do the trick :)
<ixxie> srhb: I guess there is a very good usecase for swap when you run Nix is a memory constrained environment xD
<srhb> ixxie: Oh yeah, bigtime. Especially with the 2.0.x memory leaks.
<srhb> Well, actually not leaks. But definitely not efficient memory usage.
<ixxie> does nixops already use 2.0?
<ixxie> it also seems Nix's memory usage on my machine is capped to about 4GB (I have 8)
<srhb> ixxie: iirc nixops just uses whichever Nix is on PATH
<srhb> And there's no cap, normally
<ixxie> hmm I see
<ixxie> srhb: speak of the devil: https://discourse.nixos.org/t/nix-2-1-released/875
<srhb> Yup, indeed. Not in any release yet (except as nixUnstable)
ixxie has quit [Ping timeout: 252 seconds]
ixxie has joined #nixos-kubernetes
johanot has joined #nixos-kubernetes
<ixxie> srhb: so with swap the build suceeded, but while it seems kubernetes is installed (I see k8s containers when I run `docker ps`) it seems I don't have a `kubectl` command available in my path
<ixxie> nor does the root user
<srhb> ixxie: You can just add it to environment.systemPackages
<srhb> It's funny that you expect that. I think we debated whether the module should do that. I lean towards "don't silently stick things on the global PATH"
<srhb> But good feedback.
<johanot> While testing something else today I realized that the etcd module sticks "etcdctl" on PATH, I thought I had to ask for it specifically. Apparently not :)
<johanot> my personal opinion is that the system PATH should be kept as clean as possible, but I'm not totally immune to the other argument still.
<ixxie> srhb: well, at the very least the current version on unstable DOES supply kubectl
<ixxie> because I haven't had to install it seperately
<ixxie> so if you remove it you better warn people somehow
<srhb> ixxie: Ah. My main concern is that it's easy to accidentally depend on things in /run/current-system in the actual module if you stick it on there automatically.
<ixxie> fair enough!
<srhb> ixxie: That's funny though, I don't see it being set automatically on unstable
<johanot> ixxie: kubectl is part of the outputs from the kubernetes package, also in stable.
<srhb> Oh, right, I see..
<johanot> but that doesn't put in in PATH :)
<srhb> It does...
<johanot> it in*
<srhb> environment.systemPackages = [ cfg.package ];
<johanot> damn, thought I removed that
<srhb> In current unstable?
<srhb> Jaka Hudoklin 2014-11-23 01:27:04 +0100 1147) environment.systemPackages = [ cfg.package ];
<johanot> Ah.. No, but in 1.11
<srhb> Right, I think you did
<ixxie> alrighty
<srhb> So either release notes or bring it back.
* srhb shrugs
<johanot> It didn't feel right to have all k8s thingies on global path
<srhb> I agree.
<johanot> but of course, relnotes.. aye aye :P
<ixxie> hmmm
<ixxie> [ixxie@flux-master:~]$ kubectl get pods --all-namespaces
<ixxie> The connection to the server localhost:8080 was refused - did you specify the right host or port?
<ixxie> what do I make of this?
<johanot> ixxie: export KUBECONFIG=/etc/kubernetes/cluster-admin.kubeconfig
<ixxie> [ixxie@flux-master:~]$ kubectl get pods --all-namespaces
<ixxie> error: unable to read client-key /var/lib/kubernetes/secrets/cluster-admin-key.pem for cluster-admin due to open /var/lib/kubernetes/secrets/cluster-admin-key.pem: permission denied
<johanot> ixxie: Sorry. Should have linked you here in the first place: https://github.com/NixOS/nixpkgs/pull/45670/files#diff-8c4e3a8a3bb211a53525fb97850e1fcfR115
<johanot> Basically, you need to be root to access the private key for the cluster-admin cert
<johanot> It's the reason why I made a wrapper for kubectl in my test cluster: https://github.com/johanot/machines-mirror/blob/master/configurations/nixos/nixos-1.nix#L5
<ixxie> oh right
<ixxie> you did this today right? I missed that yesterday
<ixxie> but I suppose there is a more permenant setup to be done where certain users get access to the cert?
<johanot> ixxie: nope. I added firewall ports today, because I wanted to demonstrate that the nixos firewall doesn't have to be disabled to use the k8s module. But the kubectl wrap was there yesterday as well
<johanot> The auto-generated cert and kubeconfig is a way of getting the same functionality as before, where an admin (having a console on the master-server) can obtain cluster-admin rights.
<johanot> Additional users/roles/rolebindings must be added manually, for now.
<ixxie> alrighty
<ixxie> I only dimly follow whats going on here, but I am sick and quite tired now; so I will try out the wrapper tomorrow and probably ask more questions at some point
<ixxie> thanks a bunch for all the info again!
<johanot> ixxie: any time :)
<ixxie> good night!
<johanot> Users that aren't up for a deeply granular access control scheme, are most welcome to just copy the admin cert and key around as needed. At least RBAC ensures that random people without any (or a self-signed) cert cannot access the cluster.
<johanot> Before, everyone could just access port 8080, no questions asked.
<johanot> ixxie: see ya. gn
<ixxie> right
<ixxie> see ya!
ixxie has quit [Quit: Lost terminal]
johanot has quit [Quit: leaving]