<srhb>
fwiw I'm not very happy with flannel, but it's what we use in the tests
<jybs>
But because unable to get local issuer certificate
<srhb>
Ah that's fine
<srhb>
just throw -k at it
<jybs>
Forbidden
<srhb>
That's also fine :P
<srhb>
The keys and stuff are in /var/lib/kubernetes/secrets iirc
<jybs>
And if I want to join three servers I just do it manually, not via nix right?
<srhb>
jybs: Take a look at how the multinode tests do it
<srhb>
jybs: It uses the easyCerts mechanism and makes it easy to do the joining.
<srhb>
But you can also do it by hand of course.
<jybs>
Hangon
<jybs>
"Note: Multi-master (HA) clusters are not supported by the easyCerts module."
<jybs>
Sorry this is why I assumed I couldn't
<srhb>
Right, if you're building a prod cluster, I wouldn't do easyCerts anyway.
<jybs>
Of course
<srhb>
The test exchanges the token through the shared tmpfs of the vms
<srhb>
Which is a bit cheaty, but nice for testing multinode functionality :P
<srhb>
For a real setup, by far the most involved part is setting up proper PKI.
<jybs>
Which ones are the MN tests? Are they in nixpkgs/nixos/tests/kubernetes/?
<srhb>
Which goes for.. Well, anything, not just k8s :P
<srhb>
jybs: They're in both dns and rbac, under the multinode name :)
<srhb>
None of them are multi-master though
<srhb>
iirc.
<jybs>
That's fine
<srhb>
What I would suggest you do if you want to do multimaster right away is get rid of roles = [ ... ]; and set it up manually through the low level interface
<srhb>
From there it's basically following any k8s-the-hard-way guide.
<srhb>
And working out errors in the journal as you go along :)
<srhb>
The daemons are actually quite informative about what's wrong at any given point.
<jybs>
the hard way sounds ominous, but cool
<srhb>
It should be called "kubernetes, but you do all the PKI stuff" ;-)
<srhb>
The rest is easy.
<jybs>
Amazing
<jybs>
helm deploying away
<jybs>
Thanks so much for the help!
<srhb>
You're welcome!
<srhb>
Sorry about the so-so state of the module and stuff
<srhb>
Hopefully we can fix that real soon, and definitely in time for 20.03
<jybs>
To be fair it doesn't matter as long as it works in the end
<jybs>
Once it's in it's in
<srhb>
I hope you'll stick around or at least come back, more people means more feedback means better modules :)
<jybs>
Oh I'll be back
<srhb>
Great!
<jybs>
most likely tomorrow when I can't work out the PKI