<{^_^}>
#82696 (by bhipple, 6 days ago, open): nixos-ami: update nvme_core.io_timeout for linux kernel >= 4.15
<gchristensen>
bhipple: commented
abathur has quit [Ping timeout: 250 seconds]
abathur has joined #nixops
<bhipple>
gchristensen: good idea, updated!
abathur has quit [Ping timeout: 250 seconds]
<bhipple>
I think we're actually really close to getting nixops-aws running cleanly on python3
<bhipple>
aminechikhaoui: if you get a chance, I opened two PRs attached to https://github.com/NixOS/nixops-aws/issues/29. Both of these are mergeable as-is without impacting python2 and reduce the diff of items to do for py3.
<{^_^}>
nixops-aws#29 (by bhipple, 1 hour ago, open): Port to python3
abathur has joined #nixops
abathur has quit [Ping timeout: 264 seconds]
abathur has joined #nixops
abathur has quit [Ping timeout: 250 seconds]
<gchristensen>
nice!
<gchristensen>
bhipple: open the backport PR?
abathur has joined #nixops
abathur has quit [Ping timeout: 265 seconds]
<gchristensen>
aminechikhaoui, adisbladis: btw eelco seems totally neutral on poetryifying nixops, though was there an open question w.r.t. flakes?
<{^_^}>
#83147 (by bhipple, 10 minutes ago, open): [20.03] nixos-ami: update nvme_core.io_timeout for linux kernel >= 4.15
<bhipple>
I think we're getting really close to py3 nixops-aws. 2to3 seems to get us there. It's really unfortunate that 2to3 isn't idempotent and introduces all kinds of noise though :(
<bhipple>
although it looks like it's mostly just the print() function that is non-idempotent, and for reasonable technical reasons, so not too terrible I suppose https://bugs.python.org/msg336673
<gchristensen>
once you add a print() it assumes the rest is py3 :(
<bhipple>
A straight 2to3 run works now (minus fixing a couple duplicate prints), but I think there are a handful of unnecessary list() casts that with human effort we could remove.
<gchristensen>
nice
<bhipple>
I wonder if we should just merge that to master and then clean up ex-post-facto, or go through and try to minify the diff? There are a couple more no-op in-place PRs I could send, like the except T, R syntax change and import fixers
<bhipple>
Yeah let's do 1 more no-op I think, PR incoming
<gchristensen>
bhipple: can you give me a quick review?
<bhipple>
Gonna slowly turn python into Haskell/Rust eh? :D
<gchristensen>
adding types to nixops made the 2->3 transition possible :P
<gchristensen>
mypy told me all of the errors, I just fixed them
<bhipple>
hooray for types \o/
<gchristensen>
yuuup!
<gchristensen>
bhipple: confirming that PR is 2 and 3 compatible?
<bhipple>
Yes, though may require a minimum of python 2.7 (e.g., won't work on 2.3 or w/e)
<gchristensen>
lol ok
<gchristensen>
nice :)
<bhipple>
For nixops-aws on master, are we OK going py3-only, or do we want to straddle? If the former I'll look to just run 2to3, test it, and send a PR; if the latter we can use the futurize module to maintain back compat
<bhipple>
Going to py3 only would be much simpler, but there are libs/tools to help straddle
<bhipple>
FWIW my vote is we go py3 only, since it looks like that's what NixOps itself has done, and it makes the nix packaging expressions and CI simpler, as well as the python code itself
<gchristensen>
we should go py3 only, but before we go do, we should make a pre-python3 branch, just like we did for nixops
<gchristensen>
python2 is dead, let's not pretend otherwise :P
<bhipple>
Sounds good, here's one more that can go and is still py2 compatible. I'd be really surprised if a tool like NixOps cares about the iterator vs. copy distinction in py2: https://github.com/NixOS/nixops-aws/pull/33
<bhipple>
might be one or two more things that could be done, but the 2to3 diff is now small enough that it's reviewable/testable in its own right
<gchristensen>
nice!
<gchristensen>
bhipple: merged :)
<bhipple>
I've taking a look at the remaining diff now, should have another PR in a while
<bhipple>
I'm*
<gchristensen>
cool
<gchristensen>
bhipple: what if you could configure where nixops stores its state file, likethis:
<gchristensen>
{ network.state.local = ./deployment.py; } or like this { network.state.s3 = { bucket = "foo"; key = "bar"; region = "us-east-1"; }; }
<bhipple>
Isn't the "state file" a sqlite db? Or are you talking about a different state file?
<gchristensen>
oops
<gchristensen>
{ network.state.local = ./deployment.sqlite; } or like this { network.state.s3 = { bucket = "foo"; key = "bar"; region = "us-east-1"; }; }
<gchristensen>
:P
<bhipple>
Yes, having a state file in S3 (like terraform does) is critical for any kind of non-trivial enterprise adoption, IMO. I would not in good faith recommend it at $WORK without it.
<gchristensen>
yes of course but my question is really about the mechanism of how you define it
<bhipple>
I know some people (IOHK?) have setup a shared ssh bastion host or sqlite in git or something like that , but it sounds super hacky
<bhipple>
I haven't actually looked at changing that or how it's specified / configured yet, so I don't have a well-formed opinion to give (yet).
<gchristensen>
you don't need to look carefully to have an opinion on the data layout I think :P
<bhipple>
But I vigorously support the idea :)
<bhipple>
Why network.state and not nixops.state?
<gchristensen>
nixops already has a "network" attribute with configuration
<gchristensen>
but maybe it should be different?
<bhipple>
Bikesheddy, I guess; I should take a closer look at how terraform does the state file configuration before making recommendations. Seems there's a bit of a bootstrapping problem where ideally you'd want the state file to be declared consistently like the rest of infra, but might need some cfg outside to point to where the state file is (?)
<{^_^}>
nixops-aws#39 (by bhipple, 9 minutes ago, open): Bugfix: SSH public key creation has to be encoded into bytes
<bhipple>
Previously I was just re-running nixops deploy, but now I'm able to do a full nixops destroy / delete / create / list / deploy-from-scratch
<bhipple>
gchristensen: if you get a chance ^
<gchristensen>
bhipple: maybe you'd like to take the work from adisbladis on poetry for plugins, and try mypy?
<bhipple>
Yeah that'd be interesting to try, I was wondering why mypy didn't find it (I suppose because it's going cross-repos?)
<gchristensen>
wait, no
<gchristensen>
the problem wasn't poetry
<gchristensen>
you need to be on a recent nixpkgs rev because of a bug in how we set up python's site dir
<bhipple>
At any rate, I have now tested all NixOps + NixOps-AWS functionality w/ my cluster and the master branch of both repos! (destroy, delete, create, list, deploy, etc.)
<bhipple>
So while there may be more bugs lurking in the AWS python3 port, I can at least say all the basic functionality is operational