samueldr changed the topic of #nixops to: NixOps related talk | logs:
<bhipple> Any one of you fine folks around to take a look at ?
<{^_^}> #82696 (by bhipple, 6 days ago, open): nixos-ami: update nvme_core.io_timeout for linux kernel >= 4.15
<gchristensen> bhipple: commented
abathur has quit [Ping timeout: 250 seconds]
abathur has joined #nixops
<bhipple> gchristensen: good idea, updated!
abathur has quit [Ping timeout: 250 seconds]
<bhipple> I think we're actually really close to getting nixops-aws running cleanly on python3
<bhipple> aminechikhaoui: if you get a chance, I opened two PRs attached to Both of these are mergeable as-is without impacting python2 and reduce the diff of items to do for py3.
<{^_^}> nixops-aws#29 (by bhipple, 1 hour ago, open): Port to python3
abathur has joined #nixops
abathur has quit [Ping timeout: 264 seconds]
abathur has joined #nixops
abathur has quit [Ping timeout: 250 seconds]
<gchristensen> nice!
<gchristensen> bhipple: open the backport PR?
abathur has joined #nixops
abathur has quit [Ping timeout: 265 seconds]
<gchristensen> aminechikhaoui, adisbladis: btw eelco seems totally neutral on poetryifying nixops, though was there an open question w.r.t. flakes?
abathur has joined #nixops
abathur has quit [Ping timeout: 264 seconds]
abathur has joined #nixops
<bhipple> gchristensen: thanks! I opened the backport PR as well.
<{^_^}> #83147 (by bhipple, 10 minutes ago, open): [20.03] nixos-ami: update nvme_core.io_timeout for linux kernel >= 4.15
<bhipple> I think we're getting really close to py3 nixops-aws. 2to3 seems to get us there. It's really unfortunate that 2to3 isn't idempotent and introduces all kinds of noise though :(
<bhipple> although it looks like it's mostly just the print() function that is non-idempotent, and for reasonable technical reasons, so not too terrible I suppose
<gchristensen> once you add a print() it assumes the rest is py3 :(
<bhipple> A straight 2to3 run works now (minus fixing a couple duplicate prints), but I think there are a handful of unnecessary list() casts that with human effort we could remove.
<gchristensen> nice
<bhipple> I wonder if we should just merge that to master and then clean up ex-post-facto, or go through and try to minify the diff? There are a couple more no-op in-place PRs I could send, like the except T, R syntax change and import fixers
<bhipple> Yeah let's do 1 more no-op I think, PR incoming
<gchristensen> bhipple: can you give me a quick review?
<{^_^}> nixops#1261 (by grahamc, 9 seconds ago, open): Add types to argparse calls
<{^_^}> nixops-aws#32 (by bhipple, 39 seconds ago, open): py3 prep: relative imports and exception syntax
<bhipple> Looking ...
<bhipple> Gonna slowly turn python into Haskell/Rust eh? :D
<gchristensen> adding types to nixops made the 2->3 transition possible :P
<gchristensen> mypy told me all of the errors, I just fixed them
<bhipple> hooray for types \o/
<gchristensen> yuuup!
<gchristensen> bhipple: confirming that PR is 2 and 3 compatible?
<bhipple> Yes, though may require a minimum of python 2.7 (e.g., won't work on 2.3 or w/e)
<gchristensen> lol ok
<gchristensen> nice :)
<bhipple> For nixops-aws on master, are we OK going py3-only, or do we want to straddle? If the former I'll look to just run 2to3, test it, and send a PR; if the latter we can use the futurize module to maintain back compat
<bhipple> Going to py3 only would be much simpler, but there are libs/tools to help straddle
<bhipple> FWIW my vote is we go py3 only, since it looks like that's what NixOps itself has done, and it makes the nix packaging expressions and CI simpler, as well as the python code itself
<gchristensen> we should go py3 only, but before we go do, we should make a pre-python3 branch, just like we did for nixops
<gchristensen> python2 is dead, let's not pretend otherwise :P
<bhipple> Sounds good, here's one more that can go and is still py2 compatible. I'd be really surprised if a tool like NixOps cares about the iterator vs. copy distinction in py2:
<{^_^}> nixops-aws#33 (by bhipple, 1 minute ago, open): py3 prep: replacing iteritems and itervalues
<bhipple> I've submitted what I think are probably the final 2 PRs to go before we fork off the pre-python3 branch:
<bhipple> might be one or two more things that could be done, but the 2to3 diff is now small enough that it's reviewable/testable in its own right
<gchristensen> nice!
<gchristensen> bhipple: merged :)
<bhipple> I've taking a look at the remaining diff now, should have another PR in a while
<bhipple> I'm*
<gchristensen> cool
<gchristensen> bhipple: what if you could configure where nixops stores its state file, likethis:
<gchristensen> { network.state.local = ./; } or like this { network.state.s3 = { bucket = "foo"; key = "bar"; region = "us-east-1"; }; }
<bhipple> Isn't the "state file" a sqlite db? Or are you talking about a different state file?
<gchristensen> oops
<gchristensen> { network.state.local = ./deployment.sqlite; } or like this { network.state.s3 = { bucket = "foo"; key = "bar"; region = "us-east-1"; }; }
<gchristensen> :P
<bhipple> Yes, having a state file in S3 (like terraform does) is critical for any kind of non-trivial enterprise adoption, IMO. I would not in good faith recommend it at $WORK without it.
<gchristensen> yes of course but my question is really about the mechanism of how you define it
<bhipple> I know some people (IOHK?) have setup a shared ssh bastion host or sqlite in git or something like that , but it sounds super hacky
<gchristensen> vs { network.state = { bcakend = "s3"; config = { ... }; }
<bhipple> I haven't actually looked at changing that or how it's specified / configured yet, so I don't have a well-formed opinion to give (yet).
<gchristensen> you don't need to look carefully to have an opinion on the data layout I think :P
<bhipple> But I vigorously support the idea :)
<bhipple> Why network.state and not nixops.state?
<gchristensen> nixops already has a "network" attribute with configuration
<gchristensen> but maybe it should be different?
<bhipple> Bikesheddy, I guess; I should take a closer look at how terraform does the state file configuration before making recommendations. Seems there's a bit of a bootstrapping problem where ideally you'd want the state file to be declared consistently like the rest of infra, but might need some cfg outside to point to where the state file is (?)
<bhipple> BTW, I think this is the final one before going py3:
<{^_^}> nixops-aws#36 (by bhipple, 4 minutes ago, open): py3 prep: foo.values() does not return an indexable list
<gchristensen> terraform can't bootstrap itself in to storign its state in its own self-created resource:
<gchristensen> but yeah
<gchristensen> these are great PRs, bhipple :)
<bhipple> Thanks!
<DigitalKiwi> i found this recently i think it's related to what you're talking about
<bhipple> The final one (not back compat). We got it down to a tiny diff that's human readable :)
<{^_^}> nixops-aws#37 (by bhipple, 1 minute ago, open): Migrate nixops-aws to python3
<gchristensen> bhipple: this is the only one that is spooky to me
<gchristensen> bhipple: great work! :D
<bhipple> gchristensen: Thanks! You can also check one checkbox on now!
<{^_^}> nixops#1242 (by grahamc, 2 weeks ago, open): Release 1.8
<bhipple> Tho that list is missing GCE:
<{^_^}> AmineChikhaoui/nixops-gce#2 (by bhipple, 49 minutes ago, open): Migrate to python 3
<bhipple> self.run_command("test -f /root/.ssh/id_charon_vpn")
<bhipple> Oops, bad copy-paste ^
<{^_^}> nixops-aws#39 (by bhipple, 9 minutes ago, open): Bugfix: SSH public key creation has to be encoded into bytes
<bhipple> Previously I was just re-running nixops deploy, but now I'm able to do a full nixops destroy / delete / create / list / deploy-from-scratch
<bhipple> gchristensen: if you get a chance ^
<gchristensen> bhipple: maybe you'd like to take the work from adisbladis on poetry for plugins, and try mypy?
<bhipple> Yeah that'd be interesting to try, I was wondering why mypy didn't find it (I suppose because it's going cross-repos?)
<gchristensen> wait, no
<gchristensen> the problem wasn't poetry
<gchristensen> you need to be on a recent nixpkgs rev because of a bug in how we set up python's site dir
<bhipple> At any rate, I have now tested all NixOps + NixOps-AWS functionality w/ my cluster and the master branch of both repos! (destroy, delete, create, list, deploy, etc.)
<bhipple> So while there may be more bugs lurking in the AWS python3 port, I can at least say all the basic functionality is operational
<gchristensen> yay :D
<gchristensen> let me know if you know how to go about fixing the redness in
<bhipple> I'm taking a look. Running ./ci/ seemed to pass (exit code 0) when I just ran it locally
<gchristensen> so in this case, it may be that we just merge it
<bhipple> Trying to figure out what exactly failed
<gchristensen> it is complaining that the % of the code covered went down
<bhipple> oh lol
<bhipple> Probably because it's fewer LoC in the PR?
<gchristensen> could be :P
<gchristensen> but if you could add some types to the surounding code, that would make a nice clean-up!
<gchristensen> let me know if you want to do that, if not, I will just merge
<bhipple> I think is already completely typed \o/
<gchristensen> :o :D
<bhipple> I'd say let's just merge this one
<gchristensen> Imprecision went up:
<gchristensen> nixops.known_hosts: 11.43 -> 11.59
<gchristensen> nixops.util: 18.0 -> 18.11
<gchristensen> Total: 26.39 -> 26.4
<gchristensen> so, yeah
<gchristensen> merging :P
bhipple has quit [Ping timeout: 250 seconds]
bhipple has joined #nixops