samueldr changed the topic of #nixops to: NixOps related talk | logs: https://logs.nix.samueldr.com/nixops/
syd has joined #nixops
syd has quit [Remote host closed the connection]
nuncanada has quit [Quit: Leaving]
<gchristensen> btw I rewrote my nixops 2->3 PR: https://github.com/NixOS/nixops/pull/1232
<{^_^}> nixops#1232 (by grahamc, 4 hours ago, open): 2-to-3: Start with types
<DigitalKiwi> did you add all of the type sigs
<gchristensen> heh yeah
<gchristensen> well not all
<DigitalKiwi> so when you porting it to python 5?
<gchristensen> lol
<DigitalKiwi> in python 5 it still does the type checking but the type signature is optional :p
<gchristensen> haha, python still doesn't even do type checking
<DigitalKiwi> for anyone who wants in on the in joke https://www.youtube.com/watch?v=BvECNQRrjCY
<DigitalKiwi> gchristensen: where to start adding a module? write the nix options first or the python first? do one at a time or? :(
<DigitalKiwi> if i copy paste enough i can do this without writing python
<DigitalKiwi> writing or (re)learning
<DigitalKiwi> i will do both simultaneously! 2 laptops one hand each!
<aminechikhaoui> gchristensen woah is that Guido himself excited about nixops being type checked with mypy :D
<gchristensen> :D
<gchristensen> yep
<gchristensen> also some dude on Twitter decided to help out: https://twitter.com/TheWizardTower/status/1233783926117691398
<{^_^}> nixops#1233 (by TheWizardTower, 7 minutes ago, merged): Start with types
<aminechikhaoui> fun !
<DigitalKiwi> Also along the way I fixed at least three crashers which have been there since like 2015.
<DigitalKiwi> lol
<DigitalKiwi> but if you have runtime errors you get immediate feedback!
<aminechikhaoui> gchristensen any idea how can I get mypy to see plugins ?
<aminechikhaoui> btw maybe before merging py 2->3 we should create a branch pre-python3
<aminechikhaoui> to make it easier to use plugins that are not yet migrated
<gchristensen> aminechikhaoui: good idea
<aminechikhaoui> gchristensen so what's your workflow with mypy/nixops ?
<gchristensen> nix-shell in the nixops repo, then `mypy nixops` while I add types. if I'm looking for some new area to add types to, `mypy --html-report ./report nixops` and open ./report/index.html
<gchristensen> but the report options make it much slower, so I don't use it each time
<aminechikhaoui> ./report/index.html doesn't seem to tell what's the problem just red or green :D
<gchristensen> types for plugins: I was futzing with that ... but didn't manage to make it work.
<gchristensen> try adding --pretty? it might help
<gchristensen> oh I did get mypy to sort of work with plugins, but badly
<gchristensen> in like 30min I can show you how
<aminechikhaoui> no rush :-)
<aminechikhaoui> you sure there is --pretty /
<aminechikhaoui> ? *
<gchristensen> yeah, pretty sure
<gchristensen> mypy's output is sometimes confusing / unclear, for sure
<gchristensen> ify ou need help with one, glad to
<aminechikhaoui> looks like I don't have --pretty for some reason
<gchristensen> hm
<gchristensen> try adding pretty = True to setup.cfg under [mypy] ?
<aminechikhaoui> do you remember you mypy version
<aminechikhaoui> ah seems to work with nixpkgs 20.03
<gchristensen> nice
<gchristensen> wow this person is amazing
<{^_^}> nixops#1234 (by TheWizardTower, 1 hour ago, open): Start with types
<aminechikhaoui> nice !
abathur has joined #nixops
<gchristensen> aminechikhaoui: I don't understand what this function's arguments do: def get_keys(self, *_: AnyStr) -> List[str]:
<gchristensen> oh I see:
<gchristensen> def get_keys(self, *_: AnyStr) -> List[str]:
<gchristensen> but it evidently doesn't use the argument
<aminechikhaoui> yeah that can probably be removed
<gchristensen> aminechikhaoui: so maybe fork a maint-1.8 branch, and bump to 1.9?
<gchristensen> maybe not actually do an official 1.8 "release" even
<aminechikhaoui> didn't get what you said, do a release you mean or without a release just a branch named 1.8 ?
<gchristensen> yeah, no actual tagged release for 1.8 -- just a branch people can point to if they need it
<gchristensen> or is that too weird?
<aminechikhaoui> maybe because at some point we should release and would that be 1.8 or 1.9
<aminechikhaoui> so a branch with name pre-python3 looks more clear w.r.t purpose
* aminechikhaoui gonna be afk for 1h or so
<gchristensen> aminechikhaoui: let me know when you're back
<DigitalKiwi> volume-1NYC3 / 1 GBAttach to a Droplet Just now
<DigitalKiwi> :D :D :D
<gchristensen> nice
<DigitalKiwi> [nix-shell:~/projects/github/nixops]$ nixops destroy -d volume
<DigitalKiwi> volume-1> warning: don't know how to destroy resource ‘volume-1’
<DigitalKiwi> ooops ;)
<DigitalKiwi> how do i make it reload environment :(
<DigitalKiwi> instead of having to ./dev-shell, try it, ctrl+d, ./dev-shell, repeat
<DigitalKiwi> volume-1> destroying volume e8e13bad-5b29-11ea-b5f4-0a58ac146a1b
<DigitalKiwi> :D :D :D
<DigitalKiwi> at this rate i'll have added volumes to nixops-digitalocean before anyone even looks at it :(
<DigitalKiwi> well i mean ...other than that they're added
<DigitalKiwi> but like...they don't get attached yet
<aminechikhaoui> gchristensen back now
<aminechikhaoui> DigitalKiwi yeah that's what of the painful things, having to reload nix-shell
<aminechikhaoui> donnow if there is a way to point to local checkout using an environment variable for example
<DigitalKiwi> oh now i feel less dumb
<DigitalKiwi> fewer*
<gchristensen> aminechikhaoui: I was just going to show you my failed attempt to make plugin types work, but then I saw the stupid mistake I made
<gchristensen> okay yeah it still doesn't work
<aminechikhaoui> btw seems plugins aren't loaded in your branch
<aminechikhaoui> [amine@nixos:~/src/nixops]$ ./dev-shell --arg p '(p: [ (p.callPackage ../nixops-aws/release.nix {})])' -I nixpkgs=channel:nixos-20.03
<aminechikhaoui> trace: Using plugins: []
<aminechikhaoui> warning: unknown setting 'experimental-features'
<gchristensen> ehhh....hmm
<gchristensen> I can make my nixops one work
<gchristensen> I'll push a branch
<aminechikhaoui> ok thanks, now back with playing with mypy :)
<gchristensen> aminechikhaoui: branch nixops-plugins-types is what I was trying, see https://mypy.readthedocs.io/en/latest/installed_packages.html
<aminechikhaoui> 👍
<gchristensen> aminechikhaoui: and I'm pushing my nixops-packet branch which works
<DigitalKiwi> how make python language server work
<gchristensen> aminechikhaoui: ehh I did break that
<gchristensen> aminechikhaoui: the problem is we need to callPackage ourselves from inside nixops, to make sure both the plugin and nixops have a consistent env, and a consistent nixops
<gchristensen> aminechikhaoui: overall the plugin model is a bit weird in the release.nix because of how it works ... but we can probably fix it
<gchristensen> we need to be able to pass a nixops to the plugin's build so it can test, but that means a three-phase build: no-plugins, build the plugin, build nixops-with-plugins
<gchristensen> so maybe ... hmm
<gchristensen> aminechikhaoui: I feel using the release.nix makes this over-complicated
<aminechikhaoui> yeah that's a bit complicated
<aminechikhaoui> maybe we should make that logic in shell.nix
<gchristensen> aminechikhaoui: maybe we should have a default.nix which does a simple, one-arch build
<gchristensen> and then use release.nix to fan out
<aminechikhaoui> + some plugins.nix file which we don't check into the repo
<gchristensen> yea
<aminechikhaoui> dev-shell (list of plugins) is also a bit weird
<gchristensen> very
<aminechikhaoui> so using a file might be easier
<aminechikhaoui> where we just point to local or remote git checkouts
<gchristensen> maybe we should think about how we want it to work, then make it do that :P
<aminechikhaoui> yeah :D
<DigitalKiwi> make it run in lorri
<gchristensen> yeah aminechikhaoui it'd be nice to not need to rebuild the plugin to build it again
<gchristensen> to develop it*
<gchristensen> aminechikhaoui: would you mind experimenting and thinking about this a bit? I need to get off the computer a while :)
<aminechikhaoui> sure, which part not rebuilding the plugin or moving stuff out of release.nix or both :)
<gchristensen> the whole "how do we develop on nixops + nixops plugins?" question :D
<aminechikhaoui> yes, will have lunch and then look at that
<gchristensen> thank you!
<gchristensen> aminechikhaoui: nix-shell ./release.nix -A build.x86_64-linux --arg nixops '(import ../../NixOS/nixops/release.nix {}).buildWithNoPlugins'
<gchristensen> this on https://github.com/grahamc/nixops-packet/tree/py3 seem sto work
abathur has quit [Ping timeout: 258 seconds]
<aminechikhaoui> hm so you do that from the nixops-packet repo ?
abathur has joined #nixops
<gchristensen> yea
<gchristensen> ugly
<gchristensen> ooooooh
<gchristensen> maybe for mypy to work we need to buildPythonPackage for the nixops input to the plugin's build
<aminechikhaoui> hm gchristensen so why do we need nixops as input to the plugins in the first place ?
<gchristensen> without it you can't write / run tests or do type checking
<gchristensen> which I assume we want to encourage :P
<aminechikhaoui> you want tests to run within the plugins derivations ?
<aminechikhaoui> it's using a function from nixops.util
<gchristensen> the tests for nixops-packet should be inside of nixops-packet
<aminechikhaoui> yes but maybe we can somehow run them within the nixops derivation when packet is a plugin input
<gchristensen> and nixops.utildevice_name_to_boto_expected should be deleted from nixops and moved in to nixops-aws
<aminechikhaoui> actually it's in nixops-aws but maybe it should be part of ec2_utils instead
<gchristensen> running them as part of the nixops derivation makes sense
<gchristensen> as long as a sensibel nix-shell can also becreated, sounds really good
<aminechikhaoui> I'm not sure we really need the circular dependency between nixops <-> plugins if we can run tests from within the nixops drv
<aminechikhaoui> just not sure how to do that yet
<gchristensen> (and can run the tests inside the nix-shell)
<aminechikhaoui> export PYTHONPATH=../nixops-aws/:$PYTHONPATH seems to make plugins work without reloading nix-shell
<aminechikhaoui> which I guess we can do in shellHook ?
<aminechikhaoui> well I don't know really if it works, maybe I talked too fast
<aminechikhaoui> seeing this while doing `nixops info`
<aminechikhaoui> set(definitions.keys()) | set(depl.resources.keys()), key=name_to_key
<aminechikhaoui> TypeError: '<' not supported between instances of 'method' and 'method'
<aminechikhaoui> File "/home/amine/src/nixops/nixops/script_defs.py", line 233, in print_deployment
<aminechikhaoui> something wrong during the sorting of the resource names or something
<gchristensen> nice.
<gchristensen> :|
<gchristensen> aminechikhaoui: try changing:
<gchristensen> ...
<gchristensen> aminechikhaoui: key = machine_to_key(depl.uuid, name, d.get_type) to key = machine_to_key(depl.uuid, name, d.get_type()) in script_defs.py around line 226
<aminechikhaoui> yay, now it works
<gchristensen> cool
<gchristensen> aminechikhaoui: https://gsc.io/snaps/f47e1a86-6905-4d27-b210-61c16f0f277f.png looks like that section needs more types :)
<aminechikhaoui> and it seems to PYTHONPATH hack is working 🤔
<aminechikhaoui> gchristensen that's the red/green I was complaining about :p
<gchristensen> want to go back to just get_type (minus the ()'s), add the types until mypy complains, and then fix it?
<aminechikhaoui> yeah PYTHONPATH seems to work, that's easy enough :p
<aminechikhaoui> gchristensen btw your nixops-aws branch seems to use another PR that adds deployment.ec2.profile and what not
<gchristensen> do I have a nixops-aws branch?
<{^_^}> nixops-aws#23 (by grahamc, 1 week ago, open): boto3 PR + python2->3
<aminechikhaoui> oh maybe you branched out of another PR which does that
<gchristensen> oh yeah I branched from the boto3 PR
abathur has quit [Read error: Connection reset by peer]
abathur has joined #nixops
abathur has quit [Read error: Connection reset by peer]
<aminechikhaoui> shouldn't mypy be able to infer the type here https://i.imgur.com/W5nw9Uo.png
<gchristensen> hmmm
<gchristensen> aminechikhaoui: I think there is an Any type it is unhappy about
<gchristensen> look at the definiton for machine_to_key?
<aminechikhaoui> this is the diff http://ix.io/2d37
<aminechikhaoui> also I wanted to have List[int] instead of List[object] but it kept complaining
<aminechikhaoui> not sure why int(x) doesn't return int :o
<aminechikhaoui> but this is fun :D
<gchristensen> :D
<aminechikhaoui> mypy with vim support is cool
<aminechikhaoui> :w fix :w fix
<gchristensen> oh nice!
<gchristensen> your diff looks good, I don't know why it is complaining
<gchristensen> #python is very helpful btw :P
<aminechikhaoui> cool
<aminechikhaoui> ah it needed depl.uuid and get_type() to be typed https://i.imgur.com/Pkl8V1e.png
<aminechikhaoui> poor mypy was trying to send a message
<gchristensen> ah!
<gchristensen> wow that is so beautiful
<gchristensen> nicely done, aminechikhaoui :)
abathur has joined #nixops
<aminechikhaoui> now need to figure out why it doesn't like `key = machine_to_key(depl.uuid, name, "")` still :D
abathur has quit [Read error: Connection reset by peer]
abathur has joined #nixops
abathur has quit [Read error: Connection reset by peer]
abathur has joined #nixops
abathur has quit [Ping timeout: 260 seconds]