gchristensen changed the topic of #nixos-borg to: https://www.patreon.com/ofborg https://monitoring.nix.ci/dashboard/db/ofborg?refresh=10s&orgId=1&from=now-1h&to=now "I get to skip reviewing the PHP code and just wait until it is rewritten in something sane, like POSIX shell. || https://logs.nix.samueldr.com/nixos-borg
supersandro2000 has quit [Disconnected by services]
supersandro2000 has joined #nixos-borg
cole-h has quit [Ping timeout: 265 seconds]
cole-h has joined #nixos-borg
<{^_^}> [ofborg] @cole-h merged pull request #546 → tasks/evaluate: error on PRs targeting nixos-* or nixpkgs-* branches → https://git.io/JtTBD
<{^_^}> [ofborg] @cole-h pushed to released « tasks/evaluate: error on PRs targeting nixos-* or nixpkgs-* branches »: https://git.io/Jtqyr
cole-h_ has joined #nixos-borg
adisbladis has quit [Ping timeout: 256 seconds]
cole-h has quit [Ping timeout: 256 seconds]
adisbladis has joined #nixos-borg
orivej has quit [Ping timeout: 264 seconds]
orivej has joined #nixos-borg
orivej has quit [Ping timeout: 256 seconds]
WilliButz has quit [Ping timeout: 256 seconds]
hexa- has quit [Ping timeout: 244 seconds]
WilliButz has joined #nixos-borg
orivej has joined #nixos-borg
hexa- has joined #nixos-borg
cole-h_ has quit [Ping timeout: 265 seconds]
<gchristensen> dang cole-h is gone :P
<gchristensen> anyone want to make a maybe fun contribution to the ofborg/infrastructure repository?
<gchristensen> a script which lets you ssh-for-each all the boxes -- something I don't think morph can do? -- and then a buildkite workflow step which runs nix-collect-garbage that can be triggered
orivej has quit [Ping timeout: 256 seconds]
orivej has joined #nixos-borg
orivej has quit [Ping timeout: 256 seconds]
orivej has joined #nixos-borg
kalbasit has quit [Ping timeout: 265 seconds]
cole-h_ has joined #nixos-borg
kalbasit_ has joined #nixos-borg
kalbasit_ has quit [Ping timeout: 240 seconds]
kalbasit has joined #nixos-borg
<gchristensen> cole-h_: a script which lets you ssh-for-each all the boxes -- something I don't think morph can do? -- and then a buildkite workflow step which runs nix-collect-garbage that can be triggered
<cole-h_> morph has `morph exec --on="*"`, which is basically ssh-for-each from what I can tell
<cole-h_> --on="*" Glob for selecting servers in the deployment
<gchristensen> oh neat
<gchristensen> you're half way there :P
<cole-h_> hehe
<cole-h_> gchristensen: By "can be triggered", are you imagining an actual "trigger" step, or a "block" step?
<gchristensen> uhh
<gchristensen> block step, at the end of the pipeline
<cole-h_> Thought so 👍
<cole-h_> gchristensen: To my existing PR, or a new one? :)
<gchristensen> oh oops
<gchristensen> uhh
<infinisil> About T-10 minutes until static fire
<gchristensen> new PR ..... and can you remind me of your PR, uh, tomorrow? :)
<infinisil> Oh wrong chat lol
<cole-h_> gchristensen: Sounds good :D
<gchristensen> thask
<gchristensen> I also wrote it down in my stack of todos ...
<cole-h_> hehe, I know the feeling
cole-h_ is now known as cole-h
<gchristensen> I've managed to keep on top of the stack lately ..... but I'm slow to add stuff to it
<gchristensen> which helps keep on top of it! I only add stuff if I realyl intend on it
<{^_^}> ofborg/infrastructure#39 (by cole-h, 9 seconds ago, open): ci: add nix-collect-garbage step
<cole-h> Also includes "ci: remove unnecessary `nix-shell --run` usage"
<cole-h> oh oops
<gchristensen> uh oh
<cole-h> That "unnecessary nix-shell usage" actually depends on my other PR
<cole-h> :D
<gchristensen> :D
<gchristensen> well
<gchristensen> it looked good, I was just wanting to spy it a bit the first run through
<cole-h> It'll fail cuz `./enter-env.sh: line 104: morph: command not found` :(
<gchristensen> let's see what happens :) https://buildkite.com/grahamc/ofborg/builds/160
<cole-h> you could make a temp branch that adds `nix-shell --run` back hehe
<gchristensen> cole-h: mind sending a fixup for the error in Intake?
<cole-h> oh oh yeah
* cole-h totally missed infra#38 merged
<gchristensen> :D
<gchristensen> time to rip the bandaid off :P
<cole-h> I grepped for `./shell.nix` earlier. I fixed them all, except for that single one lolllllll
<gchristensen> :D
<{^_^}> ofborg/infrastructure#40 (by cole-h, 6 seconds ago, open): terraform/enumerate-servers.sh: shell.nix is one dir up
<cole-h> We go again!
<cole-h> That "/tmp/env-vars: Permission denied" line always scares me because I see the "Permission denied" before the filename
<gchristensen> yeah I don't know what that is about ....
<cole-h> "When XDG_RUNTIME_DIR is not defined, nix-shell / nix-build falls back on /tmp. This create conflict in case of multi-user shared environments( c.f "/tmp/env-vars" problems )"
<gchristensen> hmmm....
<gchristensen> Step Dependencies
<gchristensen> collect-garbage-confirm
<gchristensen> deploy-boot
<gchristensen> probably should not make that depend on the deploy boot step
<cole-h> We don't want to wait for deploy-boot?
<cole-h> OKOK
<{^_^}> ofborg/infrastructure#41 (by cole-h, 10 seconds ago, open): ci: don't depend on deploy-boot
<cole-h> gchristensen: Maybe should also use a different concurrency group?
<gchristensen> hmm maybe
* cole-h just copy-pasted from earlier things which had ofborg-infra-deploy as the group and concurrency: 1
<gchristensen> these thinsg are sorta mysterious
<gchristensen> let's give it a go
<gchristensen> and actually, let's put it to before dry activation even
<cole-h> Maybe can just remove concurrency lol
<gchristensen> so let's just always run it before the dry activation happens altogether
<gchristensen> no block
<cole-h> oh ok
<gchristensen> that way we can deploy even if it is super low on space
<cole-h> +1
<{^_^}> ofborg/infrastructure#42 (by cole-h, 52 seconds ago, open): ci: collect garbage before dry activation
<cole-h> gchristensen: Does nixops' ssh-for-each go host-by-host, or does it do them all async?
<cole-h> Seems like morph's is one-by-one
<gchristensen> async, and yes
<gchristensen> downgrade
* cole-h files an issue
<{^_^}> DBCDK/morph#142 (by cole-h, 5 seconds ago, open): `morph exec --on="*"` should run the command on all hosts asynchronously
<gchristensen> oh no it failed because the key expired, lol
<gchristensen> cole-h: one junky but maybe useful workaround would be to generate a buildkite step per machine
Emantor has quit [Quit: ZNC - http://znc.in]
Emantor has joined #nixos-borg
<gchristensen> thank you, cole-h, that has cleared the page that has been buzzing my phone all day :P
<cole-h> gchristensen: :D
<{^_^}> ofborg/infrastructure#43 (by cole-h, 8 seconds ago, open): ci: collect garbage per machine
* cole-h wonders if that `concurrency: $NRHOSTS` will actually do what I think
<gchristensen> minor suggestions
<gchristensen> the concurrency limits is a bit of a joke at this very moment anyway: there is only one builder of that type :D
<gchristensen> but I'll add more
<cole-h> hehe
<gchristensen> okay cole-h I think I finished with that review
<cole-h> Done
<gchristensen> buildkite's ability to dynamically generate more work is so good
<gchristensen> (note: this is a monadic system!)
<cole-h> :o
<cole-h> gchristensen: btw I did make the change from -deploy to -gc
<cole-h> So, uh, we'll see what happens :P
<gchristensen> :D
<gchristensen> oh yeah that isgorgeous
<cole-h> Yeeeeeep
* cole-h wonders if it should be parallelism instead of concurrency
* cole-h doesn't know the difference
* gchristensen either
<cole-h> gchristensen: Well, guess you can cross my earlier PR off your todo list if you haven't already :P
<gchristensen> the stack of todos is on my desk up stairs :) so ... in the AM
<gchristensen> thank you!
<gchristensen> cole-h: 2x thank you for sticking through the little iterations needed to make it fully perfect
<cole-h> Always happy to help.
<cole-h> ("perfect" is debatable, but)
<gchristensen> I'm trying to let something working be considered perfect enough :P
<cole-h> Y'know, that's fair.
<cole-h> And probably better than agonizing over every tiny detail
<cole-h> s/probably /
<cole-h> /
<gchristensen> I have a hard time with that, letting mysel get bogged down in to detail
<cole-h> `102291 store paths deleted, 186184.64 MiB freed` and `79538 store paths deleted, 148736.81 MiB freed`
<cole-h> No wonder we were getting those warnings lol
<gchristensen> yeah ....