gchristensen changed the topic of #nixos-borg to: https://www.patreon.com/ofborg https://monitoring.nix.ci/dashboard/db/ofborg?refresh=10s&orgId=1&from=now-1h&to=now "I get to skip reviewing the PHP code and just wait until it is rewritten in something sane, like POSIX shell. || https://logs.nix.samueldr.com/nixos-borg
orivej has quit [Ping timeout: 265 seconds]
andi- has quit [Remote host closed the connection]
andi- has joined #nixos-borg
cole-h has quit [Quit: Goodbye]
orivej has joined #nixos-borg
<gchristensen> LnL: 1.1Gpartial.dump
<LnL> nice!
<gchristensen> LnL: wormhole receive 1-amusement-rematch
<LnL> so, how can I get this? :)
<gchristensen> run that command :)
<LnL> oh
<gchristensen> nix-shell -p magic-wormhole
<LnL> you can't pass config to -p
<gchristensen> huh?
<gchristensen> nix-shell -p magic-wormhole -- run "wormhole receive 1-amusement-rematch"
<LnL> has some openssl-1.0.2u reference on darwin
<gchristensen> ouch
<LnL> anyway
<gchristensen> hey my upload rate isn't so bad today
<LnL> urgh I broke something
<gchristensen> uh oh
<LnL> ok back to this, I'm super confused the logger is completely disabled
<LnL> but if I use the ofborg setup it does print the info message in there (but still nothing else)
<gchristensen> hrm :/
<LnL> femme works, but logs all creates
<LnL> I think this might be related to the workspace stuff
<gchristensen> if you wanted to introduce `slog`, it might be a good time :P :x :')
<LnL> heh
<tilpner> Also check `tracing` if there's been no investment into slog so far
<LnL> maybe I should create a clean branch after this and just start with the workspaces stuff
<LnL> tracing rings a bell from somewhere I think
<gchristensen> info!(number_of_yaks, "preparing to shave yaks"); -> that's the tokio spirit
<tilpner> https://tracing.rs/tracing_flame/ looks like a nice motivator
<LnL> ah, I think I've watched a talk about tracing
<gchristensen> I'm not married to slog
<gchristensen> .instrument(tracing::info_span!("my_future")) :/
<gchristensen> I should just butt out :P
<tilpner> gchristensen: But did you see the code snippet below that?
<gchristensen> cool
<gchristensen> I really need to just butt out and let y'all do the best thing :)
<gchristensen> I have a hard time not feeling grumpy about async things, and I'm trying to not be that person
<tilpner> Nah, it's probably deserved
<gchristensen> I like the slog method of explicitly passing the logger down, so you don't have to add further implicitly magic "color" to the functions (see: "what color is your function?")
<gchristensen> but I guess it doesn't have the instrumentation that tracing does
<LnL> oh yay indexes have finished
<gchristensen> actually tracing seemsto fit myown preference of pushing logging and profiling in to a rich runtime component
<gchristensen> so I do like that
<tilpner> I'm not sure how much color tracing really adds. There's already a "magic" (as in "not looked into until it stops working") components in the runtime context, but I don't think it necessarily infects surrounding code?
<tilpner> ("We should use magic, it's fine if it works" "But what about when it stops working?" "Then we shouldn't have chosen it")
<gchristensen> the answer is obviously only use perfect magic
<gchristensen> my favorite thing about Nix is it is very explicit about the rules you must follow for it to work correctly
<gchristensen> I wish magic always worked that way
<tilpner> Background context can be very incredibly convenient while still not being magic, see Rackets "parameters"
<tilpner> *-very
<tilpner> Unfortunately (but understandably), that's not what the Rust community went with
<gchristensen> have any handy links?
<tilpner> Probably not useful when viewed stand-alone
<tilpner> Essentially global variables, but not with the usual downsides
<gchristensen> scoped globals? :)
<tilpner> The idea of passing a logger, and a tokio runtime, etc., to every little thing isn't very appealing
<gchristensen> yeah, but actually that one really drives me batty: having to use the tokio runtime inside another runtime
<tilpner> Just yesterday I was putting glib and tokio into the same application, do you happen to know any other event loops to go with them? /s
<gchristensen> lol
<gchristensen> exactly
<LnL> here we go...
<LnL> never built on hydra (missing platforms)
<gchristensen> ooh
<LnL> hmm, this one should be tho
<LnL> marked broken 4 years ago:
<gchristensen> nice
<gchristensen> nice nice nice
cole-h has joined #nixos-borg
<gchristensen> btw in a few days we'll be losing 1 evaluator and 1 builder
<cole-h> Just one of those "Packet asked for it back" things?
<gchristensen> I canceled a dedicated box from Hetzner, preferring the packet spot market :)
<gchristensen> I canceled it a month ago
<cole-h> Oh, other way around
<gchristensen> but they don't cancel mid-month
<tilpner> Is Packet actually cheaper?
<tilpner> I looked at their pricing list a few times, which looked very expensive, especially compared to Hetzners auction
<tilpner> (You didn't say cheaper. "Prefer because they do thing X better" is valid)
<gchristensen> the spot market builders are run in an account which is comped by Packet, so they're effectively free. I'm going to stop the patreon at the end of the month, and ask people to instead donate to the foundation, where I'll instead send receipts for reimbursement
<gchristensen> I should write that up and do it before patreon bills :)
* cole-h was going to send some money to the Patreon sometime in the future
<cole-h> Guess I'll make a note to send it to the foundation instead :)
<cole-h> But the real question is.... where else will I become on call if not by paying $66/mo?!?!
<cole-h> Oh, wait...
<gchristensen> lol
<cole-h> s/where/how
<LnL> this is definitively slower...
<gchristensen> what's up?
<LnL> nothing, just much more data :)
<gchristensen> :)
<LnL> would just burn through all the metadata under a minute before because most of the jobs where unknown
<gchristensen> ack
<{^_^}> [ofborg] @LnL7 opened pull request #469 → introduce cargo workspace → https://git.io/JfLDn
<LnL> still don't fully understand what's going on with the logging sometimes it works and other times I had to use RUST_LOG=ofborg=info,ofborg_foo=info
<cole-h> LnL: If you're using `log::info!` and similar, you don't need the macro_use (alternatively, you can keep the macro_use and just use `info!`)
<LnL> are you sure? cannot find macro `log` in this scope
<cole-h> Maybe I'm wrong... brb local experimentation
<LnL> you do only need the macro_use once for the extern crate, which is generally in the lib and not your local file
<cole-h> Ah, it's because `info!` uses the `log!` macro internally, which needs to be in scope. `use log::{info, log};` and you can drop the `extern crate log`.
<cole-h> gchristensen: if a PR has stalled on "Pending -- Beginning Evaluations" for ~2 hours, is it a safe bet it needs to be restarted?
<gchristensen> yea
<cole-h> Sad I didn't find any obvious logs regarding it...
<cole-h> At least the comment filter is still alive this time :)
<cole-h> gchristensen: Is this the lapin you were talking about yesterday? https://github.com/sozu-proxy/lapin
<LnL> yeah, that's the one I've been trying out
<cole-h> lol, whatever you do, do not look at logs for ofborg-evaluation-filter.service
<gchristensen> oh cool lets fix
<cole-h> 👀
<LnL> doesn't seem bad and it supports filtering similar to RUST_LOG just with a superset since it has some structured features
<gchristensen> nice
<LnL> the #[instrument] thing is kind of over the top
<tilpner> LnL: Why event!(DEBUG, ...) instead of debug!(...)?
<LnL> as for spans, if it's really useful somewhere we could use it but this seems fine for probably almost everything
<LnL> still eating some pasta
<gchristensen> hehe
<cole-h> gchristensen: Do you think that "ready for review" should be "interesting"?
<gchristensen> probably not
<cole-h> (I was looking in hubcaps for this stuff... lol oops)
<cole-h> OK, then it's a one-liner change
<cole-h> let's see if there are any new variants before PR, though
<cole-h> Only missing `ready_for_review`, `locked`, and `unlocked`. Worth it to add those last 2 to the enum?
<gchristensen> please :)
<{^_^}> [ofborg] @cole-h opened pull request #470 → ghevent: add missing variants → https://git.io/JfLyQ
<{^_^}> [ofborg] @grahamc merged pull request #469 → introduce cargo workspace → https://git.io/JfLDn
<{^_^}> [ofborg] @grahamc pushed 3 commits to released: https://git.io/JfLyN
<cole-h> If anybody else has a second, I'd like to know why #85951 keeps stalling... eval-lib-tests (and therefore eval) keeps getting stuck in `Pending`
<{^_^}> https://github.com/NixOS/nixpkgs/pull/85951 (by lovesegfault, 2 hours ago, open): binutils: 2.31.1 -> 2.34
<cole-h> Nothing that leaps out to me in the logs
<{^_^}> [ofborg] @grahamc merged pull request #470 → ghevent: add missing variants → https://git.io/JfLyQ
<{^_^}> [ofborg] @grahamc pushed 2 commits to released: https://git.io/JfLSL
<cole-h> Maybe I'm just being impatient, but it just seems weird since the coreutils PR, opened afterwards and seemingly just as hefty, already has labels and reviwers...
<cole-h> (fix spelling where applicable)
<gchristensen> doing a deploy to see if it fixes it :P
<gchristensen> deploy done
<cole-h> It's `checking original out paths`, so I don't think I need to manually intervene this time...
<cole-h> But I'll do it anyways, just to be sure :D
<LnL> is it normal 2 aarch builders disappeared?
<gchristensen> nope
<LnL> seems like a strange thing to break with that change tho
<cole-h> I was looking at evaluators and was thinking "they're all there" and then scrolled a little further x)
<LnL> unless only 2 builders where deployed
<cole-h> Ayyy, binutils finally eval'd
<LnL> btw what's the deploy story now?
<LnL> say if I wake up and nice my thing broke everything
<LnL> notice*
<gchristensen> ughhhhhhhhhhhhhhhhhhhhhhhhhhhhhh I didn't realize the lib tests required building
<gchristensen> that is why it was stalled, it had to bootstrap the world. ughhhhh.
<cole-h> LOL
<cole-h> oof
<cole-h> Sorry :^)
<gchristensen> asouhsntoaheunthoasnuthosnuhsonh
<LnL> oh euh
<LnL> those are nixos tests no?
<gchristensen> I think the lib tests are mostly just evaluation time
<gchristensen> I don't know why they're building
<LnL> ah lib not the passtrough stuff
<gchristensen> yea
<gchristensen> so they bootstrap on the evaluator itself
<cole-h> This sounds annoying
<LnL> I think that's an actual build, or at least is should be
<cole-h> I'm scared all of a sudden
<gchristensen> it should be but I think it happens in the list of evaluations?
<LnL> but it just doesn't depend on much
<gchristensen> okay. deploying.
<cole-h> Deploying what, exactly? 👀
<gchristensen> I need to get you an answer on that. I could do an easy thing or the hard thing
<gchristensen> I think I'll do the easy thing and just rant you access to my other org where I deploy from :)
<gchristensen> I don't remember why I switched it :/
<cole-h> Because that yellow dot might scare people away? :P
<LnL> it's a build
<LnL> unlike everything else in the list
<gchristensen> yeah :(
<gchristensen> not sure what to do on that front
<gchristensen> cole-h: email addr?
<cole-h> Same as last -- cole.e.helbling@outlook.com
<gchristensen> invite sent
<gchristensen> to both you and LnL
<LnL> hmm, is that the right thing?
<gchristensen> hm?
<cole-h> "You've been invited to join the grahamc organization" :P
<cole-h> I think the question is because there's nothing that looks ofborg related in that org
<cole-h> ofb"org"
<gchristensen> lol
<gchristensen> I know, I'm not remembering why i moved it to my personal org
<gchristensen> I'll hav eto move it back eventually but I want to get y'all access now
<cole-h> Roger
<gchristensen> so
<gchristensen> to deploy
<gchristensen> click New Build
<LnL> aha, I can't see that
<LnL> hence the confusion
<gchristensen> try now
<cole-h> Nada here
<gchristensen> try now
<cole-h> Yep, now I can see
<cole-h> (said the blind man)
<LnL> :)
<gchristensen> unblock Dry ACivation, this does nothing but makes the second step real fast. once it is done, unblock the second deploy step, once its done its done
<LnL> right so same as before?
<gchristensen> yep
<LnL> I vaguely remember something about moving some of the ofborg infra itself but know why you also did the team switch
<cole-h> Will the locked "Deploy" become unlocked after unlocking "Dry Activation"?
<gchristensen> I think you could click either and they'd just run, but go in order
<cole-h> So, it's safe to unlock "Dry Activation" as long as I don't "Deploy", yes?
<gchristensen> ye
<LnL> ooh, I see some new stuff I like :D
<gchristensen> the vault bits? :)
<cole-h> Pretty colors
<LnL> yeah
<gchristensen> me too
<cole-h> I should really familiarize myself with Vault sometime when I have more free time (read: am not writing history papers)
<cole-h> OK, I unblocked Dry Activation and then canceled. Hope that doesn't screw anything up...
<LnL> core-0.......................> would restart the following units: nginx.service
<LnL> ^ it just shows you what it would do
<cole-h> Well, now it shows up as red on buildkite and ofborg/infrastructure because I canceled it... :P
<LnL> does copy over stuff first so if there's a disk space issue might not be a good idea
<LnL> but other then that it's pretty safe
<cole-h> Great, I don't plan on doing it again until I really need to x)
<cole-h> Back to 2 rows of pinned tabs on FF.
<gchristensen> rows? they turn in to rows?
<cole-h> Ah, I use Tree Style Tabs :D
<gchristensen> ah
<gchristensen> cool
<cole-h> Somebody took a chunk out of the community box's inodes lol
<gchristensen> volth probably
<cole-h> Hehehe
<gchristensen> they clone all of AOSP
<cole-h> Dawg
<gchristensen> lol yea
<cole-h> Haha that'll do it
<cole-h> Geez, I remember doing that back with my Nexus 7... Big pain
<cole-h> My first official action as a member of grahamc/ofborg was to make ofborg/infrastructure show a big red X (because I cancelled the job... lol). I'm so proud.
<cole-h> s/the job/my "intro to buildkite" job/
<gchristensen> good work :)
<gchristensen> keep it up
<cole-h> Muahaha
<{^_^}> ofborg/infrastructure#15 (by LnL7, 27 seconds ago, open): remove the aarch community builder from filesystem alerts
<gchristensen> I'll let you deploy it :)
<cole-h> Oooh, LnL gonna correct my big red X
<cole-h> I think it's kinda funny how "packet-nix-builder" has 0% reliability x)
<gchristensen> :)
<gchristensen> it has been broken for a while, since I replaced my home system
<cole-h> Does expanding the `./build/deploy.sh` start the deploy? Or do you have to explicitly do that by unblocking it
<gchristensen> hm?
<cole-h> Expanding the logs
<LnL> there's nothing to expand until you unblock it I think
<gchristensen> yea
<cole-h> Got it. Needs to be explicitly unblocked. Thanks :)
<LnL> would be nice to have an extra label with the environment of the hosts or something so it could filter on that instead
<gchristensen> yeah
<gchristensen> I think that is possible?
<LnL> yeah in the scrape config
<LnL> forgot how to do it exactly
<cole-h> How do you mean "environment of the hosts"? Like literally PATH and things?
<LnL> more like the roles of the hosts, a builder dying isn't a huge problem an evaluator is more of a problem
<LnL> gchristensen: ^ that adds a port label so you can sum counters over multiple processes in a stable way
<gchristensen> oh nice
<LnL> it's a pretty weird format but adding a static label is probably simpler
<LnL> maybe this {target_label: __role__, replacement: evaluator}