<LnL>
there's no autoscaling for darwin so 40k builds take a while
<LnL>
linux ran 200 concurrent builds for a about a day
cole-h has quit [Quit: Goodbye]
cole-h has joined #nixos-borg
<MichaelRaskin>
But then I think it makes little sense to auto-schedule builds that will take a few days to be started.
<MichaelRaskin>
Right now I pushed, ofborg auto-scheduled a Darwin build, but nothing will happen with that auto-scheduled build for the forseeable future, then later it creates some load when everyone has stopped caring
<LnL>
ah yeah, we might want to flush the queue before I start it again
<cole-h>
I meant to ask about that yesterday: do we have a way to flush the queue?
<LnL>
don't have access to the management interface but there's a purge button there
<gchristensen>
if we start it, won't it blow through the queue quickly?
<LnL>
yeah it might be ok
<LnL>
none of those jobs are filtered yet so a bunch are cantbuild and will be skippeed
<gchristensen>
yea
<MichaelRaskin>
I think just unrequested creation of queued jobs is a bit confusing in the current situation
<MichaelRaskin>
But maybe there is no cheap way to suspend it
<LnL>
I think disconnecting the exchange might work for that
<LnL>
so I have a patch for carnix and/or a one line change I could do to the generated Cargo.nix
<LnL>
what should I do?
<gchristensen>
*preferrably* patch Carnix..... but I've been known to check in patches to the Cargo.nix
<gchristensen>
(so that it could be regenerated, and then have patches applied automatically)
<cole-h>
+1.
<cole-h>
A little sad that Carnix's upstream seems to be stalled at the moment
<gchristensen>
LnL: could you send me the patch for carnix? maybe I can get pierre to apply it
<gchristensen>
then let me know what I should do and ping him about :) and I will
<LnL>
aww the ui doesn't work for my second branch :/
<LnL>
do you know what the correct way to push a patch do a discussion is?
<cole-h>
Anybody know why ofborg sometimes misses PRs? I can look at logs in ~an hour, but the past day or two I've had to kick off evals manually on a handful of PRs.
<cole-h>
Example PR that I haven't kicked off yet: #86336
<gchristensen>
LnL: since I can't load them, could you give me a very short thing to say about them in the ping?
<gchristensen>
I'm going to say like "hey Pierre, my friend Daiderd wrote up a couple patches to carnix to fix some issues we've been having when using it on ofborg. for example «a few words». we had some trouble with the nest, but could you take a look?"
<LnL>
first fixes cargo workspaces, second operator precedence of cfg expressions
<cole-h>
gchristensen: How often do borg's logs get rotated? There were 2 internal errors ~11 hours ago, but nothing in the logs (didn't even get paged using the Discord thing I hooked up)
<infinisil>
(which means I might have to look into how to make a good performance benchmark for NixOS itself)
<gchristensen>
LnL gogogo!
<LnL>
looks like hydra has also built ghc already so it might even succeed
<LnL>
cole-h: I'm not sure how to solve that one
<cole-h>
OK, sounds good. Just wanted to make sure it wasn't forgotten.
<LnL>
or if it should be solved, the wrapper struct is just a bit weird
<LnL>
cabal-install on x86_64-darwin — Success
<LnL>
does something at least
<gchristensen>
nice
<gchristensen>
decide between you on the unglobbing and then let's merge & deploy :)
<LnL>
yeah I'll do that, just didn't want to deal with it while working
<cole-h>
Thanks :) Sorry for the nits
<gchristensen>
it is nice having the two of you working on it :)
<cole-h>
"working"
<cole-h>
If you call what I do working, I'm afraid to ask about what was happening before... :P
<gchristensen>
less :)
<gchristensen>
LnL: I'm inclined to deploy that PR tonight if you're up for it
<cole-h>
gchristensen: Just occurred to me, but one of the most important things to know (after how to deploy) would be: how to roll back if an issue occurs?
<cole-h>
Just go back to the previous deploy and restart it or something?
<cole-h>
"Rebuild" the button says
<gchristensen>
the deploy ALWAYS deploys the current "released" branch
<gchristensen>
so if you need to roll back, do a revert PR, merge, and deploy
* cole-h
cannae merge
<LnL>
yeah, there's no nixops rollback right?
<gchristensen>
there is actually
<gchristensen>
might be good to look in to that :)
<LnL>
no job for it that is
<gchristensen>
right
<LnL>
yeah I think you do need to enabble history for it, but perhaps there could be a rollback job or something that does that
<LnL>
imports done, don't worry about the nitpicks by the way :)
<LnL>
as for deploy that's up to you two, it's getting late here so I'm going to bed soon
<gchristensen>
cool
<gchristensen>
let's do it :)
<gchristensen>
as soon as CI passes
<gchristensen>
thank you LnL!
<gchristensen>
really awesome :D
<LnL>
let's hope it actually helps with the heartbeats
<cole-h>
<3 LnL
<{^_^}>
LnL's karma got increased to 41
<gchristensen>
yah
<gchristensen>
once this is out, it'll be good to move, say, log collector over
<gchristensen>
log collector is actually pretty high bandwidth
<gchristensen>
or maybe a lower bandwidth one
<gchristensen>
like comment filter
<gchristensen>
lots of options
<cole-h>
After looking at WOP's NixOS org team stuff, I noticed there's an ofborg team... Should I "Request to join"? 👀
<LnL>
I think all the groundwork is done now so moving the other things should be easy
<gchristensen>
<3 LnL
<{^_^}>
LnL's karma got increased to 42
<LnL>
heh there we go, perfect number
<cole-h>
:D Now you just have to ++ yourself whenever somebody ++'s you again
<cole-h>
I kinda want to say "Thanks!" to myself hehe
* cole-h
does it
<gchristensen>
cole-h: fwiw, don't merge darwin access without consulting LnL and/or me
<cole-h>
Absolutely
<cole-h>
I won't merge anything without consulting either/or, tbh 👀
<cole-h>
(Unless it's an urgent revert or something)
<gchristensen>
I'm watching for your deploy whenever you're ready :)
<cole-h>
Sir, yes, sir!
<gchristensen>
let me know before you do the "actually do it" button
<cole-h>
OK. Just did the dry activation button.
* cole-h
gets some water.
<gchristensen>
LnL: do you see this? cole-h doesn't even know the pain of starting a chef deploy / puppet apply and having to sit around for however long it takes :P
<cole-h>
:D
<cole-h>
Do you usually get dinner/lunch for those times? :P
<gchristensen>
oh no, you're chained to your computer watching everything
<cole-h>
God, scrolling past those warnings scared the bejeezus out of me
<gchristensen>
because even "dry run" isn't necessarily even safe, and then the not-dry-run takes crazy long
<cole-h>
I saw red text and almost flipped out
<LnL>
neither do I, just horrible ansible stuff but apparently that's already an improvement?
<gchristensen>
same problem though :P
<gchristensen>
deploying takes $forever, and is scary
<cole-h>
gchristensen: The real reason I wanted to be on the team: I can resolve my own comments! 🥳
<gchristensen>
never would I ever do a "dry run" and walk away whil eit ran
<gchristensen>
lol
<cole-h>
(I'm really mad that GH doesn't let you resolve your OWN freaking comments......)
<cole-h>
OK, ready to Just Do It™?
<LnL>
yeah, at my previous our integration tests would deploy a vm first which took about 40min