aminechikhaoui has quit [Ping timeout: 256 seconds]
aminechikhaoui1 is now known as aminechikhaoui
fabian_a has joined #nixos-infra
cole-h has joined #nixos-infra
tilpner_ has joined #nixos-infra
tilpner has quit [Ping timeout: 264 seconds]
tilpner_ is now known as tilpner
lukegb has quit [Quit: ~~lukegb out~~]
lukegb has joined #nixos-infra
<roberth>
the darwin builders do seem to be stuck. They have plenty of backlog from trunk, staging and staging-20.09, but only one of the machines is active
<roberth>
I don't think that can be caused by shares, can it?
cole-h has quit [Ping timeout: 256 seconds]
tilpner_ has joined #nixos-infra
tilpner has quit [Ping timeout: 246 seconds]
tilpner_ is now known as tilpner
<domenkozar[m]>
ikwildrpepper: ^^
<domenkozar[m]>
btw, shouldn't shares be per platform?
<ikwildrpepper>
not sure, would have to check the hydra source code, maybe niksnut knows by heart
<gchristensen>
I don't think shares are by platform, meaning it could starve a platform
<ikwildrpepper>
looks like builds are only happening on one mac, perhaps reboot went wrong this weekend
<ikwildrpepper>
mac minis coming up
<ikwildrpepper>
domenkozar[m], roberth: all macs are building now again
<domenkozar[m]>
go team fruit
<domenkozar[m]>
thanks ikwildrpepper :)
<domenkozar[m]>
one of the hard things about hydra is that it's not clear if things are stuck or just not scheduled yet
<ikwildrpepper>
the logs were clear
<domenkozar[m]>
(which would allow for monitoring to trigger alarms)
<roberth>
thanks!
<domenkozar[m]>
ikwildrpepper: logs are clear, but I don't want to waste your time each time I think macos are stuck :D
<gchristensen>
https://hydra.nixos.org/queue-runner-status has a lot of really good information in it, but it isn't visualized anywhere that is easy to understand what it is trying to say
<ikwildrpepper>
oh sure, I wasn't suggesting that the current situation is perfect :D can't remember if we have some log aggregation setup, that we could use to alert
<gchristensen>
we don't yet, we could setup loki to the same machine eris is on -- the reason I haven't yet is logs have "scarier" information to make public
<ikwildrpepper>
oh right, I think in this case the info is in the queue status log
<ikwildrpepper>
log -> json
<ikwildrpepper>
as the macs were 'disabledUntil'
<ikwildrpepper>
sorry, a bit distracted, as i am working on my favorite thing (bookkeeping / admin)
<ikwildrpepper>
:puke:
<domenkozar[m]>
heh I just did accounting in last two days as well
<gchristensen>
I have never expensed a single ofborg expense because I just can't make myself do it, lol
<gchristensen>
it is hard to make the $50 worth the pain
<domenkozar[m]>
I only had to make two bank exports of transactions
<domenkozar[m]>
took me 2.5 months to get to it
<gchristensen>
that is familiar
<domenkozar[m]>
:D
<domenkozar[m]>
alright so we just need to wire up things to know when macs are stuck
<ikwildrpepper>
gchristensen: haha, same here. I almost never declare things at company as I dislike it too much :D (don't tell Lizi)
tilpner has quit [Remote host closed the connection]