gchristensen changed the topic of #nixos-borg to: https://www.patreon.com/ofborg https://monitoring.nix.ci/dashboard/db/ofborg?refresh=10s&orgId=1&from=now-1h&to=now "I get to skip reviewing the PHP code and just wait until it is rewritten in something sane, like POSIX shell. || https://logs.nix.samueldr.com/nixos-borg
cole-h_ has joined #nixos-borg
cole-h has quit [Quit: Goodbye]
cole-h_ is now known as cole-h
cole-h has quit [Quit: Goodbye]
cole-h has joined #nixos-borg
cole-h has quit [Quit: Goodbye]
cole-h has joined #nixos-borg
cole-h has quit [Client Quit]
cole-h has joined #nixos-borg
hmpffff has joined #nixos-borg
hmpffff_ has joined #nixos-borg
hmpffff has quit [Ping timeout: 260 seconds]
cole-h has quit [Quit: Goodbye]
orivej has joined #nixos-borg
orivej has quit [Read error: Connection reset by peer]
orivej has joined #nixos-borg
orivej has quit [Quit: No Ping reply in 180 seconds.]
orivej has joined #nixos-borg
orivej_ has joined #nixos-borg
orivej has quit [Ping timeout: 265 seconds]
orivej_ has quit [Read error: Connection reset by peer]
orivej has joined #nixos-borg
orivej has quit [Ping timeout: 246 seconds]
orivej has joined #nixos-borg
hmpffff has joined #nixos-borg
hmpffff__ has joined #nixos-borg
hmpffff_ has quit [Ping timeout: 246 seconds]
hmpffff has quit [Ping timeout: 246 seconds]
orivej has quit [Ping timeout: 240 seconds]
hmpffff has joined #nixos-borg
hmpffff__ has quit [Ping timeout: 260 seconds]
hmpffff_ has joined #nixos-borg
orivej has joined #nixos-borg
hmpffff has quit [Ping timeout: 265 seconds]
hmpffff has joined #nixos-borg
orivej has quit [Quit: No Ping reply in 180 seconds.]
orivej has joined #nixos-borg
hmpffff_ has quit [Ping timeout: 265 seconds]
hmpffff_ has joined #nixos-borg
hmpffff has quit [Ping timeout: 260 seconds]
orivej has quit [Ping timeout: 258 seconds]
orivej has joined #nixos-borg
hmpffff_ has quit [Ping timeout: 258 seconds]
sphalerite has quit [Ping timeout: 244 seconds]
sphalerite has joined #nixos-borg
orivej has quit [Ping timeout: 256 seconds]
orivej has joined #nixos-borg
<hexa-> ofborg seems to be lagging behind
<MichaelRaskin> Well, there are zero evaluators and zero x86_64-linux builders running
<LnL> yeah
<MichaelRaskin> packet maintenance yanked the machines?
<LnL> yep, ~4h after they where just brought back :/
<gchristensen> ouch
<gchristensen> sorry :( will bring them back again
<gchristensen> LnL: can you trigger a deploy?
<LnL> oh should that work now?
<gchristensen> maybe there is a way we could add an optional button before the dry-activation deploy step which runs: nixops destroy --include packet-spot-eval-{1,2,3}nixops destroy --include packet-spot-eval-{1,2,3}
<gchristensen> I just ran ^ which is the only thing we need to do for redeploy to work right now
<LnL> sounds like a good idea, let's see
hmpffff has joined #nixos-borg
hmpffff has quit [Client Quit]
<LnL> gchristensen: looking good
<gchristensen> yay :D
<LnL> could have easily done that this morning if we had the destroy option
orivej has quit [Ping timeout: 256 seconds]
orivej has joined #nixos-borg
orivej has quit [Ping timeout: 258 seconds]
<LnL> everything recovered :D
<gchristensen> w00t
<LnL> not bad, I was a bit worried before about only having 3 evaluators when the hetzner host was decommissioned
<gchristensen> yeah..
<LnL> but since we've been able to catch of a bunch of times now seems like the current capacity is probably fine
<gchristensen> with a bit of work, these evaluators could be diversified across a few regions and instance classes
<gchristensen> to prevent big-bang "oops all gone" moments
<LnL> yeah that would be a good idea
orivej has joined #nixos-borg
cole-h has joined #nixos-borg
cole-h has quit [Client Quit]
cole-h has joined #nixos-borg
orivej has quit [Ping timeout: 258 seconds]
orivej has joined #nixos-borg
orivej has quit [Quit: No Ping reply in 180 seconds.]
orivej has joined #nixos-borg
cole-h has quit [Quit: Goodbye]
cole-h has joined #nixos-borg
orivej has quit [Quit: No Ping reply in 180 seconds.]
orivej has joined #nixos-borg
qyliss has quit [Quit: bye]
qyliss has joined #nixos-borg
orivej has quit [Quit: No Ping reply in 180 seconds.]
orivej has joined #nixos-borg
orivej has quit [Quit: No Ping reply in 180 seconds.]
orivej has joined #nixos-borg
orivej has quit [Ping timeout: 256 seconds]
orivej_ has joined #nixos-borg
orivej has joined #nixos-borg
orivej_ has quit [Ping timeout: 260 seconds]
orivej has quit [Ping timeout: 246 seconds]
orivej has joined #nixos-borg
orivej has quit [Read error: Connection reset by peer]
orivej has joined #nixos-borg
orivej has quit [Quit: No Ping reply in 180 seconds.]
orivej has joined #nixos-borg
<cole-h> gchristensen: When you got a sec, mind restarting `ofborg-github-comment-poster.service` -- seems dead.
<cole-h> `ofborg-stats.service` also dead
orivej has quit [Read error: Connection reset by peer]
orivej has joined #nixos-borg
<cole-h> gchristensen: `error: could not lock config file /var/lib/ofborg/.gitconfig: File exists` from comment-poster
<cole-h> But not an issue I guess
<cole-h> See logs now -- yay!
<gchristensen> w00t :)
orivej has quit [Ping timeout: 256 seconds]
orivej has joined #nixos-borg
orivej has quit [Ping timeout: 256 seconds]
orivej has joined #nixos-borg
cole-h_ has joined #nixos-borg
cole-h has quit [Quit: Goodbye]
cole-h has joined #nixos-borg
orivej has quit [Quit: No Ping reply in 180 seconds.]
orivej has joined #nixos-borg
cole-h has quit [Ping timeout: 256 seconds]
cole-h has joined #nixos-borg
cole-h has quit [Client Quit]
cole-h has joined #nixos-borg
orivej has quit [Ping timeout: 260 seconds]
orivej_ has joined #nixos-borg
cole-h has quit [Quit: Goodbye]
cole-h has joined #nixos-borg
cole-h has quit [Client Quit]
cole-h has joined #nixos-borg
orivej_ has quit [Ping timeout: 264 seconds]
orivej has joined #nixos-borg
orivej has quit [Ping timeout: 258 seconds]
orivej has joined #nixos-borg
orivej has quit [Ping timeout: 258 seconds]
cole-h has quit [Quit: Goodbye]