<samueldr>
were they *that* failure or another one?
<srhb>
Hard to say.
<samueldr>
the fact hydra clobbers builds when restarting is annoying here
<srhb>
Yeah.
<srhb>
Ask vcunat? :)
JosW has quit [Ping timeout: 252 seconds]
<samueldr>
I was doing it, I was asking more rethorically than to get an answer here :/
<srhb>
Ah. :P
drakonis has joined #nixos-dev
<samueldr>
there *are* other failures than failing to connect to the VMs, so it cannot be assumed it was the fault here
<samueldr>
transient failures
<samueldr>
hmm, it sounds like a full-time-job thing, but it would help greatly, imho, if the builds were immutable
<gchristensen>
like an attempt-based mechanism, which ofborg does?
<samueldr>
well, immutable is probably not the right term, since it needs to go through steps, but a build should be *one* build, not an ID for a job for an eval
<samueldr>
yeah
<gchristensen>
yeah
<gchristensen>
hydra already tracks attempts, but logs get stored at drvpath.log which is not supportive of that
<samueldr>
there are other weird things coming from the fact that builds "act as mutable"
<samueldr>
like when a build fails and its dependents are put in a failing state, and that build is restarted and fine
<gchristensen>
complicated stuff
<samueldr>
yeah, I presume
<gchristensen>
might be grant money to improve it
<roberth>
has anyone tried to run a debian systemd as pid 1 using vmTools?
drakonis_ has joined #nixos-dev
drakonis has quit [Ping timeout: 252 seconds]
drakonis_ has quit [Read error: Connection reset by peer]
<Guest28061>
no idea, don't think I've ever ran into that
<gchristensen>
who is mysterious Guest28061?
Guest28061 is now known as LnL
<LnL>
hrm
<gchristensen>
ah!
<samueldr>
okay, so running both tests locally work, so my guess is that there might still be something going on when the machines are overloaded, but then it's hard to pinpoint
<samueldr>
the grub failure seems to be a completely different failure mode (grub kinda froze?)
<samueldr>
the networkd.loopback test looks like a honest timeout
<samueldr>
and IIRC installer tests are "often bad" since they incurr many boots
<samueldr>
I guess let's backport the revert now
<gchristensen>
samueldr++
<{^_^}>
samueldr's karma got increased to 52
<timokau[m]>
I guess it makes sense that small is eval'd more often doesn't it?
<samueldr>
sure!
<samueldr>
I mostly didn't think about it
<timokau[m]>
I've pretty much switched to small and noticed I barely have to do any rebuilds anyways
<timokau[m]>
Even though I have some big "leaf" software installed that takes long to build, i.e. sage and graph-tool
<gchristensen>
nice. I have a nag tool annoy me when my system needs updating. not sure I could handle the -small one :P
<samueldr>
gchristensen: unlucky you, you might be nagged more often in the near future
<samueldr>
(assuming no other major failures happen)
<gchristensen>
I deserve that.
<timokau[m]>
Its not really intentional. Mostly I notice an issue with a package I need *right now*, fix it, do one rebuild from master and then have to wait for the channel
<timokau[m]>
So at least up until now regular unstable would take way too long
<timokau[m]>
It would be neat if nix-channel could apply a patch file to any revision it fetches :D
worldofpeace has quit [Quit: worldofpeace]
worldofpeace has joined #nixos-dev
<ekleog>
gchristensen: does your nag tool include a diff of the system? I definitely need one of those sending me an email with the changes I should enable, but nix-diff still has a few issues that'd make its output illegible I think :/
<gchristensen>
no, it doesn't update anything, just tells me my version is not matching the channel
<ekleog>
oh ok :( I was planning to download the most recent version of the nix-channel, nixos-rebuild on this channel build and nix-diff with my /run/current-system
<gchristensen>
that'd be cool
<timokau[m]>
What is nix-diff?
<ekleog>
s/nixos-rebuild on this chnnel build/nixos-rebuild build on this channel/ (sorry, mosh over train connection, not really able to re-read what I type before sending)
<{^_^}>
Gabriel439/nix-diff#7 (by Ekleog, 40 weeks ago, merged): Output all differences between the compared derivations
<{^_^}>
Gabriel439/nix-diff#8 (by Ekleog, 40 weeks ago, open): Shorten trees consisting only of “These two derivations have already been compared” as the leafs.
<timokau[m]>
Looks very neat
<timokau[m]>
But why would you want that at every update?
<ekleog>
basically, to know what would be upgraded if there's a channel update
<ekleog>
this way I can ignore bumping useless stuff, while not missing important updates
<ekleog>
basically like debian's update nagging system
* gchristensen
plugs in the power and starts poking at this idea
<ekleog>
:3
<timokau[m]>
Is that for a server?
<timokau[m]>
Because for a home machine, why not just try it?
* ekleog
hopes to have nerd-sniped gchristensen into solving nix-diff#8
<gchristensen>
lol, no
<ekleog>
it's for servers mostly indeed
<timokau[m]>
Okay for servers it makes a lot of sense
<ekleog>
though it'd be useful for laptops too I think, because then I don't have to even attempt an update often, it'd be automatically built, and I'd just have to switch to the correct nixpkgs revision and nixos-rebuild switch to use the already-built-in-the-background system
<ekleog>
(when poked by a mail that looks interesting)
<gchristensen>
ekleog: eh... it doesn't really work because the update checker runs as a restricted user, but my nixos config is owned by me...
<clever>
ekleog: i have my hydra setup to build my entire nixops deployment against unstable
<clever>
ekleog: so it both acts as an advanced warning as to when things are broken, and a cache
<ekleog>
gchristensen: same “nixos config owned by me”, but I'd have just run the update checker cron as myself (or, if I'm really motivated enough, another user that'd be in the right group to be able to only read the config, but I'm not sure what's being really protected against by doing that)
<gchristensen>
or root
<ekleog>
clever: that's nice :) though it's still missing the nix-diff (and I must say I don't have a hydra yet (a))
<timokau[m]>
ekleog But how would you know which change is security relevant for example?
orivej has joined #nixos-dev
<ekleog>
timokau[m]: first step would be manually reviewing, and then sometime later hopefully we'll gain a way to flag security updates in nixpkgs
<ekleog>
already knowing “ok this change bumps packages X, Y and changed configuration Z” is much better than “well the built-system-hash changed”
<timokau[m]>
Thats the part I understand for servers, but for end-user machines it seems like too much overhead compared to a (systemd) cron job that just does the update
<timokau[m]>
But then again, I'm not you :)
<ekleog>
well, let's say I tend to be cautious and not update random stuff at random times :)
<ekleog>
and given I'll be handling my servers this way, may as well just use it for my laptop too
<timokau[m]>
For fear of breakage or security issues?
<ekleog>
mostly breakage
<timokau[m]>
Okay, not worth the effort for me but I do see why its a good idea
<ekleog>
(for security issues I might as well just always upgrade everything, but I have for instance one server that's completely headless with no way of accessing to a physical shell without jumping through hoops and no otherconsole than the ssh, so… ^^'))
<timokau[m]>
I was thinking more the "somebody could have made a malicious change" security aspect
* ekleog
still should setup some raspberry pi just to get some serial console just in case, but them being armv7 without hydra is still a PITA, despite dezgeg's cache
<timokau[m]>
But you'd go insane if you'd try to review everything for a personal machine
<ekleog>
oh well I'm not part of SLNOS so I can't reasonably handle that aspect yet :p
<ekleog>
heard they do it
<ekleog>
some day we'll get expression integrity, but for now the work is blocked on git-wotr being completed I think
<timokau[m]>
Not sure if I want to be a part of that, but good to know that there are some eyes on that
<timokau[m]>
My only "server" is a raspberry pi that still runs on arch since I am pardox-of-choice'd out of upgrading to aarch64
<ekleog>
too many projects at too many places
<ekleog>
heh, I don't really plan to upgrade to aarch64
<ekleog>
will likely wait for cross-compilation of nixpkgs becoming Really Good, and just setting up a cross-compiling hydra that lies and says it built the stuff natively
<ekleog>
and then start using my rpis again
* ekleog
won't have enough time for doing this before a few years anyway, with all the stuff on the fire…
<timokau[m]>
Doesn't really work without a binary cache though. And completely relying on a private hydra seems fishy, even if it is operated by a well trusted and respected community member
<timokau[m]>
ekleog: Thats optimistic, I think I'll just throw a little bit of money at the issue to make it go away
<ekleog>
heh, that's ruining all the fun
* ekleog
doesn't really need an aarch64 server, though, with the already setup nixos x64 server, so isn't in the same position as you
<timokau[m]>
Already more fun than I have time for :D
<timokau[m]>
By the way does anyone know why hydra doesn't *finish* to build an eval before it starts a new one? Seems weird to me that tests already passed twice but the unstable channel is still 4 days old
<samueldr>
tests have finished successfully, but that's not enough, it needs to finish building (successfully or not) everything for the channel to advance
<samueldr>
as to why it doesn't wait? does need to? jobs are already scheduled, presumably in order
<timokau[m]>
Yes but why doesn't it do that before starting the next one? Just "because its not implemented that way"?
<samueldr>
does it need to*
<timokau[m]>
It wouldn't idle, it would just put more resources into finishing the first eval. That way the first eval would be done earlier. The two evals should still take about the same time together and the second one will be more up-to-date.
<samueldr>
(I'm really asking, hadn't thought much about it beforehand)
<samueldr>
"just put more resources into finishing the first eval" -> I'm not sure I follow, doesn't it queue jobs and builds in order?
<samueldr>
doesn't the eval happen on another machine than the build machines anyway?
<ekleog>
I think timokau[m]'s point is that it shouldn't queue a new eval before having finished building jobs for a previous eval or before having failed a test of a previous eval
<timokau[m]>
exactly
<timokau[m]>
Apparently its not run in order, since the tests for the new eval are already run before the first eval has finished building completely
<samueldr>
hmmm, right
<samueldr>
what I had in mind, but I think anyways it's moot, is that for many evals, many builds are automatically resolved as being built, so it "doesn't matter"
<samueldr>
but it clearly matters
<gchristensen>
interesting
<gchristensen>
someone would probably need to bust out little's law to determine if that would hurt or help the throughput of channel updates
<timokau[m]>
How could it hurt it?
<samueldr>
some quick thoughts I think it could reduce the prebuilt binaries coverage between long rebuilds, but might help speed up updates for multiple successive large renbuilds?
<samueldr>
it might depend on what the goal is
<samueldr>
coverage or updates
<timokau[m]>
That is what the small channel is for though
<samueldr>
after all, hydra isn't only for NixOS/Nixpkgs
* gchristensen
still needs to work on his TAGS thing :|
<samueldr>
gchristensen: work on it every day that ends in TAG (in german)
<timokau[m]>
True, sounds like it most likely just is "because it was implemented that way"
<gchristensen>
page 9 shows the original, epic logo
<srhb>
timokau[m]: It's funny that Mittwoch even happened. It seems like a censure.
<samueldr>
hmm, that interface makes more sense than the bootstrapped one
<timokau[m]>
So it *was* hacked together by eelco, at least in part
<gchristensen>
it was built by eelco
<timokau[m]>
srhb: I don't get that
<gchristensen>
I think hydra doesn't get a fraction of the credit it should :/
<timokau[m]>
gchristensen: so my torvalds analogy holds
<srhb>
timokau[m]: I mean, most other languages that stem for germanic have kept the connotation to Odin for wednesday (wotans day...) but German lost it.
<timokau[m]>
Well just like git it gets a lot of hate but in the end everyone uses it and its an essential tool
<samueldr>
if I ever sound critical, it's not that it's bad, it's that it's complex!
<gchristensen>
ah, here it is! it was funded by NIRICT LaQuSo BuildFarm
<timokau[m]>
srhb: Didn't know that. Odin never really mattered in germany though
<srhb>
timokau[m]: That's not true :P
<timokau[m]>
Its not?
<srhb>
No, old north germanic religions containing some sort of Odin figure predate the norse Odin image.
<timokau[m]>
gchristensen: That is the "project" though, not a company right?
* gchristensen
shrugs
<timokau[m]>
srhb: TIL, I'll complain to my high school history teacher right away. They never teach the interesting stuff
<srhb>
xD
<gchristensen>
btw, it is likely that funding could be procured for improvements to hydra
<timokau[m]>
(I still wish we could fix the email issue, though thats not really hydra-the-project but hydra-the-deployment)
<samueldr>
I started also some other misc. work in a branch to fix "internet-OCD" issues, but seeing a lack of reviews on PRs was a bit discouraging, can't invest time if I'm working in wrong ways :/
<timokau[m]>
Who would I have to ask about hydra permissions? I have a job that needs restarting again and don't want to have to ask every time.