<ekleog>
<zimbatm> ekleog: do you know if signed commits change the commit hash? <-- so there are multiple ways to sign commits, 3 have been discussed on the RFC
<ekleog>
Way 1 is git's `git commit -S` / `git merge -S`. This one changes the commit hash, you can't sign a posteriori
<ekleog>
Way 2 and 3 are both signing metadata relating to the commit and storing it in `git notes`. These ones don't change the commit hash, but rather push a commit on an unrelated branch `refs/signatures`
<ekleog>
Way 2 signs basically the commit SHA-1 + metadata
<ekleog>
Way 3 signs the patch-id, commit on which it's supposed to apply and [I don't really remember everything, it seems way too complex for no benefit to me]
* ekleog
is in favor of way 2, which allows multiple persons to sign the same commit without any workflow change if we want to switch to it for sensitive packages later
Synthetica has quit [Quit: Connection closed for inactivity]
orivej has quit [Ping timeout: 252 seconds]
<gchristensen>
ekleog: the problem is git-notes suck, aren't nice for rebasing, pushing, pulling, merging, conflict-resolution, etc.
<gchristensen>
there is a reason github dropped support for displaying git notes years ago
<ekleog>
gchristensen: well, git's internal signatures aren't nice for any of those things either, and I think that's actually a good thing :)
<ekleog>
like, I want to sign a commit in a known context, not sign a commit that'll maybe be rebased in a situation where it introduces a vulnerability
<ekleog>
the point of git notes being only to have a way to associate (multiple) signature(s) to a commit after the commit is pushed
<ekleog>
(tbh, I don't really understand why git didn't make its internal signatures based on git-notes in the first place, it's much nicer for ability to rebase, because you can rebase and then ask people to re-sign their commits, while with internal signatures you'd need to be synchronized with the others to in turn re-sign and re-push the correct commits)
* ekleog
discovered about git notes only recently but fell in love with the idea, even if not really with the tool
<gchristensen>
git notes are a disaster because of how detached they are
<gchristensen>
git commit signatures work properly because they are part of the commit, not a forgotten appendage
<gchristensen>
I used git notes to store security annotation data for commits, it was terrible to deal with conflicts, and I didn't even have to deal with more than 1 person committing to them
<gchristensen>
everybody falls in love with git-notes because they're so cool, but man
<ekleog>
the cat_uniq_sort merge strategy didn't do the trick?
<ekleog>
Here it'd be storing an ever-growing set of signatures in the notes for each commit, so I'd guess cat_uniq_sort would take care of all conflict
<ekleog>
s
<ekleog>
the fear I have with regular git commit signatures is that it's not possible to sign an old commit we forgot to sign, which isn't really a problem with few people committing… but in nixpkgs I guess we should expect this kind of mistakes to happen
<gchristensen>
well there is no need to sign an old commit, every commit inherently signs all of history
<gchristensen>
maybe it would work fine, but since it is a weird part of git nobody really knows how to use it and the tools work less nicely, and external tools which know about git worked barely at all with it
<gchristensen>
I'm not sure mashing up two user-hostile tools in to one feature is a good idea
<gchristensen>
anywho, it is long past my bedtime, see you
lopsided98 has quit [Quit: Disconnected]
lopsided98 has joined #nixos-dev
<ekleog>
see you :)
<ekleog>
gchristensen: (hl so you see when you come back) the problem with “every commit inherently signs all of history” is “A pushes an unsigned backdoor, B pulls, signs a merge, pushes”: here, B has taken responsibility for A's backdoor
<ekleog>
it's possible to say B must check all commits on fetch, but that requires tooling that's not yet present and adds a failure mode, or requires a *lot* of human brainpower
<ekleog>
otoh, if we consider each commit to be signed independently (ie. not signing the previous state of history), then we can just say “ok A's commit has never been signed, there's something weird in the history”, hydra would be automatically blocked and people would look into the issue
<ekleog>
but yeah, the problem is mostly about external tools that know about git, they wouldn't know at all about said signatures… though I can't see a practical example of issues happening due to it right now :)
orivej has joined #nixos-dev
srk has quit [Ping timeout: 246 seconds]
srk has joined #nixos-dev
orivej has quit [Ping timeout: 268 seconds]
FRidh has joined #nixos-dev
jtojnar has quit [Remote host closed the connection]
{^_^} has quit [*.net *.split]
globin has quit [*.net *.split]
phreedom_ has quit [*.net *.split]
{^_^} has joined #nixos-dev
globin has joined #nixos-dev
phreedom has joined #nixos-dev
Sigyn has quit [Quit: People always have such a hard time believing that robots could do bad things.]
Sigyn has joined #nixos-dev
{^_^} has quit [*.net *.split]
globin has quit [*.net *.split]
phreedom has quit [*.net *.split]
FRidh has quit [Quit: Konversation terminated!]
<andi->
what can we do about the issue with the build timeouts of 10h due to local nix-daemon configuration? 18.03 has been blocked for about 12 days or so because the chromium builds have been terminated after 10h. There is a more serious firefox update arriving today and it would be nic to release within a few days at most.
phreedom has joined #nixos-dev
<srhb>
andi-: What's this about local nix-daemon configuration? A situation where meta.timeout is not respected?
<andi->
If I could hit the restart button I could at least hit it until we get lucky with a faster/idle build machine :/
<srhb>
andi-: Our options are certainly limited, because removing those large builds from the tested set is not on the table (has been discussed before) -- so I guess we're down to 1) fixing the override of the meta.timeout and friends values, disabling the nix-daemon on all builders, creating a new feature flag for non-daemonized stores, or restarting ad hoc.
<srhb>
Oh, I gave up on numbering after 1... *needs more coffee)
<clever>
srhb: nix-daemon doesnt have to be disabled, you just have to ssh into root and run `nix-store --serve` as root
<clever>
then it will ignore the daemon
<clever>
but then hydra has root on all build slaves...
<srhb>
That seems less likely to be a thing we can/want to do in the short term. :-)
<srhb>
I think feature flag is the most palatable quickfix.
<clever>
the protocols between hydra<->nix-store --serve<->nix-daemon have to be improved to allow forwarding this all the way
<clever>
i think the problem is the serve<->daemon layer
<ekleog>
wouldn't a quickfix just to increase builders' timeout setting?
<ekleog>
+be
<clever>
yeah
<srhb>
Definitely.
<clever>
in the case of iohk, the problem is the reverse, tests that just deadlock and stay running for 48 hours, so we needed to decrease the timeout
<ekleog>
that can be done with meta.timeout, though, can't it?
<srhb>
ekleog: The issue is that it's ignored on daemonized builder nodes.
<clever>
ekleog: meta.timeout is ignored when nix-daemon is at play
<ekleog>
oh. I thought nix-daemon took the min between the two values
<ekleog>
which would make sense imo
<srhb>
If it does, the problem is the same..
<srhb>
(At least unless we do bump the nix.conf setting)
<ekleog>
well, bump daemon's timeout to a lot, down iohk's timeout to not a lot
<clever>
ekleog: i believe the protocol between `nix-store --serve` and `nix-daemon` doesnt support pushing over the timeout value
<{^_^}>
nix#50 (by rbvermaa, 6 years ago, closed): Build timeout not passed to nix daemon
<ekleog>
oh wait
<ekleog>
it's good actually, didn't see it was closed with a commit
<clever>
that commit is in the perl code for build remotes
<clever>
all perl is gone
<ekleog>
I can confirm that for a local daemon, the --timeout option works, so the daemon protocol can understand it
<ekleog>
now, between hydra and nix-store --serve, I have no easy way of testing :/
<ekleog>
well, in the meantime, maybe it'd make sense to bump the timeout to 15 hours or similar, so that builds are unblocked? having overloaded hydra is still better than having completely blocked hydra
<clever>
sure
<srhb>
It's more like 48 hours :-)
<srhb>
(That's what we determined on the builders that _do_ respect meta.timeout)
<andi->
I saw a build that succeeded in ~3.5h on one of the epyc machines
<srhb>
Yeah, it can go really fast in good situations.
<andi->
How do we force good situations? Break all other channels? ;-)
<srhb>
iirc it's mostly about load at any given time.
<srhb>
We don't have that sort of granularity.
<srhb>
You'd want certain builds to "balloon" taking up more slots than usual.
<ekleog>
niksnut: (later solutions would be fixing https://github.com/NixOS/hydra/issues/591 to make hydra always respect meta.timeout, as I understand it, cf. the discussion preceding these hl)
<{^_^}>
hydra#591 (by cleverca22, 4 weeks ago, open): meta.timeout does not always work
<ekleog>
andi-: hmm… what's there to review? it's all just version and hash bumps, so if it builds it should be ok?
<andi->
ekleog: just making sure I didn't miss anything :-)
<andi->
I build them all and tested in the mean time..
<ekleog>
well, then I can confirm these look very much the same as your previous security fixes from https://github.com/NixOS/nixpkgs/pull/47277 ; so if those were correct the new ones should be too :)
<andi->
Ohhh, srhb I think I figured what I am permitted to restart (and probably you as well). If you go to a jobset (e.g. https://hydra.nixos.org/jobset/nixos/release-18.03) you can click actions -> restart all failed/aborted
<andi->
It feels like a shotgun mode where I'd have expected to just be able to retry individual builds
roberth has joined #nixos-dev
<roberth>
domenkozar and I are working on a brand new Continuous Integration service for Nix users. Check out https://hercules-ci.com!
<andi->
The build machines have the big-parallel feature but still it takes them a while to build it.
orivej has joined #nixos-dev
orivej has quit [Ping timeout: 252 seconds]
<andi->
Looking at the output I get the impression it is building on a single core..
lassulus has quit [Ping timeout: 268 seconds]
lassulus has joined #nixos-dev
Synthetica has joined #nixos-dev
LnL has quit [Ping timeout: 244 seconds]
orivej has joined #nixos-dev
LnL has joined #nixos-dev
<ekleog>
domenkozar: you're welcome :) (and sorry for not finishing with a “thank you!”, I've been interrupted IRL :°
<ekleog>
)
<gchristensen>
does anyone have feelings about adding "MLX5_CORE_EN m" or "MLX5_CORE_EN y" to our default kernel config?
<thoughtpolice>
gchristensen: That's a fancy network card, is what I feel.
<thoughtpolice>
Jokes aside, adding kernel modules to the default closure mostly gets annoying when you want to override things, like slimmer kernel builds, in my experience. (Really this is also due to the fact our kernel config driver tool is a huge hack, too). Maybe this has changed. I don't think that's worth holding off though. Mellanox cards aren't exactly unheard of.
<samueldr>
I'll have to have eyes on #47697, I think I had a couple peeps checking, but no one left a review or approval :/
<thoughtpolice>
Plus it's obviously quite grating to plug in some hardware and realise "I need to rebuild my kernel" suddenly, on its own.
<samueldr>
thoughtpolice: I think this case is pretty self-serving where the nixos aarch64 community builder uses such a card (unless it's yet another one)
<thoughtpolice>
I'm guessing the only reason we don't have it already, honestly, is because a lot of people don't just have that kind of gear sitting around, but they're pretty popular cards for high-end network deployments. Most people don't get them on a cloud.
<samueldr>
so if I were to give a plus or a minus, it would be a +1
<thoughtpolice>
Right, exactly.
<thoughtpolice>
I think of it as, "If someone had done this sooner, they would have definitely added it", so in that sense it seems pretty good to just go ahead and add it and be happy.
<samueldr>
I think the other detail to check would be: why does upstream (kernel) keeps it =n ? if no real reason or nothing bad, just do it :)
<ekleog>
now… well, =m can make the job in the meantime :°
<ekleog>
oh actually teto finished it 4 hours ago, or so it seems… nice timing :°
<Dezgeg>
why should this particular module be any different from the other bajillon modules we have on by default?
<ekleog>
my opinion is the other modules should also be auto-detected this way, but it'd need benchmarking to figure out whether by compiling only the required modules we can drop down to an acceptable build time (like gentoo's 2-3 min max, not like nixos' current 20-30 minutes)… if it's not the case, staying with everything-on at least makes the cache work, but… :/
* ekleog
wonder whether it'd be possible to have =m-compiled modules in separate outputs and to fetch them on-demand
<sphalerite>
ekleog: not sure how that would work with module dependencies. It also wouldn't help with build time
<Dezgeg>
how do you autodetect say, some USB hardware in advance?
<samueldr>
the usb installer image will be some hell to build :)
<ekleog>
sphalerite: it would make the cache able to build all modules and people able to selectively pick which ones they use, which would be a huge improvement over people currently having to rebuild the whole kernel+modules when they want any configuration not pre-built by hydra (eg. MLX5_CORE_EN, currently) -- for dependencies, can't we have cross-output dependencies?
<Dezgeg>
if people find that they need MLX5_CORE_EN for their hardware they should submit a patch adding that
<ekleog>
Dezgeg: the default config could still default-load undetectable external hardware
<Dezgeg>
that's what the majority of modules are
<ekleog>
well, MLX5_CORE_EN is a good example of one that isn't
<Dezgeg>
why not? isn't it a PCI device?
<ekleog>
because in practice people don't expect to hotplug pci devices without any issue
<ekleog>
usb devices are a good point, pci devices are not
<sphalerite>
ekleog: yes they do nowadays
<sphalerite>
thunderbolt :)
<ekleog>
nix defaults to turning on thunderbolt hotplugging? :/
<ekleog>
nixos*
<sphalerite>
idk
* ekleog
sees that as a huge security vulnerability
<Dezgeg>
besides, purely from the kernel config point of view you're not going to know if an option enables a PCI device or USB device
<{^_^}>
"Unless enabled, encryption keys can be easily recovered by an attacker with physical\naccess to any machine with PCMCIA, ExpressCard, ThunderBolt or FireWire port.\nMore information is available at <...
<ekleog>
Dezgeg: well for sure it'd require vetting
<ekleog>
sigh :)
<sphalerite>
but the option description suggests it only disables firewire
<sphalerite>
so the problem remains
<Dezgeg>
there is no way for anyone to vet 10000 options
<Synthetica>
Is there a way to enable hotplugging, but only after a root password check?
<ekleog>
people already have vetted 10000 options
<Dezgeg>
really? where?
<ekleog>
I *hope* the enabled options have been checked at least once before being activated
<Dezgeg>
of course not, the build script just answers 'm' where possible
<ekleog>
be it by a nixpkgs committer, an ubuntu member or whomever
<ekleog>
waaaait… really?
<ekleog>
now, in practice the idea of having the cache build all modules as separately-fetchable outputs|derivations|whatever is much more reasonable than the one of parameterizing the kernel automatically for everyone
<andi->
That one really needs some work :/ It isn't fun trying to add builtins etc.. (not just modules)
<andi->
ekleog: Dependency resolution between modules would have to be mirrored to nix then
<ekleog>
andi-: in the building derivation, use modinfo to get the dependencies, and add links to the dependencies in a nix-whatever/ directory
<ekleog>
(if multiple outputs can indeed have dependencies between them)
<Dezgeg>
you need to know the outputs beforehand
<ekleog>
it'd “just” require having a list of modules in .nix
<ekleog>
which can be automated, I think, but I'm not sure how
<sphalerite>
mumble mumble we now have recursive nix
<andi->
it probably can be automated, pushes a bit of the work towards updating the build expression
<sphalerite>
(although that'll never fly in nixpkgs also it's not in a release)
<sphalerite>
(not sure if it's even on master)
<andi->
optional dependencies and stuff probably also exist in the kernel :/
<andi->
It will end up being cargo2nix, npm2nix,... for the kernel..
<sphalerite>
Kconfig2nix :)
obadz has joined #nixos-dev
<ekleog>
andi-: modinfo aesni-intel -> can't see anything that makes me think there could be optional dependencies
<ekleog>
(random module picked just to check modinfo output
<ekleog>
)
<Dezgeg>
try something like ext4 which will optionally depend on crc32 if certain checksums are enabled
<ekleog>
there's a hard depend on crc16 and no mention of crc32 here
<andi->
the "cheap" way would be to depend on all of them in that case.
<ekleog>
oh nvm found it, it's not the same line
<ekleog>
so yeah there's also “softdep” in addition to “depends”
<ekleog>
and yeah, depending on all of them sounds like it'd make sense
<andi->
it would be a few megabytes extra but probably not tooo bad
<andi->
compared to having ALL modules :D
<samueldr>
would it be realistic to implement such a feature, but still default to an all-encompassing default kernel (so not changing the defaults), but it would be built using that modular kernel?
<samueldr>
so users that want to pick and match will have the independent modules already available, and no change for the defaults
<ekleog>
samueldr: I can't see why it wouldn't be, insofar as implementing such a feature would not in itself be unrealistic
<sphalerite>
wait so what's the goal here? :p
<ekleog>
being able to add/remove modules without rebuilding for half an hour :D
<ekleog>
(also, hopefully tweak the configuration of builtins, but I guess that won't be possible because changing an option for the core kernel could necessitate rebuilding some modules)
<sphalerite>
I don't think that makes much sense from a practical perspective
<ekleog>
well, that's how other distros do it : a main kernel package, and additional module packages for more rarely-used stuff that doesn't deserve inclusion in the main kernel package (for instance the iwlwifi network cards, always fun when you forget to install it… I'm not suggesting we go up to there)
<sphalerite>
if you want to disable a module because you're concerned about the implications of it being autoloaded on a hotplug, you can blacklist it. If you want to disable a module to save space (and still want to avoid rebuilds), you can make a custom derivation that just copies bits and pieces from the stock kernel
<sphalerite>
and if you want to enable something that's missing from the stock kernel, you'll be building it yourself anyway
<ekleog>
unless the stock kernel starts building *everything* and not pushing everything to the end-user
<ekleog>
(also, copying bits and pieces… avoiding that is exactly the reason why multiple outputs were made, isn't it?)
<sphalerite>
well it should already be including everything it can as modules
<ekleog>
like, the kernel modules are 58M currently… from memory (old one) my gentoo kernel was totalling only a few megs (like 5 or 10) when I used gentoo… that's a quite big difference, imo
<ekleog>
to me the point of multiple derivations is exactly to not have to make a derivation that copies bits and pieces, actually :)
<sphalerite>
shouldn't you also count the sources towards the size when talking about gentoo? :)
<ekleog>
I'm not trying to compare NixOS to gentoo, just to mention that we could shave a *lot* on modules if we wanted to
<ekleog>
(also, people have been using gentoo to build minimal-sized images that don't include the sources, in a way similar to not-os)
<sphalerite>
yes, the latter is the perfect use case for a derivation that copies bits and pieces. None of the fuss with translating the whole kernel dependency structure into nix
<ekleog>
parenthesis are not the main point
<ekleog>
(otherwise I'd be comparing with not-os, which does handle this quite well)
<sphalerite>
you can just use the existing information in an already built kernel tree to get the closure of the modules you care about and copy that into a new derivation which includes only the modules you're interested in — without building the kernel from source, and keeping the full 60MB things only on the machine building the image
<sphalerite>
I'm not convinced that the potential size savings are worth the engineering effort
<sphalerite>
that's my point
<ekleog>
well, what I'm saying is that the kernel-building derivation could use the information from the built kernel tree to copy each module into an appropriate output
<Dezgeg>
how do you know a priori what outputs are needed?
<ekleog>
the only translation that's needed to nix is the list of modules
<sphalerite>
no it can't, because that requires A) knowing which modules are going to be there at evaluation time
<sphalerite>
or B) IFD
<ekleog>
that's not the whole dependency tree
<sphalerite>
if nix doesn't know about the dependencies, loading a module with dependencies will be Fun
<ekleog>
it doesn't need to know dependencies at eval-time, only at build-time
<ekleog>
and at build-time it can get them with modinfo
<sphalerite>
yes it does, ebcause the list of outputs needs to be known before the build happens
<ekleog>
again, the list of outputs is not the dependencies
<sphalerite>
so what happens with the dependencies?
<ekleog>
I'll just walk through the idea, because I feel like I can't explain otherwise
<ekleog>
1. The kernel.nix has a list of ["modulea", "moduleb"]
<ekleog>
2. this list is set as the list of outputs (plus a “kernel” output and whatever)
<ekleog>
3. when building the kernel, make a standard build-from-source-to-all-modules
<ekleog>
4. then, install the kernel to its output, and each module to its output
<Dezgeg>
how do you know what is ["modulea", "moduleb"] going to be?
<ekleog>
5. and here is the trick, in each module output, add a link to the module outputs of modules it depends on, in nix-build-support or whatever
<ekleog>
this way the dependency structure is automatically generated
<Dezgeg>
I get the idea, but you cannot solve steps 2.-5. without solving step 1.
<sphalerite>
but those module outputs don't exist because they weren't known at instantiation time!
<ekleog>
Dezgeg: please tell me, what part of “yes you need to auto-generate the list of modules to list the outputs, but you don't need the whole dependency structure of the kernel” isn't clear? I've repeated it like three times, so I guess my grammar isn't correct?
<sphalerite>
if each module output has links to its dependency outputs, where are the dependency outputs created?
<ekleog>
you only need the list of modules for that, which I've assumed to be given four times now :)
<samueldr>
ekleog: you assume something akin to the other "generators" in nixpkgs, something like you called Kconfig2nix, right?
<Dezgeg>
that's not feasible to implement without using import-from-derivation
<ekleog>
samueldr: I'm not the one who called it Kconfig2nix, but that's more or less the idea, except you could just trigger a full non-multiple-output kernel build and list the modules from there :)
* samueldr
must have crossed lines while reading
<ekleog>
(and Dezgeg ^ too, we can hardcode the list of outputs if it's auto-generated)
<Dezgeg>
it cannot be hardcoded because it will depend on the kernel version, architecture and config options
<ekleog>
which are all given as arguments… ok so *that* is a convincing argument, thanks! :) I guess it'd be possible to also take as parameter the list of outputs to generate, and if someone customizes their kernel they would anyway have to things like that / could build a non-multiple-output kernel ?
<ekleog>
(and then have the defaults set for linuxPackages.whatever)
<sphalerite>
oooh I thought we were assuming there *wouldn't* be a Kconfig2nix
<sphalerite>
>_<
<ekleog>
\o/
<ekleog>
quid pro quo solved
<gchristensen>
is there a way we could take our existing config and adding a new module to the already compiled one? even if it does take some reduplication
<sphalerite>
still, just copying bits and pieces when you really can't afford to build your own kernel and building your own when you can seems good enough to me :p
<sphalerite>
gchristensen: that's a good question. I'm going to try it. :D
<sphalerite>
I imagine it would be similar to building out-of-tree modules like zfs
<gchristensen>
yeah, then we don't have to solve all this hard stuff but users can stil get some modules
<sphalerite>
(by "try it" I mean give it a quick shot and give up if I fail!)
<samueldr>
I wonder if it'd make sense to snapshot the whole kernel source directory after configuration as a package so you can "hop in and build what you need"
<ekleog>
gchristensen: we have that for eg. wireguard / virtualbox, if I understood your question
<gchristensen>
yeah so if I could cheaply just add the mellanox module, that would make me way happy :)
<samueldr>
(though, I think it'd still need to re-build a bunch)
<sphalerite>
samueldr: that's basically what we have already with the separate configFile derivation I think?
<samueldr>
I should read on that to know
<ekleog>
I think you'd need to rebuild the whole kernel in order to be able to build the mellanox module, though :(
<gchristensen>
oh?
<ekleog>
<sphalerite> still, just copying bits and pieces when you really can't afford to build your own kernel and building your own when you can seems good enough to me :p <-- well, yeah, but it's implementing basically the same logic as multiple-outputs, except everyone must do it themselves and it duplicates stuff on the building machine :)
<sphalerite>
ekleog: not really — I imagine it would be significantly easier to make a generic thing that can make a "trimmed" kernel modules tree from a full one based on modules.dep than to make Kconfig2nix
<ekleog>
gchristensen: when I used gentoo I always build the required with =y and the rest with =n (including module support =n), so I may be wrong here
<ekleog>
sphalerite: well, the Kconfig2nix really only needs to list the modules, if you give me a tick I can make it with a one-liner
<sphalerite>
oh?
<ekleog>
find /run/current-system/kernel-modules/lib/modules/4.14.71/kernel/ -type f | sed 's;^.*/\([^/]*\)\.ko\.xz;\1;'
<sphalerite>
that doesn't work because you need the kernel to be already built
<ekleog>
yeah, the steps would be 1. build the kernel as a monorepo, 2. run this command and generate the kernel.nix, 3. build the kernel and split it into multiple outputs
<ekleog>
potentially using your copying bits of the kernel trick to copy said bits to the multiple-output derivation from the one-output derivation directly on hydra
<ekleog>
(in order to avoid doing two builds)
<ekleog>
strike that, it'd require IFD indeed to be useful, anyway hydra would have to do one build and the contributor of kernel.nix one
<sphalerite>
:|
<ekleog>
now yeah, ideally there'd be IFD… but that world isn't coming
<gchristensen>
a project I work on uses a lot of IFD and I think avoiding it is the right choice
<gchristensen>
it makes for a weird phase of having to build stuff for a while before you know what you can build
<sphalerite>
ekleog: I'd say the best solution is to have a derivation that uses the hydra-built kernel as an input, some function that takes a list of desired modules, and runs through modules.dep to obtain the closure of those modules then copies them into the output
<sphalerite>
small resulting module set, no building a kernel, but you do require the full kernel to be able to build it
<sphalerite>
but AFAICT your suggestion doesn't avoid the latter part either?
<ekleog>
hmm… we can fetch only an output from a derivation from hydra, right?
<ekleog>
so my suggestion would be basically 1. have hydra build the kernel and split it into outputs, 2. the client can ask for just the required outputs and hardlink them into a kernel tree
<ekleog>
the client never needs to touch the whole kenrel
<sphalerite>
yes 2 is easy when you've got 1. The problem is, *how do you get 1*
<ekleog>
well, with the afore-mentioned process of listing outputs on the compiler of the person who did the last kernel update in nixpkgs :)
<ekleog>
s/compiler/computer/
* ekleog
needs to sleep
<sphalerite>
that sounds like it would cause a bootstrap problem
<sphalerite>
because new modules added in a new kernel version won't be taken into account
<ekleog>
yeah, it means people who bump the kernel version need to refresh the module list, like with things like firefox langpacks or the like
<ekleog>
actually firefox langpacks are the exact thing I should be comparing kernel modules with :D
<ekleog>
then… what I say is by no means a priority for anyone I know of, so it's all theoretical speech anyway
<ekleog>
(that said, 'night :))
<sphalerite>
gnight!
<{^_^}>
Night!
<sphalerite>
gchristensen: so I've got just that building in a nix-shell, and written an expression that will hopefully produce the same thing…
<gchristensen>
no idea :D I can try it ~sometime~ but not for the next many hours at least
<gchristensen>
(I'm working from unusual locations for the next few days)
<Dezgeg>
why all this effort instead sticking it to common-config.nix just like the other bajillon options?
<gchristensen>
well.... if there was a nice option to not have to do that, it'd be cool to document it
<Dezgeg>
but now everyone who has this particular card needs to somehow figure it out instead of having it work out of the box
<sphalerite>
oh yeah, build using nix-build mlx5.nix --arg linux '(import <nixpkgs> {}).linux'
<gchristensen>
yeah, this option might be good to enable by default
<sphalerite>
Dezgeg: sure, but having this way of building just one module documented is useful for future things — if someone wants to use module xyz *now* and not have to wait for everything to build after modifying the common one
<sphalerite>
I am in favour of enabling it by default too
<sphalerite>
but if this works, it's nice to have for the purpose of trying out a module quickly
<sphalerite>
in fact, I'll probably use this to build a patched radeon module without building the whole kernel for one of my machines :D
<sphalerite>
once I have that machine again.
<gchristensen>
yeah, I think it is fine to enable by default but if we can make a nice thing for other modules for other niche use cases, it'd be good
<Dezgeg>
well, if this doesn't work for the general case or bitrots in some way, there's a real risk in someone spending half hour of human time to save half hour of compilation time
jtojnar has joined #nixos-dev
<gchristensen>
yeah I agree, the solution would need to be good and not bitrot
<gchristensen>
ok so andi- pointed out a bunch of builds were deadlocked on hydra. making sure the packet machines are all running 2.1.3 which has a fix for that
* sphalerite
is now testing mlx5.nix with linux and linux_latest across unstable, 18.03 and 16.03 to see how much it might bitrot :p
<sphalerite>
ok, doesn't work on 16.03 because it didn't have overrideAttrs :D
<gchristensen>
ok so andi- pointed out a bunch of builds were deadlocked on hydra. I'm doing a rolling deploy with --force-reboot to each one to solve it.
<gchristensen>
the good news is my monitoring did catch it, next step might be some form of alerting.
jtojnar has quit [Ping timeout: 252 seconds]
jtojnar has joined #nixos-dev
jtojnar has quit [Ping timeout: 252 seconds]
jtojnar has joined #nixos-dev
jtojnar has quit [Remote host closed the connection]
Lisanna has quit [Quit: Lisanna]
jtojnar has joined #nixos-dev
orivej has joined #nixos-dev
matthewbauer has joined #nixos-dev
matthewbauer has quit [Read error: Connection reset by peer]
matthewbauer has joined #nixos-dev
matthewbauer has quit [Remote host closed the connection]
orivej has quit [Ping timeout: 260 seconds]
<gchristensen>
it would be nice to be able to access the arguments which were first used to import Nixpkgs
Lisanna has joined #nixos-dev
<LnL>
we could do something similar to pkgs.path
orivej has joined #nixos-dev
jtojnar has quit [Remote host closed the connection]