init_6 has quit [Read error: Connection reset by peer]
<samueldr>
maybe related to MichaelRaskin's proposal, but which instruction set should nixpkgs target, #56786 disables checks for a package since it might end up on a machine without AVX
<samueldr>
I don't like the idea of disabling the checks and still build something that the farm can't handle, and it might not be the only thing in that situation too
<ma27>
samueldr: may I ask what you'd do instead? I thought that without precise matching to tell which optimizations can be used, the best thing to do would be to use defaults (AVX enabled in this case) and run tests without AVX to ensure that they don't fail on a Hydra machine (building python-dlib costs time, so it shouldn't fail IMHO)
<samueldr>
I see three cases: use upstream default and nothing else (current situation); ensure all of nixpkgs can be executed on a target minimum platform; mark AVX as a feature a build machine needs for that derivation and mark hydra instances which can do AVX
<samueldr>
the latter is fine if it's fine that not the whole of nixpkgs can be executed on the minimum platform
<samueldr>
at least, that's what I see
<samueldr>
though I think this could point towards a more subtle issue, where we may have things building with features not the whole of hydra instances can handle, waiting for a long build on qemu non-kvm to see if I can repro an earlier feature
init_6 has joined #nixos-dev
qyliss_ has joined #nixos-dev
averell- has joined #nixos-dev
Haskellfant has joined #nixos-dev
qyliss has quit [*.net *.split]
averell has quit [*.net *.split]
NinjaTrappeur has quit [*.net *.split]
angerman has quit [*.net *.split]
cbarrett has quit [*.net *.split]
cocreature has quit [*.net *.split]
qyliss_ is now known as qyliss
Haskellfant is now known as cocreature
averell- is now known as averell
init_6 has quit [Read error: Connection reset by peer]
<MichaelRaskin>
Right, thought about mentioning it explicitly andforgot
<MichaelRaskin>
Added
johanot has joined #nixos-dev
johanot has quit [Quit: WeeChat 2.2]
init_6 has joined #nixos-dev
<ma27>
samueldr: I don't think that using a minimum set of features is a good idea (at least for dlib) as people will experience significantly less performance (http://dlib.net/compile.html). Matching where to build (a.k.a which flags to use) is basically what volth and I agreed on, however this isn't implemented in nixpkgs yet. I mainly wanted to work around the issue with the probably broken builds on Hydra so
<ma27>
as soon as we have proper matching of advanced hardware features I intend to improve the code in the dlib derivation %)
orivej has joined #nixos-dev
cbarrett has joined #nixos-dev
angerman has joined #nixos-dev
NinjaTrappeur has joined #nixos-dev
init_6 has quit [Read error: Connection reset by peer]
jtojnar has quit [Quit: jtojnar]
jtojnar has joined #nixos-dev
<samueldr>
the question was more general than for dlib; this is something we need to address I think since as new instruction sets arrive, it'll happen more and more
<samueldr>
there are different kind of situations, specialized software which benefit greatly from them, and routine optimizations, I guess
<samueldr>
should the whole of nixpkgs be usable on that minimum platform without recompilation? probably; what about specialized software? don't know how to handle it, but optimizations would be good too :/
jtojnar has quit [Read error: Connection reset by peer]
jtojnar has joined #nixos-dev
jtojnar has quit [Remote host closed the connection]
JosW has joined #nixos-dev
contrapumpkin has joined #nixos-dev
copumpkin has quit [Ping timeout: 255 seconds]
copumpkin has joined #nixos-dev
contrapumpkin has quit [Ping timeout: 245 seconds]
drakonis has joined #nixos-dev
asymmetric has joined #nixos-dev
lopsided98 has quit [Quit: No Ping reply in 180 seconds.]
lopsided98 has joined #nixos-dev
<catern>
hmm so there's a regression in restrict-eval and pure-eval modes, if you use a non-default store location by setting NIX_REMOTE, it won't work in Nix 2.1.3 and will work in Nix 2.0.1
<LnL>
you can't use <nix> unless it's in the target store
<catern>
isInStore is incorrectly firing on the <nix> added to the include path
<catern>
LnL: no, I'm not using it, it fails before any evaluation takes place, when it tries to process the search path as done in that commit I just linked, and fails when it hits the nix entry
<catern>
LnL: right, so isInStore should definitely be false, but it's going off of storeDir instead of realStoreDir... why do both of those two exist instead of just having one storeDir?
<clever>
and this builds a modified nix within that store, that has all of those baked into it!
<clever>
then, you can tar /home/clever/rootfs up, and unpack it such that /home/clever/rootfs/home/$X/nix/store lands at /home/$X/nix/store
<LnL>
the build doesn't know the real path but nix needs to know both to put stuff in the right place
<catern>
I don't think that's relevant, this is just a bug in the interaction between localStore with a non-default store path and restricted evaluation mode
<LnL>
but it's relatively new so not all cases use the new realStoreDir correctly
<clever>
LnL: i have 2 or 3 open bugs on nix involving that
<catern>
LnL: hmm I see
<catern>
LnL: so, I want to actually change the store dir, though
<catern>
like, have builds using /whatever/nix/store
<manveru>
can someone explain how the nix hashing algo works?
<manveru>
i basically have a lockfile with the hashes of `cat $lib/VERSION $lib/metadata.config $lib/contents.tar.gz | sha256sum -` and would like to build a fetcher for it so i don't need to build a separate lockfile for nix that has to be generated
<manveru>
in theory, i could put that concatenated thing in the nix store, and in the next step cut everything until the gzip header to just get the contents.tar.gz, would that be a good idea?
drakonis2 has quit [Quit: WeeChat 2.3]
asymmetric has quit [Ping timeout: 245 seconds]
asymmetric has joined #nixos-dev
<clever>
manveru: nix uses the hash of the nar, so nix-store --dump $lib | sha256sum -
<manveru>
i guess that's order-dependent though :|
<clever>
manveru: the rules of a nar file enforce a consistant sorting order, so it always gives the same hash
<manveru>
yeah, but that doesn't match the order of the hashing algo in the lockfile
<manveru>
anw, i'll try my way for now, i think it has a chance to work :)
<manveru>
problem is that it will still rely on IFD to convert that lockfile from elixir syntax to nix...
<clever>
manveru: one option, is to not try to convert the lock file and hashes into a single recursive nix hash
<clever>
manveru: instead, create a dozen flat nix hash derivations (basically, fetchurl), for every input in the upstream lock file
<clever>
manveru: then have a non-fixed derivation, that combines them all together
<clever>
133 outputHashMode = if (recursiveHash || executable) then "recursive" else "flat";
<clever>
if you set `outputHashMode = "recursive";` then nix will use the hash of the NAR, which is hard for you to convert to
<manveru>
but that leads to refetching of all the constituents
<clever>
but if you set `outputHashMode = "flat";` then its just the raw hash of $out, which must be a single file
<clever>
thats going to match your lock file a lot better
<clever>
and each constituent dep is a single derivation, that it can fetch&cache seperately
<manveru>
hmm
<gchristensen>
that'll be much nicer
<manveru>
i think i'm lost there :)
<clever>
manveru: go thru the lock file, and find every URL + sha256 pair, and then create a fetchurl call for each one
<clever>
manveru: and then have a non-fixed derivation that joins all of those into a single directory (preferably with symlinks)
<manveru>
so, you'd only specify a single hash for the toplevel symlink tree of all the derivations?
<clever>
the symlink tree is not a fixed-output drv
<clever>
so its $out will depend on the sha256 of every input fetchurl (and also the build of coreutils)
<manveru>
lemme see if i can make that work
MichaelRaskin has quit [Quit: MichaelRaskin]
asymmetric has quit [Ping timeout: 250 seconds]
lopsided98 has quit [Quit: Disconnected]
lopsided98 has joined #nixos-dev
lopsided98 has quit [Client Quit]
<thoughtpolice>
gchristensen: Around still? You pinged yesterday