<NinjaTrappeur>
Hey, I'm new to nix and I wanted to dive a bit in the implementation. I started to fix a minor issue on nix-store as an excuse to dive in the code. I raged for 30 minutes in front of GDB not understanding why my changes to lib-store were not reflected before realizing I was actually not using my local build but my nixos nix-daemon ><
<NinjaTrappeur>
Is there any clean way to prevent the nix-shell "nix" environment from calling my nix-daemon?
<aminechikhaoui>
are you on NixOS ?
<NinjaTrappeur>
yes
<clever>
NinjaTrappeur: NIX_REMOTE=local
<clever>
it will also need write access to /nix/store for that to work
<clever>
so it only works as root
<NinjaTrappeur>
clever, awesome, thanks! Yes, of course.
<NinjaTrappeur>
May I add this tip in the hacking section?
<clever>
though you can also use a different rootfs, NIX_REMOTE=local?root=/home/clever/rootfs/
<clever>
it will then act on /home/clever/rootfs/nix/store/
<clever>
the binaries within will only work if you chroot to the rootfs dir
<aminechikhaoui>
I guess you could also set `nix.package = (builtins.storePath <path>); where path is your own nix versions built with nix-build
<clever>
and still expect a /nix/store path for the store
<NinjaTrappeur>
ah, good idea
<NinjaTrappeur>
thanks
<clever>
that can also help prevent it from ever corrupting the real store
coconnor has joined #nixos-dev
abbradar has quit [Remote host closed the connection]
zybell_ has joined #nixos-dev
abbradar has joined #nixos-dev
<abbradar>
,b
<abbradar>
wrong window, sorry
orivej has joined #nixos-dev
<copumpkin>
FRidh: I'm seeing some odd behavior with a private buildPythonPackage between 17.09 and 18.03 that I can't explain. I didn't have doCheck specified before, and I don't think it ran tests, but now it does seem to, and it's not a big deal but I also don't understand what changed in the python machinery to run tests automatically
<LnL>
copumpkin: I think the checkPhase was enabled by default on master a while back
Lisanna has joined #nixos-dev
<Profpatsch>
Sonarpulse: What’s the best way to collect all dependencies of a derivation?
<Sonarpulse>
Profpatsch: in what sense?
<Sonarpulse>
buildInputs and friends?
<Sonarpulse>
nix references if it is built?
<Profpatsch>
Every drv that goes into building a drv basically.
<Sonarpulse>
there is some nix-level thing for that
<Sonarpulse>
I don't remember what it is called
<Profpatsch>
There’s a bunch of new fields I’ve never seen.
<Profpatsch>
hello.depsBuildBuildPropagated
<Profpatsch>
hello.all
<Sonarpulse>
oh those are the new dep types for cross
<Sonarpulse>
no idea what "all" is though
<Sonarpulse>
is this for check meta?
<Profpatsch>
nix-repl> map (v: v == glibc) glibc.all
<Profpatsch>
[ true false false false false ]
<Profpatsch>
Sonarpulse: I’m investigating if getting a full tree of dependencies is possible from within nix
<Profpatsch>
There’s just *so* many circular dependencies.
<Profpatsch>
Sonarpulse: Yeah, but all the meta info is missing.
<Sonarpulse>
this is for meta check?
<Profpatsch>
Which is the relevant bit for finding the corresponding maintainers.
<Profpatsch>
Nope, for pinging maintainers.
<Sonarpulse>
ah ok
<Sonarpulse>
hmm
<Profpatsch>
Maybe it’s already enough. But it’s probably missing half the outputs.
<Sonarpulse>
no bootstrap tools on gcc seems wrong
<Sonarpulse>
yeah
<Profpatsch>
*err dependencies.
<Profpatsch>
I’m collecting them from drvAttrs, which is probably wrong?
<Sonarpulse>
I suppose meta is like passthru
<Sonarpulse>
doesn't influence drb
<Profpatsch>
exactly.
<Sonarpulse>
the point of pinging all the deps is what exactly?
<Sonarpulse>
(basic concept makes sense)
<Sonarpulse>
(just trying to be really precise on goal)
<Profpatsch>
Ping the maintainers whose drvs changed.
<Sonarpulse>
ah from a PR?
<Profpatsch>
yeah
<Sonarpulse>
that would be reverse dependencies of change, right?
<Profpatsch>
Yep.
<Sonarpulse>
err but this is looking at dependencies not reverse dependencies
<Sonarpulse>
if gcc is changed
<Sonarpulse>
those will be the only things that *haven't* changed
<Profpatsch>
That’s just one small part.
<Sonarpulse>
ok
<Profpatsch>
The algorithm is:
<Profpatsch>
Take a list of all packages (pkgs)
<Profpatsch>
Build up a tree of all deps for each of them
<Profpatsch>
Do that twice, with and without patch
<Profpatsch>
Diff the tree for changed drvs.
<Profpatsch>
The deepest leaves are the reverse dependencies that caused the changes
<Sonarpulse>
the deepest leavs of the diff tree you mean?
<Profpatsch>
yes.
<Sonarpulse>
ok
<Sonarpulse>
so there are ways with like "......${asdf}....." to add dependencies
<Sonarpulse>
that are very hard to catch
<Sonarpulse>
the string contexts or whatever i dunno how it works
<Profpatsch>
Right!
<Profpatsch>
They are not in the type="derivation" attrset I suppose? I can test, actually.
<Sonarpulse>
so the drv files list all the deps
<Sonarpulse>
including those secret ones
<Sonarpulse>
but I don't think the drv attrset does
<Sonarpulse>
try importing a drv file in the repl and see what happens
<Sonarpulse>
Profpatsch: I'd consider going straight to hnix for this one
<Sonarpulse>
I don't think its possible to be 100% correct in nix
<Sonarpulse>
unless we can override things to put the meta attr in the drv
<Profpatsch>
Ah, they are not.
<Profpatsch>
Well, then the algorithm is changed.
<Sonarpulse>
if we can annotate derivations
<Sonarpulse>
and shlevy has talked about that
<Profpatsch>
Ah, no, I can’t pick those up and reassiciate them with meta info sadly.
<Sonarpulse>
then we can put meta in drvs
<Sonarpulse>
and then your algorithm works on drvs
<Profpatsch>
So there is no way to go from drv or storepath to a meta attribute.
<Sonarpulse>
nope
<Sonarpulse>
not that I know of at least
<Profpatsch>
Hm, maybe there is.
<Sonarpulse>
it's stripped away
<Sonarpulse>
well you could make index
<Sonarpulse>
I suppose
<Profpatsch>
If I constructed a hashmap from drv paths/out paths to drvs
<Profpatsch>
yeah, index
<Sonarpulse>
drvpaths -> meta.maintainers
<Sonarpulse>
nix-diff should probably know how to find the leaves
<Sonarpulse>
(in general its a dag not tree, then the root changes are the *only* leaves)
<Profpatsch>
Every dag can be made into a tree. :)
<Profpatsch>
Yeah, so building an index and working on that is going to be the way to go I think.
<Profpatsch>
And probably working breadth-first, calling into nix-instantiate multiple times.
goibhniu has quit [Ping timeout: 260 seconds]
<Profpatsch>
To prune when already known drvs are reached.
<Profpatsch>
Building the index is a best-effort thing, but because pkgs contains references to nearly all derivations of note, we shouldn’t miss too many packages.
<Profpatsch>
Or maintainers fields in this case.
<Profpatsch>
And one can warn on differences in dependencies between the drv files and what is found on the nix level.
<niksnut>
Profpatsch: there is some code for traversing derivation graphs in maintainers/scripts/find-tarballs.nix
<niksnut>
it uses genericClosure to make the traversal efficient
<Profpatsch>
niksnut: Thanks for the tip!
Lisanna has quit [Ping timeout: 260 seconds]
Lisanna has joined #nixos-dev
Lisanna has quit [Client Quit]
Jackneill has quit [Read error: Connection reset by peer]
<Sonarpulse>
even if the user is confused, it makes them think the wrong things
<Dezgeg>
how will it un-confuse all the existing users who have --argstr system foo in their scripts that has worked for years and have to go modify them?
<Sonarpulse>
Dezgeg: you value backwards compatability more than me
<Sonarpulse>
after isArm
<Sonarpulse>
I don't think I'll convince you this is a good idea either
<Sonarpulse>
but here i goes anyways
<Sonarpulse>
I made localSystem to match crossSystem
<Sonarpulse>
and {build,host,target}Platform
<Sonarpulse>
so that everything had the same structure
<Sonarpulse>
having system and platform as top level arguments to nixpkgs is...
<Sonarpulse>
a) weird because the division of labor is kind of arbitrary
<Sonarpulse>
b) obscures the fact that everything is now the same schema/type
<Sonarpulse>
c) obscures cross in general
<Sonarpulse>
d) doesn't allow setting all options
<Sonarpulse>
removing them and leaving localSystem
<Sonarpulse>
a) makes fewer redundant ways to sets things
<Sonarpulse>
b) shows the relation
<Sonarpulse>
c) hints at cross being available
taktoa has quit [Remote host closed the connection]
FRidh has quit [Quit: Konversation terminated!]
abbradar has quit [Remote host closed the connection]
phreedom has quit [Remote host closed the connection]
phreedom has joined #nixos-dev
jtojnar has joined #nixos-dev
goibhniu has joined #nixos-dev
orivej has quit [Ping timeout: 276 seconds]
xeji has joined #nixos-dev
goibhniu has quit [Ping timeout: 265 seconds]
taktoa has joined #nixos-dev
<obadz>
so I remember there's a way to either check that a derivation output doesn't contain a specific dependency (and fail if it does) or maybe even to prune the dependency, but I don't remember how to do it.. anyone knows?