<Sonarpulse>
gchristensen: I've was thinking about writing the next nix pill that lethelman never wrote/finished
<Sonarpulse>
but
<Sonarpulse>
at some point as the nix pills get more advanced
<Sonarpulse>
they become more specific to a single nixpkgs version
<Sonarpulse>
rather than general design patterns
<Sonarpulse>
separately, bgamari was reminding me of the usefulness of "maintainer-notes" implementation-specific documentation
<Sonarpulse>
I'm wary of over engineering things, but I'm somewhat tempted to split the nix pills into sort of "nix / timeless" and nixpkgs-specific
<Sonarpulse>
orivej: vaguely related to the above, bgamari once reminded me we'll propably needs a pkgconfig-wrapper
<Sonarpulse>
at the point, the redundancies between the tree wrappers mean we should factor things out
<Sonarpulse>
I have a bunch of comments in cc-wrapper that I think no one is seeing, and furthermore is annoying to modify because it will cause mass-rebuilds
<gchristensen>
I'd be cautious about making too many disparate docs
<Sonarpulse>
this *-wrapper stuff might be lower hanging fruit
<Sonarpulse>
get the documentation easily editable in some new spot
<Sonarpulse>
then think about how that may/may not overlap with a *-wrapper / env hook nix pill
<Sonarpulse>
dtz: I have an old WIP branch for clang cross compile
<Sonarpulse>
dtz: its a bit subtle as tools and libraries (libcxx, compiler-rt) need to come from different stages
<Sonarpulse>
so might be of some use
<bgamari>
Is it just me or does cabal2nix not workwith cabal-install 2?
<bgamari>
it seems to look for the old 00-index.gz, which cabal no longer downloads iirc
<bgamari>
either that or I am doing something terribly wrong
<dtz>
hmm might be good to look at old clang cross-compile if you have around
<dtz>
i mostly just started building "all" llvm bits together and it became easier lol :P
<Sonarpulse>
dtz: I'll dredge it up
<Sonarpulse>
dtz: last thing on the texinfo is that the "buildPackages." in buildPackages.perl shouldn't be needed
<bgamari>
Sonarpulse, didn't I have a patch for that?
<Sonarpulse>
bgamari: this was yours cherry-picked many times I'd think
<Sonarpulse>
I'll check
<dtz>
Sonarpulse: re:pciutils nah we can just drop it if can't get it working reasonably--I'm grabbing the commit you pushed now and will stash locally thank you :)
<bgamari>
that being said, it's possible that I just worked around pciutils
<bgamari>
since my target doesn't have any PCI devices
<Sonarpulse>
dtz: sounds good
<bgamari>
dtz, thanks again for taking the time to merge sort through my trail of chaos
<dtz>
Sonarpulse: building texinfo5 and texinfo6 on aarch64 presently, finally got through all the deps xD
<Sonarpulse>
dtz: cool!
{^_^} has quit [Remote host closed the connection]
{^_^} has joined #nixos-dev
<dtz>
texinfo looks good ^_^
<Sonarpulse>
dtz: cool!
<Sonarpulse>
dtz: stashed it?
<Sonarpulse>
err wrong branch
<dtz>
:)
<dtz>
and yeah grabbed your pciutils commit, ty sir :)
orivej has quit [Ping timeout: 268 seconds]
mbrgm has quit [Ping timeout: 256 seconds]
mbrgm has joined #nixos-dev
acowley has quit [Ping timeout: 268 seconds]
acowley has joined #nixos-dev
Sonarpulse has quit [Ping timeout: 256 seconds]
la_putin has joined #nixos-dev
el_putin has quit [Read error: Connection reset by peer]
Sonarpulse has joined #nixos-dev
ma27 has joined #nixos-dev
ma27 has quit [Ping timeout: 265 seconds]
Sonarpulse has quit [Ping timeout: 246 seconds]
la_putin has quit [Quit: Konversation terminated!]
orivej has joined #nixos-dev
pie__ has quit [Ping timeout: 246 seconds]
orivej has quit [Ping timeout: 256 seconds]
pie_ has joined #nixos-dev
orivej has joined #nixos-dev
FRidh has joined #nixos-dev
* domenkozar
waves
pie_ has quit [Ping timeout: 240 seconds]
<FRidh>
This is is quite nice: `with import (fetchTarball channel:nixos-17.09) {};` Does a nix command exist for substituting this with a fixed revision and hash? Something like `pip freeze` but then pinning Nixpkgs commit.
infinisil has quit [Quit: ZNC 1.6.5 - http://znc.in]
viric has joined #nixos-dev
infinisil has joined #nixos-dev
ma27 has joined #nixos-dev
ma27 has quit [Ping timeout: 255 seconds]
ma27 has joined #nixos-dev
Sonarpulse has joined #nixos-dev
<copumpkin>
niksnut: I'm making progress on my mysterious sandbox bug. It only shows up with --check! Still no clean repro unfortunately, but I think I'm honing in on it
<copumpkin>
homing
<niksnut>
weird
<copumpkin>
yeah, can't explain it yet based on the code
<Dezgeg>
maybe without --check it manages to read nsswitch.conf from before entering the sandbox and caches that?
<Dezgeg>
and/or resolv.conf
<copumpkin>
now trying to get it to fail in a builder that I control rather than builtin:fetchurl
pie__ has joined #nixos-dev
<copumpkin>
maybe that'll clarify what's going on
pie_ has quit [Ping timeout: 256 seconds]
<niksnut>
copumpkin: have you tried strace?
<copumpkin>
yup, nothing obvious jumped out at me, but I'll look more carefully soon if this path shows me nothing
ma27 has quit [Ping timeout: 255 seconds]
<niksnut>
I would look for connect() calls
<niksnut>
and grep for nscd
<copumpkin>
yeah I did, it kept trying to connect to the nscd socket but it wasn't there, iirc
<copumpkin>
will look again soon
ma27 has joined #nixos-dev
<copumpkin>
okay, so it doesn't seem to happen for my non-builtin builder with other conditions identical
<copumpkin>
so I guess strace it is
<copumpkin>
so I see various messages about trying to connect to nscd socket, but ENOENT
<copumpkin>
e.g.,
<copumpkin>
[pid 1347] connect(3, {sa_family=AF_UNIX, sun_path="/var/run/nscd/socket"}, 110) = -1 ENOENT (No such file or directory)
<copumpkin>
same in the child
<copumpkin>
although it's broken up with other nonsense due to concurrency
<Dezgeg>
actually, now that I see that those lines don't match: [pid 1360] stat("/etc/resolv.conf", <unfinished ...> [pid 1360] <... open resumed> ) = -1 ENOENT (No such file or directory)
<niksnut>
probably we shouldn't bind-mount the host's nsswitch.conf, but install a minimal one
<Dezgeg>
stat() is unfinished but open() gets ENOENT
<copumpkin>
niksnut: in my case I had to avoid nscd on purpose because it was somehow going out to the host (as far as I could tell) and getent would fail for nix looking up the build user groups
<niksnut>
edolstra, We found a potential security vulnerability in a repository for which you have been granted security alert access.
<copumpkin>
I could probably be more precise about that
<niksnut>
github complaining about gemfile.lock files again
<copumpkin>
niksnut: nice!
<copumpkin>
lol
<copumpkin>
Dezgeg: yeah the lines might not match up properly, because I'm getting junk from all the concurrent processes with strace -f
<copumpkin>
is tehre a good way to disentangle those?
<Dezgeg>
you can split it to one file per process with some strace flag
<copumpkin>
ah, cool
<copumpkin>
didn't know about that one
pie__ has quit [Ping timeout: 240 seconds]
Sonarpulse has joined #nixos-dev
ma27 has joined #nixos-dev
<jtojnar>
why does nix why-depends build packages?
<domenkozar>
otherwise it can't know?
ma27 has quit [Ping timeout: 246 seconds]
<jtojnar>
is there something similar for the drv files?
<copumpkin>
Dezgeg: okay I now have a per-PID breakdown :P
ma27 has joined #nixos-dev
jtojnar_ has joined #nixos-dev
<jtojnar_>
something like nix-store --tree but in reverse
jtojnar has quit [Ping timeout: 246 seconds]
jtojnar_ is now known as jtojnar
jtojnar has quit [Ping timeout: 256 seconds]
ma27 has quit [Ping timeout: 276 seconds]
ma27 has joined #nixos-dev
JosW has joined #nixos-dev
pie__ has joined #nixos-dev
<copumpkin>
Dezgeg, niksnut: so I'm looking for write(2, "\1\n", 2) as what happens right before builtinFetchurl
<copumpkin>
there's no nix-daemon involved so I wouldn't expect much IPC, but perhaps the downloader stuff is weird?
<copumpkin>
I guess it must be in a different process
<copumpkin>
because I don't see any writes with the warnings about resolving the host name
<copumpkin>
niksnut: oh, so the actual download happens in the parent process? we seem to fork, then builtinFetchurl in the child somehow gets the parent process to run the actual download? definitely seeing the resolution messages coming from the parent in strace
<niksnut>
copumpkin: no
<niksnut>
download is done in the child
<copumpkin>
well, then this strace is super confusing
<copumpkin>
because the write(2, "warning: unable to download" is coming from the parent
<copumpkin>
whereas the final error: download failed comes from teh child
<copumpkin>
I think that's the clone = 2 from above
<copumpkin>
niksnut: I'm guessing the 8 might be trying tukaani after tarballs.nixos.org failed?
<copumpkin>
<copumpkin>
openat(AT_FDCWD, "/nix/store/1zv5dwifxg5fh08gif8ld3h9f40y8czh-glibc-2.26-115/lib/libnss_dns.so.2", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
<copumpkin>
if instead of builtins.fetchurl, I seq `5`, then it fails
<copumpkin>
niksnut: :P
<copumpkin>
so I think this is actually a more general bug but is almost always masked by nix doing DNS resolution of its own prior to calling a builder
<copumpkin>
in my case, being a super narrow build, it didn't do that and failed
<copumpkin>
it probably also relies on nscd being turned off
<Dezgeg>
yes
<copumpkin>
so basically, I'm extremely unlucky :)
<copumpkin>
but I'm not imagining things
<niksnut>
copumpkin: hm, I wonder why it's doing DNS resolution. Probably checking the binary cache.
<copumpkin>
yeah, or in my case (when I disable substitutes), builtins.fetchurl
<niksnut>
ah right
<copumpkin>
so the hacky fix would be to just insert a dummy resolution :P
<copumpkin>
always
<copumpkin>
but that doesn't feel ideal
<copumpkin>
maybe always insert nix's glibc into the sandbox?
<copumpkin>
I think the key issue is that if you're using a builtin builder, you could in theory use all of nix's own closure
<copumpkin>
in practice builtin:fetchurl is the only one and that doesn't do much
<niksnut>
the whole point of builtins:fetchurl was that it wouldn't require any sandbox configuration
<copumpkin>
but if someone were to add another builtin builder, that might be very confusing if it used any more of nix's dependencies
<niksnut>
we may as well get rid of it if that's not the case
<copumpkin>
well, I just mean in theory, if builder.isBuiltin then sandbox += nix.closure
<copumpkin>
hmm
<niksnut>
there is no nix closure necessarily
<copumpkin>
oh, because it might be installed in /usr
<niksnut>
right, or another nix store
<copumpkin>
hmm
<niksnut>
but maybe we can just force nss modules to be loaded
<niksnut>
in the parent
MichaelRaskin has joined #nixos-dev
<copumpkin>
yeah, that might be easiest
ma27 has quit [Ping timeout: 265 seconds]
<copumpkin>
nastiest solution of all: whenever the evaluator evaluates something, it implicitly prepends "builtins.sec (buitlins.fetchurl ...)"
<copumpkin>
:P
<copumpkin>
seq
jtojnar has joined #nixos-dev
FRidh2 has quit [Quit: Konversation terminated!]
<copumpkin>
niksnut: still curious about the other thing I was asking the other day: why is builtin:fetchurl even run in a child if it's purely nix code? it seems like we can trust that it'll do the right thing
michaelpj_ has joined #nixos-dev
<copumpkin>
can't it just enqueue a download in the parent process and be done with it?
<copumpkin>
that would avoid this issue and maybe simplify the whole thing
pie__ has quit [Ping timeout: 240 seconds]
<copumpkin>
niksnut: I don't see anything in the NSS machinery to preload stuff, except by actually performing a resolution. I could ask gethostbyname("nixos.org") at startup and not pay attention to the result, I suppose :P
<dtz>
aw man I remember doing that WAY back to get some stuff working on iOS lol
<copumpkin>
gonna run another test to make sure it's quiet
<copumpkin>
niksnut: looks good and quiet now :D
<copumpkin>
I suppose you might prefer the once_flag being a static variable inside preloadNSS?
<niksnut>
copumpkin: thanks!
<copumpkin>
thank you (and dezgeg) for all the help tracking it down :)
<copumpkin>
I feel very important to this project: it's essential to have someone very unlucky bug testing stuff :P
<copumpkin>
I'm gonna trigger a hydra nix eval and then update nixUnstable in nixpkgs
<gchristensen>
<3 copumpkin
<gchristensen>
<3 niksnut
<copumpkin>
niksnut: I think nix-build --hash might have broken
<copumpkin>
thanks gchristensen :)
<copumpkin>
niksnut: if I run the example from your original --hash buildmode commit, I now get:
<copumpkin>
output path ‘/nix/store/wyrj496mdypcqxik1fh95fjjfmma5yir-nix’ has r:sha256 hash ‘11clfc8fh8q8s3k4canmn36xhh3zcl2zd8wwddp4pdvdal16b5n6’ when ‘0fffffffffffffffffffffffffffffffffffffffffffffffffff’ was expected
<copumpkin>
(I had to replace the first f with a 0 to get it to pass the validation)
<copumpkin>
oh maybe it's just because my daemon is still 1.11
michaelpj_ has quit [Ping timeout: 240 seconds]
<copumpkin>
is setting `hashed-mirrors =` in nix.conf sufficient to disable hashed mirrors everywhere? (for builtins and nixpkgs fetchurl?
<copumpkin>
I think so, right?
<copumpkin>
it seems like the nixpkgs fetchurl supports a NIX_HASHED_MIRRORS but nothing in nix sets it
<copumpkin>
oh I see, mirrors.nix sets hashedMirrors :(
<copumpkin>
doesn't let me set it to an empty list
<copumpkin>
oh, I can set it to a single space
<LnL>
is that also used by fetchurl?
<LnL>
or just the builtin
<copumpkin>
NIX_HASHED_MIRRORS is used by nixpkgs fetchurl, and hashed-mirrors in nix.conf is used only by the builtin
<LnL>
I see, that's why it wasn't working
MichaelRaskin has quit [Ping timeout: 248 seconds]
<copumpkin>
niksnut: maybe it makes sense to put NIX_HASHED_MIRRORS purely into nix now (so that the hashed-mirrors option controls it), instead of hardcoding it in mirrors.nix in nixpkgs?
MichaelRaskin has joined #nixos-dev
<copumpkin>
that way we can stop using the impure env var in fetchurl
JosW has quit [Quit: Konversation terminated!]
infinisil has quit [Quit: ZNC 1.6.5 - http://znc.in]
infinisil has joined #nixos-dev
infinisil has quit [Quit: ZNC 1.6.5 - http://znc.in]
infinisil has joined #nixos-dev
<MichaelRaskin>
Well, whatever you do there is always proxy…
la_putin has joined #nixos-dev
<contrapumpkin>
oh I just mean for NIX_HASHED_MIRRORS
<contrapumpkin>
not all the impure env vars
<contrapumpkin>
it's just awkward to have this defined in a bunch of different places if you want to turn it off
<ekleog>
Hmm... Anyone knows whether there is a dirNameOf equivalent to baseNameOf? I can't find any :/
<LnL>
builtins.dirOf
<ekleog>
oh, thanks!
* ekleog
feels stupid right now
<contrapumpkin>
the hashed mirrors thing is hiding a lot of bad downloads now
<LnL>
I bet, didn't even know about that until recently
<gchristensen>
yikes
<gchristensen>
you mean b/c of tarballs.nixos.org?
<gchristensen>
that is why we have it :$
<contrapumpkin>
it sort of diminishes accountability though, too
<gchristensen>
accountability of upstreams?
<contrapumpkin>
of our packages
<contrapumpkin>
because there's not necessarily any trace left anywhere but nixos infra of where their inputs came from
<contrapumpkin>
it's much easier to trust package X if it just grabs source from *.gnu.org and builds it with a small script
<contrapumpkin>
than if it fetches something from tarballs.nixos.org/sha256/kljkljklawjfkelwjfwklejfkal that ostensibly came from gnu.org but doesn't match anything on there
* contrapumpkin
shrugs
<contrapumpkin>
it's a bit of a trade-off I guess
<contrapumpkin>
maybe we should have a fall-back with a warning
<gchristensen>
I suppose I see where you're coming from, but there is real value in the mirror in terms of being able to maintain thee distro, and if you don't trust tarballs.nixos.org to contain valid data then I suspect there are deeper problems
<contrapumpkin>
"original source didn't work, falling back to hashed mirror" or something
<gchristensen>
that would be great
infinisil has quit [Quit: ZNC 1.6.5 - http://znc.in]
<LnL>
how does that actually get populated?
<contrapumpkin>
it's also less clear to me what value hashed mirrors add over binary substitutes
<gchristensen>
hmm maybe not deeper problems if you don't trust anything produced by NixOS other than a given copy of nixpkgs and a given nix...
<contrapumpkin>
since we now seem to have two content-addressable substitution mechanisms
infinisil has joined #nixos-dev
<gchristensen>
there is a mirror-tarballs script somewhere
<gchristensen>
but I'm not sure if / when it is run
<gchristensen>
are FO outputs in the cache stable when the tools to fetch them change?
<contrapumpkin>
yeah
<LnL>
also things like changing the name
<contrapumpkin>
shot an email to nix-devel
<contrapumpkin>
will see
<LnL>
I was actually looking for some kind of content addressable fetch helper when I first came across it
<LnL>
was expecting something like fetchSha256 "..." for bootstrap tarballs
infinisil has quit [Quit: ZNC 1.6.5 - http://znc.in]
infinisil has joined #nixos-dev
<MichaelRaskin>
contrapumpkin: technically, hashed mirror can work with non-standard store path.
<contrapumpkin>
fair enough, but that seems somewhat far-fetched and depends on eelco remembering to populate it