<stephank>
Hmm, apparently Xcode does not install some pf headers by default, and Nix sort of follows that? I'm not sure if we can fix this package without changing the xnu package and causing a stdenv rebuild: https://hydra.nixos.org/build/143298626/log
__monty__ has joined #nix-darwin
<__monty__>
thefloweringash: Do you remember modifying rpathify in make-bootstrap-tools.nix?
<__monty__>
It prevents expat from building for me unless I drop the /lib from the "@rpath/lib/libc++abi.1.dylib".
<__monty__>
I'm very not familiar with the rpath mechanism though so I'm unsure whether patching rpathify is the proper fix or whether I need to do something to the expat expression so it sets the correct rpath on non-ARM Darwin.
<__monty__>
Looks like the Aarch64 bootstrap-tools job finished btw. Almost there \o/ : )
<thefloweringash>
__monty__: hmm, that's interesting, which branch is that?
<thefloweringash>
there's a slight mismatch between the bootstrap tools and the unpack script, but I wouldn't expect it to cause problems. which step of the build fails?
<thefloweringash>
the bootstrap tools should be able to unpack and run without modification. if there's a root, say $out, then the rpath points to $out relative to the executable (ie, `$out/bin/foo` has rpath including `@executable_path/..` == `$out`). libraries are loaded from @rpath/lib/libfoo.dylib == $out/lib/libfoo.dylib.
<thefloweringash>
the install name of the extracted libraries still needs to be fixed for freshly compiled binaries that link against them, which the unpack script also handles
<__monty__>
thefloweringash: The error is in the most recent section here, https://git.io/JsgUk
<__monty__>
I think maybe the bootstrap-tools are self-consistent but expat only sets rpath to its own lib, which breaks things.
<andi->
How reproducible are the darwin bootstrap tarballs? Just doing a native built after having built the cross toolchain.
<__monty__>
Afaict they're mostly just copying files. But they do depend on stdenv.
<andi->
I am curious if I end up with other hashes than the hydra native build :-)
<__monty__>
thefloweringash: Fwiw, the bootstrap-tools build I kicked off removing /lib in rpathify is still going. So it does seem like it helps with the expat build. Do you know of a better way to patch just that build maybe?
<thefloweringash>
that looks like a transitive load. a bin in expat -> absolute path to libc++ -> rpath relative path to libc++abi
<__monty__>
Yes, exactly.
<thefloweringash>
I’m curious why it passes with @rpath/libc++abi.dylib
<__monty__>
Actually, maybe it doesn't. I changed the bootstrap-tools used when building these bootstrap-tools, so it's probably still rebuilding LLVM 11... >.<
<__monty__>
I was simply trying to get back to the behavior before. Where just the lib's basename was used.
<thefloweringash>
We could rewrite the reference in libc++.dylib to be absolute. It seems like the simplest answer
<thefloweringash>
andi-: hydra has finished, if you’ve build the tools you can compare
<andi->
thefloweringash: I am using the one of hydra to build the native bootstrap seed
<andi->
so no longer cross bootstrap but natively built bootstrap
<thefloweringash>
Oh, right. I don’t know what to expect, let me know when you find out :-)
<andi->
at first I hope this M1 doesn't just power off again when it his the thermal limit..
<thefloweringash>
I ran into a few panics during “intltest” while testing it, always with some panic message like “spinlock timeout”. I really hope it’s a quirk of my local machine.
<andi->
tbh as soon as linux works reliably well on this box I'll switch..
<andi->
I still haven't understood how the code signing works with Nix. Did we disable that requirement? Wouldn't each machine produce a different binary if we signed the code? Are the signatures detached into some local dir?
mjlbach has quit [Ping timeout: 245 seconds]
thefloweringash has quit [Ping timeout: 276 seconds]
siraben has quit [Ping timeout: 245 seconds]
berbiche has quit [Ping timeout: 260 seconds]
thefloweringash has joined #nix-darwin
angerman has quit [Ping timeout: 260 seconds]
hhes has quit [Ping timeout: 260 seconds]
siraben has joined #nix-darwin
berbiche has joined #nix-darwin
mjlbach has joined #nix-darwin
<__monty__>
thefloweringash: So special-casing a call to install_name_tool to replace @rpath/lib/libc++abi.1.dylib with the $out/lib path, right? Any reason that's preferable to adding to expat's rpath?
hhes has joined #nix-darwin
<thefloweringash>
andi-: can you let me know if that build produces compiler-rt? I just noticed it’s broken on x86_64-darwin
<andi->
can do
<__monty__>
Yeah, I had to fix compiler-rt to include Libsystem in the nativeBuildInputs.
<__monty__>
Not sure whether buildInputs would be more appropriate.
angerman has joined #nix-darwin
ece has quit [Quit: Ping timeout (120 seconds)]
ece has joined #nix-darwin
supersandro2000 has quit [Killed (verne.freenode.net (Nickname regained by services))]
supersandro2000 has joined #nix-darwin
pingiun has quit [Quit: Bye!]
ris has quit [Remote host closed the connection]
pingiun has joined #nix-darwin
<__monty__>
thefloweringash: Hmm, how does #123524 fix compiler-rt to find TargetConditionals.h? It looks like your changes should only affect aarch64?
<andi->
Intresting the build of expat now fails while running tests
<andi->
https://termbin.com/r2sk - the non-cross bootstrap tarball is missing Library/Frameworks?
<andi->
ahh this must be what __monty__ was talking about earlier. I am slow m(
<thefloweringash>
So strange that it’s only on native builds.
<thefloweringash>
If you have both sets of tools side by side, you could check the load paths output (“otool -L”) of libc++.dylib to confirm the difference.
<andi->
First gonna put all that stuff in my binary cache.. don't want to spent several hours rebuilding this stuff..
<thefloweringash>
I'm only more confused. Can you dump the `-l` as well?
<andi->
thefloweringash: added two files to the same gist
<andi->
I am actually confused that we have @rpath in there. Is that common on darwin?
<andi->
At first I thought some string replacement went wrong
<thefloweringash>
it's not unusual, there are a few magic tokens like @rpath, @executable_path, @loader_path. mostly used so you can put your .app anywhere and it can find its embedded frameworks and libraries
<prusnak>
since the aarch64-darwin bootstrap is built in hydra, how do I use aarch64-darwin with nixpkgs?
<prusnak>
I checked out the 20acd4 commit and used the following:
<prusnak>
nix-shell --argstr system aarch64-darwin -I nixpkgs=/path/to/nixpkgs -p hello
<__monty__>
prusnak: Note that a new bootstrap means a new stdenv so I'm not sure you already want to be using it.
<prusnak>
but file $(which hello) says it's x86_64 executable
<andi->
prusnak: most likely --system aarch64-darwin or changing the default system in /etx/nix/nix.conf but as monty said it is not at a stage where you can build your system with it
<__monty__>
People are really jonesing for native aarch64 stuff, huh? : )
<prusnak>
ok, so I should wait a little more then, right?
<andi->
Yeah, I am using an older snapshot of the in-flight PR that was open earlier and that has been working great :)
<andi->
At least to the point where I do not have to use imperative shit to get my work done.
<__monty__>
prusnak: Depends on how much tolerance for rebuilds you have. If you want things like stdenv cached, yes some more patience is required : )
<prusnak>
i don't mind rebuilds - issue is I can't get it working
<prusnak>
I am on staging-next branch - which contains 20acd4c4f14040485f40e55c0a76c186aa8ca4f3 (the commit from hydra aarch64 bootstrap)
<prusnak>
this produces x86_64 executable:
<prusnak>
nix-shell --argstr system aarch64-darwin -I nixpkgs=/path/to/nixpkgs -p hello
<prusnak>
setting system = aarch64-darwin in /etc/nix/nix.conf produces weird error:
<prusnak>
error: The path name '.drv' is invalid: it is illegal to start the name with a period.
<prusnak>
but I am fine with waiting a couple of days if that's what I should do :)
<thefloweringash>
maybe nix-shell doesn't handle arguments? maybe try `nix-build --argstr system aarch64-darwin -I nixpkgs=/path/to/nixpkgs '<nixpkgs>' -A hello`
philr has quit [Ping timeout: 240 seconds]
<thefloweringash>
though there's no branch yet that has the bootstrap tools provided, so it won't work without changes
<prusnak>
I get the same "error: The path name '.drv' is invalid" error
<prusnak>
(so it seems that nix-shell is not taking arguments, but it is correctly interpreting system from /etc/nix/nix.conf)
<andi->
yeah that is due to the bootstrap tools not being plugged in yet. So far hydra has just produced a "trustworthy" bootstrap tarball that we can now use to plug it into the pkgs/stdenv/darwin/default.nix file
<andi->
prusnak: see my github URL from earlier today where I did basically that.
<prusnak>
andi-: ah, great - that was the missing piece of the puzzle
<andi->
Just tested those locally and as expected it yields the same output that I produced earlier :0
<gchristensen>
it is really nice and easy to upload these tarballs when the results are made with such clear provenance
<andi->
I'd love if we had another distro bootstrap source to show that we can converge on the same hashes again.
<gchristensen>
?1
<gchristensen>
+1
<andi->
We would probably use Guix to create bootstrap tarballs for NixOS... mhmm
<stephank>
I think I'm just about done doing darwin PRs. I went through the entire list of failures of nixpkgs:trunk, except lang ecosystem stuff (perlPackages, pythonPackages, etc.), and fixed most of the trivial stuff. :o
<stephank>
The next staging merge should also clear up dozens of failures, from packages that have a wayland dependencies. (Though they could also do without that.)
<stephank>
Plus the haskell-updates should also happen before Friday I'm told. That should also make another small improvement.
<stephank>
Is there any automation that reads build logs from Hydra? I was thinking of doing that, or extending something existing, to pick up on simple stuff, like missing libiconv, hardcoded gcc, etc.
<stephank>
even if it’s just to generate a report
<__monty__>
thefloweringash: Fwiw, dropping the /lib so it is @rpath/libc++abi.1.dylib worked. (I didn't interrupt the build.)
<clever>
has the M1 fixed the silly thing about the dynamic linker being hard-coded into the kernel?