worldofpeace changed the topic of #nixos-dev to: NixOS Development (#nixos for questions) | NixOS 20.09 Nightingale ✨ | | | 20.09 RMs: worldofpeace, jonringer |
BaughnLogBot has joined #nixos-dev
BaughnLogBot has quit [Client Quit]
BaughnLogBot has joined #nixos-dev
BaughnLogBot has quit [Quit: ZNC 1.6.2+deb1 -]
BaughnLogBot has joined #nixos-dev
BaughnLogBot has quit [Client Quit]
<supersandro2000> hexa-: I think I found the issue. Didn't properly quote the jq expression
<hexa-> we'll see
mkaito has quit [Quit: WeeChat 3.0]
supersandro20005 has joined #nixos-dev
supersandro2000 has quit [Ping timeout: 265 seconds]
supersandro20005 is now known as supersandro2000
tilpner_ has joined #nixos-dev
rajivr has joined #nixos-dev
cole-h has quit [Ping timeout: 260 seconds]
tilpner has quit [Ping timeout: 260 seconds]
tilpner_ is now known as tilpner
orivej has quit [Ping timeout: 272 seconds]
<siraben> Maybe it would be a good idea to add gnu32 and gnu64 to the cross-trunk jobset
<samueldr> ?
<samueldr> what are they even supposed to represent?
Jackneill has quit [Ping timeout: 256 seconds]
Jackneill has joined #nixos-dev
blueberrypie has quit [Quit: leaving]
blueberrypie has joined #nixos-dev
<supersandro2000> found something intersting for archiving our repos
<supersandro2000> used by the debian people
mjsir911 has quit [Ping timeout: 256 seconds]
mjsir911 has joined #nixos-dev
<supersandro2000> Has someone tried to write a script/function/program that tries to clear all {,native,propogated}buildInputs etc and check if the program still compiles or the output is different?
<samueldr> propagated wouldn't apply here, as the build may succeed, but things using that derivation could fail
vdemeester has quit [Ping timeout: 260 seconds]
vdemeester has joined #nixos-dev
jonringer has quit [Remote host closed the connection]
<jtojnar> rmcgibbo: I think we will want to combine checking logs with other types of checks too so they should get passed attrpath and file path too
<jtojnar> also porting the tool to rust first will allow us to switch to pull model and thus get rid of the need for distinguishing check types altogether
<jtojnar> i.e. we would extend the Files object with attributes and logs that could be requested by the checks lazily
<ashkitten> supersandro2000: sounds like you want something like the inverse of
<rmcgibbo[m]> Jan Tojnar: sure. i'm definitely less comfortable in rust -- i'm not sure if re-writing grep in rust for the log checks is better than having them in external binaries that can use shell, but i'm fine either way.
<jtojnar> worst case scenario we can shell out
<jtojnar> but IIRC ripgrep is a library
<rmcgibbo[m]> my counter-argument is that it feels like overengineering for something that is only 3 lines worth of logic, but i won't push it.
<jtojnar> we can definitely go the simple route first, and RiiR when as more complex logic is needed for the checks
<rmcgibbo[m]> Obviously your call -- I'm fine with the RewriteItInRust route.
<siraben> I don't quite get what's happening in
<siraben> I'm trying to cross-compile something that has SDL2 as a dependency but it's failing on dariwn
<qyliss> siraben: wayland is often a nativeBuildInput as well as a buildInput because of wayland-scanner
<siraben> There's this line in the SDL2 expression
<siraben> `waylandSupport ? stdenv.isLinux && !stdenv.hostPlatform.isAndroid`
<siraben> maybe it should be adjusted to be false when buildPlatform.isDarwin?
<qyliss> it would be a shame, because I'm sure there's no reason we can't at least build wayland-scanner on Darwin
<qyliss> plus you'd have to make that change to every Wayland dependent
<siraben> new error if I do that:
<qyliss> yeah exactly
<siraben> hmm
<qyliss> I think the Wayland attribute on Darwin should be something that just gives you wayland-scanner, for cross-compiling
<siraben> I just need SDL2 to cross-build
<qyliss> looks like it's coming from mesa
rajivr has quit [Quit: Connection closed for inactivity]
<V> I mean, what's stopping someone from writing a Wayland compositor for not-Linux? Windows is implementing one, NetBSD has work to support it, and I imagine the other BSDs will be working on doing so in the future
rajivr has joined #nixos-dev
* V . o O ( and this is why we need USE flags )
supersandro2000 has quit [Quit: Ping timeout (120 seconds)]
supersandro2000 has joined #nixos-dev
srk has quit [Remote host closed the connection]
srk has joined #nixos-dev
pmy has quit [Quit: WeeChat 3.0.1]
rj has quit [Remote host closed the connection]
rj has joined #nixos-dev
<siraben> qyliss: testing
<siraben> wow SDL2 has such a big build graph
<siraben> qyliss: a dependency of SDl2 fails to build
<siraben> more and more I feel like we should have some sort of cross-compilation team
<siraben> there's a lot of potential and good infra in Nixpkgs but it's often inaccessible to newcomers (long build times, poor documentation, cross-compilation failing)
<siraben> For instance myself running on a MacBook Pro with limited time am only able to test cross-compilation for relatively small packages. Anything GUI related is usually out because of huge build graphs
cole-h has joined #nixos-dev
<symphorien[m]> I suspect many gui related things will not compile
<symphorien[m]> x libs don't cross compile to musl for example
saschagrunert has joined #nixos-dev
<siraben> how should I cross-compile a shell script? When I run patchShebangs it points to the buildPlatform bash not the hostPlatform bash
<symphorien[m]> patchShebangs takes a flag to indicate that
pmy has joined #nixos-dev
pmy has quit [Read error: Connection reset by peer]
pmy has joined #nixos-dev
<siraben> symphorien: it didn't seem to change `#!/usr/bin/env bash`
<siraben> I did `patchShebangs --host script`
<symphorien[m]> Add bash to nativeBuildInputs?
<siraben> do you mean buildInputs? (the script needs to run on the host)
<siraben> ok that seems to have worked
<symphorien[m]> Hum I'm a bit fuzzy on the host terminology...
<symphorien[m]> Glad you worked it out
<siraben> #113373 :D
<{^_^}> (by siraben, 31 seconds ago, open): neofetch: fix cross-compilation
<siraben> having at least some cross-compilation cache on hydra is awesome
<siraben> s/is/has been/
<symphorien[m]> <siraben "#113373 :D"> I think it makes sense to add strictDeps=true on cross related PRs
<siraben> to prevent regressions?
<symphorien[m]> Otherwise someone could remove the bash in buildinputs incorrectly thinking it serves no purpose
<siraben> hmm yes, then eventually we can make strictDeps=true to be the default and remove those statements
<siraben> done
pingiun has quit [Quit: Bye!]
pingiun has joined #nixos-dev
<supersandro2000> siraben: If you have some time you can try to collect some common error patterns with me and I try to create a rule out of it which then can be implemented in hammering or my little script
<supersandro2000> also if you have something that takes a bit longer I can try building in on a beefier machine and fix some common errors for some core GUI things
<siraben> supersandro2000: sure thing
<siraben> So for CC I've seen CC=cc, CC=gcc, CC=g++, CC=c++
<siraben> I think `CC\s=(cc|gcc|g\+\+|c\+\+)` might be a better regex
<siraben> er, `\s=\s` even
<supersandro2000> I wrote an ripgrep command for that earlier and did not think about regex...
<supersandro2000> also would it be a smart idea to grep Makefiles for such patterns or do you think it would yield many false positive resultsß
<supersandro2000> ?
<siraben> yeah I used ripgrep for that
<siraben> I think grepping .nix files would be a great start, not sure about non-Nix files though
<supersandro2000> greping nix files is in the works for my review script and hammering has an issue about it
<supersandro2000> > ${buildPackages.pkg-config}/bin/${buildPackages.pkg-config.targetPrefix}pkg-config
<{^_^}> error: syntax error, unexpected DOLLAR_CURLY, at (string):475:1
<supersandro2000> there is an pkg-config with a prefix?
<symphorien[m]> yes
<supersandro2000> grex CC=cc CC=gcc CC=g++ CC=c++
<supersandro2000> ^CC=(?:c(?:\+\+|c)|g(?:\+\+|cc))$
<siraben> man perl and SDL failing to cross-compile are really make large swaths of Nixpkgs un-crossbuildable
<siraben> s/make/making
<supersandro2000> also I can recommend grex for writing regex from examples
<symphorien[m]> also in a $PKG_CONFIG env var
<supersandro2000> siraben: pkgCross to which platform?
<supersandro2000> musl probably not but like aarch64 or what would you like?
<symphorien[m]> well grex seems to obfuscate regex very efficiently
<siraben> `nix build .#pkgsCross.raspberryPi.SDL2`
<supersandro2000> flakes...
<supersandro2000> symphorien[m]: it makes short regex, not readable ones
<siraben> supersandro2000: not a fan of flakes? hehe
<siraben> nix-build . -A pkgsCross.raspberryPi.SDL2
<symphorien[m]> I think aarch64-multiplatform has the most cached toolchain
<supersandro2000> siraben: does not work with nix and home-manager
<siraben> yeah I'm not even using flakes to manage home-manager yet
<stigo> siraben: does perl fail to cross compile?
<siraben> stigo: yes
<siraben> stigo: fails with something like this:
<siraben> patchPhase
<stigo> what commit?
<siraben> stigo: master
<siraben> supersandro2000: I think if gcc is found in nativeBuildInputs nixpkgs-hammering should recommend to use `depsBuildBuild = [ ];` instead
<siraben> I guess it depends
<siraben> you'd need depsBuildBuild if you want a build β†’ build gcc available at build time
<supersandro2000> depsBuildBuild is a confusing name
<symphorien[m]> it sounds clearer than nativeBuildInputs
rj has quit [Remote host closed the connection]
<supersandro2000> no..
<supersandro2000> native building platform
rj has joined #nixos-dev
<symphorien[m]> dependencies to compile things on the build machine to be executed on the build machine
<supersandro2000> depsBuildBuild is like I pressed the wrong key in vim and copied Build
<siraben> lol
<supersandro2000> why not depsBuildTarget?
<siraben> symphorien: yeah when put like that depsBuildBuild makes sense
<siraben> supersandro2000: there is depsBuildTarget
<supersandro2000> yeah but why gcc go to things compiled for the build host, not the target host
<siraben> sometimes packages produce binaries that are used at build time
<siraben> i retract that suggestion, it's not clear when gcc should be put into depsBuildBuild
<siraben> tbh we should rename nativeBuildInputs to depsBuildHost
<symphorien[m]> I think you can already do that
<symphorien[m]> nativeBuildInputs is just an alias iirc
ckauhaus has joined #nixos-dev
<supersandro2000> why is my remote builder always aborting the builds when I compile more than one at the time?
<siraben> can someone explain what propagatedBuildInputs is or propagated dependencies in general?
<adisbladis> siraben: Any package that depends on the propagated package also runs it's setup hooks
<supersandro2000> in python you can't check if a build time dependency is used at runtime
<supersandro2000> for objects and binaries we can check against what things they are linked
<symphorien[m]> when you put A in buildinputs of C, if A has B in propagatedBuildInputs, then B is automatically and implicitely added to A's buildInputs as well
<adisbladis> Some package ecosystems (like python) don't have automatic dependency tracking, and in those cases propagatedBuildInputs are the list of runtime deps
<adisbladis> ^ This is quite unfortunate
<adisbladis> I'd like a separation
<symphorien[m]> it's also required for C libs that musl always be linked with a dependency
<supersandro2000> talk to jonringer about that
<stigo> siraben:
<supersandro2000> I am not doing language or eco system design and refusing any bikeshedding about it
<stigo> siraben: pkgsCross.armv7l-hf-multiplatform.perl and pkgsCross.aarch64-multiplatform.perl build fine on master it seems.
<siraben> symphorien, adisbladis: I see
<stigo> siraben: how to reproduce? :)
<siraben> stigo: build `nix build nixpkgs#pkgsCross.raspberryPi.SDL2`
<siraben> armv6l
<siraben> build = x86_64-darwin here
<siraben> also OpenSSL fails to cross-build to raspberyPi
<{^_^}> #82924 (by ebkalderon, 47 weeks ago, open): Cross-compiling OpenSSL 1.1 is broken on most platforms
<stigo> thx
<supersandro2000> it seems that nix really dislikes ssh ControlMaster
<siraben> as a result most games do not cross-compile (games in nixpkgs are notorious for having incorrectly specific inputs for some reason)
<siraben> s/specific/specified
<adisbladis> siraben: Cross compilation in nixpkgs isn't _that_ old
<supersandro2000> probably because games are often not used by a lot of people
<siraben> right, it's a combination of age and not many people using them
<adisbladis> And since nixpkgs is dogfooding cross, I'd wager that's one of the last things you'd try to cross compile
<symphorien[m]> add to that that it's easy to introduce regressions without noticing
<siraben> what are some other package managers/distros that have good cross-compilation infra?
<siraben> hmm, yocto, maybe?
<adisbladis> siraben: Gentoo
cole-h has quit [Ping timeout: 260 seconds]
<siraben> stigo: failure on hydra
<supersandro2000> why is guile not multi threaded...
<supersandro2000> guile tests take a good amount of time
<siraben> TIL Nix manpages are generated by a Nix script
<adisbladis> siraben: Just wait until you see the chrome updater script
__monty__ has joined #nixos-dev
<supersandro2000> πŸ˜‚ what the actual
<sterni> siraben: only on nixUnstable though
<sterni> siraben: and they are still pretty bad because they convert markdown to man page which doesn't really work since markdown can't represent all things man page
<sterni> siraben: but if the tool author advises against it, it must be a good idea :p
<supersandro2000> guile takes forever to compile...
<V> if only we didn't have to recompile everything that doesn't depend on something that changed
<V> I spent a while working on improving the Nixpkgs guile ecosystem support, and holy hell those rebuilds got annoying
<V> plus guile is used in stdenv
<V> 😱
<supersandro2000> 45 min on 40 cores
<supersandro2000> siraben: SDL2 is now building for raspberrypi
<sterni> V: 😳😳😳 what for
<sterni> nixos already half guix lol
<supersandro2000> guile not guix
<V> sterni: which bit? why did I improve the support/why is guile used in stdenv/...
<adisbladis> V: This is something the CAS work aims to solve
<V> IIRC it's an autotools dep, might just be the gnumake support though
<sterni> V: why guile in stdenv
<sterni> supersandro2000: ikr, but still guile in stdenv is something i'd rather expect guix to have than nixos
<V> adisbladis: unfortunately CAS won't fix stdenv rebuilds since it does not discriminate on behaviour, just identity
<V> or a large class of mass rebuilds in general
<V> it's part of the solution, certainly
<adisbladis> Not claiming it's a silver bullet by any means :)
<V> I think it wouldn't solve issues when doing that, either, assuming it's due to the make dependency?
<V> Nix can't guarantee that make will behave the same if it's linked against a slightly different guile, which will still cause the stdenv mass rebuild
<V> Please correct me if I'm wrong
<qyliss> V: I've been thinking about looking into speeding up stdenv rebuilds
<supersandro2000> I would appreciate some general feedback/guidance with
<{^_^}> #113388 (by Emantor, 1 hour ago, open): python3Packages.pyserial: patches for RFC2217
orivej has joined #nixos-dev
<Emantor> cc worldofpeace ^ these are the patches for pyserial I was talking about 24 days ago. They are more or less a requirement for
<{^_^}> #110717 (by Emantor, 3 weeks ago, open): python3Packages.labgrid: init at 0.3.1
<siraben> supersandro2000: nice!
<siraben> yeah why is Guile a dep of SDL2 anyway
<supersandro2000> siraben: send you a few private messages with some other build failures
<supersandro2000> if you have another good idea to build let me know
<stigo> siraben: pkgsCross.raspberryPi.SDL2 builds fine for me on linux x64, need to look into why the patch fails to apply on darwin
ris has quit [Ping timeout: 264 seconds]
<supersandro2000> case insensitive fs?
<niksnut> guile is used in stdenv? where?
<V> niksnut: as a transitive dependency IIRC
<niksnut> I'm not seeing it
<niksnut> nix-store -q --tree $(nix-instantiate -A stdenv) | grep -i guile => nothing
<V> gnu make has guile support, might be that unless we're explicitly disabling it
<adisbladis> nix-store --query --requisites (nix-instantiate '<nixpkgs>' -A stdenv) | grep guile
<adisbladis> Returns nothing
<V> strange
<niksnut> yeah it's disabled in gnumake
<niksnut> regarding SDL2, it's because autogen depends on it
<niksnut> maybe that could be disabled too
<V> I swear modifying it resulted in a mass rebuild
<niksnut> well, it's the official scripting language of the GNU project...
<V> indeed
<adisbladis> I wouldn't be surprised to see it in the critical path for something else, but at least it's not in stdenv
<V> might have been something in NixOS
<siraben> I meant that SDL2 depends on Guile
<siraben> not stdenv
<siraben> but yeah
ris has joined #nixos-dev
<domenkozar[m]> niksnut: how could I pass arguments to a flake without triggering the dirty bit?
<siraben> how can I make this executable use the build β†’ build compiler?
<siraben> when cross-compiling, it builds using the cross-compiler then tries to run the executable which doesn't work
<V> niksnut: seems plausible it was just triggering a rebuild of most of NixOS via the autogen dependency; it's a transitive dependency of systemd [systemd -> libmicrohttpd -> gnutls -> autogen -> guile]
<V> which does beg the question why gnutls is being used in the default system, when it's repeatedly been shown to have unfortunate security issues
<V> TIL systemd contains a web server
<supersandro2000> a real webserver or nc?
tilpner_ has joined #nixos-dev
<V> supersandro2000: see the dep chain
tilpner has quit [Ping timeout: 264 seconds]
tilpner_ is now known as tilpner
<V> oh, I see. it's for the remote journal feature, and is only a dependency of that one service
<V> if the systemd subpackages were split up better this wouldn't be a thing I think?
<siraben> how do I get the cross and native compiler names from cmake?
<V> I'm confused. you have a compiler that is being run via cmake (native); what else is there to worry about?
<siraben> it compiles the generate_tables binary using the buildHost compiler then runs the binary
<siraben> So I need to find a way to compile that using the buildBuild compiler
<V> if it immediately runs the binary, isn't that correct?
<V> since you want the binary to be for the host system
<siraben> V: but exec format error
<siraben> it's running the binary for host on build
tdeo has quit [Ping timeout: 260 seconds]
<V> oh is this using the gcc host = system that builds the cross compiler / build = system that runs the cross compiler / target = system that runs the final output terminology
<siraben> er, I'm building a program on darwin (build) to be run on raspberryPi (host)
<siraben> since it's not a compiler there is no target
<siraben> unless I'm mixing up terminology
<V> the build/host split should be irrelevant unless you're building a crosscompiler, all that matters is the system you're running the compiler on and the system it compiles to. sounds like you need a compiler that outputs a binary for the system it runs on
<V> "There are three system names that the build knows about: the machine you are building on (build), the machine that you are building for (host), and the machine that GCC will produce code for (target). When you configure GCC, you specify these with --build=, --host=, and --target=. "
<siraben> and I do have that cross-compiler, the problem is that the build script runs one of the output binaries during building
<V> what I said above was incorrect; I flipped build/host
<siraben> which results in an exec format error
<siraben> Ok, yeah.
tdeo has joined #nixos-dev
<V> so you want a compiler where host = target = the system you're running nix on
<siraben> Why is target involved?
<V> in normal usage, host and target are the only two relevant fields
<V> you only care where you're running the compiler and which system it compiles for
<siraben> to be concrete, there is a generate_tables.c file that gets compiled (using build β†’ build) and run to generate another C file that gets compiled (using build β†’ host) to the final binary.
<siraben> the problem is that the first compilation step is build β†’ host
<siraben> so I need to do someting like, `... use_build_build() <build and run generate_tables> use_build_host() ...` in cmake
<V> 'build' is an extra concept in gcc that lets you build a cross-compiler that runs on another system (host) that produces binaries for another system (target). hence, as soon as you've built this compiler, the build system is no longer relevant as all you have is a gcc that you want to use to build stuff
<V> AFAIK it's only relevant in that one place
<siraben> oh really? ok I'm confused
<siraben> hm
<symphorien[m]> I think in the nix terminology, it's target which is the extra one
<qyliss> yeah
<qyliss> the manual covers this quite extensively
<V> ...
<siraben> I'm aware of what build/host/target does but target only really matters when build a cross-compler
<symphorien[m]> ^
<niksnut> domenkozar[m]: there is currently no way to pass arguments to a flake
<V> okay, wasn't aware that nixpkgs uses conflicting terminology
<qyliss> > he third attribute, target, is relevant only for certain specific compilers and build tools.
<{^_^}> error: syntax error, unexpected ',', expecting ')', at (string):475:19
<siraben> matthewbauer: I learned this definition from matthewbauer's presentation (slide 10)
<siraben> which is borrowed from autoconf
<siraben> niksnut: but you can override inputs right?
<siraben> from my history: `nix build --override-input nixpkgs nixpkgs --no-write-lock-file nickel`
<niksnut> yes but that also dirties the flake
<V> or I guess it's not technically conflicting depending on how you look at it
<symphorien[m]> it's about the same, but with an offset, if I understand correctly
<worldofpeace> Emantor: looking
<siraben> the manual isn't very illuminating when it comes to the "specifying dependencies" section: #89389
<{^_^}> (by mpickering, 37 weeks ago, open): Documentation of "Specifying Dependencies" could be better
<domenkozar[m]> niksnut: is there a good way to chain flakes within a repo?
<worldofpeace> Emantor: merged pyserial ✨
<Emantor> worldofpeace: thanks!
<eyJhb> pyserial! :o Are you doing Arduino stuff Emantor ?
<siraben> domenkozar: so you have multiple flakes in a repo and want to call one from another?
<Emantor> eyJhb: No, I am the maintainer of a remote hardware control library, we use pyserial with RFC2217 and ser2net to make serial ports network transparent.
<eyJhb> Ohh cool Emantor :D I have only used it in regards to doing some Arduino stuff, but granted, and don't do as much with serial as I would want to
<Emantor> I am an embedded Linux developer, we can't work without serial ports :-)
<eyJhb> Emantor: pinging you in #nixos-chat, sounds interesting :D
<domenkozar[m]> siraben: well what I want is to change inputs only
<domenkozar[m]> but it seems like having two flakes is the closest to that
<niksnut> domenkozar[m]: I think the easiest solution is to add another flake output (could be adding it to hydraJobs, but could also be another flake output)
<domenkozar[m]> ah yeah!
<niksnut> that does the same thing as hydraJobs.installerScript but with fewer systems
<domenkozar[m]> good point :)
ris has quit [Ping timeout: 260 seconds]
ris has joined #nixos-dev
<qyliss> I've seen the button so I assume so
<supersandro2000> I think that was refined github
<supersandro2000> at least for me
<qyliss> I don't use that
<niksnut> supersandro2000: for what repo?
mkaito has joined #nixos-dev
mkaito has quit [Changing host]
mkaito has joined #nixos-dev
pie_ has joined #nixos-dev
jonringer has joined #nixos-dev
<supersandro2000> nixpkgs mainly
<supersandro2000> If I push a formatting change which I know won't affect eval it would be nice if you could enable this to merge it with green ofborg
<gchristensen> not so infrequently
* lukegb likes for 'Revert "Revert "Revert "Revert "Revert "(dl)Close native libraries on unload"""""'
<siraben> What's going on with ? I can't seem to find the hashes the satisfy both macOS and Nixos!
<{^_^}> #113402 (by siraben, 1 hour ago, open): zasm: 4.2.6 -> 4.4.7, fix darwin build and cross-compilation
<gchristensen> maybe it is unicode normalization
<siraben> Should I have a conditional hash then? `sha256 = if stdenv.isDarwin ...`
<siraben> strange
<siraben> I could fetch tarball maybe
<gchristensen> which one is busted?
<gchristensen> siraben: which source file is broken?
<siraben> github:megatokio/zasm is
<siraben> there's symlinks in the git repo too
<siraben> so maybe that's the culprit
<siraben> Ok just pushed a changed where src is fetched with fetchrul
<siraben> fetchurl
<gchristensen> fetchFromGitHub uses the tarball already no?
<sterni> siraben: don't use fetchurl github changes the tarball generation from time to time
<siraben> bah
<sterni> siraben: fetchFromGitHub already uses fetchzip to download the tarball generated by github and does a nar hash on the output
<siraben> hmm
<sterni> yeah it's pretty annoying and mostly why fetchFromGitHub exists in the first place
<sterni> siraben: maybe do a find -type l | xargs rm if you don't need the symlinks in postFetch?
<qyliss> i'd be surprised if the symlinks were the problem
<siraben> sterni: but the problem is that the original tarball already has the wrong hash
<siraben> so changing the source after the fact wouldn't work
<qyliss> siraben: you can postFetch inside fetchers
<qyliss> before the hash is calculated
<siraben> hmm, do you think removing the symlinks really will fix the issue?
<gchristensen> I doubt it
<siraben> (the repo author shouldn't be using symlinks as a sub for submodules anyway but, here we are)
<gchristensen> this project, like all that hash differently on macos, has funny filenames
<gchristensen> ./Test/SDCC/s/--nostdinc/-I:pub:Develop:Projects:zasm-4.0:sdcc:include:/--reserve-regs-iy/--max-allocs-per-node/100/pub:Develop:Projects:zasm-4.0:Test:SDCC:library:_put_char_to_stdout.c.s
<gchristensen> ./Test/ZXSP/original/s/--nostdinc/-I:home:kio:pub:Develop:Projects:zasm:sdcc:include:/home:kio:pub:Develop:Projects:zasm:Test:ZXSP:main.c.s
<siraben> maybe remove the test directory entirely
<sterni> should be possible to do that in postFetch
<siraben> lol those look like local paths?
<sterni> for fetchzip the hash is calculated after that
<andi-> how do you run tests if you remove the tests? :)
<sterni> gchristensen: why exactly do they hash differently then? this is relatively weird behavior for β€œwe have a standard way of hashing directories, the NAR hash”
<siraben> andi-: simple, don't run the tests! /s
<siraben> doCheck is false by default anyway
<gchristensen> sterni: macos has different ideas of filenames
<gchristensen> for example it has an case-insensitive filesystem, it does unicode normalization, and other things that most linux filesystems don't.
<gchristensen> NARs follow the unix idea that paths are a string of bytes, macos doesn't
<siraben> I did rm -rf Test in postFetch but the hash is still the same?
<siraben> perturbed the hash accordingly to invalidate it
<siraben> error: hash mismatch in fixed-output derivation '/nix/store/5wkhiwsg4sxizmwxh1i7jxyp5dxqkmiz-source.drv':
<siraben> specified: sha256-XBf852JYply00QDyPKTFw+THaL8c7Cm5pFhTVbeh2FE=
<siraben> got: sha256-XBf852JYplyw3QDyPKTFw+THaL8c7Cm5pFhTVbeh2FE=
<sterni> gchristensen: uhh yeah that's not nice and probably can't be fixed without breaking linux compat
<gchristensen> I dunno we could probably mandate paths are valid and normalized utf8 without breaking too much stuff
<gchristensen> since we target macos too, we might as well since we're effectively forced to do it anyway
<sterni> siraben: extraPostFetch for fetchzip I think
<sterni> gchristensen: yeah but forcing case insensitivity might be a problem
<sterni> gchristensen: but I guess we could have a switch for the old behavior then at the expense of darwin support?!
<siraben> earlier today I ran into substitueInPlace having different behavior on macOS and linux because makefile != Makefile but on macOS it is
<gchristensen> Nix actually secretly deals with it with a case insensitivity hack
<siraben> sterni: !!
<clever> darwin also lets you enable case sensitivity
<clever> its in the stackage2nix docs
<rmcgibbo[m]> Does macos have fuse? Clearly /nix/store just needs to be running on NixFS rather an AppleFS
<siraben> clever: is that recommended? I think I recall something about partitioning a FS like that being weird
<siraben> sterni: even in extraPostFetch it seems to be the same hash
<siraben> error: hash mismatch in fixed-output derivation '/nix/store/6m7jl7v4anc615z80k7fij6gfgzpj4l7-source.drv':
<clever> siraben: ive not actually tried doing it
<siraben> specified: sha256-XBf852JYplyw3QDyPKTFw+00aL8c7Cm5pFhTVbeh2FE=
<siraben> got: sha256-XBf852JYplyw3QDyPKTFw+THaL8c7Cm5pFhTVbeh2FE=
<clever> whitespace?
<clever> oh, there, 0/T
<siraben> argh
<gchristensen> siraben: it is *not* recommended
<gchristensen> also case sensitivity is not the issue here either
<siraben> gchristensen: right. as I recall.
<gchristensen> maybe someone on a mac could tar up the created store path and upload it for comparison
<siraben> i could do it
<gchristensen> or even better, the nar :)
<siraben> How do I eval `with import <nixpkgs> { }; src = fetchFromGitHub { owner = "megatokio"; repo = "zasm"; rev = "4.4.7"; sha256 = "sha256-XBf852JYplyw3QDyPKTFw+THaL8c7Cm5pFhTVbeh2FE="; };` ?
<siraben> trying `nix eval --impure --expr ...`
<clever> siraben: nix-instantiate -E is my go-to
<qyliss> that's not valid syntax
<qyliss> but i imagine nix told you that already
<sterni> siraben: try rm -r $out/Test
<siraben> oops copy/paste error: `nix eval --impure --expr 'with import ./. { }; fetchFromGitHub { owner = "megatokio"; repo = "zasm"; rev = "4.4.7"; sha256 = "sha256-XBf852JYplyw3QDyPKTFw+THaL8c7Cm5pFhTVbeh2FE="; }`
<siraben> there
<sterni> siraben: you are not cd'd into $out in postFetch
<siraben> ok I have the .drv
<qyliss> siraben: we don't need the drv, we need the store path
<siraben> /nix/store/hcgp47l5x2zcsbyvd2bw9a0ishf09fc2-source.drv ?
<qyliss> no, the store path of the built drv
<sterni> siraben: sri on NixOS is sha256-Zbno8kmzss1H2FjwzHB4U7UXxa6oDfsPV80MVVFfM68= with Test deleted
<qyliss> because the problem here is that the drv evaluates to different things on different systems
<siraben> /nix/store/0dn4khwiycs6izfdxxfywjq1lk18904x-source
<siraben> sha256-XBf852JYplyw3QDyPKTFw+THaL8c7Cm5pFhTVbeh2FE=
<sterni> siraben: you need the extraPostFetch *in* the fetchFromGitHub attrset
<sterni> siraben: ^
<siraben> dangit! thanks.
<gchristensen> realizes to different things*
<siraben> I can replicate. That works.
<sterni> also on darwin?
<gchristensen> I'd still like to get the NAR from darwin which is producing the different hash
<siraben> Yes, on Darwin.
<siraben> gchristensen: how do I do that?
<gchristensen> what does nix-build /nix/store/pnrhmr65hbrckdybzfay81cc2rf7whch-source.drv get you?
<gchristensen> (note nix-build is exactly the command I want you to run)
srk has quit [Remote host closed the connection]
<siraben> error: hash mismatch in fixed-output derivation '/nix/store/pnrhmr65hbrckdybzfay81cc2rf7whch-source.drv':
<siraben> specified: sha256-OfbehpAyS77GslUNVzp9ZbXYEjcMi06yO3VLK0xP/2Y=
<siraben> got: sha256-XBf852JYplyw3QDyPKTFw+THaL8c7Cm5pFhTVbeh2FE=
srk has joined #nixos-dev
<gchristensen> ack, stepping back a second.
<gchristensen> I want you to do this and get that error, then update th ederivation to have this corrected hash, do the build again, and then we'll export the /nix/store/... -source store path
<siraben> done and done.
<gchristensen> nix-store --export /nix/store/...-source
<gchristensen> uh
<gchristensen> > meh.nar
<siraben> ack my terminal went crazy
<gchristensen> yeah sorry it spews to stdout
<gchristensen> anyone able to get the nar from linux?
<qyliss> on it
<gchristensen> then I suppose diffoscope
<siraben> I'd be curious to know what the differences are as well
<qyliss> diffoscoping
<qyliss> it's case sensitivity
<qyliss> -Test/Z80/ti83plus/Hello.asm
<qyliss> there are files in that directory called hello.asm and Hello.asm
<gchristensen> cool
<gchristensen> maybe we could open an issue upstream and invite them to change it
<siraben> nice
<qyliss> sounds like a good way to go
<gchristensen> *sigh* the reason I didn't catch the difference before is ... zfs create tank/local/junk/utf8 -o utf8only=on -o normalization=formD -o casesensitivity=sensitive -o mountpoint=/junk-roots/case
<qyliss> oh no
<qyliss> why would you do that to yourself
<gchristensen> hrm?
<gchristensen> the issue being I forgot to say *in*sensitive
<qyliss> ohhhhh
<qyliss> whoops
<gchristensen> how about an ofborg feature which fetches all the changed sources on variously tortured filesystem configurations
<gchristensen> I mean actually that'd be an interesting thing for r13y to do
<qyliss> that would be very cool
lechner has left #nixos-dev [#nixos-dev]
<rmcgibbo[m]> anyone understand the ofborg errors here? it's related to... ripgrep?
<{^_^}> #113412 (by danieldk, 1 hour ago, open): python3Packages.cvxpy: 1.1.8 -> 1.1.10
<gchristensen> rmcgibbo[m]: did you see the Meta Check "Details" link?
<gchristensen> erm, check-meta
<gchristensen> rmcgibbo[m]: it looks like a push-to-master broke evaluation:
rajivr has quit [Quit: Connection closed for inactivity]
<gchristensen> maybe somebody could either revert that commit or add the field to the meta validation
saschagrunert has quit [Remote host closed the connection]
<rmcgibbo[m]> I see.
Mic92 has quit [Quit: WeeChat 3.0]
Mic92 has joined #nixos-dev
<gchristensen> anyone available to address this fixup before a ton of PRs fail?
<qyliss> gchristensen: reverted
<gchristensen> thanks
mcint has joined #nixos-dev
cole-h has joined #nixos-dev
<supersandro2000> siraben: if the hashes are different on darwin and linux it should be fixed via postfetch so that they get the same hash
<supersandro2000> How do I get the host shell for cross compiling?
<supersandro2000> the configure script has a /bin/bash and replacing that with does not work
<samueldr> buildPackages.runtimeShell
<samueldr> buildPackages is "pkgs, but those that would get magically used for nativeBuildInputs"
<supersandro2000> ah thanks
<supersandro2000> need to remember that
<supersandro2000> grub2 cross compiling here we come!
<samueldr> neat
Jackneill has quit [Ping timeout: 240 seconds]
<symphorien[m]> <supersandro2000 "grub2 cross compiling here we co"> is already there
<{^_^}> #112857 (by r-burns, 5 days ago, open): grub2: fix cross-compilation
<symphorien[m]> it's blocked on
<{^_^}> #112961 (by r-burns, 4 days ago, open): nixosTests.installer: fix non-zfs grub tests
<symphorien[m]> I won't have time to review it before some time, so if someone is interested...
<supersandro2000> well...
<supersandro2000> at least I got to know that 2 minutes in
Jackneill has joined #nixos-dev
Jackneill has quit [Ping timeout: 246 seconds]
<supersandro2000> If someone familiar with the installer could take a look at I would be very happy to get grub2 cross compiling merged.
<{^_^}> #112961 (by r-burns, 4 days ago, open): nixosTests.installer: fix non-zfs grub tests
Jackneill has joined #nixos-dev
<jtojnar> which broke ripgrep.passthru.tests
<supersandro2000> Was that silently removed?
<cole-h> "silently" in that it was pushed to master
<cole-h> sans PR
<sterni> also this could have been moved into a derivation in passthru.tests
<supersandro2000> not sure if github links links to it
<sterni> pretty lazy change
<sterni> supersandro2000: the last PR by eelco was last month
<supersandro2000> I mean he could have linked it
<sterni> huh?
<supersandro2000> linked it in the PR it was introduced so that people know they they need to change it
<supersandro2000> now it is just: I could have sworn I've added a test but it is gone
<supersandro2000> if I don't check git log right away
<sterni> maybe tests shouldn't be removed without replacement or discussion
<supersandro2000> I think I can now also build staging if anyone needs it.
<supersandro2000> 40 cores make big difference
tdeo has quit [Remote host closed the connection]
tdeo has joined #nixos-dev
ckauhaus has quit [Quit: WeeChat 2.7.1]
pmy has quit [Ping timeout: 272 seconds]
pmy has joined #nixos-dev
__monty__ has quit [Quit: leaving]
<supersandro2000> Is there a way I can easily know what blocks staging-next?