<supersandro2000>
Has someone tried to write a script/function/program that tries to clear all {,native,propogated}buildInputs etc and check if the program still compiles or the output is different?
<samueldr>
propagated wouldn't apply here, as the build may succeed, but things using that derivation could fail
vdemeester has quit [Ping timeout: 260 seconds]
vdemeester has joined #nixos-dev
jonringer has quit [Remote host closed the connection]
<jtojnar>
rmcgibbo: I think we will want to combine checking logs with other types of checks too so they should get passed attrpath and file path too
<jtojnar>
also porting the tool to rust first will allow us to switch to pull model and thus get rid of the need for distinguishing check types altogether
<jtojnar>
i.e. we would extend the Files object with attributes and logs that could be requested by the checks lazily
<rmcgibbo[m]>
Jan Tojnar: sure. i'm definitely less comfortable in rust -- i'm not sure if re-writing grep in rust for the log checks is better than having them in external binaries that can use shell, but i'm fine either way.
<jtojnar>
worst case scenario we can shell out
<jtojnar>
but IIRC ripgrep is a library
<rmcgibbo[m]>
my counter-argument is that it feels like overengineering for something that is only 3 lines worth of logic, but i won't push it.
<jtojnar>
we can definitely go the simple route first, and RiiR when as more complex logic is needed for the checks
<rmcgibbo[m]>
Obviously your call -- I'm fine with the RewriteItInRust route.
<V>
I mean, what's stopping someone from writing a Wayland compositor for not-Linux? Windows is implementing one, NetBSD has work to support it, and I imagine the other BSDs will be working on doing so in the future
rajivr has joined #nixos-dev
* V
. o O ( and this is why we need USE flags )
supersandro2000 has quit [Quit: Ping timeout (120 seconds)]
supersandro2000 has joined #nixos-dev
srk has quit [Remote host closed the connection]
srk has joined #nixos-dev
pmy has quit [Quit: WeeChat 3.0.1]
rj has quit [Remote host closed the connection]
rj has joined #nixos-dev
<siraben>
qyliss: testing
<siraben>
wow SDL2 has such a big build graph
<siraben>
qyliss: a dependency of SDl2 fails to build http://ix.io/2PHy
<siraben>
more and more I feel like we should have some sort of cross-compilation team
<siraben>
there's a lot of potential and good infra in Nixpkgs but it's often inaccessible to newcomers (long build times, poor documentation, cross-compilation failing)
<siraben>
For instance myself running on a MacBook Pro with limited time am only able to test cross-compilation for relatively small packages. Anything GUI related is usually out because of huge build graphs
cole-h has joined #nixos-dev
<symphorien[m]>
I suspect many gui related things will not compile
<symphorien[m]>
x libs don't cross compile to musl for example
saschagrunert has joined #nixos-dev
<siraben>
how should I cross-compile a shell script? When I run patchShebangs it points to the buildPlatform bash not the hostPlatform bash
<symphorien[m]>
patchShebangs takes a flag to indicate that
pmy has joined #nixos-dev
pmy has quit [Read error: Connection reset by peer]
pmy has joined #nixos-dev
<siraben>
symphorien: it didn't seem to change `#!/usr/bin/env bash`
<siraben>
I did `patchShebangs --host script`
<symphorien[m]>
Add bash to nativeBuildInputs?
<siraben>
do you mean buildInputs? (the script needs to run on the host)
<siraben>
ok that seems to have worked
<symphorien[m]>
Hum I'm a bit fuzzy on the host terminology...
<siraben>
having at least some cross-compilation cache on hydra is awesome
<siraben>
s/is/has been/
<symphorien[m]>
<siraben "#113373 :D"> I think it makes sense to add strictDeps=true on cross related PRs
<siraben>
to prevent regressions?
<symphorien[m]>
Otherwise someone could remove the bash in buildinputs incorrectly thinking it serves no purpose
<siraben>
hmm yes, then eventually we can make strictDeps=true to be the default and remove those statements
<siraben>
done
pingiun has quit [Quit: Bye!]
pingiun has joined #nixos-dev
<supersandro2000>
siraben: If you have some time you can try to collect some common error patterns with me and I try to create a rule out of it which then can be implemented in hammering or my little script
<supersandro2000>
also if you have something that takes a bit longer I can try building in on a beefier machine and fix some common errors for some core GUI things
<siraben>
supersandro2000: sure thing
<siraben>
So for CC I've seen CC=cc, CC=gcc, CC=g++, CC=c++
<siraben>
I think `CC\s=(cc|gcc|g\+\+|c\+\+)` might be a better regex
<siraben>
er, `\s=\s` even
<supersandro2000>
I wrote an ripgrep command for that earlier and did not think about regex...
<supersandro2000>
also would it be a smart idea to grep Makefiles for such patterns or do you think it would yield many false positive resultsß
<siraben>
supersandro2000: I think if gcc is found in nativeBuildInputs nixpkgs-hammering should recommend to use `depsBuildBuild = [ buildPackages.stdenv.cc ];` instead
<siraben>
I guess it depends
<siraben>
you'd need depsBuildBuild if you want a build → build gcc available at build time
<supersandro2000>
depsBuildBuild is a confusing name
<symphorien[m]>
it sounds clearer than nativeBuildInputs
rj has quit [Remote host closed the connection]
<supersandro2000>
no..
<supersandro2000>
native building platform
rj has joined #nixos-dev
<symphorien[m]>
dependencies to compile things on the build machine to be executed on the build machine
<supersandro2000>
depsBuildBuild is like I pressed the wrong key in vim and copied Build
<siraben>
lol
<supersandro2000>
why not depsBuildTarget?
<siraben>
symphorien: yeah when put like that depsBuildBuild makes sense
<siraben>
supersandro2000: there is depsBuildTarget
<supersandro2000>
yeah but why gcc go to things compiled for the build host, not the target host
<siraben>
sometimes packages produce binaries that are used at build time
<siraben>
i retract that suggestion, it's not clear when gcc should be put into depsBuildBuild
<siraben>
tbh we should rename nativeBuildInputs to depsBuildHost
<symphorien[m]>
I think you can already do that
<symphorien[m]>
nativeBuildInputs is just an alias iirc
ckauhaus has joined #nixos-dev
<supersandro2000>
why is my remote builder always aborting the builds when I compile more than one at the time?
<siraben>
can someone explain what propagatedBuildInputs is or propagated dependencies in general?
<adisbladis>
siraben: Any package that depends on the propagated package also runs it's setup hooks
<supersandro2000>
in python you can't check if a build time dependency is used at runtime
<supersandro2000>
for objects and binaries we can check against what things they are linked
<symphorien[m]>
when you put A in buildinputs of C, if A has B in propagatedBuildInputs, then B is automatically and implicitely added to A's buildInputs as well
<adisbladis>
Some package ecosystems (like python) don't have automatic dependency tracking, and in those cases propagatedBuildInputs are the list of runtime deps
<adisbladis>
^ This is quite unfortunate
<adisbladis>
I'd like a separation
<symphorien[m]>
it's also required for C libs that musl always be linked with a dependency
<supersandro2000>
talk to jonringer about that
<stigo>
siraben:
<supersandro2000>
I am not doing language or eco system design and refusing any bikeshedding about it
<stigo>
siraben: pkgsCross.armv7l-hf-multiplatform.perl and pkgsCross.aarch64-multiplatform.perl build fine on master it seems.
<sterni>
siraben: and they are still pretty bad because they convert markdown to man page which doesn't really work since markdown can't represent all things man page
<sterni>
siraben: but if the tool author advises against it, it must be a good idea :p
<supersandro2000>
guile takes forever to compile...
<V>
if only we didn't have to recompile everything that doesn't depend on something that changed
<V>
I spent a while working on improving the Nixpkgs guile ecosystem support, and holy hell those rebuilds got annoying
<V>
plus guile is used in stdenv
<V>
😱
<supersandro2000>
45 min on 40 cores
<supersandro2000>
siraben: SDL2 is now building for raspberrypi
<sterni>
V: 😳😳😳 what for
<sterni>
nixos already half guix lol
<supersandro2000>
guile not guix
<V>
sterni: which bit? why did I improve the support/why is guile used in stdenv/...
<adisbladis>
V: This is something the CAS work aims to solve
<V>
IIRC it's an autotools dep, might just be the gnumake support though
<sterni>
V: why guile in stdenv
<sterni>
supersandro2000: ikr, but still guile in stdenv is something i'd rather expect guix to have than nixos
<V>
adisbladis: unfortunately CAS won't fix stdenv rebuilds since it does not discriminate on behaviour, just identity
<V>
or a large class of mass rebuilds in general
<V>
it's part of the solution, certainly
<adisbladis>
Not claiming it's a silver bullet by any means :)
<V>
I think it wouldn't solve issues when doing that, either, assuming it's due to the make dependency?
<V>
Nix can't guarantee that make will behave the same if it's linked against a slightly different guile, which will still cause the stdenv mass rebuild
<V>
Please correct me if I'm wrong
<qyliss>
V: I've been thinking about looking into speeding up stdenv rebuilds
<Emantor>
cc worldofpeace ^ these are the patches for pyserial I was talking about 24 days ago. They are more or less a requirement for https://github.com/NixOS/nixpkgs/pull/110717.
<V>
niksnut: seems plausible it was just triggering a rebuild of most of NixOS via the autogen dependency; it's a transitive dependency of systemd [systemd -> libmicrohttpd -> gnutls -> autogen -> guile]
<V>
which does beg the question why gnutls is being used in the default system, when it's repeatedly been shown to have unfortunate security issues
<V>
TIL systemd contains a web server
<supersandro2000>
a real webserver or nc?
tilpner_ has joined #nixos-dev
<V>
supersandro2000: see the dep chain
tilpner has quit [Ping timeout: 264 seconds]
tilpner_ is now known as tilpner
<V>
oh, I see. it's for the remote journal feature, and is only a dependency of that one service
<V>
if the systemd subpackages were split up better this wouldn't be a thing I think?
<siraben>
how do I get the cross and native compiler names from cmake?
<V>
I'm confused. you have a compiler that is being run via cmake (native); what else is there to worry about?
<siraben>
it compiles the generate_tables binary using the buildHost compiler then runs the binary
<siraben>
So I need to find a way to compile that using the buildBuild compiler
<V>
if it immediately runs the binary, isn't that correct?
<V>
since you want the binary to be for the host system
<siraben>
V: but exec format error
<siraben>
it's running the binary for host on build
tdeo has quit [Ping timeout: 260 seconds]
<V>
oh is this using the gcc host = system that builds the cross compiler / build = system that runs the cross compiler / target = system that runs the final output terminology
<siraben>
er, I'm building a program on darwin (build) to be run on raspberryPi (host)
<siraben>
since it's not a compiler there is no target
<siraben>
unless I'm mixing up terminology
<V>
the build/host split should be irrelevant unless you're building a crosscompiler, all that matters is the system you're running the compiler on and the system it compiles to. sounds like you need a compiler that outputs a binary for the system it runs on
<V>
"There are three system names that the build knows about: the machine you are building on (build), the machine that you are building for (host), and the machine that GCC will produce code for (target). When you configure GCC, you specify these with --build=, --host=, and --target=. "
<siraben>
and I do have that cross-compiler, the problem is that the build script runs one of the output binaries during building
<V>
what I said above was incorrect; I flipped build/host
<siraben>
which results in an exec format error
<siraben>
Ok, yeah.
tdeo has joined #nixos-dev
<V>
so you want a compiler where host = target = the system you're running nix on
<siraben>
Why is target involved?
<V>
in normal usage, host and target are the only two relevant fields
<V>
you only care where you're running the compiler and which system it compiles for
<siraben>
to be concrete, there is a generate_tables.c file that gets compiled (using build → build) and run to generate another C file that gets compiled (using build → host) to the final binary.
<siraben>
the problem is that the first compilation step is build → host
<siraben>
so I need to do someting like, `... use_build_build() <build and run generate_tables> use_build_host() ...` in cmake
<V>
'build' is an extra concept in gcc that lets you build a cross-compiler that runs on another system (host) that produces binaries for another system (target). hence, as soon as you've built this compiler, the build system is no longer relevant as all you have is a gcc that you want to use to build stuff
<V>
AFAIK it's only relevant in that one place
<siraben>
oh really? ok I'm confused
<siraben>
hm
<symphorien[m]>
I think in the nix terminology, it's target which is the extra one
<qyliss>
yeah
<qyliss>
the manual covers this quite extensively
<V>
...
<siraben>
I'm aware of what build/host/target does but target only really matters when build a cross-compler
<symphorien[m]>
^
<niksnut>
domenkozar[m]: there is currently no way to pass arguments to a flake
<domenkozar[m]>
niksnut: is there a good way to chain flakes within a repo?
<worldofpeace>
Emantor: merged pyserial ✨
<Emantor>
worldofpeace: thanks!
<eyJhb>
pyserial! :o Are you doing Arduino stuff Emantor ?
<siraben>
domenkozar: so you have multiple flakes in a repo and want to call one from another?
<Emantor>
eyJhb: No, I am the maintainer of a remote hardware control library, we use pyserial with RFC2217 and ser2net to make serial ports network transparent.
<eyJhb>
Ohh cool Emantor :D I have only used it in regards to doing some Arduino stuff, but granted, and don't do as much with serial as I would want to
<Emantor>
I am an embedded Linux developer, we can't work without serial ports :-)
<eyJhb>
Emantor: pinging you in #nixos-chat, sounds interesting :D
<domenkozar[m]>
siraben: well what I want is to change inputs only
<domenkozar[m]>
but it seems like having two flakes is the closest to that
<niksnut>
domenkozar[m]: I think the easiest solution is to add another flake output (could be adding it to hydraJobs, but could also be another flake output)
<domenkozar[m]>
ah yeah!
<niksnut>
that does the same thing as hydraJobs.installerScript but with fewer systems
<siraben>
maybe remove the test directory entirely
<sterni>
should be possible to do that in postFetch
<siraben>
lol those look like local paths?
<sterni>
for fetchzip the hash is calculated after that
<andi->
how do you run tests if you remove the tests? :)
<sterni>
gchristensen: why exactly do they hash differently then? this is relatively weird behavior for “we have a standard way of hashing directories, the NAR hash”
<siraben>
andi-: simple, don't run the tests! /s
<siraben>
doCheck is false by default anyway
<gchristensen>
sterni: macos has different ideas of filenames
<gchristensen>
for example it has an case-insensitive filesystem, it does unicode normalization, and other things that most linux filesystems don't.
<gchristensen>
NARs follow the unix idea that paths are a string of bytes, macos doesn't
<siraben>
I did rm -rf Test in postFetch but the hash is still the same?
<siraben>
perturbed the hash accordingly to invalidate it
<siraben>
error: hash mismatch in fixed-output derivation '/nix/store/5wkhiwsg4sxizmwxh1i7jxyp5dxqkmiz-source.drv':
<gchristensen>
maybe we could open an issue upstream and invite them to change it
<siraben>
nice
<qyliss>
sounds like a good way to go
<gchristensen>
*sigh* the reason I didn't catch the difference before is ... zfs create tank/local/junk/utf8 -o utf8only=on -o normalization=formD -o casesensitivity=sensitive -o mountpoint=/junk-roots/case
<qyliss>
oh no
<qyliss>
why would you do that to yourself
<gchristensen>
hrm?
<gchristensen>
the issue being I forgot to say *in*sensitive
<qyliss>
ohhhhh
<qyliss>
whoops
<gchristensen>
how about an ofborg feature which fetches all the changed sources on variously tortured filesystem configurations
<gchristensen>
I mean actually that'd be an interesting thing for r13y to do
<symphorien[m]>
I won't have time to review it before some time, so if someone is interested...
<supersandro2000>
well...
<supersandro2000>
at least I got to know that 2 minutes in
Jackneill has joined #nixos-dev
Jackneill has quit [Ping timeout: 246 seconds]
<supersandro2000>
If someone familiar with the installer could take a look at https://github.com/NixOS/nixpkgs/pull/112961 I would be very happy to get grub2 cross compiling merged.