gchristensen changed the topic of #nixos-dev to: NixOS Development (#nixos for questions) | https://hydra.nixos.org/jobset/nixos/trunk-combined https://channels.nix.gsc.io/graph.html | 18.03 release managers: fpletz and vcunat | https://logs.nix.samueldr.com/nixos-dev
lassulus_ has joined #nixos-dev
lassulus has quit [Ping timeout: 240 seconds]
lassulus_ is now known as lassulus
Cale has quit [Remote host closed the connection]
orivej has joined #nixos-dev
<domenkozar> /nix/store/ckq71kkymh1ji2b44xn80wmr7fmi6wr5-clang-wrapper-5.0.2/bin/cc: line 183: /nix/store/bcl9zj60h52p47dy85s326mdrqx52417-clang-5.0.2/bin/clang: Argument list too long
<domenkozar> :'(
<domenkozar> haskell is 10% and 90% fighting limits :D
goibhniu has joined #nixos-dev
<LnL> yeah...
<domenkozar> LnL: is that a known issue?
<domenkozar> what's interesting is, why did my project compile with 18.03 and GHC 8.2?
<domenkozar> oh this is the old linker limit
<srhb> domenkozar: The only change I'm aware of that drastically changed this was during 17.09 cycle, I think
<domenkozar> just a different error :)
<LnL> kind of, something recent increased the amount of flags significantly
<srhb> Really? I thought most of the unintentional cross blowup was fixed recently, with strictDeps and such
<domenkozar> wish I had a macos to debug this
<srhb> domenkozar: can't you just lower ARG_MAX to emulate this specific issue?
<srhb> Oh, it wasn't merged yet...
<srhb> 41420 was what I was talking about.
<domenkozar> the breakage seems to have occurred sometime after commit 91ca88b and before commit d91c701
<domenkozar> I really think we should just revert whatever broke this
<LnL> that looks like a pretty big diff, you can probably get a better delta from hydra
<domenkozar> yikes :D
<srhb> I'm really not certain that the breakage occurred between those commits. As far as I've seen, the limit was hit during before 18.03 for a lot of things
<srhb> But since it's just a limit, and dependencies can change that, it's hard to tell
<LnL> that's like half
<domenkozar> yeah, but it got lower
* srhb nods
<domenkozar> I was easily able to build my project before
<domenkozar> angerman actually fixed this properly in ghc
<domenkozar> but it will only hit 8.6
<srhb> :(
<angerman> O_o?
<domenkozar> oh it's actually in master
<domenkozar> I'm trying nixpkgs commit 91ca88b3e6b4d57640c213e755077b2d93d0a8bd
taktoa has joined #nixos-dev
<angerman> Yep the dylib stuff was baclported for 8.2 and 8.4
<angerman> It reduces the load command size by ~66%
<domenkozar> yeah I think this error is not about linker command but rather actualy clang arg list
<LnL> yes, this is something different
<LnL> ghc isn't the only thing that suffers from it either
<domenkozar> nor gcc it seems :)
<srhb> Really? I can't imagine that any clang call that isn't related to linking is long enough to hit arg_max when combined with the environment.
<srhb> Unless the env is already really large.
<srhb> (Which it probably is...)
<LnL> maybe arg_max is a bit lower on darwin?
<domenkozar> /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/syslimits.h
<domenkozar> need to check that for each macos version
<srhb> LnL: Yes, about one eighth of that of Linux
<LnL> eh, /Library/Developer/CommandLineTools?
<domenkozar> well <sys/syslimits.h>
<LnL> :p
<domenkozar> it's 256k
<domenkozar> should have a set structure in nix :P
<domenkozar> or unique list that's not O(n^2)
<domenkozar> right
taktoa has quit [Remote host closed the connection]
<domenkozar> this was broken in last 14 days: https://hydra.nixos.org/job/nixpkgs/trunk/stack.x86_64-darwin
<domenkozar> down to 318 commits :)
<domenkozar> anyone has a macos I can borrow?
<angerman> Ahh that clang args limit.
<angerman> I’ve a fix for that in my local nixpkgs branch as well.
<angerman> Essentially you need to stop the Haskell builder from producing `lib` folders.
<domenkozar> is it a ghc fix?
<domenkozar> or nix builder?
<angerman> the nix builder.
<domenkozar> angerman: if you can dig up that fix I'd be grateful :)
<angerman> The underlying issue is the following: we have `$out/lib` for each haskell library we build. However that folder is mostly useless for anything but haskell.
<angerman> However the cc-wrapper logic, puts any `$out/lib` folder of the dependencies into the NIX_CLFAGS.
<angerman> Now you end up with $pkg/lib for each haskell dependency, as well as `$pkg/lib/ghc8.4.../` that GHC forwards to the compiler/linker.
<angerman> and that overruns your limit.
<angerman> So a qhick fix is to just mutilate the haskell pkg builder, to stop it from generating `lib` folders.
<angerman> The *proper* fix would be to use response-files in the cc-wrapper, so that we pass `@args-file` to `clang` or `gcc` instead of passing the flags on the command line.
<domenkozar> does clang support that?
<angerman> Stopping the cc-wrapper from adding `$pkg/lib` for haskell packages, is also sensible.
<angerman> domenkozar yes, for quite a few years already, the commit was landed in clang in 2014. See https://reviews.llvm.org/D4897
<domenkozar> I wonder if /lib reliance breaks anything
<angerman> hasn't for me :-)
<angerman> of course if you try to link against haskell libraries from else where, you might end up with issues.
<domenkozar> interesting :)
<angerman> but even taht wouldn't work yet properly, as you'd need to custom-add the proper search path's anyway.
<angerman> in $out/lib, there's nothing you could properly link against. It's all in `$out/lib/ghc-$version`.
<domenkozar> this also pushes the linker limit a few 100 chars further :P
<domenkozar> LnL: btw, is /usr/bin/security allowed in macos sandbox?
<domenkozar> right now lots of haskell packages on macos depend on x509, which brings in SecurityTool
<domenkozar> which pulls in the world
<LnL> no, what's wrong with darinw.security_tool?
<domenkozar> pulls in ton of stuff
<LnL> like what? just some frameworks afaik
<domenkozar> $ nix-store -qR /nix/store/gsvjs6fp1hrpxady3hgb7d6c1264y1z8-SecurityTool-55115 | wc -l
<domenkozar> 69
<domenkozar> $ nix-store -qR /nix/store/gsvjs6fp1hrpxady3hgb7d6c1264y1z8-SecurityTool-55115 | du -sh
<domenkozar> 2.2G .
<LnL> huh, that's probably one culprit the most of those paths should be really small
<domenkozar> eh, du doesn't work as a pipe, lol
<domenkozar> 355M
<LnL> yeah, the sdk shouldn''t be in there
<domenkozar> and there's duplicate store paths
<domenkozar> 31M /nix/store/blawd9czgsl7bxb4i63z9rw0bmfxn073-ICU-osx-10.10.5
<domenkozar> 31M /nix/store/x17m259z4cac95y0py8s4y78b6s95iii-ICU-osx-10.10.5
<domenkozar> 2.5M /nix/store/wv04hpqqfvr23m6f2kksv7ad5kaj69r3-CF-osx-10.10.5
<domenkozar> 2.5M /nix/store/z9imly1f3lwc90irij4gx9j9b1nn43ij-CF-osx-10.10.5
<LnL> and might be possible to avoid python, not sure
<domenkozar> "/nix/store/x38ln6gxa3myclbcw8a2mcg140s3mmmn-Libsystem-osx-10.11.6" -> "/nix/store/wv04hpqqfvr23m6f2kksv7ad5kaj69r3-CF-osx-10.10.5" [color = "red"];
<domenkozar> "/nix/store/x38ln6gxa3myclbcw8a2mcg140s3mmmn-Libsystem-osx-10.11.6" -> "/nix/store/z9imly1f3lwc90irij4gx9j9b1nn43ij-CF-osx-10.10.5" [color = "green"];
* domenkozar scratches head
<LnL> interesting
<LnL> that's darwin.CF and stdenv.__bootPackages.darwin.CF
<domenkozar> so the latter shouldn't be propagated?
<LnL> yeah, not sure how it ends up in the closure
<domenkozar> $ grep -Ra /nix/store/12r38kqsdlgn9h1k49l43hzhjgrnkaxx-python-2.7.15 /nix/store/a75h1bpv686dachi1wg5rcrcvc5rvk9y-MacOS_SDK-10.10 2>/dev/null
<domenkozar> /nix/store/a75h1bpv686dachi1wg5rcrcvc5rvk9y-MacOS_SDK-10.10/Library/Frameworks/Python.framework/Examples/PyObjC/2.5.1/AppKit/SimpleService/rebuild.py:#!/nix/store/12r38kqsdlgn9h1k49l43hzhjgrnkaxx-python-2.7.15/bin/python
<domenkozar> /nix/store/a75h1bpv686dachi1wg5rcrcvc5rvk9y-MacOS_SDK-10.10/Library/Frameworks/Python.framework/Examples/PyObjC/2.5.1/AppKit/TinyURLService/rebuild.py:#!/nix/store/12r38kqsdlgn9h1k49l43hzhjgrnkaxx-python-2.7.15/bin/python
<domenkozar> /nix/store/a75h1bpv686dachi1wg5rcrcvc5rvk9y-MacOS_SDK-10.10/Library/Frameworks/Python.framework/Examples/PyObjC/2.5.1/Scripts/autoreadme.py:#!/nix/store/12r38kqsdlgn9h1k49l43hzhjgrnkaxx-python-2.7.15/bin/python
<domenkozar> lots of low hanging fruit apparently
<LnL> ah the sdk pulls in python, that's good news since it shouldn't be in the closure anyway
<domenkozar> oh it shouldn't?
<LnL> probably a good idea to use disallowedReferences for basically anything that uses the sdkk
<LnL> didn't you fix a bunch of stuff to copy headers over instead of symlinking them a while back?
<domenkozar> yeah for private sdk
<domenkozar> that was low hanging fruit back then
<LnL> ah, well same goes for the sdk no need to download the headers of _all_ frameworks if you just depend on Foundation
<domenkozar> nice :)
<domenkozar> ok let me open an issue, go surfing and fix that once I'm back :)
<LnL> heh, have fun! :)
<Mic92> I wonder if I should enforce strict-eval in NUR: https://github.com/mpickering/nur-packages/blob/master/haskell-plugins/default.nix
goibhniu has quit [Ping timeout: 256 seconds]
goibhniu has joined #nixos-dev
orivej has quit [Ping timeout: 264 seconds]
Lisanna has joined #nixos-dev
obadz has quit [Ping timeout: 265 seconds]
obadz has joined #nixos-dev
aszlig has quit [Quit: Kerneling down for reboot NOW.]
aszlig has joined #nixos-dev
<aszlig> niksnut/ikwildrpepper: what was the last time a kernel upgrade has been done on the Hydra build slaves?
<aszlig> because since 892a0e8ff40e8282fb09ee6c8a132979c5539191 the openblas build fails for i686-linux: https://nix-cache.s3.amazonaws.com/log/wi79zyfmwdpwx7bm29dzqh4vglx3x550-openblas-0.3.0.drv
<aszlig> and when i force a rebuild for 892a0e8ff40e8282fb09ee6c8a132979c5539191^ it still fails on my kernel, so i suppose the build has passed on a machine with an older kernel
<aszlig> this is the upstream issue (already included in 0.3.1): https://github.com/xianyi/OpenBLAS/issues/1575
<aszlig> s/already/fix is &/
<aszlig> so my idea would be to add a patch to openblas which is only used on i686-linux and the next staging merge will drop that patch so we avoid a mass-rebuild
<aszlig> does that sound good or should i just bump to 3.0.1?
<aszlig> err 0.3.1
<niksnut> aszlig: last week or so
<niksnut> upgraded to 18.03
<aszlig> niksnut: okay, that would explain that
<aszlig> so bumping openblas would mean 4795 rebuilds
<aszlig> so i'd go for just bumping without going through staging, any objections?
<niksnut> sure
<niksnut> why does so much depend on openblas btw?
<aszlig> niksnut: probably because of boost
<dtz> "nix why-depend everything openblas", lol :P (dunno)
<aszlig> ah, no
<aszlig> well, opencv would be a larger one, but that's it
<niksnut> I guess the real question is "why do the tests of a math package depend on the kernel version"
<aszlig> niksnut: i haven't dug into that sufficently but from what i can see it seems that it wasn't just the tests
<aszlig> hm, even looking through swap.S i can't see why anything would depend on the kernel version
aristid has quit [Ping timeout: 260 seconds]
aristid has joined #nixos-dev
<aszlig> niksnut: ah, it's numpy and boost indeed, because it depends on numpy
<niksnut> boost depends on numpy? :o
<aszlig> niksnut: if enablePython is true, yes
<aszlig> and/or enableNumpy
<niksnut> then that should be disabled by default
<niksnut> we can't have boost pulling in crap like that
<aszlig> niksnut: i guess numpy-support was accidentally enabled by default in https://github.com/NixOS/nixpkgs/pull/33359
<aszlig> oh...
<aszlig> you already turned it off
<aszlig> let's see how much will break now
<aszlig> i guess it should only be a small fraction of packages, for that we can still override boost with python support
orivej has joined #nixos-dev
orivej has quit [Ping timeout: 256 seconds]
Sonarpulse has joined #nixos-dev
Sonarpulse has quit [Ping timeout: 245 seconds]
<dtz> \o/ good find
goibhniu has quit [Ping timeout: 240 seconds]
<domenkozar> gchristensen: btw I'm not on the list for 2FA issue
<domenkozar> is that a bad script or?
<domenkozar> ah probably because I have 2FA :D
<gchristensen> I assume it is because, indeed, you have 2FA :)
<domenkozar> always a minority
<domenkozar> grr
<gchristensen> luckily the majority of nixpkgs committers have 2fa turned on
<gchristensen> (they were majority by like 2 people when we made the list)
<domenkozar> nice :)
<domenkozar> either most of people are working for cryptocurrency startups or we've been hacked a couple of times
<domenkozar> ok I'm back to serious now :P
<gchristensen> I have to leave my hotel tomorrow at 5am to get to the flight on time ...
<domenkozar> from london?
<gchristensen> yeah
<domenkozar> well at least it's tomorrow, france ATC were having strikes over all weekends in June
<gchristensen> oh man
<domenkozar> was stuck in Milano eating just rice
<domenkozar> (they have to provide meals)
orivej has joined #nixos-dev
<samueldr> minor but irritating issue with the options page fixed here: https://github.com/NixOS/nixos-homepage/pull/229
<infinisil> Literally unreadable
<dtz> if we level-up our testing by quite a bit :P we could maybe use something like https://mergify.io/ haha
goibhniu has joined #nixos-dev
Sonarpulse has joined #nixos-dev
Sonarpulse has quit [Ping timeout: 265 seconds]