ckauhaus has quit [Remote host closed the connection]
ckauhaus has joined #nixos-dev
ckauhaus has quit [Remote host closed the connection]
FRidh has joined #nixos-dev
ckauhaus has joined #nixos-dev
ckauhaus has quit [Remote host closed the connection]
ckauhaus has joined #nixos-dev
ckauhaus has quit [Remote host closed the connection]
ckauhaus has joined #nixos-dev
ckauhaus has quit [Remote host closed the connection]
ckauhaus has joined #nixos-dev
ckauhaus has quit [Remote host closed the connection]
ckauhaus has joined #nixos-dev
goibhniu has quit [Ping timeout: 240 seconds]
<peti>
domenkozar: Yes, getting Darwin builds back on track is important.
<domenkozar>
I don't know if reverting cross changes it's feasible
<domenkozar>
those span over whole 18.09
<domenkozar>
and involve complex changes across stdenv, etc
<domenkozar>
it's quite some work identifying those and reverting them
<domenkozar>
not to mention going through staging
<LnL>
what's the state of strictDeps for haskell stuff?
<domenkozar>
well peti is not happy with neither of two solutions
<domenkozar>
which I understand
<domenkozar>
but I don't know who will spend time reverting all that code
<domenkozar>
it's 4 months of stdenv changes
<domenkozar>
on top of that
<domenkozar>
I tried for 1h to understand that stdenv code
<domenkozar>
and I'm too stupid
<domenkozar>
it's a state machine with too many code paths
<domenkozar>
for me it's unacceptable and gives bad reputation for nixpkgs
<peti>
I feel like StrictDeps=true is the morally right choice because it undoes a hack that caused all this flag duplication. Of all the poisons we can choose from, this one feels like the least bad one.
<domenkozar>
peti: do you understand why we need that?
<peti>
Kind of.
<domenkozar>
the reason why we have all these duplicate flags is
<domenkozar>
this code calls now setup hooks many times
<domenkozar>
it's a 3-level for loop
<domenkozar>
that's indexed by different arrays of integers
<domenkozar>
it's madness :)
<peti>
Apparently, there's some issue in the stdenv hooks where their results depend on the order in which they are run. StrictDeps=false will, apparently, re-run those hooks multiple times to avoid those issues. A side effect is, though, that -I... and -L... flags get added to the environment multiple times.
<peti>
In hindsight, that code should probably not have made it into Nixpkgs master.
<LnL>
domenkozar: I feel like we shouldn't be calling setup hooks multiple times, however I realised that strictDeps isn't what I thought it was yesteday
<peti>
I am worried because it seems like people add one layer of hacks on top of another layer of hacks to fix issues caused the older layer, but the new layer actually doesn't fix those and adds new issues as an unintended consequence.
<LnL>
yeah
<domenkozar>
peti: not to say now there's probably two people in our community that understand stdenv
<LnL>
but strictDeps isn't a hack for this and would ideally be the default in the stdenv
<LnL>
that said, seeing it as a solution for duplicate flags isn't really correct
<Dezgeg>
if it's the thing that causes buildInputs not to be put on PATH, then it will never happen since it would break way too much code
<LnL>
I thought it was literally a hack to deduplicate inputs
<Dezgeg>
well, as one side-effect
<peti>
It does more than de-duplicate inputs. If it were only about de-duplication, then turning it on wouldn't break half the Haskell packageg set. :-)
<peti>
StrictDeps=true has some effect on trasitive dependencies, too. Not sure what that is, though.
<LnL>
Dezgeg: true, unless we slowly enable it in packages over time
<Dezgeg>
that's even more confusing
<domenkozar>
well just fixing haskell is a mess
<domenkozar>
becuase community can't agree if it's a feature or a bug
<LnL>
why? it's the same as moving packages to nativeBuildInputs, only stricter
<Dezgeg>
the problem is IMHO that now those people who have no use for cross compiling need to understand this whether they need it or not
<peti>
Yes, exactly.
<LnL>
well, PATH vs CFLAGS is a good distinction if you ask me, same for other types of inputs like python packages
goibhniu has joined #nixos-dev
<Dezgeg>
but it will break down with say, perl where you might need to either (or even both) run a configure script written in perl or have patchShebangs patch the shebang in an installed perl script in $out
Lisanna has quit [Remote host closed the connection]
<Dezgeg>
e.g. having to explain to someone who doesn't need cross compilation why they need to have both nativeBuildInputs = [ perl ]; and buildInputs = [ perl ]; in their derivation for it to work is going to be quite awkward I bet
ckauhaus has quit [Remote host closed the connection]
Lisanna has joined #nixos-dev
ckauhaus has joined #nixos-dev
ckauhaus has quit [Ping timeout: 276 seconds]
ckauhaus has joined #nixos-dev
ckauhaus has quit [Remote host closed the connection]
orivej has joined #nixos-dev
<gchristensen>
master takes >8G of RAM to evaluate now
<gchristensen>
(on ofborg)
<gchristensen>
what do we do?
<gchristensen>
cc niksnut do you have any secret "use less ram" branches that might be useful here?
orivej has quit [Ping timeout: 260 seconds]
orivej has joined #nixos-dev
ckauhaus has joined #nixos-dev
orivej has quit [Ping timeout: 240 seconds]
ckauhaus has quit [Remote host closed the connection]
ckauhaus has joined #nixos-dev
__Sander__ has joined #nixos-dev
<gchristensen>
OK I set the initial heap size to 10G which mirrors hydra
<gchristensen>
we should start doing something about nixpkgs eval now so that next time PRs start failing like this, we have somewhere to turn to fix it
ckauhaus has quit [Remote host closed the connection]
ckauhaus has joined #nixos-dev
<Profpatsch>
Huh, the pain is real
<Profpatsch>
gchristensen: Do you think that was enough?
<gchristensen>
this is a day where everything feels like it is on fire
ckauhaus has quit []
<Profpatsch>
I’m not sure what the StrictDeps stuff from above is about, but it sounds like it could be related.
<Profpatsch>
It definitely sounds like a laziness bug.
<LnL>
nah, that's all stdenv runtime
<gchristensen>
I wish sonarpulse and matthewbauer were actually available for easy communication since they've been changing a bunch of big stuff lately
<gchristensen>
just messaged sonarpulse
<Profpatsch>
gchristensen: hm, total allocations: 53810856 bytes
<Profpatsch>
On the master of exactly the PR from above.
<Profpatsch>
And that should have been the current master at the time, minus one or two commits.
<gchristensen>
master you're based off of, or master of nixpkgs upstream? the bulid is always done post-merge
<Profpatsch>
Well, I had rebased the commit about one minute earlier.
<gchristensen>
ah
<Profpatsch>
It’s on 8432dec854f7e9f622217671958a92b535dc0d51
<gchristensen>
:|
<gchristensen>
maybe just try re-triggering the build then
<Profpatsch>
So something to do with the ofborg setup?
<Profpatsch>
Okay.
<gchristensen>
shouldn't be!
<gchristensen>
it is just doing a nix-build
<Profpatsch>
gchristensen: So interesting. It still happens https://logs.nix.ci/?key=nixos/nixpkgs.43530&attempt_id=b8c45400-bc39-4b32-b304-ef548756b1f2
<Profpatsch>
Maybe one of those three commits really broke something.
<gchristensen>
oh by the way, don't break anything between Thursday night and like, next Wednesday, I won't have my computers or keyboards post-surgery
andi- has joined #nixos-dev
<manveru>
hopefully we'll survive :)
<LnL>
should somebody learn debugging ofborg 101 so you don't have to worry?
<gchristensen>
yeah that'd be good
<gchristensen>
maybe, LnL, I could get you the abililty to ssh in / deploy / etc before then?
<gchristensen>
... if you want that :)
FRidh has quit [Quit: Konversation terminated!]
<LnL>
a bit scary, but I'm up for it :)
* gchristensen
feels the same way :D
<gchristensen>
Profpatsch: should be fixed
<thoughtpolice>
IIRC, nix copy runs with much less memory usage/constant space these days. Is there any work on doing this for e.g. `nix-prefetch-url` or `nix add-to-store`, or has anyone tried/thought about it?
<thoughtpolice>
In case you're wondering I have a 30GB file to add to the store, so.
<Profpatsch>
->}(/[-}? why does substituteInPlace still not fail on missing replacements.
<gchristensen>
it isn't guaranteed to be missed replacement
ixxie has joined #nixos-dev
<thoughtpolice>
gchristensen: Ping. I have a question about the multi-user Nix installer
<gchristensen>
ok
<gchristensen>
what's up? I can try and help now
<thoughtpolice>
gchristensen: On a fresh install, using the latest 2.1pre installer from Hydra, a run of `systemctl cat nix-daemon | grep ExecStart` gives me
<thoughtpolice>
Wouldn't it be better to use /nix/var/nix/profiles/default/bin/nix-daemon (which is what `which nix-daemon` returns by default with the sourced /etc/profile.d/nix.sh) ?
<gchristensen>
hmm maybe yes
<thoughtpolice>
For example, if I upgraded nix by building a new copy and installing it with the `root` user, e.g. `nix-env -i /nix/store/<path to a new version of nix>`, that would change the roots
<thoughtpolice>
then a GC later would delete the old version
<gchristensen>
hmm no I don't think so
<thoughtpolice>
and, broken
<gchristensen>
one sce
<gchristensen>
sec*
<thoughtpolice>
Or, at least I think so
<thoughtpolice>
gchristensen: sure. Really my complaint is maybe a bit different -- I just remembered this behavior because I had to ask myself once, "What is the procedure for upgrading now, with the new installer, on Non-NixOS?"
<gchristensen>
so I believe the enabled unit is at /nix/var/nix/profiles/default/lib/systemd/system/nix-daemon.service
<thoughtpolice>
yep
<gchristensen>
right
<thoughtpolice>
(you can see that with `systemctl cat` actually, it shows the path)
<gchristensen>
so if you install a new nix and daemon-reload and restart nix-daemon, it should just work
<thoughtpolice>
Oh, I see, OK, that's good.
<thoughtpolice>
Just wondering, I found that weird.
<thoughtpolice>
I think before on my laptop it might have been slightly different, I can't remember. I've been using the installer since, well, basically you started working on it, so it's gone through a few revisions :P
<thoughtpolice>
Like before I think I did see this but it didn't link the enabled unit from /nix, it was in /etc. Maybe. You'd actually know! But if it works now, it doesn't matter :)
<gchristensen>
ah :)
<gchristensen>
I _think_ it works like I describe, but I haven't fully tested it :$
<thoughtpolice>
Yeah, the upgrade on-non-NixOS story probably has to be thought about carefully I guess...
<gchristensen>
I have a grand test matrix in mind but am not able to spend too much engineering time outside of stuff I'm doing for work due to my hand injuries
<thoughtpolice>
ah, sorry to hear that
<gchristensen>
its ok. surgery in like 42 hours from now, not that I'm counting :|
<aminechikhaoui>
gchristensen: hope you'll feel better after :-)
<gchristensen>
me too!
<gchristensen>
thanks!
<thoughtpolice>
Is qemu considered off limits for master (or 'staging only') because it causes tons of rebuilds, or
<gchristensen>
probably best to send a PR and get the rebuild estimation labels
<thoughtpolice>
qemu is presumably old enough where it doesn't actually have a `version` attribute, it bakes it directly into `name` :| I fixed this recently in postgres actually
<thoughtpolice>
But it's nice for overrides
<thoughtpolice>
Maybe I should just wait and submit the PR/work for QEMU 3.0 when it comes out
<thoughtpolice>
which should be Real Soon Now, anyway
<thoughtpolice>
gchristensen: good idea, thanks!
<gchristensen>
:)
<thoughtpolice>
I should probably figure out how ofborg works one of these days too, so I can use it.
<gchristensen>
its biggest features are totally automatic on every PR: (1) did it break eval? (2) how many packages did it trigger rebuilds for on mac and linux?
FRidh has quit [Quit: Konversation terminated!]
aristid has quit [Quit: WeeChat 2.1]
sir_guy_carleton has quit [Quit: WeeChat 2.0]
sir_guy_carleton has joined #nixos-dev
pie_ has joined #nixos-dev
ixxie has quit [Ping timeout: 240 seconds]
pie_ has quit [Ping timeout: 256 seconds]
<Sonarpulse>
gchristensen: pong, belated
pie_ has joined #nixos-dev
<gchristensen>
let' s talk in a bit
<gchristensen>
dinner
<Sonarpulse>
ok
pie_ has quit [Read error: Connection reset by peer]
pie__ has joined #nixos-dev
pie__ has quit [Remote host closed the connection]
pie__ has joined #nixos-dev
<LnL>
Sonarpulse: o/
<Sonarpulse>
LnL: belated pong
<LnL>
I think I only know half of the context around the duplicate setup hooks / strict deps stuff
<LnL>
so first off I only realised this yesterday but strictDeps doesn't deduplicate flags right, it's just the PATH/CFLAGS separation of native/regular inputs
<LnL>
and because the haskell infra uses nativeInputs quite extensively it 'fixes' ARG_MAX for cflags/ldfags
<Sonarpulse>
LnL: errr you saying we have a good thing or bad thing here?
<LnL>
second, I recall noticing the duplicate setup hook calls while reviewing or shortly after one of the cross changes, pretty sure I talked to you about it but I forgot what the conclusion what
<LnL>
was*
<Sonarpulse>
LnL: it is intended that the setup hooks are called twice
<Sonarpulse>
because the offsets are different
<Sonarpulse>
and it does use the offset
<Sonarpulse>
now the CFLAGS stuff I suppose still gets duplicated at the end of the day
<Sonarpulse>
due to the way cc-wrapper works
<LnL>
yeah, I remember that but domen noticed paths repeating 6+ times
<Sonarpulse>
right that is with strictDeps = false
<Sonarpulse>
LnL: so, I mentinoed the pkg-config wrapper
<Sonarpulse>
I think a simpler thing to do
<Sonarpulse>
for angerman
<Sonarpulse>
's thing
<Sonarpulse>
is /nix-support/{c_include_dirs,lib_dirs}
<Sonarpulse>
which will tell the wrappers what do to do
<LnL>
not sure how that helps
<Sonarpulse>
LnL: we can tell it to skip lib dir
<Sonarpulse>
while keeping lib dir
<Sonarpulse>
also for stdlib c++ do both include dirs
<Sonarpulse>
two old problems, one solution!
<LnL>
oh, like input -> input/lib && input/include
<LnL>
if they exist
{`-`} has joined #nixos-dev
<LnL>
but that doesn't really help with the explosion of duplicates, does it?
pie__ has quit [Remote host closed the connection]
pie__ has joined #nixos-dev
<Sonarpulse>
LnL: yeah it does
<Sonarpulse>
so angerman's thing
<Sonarpulse>
is just don't do any -L for haskell at all
<Sonarpulse>
since the libs are in subdirs
<Sonarpulse>
that's a fine fix
<Sonarpulse>
we can do it in addition
<Sonarpulse>
that will probably cut my 3 to 2
<Sonarpulse>
or 3 to 1
shlevy has quit [Ping timeout: 265 seconds]
orivej has joined #nixos-dev
pie__ has quit [Remote host closed the connection]