<{^_^}>
nix#2459 (by cleverca22, 1 year ago, open): add a sublist primop
<clever>
Profpatsch: the biggest cost of lib.unique, is lib.sublist
<clever>
which can easily be re-written as a primop
<Profpatsch>
gchristensen: For all of nixpkgs?
<clever>
i just had a few edge cases, like specifcing an index greater then the list, that i havent covered
<clever>
Profpatsch: that sublist pr, for a given expr, reduces the calls of elemAt from 108 MILLION to 59 thousand!
<gchristensen>
Profpatsch: yes
<Profpatsch>
clever: How often is that function used?
<andi->
3r2
<andi->
(ignore that)
<clever>
Profpatsch: i'm not sure how often its used in general nixpkgs/nixos, but it was heavily abused in snack (a haskell built tool, written in nix)
<Profpatsch>
Oh fuck, it’s in make-derivation.nix. four times.
<Profpatsch>
In the cross code
<clever>
Profpatsch: lib.uniq performance is fine for lists <10 elements
<clever>
Profpatsch: it only gets insane when you deal with lists of over 100 elements
<Profpatsch>
yeah, but that’s for dependencies
<clever>
because you have to copy 100 + 99 + 98 + 97 ... elements
<Profpatsch>
Propagated dependencies
<Profpatsch>
In friggin mkDerivation
<FRidh>
Is there a hook to create symbolic links to versioned shared objects?
<clever>
Profpatsch: you could try building the nix from my pr, and then profile an example drv to see what diff it makes
<clever>
,profiling
<{^_^}>
Use NIX_COUNT_CALLS=1 and/or NIX_SHOW_STATS=1 to profile Nix evaluation
stears has quit [Ping timeout: 268 seconds]
<Profpatsch>
clever: Haha, that’s blocked on a fix from your side
<Profpatsch>
I think we can merge otherwise
<Profpatsch>
gchristensen: I’m guessing producing this flamegraph needs a lot of RAM?
<clever>
Profpatsch: the problem was just segfaults if you give it an invalid index, and then i never got around to finishing it
<Profpatsch>
clever: do eeet
stears has joined #nixos-dev
<Profpatsch>
gchristensen: Is it easy to produce this flamegraph locally? Because that might be very handy for smaller things, like a local shell.nix
<clever>
Profpatsch: funny thing is, i started that at the 2018 nixcon
<clever>
Profpatsch: and then for the 2019 nixcon, i started builtins.fork
<Profpatsch>
gchristensen: Cool, I think I’ll fire that at something with lots of folds and see what happens.
<gchristensen>
rig8ht on
<gchristensen>
caution though, if you do it on all of nixpkgs you need approximately all of the RAM (32-64G I think?)
<Profpatsch>
Yeah, not trying that :P
<clever>
gchristensen: hydra-eval-jobs can be configured to restart itself at regular intervals to keep heap usage down
<clever>
but that also messes with NIX_SHOW_STATS, since it will overwrite the file on each restart
<clever>
i have an unpushed change to modify NIX_SHOW_STATS to append instead
<gchristensen>
ah, it isn't nix that has the problem, it is flamegraph.pl which takes 64g of ram
<clever>
oh, nvm then, lol
<clever>
oh, and i was talking to angerman a few days ago, about having nix itself generate flamegraphs
<clever>
basically, record the start and end timestamp, of every call to the derivation primop
<gchristensen>
it does lol
<clever>
along with the stack depth (how many derivations deep it is)
<gchristensen>
look at that link :)
<clever>
somebody beat me to it?
<angerman>
yea that flamgraph.pl thing is shit.
<clever>
does it also include the ifd codepath?
<gchristensen>
angerman: shit or the shit? :P
<Profpatsch>
gchristensen: Hm, it might work if you used generators instead. Or we rewrite it in rust, prob faster than debugging python code )
<angerman>
Awesome. I just keep planing ideas into clever; and see what transpires.
<Profpatsch>
Ah, flamegraph, was looking at stack-collapse.py
<angerman>
can we get /opt/nix now? :-)
* angerman
ducks and runs.
<clever>
gchristensen: ah, --trace-function-calls is new, i hadnt seen that before
<angerman>
gchristensen: the shiiit!
<gchristensen>
Profpatsch: yeah, I'm not rewriting flamegraph.pl in rust
<angerman>
gchristensen: in all honesty though, I've tried using it on some ghc traces and had to give up as it just kept eating more and more ram and then swap.
<gchristensen>
brendan gregg will do it some day when he's tired of its performance :P
<gchristensen>
okay they're firing because new alerts keep getting added :P
drakonis has joined #nixos-dev
<tilpner>
gchristensen: That's controlled by group_interval, which defaults to 5m
drakonis_ has joined #nixos-dev
<xwvvvvwx>
I'm getting `error: cannot auto-call a function that has an argument without a default value ('stdenv')` when calling `./update.sh` for wireguard-tools