samueldr changed the topic of #nixos-dev to: #nixos-dev NixOS Development (#nixos for questions) | NixOS 19.09 is released! https://discourse.nixos.org/t/nixos-19-09-release/4306 | https://hydra.nixos.org/jobset/nixos/trunk-combined https://channels.nix.gsc.io/graph.html | https://r13y.com | 19.09 RMs: disasm, sphalerite | https://logs.nix.samueldr.com/nixos-dev
phreedom_ has joined #nixos-dev
phreedom has quit [Remote host closed the connection]
drakonis has quit [Ping timeout: 265 seconds]
ris has quit [Ping timeout: 276 seconds]
drakonis has joined #nixos-dev
drakonis2 has joined #nixos-dev
drakonis has quit [Client Quit]
drakonis2 is now known as drakonis
drakonis1 has quit [Ping timeout: 276 seconds]
dongcarl has quit []
justanotheruser has joined #nixos-dev
orivej has joined #nixos-dev
orivej has quit [Ping timeout: 268 seconds]
drakonis has quit [Ping timeout: 246 seconds]
lopsided98 has quit [Quit: Disconnected]
lopsided98 has joined #nixos-dev
orivej has joined #nixos-dev
Jackneill has joined #nixos-dev
FRidh has joined #nixos-dev
FRidh has quit [Ping timeout: 240 seconds]
FRidh has joined #nixos-dev
Moredread[m] has quit [Remote host closed the connection]
yegortimoshenko has quit [Remote host closed the connection]
bennofs[m] has quit [Write error: Connection reset by peer]
atopuzov[m] has quit [Write error: Connection reset by peer]
arcnmx has quit [Write error: Connection reset by peer]
worldofpeace has quit [Read error: Connection reset by peer]
timokau[m] has quit [Write error: Connection reset by peer]
Nyanloutre[m] has quit [Read error: Connection reset by peer]
rycee has quit [Read error: Connection reset by peer]
aanderse has quit [Read error: Connection reset by peer]
layus[m] has quit [Write error: Connection reset by peer]
Ox4A6F has quit [Remote host closed the connection]
codyopel has quit [Read error: Connection reset by peer]
alienpirate5 has quit [Write error: Connection reset by peer]
thefloweringash has quit [Write error: Connection reset by peer]
Ericson2314 has quit [Write error: Connection reset by peer]
abbradar[m] has quit [Write error: Connection reset by peer]
nh2[m] has quit [Write error: Connection reset by peer]
vaibhavsagar has quit [Read error: Connection reset by peer]
jtojnar has quit [Read error: Connection reset by peer]
ma27[m] has quit [Read error: Connection reset by peer]
domenkozar[m] has quit [Write error: Connection reset by peer]
jonge[m] has quit [Write error: Connection reset by peer]
Ox4A6F has joined #nixos-dev
stears has joined #nixos-dev
__monty__ has joined #nixos-dev
tilpner has quit [Quit: tilpner]
tilpner has joined #nixos-dev
psyanticy has joined #nixos-dev
domenkozar[m] has joined #nixos-dev
dtz has joined #nixos-dev
alienpirate5 has joined #nixos-dev
timokau[m] has joined #nixos-dev
aanderse has joined #nixos-dev
jtojnar has joined #nixos-dev
bennofs[m] has joined #nixos-dev
atopuzov[m] has joined #nixos-dev
layus[m] has joined #nixos-dev
codyopel has joined #nixos-dev
nh2[m] has joined #nixos-dev
Ericson2314 has joined #nixos-dev
roberth has joined #nixos-dev
Nyanloutre[m] has joined #nixos-dev
ma27[m] has joined #nixos-dev
vaibhavsagar has joined #nixos-dev
rycee has joined #nixos-dev
thefloweringash has joined #nixos-dev
jonge[m] has joined #nixos-dev
worldofpeace has joined #nixos-dev
yegortimoshenko has joined #nixos-dev
abbradar[m] has joined #nixos-dev
arcnmx has joined #nixos-dev
Synthetica has joined #nixos-dev
<Profpatsch> I wonder if we should get rid of foldl for foldl' in nixpkgs.
<Profpatsch> Usually it’s a bad idea to use non-stricly accumulating foldl in a lazy language.
<Profpatsch> Grepping I see maybe two dozen uses of lib.foldl
<Profpatsch> Also, fold is still an alias of foldr
<Profpatsch> We might even get a nice speedup if we change every fold/foldr/foldl to foldl'
<Profpatsch> especially since foldl' is a builtin now
<gchristensen> give it a go and see what the evaluation performance report says :)
<Cale> Profpatsch: Be careful replacing foldr with foldl
<Cale> It depends on the combining function whether foldl' or foldr will be better
pie_ has quit [Ping timeout: 276 seconds]
<clever> Profpatsch: and in some cases, foldl and // is more expensive then map and listToAttrs
<Profpatsch> gchristensen: Is that report for all of nixpkgs?
<clever> // must copy the entire key/value list from both sets, so it will get slower with each append
<Profpatsch> clever: Ah, that’s cool information
<clever> ++ also has similar performance problems
<Profpatsch> Nix doesn’t do any rewrite passes, right?
<clever> rewrite passes?
<gchristensen> Profpatsch: yes
<Profpatsch> Well, it’s not a compiler. Though I guess even an interpreter could fuse chains of builtins together if it was smart enough
<clever> [ 1 2 3 ] ++ [ 4 ] involves copying 4 elements
<Profpatsch> clever: Because nix can’t know if the first list is used anywhere again, right.
<clever> [1 2 3] ++ [4] ++ [5] involves 9 element copies, i believe
<clever> yeah, it will always copy things, never modify
<clever> and its up to the GC to clean up the mess left behind
<clever> thunks being forced, at the only time it ever modifies the type of something
<Profpatsch> That’s why hnix should be faster in many/some cases
<clever> builtins.concatLists [ [1 2 3] [4] [5] ] involves only 5 element copies
<Profpatsch> Ah, wait, only if it does rewrites of certain expression forms to hit GHC optimizations. idk how much it does that
<clever> so its faster then the 9 copies from earlier
<Profpatsch> clever: This would be a great idea for a simple static linter.
<Profpatsch> Because I’m fairly sure we are using list concatenation a lot
<clever> the biggest offender, is lib.uniq
<clever> that list concats things to death
<Profpatsch> clever: Can you make that faster by converting to attrs and using // instead?
<clever> only if the elements can be cast to string
<gchristensen> http://gsc.io/graph.unstable.post.svg here is a flame graph
<{^_^}> nix#2459 (by cleverca22, 1 year ago, open): add a sublist primop
<clever> Profpatsch: the biggest cost of lib.unique, is lib.sublist
<clever> which can easily be re-written as a primop
<Profpatsch> gchristensen: For all of nixpkgs?
<clever> i just had a few edge cases, like specifcing an index greater then the list, that i havent covered
<clever> Profpatsch: that sublist pr, for a given expr, reduces the calls of elemAt from 108 MILLION to 59 thousand!
<gchristensen> Profpatsch: yes
<Profpatsch> clever: How often is that function used?
<andi-> 3r2
<andi-> (ignore that)
<clever> Profpatsch: i'm not sure how often its used in general nixpkgs/nixos, but it was heavily abused in snack (a haskell built tool, written in nix)
<Profpatsch> Oh fuck, it’s in make-derivation.nix. four times.
<Profpatsch> In the cross code
<clever> Profpatsch: lib.uniq performance is fine for lists <10 elements
<clever> Profpatsch: it only gets insane when you deal with lists of over 100 elements
<Profpatsch> yeah, but that’s for dependencies
<clever> because you have to copy 100 + 99 + 98 + 97 ... elements
<Profpatsch> Propagated dependencies
<Profpatsch> In friggin mkDerivation
<FRidh> Is there a hook to create symbolic links to versioned shared objects?
<clever> Profpatsch: you could try building the nix from my pr, and then profile an example drv to see what diff it makes
<clever> ,profiling
<{^_^}> Use NIX_COUNT_CALLS=1 and/or NIX_SHOW_STATS=1 to profile Nix evaluation
stears has quit [Ping timeout: 268 seconds]
<Profpatsch> clever: Haha, that’s blocked on a fix from your side
<Profpatsch> I think we can merge otherwise
<Profpatsch> gchristensen: I’m guessing producing this flamegraph needs a lot of RAM?
<clever> Profpatsch: the problem was just segfaults if you give it an invalid index, and then i never got around to finishing it
<Profpatsch> clever: do eeet
stears has joined #nixos-dev
<Profpatsch> gchristensen: Is it easy to produce this flamegraph locally? Because that might be very handy for smaller things, like a local shell.nix
<clever> Profpatsch: funny thing is, i started that at the 2018 nixcon
<clever> Profpatsch: and then for the 2019 nixcon, i started builtins.fork
<gchristensen> Profpatsch: one sec
<Profpatsch> gchristensen: Cool, I think I’ll fire that at something with lots of folds and see what happens.
<gchristensen> rig8ht on
<gchristensen> caution though, if you do it on all of nixpkgs you need approximately all of the RAM (32-64G I think?)
<Profpatsch> Yeah, not trying that :P
<clever> gchristensen: hydra-eval-jobs can be configured to restart itself at regular intervals to keep heap usage down
<clever> but that also messes with NIX_SHOW_STATS, since it will overwrite the file on each restart
<clever> i have an unpushed change to modify NIX_SHOW_STATS to append instead
<gchristensen> ah, it isn't nix that has the problem, it is flamegraph.pl which takes 64g of ram
<clever> oh, nvm then, lol
<clever> oh, and i was talking to angerman a few days ago, about having nix itself generate flamegraphs
<clever> basically, record the start and end timestamp, of every call to the derivation primop
<gchristensen> it does lol
<clever> along with the stack depth (how many derivations deep it is)
<gchristensen> look at that link :)
<clever> somebody beat me to it?
<angerman> yea that flamgraph.pl thing is shit.
<clever> does it also include the ifd codepath?
<gchristensen> angerman: shit or the shit? :P
<Profpatsch> gchristensen: Hm, it might work if you used generators instead. Or we rewrite it in rust, prob faster than debugging python code )
<angerman> Awesome. I just keep planing ideas into clever; and see what transpires.
<Profpatsch> Ah, flamegraph, was looking at stack-collapse.py
<angerman> can we get /opt/nix now? :-)
* angerman ducks and runs.
<clever> gchristensen: ah, --trace-function-calls is new, i hadnt seen that before
<angerman> gchristensen: the shiiit!
<gchristensen> Profpatsch: yeah, I'm not rewriting flamegraph.pl in rust
<angerman> gchristensen: in all honesty though, I've tried using it on some ghc traces and had to give up as it just kept eating more and more ram and then swap.
<gchristensen> brendan gregg will do it some day when he's tired of its performance :P
<{^_^}> firing: BuildsStuckOverTwoDays: https://status.nixos.org/prometheus/alerts
* gchristensen will look after this call
<{^_^}> firing: BuildsStuckOverTwoDays: https://status.nixos.org/prometheus/alerts
<{^_^}> firing: BuildsStuckOverTwoDays: https://status.nixos.org/prometheus/alerts
eraserhd has quit [Quit: WeeChat 2.6]
eraserhd has joined #nixos-dev
<{^_^}> firing: BuildsStuckOverTwoDays: https://status.nixos.org/prometheus/alerts
<gchristensen> I get it :x
<gchristensen> okay they're firing because new alerts keep getting added :P
drakonis has joined #nixos-dev
<tilpner> gchristensen: That's controlled by group_interval, which defaults to 5m
drakonis_ has joined #nixos-dev
<xwvvvvwx> I'm getting `error: cannot auto-call a function that has an argument without a default value ('stdenv')` when calling `./update.sh` for wireguard-tools
<xwvvvvwx> Am I calling it wrong or is the script broken?
<xwvvvvwx> ah ok, I figured it out after I asked :D
<xwvvvvwx> nix-shell maintainers/scripts/update.nix --argstr package wireguard-tools from the root
orivej has quit [Ping timeout: 268 seconds]
drakonis has quit [Quit: WeeChat 2.6]
<puck> https://github.com/NixOS/nix/issues/3246 i accidentally side-effects in nix
<{^_^}> nix#3246 (by puckipedia, 32 seconds ago, open): The state of thunks is observable
<gchristensen> oh no
<puck> this took me quite some time to perfect; ensuring the thunks happen exactly where i want is suprisingly difficult
<gchristensen> so cool!!
<gchristensen> puck: you're finding some great bugs, can you tell me about your research?
<puck> basically, i have too much spare time on my hands
<gchristensen> well, I'm honored to have you spend it on nix like this =)
<puck> a bunch of it is since i'm working on a custom nix evaluator myself, and i end up picking up on a ton of edge cases quickly
<gchristensen> neato
<gchristensen> I wonder what valueSize is for ...?
<puck> it never gets used in nixpkgs and seems to have been added for debugging memory consumption issues
drakonis has joined #nixos-dev
<qyliss> can you use it in strict mode?
<qyliss> or pure mode or whatever it's called
<tilpner> > builtins.valueSize (x: x)
<{^_^}> 4590729
<gchristensen> imo we should just delete that one :)
<puck> seems so, it's not limited to e.g. pureEval
<puck> > with builtins; let val = (x: x) 5; preSeq = valueSize val; postSeq = seq preSeq (seq val (valueSize val)); in [ preSeq postSeq ]
<{^_^}> [ <CODE> <CODE> ]
<puck> huh.
<tilpner> > :p with builtins; let val = (x: x) 5; preSeq = valueSize val; postSeq = seq preSeq (seq val (valueSize val)); in [ preSeq postSeq ]
<{^_^}> attempt to call something which is not a function but an integer, at (string):209:40
drakonis_ has quit [Ping timeout: 265 seconds]
<tilpner> Huh, I didn't know Nix specialised lists with few elements
<tilpner> But I still haven't found how the size of a function is determined
<tilpner> I doubt (x: x) actually weighs 4.5MB
<puck> it's the size of the expr and all the environments
<puck> (actually, not sure it counts the expr)
<tilpner> Ahh
drakonis has quit [Ping timeout: 276 seconds]
drakonis has joined #nixos-dev
<tilpner> gchristensen: That repetition-on-new-alert is controlled by group_interval, which defaults to 5m
<gchristensen> gotcha
<gchristensen> I'm giving niksnut a few minutes to look in to why they're stalled for now
pie_ has joined #nixos-dev
FRidh has quit [Quit: Konversation terminated!]
andi- has quit [Ping timeout: 276 seconds]
pie_ has quit [Remote host closed the connection]
andi- has joined #nixos-dev
pie_ has joined #nixos-dev
Jackneill has quit [Remote host closed the connection]
lassulus has quit [Ping timeout: 252 seconds]
kcalvinalvin has quit [Ping timeout: 250 seconds]
kcalvinalvnn has joined #nixos-dev
sheenobu has quit [Ping timeout: 250 seconds]
page has quit [Ping timeout: 250 seconds]
sheenobu has joined #nixos-dev
page has joined #nixos-dev
__monty__ has quit [Ping timeout: 246 seconds]
drakonis has quit [Ping timeout: 265 seconds]
drakonis has joined #nixos-dev
edef has quit [Ping timeout: 250 seconds]
arianvp has quit [Ping timeout: 250 seconds]
edef has joined #nixos-dev
arianvp has joined #nixos-dev
pie_ has quit [Ping timeout: 246 seconds]
lassulus has joined #nixos-dev
<{^_^}> firing: BuildsStuckOverTwoDays: https://status.nixos.org/prometheus/alerts
pie_ has joined #nixos-dev
ris has joined #nixos-dev
__monty__ has joined #nixos-dev
<samueldr> are there still issues with getting the aarch64 builders?
psyanticy has quit [Quit: Connection closed for inactivity]
pie_ has quit [Ping timeout: 250 seconds]
drakonis has quit [Quit: Leaving]
drakonis has joined #nixos-dev
<jtojnar> aarrg, "gmic.h" includes "./CImg.h" and also tells it to include "gmic.cpp", neither of the two included files is installed
pie_ has joined #nixos-dev
drakonis has quit [Quit: WeeChat 2.6]
<gchristensen> dang, I can't use /dev/disk/by-id/ata-SSDSCKJB240G7R_PHDW83710400240E1 as a grub device from within nixos-install
drakonis has joined #nixos-dev
worldofpeace_ has joined #nixos-dev
<samueldr> how come?
worldofpeace_ has quit [Ping timeout: 268 seconds]
drakonis has quit [Ping timeout: 250 seconds]
drakonis has joined #nixos-dev
__monty__ has quit [Quit: leaving]
<gchristensen> udev hasn't run to create all the symlinks I think?
<gchristensen> not sure exactly
Jackneill has joined #nixos-dev
drakonis has quit [Ping timeout: 265 seconds]
Jackneill has quit [Remote host closed the connection]
<samueldr> [17:17:27] <lovesegfault> https://hydra.nixos.org/eval/1556762#tabs-unfinished
<samueldr> [17:17:37] <lovesegfault> Why are finished jobs listed in the unfinished tab?
worldofpeace_ has joined #nixos-dev
<{^_^}> firing: BuildsStuckOverTwoDays: https://status.nixos.org/prometheus/alerts