gchristensen changed the topic of #nixos-dev to: NixOS Development (#nixos for questions) | https://hydra.nixos.org/jobset/nixos/trunk-combined https://channels.nix.gsc.io/graph.html | 18.09 release managers: vcunat and samueldr | https://logs.nix.samueldr.com/nixos-dev
pie__ has joined #nixos-dev
pie___ has quit [Remote host closed the connection]
orivej has quit [Ping timeout: 250 seconds]
jtojnar has quit [Ping timeout: 245 seconds]
jtojnar has joined #nixos-dev
pie__ has quit [Remote host closed the connection]
pie___ has joined #nixos-dev
sir_guy_carleton has quit [Quit: WeeChat 2.2]
Lisanna has quit [Ping timeout: 245 seconds]
sir_guy_carleton has joined #nixos-dev
lassulus_ has joined #nixos-dev
sir_guy_carleton has quit [Quit: WeeChat 2.2]
lassulus has quit [Ping timeout: 272 seconds]
lassulus_ is now known as lassulus
pie__ has joined #nixos-dev
pie___ has quit [Remote host closed the connection]
orivej has joined #nixos-dev
Lisanna has joined #nixos-dev
orivej has quit [Ping timeout: 246 seconds]
asymmetric has joined #nixos-dev
asymmetric has quit [Remote host closed the connection]
asymmetric has joined #nixos-dev
asymmetric_ has joined #nixos-dev
asymmetric has quit [Ping timeout: 240 seconds]
hedning has joined #nixos-dev
orivej has joined #nixos-dev
orivej has quit [Ping timeout: 250 seconds]
init_6 has joined #nixos-dev
ma27 has quit [Quit: WeeChat 2.2]
ma27 has joined #nixos-dev
ma27 has quit [Quit: WeeChat 2.2]
ma27 has joined #nixos-dev
drakonis has joined #nixos-dev
asymmetric_ is now known as asymmetric
orivej has joined #nixos-dev
init_6 has quit [Ping timeout: 245 seconds]
orivej has quit [Ping timeout: 245 seconds]
pie___ has joined #nixos-dev
pie__ has quit [Remote host closed the connection]
__Sander__ has joined #nixos-dev
orivej has joined #nixos-dev
edwtjo has quit [Ping timeout: 240 seconds]
<arianvp> Wondering if people are still interested to have something like a networkd hack day next year
<arianvp> I think we can get pretty far. We'll probably not make 19.03 but I think 19.09 might be feasible if we get a few people interested
<ckauhaus> I would be interested ;-)
edwtjo has joined #nixos-dev
ckauhaus has quit [Quit: WeeChat 2.2]
<domenkozar> arianvp: that sounds interesting
<domenkozar> I'd love sprints to happen, do you want to do it remote?
<samueldr> eval of nixos:trunk-combined is failing (due to a change of mine) Too many heap sections: Increase MAXHINCR or MAX_HEAP_SECTS
<samueldr> how can we make this not an issue anymore, without reverting making aarch64 a supported platform?
<samueldr> if I read the output of `time` right, it's taking about 10.9 GiB on my machine to nix-instantiate nixos/release-combined.nix --show-trace
<Mic92> fpletz might be also interested in the networkd sprint
<arianvp> I think remote is probably the best option
<arianvp> Though having a few people together irl is always good
<Mic92> arianvp: that one might help: https://videochat.mayflower.de
sir_guy_carleton has joined #nixos-dev
<gchristensen> I'm asking around about that
<samueldr> 8.12GB with aarch64-linux in limitedSupport (waiting to see how it compares to previously with it entirely removed)
asymmetric has quit [Ping timeout: 246 seconds]
<gchristensen> (that number will be good to know)
<samueldr> eh, got 8.32 but let's chalk it up to being equivalent limited support is about equal to not adding it at all
<gchristensen> interesting
<samueldr> limitedSupportedSystems doesn't do much though
<gchristensen> right
<gchristensen> means build build the absolute minimum closure
<samueldr> though we would be able to *also* pick some attributes from aarch64-linux to start off with, like the iso... which I forgot to add in the initial PR oops
ma27 has quit [Quit: WeeChat 2.2]
ma27 has joined #nixos-dev
Jackneill has quit [Remote host closed the connection]
pie__ has joined #nixos-dev
pie___ has quit [Remote host closed the connection]
init_6 has joined #nixos-dev
init_6 has quit [Ping timeout: 246 seconds]
__Sander__ has quit [Quit: Konversation terminated!]
orivej has quit [Ping timeout: 250 seconds]
jtojnar has quit [Remote host closed the connection]
drakonis_ has joined #nixos-dev
drakonis has quit [Ping timeout: 250 seconds]
orivej has joined #nixos-dev
obadz has quit [Ping timeout: 246 seconds]
<flokli> arianvp: domenkozar: Mic92: what about hacking around some days before or after fosdem, as suggested in https://discourse.nixos.org/t/fosdem-brussels-2-3-february-2019/1307 ?
<flokli> I'll be in too :-)
orivej has quit [Ping timeout: 250 seconds]
<arianvp> Im skiing in the Alps then
<arianvp> So no Fosdem for me unfortunately
<flokli> Could be worse ;-)
Synthetica has quit [Quit: Connection closed for inactivity]
<domenkozar> I used to attend ski sprints
<domenkozar> we could have that
drakonis has joined #nixos-dev
drakonis_ has quit [Ping timeout: 252 seconds]
drakonis1 has joined #nixos-dev
drakonis has quit [Ping timeout: 252 seconds]
orivej has joined #nixos-dev
jtojnar has joined #nixos-dev
orivej has quit [Ping timeout: 246 seconds]
sir_guy_carleton has quit [Quit: WeeChat 2.2]
orivej has joined #nixos-dev
orivej has quit [Ping timeout: 246 seconds]
orivej has joined #nixos-dev
orivej has quit [Ping timeout: 246 seconds]
<samueldr> so uh, sorry to nag you all; any ideas for the eval problem I caused?
<samueldr> (I will back out the changes and set limited support for the time being by my evening so trunk isn't held back for too long otherwise)
<clever> samueldr: got a link to the eval error?
<samueldr> clever: I made sure not to say error :)
<samueldr> it evals fine, unless it's low in memory
<clever> ahh
<clever> of note, hydra will flush the heap at regular intervals
<clever> so it needs less ram when compared to nix-instantiate
<samueldr> and turns out hydra's machine where eval takes place is in such a situation
<samueldr> oh
<samueldr> still has issues it looks like :)
<clever> one min
<samueldr> but good to know
<clever> run hydra-eval-jobs against nixos/release-combined.nix
<clever> no postgresql or config required
<clever> internally, hydra will fork out a child that does the bulk of the eval
<clever> and when the heap goes over a set limit, it will note the last attrpath, and restart the child
<samueldr> clever: time ~/Projects/nixos/hydra/result/bin/hydra-eval-jobs -I "$PWD" nixos/release-combined.nix # error: access to path '/nix/store/yy6cr71imd682dis2z2lsqdi255a9v9r-nixpkgs' is forbidden in restricted mode
<clever> 5 { nixpkgs ? { outPath = (import ../lib).cleanSource ./..; revCount = 56789; shortRev = "gfedcba"; }
<clever> you need to add a `--arg nixpkgs '{ outPath = ./.; revCount = 123; shortRev = "4567"; }'`
<clever> to make it not try to clean itself
<samueldr> --arg nixpkgs "$PWD" seems to do it
<clever> there will be a .git in the nixpkgs it embeds into all things, but it should work
<samueldr> if I get a bunch of json objects I guess it's right?
<clever> samueldr: that will be missing the shortrev, and break some things
<clever> samueldr: yeah, but some of those json's may have an error attr on them
<samueldr> yeah
<clever> the jq commands in check-hydra.sh review it for such errors
<samueldr> ah, I missed where you gave a solution
* samueldr restarts with the given solution
<clever> the real hydra, has a perl script called hydra-eval-jobset, which queries the psql db, fetches all inputs, and runs hydra-eval-jobs with the --arg's like above, for every configured input
<clever> with the real rev of the things it fetched
<samueldr> yeah, I assumed something along the lines :)
<clever> it will also parse the json, and generate builds in psql
<samueldr> (it's still running)
<clever> oh, also
<clever> samueldr: you may need to pass this via --option i think
* samueldr restarts with
<samueldr> I also used the initial heap size option
<clever> samueldr: you could also maybe file a PR to that repo to reduce it, which would make it use less ram, but restart more often
<clever> it may also use more cpu time, because it has to recalc things every time the heap is wiped clean
<samueldr> that's not a solution imho :/
<clever> its just one of many solutions
<clever> ,profile
<{^_^}> clever: Did you mean profiling?
<{^_^}> Use NIX_COUNT_CALLS=1 and/or NIX_SHOW_STATS=1 to profile Nix evaluation
<clever> this is another
<clever> show_stats includes the number of things created on the heap
<clever> so you could compare before/after
<samueldr> I'm guessing those stats are going to be dizzying to look at for <nixos/release-combined.nix>
<clever> its just the low-level types in nix, so it will only have big numbers, not a lot of them
<samueldr> I'm thinking more about the fact that I won't really know what to do with the info :)
jtojnar has quit [Read error: Connection reset by peer]
jtojnar has joined #nixos-dev
orivej has joined #nixos-dev
contrapumpkin has quit [Quit: My MacBook Pro has gone to sleep. ZZZzzz…]
copumpkin has joined #nixos-dev
obadz has joined #nixos-dev
copumpkin has quit [Ping timeout: 240 seconds]
<samueldr> took 7.46GiB of memory, and 37 minutes to do the limited support aarch64-linux eval
<gchristensen> it went down? haha
<samueldr> well, as said, hydra aggressively does something with those heap allocations
<samueldr> but wow did it spike the eval time
<samueldr> I think it was 4 minutes for a classic nix eval
<samueldr> well, nix-instantiate
<gchristensen> hydra has a magic evaluator
<samueldr> yeah, clever touched the subject
<gchristensen> it is regularly killed and restarted part way through
<samueldr> I'm now waiting on a supported aarch64-linux eval, see you in at least 37 minutes, maybe ~50?
<gchristensen> god speed
<Profpatsch> ekleog: For the rss2email module you can’t have multiple mails to send different feeds to, right? You can’t start two rss2email instances?
<samueldr> 7.98 GiB and 40 minutes, so it looks like it's not a huge increase :/
jtojnar_ has joined #nixos-dev
jtojnar has quit [Read error: Connection reset by peer]
jtojnar_ is now known as jtojnar
<samueldr> hm, I just looked at it quickly: it also fails here with `Too many heap sections: Increase MAXHINCR or MAX_HEAP_SECTS` using the hydra eval, so it's not a case of "the server not having enough ram"
<samueldr> (I missed the error message earlier while I was still working)
<clever> samueldr: oh, i have been able to make things pass before, by building nix without gc support
<clever> if gc is off, it just malloc's everything, and never frees it
<clever> with 64gig of ram, that has solved things that bumping the max heap size couldnt
<samueldr> I'm not sure this is something that will help with nixos' hydra :)
<clever> it may also make things worse
<clever> samueldr: oh, have you seen hydraJob?
<samueldr> I don't think so?
<clever> it strips out a lot of junk hydra doesnt need, and deepseq's the derivation
<clever> to reduce the amount of garbage the thunks indirectly hold a reference to
<clever> which improves GC
<samueldr> looks like it's already in use by <nixos/release-combined.nix> (as expected I guess)
<clever> yeah, just need to be aware of it, so when you add more jobs, you dont not-use-it
phreedom_ has joined #nixos-dev
phreedom has quit [Ping timeout: 256 seconds]