<samueldr>
8.12GB with aarch64-linux in limitedSupport (waiting to see how it compares to previously with it entirely removed)
asymmetric has quit [Ping timeout: 246 seconds]
<gchristensen>
(that number will be good to know)
<samueldr>
eh, got 8.32 but let's chalk it up to being equivalent limited support is about equal to not adding it at all
<gchristensen>
interesting
<samueldr>
limitedSupportedSystems doesn't do much though
<gchristensen>
right
<gchristensen>
means build build the absolute minimum closure
<samueldr>
though we would be able to *also* pick some attributes from aarch64-linux to start off with, like the iso... which I forgot to add in the initial PR oops
ma27 has quit [Quit: WeeChat 2.2]
ma27 has joined #nixos-dev
Jackneill has quit [Remote host closed the connection]
pie__ has joined #nixos-dev
pie___ has quit [Remote host closed the connection]
init_6 has joined #nixos-dev
init_6 has quit [Ping timeout: 246 seconds]
__Sander__ has quit [Quit: Konversation terminated!]
orivej has quit [Ping timeout: 250 seconds]
jtojnar has quit [Remote host closed the connection]
<clever>
internally, hydra will fork out a child that does the bulk of the eval
<clever>
and when the heap goes over a set limit, it will note the last attrpath, and restart the child
<samueldr>
clever: time ~/Projects/nixos/hydra/result/bin/hydra-eval-jobs -I "$PWD" nixos/release-combined.nix # error: access to path '/nix/store/yy6cr71imd682dis2z2lsqdi255a9v9r-nixpkgs' is forbidden in restricted mode
<clever>
you need to add a `--arg nixpkgs '{ outPath = ./.; revCount = 123; shortRev = "4567"; }'`
<clever>
to make it not try to clean itself
<samueldr>
--arg nixpkgs "$PWD" seems to do it
<clever>
there will be a .git in the nixpkgs it embeds into all things, but it should work
<samueldr>
if I get a bunch of json objects I guess it's right?
<clever>
samueldr: that will be missing the shortrev, and break some things
<clever>
samueldr: yeah, but some of those json's may have an error attr on them
<samueldr>
yeah
<clever>
the jq commands in check-hydra.sh review it for such errors
<samueldr>
ah, I missed where you gave a solution
* samueldr
restarts with the given solution
<clever>
the real hydra, has a perl script called hydra-eval-jobset, which queries the psql db, fetches all inputs, and runs hydra-eval-jobs with the --arg's like above, for every configured input
<clever>
with the real rev of the things it fetched
<samueldr>
yeah, I assumed something along the lines :)
<clever>
it will also parse the json, and generate builds in psql
<clever>
samueldr: you may need to pass this via --option i think
* samueldr
restarts with
<samueldr>
I also used the initial heap size option
<clever>
samueldr: you could also maybe file a PR to that repo to reduce it, which would make it use less ram, but restart more often
<clever>
it may also use more cpu time, because it has to recalc things every time the heap is wiped clean
<samueldr>
that's not a solution imho :/
<clever>
its just one of many solutions
<clever>
,profile
<{^_^}>
clever: Did you mean profiling?
<{^_^}>
Use NIX_COUNT_CALLS=1 and/or NIX_SHOW_STATS=1 to profile Nix evaluation
<clever>
this is another
<clever>
show_stats includes the number of things created on the heap
<clever>
so you could compare before/after
<samueldr>
I'm guessing those stats are going to be dizzying to look at for <nixos/release-combined.nix>
<clever>
its just the low-level types in nix, so it will only have big numbers, not a lot of them
<samueldr>
I'm thinking more about the fact that I won't really know what to do with the info :)
jtojnar has quit [Read error: Connection reset by peer]
jtojnar has joined #nixos-dev
orivej has joined #nixos-dev
contrapumpkin has quit [Quit: My MacBook Pro has gone to sleep. ZZZzzz…]
copumpkin has joined #nixos-dev
obadz has joined #nixos-dev
copumpkin has quit [Ping timeout: 240 seconds]
<samueldr>
took 7.46GiB of memory, and 37 minutes to do the limited support aarch64-linux eval
<gchristensen>
it went down? haha
<samueldr>
well, as said, hydra aggressively does something with those heap allocations
<samueldr>
but wow did it spike the eval time
<samueldr>
I think it was 4 minutes for a classic nix eval
<samueldr>
well, nix-instantiate
<gchristensen>
hydra has a magic evaluator
<samueldr>
yeah, clever touched the subject
<gchristensen>
it is regularly killed and restarted part way through
<samueldr>
I'm now waiting on a supported aarch64-linux eval, see you in at least 37 minutes, maybe ~50?
<gchristensen>
god speed
<Profpatsch>
ekleog: For the rss2email module you can’t have multiple mails to send different feeds to, right? You can’t start two rss2email instances?
<samueldr>
7.98 GiB and 40 minutes, so it looks like it's not a huge increase :/
jtojnar_ has joined #nixos-dev
jtojnar has quit [Read error: Connection reset by peer]
jtojnar_ is now known as jtojnar
<samueldr>
hm, I just looked at it quickly: it also fails here with `Too many heap sections: Increase MAXHINCR or MAX_HEAP_SECTS` using the hydra eval, so it's not a case of "the server not having enough ram"
<samueldr>
(I missed the error message earlier while I was still working)
<clever>
samueldr: oh, i have been able to make things pass before, by building nix without gc support
<clever>
if gc is off, it just malloc's everything, and never frees it
<clever>
with 64gig of ram, that has solved things that bumping the max heap size couldnt
<samueldr>
I'm not sure this is something that will help with nixos' hydra :)