<gchristensen>
I was thinking I'd make this part of every ofborg run, but (1) github won't let you display SVGs inline (2) the ofborg evaluators don't have enough ram
<pie___>
hm
<pie___>
is it because its perl?
<gchristensen>
not sure
<gchristensen>
ma27: seems then that 60138 is ready to go?
<samueldr>
[18:51:40] -*- samueldr is interested to read more about the precise-gc branch
<clever>
samueldr: i have had memory leaks before that where "solved" by just building nix without GC
<clever>
the ram usage still sky-rocketed, but the GC no longer compained when i hit some arb number
jtojnar has quit [Read error: Connection reset by peer]
jtojnar_ has joined #nixos-dev
<samueldr>
I was thinking about the GC issues with too many heap sections beginning to show up with increasing frequency in hydra; and how if it helps with that we might be able to set aarch64-linux as a supported platform back
jtojnar_ is now known as jtojnar
<gchristensen>
samueldr:let's chat in about 15min
Synthetica has quit [Quit: Connection closed for inactivity]
<matthewbauer>
who is normally responsible for merging staging into staging-next? i can take that up if needed
<gchristensen>
matthewbauer: maybe send vcunat an email saying you'd be interested in helping?
<gchristensen>
part of that is also merging staging-next in to master :)
<matthewbauer>
gchristensen: ok nice. it looks like fridh normally is doing it, but i could at least open a pr from staging-next to master
<gchristensen>
matthewbauer: having more people doing it could only be a good thing
<gchristensen>
I'd love to see a larger group collaborating on it
pie___ has quit [Ping timeout: 258 seconds]
zarel has joined #nixos-dev
<samueldr>
gchristensen: I am now available for about an half an hour still if you have the time still
hedning__ has joined #nixos-dev
<gchristensen>
oh sure
<gchristensen>
moving up to my desk and I'll be back.
<gchristensen>
ok so I met with Eelco today about that patch
<gchristensen>
the idea is boehm's scanning means random data can look like a pointer, and can keep things alive unnecessarily -- this is part of why the fork is necessary in the hydra evaluator to successfully prune th egc.
<samueldr>
ooh
asymmetric has quit [Read error: Connection reset by peer]
<gchristensen>
another problem is nixpkgs is huge and everything points to everything, so it is very hard to find anything to GC anyway
<samueldr>
and I guess those compounds a bit
<gchristensen>
right
<gchristensen>
with a non-boehm, precise GC, Nix could keep the thunk and then a weak pointer to the evaluated thunk. when GC runs, it frees the other end of the weak pointer and if it is needed again, just re-evaluates it
<gchristensen>
initial testing with the precise-gc branch showed a reduction from about 7gb to 5gb of evaluation time memory, but he's expecting it to be much reduced
<gchristensen>
much more reduced* to the point that it could operate in sort of constant memory (not literally, but also not unbounded by the growth of nixpkgs)
<samueldr>
even if there was no reduce in memory use, depending on how the allocation and GC is done, I guess it would be an improvement if there is no _whatever caused the amount of heap sections_ to happen
<gchristensen>
right, so under ideal scenarios where there is no memory pressure, ideally it would never GC and just go really fast
<gchristensen>
but the idea of keeping the unevaluated thunk and a weak ref to the evaluated result is that under memory pressure it can prune old evaluated results as often as needed
<gchristensen>
and in a more application-specific, smarter way than Boehm can do based on its simple design
<aminechikhaoui>
question: do we allow IFDs in our current hydra.nixos.org ?
<gchristensen>
no
<aminechikhaoui>
any idea how do we disable that ? I want to do that on our private hydra
* gchristensen
takes this to a PM
apaul1729 has joined #nixos-dev
<clever>
gchristensen: ahhh, i was wondering how boehm found links
<clever>
gchristensen: the SRT bitmap is part of it
srhb has quit [Client Quit]
srhb has joined #nixos-dev
srhb has quit [Remote host closed the connection]
srhb has joined #nixos-dev
<gchristensen>
oh cool
<gchristensen>
that is neat!
drakonis has joined #nixos-dev
avn has joined #nixos-dev
<gchristensen>
hey everyone, I'm going to be hosting regular Nix ecosystem office hours starting next week. It'll be a video call where anyone can drop in and chat about the Nix ecosystem: cool PRs, problems, improvements, how to help each other to improve Nix. If you're interested, please tell me when you're available over at https://doodle.com/poll/aq9iytabi3k7z9da#calendar -- thank you!
asymmetric has joined #nixos-dev
drakonis has quit [Ping timeout: 246 seconds]
pie_ has joined #nixos-dev
<worldofpeace>
Maybe a discourse post for persistent visability?
<infinisil>
While the author there didn't really mean /lib specifically, I think it would still be useful
<infinisil>
I think I've used lib without nixpkgs before
<infinisil>
Oh yeah, for running the module system for a custom option set
<gchristensen>
if it is a just "I think" I'm not sure it is worth the headache, I feel it is quite complicated to split it out: makes API expectations different, and it may become much more easy to accidentally mix up versions
<infinisil>
It shouldn't be that hard. It's really just a mirror of a subdirectory. Not sure what you mean with API expectations, /lib is self-contained
<infinisil>
And it can be fully automatic, one-time setup
<samueldr>
I wonder about a plain mirror, for the expectation of purity outside nixpkgs
<samueldr>
something tested
<infinisil>
samueldr: (I have no idea what you mean by what you just said)
<samueldr>
if one copies lib outside of nixpkgs, how much of it will break since it relies on nixpkgs?
<samueldr>
(if it relies on nixpkgs)
<infinisil>
I'm pretty sure it doesn't rely on nixpkgs at all
<infinisil>
Like 99.9% sure
<samueldr>
right, then is it a property that's important? might a mirror of it be useful to test that? (though I guess there would be other ways to test that)
<infinisil>
Ah, there are some parts that rely on stuff outside /lib, like the maintainers list, which is in nixpkgs/maintainers now.. (previously it was in /lib) :/
<infinisil>
And nixpkgs/.version and nixpkgs/.version-suffix
<infinisil>
And the tests of lib also do..
<infinisil>
So much for that 99.9% lol, but this could be fixed
<infinisil>
Probably not worth it though
<samueldr>
is there value though? that's the other point that might be harder to gauge, the value has to outweigh the concerns even if it's just a mirror
<samueldr>
(though, might be a fun experiment for your sake to do and see how useful or not it is?)
<infinisil>
Yeah I don't have enough value out of it to justify trying to purify it
<infinisil>
This makes me think. lib.maintainers shouldn't really be in lib, it's got nothing to do with a library, it's just a registry for nixpkgs contributors
<samueldr>
through lazyness I guess most uses can be done anyway through referrencing "a nixpkgs", even if not ending up used with nixpkgs
<infinisil>
But it also doesn't fit in pkgs
<Profpatsch>
Since hydra doesn’t support IFD, we have to keep & maintain an exact duplicate. Not worth it imo
<infinisil>
Profpatsch: No, I don't mean to split nixpkgs, nixpkgs stays exactly as it is now
<infinisil>
Just *mirror* /lib
apaul1729 has quit [Remote host closed the connection]
zarel has quit [Quit: Leaving]
n_db has joined #nixos-dev
<ajs124>
What's the policy on dropping unmaintained/useless software from nixpkgs, if there is one? Specifically, I'm talking about evopedia, because I saw it while scolling over the list in #33248
<ajs124>
I used to use it back in the day, but it hasn't been developed for quite some time and I doubt there is up to date data that it can be used with.