<multun>
I'm don't know about the internals, but I guess something like .pyc files for python would help as well
<multun>
or even better, can the loaded nixpkgs be serialized and cached ?
<gchristensen>
I go log in to look at why those builds are stuck and they resolve, lol
<simpson>
The interpretative overhead of Nix is pretty bad, isn't it? I'm not sure that there's a suitable JIT toolkit for Nix, but if there were, then that would be a solution.
<simpson>
(LLVM's is bad, RPython requires Python 2 to build and has runtime libs, Graal is owned by Oracle, libfirm can't actually JIT?, libgccjit has runtime libs, GNU Lightning maaaaybee?)
<Profpatsch>
simpson: Is JIT a good solution for something that’s not long-running?
<Profpatsch>
Or at least shouldn’t be long-running
<yorick>
simpson: I was looking at rpython, yeah
<yorick>
Profpatsch: it'll work for things that repeat a lot, like stdenv.mkDerivation
<simpson>
I don't know; I'm only going on rumors. I haven't done the profiling myself. But yeah, even processes as short as 1s in length can benefit from 1ms of JIT if it removes 2x or more of the overhead.
<puck>
also, you can cache the jit results between evaluations if the file doesn't change
<yorick>
simpson: I think pypy is all python 3 now?
<simpson>
yorick: The main problem with RPython is one that Nix neatly solves: We need to maintain a PyPy Python 2.7.x toolchain indefinitely. Not a problem; upstream is going to do this, so we just have to provide the build rules.
<yorick>
simpson: why do you need that?
<simpson>
RPython is Python 2.7, and that's never going to change.
<yorick>
luajit
<yorick>
simpson: huh? why?
<simpson>
The secondary problem is Nix-specific: We want to minimize the shared-library footprint, in order to minimize the bootstrap and the amount of implicitly-trusted code.
<yorick>
simpson: nix has about 60 runtime dependencies
<simpson>
yorick: Speaking informally, and not as a PyPy core contributor: There's no manpower or money available to do the port to Python 3, and nobody really cares because Python 3 doesn't have any compelling features to add to RPYthon.
<simpson>
puck: If we did *that*, then I humbly suggest that we do something closer to per-module AOT compilation. JIT toolkits don't really support that use case. But I gather that using Nix in non-AOT ways is a big design goal.
<yorick>
dependency injecting `lib`, containing things like loops, into modules can make them hard to optimize AOT
<simpson>
Yep. But the small size of the core language, combined with the pre-existing analysis of the Nix store, is promising for JIT. We may even be able to reflect purity and caching in the store into the JIT's understanding of derivations.
<FRidh>
simpson: rpython is a very limited language. You can't say whether it is 2 or 3, because there is no stdlib and the language itself is more limited than either 2 or 3
<FRidh>
though maybe strings are still bytes, I don't know about that
<simpson>
FRidh: It is Python 2.7; the translation process uses a Python 2.7 interpreter (CPython or PyPy) to import the target's modules, executing arbitrary Python 2 code. After import, the modules' bytecode is then broken down and analyzed to create the JIT.
<simpson>
There *is* a stdlib, rpython.rlib, and while it is limited in some ways, it also has many useful tools common to building programming languages. It's not impossible, but does require a couple person-years of effort.
justanotheruser has quit [Quit: WeeChat 2.6]
cole-h has joined #nixos-dev
cole-h has quit [Client Quit]
justanotheruser has joined #nixos-dev
ixxie has joined #nixos-dev
<andi->
With current Nix master I can't eval the tested job nixpkgs master (without changing GC settings) but with the older "84de821" I can... Maybe we also have a regression in Nix?
cole-h has joined #nixos-dev
<yorick>
andi-: sounds like a good candidate for bisecting all night
justanotheruser has quit [Read error: Connection reset by peer]
psyanticy has joined #nixos-dev
claudiii has quit [Quit: Connection closed for inactivity]
justanotheruser has joined #nixos-dev
drakonis has joined #nixos-dev
lovesegfault has quit [Quit: WeeChat 2.7]
justanotheruser has quit [Read error: Connection reset by peer]
Jackneill has quit [Remote host closed the connection]
orivej has quit [Ping timeout: 240 seconds]
ris has joined #nixos-dev
justanotheruser has joined #nixos-dev
lovesegfault has joined #nixos-dev
<lovesegfault>
Is Hydra okay?
<lovesegfault>
Is it still doing nightly evals of trunk-combined?
<niksnut>
there are some OOM issues that we're working on
<lovesegfault>
Uh-oh
orivej has joined #nixos-dev
claudiii has joined #nixos-dev
justanotheruser has quit [Read error: Connection reset by peer]
Jackneill has joined #nixos-dev
Jackneill has quit [Remote host closed the connection]
psyanticy has quit [Quit: Connection closed for inactivity]
justanotheruser has joined #nixos-dev
edwtjo has quit [Ping timeout: 268 seconds]
justanotheruser has quit [Ping timeout: 246 seconds]