justanotheruser has quit [Ping timeout: 244 seconds]
<elvishjerricco>
So I'm trying to bootstrap Nix on Ubuntu, then build a NixOS iso with it (using binary cache for now). But I get `install-info: Cannot allocate memory for gzip -d` at the end of `system-path.drv`
<elvishjerricco>
What could cause that?
<elvishjerricco>
(I think this is the right channel for such a question)
<elvishjerricco>
Nix builds fine though, which is sweet
<elvishjerricco>
And the cause definitely isn't running out of memory; it's got plenty
justanotheruser has joined #nixos-dev
drakonis- has joined #nixos-dev
drakonis has quit [Quit: WeeChat 2.9]
drakonis has joined #nixos-dev
drakonis- has quit [Client Quit]
drakonis- has joined #nixos-dev
drakonis has quit [Client Quit]
drakonis- is now known as drakonis
AlwaysLivid has joined #nixos-dev
orivej has joined #nixos-dev
alp has joined #nixos-dev
<Mic92>
elvishjerricco: is this the nix installer?
<elvishjerricco>
Mic92: No, I'm just doing ./configure and make with deps installed via apt. Testing out bootstrapping
<Mic92>
elvishjerricco: sorry no idea. bash -x ./configure and than maybe open a github issue.
<elvishjerricco>
Mic92: The build/install of nix works fine. It's just actually building the nixos iso from nixpkgs with this build results in this error
<elvishjerricco>
I doubt it's the actual build of Nix that's the issue. I bet it's some ubuntu compatibility thing
<Mic92>
elvishjerricco: Alright, are you using nix-daemon or single-user?
<elvishjerricco>
Mic92: Single user
<Mic92>
Maybe you can try nix-daemon to narrow down if it is due to that.
<Mic92>
I remember having broken coreutils tests with single-user for instance.
<Mic92>
Also I don't see how this should affect gzip
<elvishjerricco>
Mic92: Yea it's weird. Like it gets most of the way done building the iso, but then the system-path derivation fails with `install-info: Cannot allocate memory for gzip -d`, and googling that error returns zero results
<Mic92>
elvishjerricco: not even sure why it does gzip in that derivation. Are there info pages generated at this point?
<Mic92>
You could try to disable info pages
<elvishjerricco>
Yea no idea.
<Mic92>
documentation.info.enable = false;
<elvishjerricco>
I'm more concerned about the memory allocation thing than this specific derivation. Like why wouldn't it be able to allocate memory? I need to find other derivations that exhibit the same issue I guess
<elvishjerricco>
Huh, `--no-sandbox` seemed to fix it
<elvishjerricco>
I guess ubuntu is stricter on user namespaces or something?
cole-h has quit [Quit: Goodbye]
<Mic92>
NixOS does not have any kernel packages in this regard.
<Mic92>
*patches
<Mic92>
strace might help to figure out what install-info actually does.
<Mic92>
elvishjerricco: you can use https://github.com/Mic92/cntr to attach to the nix sandbox and run install-info from the inside
<elvishjerricco>
Mic92: Well I know there's a lot of controversy among distros about user namespaces, and a lot of distros patch the crap out of it. Ubuntu may be one of them
<Mic92>
elvishjerricco: ubuntu has it enabled for lxc afaik
<elvishjerricco>
Mic92: Yea but it might have different memory allocation rules or some kinda crap like that
<Mic92>
some applications seem to be built with it
<JJJollyjim>
on my system atom+vscode+element+geogebra are using it
<JJJollyjim>
yeah
<JJJollyjim>
there are electron5 and electron7 packages too
<Mic92>
I wonder how much memory this saves in the end.
<JJJollyjim>
oh yeah i guess it would have that benefit
<JJJollyjim>
with the binaries at least right?
<Mic92>
they are only mapped once than.
<JJJollyjim>
yeah nice
<jtojnar>
but I am starting to question whether it makes sense to have hermetic fontconfig configuration – we can just use unversioned config like other distros do and hope nothing breaks
<flokli>
jtojnar's approach might also be more compatible when we run nixpkgs-built applications on non-NixOS.
<flokli>
as their fontconfig probably needs to be compatible with all these versions as well
<Mic92>
probably
<jtojnar>
flokli: outside of NixOS, fontconfig from nixpkgs will use configs from the package
<drakonis>
i find it to be important to remember that nix app will be shipped with nix 3.0
<flokli>
jtojnar: can you elaborate?
<jtojnar>
see the config-compat.patch
<jtojnar>
it checks FONTCONFIG_FILE env var, if that does not exists versioned config in /etc, and if that does exist ${fontconfig}/etc/fonts/fonts.conf
<jtojnar>
never /etc/fonts/fonts.conf
<flokli>
jtojnar: but chromium and all electron apps at least won't have that custom logic
<jtojnar>
right
<flokli>
can't we make the nixos-provided /etc/fonts/fonts.conf compatible with all fontconfig flavors, and just use that consistently?
<jtojnar>
I am not sure that is possible
<flokli>
then we wouldn't need to manually wrap everything not using our fontconfig flavor with FONTCONFIG_FILE
<flokli>
I'd assume other distros would have the same problem, wouldn't they?
<jtojnar>
without degraded experience
<jtojnar>
other distros use single fontconfig (and bundled libs is unsupported), we can use packages from stable branches, which can have different fontconfig
alp has joined #nixos-dev
<flokli>
I assume for many electron apps, people use the binaries (or packages) provided by the ones producing them, so they might come with whatever bundled libraries
<jtojnar>
but maybe the incompatibilities are not so bad, fc 2.12.6 app on fc 2.13.92 system seems to print ton of errors but at least it displays text (as long as you generate fc cache manually)
<jtojnar>
and same thing the other way around
<jtojnar>
yeah, but then it is the responsibility of the app developer
<flokli>
deviating from the default location, and trying to chase every other bundled fontconfig library, then wrapping env vars sounds worse, than making some compromise on what funky stuff we set in the config
<flokli>
if all the stuff it does it if doesn't know the option is show a warning on the CLI
<jtojnar>
though, they typically say what platforms are tested (e.g. Ubuntu 12.04 or whatever their customers use) and do not support anythng else
<jtojnar>
flokli: we basically use the upstream fontconfig configs, nothing funky
<flokli>
in that case, removing the versioned patch, and just using /etc/fonts/fonts.conf sounds fine to me
<flokli>
it's less stuff to worry about, and IIRC, the patch was only required for some really old nasty fontconfig stuff, which is now gone
<Mic92>
Is there actually a reason our make does not use sh and /bin/sh instead?
<Mic92>
I see the point for glibc but not for make
justanotheruser has quit [Ping timeout: 246 seconds]
<jtojnar>
flokli: I will open a PR reverting the versioned config patch
FRidh has joined #nixos-dev
cole-h has joined #nixos-dev
<flokli>
👍
justanotheruser has joined #nixos-dev
<sphalerite>
Mic92: works for me
alp has quit [Ping timeout: 272 seconds]
AlwaysLivid has quit [Ping timeout: 272 seconds]
AlwaysLivid has joined #nixos-dev
rajivr has quit [Quit: Connection closed for inactivity]
<worldofpeace>
Mic92: yep
AlwaysLivid has quit [Remote host closed the connection]
orivej has joined #nixos-dev
kalbasit has joined #nixos-dev
kalbasit_ has joined #nixos-dev
FRidh has quit [Quit: Konversation terminated!]
alp has joined #nixos-dev
kalbasit__ has joined #nixos-dev
kalbasit_ has quit [Ping timeout: 240 seconds]
kalbasit__ has quit [Ping timeout: 240 seconds]
kalbasit has quit [Ping timeout: 240 seconds]
kalbasit_ has joined #nixos-dev
kalbasit has joined #nixos-dev
kalbasit__ has joined #nixos-dev
kalbasit_ has quit [Ping timeout: 265 seconds]
alp has quit [Ping timeout: 272 seconds]
cole-h has quit [Quit: Goodbye]
alp has joined #nixos-dev
urkk has quit [Ping timeout: 240 seconds]
drakonis1 has joined #nixos-dev
<kalbasit>
I was finally able to get a backtrace of the segfault that happens only using Nix flakes, same nixos configuration: https://github.com/NixOS/nix/issues/3931
<Mic92>
kalbasit: p $_siginfo._sifields._sigfault.si_addr
<Mic92>
gives you the address that was tried to access
<Mic92>
might be useful for the reviewer
<kalbasit>
sure
<kalbasit>
$1 = (void *) 0x13693000
<kalbasit>
I added that to the description of the issue. Anything else?
<Mic92>
kalbasit: maybe the memory mapping. I cannot remember the command right from my head. I have to look it up
<kalbasit>
Mic92: do you have the same version of Nix installed? I can probably guide you to reproduce and if it does will add them to the ticket as STR
<Mic92>
kalbasit: there is `info proc mappings`, but this might not work if run it on a coredump
<kalbasit>
from shabka dir run: nix-build --no-out-link -A external.nixpkgs.release-20-03.path
<Mic92>
kalbasit: Interesting enough your pointer is at the very end of the heap
<Mic92>
If you grep for 0x13693000 in info proc mappings
<kalbasit>
then run: sudo --preserve-env=RELEASE gdb --args nix-build {path_from_above}/nixos -A system -k -I nixos-config={path_to_dotshabka}/hosts/achilles/configuration.nix --show-trace
<Mic92>
I wonder why you need sudo here?
<kalbasit>
Mic92: I could only reproduce with sudo, not sure why
<kalbasit>
nixos-rebuild build works
<kalbasit>
but sudo does not
<Mic92>
ok
<kalbasit>
You should be able to reproduce with the above instructions, I've pushed everything to master branch
<Mic92>
Its evaluating now
<Mic92>
kalbasit: at which point did it crash?
<Mic92>
got it
<Mic92>
the crash
<kalbasit>
Mic92: :tada:
<kalbasit>
I added STR to the issue
<Mic92>
So looks like a heap overflow on the first glance
<kalbasit>
Mic92: I wonder why I can only reproduce with sudo, must have something to do with it
<Mic92>
Let me think
<Mic92>
kalbasit: it would probably work if boehmgc was disabled.
<clever>
ive done it before, and it just never reclaims ram while doing the eval
<kalbasit>
clever: will the nix daemon ever run OOM with this?
<clever>
the nix-daemon never does any evals
<clever>
the eval always happens as a client to the daemon, within nix-env/nix-instantiate/nix-build/nix build
<kalbasit>
clever: cool, thanks for the info
<clever>
the eval then creates the contents of each .drv file, which is passed to nix-daemon
<kalbasit>
Mic92: since I can't test/switch my configuration I'll have to reboot into an older one. Is it possible to avoid rebooting? i.e can I use nix stable for the rebuild?
<Mic92>
kalbasit: I would expect it to go out of memory in that case rather than doing a crash
<Mic92>
kalbasit: you could just add it to a shell.nix
<Mic92>
clever: was there not a way to get tracing information i.e. memory allocations from nix?
<clever>
,profiling Mic92
<Mic92>
--trace-function-calls is something
<{^_^}>
Mic92: Use NIX_COUNT_CALLS=1 and/or NIX_SHOW_STATS=1 to profile Nix evaluation
<Mic92>
ah
<clever>
NIX_SHOW_STATS counts the number of objects allocated
<Mic92>
OK it might work so if it crashes before
<clever>
NIX_SHOW_STATS can only work if the process exits cleanly
<clever>
its just printing a bunch of global vars to stderr
<Mic92>
kalbasit: right now I am not seeing a way how nix could use the api of boehmgc wrong. so it could be a bug in boehm gc..
<Mic92>
because void *p = GC_MALLOC(n); should for any input either return a valid pointer or NULL;
<Mic92>
kalbasit: you should try to disable nix-gitignore
<Mic92>
I see a lot of calls to that before it crashes
<kalbasit>
hmmm let me try that
<kalbasit>
Mic92: am I using it? I don't find anything grepping gitignore
<Mic92>
interesting
<kalbasit>
how do you see calls to i?
<kalbasit>
it*
<Mic92>
--trace-function-calls
<Mic92>
but it is a very busy graph
<Mic92>
if you want to search it, you likely want to write it to a file
<kalbasit>
it's huge :o
<kalbasit>
I always thought my config is bloated :)
<Mic92>
kalbasit: do you use poetry2nix?
<Mic92>
It seemed to be used there
<kalbasit>
Mic92: no I do not, although I was trying it out for work yesterday/today not sure if any of it snuck in
drakonis1 has quit [Quit: WeeChat 2.9]
<kalbasit>
I don't see anything grepping for it
<kalbasit>
Mic92: can we use dotgraph to visualize the --trace-function-calls output?
<Mic92>
kalbasit: there was something for flamegraphs.
<Mic92>
in the nix repo
<Mic92>
Ok. I am going to stop here. but it should give some pointers
evils has quit [Read error: Connection reset by peer]
<kalbasit>
Mic92: I have to go for a bit, I'll be back later. Thanks!
evils has joined #nixos-dev
alp has quit [Remote host closed the connection]
aszlig has quit [Quit: Kerneling down for reboot NOW.]