00:12
orivej has joined #nixos-dev
00:18
justanotheruser has quit [Ping timeout: 264 seconds]
00:31
orivej has quit [Ping timeout: 256 seconds]
00:33
justanotheruser has joined #nixos-dev
00:36
orivej has joined #nixos-dev
02:22
Scriptkiddi has quit [Quit: killed]
02:22
ajs124 has quit [Quit: killed]
02:22
das_j has quit [Quit: killed]
02:23
ajs124 has joined #nixos-dev
02:23
Scriptkiddi has joined #nixos-dev
02:23
das_j has joined #nixos-dev
02:28
lovesegfault_ has joined #nixos-dev
02:31
lovesegfault has quit [Ping timeout: 260 seconds]
02:59
justanotheruser has quit [Ping timeout: 260 seconds]
03:08
orivej has quit [Quit: No Ping reply in 180 seconds.]
03:08
orivej has joined #nixos-dev
03:10
justanotheruser has joined #nixos-dev
03:28
julm has quit [Remote host closed the connection]
03:29
julm has joined #nixos-dev
05:33
drakonis has quit [Quit: WeeChat 2.8]
05:34
lightbulbjim has joined #nixos-dev
06:02
cole-h has quit [Quit: Goodbye]
06:28
FRidh has joined #nixos-dev
07:03
alp has joined #nixos-dev
07:17
orivej has quit [Ping timeout: 260 seconds]
07:18
copumpkin has quit [Ping timeout: 272 seconds]
07:19
orivej has joined #nixos-dev
07:29
FRidh has quit [Ping timeout: 272 seconds]
07:44
lightbulbjim has quit [Quit: Connection closed for inactivity]
07:57
obadz has quit [Ping timeout: 246 seconds]
07:57
orivej has quit [Quit: No Ping reply in 180 seconds.]
07:57
obadz has joined #nixos-dev
07:58
orivej has joined #nixos-dev
08:07
obadz has quit [Ping timeout: 272 seconds]
08:08
obadz has joined #nixos-dev
08:15
alp has quit [Ping timeout: 244 seconds]
08:15
obadz has quit [Ping timeout: 256 seconds]
08:17
FRidh has joined #nixos-dev
08:17
obadz has joined #nixos-dev
08:32
orivej has quit [Ping timeout: 246 seconds]
08:32
orivej_ has joined #nixos-dev
08:35
justanotheruser has quit [Ping timeout: 272 seconds]
08:53
alp has joined #nixos-dev
09:03
Jackneill has quit [Ping timeout: 256 seconds]
09:10
<
yorick >
Mic92: seems like it uses the native python instead of the cross python to include/link
09:11
<
Mic92 >
yorick: Yes. This is also how I understand our cross python setup
09:16
Jackneill has joined #nixos-dev
09:42
<
manveru >
any ideas what changed about `amazonImage` that it now requires KVM to build? i get `qemu-system-x86_64: CPU model 'host' requires KVM`...
09:42
<
manveru >
it used to work without KVM, albeit quite slow
09:43
<
manveru >
now i can't build AMIs on EC2 anymore :P
09:45
Jackneill has quit [Ping timeout: 264 seconds]
09:47
orivej has joined #nixos-dev
09:47
orivej_ has quit [Ping timeout: 272 seconds]
09:58
Jackneill has joined #nixos-dev
09:59
<
{^_^} >
#83920 (by offlinehacker, 5 weeks ago, open): runInLinuxVM, test-driver: disable vmx cpu feature for guest vm
10:01
<
manveru >
ah, that seems helpful, thanks
10:03
<
Mic92 >
adisbladis: do you think it would be feasible to run some of our nixos tests with podman?
10:06
<
JJJollyjim >
I would love to see faster nixos test startups
10:07
<
JJJollyjim >
Idk how much room there is to tweak the guest kernel and stuff to make that happen in the current setup though
10:08
<
gchristensen >
how much do we value testing that the kernel actually works?
10:11
<
JJJollyjim >
I wonder if something like the Firecracker hypervisor might provide better performance while still testing the kernel
10:12
<
gchristensen >
yeah, good question
10:18
<
Mic92 >
JJJollyjim: I think the biggest performance drop right now might be from the use of 9p since it's is implemented as a userspace filesystem.
10:18
<
JJJollyjim >
Ooo did someone say virtiofs? :3
10:20
<
Mic92 >
JJJollyjim: cool I have not heard of that yet. Is host-native?
10:20
<
{^_^} >
#85568 (by volth, 3 weeks ago, open): runInLinuxVM, test-driver: investigate virtiofs as 9p replacement
10:20
<
JJJollyjim >
Might take up the issue if nobody else is
10:21
<
JJJollyjim >
Host native?
10:22
<
Mic92 >
JJJollyjim: Yes from the description the host server part is running in the kernel.
10:24
<
JJJollyjim >
Oh, I thought that was only the guest part
10:32
<
aszlig >
JJJollyjim: oh yeah, would be very good to get rid of 9p and our patches... less stuff to maintain
10:58
<
Mic92 >
JJJollyjim: I re-read, this is indead the case. It's performance comes from zero-copy
10:58
<
JJJollyjim >
it's hopefully faster without dax as well, because it at least skips going through the network layer
11:02
<
Mic92 >
JJJollyjim: is 9p actually using the network stack?
11:02
<
JJJollyjim >
oh i guess not with virtio
11:02
<
Mic92 >
I thought it behaves like a unix socket
11:03
<
Mic92 >
I don't see any reason not to enable dax.
11:07
<
aszlig >
Mic92: nope, it doesn't it's actually quite similar than virtiofs as far as i can see
11:08
<
aszlig >
similar in the way it's implemented, virtiofs uses the FUSE protocol, while 9p is using similar callbacks but it's not FUSE =)
11:08
<
Mic92 >
aszlig: so is this some sort of shared memory queue between qemu and guest kernel?
11:08
<
Mic92 >
For 9p I mean
11:11
<
{^_^} >
#87639 (by mitchellh, 10 hours ago, open): vimUtils: customize accepts alternate {g}vimExecutable value
11:11
<
aszlig >
Mic92: unfortunately not :-/
11:12
<
aszlig >
it's more like a proxy between VFS operations and a virtual 9p host
11:12
<
aszlig >
which of course introduces a bit of overhead
11:13
<
Mic92 >
aszlig: Yes. but the communication between qemu the the virtual 9p host should be done through shared memory, no?
11:14
<
Mic92 >
I can see why they want FUSE. The fuse protocol is inode based and less racy than 9p.
11:14
<
aszlig >
Mic92: i
*think* it doesn't, but let me check
11:20
__monty__ has joined #nixos-dev
11:21
<
aszlig >
so essentially, it's copies all over the place
11:23
<
JJJollyjim >
virtiofsd is trying to create /var/run/virtiofsd on startup
11:23
<
JJJollyjim >
with no obvious option to change that
11:23
<
JJJollyjim >
which is something we can work around as long as we're confident the more complex parts will work as non-root?
11:23
<
aszlig >
JJJollyjim: mhm, as said, we need to patch it as well, not only because of this, but also because of the sandbox init
11:24
<
aszlig >
seccomp BPF is okay, but capability setup, pivot_root and namespaces are tricky
11:25
<
JJJollyjim >
just because the build is non-root?
11:25
<
JJJollyjim >
or complex nixy reasons?
11:25
<
aszlig >
JJJollyjim: because we're using a user namespace in nix builds
11:25
<
Mic92 >
JJJollyjim: it is quite nice not having to run qemu as root in the first place.
11:25
<
aszlig >
so we already have a mapping from hostuid to nobody
11:25
<
aszlig >
s/host/parent namespace /
11:26
<
aszlig >
so if we want to make another user namespace to uid 0, this is not going to work
11:27
<
aszlig >
and it shouldn't since it would allow privilege escalation
11:27
<
JJJollyjim >
it looks like it doesn't make a namespace
11:27
<
JJJollyjim >
*a user namespace
11:27
<
JJJollyjim >
it uses CLONE_NEWPID | CLONE_NEWNS | CLONE_NEWNET
11:27
<
JJJollyjim >
so process/mount/net namespaces
11:27
<
JJJollyjim >
i.e. another thing that's not gonna work as non-root :)
11:28
<
aszlig >
JJJollyjim: look into build.cc
11:28
<
aszlig >
JJJollyjim: ah, wait, you mean virtiofsd?
11:28
<
Mic92 >
Yeah. I am surprised it re-invents systemd hardening options here
11:28
<
JJJollyjim >
yeah aszlig
11:29
<
aszlig >
JJJollyjim: ah, yeah, then you're right, but you need to be "root" (even namespaces) for that
11:29
<
Mic92 >
JJJollyjim: let me have a look at the fuse part. This might also require root
11:29
<
JJJollyjim >
virtiofsd man page plainly states "This program must be run as the root user"
11:29
<
JJJollyjim >
i wonder what the limitations are
11:29
<
JJJollyjim >
Mic92: ty
11:29
<
Mic92 >
JJJollyjim: when I was implementing cntr I also needed root
11:29
<
aszlig >
Mic92: the FUSE part shouldn't require root, since it only implements the FUSE protocol, not an actual FUSE fs
11:30
<
Mic92 >
aszlig: no not because of the FUSE part
11:30
<
Mic92 >
aszlig: I think they might want DAC_OVERRIDE in virtiofsd
11:31
<
aszlig >
Mic92: they do
11:31
<
aszlig >
see setup_capabilities()
11:33
<
aszlig >
Mic92: however, i don't see why it couldn't be emulated
11:33
<
JJJollyjim >
can the dax stuff work as non-root?
11:33
<
Mic92 >
aszlig: We don't need DAC_OVERRIDE for our use case
11:34
<
JJJollyjim >
^virtio-fs without dax is marginally faster than virtio-9p, but it's not massive
11:35
<
Mic92 >
Yeah otherwise 9p and virtio-fs are not that different.
11:39
<
JJJollyjim >
okay, i guess i'll leave this for now while people smarter than me find out if it's possible :)
11:42
<
Mic92 >
JJJollyjim: I don't see why dax should not work. It's implemented in guest kernel.
11:43
<
JJJollyjim >
how does the virtiofsd provide access to the host kernel's page cache though?
11:43
<
JJJollyjim >
right it looks like the qemu process mmaps the file
11:44
<
JJJollyjim >
i think that makes sense
11:45
<
Mic92 >
JJJollyjim: yes. one can achieve the same by using DIRECT_IO when opening file
11:45
<
JJJollyjim >
doesn't direct_io entirely skip the page cache?
11:47
<
Mic92 >
JJJollyjim: right, it basically inverts sharing, the kernel space borrows pages for writing from userspace.
11:48
<
JJJollyjim >
oddly i can't find the original post in the mailing list archive
11:49
<
JJJollyjim >
if i understand that patch won't work
11:49
<
JJJollyjim >
but it verifies that someone has things working in non-root mode :)
11:50
alp has quit [Ping timeout: 260 seconds]
11:52
<
JJJollyjim >
oh except they're root inside the userns
11:52
<
JJJollyjim >
so maybe it still wouldn't work for us
11:53
<
JJJollyjim >
But I guess we think it will
11:53
<
Mic92 >
JJJollyjim: So I read the fuse code a bit. They actually use splice() from libfuse to move page cache pages to userpace.
11:53
<
JJJollyjim >
I'll poke at getting it running tomorrow
11:54
<
Mic92 >
So basically what need is skipping this sandbox buisness and forcing uid 0.
11:54
<
Mic92 >
aszlig: Would it not make sense if we fix the uid problem in nix itself? We could map uid 0 properly in the sandbox.
11:55
<
Mic92 >
Or does this not work if nix runs as unprivileged sandbox?
11:56
alp has joined #nixos-dev
11:56
<
aszlig >
Mic92: this unfortunately doesn't work, since we'd need to run the builder sandbox as root
11:56
<
aszlig >
Mic92: we can't simply map files to other users which we don't own
11:56
<
aszlig >
s/files/file uids/
11:57
<
aszlig >
or at least, run it as root, map the uids and then drop privileges, than could work in theory
11:58
<
aszlig >
it's been a while since i've dug into that, but IIRC there were a few problems with the run-as-root-drop-priviliges way
12:03
tv1 has quit [Quit: WeeChat 2.6]
12:03
tv has joined #nixos-dev
12:08
Jackneill has quit [Ping timeout: 260 seconds]
12:19
Jackneill has joined #nixos-dev
12:39
orivej has quit [Quit: No Ping reply in 180 seconds.]
12:40
orivej has joined #nixos-dev
12:45
<
Mic92 >
aszlig: I think in nix-user-chroot, I solved this problem.
12:45
<
Mic92 >
for nix-user-chroot itself.
13:03
<
aszlig >
Mic92: ah, just did a quick look on it, but isn't that doing the same as nix already does?
13:03
<
aszlig >
it just maps a single uid from the parent/child namespace
13:05
<
aszlig >
so i think we'd have the same problem here as well
13:06
<
aszlig >
s/already/also/
13:08
<
Mic92 >
aszlig: I think you are right
13:10
<
Mic92 >
aszlig: However I think to remember that it was possible to map multiple uids
13:13
<
yorick >
Mic92: well, obviously it can't link to the host python when cross-compiling, so it fails
13:14
<
yorick >
I'm currently hacking cflags to make it find the cross python
13:15
<
aszlig >
Mic92: it is, but IIRC you need to be root for that
13:16
<
aszlig >
or more specifically need to have CAP_SETUID
13:19
<
Mic92 >
aszlig: makes sense sort of.
13:19
<
Mic92 >
I am still waiting for bindfs to happen
13:21
<
aszlig >
Mic92: you mean bind mounts as FUSE?
13:23
<
Mic92 >
aszlig: There was bindfs kernel implementation that does uid shifts.
13:24
<
Mic92 >
aszlig: right, it was called shifts
13:24
<
Mic92 >
Also this would not solve the problem for us, since we cannot mount it in nix.
13:38
orivej has quit [Quit: No Ping reply in 180 seconds.]
13:39
orivej has joined #nixos-dev
13:48
qyliss has quit [Quit: bye]
13:51
qyliss has joined #nixos-dev
14:21
<
emily >
I feel like much of the tests don't need to be running kernels/VMs at all
14:21
<
emily >
it's valuable to test that stuff but it feels like there's a lot more userspace tests that would work fine in regular containers than there are full "VM tests" poking at kernel stuff
14:22
<
emily >
it would be nice not to run the kernels if only so that the test output was vaguely readable >_>
14:25
<
aszlig >
yeah, i think there was some work a while ago to implement this, since it needs to have some help from nix itself
14:25
<
aszlig >
well, you could still silence the kernel messages :-D
14:26
<
aszlig >
and the "help from nix"[TM] would pretty much boil down to the user namespace issue above
14:31
<
aszlig >
and also this would introduce a few more headaches, like for example you suddenly need to take care of networking instead of going through a user space switch
14:31
cole-h has joined #nixos-dev
14:32
<
aszlig >
the networking part should be manageable at some point, but how would you integrate it into nix?
14:32
<
aszlig >
like just some big privilege elevation escape hatch or tailored towards container runs?
14:33
<
aszlig >
and how do you prevent other derivations from abusing this to eg. escape the sandbox?
14:34
<
aszlig >
something like CSP on the web?
14:34
<
aszlig >
special build mode? what about hydra then?
14:37
<
MichaelRaskin >
There is a ton of things that need just a couple things running (service supervision not really needed) and definitely no networking and no kernel.
14:38
<
gchristensen >
someone might want to extend hydra to make this a configurable
14:38
<
{^_^} >
#86166 (by Ericson2314, 2 weeks ago, open): *-wrapper; Switch from `infixSalt` to `suffixSalt`
14:38
<
Ericson2314 >
looking to make a pkg-config wapper after it
14:42
<
aszlig >
gchristensen: geesh --no-allow-... sounds so odd (and yes, I know it's because --no-* is handled in a generic way)
14:42
<
aszlig >
gchristensen: configurable on a per-jobset basis?
15:17
FRidh has quit [Ping timeout: 256 seconds]
15:18
FRidh has joined #nixos-dev
15:32
<
cole-h >
It's beautiful. nixos-unstable is unstuck. niksnut++
15:32
<
{^_^} >
niksnut's karma got increased to 17.00000000000000004
16:02
evanjs- has joined #nixos-dev
16:02
orivej has quit [Ping timeout: 246 seconds]
16:04
evanjs has quit [Ping timeout: 272 seconds]
16:09
justanotheruser has joined #nixos-dev
16:10
orivej has joined #nixos-dev
16:10
<
julm >
floating karma, is... floating
16:20
drakonis has joined #nixos-dev
16:21
alp has quit [Ping timeout: 260 seconds]
16:25
alp has joined #nixos-dev
16:44
alp has quit [Remote host closed the connection]
16:44
alp has joined #nixos-dev
16:47
<
Profpatsch >
I want karma to rise exponentially, similar to power levels
16:47
<
Profpatsch >
By a random value between 2^(n-1) and 2^(n)
16:55
drakonis has quit [Ping timeout: 244 seconds]
16:58
orivej has quit [Read error: Connection reset by peer]
16:59
orivej has joined #nixos-dev
17:07
janneke has quit [Quit: janneke quits Mes'sing]
17:08
janneke has joined #nixos-dev
17:17
<
gchristensen >
adisbladis: I'm looking at ways to do that automatically (take the options from the Nix modules) and output structural json, but am tripping myself up. maybe you have some ideas with your recent spelunking
17:18
<
clever >
gchristensen: nixos options?
17:19
<
gchristensen >
NixOps specifically, but yaeh
17:20
alp has quit [Remote host closed the connection]
17:20
alp has joined #nixos-dev
17:23
<
clever >
# nix-build -o analyzer ; nix-build '<nixpkgs/nixos/release.nix>' -A options -o options ; ./analyzer/bin/option-analyzer ./options/share/doc/nixos/options.json
17:23
<
clever >
gchristensen: this is how the options page on the site got its data
17:23
<
gchristensen >
hmm yeah
17:24
<
clever >
gchristensen: and how my installer gui would generate it from whatever <nixpkgs> currently is, so its always up to date
17:24
<
gchristensen >
hmm!
17:24
<
clever >
135 options = (buildFromConfig ({ ... }: { }) (config: config.system.build.manual.optionsJSON)).x86_64-linux;
17:25
<
clever >
nixos/lib/make-options-doc/default.nix: optionsJSON = pkgs.runCommand "options.json"
17:25
<
clever >
gchristensen: so just add s3-bucket.nix to the imports of your thing (maybe even -I nixos-config=?) and build that attr of nixos
17:26
<
clever >
gchristensen: oh, theres even an example at the top of nixos/lib/make-options-doc/default.nix
17:44
<
infinisil >
Profpatsch: I like that idea
17:52
orivej has quit [Quit: No Ping reply in 210 seconds.]
17:52
teto has joined #nixos-dev
17:53
orivej has joined #nixos-dev
18:19
evanjs has joined #nixos-dev
18:22
drakonis has joined #nixos-dev
18:31
alp has quit [Remote host closed the connection]
18:31
alp has joined #nixos-dev
18:44
julm has quit [Remote host closed the connection]
18:45
julm has joined #nixos-dev
19:21
alp has quit [Ping timeout: 260 seconds]
19:23
orivej has quit [Ping timeout: 256 seconds]
19:24
orivej has joined #nixos-dev
19:58
alp has joined #nixos-dev
20:06
orivej has quit [Ping timeout: 264 seconds]
20:20
lovesegfault_ has quit [Quit: WeeChat 2.8]
20:20
orivej has joined #nixos-dev
20:29
lovesegfault has joined #nixos-dev
20:53
orivej has quit [Ping timeout: 256 seconds]
21:07
orivej has joined #nixos-dev
21:15
qyliss has quit [Quit: bye]
21:17
qyliss has joined #nixos-dev
21:52
orivej has quit [Ping timeout: 256 seconds]
21:59
kraem has quit [Ping timeout: 272 seconds]
22:06
orivej has joined #nixos-dev
22:07
alp has quit [Ping timeout: 260 seconds]
22:12
FRidh has quit [Quit: Konversation terminated!]
22:14
__monty__ has quit [Quit: leaving]
22:41
kraem has joined #nixos-dev
22:47
justanotheruser has quit [Ping timeout: 256 seconds]
23:01
justanotheruser has joined #nixos-dev