supersandro2000 has quit [Disconnected by services]
supersandro2000 has joined #nix-darwin
ajs124 has left #nix-darwin ["User left"]
<supersandro2000> Is nix daemon broken on catalina?
<supersandro2000> I think the sandbox prevents the daemon from launching and after a reboot the nix store is empty
<thefloweringash> I've been using catalina with the nix-daemon since catalina was released. works for me, but there might be some cases I haven't hit
<supersandro2000> file system sandbox blocked open() of '/nix/store/i1cg0wfns9j4lmfzvx5dz6rc436vs6ms-libsodium-1.0.18/lib/libsodium.23.dylib'
<supersandro2000> and after doing a reboot /nix is empty
<supersandro2000> I run the installer at least 30 times already today...
<thefloweringash> this seems like two problems. for /nix being empty, assuming a apfs setup, are the contents of the volume removed or is the volume unmounted? for the sandbox, where's that error coming from? why is there a sandbox active?
<thefloweringash> I do use nix-darwin which provides its own version of the plist for the daemon, so that might be different to nix itself. though skimming on github I don't see much of a difference
stephank has quit [Quit: stephank]
<supersandro2000> I am honest I have no clue about apfs
stephank has joined #nix-darwin
<supersandro2000> probably because it is not deactivated in the /etc/nix/nix.conf
<supersandro2000> which I did not touch
<supersandro2000> I've disabled the nix sandbox in nix.conf and still the same error
<thefloweringash> I think that only applies to builds, not the daemon itself
<thefloweringash> I tried to install a catalina in a vm to follow along at home, but the installation isn't succeeding
<thefloweringash> is this a single user or multi user install?
supersandro2000 has quit [Quit: The Lounge -]
supersandro2000 has joined #nix-darwin
supersandro2000 has quit [Quit: The Lounge -]
supersandro2000 has joined #nix-darwin
cransom has quit [Ping timeout: 260 seconds]
supersandro2000 has quit [Quit: The Lounge -]
supersandro2000 has joined #nix-darwin
cransom has joined #nix-darwin
Chiliparrot has joined #nix-darwin
Chiliparrot has quit [Ping timeout: 260 seconds]
Chiliparrot has joined #nix-darwin
heywoodlh has quit [Quit: ZNC 1.8.2 -]
Chiliparrot has quit [Ping timeout: 246 seconds]
Chiliparrot has joined #nix-darwin
__monty__ has joined #nix-darwin
hedgie has quit []
das_j has quit [Ping timeout: 272 seconds]
stephank has quit [Ping timeout: 272 seconds]
das_j has joined #nix-darwin
philr has quit [Ping timeout: 260 seconds]
<LnL> urgh! why the hell does the flake build suddenly timeout now :/
hedgie has joined #nix-darwin
stephank has joined #nix-darwin
<supersandro2000> strange. When running the nix-daemon in a terminal it just works ™️
<supersandro2000> when it is a service and run via launchctl I get this error "file system sandbox blocked open() of '/nix/store/i1cg0wfns9j4lmfzvx5dz6rc436vs6ms-libsodium-1.0.18/lib/libsodium.23.dylib'"
<supersandro2000> Does this file need an update?
<supersandro2000> also launchctl is horrible compared to systemd
<thefloweringash> I’ve seen something like that with lorri as a user service.
<thefloweringash> Haven’t seen it with the nix-daemon as a system service though.
<supersandro2000> Do you know if there is a solution for lorri I could try out?
<supersandro2000> or do you have any idea how to fix that. I have totally no clue about osx services
cptrbn has joined #nix-darwin
<abathur> did you encrypt your volume?
<supersandro2000> nope
<abathur> and when you reboot, /nix is empty?
<abathur> and if so, is nix in /etc/fstab? listed in the output of 'mount'? does it mount if you run `mount -a`?
<abathur> did the installer make your Nix volume, or are you creating it?
cptrbn has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
cptrbn has joined #nix-darwin
<abathur> I don't have a spare Catalina system to play with, but I just tested clean nix 2.3.8 installs (both --daemon and single-user) on Big Sur and they install fine, run something simple like cowsay, mount the volume on reboot, and still run something simple like cowsay
<abathur> also, is your Nix volume on your primary internal system drive?
<abathur> I doubt there's a general catalina break here; any CI jobs on github actions using install-nix-action that specify macos-latest should be running in Catalina on --daemon installs
<gchristensen> I nee to see about getting catalina in a VM ...
johnw has quit [Quit: ZNC -]
<LnL> does the qemu repo have an apfs setup?
Chiliparrot has quit [Ping timeout: 260 seconds]
<supersandro2000> '/nix is empty after the reboot, it is in /etc/fstab, it mounts after I remount it in the disk util app
<supersandro2000> did not try mount -a
<supersandro2000> first the installer but that files and then I used the installer guide
Chiliparrot has joined #nix-darwin
<supersandro2000> yeah, no external drive involved
<gchristensen> OSX-KVM is what I've used in the past
<gchristensen> (OSX-KVM is how the nixos foundation operates all its macs)
<LnL> yeah I know
<LnL> but the last version I tried still used hfs+
* LnL has no idea if 10.14 can still run without apfs
<supersandro2000> mount_apfs: volume could not be mounted: Operation not permitted
<supersandro2000> mount_apfs: volume could not be mounted: No such file or directory
<supersandro2000> mount: / failed with 77
<supersandro2000> mount: /nix failed with 66
<supersandro2000> mount -a does not work
<supersandro2000> LnL: uses apfel fs
<supersandro2000> *APFS
<abathur> is nix in /etc/synthetic.conf?
<supersandro2000> yes
<abathur> what does `stat /dix --format="%F"` say?
<supersandro2000> I think I found a solution but I am not sure if I tested it correct
<abathur> oops nix
<abathur> stat /nix --format="%F"
<supersandro2000> doing `LABEL=Nix\040Store /nix apfs rw,nobrowse,auto` in /etc/fstab did work ™️
<abathur> how is that different than what was already there?
<supersandro2000> 😂 the packages in default darwin are so damn old: stat: --format=%F: stat: No such file or directory
<supersandro2000> the installer does not add ,auto
<abathur> oh, sorry, I'm using gnu stat I guess
<supersandro2000> and my other machine uses the default and just works
<supersandro2000> don't worry. darwin utils are generally a pain to use
<abathur> I just wanted to see what type it said
<abathur> feel free to use stat /nix :)
<supersandro2000> their base64 needs - as a file input when used between two pipes...
<supersandro2000> '/stat nix
<supersandro2000> File: /nix
<supersandro2000> Access: (0775/drwxrwxr-x) Uid: ( 0/ root) Gid: ( 80/ admin)
<supersandro2000> Device: 100000eh/16777230d Inode: 2 Links: 7
<supersandro2000> Size: 224 Blocks: 0 IO Block: 4096 directory
<abathur> and that's after you've manually mounted it, or freshly rebooted?
<supersandro2000> wrong ssh session. that is gnu stat on the other mac
<supersandro2000> 16777224 2 drwxr-xr-x 5 root wheel 0 160 "Nov 5 00:59:51 2020" "Nov 5 00:57:30 2020" "Nov 5 00:57:30 2020" "Nov 4 09:15:15 2020" 4096 0 0 /nix
<supersandro2000> that is the output after adding auto to fstab
<supersandro2000> and a fresh reboot
<supersandro2000> maybe I changed some default while trying to debug this and now it does not do automount
<abathur> ah I guess like `/usr/bin/stat -x /nix`
<abathur> there's something weird here yes, we generally need `noauto` to stop it from auto-mounting I think
<abathur> I do recall reading something about the behavior differing by volume "role" so that might be, but I don't think that should be the case by default
<abathur> i.e. as the installer would create it
<supersandro2000> File: "/nix"
<supersandro2000> Size: 160 FileType: Directory
<supersandro2000> Device: 1,9 Inode: 2 Links: 5
<supersandro2000> Mode: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ wheel)
<supersandro2000> what is a volume role and how can I check that?
<supersandro2000> also the daemon still complains about the darwin sandbox when started as a service
<abathur> if you run `diskutil apfs list` and look for the line `APFS Volume Disk (Role):`
<abathur> for example mine is "APFS Volume Disk (Role): disk1s6 (No specific role)"
<supersandro2000> APFS Volume Disk (Role): disk1s6 (No specific role)
<abathur> hmm
<supersandro2000> and when I manually start the daemon in my ssh session I get when trying to create a nix-shell: libc++abi.dylib: terminating with uncaught exception of type nix::Interrupted: interrupted by the user
<supersandro2000> Do you think it could be that the VM uses opencore to boot?
<abathur> not sure
<abathur> with the asterisk that AFAIK we're still a little fuzzy on exactly which macOS component is even triggering the auto-mount, I'm curious what the value of AUTOMOUNTD_MNTOPTS in /etc/autofs.conf is
<supersandro2000> AUTOMOUNTD_MNTOPTS=nosuid,nodev
<supersandro2000> for the problematic and working machine
<abathur> do you see any difference between the contents of /etc/auto_master on the two?
<supersandro2000> I did not compare the comments but everything else is the same
<supersandro2000> on the working VM the hard drive is disk3s6 because the layout is a bit strange
<supersandro2000> I actually did not configure this one but a friend of mine
<abathur> the working one, or the one that isn't?
<abathur> can see if the logs say something interesting; assuming you rebooted once without it mounting today, maybe: `log show --start '2020-11-4' --predicate 'eventMessage contains "disk1s6"'`
<supersandro2000> the working one
<abathur> well, use the appropriate diskxsy for whichever
<abathur> on a correct mount I don't see much aside from the mount message and 4 volumeRole messages that follow it
<supersandro2000> kernel: (IOStorageFamily) disk1s6: device is not readable
<abathur> hmm
<abathur> and that was from one of the bad reboots, and not the most-recent?
<abathur> not sure it's related, but was one of the top few results when I search the error minus disk1s6
<supersandro2000> just tried with reboot with and without auto and both did not display the error
<supersandro2000> IIRC I can't run clover because that needs a patched proxmox
<supersandro2000> and I can't really do that
Chiliparrot has quit [Ping timeout: 244 seconds]
<supersandro2000> oh, yeah. I think I got it partially working
<supersandro2000> adding ,auto to fstab and then starting the daemon manually with the correct env vars exported works
<supersandro2000> 🎉
<supersandro2000> abathur++
<{^_^}> abathur's karma got increased to 7
<supersandro2000> thanks for your help and motivation to keep debugging this
<supersandro2000> so the last issue I have no work around for is why the system service has no access to its dylibs
Chiliparrot has joined #nix-darwin
<abathur> I recall something while monkeying around looking for a viable sandboxing approach for something that I couldn't double-sandbox
<abathur> so maybe if there's something weird about this setup that is already sandboxing the daemon, the sandbox inside nix-build might be failing?
<abathur> but I don't recall exactly what the error was, and I feel like it was more direct/obvious than what you're seeing
<supersandro2000> would sound reasonable but my darwin knowledge is not that great to debug this
<supersandro2000> it works when starting manually which would speak for the double sandbox issue
johnw has joined #nix-darwin
<abathur> is your daemon plist directly from the installer? is there anything in /var/log/nix-daemon.log?
<abathur> there's a RootDirectory setting that specifies a chroot, and I guess chroot may ultimately be a sandbox
__monty__ has quit [Quit: leaving]
cptrbn has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
cptrbn has joined #nix-darwin
cptrbn has quit [Client Quit]
cptrbn has joined #nix-darwin
philr has joined #nix-darwin
<supersandro2000> '/var/log/nix-daemon.log
<supersandro2000> dyld: Library not loaded: /nix/store/i1cg0wfns9j4lmfzvx5dz6rc436vs6ms-libsodium-1.0.18/lib/libsodium.23.dylib
<supersandro2000> Referenced from: /nix/var/nix/profiles/default/bin/nix-daemon
<supersandro2000> Reason: no suitable image found. Did find:
<supersandro2000> /nix/store/i1cg0wfns9j4lmfzvx5dz6rc436vs6ms-libsodium-1.0.18/lib/libsodium.23.dylib: stat() failed with errno=1
<supersandro2000> file system sandbox blocked open() of '/nix/store/i1cg0wfns9j4lmfzvx5dz6rc436vs6ms-libsodium-1.0.18/lib/libsodium.23.dylib'
<supersandro2000> file system sandbox blocked open() of '/nix/store/i1cg0wfns9j4lmfzvx5dz6rc436vs6ms-libsodium-1.0.18/lib/libsodium.23.dylib'
<supersandro2000> and yes it is from the installer
<supersandro2000> abathur: Where should the rootdirectory be specified? In the plist is no such setting
<abathur> yeah, that's where it would be; I didn't really expect so
<abathur> ponder
Chiliparrot has quit [Quit: My iMac has gone to sleep. ZZZzzz…]
<abathur> I guess another thing that might explain the circumstance is if it's somehow mounting the wrong thing? do you see more than one Nix Store in `diskutil list`? is the outer system Linux? when you boot it without `,auto` in fstab, and then run `mount`, does it still indicate something is mounted at /nix?
<abathur> (imagining cases like a 2nd empty Nix Store volume, or maybe a network share that has a linux system's Nix store, and the VM was mounting either...)
<abathur> in fact hmm
<supersandro2000> don#t know what else it should mount
<supersandro2000> outer system is linux
<supersandro2000> without the #
<supersandro2000> nothing else mounted. just not mounted at all
<abathur> nod
* supersandro2000 dislikes launchctl and Apple
supersandro2000 has quit [Ping timeout: 265 seconds]
andremedeiros has quit [Quit: ZNC 1.8.2 -]
andremedeiros has joined #nix-darwin
supersandro2000 has joined #nix-darwin