supersandro2000 has quit [Disconnected by services]
supersandro2000 has joined #nix-darwin
eraserhd3 has joined #nix-darwin
eraserhd2 has quit [Ping timeout: 256 seconds]
<abathur> grumble
<abathur> I think maybe this beta didn't have a working asr yet, I haven't been able to get a bootable clone
philr_ has joined #nix-darwin
<abathur> at least via ccc
philr has quit [Ping timeout: 264 seconds]
philr has joined #nix-darwin
philr_ has quit [Ping timeout: 265 seconds]
Mic92 has quit [Ping timeout: 260 seconds]
Mic92 has joined #nix-darwin
spacekookie has joined #nix-darwin
<abathur> meh, this has been rage-inducing
<abathur> does @darwin from github hang out here? I stumbled on a gist of his on the topic and realized he has a Nix Store volume :P
<thefloweringash> fwiw, one of the macs here that went to recovery while updating to 11.2 had nix but not nix-darwin
<abathur> I installed Nix, removed the volume, synthetic.conf, and fstab, and the upgrade still booted to recovery
<abathur> I've had a suspicion that it's actually the users, but idk, could be the daemon or the shell add-ins I guess
<stephank> Oh, interesting. Btw, I’m not sure you have a setup going already, but I remembered there’s a 11.1 install usb here somewhere. Could try look for it.
hedgie has quit [Read error: Connection reset by peer]
hedgie has joined #nix-darwin
<dhess> So our 5th Mac (M1), which has MDM but no Nix and no extra volume, just updated from 11.1 to 11.2 with no problems.
<dhess> so the issue is pretty clearly either something about what the Nix installer does, or something about extra volumes.
klardotsh has joined #nix-darwin
<klardotsh> rather than copy-pasting the whole spew from #nixos, I'll link in the logs;
<klardotsh> but anyhoot, if anyone has ideas on getting around this sandbox error keeping nix-daemon from starting on a brand new Catalina EC2 install, I'm all ears, I feel like I must be doing something wrong if I'm the lone one out with the unworking setup :)
<klardotsh> aha. I worked around it for now by granting "sh" Full Disk Access (killing the kernel sandboxing)
<siraben> Is there a darwin → gnu64 cross compilation cache anywhere?
<klardotsh> there is nothing enjoyable about setting up devtools on a mac, lemme tell ya
<siraben> it looks like I have to build gcc from source
<LnL> dhess: oh btw, isn't there an install log? or is there nothing interesting to find in there
<abathur> also removed the volume-mount plist and the shell profile hook this run (though I didn't think to do a literal revert--I just removed the lines) and it still failed into recovery
<abathur> klardotsh: I guess we need to figure out why you need to add an exemption to sh; I've installed this scores of times on fresh macOS installs without any such need (and also in CI, though I suppose the CI images might already have full-disk-access added)
eraserhd3 has quit [Ping timeout: 256 seconds]
<dhess> LnL: an install log from what, exactly?
<abathur> LnL: there is an install log from the updates, but figuring out what's relevant in it is another story; it's fairly long/noisy
eraserhd has joined #nix-darwin
<LnL> right
<abathur> klardotsh: though if you're only seeing it in builds, I guess it's possible that I just haven't seen it; back when I did most of the initial testing the work in nixpkgs to support builds on big sur wasn't far enough along to actually test builds
<abathur> I guess whatever klardotsh is reporting is the same thing supersandro2000 but I don't think we got a clear sense of why that was so FUBAR and how to get it right
<abathur> *sandro ran into while setting up a vm
<cransom> weird. on my macbook air running nix+nix-darwin, the update to 11.2 rebooted to recovery. i hit the restart button instead of monkeying with any of it and it booted to 11.2 and is normal, so far as i can tell.
<abathur> yeah, I think that squares with what dhess was saying
<cransom> i'll have one more machine to update that's similar config (a 2018 mini). are there logs i can pull out while in recovery (or after) that tell me why it decided to panic to recovery?
<supersandro2000> I didn't upgrade a machine yet
<abathur> there's at least one big log for the failed install, though I don't remember the path off the top of my head; I didn't spot anything obvious/conclusive in it
<abathur> not /var/log, though
<abathur> root of a volume probably
<abathur> sandro are you running your macOS VM on linux, or inside macOS?
<supersandro2000> on Linux
<abathur> k, thought so
<abathur> not quite out of things to try on the upgrade/recovery issue, but getting close; my next few tries will be: 1) also remove the Nix daemon and /etc/nix, 2) also remove the users and groups, 3) take a tmutil snapshot, install, roll back
supersandro2000 has quit [Quit: The Lounge -]
supersandro2000 has joined #nix-darwin
philr has quit [Ping timeout: 240 seconds]
supersandro2000 has quit [Quit: The Lounge -]
supersandro2000 has joined #nix-darwin
<klardotsh> abathur: what can I do to get you better debugging information on this? happy to spin up another box without this workaround and probe it for anything you'd find necessary
<klardotsh> or spin one up in a public VPC + share an SSH key your way if it's something EC2 specific you'd find use in playing with
<abathur> maybe; I'm trying to get a catalina VM set up locally to see if I can repro there, if I can that's probably ideal
<abathur> did it actually prompt you for the exemption, or just not work right?
<abathur> I know sandro saw something similar in a VM, so I'm hoping that'll be enough to trigger it, but idk
<abathur> I guess a good place to start is: can you write a gist/paste/email that outlines, in as much detail as you recall, what you did/saw across the different attempts? some specifics I can think of: how you were connected, user/account name, terminal, shell, installer, filevault status
<abathur> I did just get a VM in virtualbox up to the initial OS/account config gui, so maybe I'll have a chance to try soon :)
<abathur> erg, I lied, it's up to recovery/install I guess, the language selector tricked me
<abathur> cransom LnL dhess just hit another boot-to-recovery, I see a 2.1M log at "/Volumes/Macintosh HD - Data/.install-failure.log"
<cransom> ah, there one is in /System/Volumes/Data
<abathur> I'
<abathur> oops; I'm also just realizing you can look under window or use cmd+L in the installer to open an installer log as you go
<abathur> which can be filtered by detail level and searched; not sure if that'll be a more useful experience or not
<abathur> well, this is in recovery
<abathur> IDK if that applies in the update installers, maybe the full ones
<abathur> ah, I guess it's probably useless though; my failures are always during the reboot phase where that'd be missing
<klardotsh> abathur: I'll get you a ticket writeup in a bit, but for context here, I was running completely headless last night until I finally set up VNC to dive into System Preferences to disable filesystem sandboxing. No prompts for exemption, it just didn't work and I noticed by poking at launchctl list / cat /var/log/nix-daemon.log (after running into the famous chmod 755 red herring)
<dhess> abathur: an interesting test would be to install Nix to an alternate Nix store location that doesn't require a separate volume, like /opt/nix. Then upgrade to 11.x and see if it breaks.
<domenkozar[m]> macos funding campaign is going live, hopefully tomorrow.
<domenkozar[m]> Let me know if anyone has questions I should answer on the campaign - you can already start adding donations (it's US non-profit)
<dhess> yay!
<supersandro2000> nix-daemon did not start. Take a look at the system log app and filter for sandbox
<supersandro2000> it fails to load dylibs
<abathur> hmm
<abathur> well
<abathur> I got catalina installed in virtualbox and Nix installed with the daemon running
<abathur> a nix-shell command with packages from cache worked fine
<abathur> currently waiting on CLI tools to install to explicitly try something I know isn't cached
<abathur> were you trying to use the VM headlessly at that point as well?
<supersandro2000> I did it fully headless
<abathur> yeah, I think that's the missing variable here
<abathur> I did get a build to start as well
skrzyp has joined #nix-darwin
<abathur> I suspect either the GUI session/terminal are lending permission that ssh or whatever isn't, or that there's something different about the GUI setup process
<supersandro2000> if you tell me how to get a path in finder I can tell you which sh I needed to allow
<abathur> how did you set up the OS?
<supersandro2000> just installed it
<abathur> cmd + alt + c I think will copy the path
<abathur> did you go through the country selection gui and stuff?
<supersandro2000> 🤔 I think so
<supersandro2000> I needed to allow sshd
<supersandro2000> actually it was /bin/sh
<supersandro2000> sudo ln -s /nix/var/nix/profiles/system /run/current-system
<supersandro2000> I needed to create that symlink by hand at every reboot
<supersandro2000> maybe my disk layout is bad so that this does not work on boot
<supersandro2000> maybe it was because the services use /bin/sh to bootstrap their tools
<supersandro2000> I somehow switched my keyboard to greek with a key combo....
<abathur> heh, removing the nixbld group & users appears to have gotten me through the big sur update without the boot into recovery
<dhess> interesting.
<dhess> Maybe something about how they're being created?
<abathur> yeah, I should've tried it earlier, I've had a hunch but didn't want to deal with figuring out how to clean them up :P
<dhess> abathur: thanks for all the work you're doing on this!
<abathur> not that I can swear this is it, but, cleaning them got me by it
<abathur> so the next step is probably trying again and *only* cleaning the users up
<abathur> not really sure what about them may be causing trouble; this may be where logs help (I only had the hunch because I saw thousands of lines of logspam that involved some templateMigrator process iterating over users
<cransom> my system yet to be upgraded doesn't have the nixbld users, i can tell you in a few hours if that's a common thread for me
<abathur> cool
<abathur> so single-user install?
<cransom> yeah.
johnw has joined #nix-darwin
<abathur> I found one case of the same GUI error I see on reddit, but it's a low-info thread
philr has joined #nix-darwin
<dhess> Oh right, I wonder if most people are using a single-user install and that's why this problem isn't rampant.
<dhess> That would make a lot of sense.
<abathur> gchristensen LnL do either of you happen to know/recall much about how we arrived at the existing user setup process? I'm not familiar with any of this stuff and have been really hoping to avoid touching it :P
<gchristensen> which part?
<abathur> yeah, it's been the default for a while
<abathur> sorry, the nixbld group/users
<abathur> it looks like something about them *might* be causing macOS to boot into recovery when people update big sur
<LnL> what about them?
<gchristensen> wow
<abathur> I'm not really sure, just curious if you're familiar with the decisions/pressures or if it predated the times you've touched it
<abathur> I see that they were present in the installer ~4y ago at graham's init on it
<LnL> there's nothing special about those, basically the same as default system users like _postgres
<abathur> and most of the source scripts also use them in some form
<abathur> that's a good point; I should compare them to something like the pg user hmm
<LnL> it's just a user regular account, only special thing is the hidden flag so these don't show up on the login screen
<antifuchs> haha, I saw you can now run nixos-rebuild from nixpkgs unstable, which even makes sense if you use --target-host and --build-host
<antifuchs> (except it refuses to do a flake build)
<abathur> cransom dhess can you run `dscl . -read /Users/nixbld1` on a system that has taken the update and see what the NFSHomeDirectory is?