supersandro2000 has quit [Disconnected by services]
supersandro2000 has joined #nix-darwin
eraserhd3 has joined #nix-darwin
eraserhd2 has quit [Ping timeout: 256 seconds]
<abathur>
grumble
<abathur>
I think maybe this beta didn't have a working asr yet, I haven't been able to get a bootable clone
philr_ has joined #nix-darwin
<abathur>
at least via ccc
philr has quit [Ping timeout: 264 seconds]
philr has joined #nix-darwin
philr_ has quit [Ping timeout: 265 seconds]
Mic92 has quit [Ping timeout: 260 seconds]
Mic92 has joined #nix-darwin
spacekookie has joined #nix-darwin
<abathur>
meh, this has been rage-inducing
<abathur>
does @darwin from github hang out here? I stumbled on a gist of his on the topic and realized he has a Nix Store volume :P
<thefloweringash>
fwiw, one of the macs here that went to recovery while updating to 11.2 had nix but not nix-darwin
<abathur>
I installed Nix, removed the volume, synthetic.conf, and fstab, and the upgrade still booted to recovery
<abathur>
I've had a suspicion that it's actually the users, but idk, could be the daemon or the shell add-ins I guess
<stephank>
Oh, interesting. Btw, I’m not sure you have a setup going already, but I remembered there’s a 11.1 install usb here somewhere. Could try look for it.
hedgie has quit [Read error: Connection reset by peer]
hedgie has joined #nix-darwin
<dhess>
So our 5th Mac (M1), which has MDM but no Nix and no extra volume, just updated from 11.1 to 11.2 with no problems.
<dhess>
so the issue is pretty clearly either something about what the Nix installer does, or something about extra volumes.
<klardotsh>
but anyhoot, if anyone has ideas on getting around this sandbox error keeping nix-daemon from starting on a brand new Catalina EC2 install, I'm all ears, I feel like I must be doing something wrong if I'm the lone one out with the unworking setup :)
<klardotsh>
aha. I worked around it for now by granting "sh" Full Disk Access (killing the kernel sandboxing)
<siraben>
Is there a darwin → gnu64 cross compilation cache anywhere?
<klardotsh>
there is nothing enjoyable about setting up devtools on a mac, lemme tell ya
<siraben>
it looks like I have to build gcc from source
<LnL>
dhess: oh btw, isn't there an install log? or is there nothing interesting to find in there
<abathur>
also removed the volume-mount plist and the shell profile hook this run (though I didn't think to do a literal revert--I just removed the lines) and it still failed into recovery
<abathur>
klardotsh: I guess we need to figure out why you need to add an exemption to sh; I've installed this scores of times on fresh macOS installs without any such need (and also in CI, though I suppose the CI images might already have full-disk-access added)
eraserhd3 has quit [Ping timeout: 256 seconds]
<dhess>
LnL: an install log from what, exactly?
<abathur>
LnL: there is an install log from the updates, but figuring out what's relevant in it is another story; it's fairly long/noisy
eraserhd has joined #nix-darwin
<LnL>
right
<abathur>
klardotsh: though if you're only seeing it in builds, I guess it's possible that I just haven't seen it; back when I did most of the initial testing the work in nixpkgs to support builds on big sur wasn't far enough along to actually test builds
<abathur>
I guess whatever klardotsh is reporting is the same thing supersandro2000 but I don't think we got a clear sense of why that was so FUBAR and how to get it right
<abathur>
*sandro ran into while setting up a vm
<cransom>
weird. on my macbook air running nix+nix-darwin, the update to 11.2 rebooted to recovery. i hit the restart button instead of monkeying with any of it and it booted to 11.2 and is normal, so far as i can tell.
<abathur>
yeah, I think that squares with what dhess was saying
<cransom>
i'll have one more machine to update that's similar config (a 2018 mini). are there logs i can pull out while in recovery (or after) that tell me why it decided to panic to recovery?
<supersandro2000>
I didn't upgrade a machine yet
<abathur>
there's at least one big log for the failed install, though I don't remember the path off the top of my head; I didn't spot anything obvious/conclusive in it
<abathur>
not /var/log, though
<abathur>
root of a volume probably
<abathur>
sandro are you running your macOS VM on linux, or inside macOS?
<supersandro2000>
on Linux
<abathur>
k, thought so
<abathur>
not quite out of things to try on the upgrade/recovery issue, but getting close; my next few tries will be: 1) also remove the Nix daemon and /etc/nix, 2) also remove the users and groups, 3) take a tmutil snapshot, install, roll back
<klardotsh>
abathur: what can I do to get you better debugging information on this? happy to spin up another box without this workaround and probe it for anything you'd find necessary
<klardotsh>
or spin one up in a public VPC + share an SSH key your way if it's something EC2 specific you'd find use in playing with
<abathur>
maybe; I'm trying to get a catalina VM set up locally to see if I can repro there, if I can that's probably ideal
<abathur>
did it actually prompt you for the exemption, or just not work right?
<abathur>
I know sandro saw something similar in a VM, so I'm hoping that'll be enough to trigger it, but idk
<abathur>
I guess a good place to start is: can you write a gist/paste/email that outlines, in as much detail as you recall, what you did/saw across the different attempts? some specifics I can think of: how you were connected, user/account name, terminal, shell, installer, filevault status
<abathur>
I did just get a VM in virtualbox up to the initial OS/account config gui, so maybe I'll have a chance to try soon :)
<abathur>
erg, I lied, it's up to recovery/install I guess, the language selector tricked me
<abathur>
cransom LnL dhess just hit another boot-to-recovery, I see a 2.1M log at "/Volumes/Macintosh HD - Data/.install-failure.log"
<cransom>
ah, there one is in /System/Volumes/Data
<abathur>
I'
<abathur>
oops; I'm also just realizing you can look under window or use cmd+L in the installer to open an installer log as you go
<abathur>
which can be filtered by detail level and searched; not sure if that'll be a more useful experience or not
<abathur>
well, this is in recovery
<abathur>
IDK if that applies in the update installers, maybe the full ones
<abathur>
ah, I guess it's probably useless though; my failures are always during the reboot phase where that'd be missing
<klardotsh>
abathur: I'll get you a ticket writeup in a bit, but for context here, I was running completely headless last night until I finally set up VNC to dive into System Preferences to disable filesystem sandboxing. No prompts for exemption, it just didn't work and I noticed by poking at launchctl list / cat /var/log/nix-daemon.log (after running into the famous chmod 755 red herring)
<dhess>
abathur: an interesting test would be to install Nix to an alternate Nix store location that doesn't require a separate volume, like /opt/nix. Then upgrade to 11.x and see if it breaks.
<domenkozar[m]>
macos funding campaign is going live, hopefully tomorrow.
<domenkozar[m]>
Let me know if anyone has questions I should answer on the campaign - you can already start adding donations (it's US non-profit)
<dhess>
yay!
<supersandro2000>
nix-daemon did not start. Take a look at the system log app and filter for sandbox
<supersandro2000>
it fails to load dylibs
<abathur>
hmm
<abathur>
well
<abathur>
I got catalina installed in virtualbox and Nix installed with the daemon running
<abathur>
a nix-shell command with packages from cache worked fine
<abathur>
currently waiting on CLI tools to install to explicitly try something I know isn't cached
<abathur>
were you trying to use the VM headlessly at that point as well?
<supersandro2000>
I did it fully headless
<abathur>
yeah, I think that's the missing variable here
<abathur>
I did get a build to start as well
skrzyp has joined #nix-darwin
<abathur>
I suspect either the GUI session/terminal are lending permission that ssh or whatever isn't, or that there's something different about the GUI setup process
<supersandro2000>
if you tell me how to get a path in finder I can tell you which sh I needed to allow
<abathur>
how did you set up the OS?
<supersandro2000>
just installed it
<abathur>
cmd + alt + c I think will copy the path
<abathur>
did you go through the country selection gui and stuff?
<supersandro2000>
I needed to create that symlink by hand at every reboot
<supersandro2000>
maybe my disk layout is bad so that this does not work on boot
<supersandro2000>
maybe it was because the services use /bin/sh to bootstrap their tools
<supersandro2000>
I somehow switched my keyboard to greek with a key combo....
<abathur>
heh, removing the nixbld group & users appears to have gotten me through the big sur update without the boot into recovery
<dhess>
interesting.
<dhess>
Maybe something about how they're being created?
<abathur>
yeah, I should've tried it earlier, I've had a hunch but didn't want to deal with figuring out how to clean them up :P
<dhess>
abathur: thanks for all the work you're doing on this!
<abathur>
not that I can swear this is it, but, cleaning them got me by it
<abathur>
so the next step is probably trying again and *only* cleaning the users up
<abathur>
not really sure what about them may be causing trouble; this may be where logs help (I only had the hunch because I saw thousands of lines of logspam that involved some templateMigrator process iterating over users
<cransom>
my system yet to be upgraded doesn't have the nixbld users, i can tell you in a few hours if that's a common thread for me
<abathur>
cool
<abathur>
so single-user install?
<cransom>
yeah.
johnw has joined #nix-darwin
<abathur>
I found one case of the same GUI error I see on reddit, but it's a low-info thread
philr has joined #nix-darwin
<dhess>
Oh right, I wonder if most people are using a single-user install and that's why this problem isn't rampant.
<dhess>
That would make a lot of sense.
<abathur>
gchristensen LnL do either of you happen to know/recall much about how we arrived at the existing user setup process? I'm not familiar with any of this stuff and have been really hoping to avoid touching it :P
<gchristensen>
which part?
<abathur>
yeah, it's been the default for a while
<abathur>
sorry, the nixbld group/users
<abathur>
it looks like something about them *might* be causing macOS to boot into recovery when people update big sur
<LnL>
what about them?
<gchristensen>
wow
<abathur>
I'm not really sure, just curious if you're familiar with the decisions/pressures or if it predated the times you've touched it
<abathur>
I see that they were present in the installer ~4y ago at graham's init on it
<LnL>
there's nothing special about those, basically the same as default system users like _postgres
<abathur>
and most of the source scripts also use them in some form
<abathur>
that's a good point; I should compare them to something like the pg user hmm