<{^_^}>
nix#2523 (by periklis, 5 weeks ago, open): darwin: nix-daemon crashes due to OBJC_DISABLE_INITIALIZE_FORK_SAFETY
<LnL>
I'm kind of stuck tho :/
<gchristensen>
ouch
<LnL>
don't know how to trace the source of the dlopen
<copumpkin>
hey hey, perhaps try dtrace? you can turn off just the dtrace protection in SIP (with a restart) and then write a small probe to show you dlopens
<copumpkin>
or just launch under a debugger
<gchristensen>
copumpkin!
<copumpkin>
sorry I've been missing :)
<LnL>
!!!
<gchristensen>
delighted to see you were so impressed by the docker PR :)
<gchristensen>
don't be sorry
<LnL>
long time no see
<copumpkin>
I look forward to trying that PR btw, might have a good use for it
<copumpkin>
yeah! I feel like some sort of lone cowboy from RDR2 who shows up randomly every so often with a gun and can help out
<copumpkin>
but is otherwise largely absent :P
<LnL>
copumpkin: so I narrowed down the problem using DYLD_PRINT_LIBRARIES & DYLD_PRINT_LIBRARIES_POST_LAUNCH
<gchristensen>
lol
<copumpkin>
LnL: yeah I was seeing the ticket
<LnL>
but would dtrace help with seeing what library that originated from?
__Sander__ has joined #nix-darwin
<LnL>
cowboy or not, I might need some help for this one :)
<copumpkin>
yeah I think you can call a magic dtrace function that prints a callstack
<copumpkin>
so you'd add a probe for dlopen and then whenever it triggers you print the callstack
<copumpkin>
might just be easier from lldb though
<LnL>
I also purged kerberos on a branch because I had a suspicion that was the problem but it didn't change anything
<copumpkin>
so it looks like there are only two or three possible culprits in the diff, too?
<copumpkin>
nghttp2 1.34->1.35?
<copumpkin>
openssl p -> q?
<copumpkin>
or do you think it's a system library that changed out from under us?
<LnL>
I have no idea
<LnL>
but it kind of feels like a built time difference to me
<LnL>
I removed the hashes to get an overview, but there's a stdenv rebuild in between
<copumpkin>
I see
<copumpkin>
anyway yeah, I'd probably either using lldb or dtrace attach to dlopen calls and try to see who's doing it that way
<copumpkin>
I need to go catch a plane soon so can't really look unfortunately :(
<LnL>
^ /usr/lib/libnetwork.dylib and /usr/lib/system/libsystem_network.dylib sound very similar
<LnL>
gchristensen: could you run this on one of the macos builders? nix run nixpkgs.darwin.cctools -c otool -L /usr/lib/libnetwork.dylib
trcc has quit [Remote host closed the connection]
<gchristensen>
trying... :)
<gchristensen>
SSH in to the mac is not easy
<gchristensen>
so I usually use vnc, but its graphics are terrible
<copumpkin>
do we have a good place to host macs? I've proven myself incapable of actually running the ones I bought for nix
<copumpkin>
so would love to hand them off to someone who can
<copumpkin>
they're pretty nice machines
<gchristensen>
I can put them in my basement, you could ship them to ikwildrpepper
<gchristensen>
copumpkin: not sure if you caught it or not, but we deploy to the macs via nixops now
<copumpkin>
oh nice! I hadn't caught that
<copumpkin>
and you said something about VMs up above?
<gchristensen>
they run nixos, and macos in a VM
<copumpkin>
oh nice
<copumpkin>
so *cough*
<gchristensen>
each time the systemd service for the macos vm restarts, it has a fresh install to work from
<copumpkin>
oh actually on apple hardware that's legit
<copumpkin>
nice
<gchristensen>
yeah :)
<copumpkin>
so yeah
<copumpkin>
I mostly don't want to burden you if you don't want them, but it also makes me sad to see them gathering dust on my shelf and no time to actually do anything useful with them
<copumpkin>
could ship them to rob if that's better
<copumpkin>
it's 3 of them
<gchristensen>
hmm
<gchristensen>
I'll chat uprob
<gchristensen>
it'd be best to have them with the rest, I think, but we're out of IP space with the current setup
<copumpkin>
ah I see
<copumpkin>
cool, no rush obviously since they've been sitting for so long anyway :)
<copumpkin>
plus I can't ship them until nex week anyway
<gchristensen>
:)
<gchristensen>
our build queue would definitely appreciate more fire power
<LnL>
a few more builders would be nice
<copumpkin>
yeah, these have four cores each
<copumpkin>
they're the nicer old model
<gchristensen>
nice
<copumpkin>
not the newer nice model
<gchristensen>
I would very much prefer a *cough* setup than anything else, but also staying above board is a good thing
<copumpkin>
nah, the nixos host thing sounds great
<copumpkin>
IO perf could be impacted a bit I guess
<copumpkin>
due to no virtio-ish drivers
<copumpkin>
but *shrug*
<gchristensen>
yeah
<gchristensen>
more for the reason that having one node with 20 cores is easier to manage than 10 nodes with 4 cores each
<copumpkin>
oh so it shows up as a single logical node to hydra? neat!
<gchristensen>
oh, no
<gchristensen>
that is why I wish we had a hackintosh node instead of many legit ones :)
<copumpkin>
oh I see
<copumpkin>
well, hackintosh isn't really out of the question... :)
<gchristensen>
:see-no-evil:
<copumpkin>
nobody needs to (or should) know what infra we run our builds on
<gchristensen>
yeah
<copumpkin>
and I doubt apple would actually give a shit even if they did know
<gchristensen>
well... let's see what happens.
<copumpkin>
yeah, we can at least get a bit more legit hardware
<copumpkin>
and then if we're still struggling, maybe get a beefy packet box and coax it to run macOS :P
<copumpkin>
then if C&D, we take it down and hate apple just a little bit more
<gchristensen>
true
<LnL>
I sometimes wonder if there's somebody at apple laughing at all the macminis that people setup for infrastructure
<gchristensen>
their CFO laughs every time MacStadium buys another rack full
<gchristensen>
dries away the tears of joy with crisp $100 bills
<copumpkin>
lol
trcc has joined #nix-darwin
<gchristensen>
it has become a lot easier to get lots of aarch64 hw than apple hw
<clever>
last time i tried to get aarch64 from packet, it failed, i think they ran out of hardware
<gchristensen>
could be
<gchristensen>
depends where you were looking
<copumpkin>
now you can just pay amazon for it too :)
<copumpkin>
price isn't great but it's amazon
<clever>
gchristensen: i set the facility to any
<gchristensen>
I have 450 aarch64 cores at packet right now