<adelbertc>
johnw: do you have any automation that starts the emacs daemon if one doesnt already exist? it seems emacsclient has --alternate-editor which when set to empty string will call `emacs --daemon` but last time we chatted we learned Nix-installed emacs on macOS wants `~/.nix-profile/Applications/Emacs.app/Contents/MacOS/Emacs --daemon`
puffnfresh has joined #nix-darwin
Sonarpulse has quit [Ping timeout: 256 seconds]
copumpkin has joined #nix-darwin
sphalerite has joined #nix-darwin
dtz has joined #nix-darwin
puffnfresh has quit [Ping timeout: 248 seconds]
dtz has quit [Ping timeout: 255 seconds]
sphalerite has quit [Ping timeout: 255 seconds]
copumpkin has quit [Ping timeout: 256 seconds]
LnL has quit [Ping timeout: 260 seconds]
philr has joined #nix-darwin
Sonarpulse has joined #nix-darwin
<dhess>
LnL: do you ever see this thing where Hydra gets stuck downloading packages
<dhess>
My most common failure mode with Hydra is, one or more jobs on the same host at the same time get stuck while downloading packages from the binary caches
philr has quit [Ping timeout: 260 seconds]
LnL has joined #nix-darwin
hamishmack has quit [Quit: hamishmack]
hamishmack has joined #nix-darwin
hamishmack has quit [Client Quit]
hamishmack has joined #nix-darwin
puffnfresh has joined #nix-darwin
<johnw>
adelbertc: I don't use the emacs daemon, but nix-darwin should be able to nicely setup a launchd daemon for you
<johnw>
have you guys ever seen nix-copy-closure just stall out?
<johnw>
it's been stuck on copying path '/nix/store/vad9hkq4f5d81lc31lfxrjg6pfbw2w86-SageMath-8.1' to 'ssh://hermes'... for hours now
<johnw>
with no CPU activity, and absolutely no network activity
<dhess>
johnw: I've seen it go very, very slowly from time to time. On hermes, "ls -alFtr" on the store and look for the most recent directory, see if there are any changes over a minute or two
<johnw>
ok, that's a good idea
<dhess>
it'll be .something
<dhess>
I don't know why that happens though
<dhess>
and come to think of it I haven't seen it in quite awhile, but it used to happen on my Macs quite a bit
<johnw>
so, it's fully transferred
<johnw>
exactly same number of files and total size as the source
<johnw>
it's 103,038 files
<johnw>
is there some slowness updating the SQLite database or something?
<dhess>
Do you have another SSH session open to hermes, maybe something with a ControlMaster ?
<johnw>
but I was seeing 0% CPU
<dhess>
or do you have that set in your .ssh/config for hermes?
<johnw>
i've just disabled control mastering to try this again
<johnw>
i've seen such problems with it in the past
<dhess>
I have seen ssh sessions hang on close from time to time when that is enabled, or other options that, say, forward connections over them
<johnw>
is it bad to have tons of *.lock files in /nix/store?
<dhess>
I have seen that also, might be related to interrupted copies
<dhess>
or maybe not, I have a bunch right now and everything is working ok for me at the moment
<dhess>
so I guess I don't know why those would appear suddenly
<johnw>
they recently added ZFS native encryption, but I don't trust it yet
<dhess>
does that run via FUSE?
<dhess>
or is it a kext
<johnw>
kext
<dhess>
interesting. No issues with it?
<johnw>
no, none at all in fact
<dhess>
huh. Well maybe I'll have to try it out
<johnw>
I've been using it continuously since... 2012?
<johnw>
i mean, it's based on the Solaris code, and the zfsonlinux code
<johnw>
a few years ago it didn't work so well for things like iTunes
<johnw>
but in the last 2 years they fixed that too
<johnw>
you can set an attribute that tells the ZFS filesystem to act like HFS
<johnw>
and now I run iTunes on it
<dhess>
The wiki doesn't say anything about 10.13, is the wiki just out of date?
<johnw>
1.7.0 is only for 10.13
<johnw>
I'm running 1.6.1
<dhess>
I'm assuming there's no way to run / on it :)
<johnw>
no, that you can't do
<johnw>
i use this on USB drives
<johnw>
but all kinds: thumb drives, big drives, JBOD arrays, etc.
<johnw>
the internal boot drive is the only drive I let Apple use HFS+ on
<dhess>
I am tempted to try that. All of my music is on a USB drive
<johnw>
it's so nice to know there's no bitrot
<dhess>
yeah
<johnw>
especially when things like RAM modules fail... :)
<johnw>
for single USB drives, I use copies=2
<johnw>
but I prefer mirrors when I can
<dhess>
hehe
<johnw>
also, the ZFS filesystems are shareable with NixOS
<johnw>
in fact, right now I'm using zfs send+recv to copy 1 TB from my old NixOS machine
<dhess>
so they're keeping up with zfsonlinux features?
<johnw>
yes
<dhess>
that's sweet. How the hell haven't I heard of this?
<johnw>
sha512, lz4... and now native encryption
<johnw>
dunno, glad I told you :)
<dhess>
me too!
<johnw>
snapshots are just as wondrous as they are in Nix
<dhess>
that's what I would really like because Apple hasn't really exposed snapshotting to end users and/or the CLI with APFS
<johnw>
snapshots have saved my bacon a few times too
<johnw>
i run a backup that keeps rolling daily/weekly/monthly/yearly snapshots
<johnw>
i do wish I had this on my root volume
<johnw>
*however*
<johnw>
it's a big performance hit
<johnw>
so for active development it's not so bad to have a weaker but faster filesystem
<dhess>
is it slower than zfsonlinux on the same drives?
<johnw>
I don't see why it would be
<johnw>
it's just thet ZFS is a memory and CPU pig
<johnw>
if you're doing lots of heavy asynchronous I/O, you'll notice it
<johnw>
that's why I mainly use it for archival storage
<johnw>
I'm not using deduplication, which makes things even worse
<johnw>
but then again, I'm also not using an external ARC cache
<johnw>
which can really speed things up
<dhess>
yeah my Linux ZFS fileserver is dedicated to just fileserving because ZFS is such a memory hog
<johnw>
I want ZFS to *keep* files
<johnw>
not for making them :)
<johnw>
like, I avoid running my VMs directly from the ZFS volume
<johnw>
I prefer to copy the active VMs down to the SSD
<johnw>
but I keep the old ones in ZFS, in case I want them in future
<dhess>
I do have an SSD ARC on my fileserver
<dhess>
but I still don't do de-dup
<johnw>
do you use a ZIL volume?
<dhess>
yes
<dhess>
also on an SSD
<johnw>
very cool
<johnw>
you're no zfs schmoe
<dhess>
I dunno about that. I carefully set it up a few years ago and it has run so reliably ever since I never even have to touch it anymore
<dhess>
so I used to know more than I do now, but I guess that is to ZFS's credit.
<dhess>
/know/remember/
<johnw>
have you done performance testing?
<johnw>
when I did some testing, I discovered that I had a bad batch of WD REDs that were parking the heads continuously
<johnw>
like, normal drives had <50 power cycles since purchase, and these had >20,000
<dhess>
I haven't dont any performance testing since I first set it up. I can do about 600MB/s reads and near that for writes so it's better than any network connection I've got to it :)
<johnw>
sweet
<dhess>
In fact I'm not even using fast drives with it anymore, I just buy the cheapest, most power efficient ones I can
<johnw>
what configuration are you using?
<johnw>
i used to use RAIDZ2
<johnw>
but then I switched to striped mirrors
<dhess>
it's 2 striped raidz2's
<dhess>
6 drives per raidz2
<johnw>
ah, you have a 12-bay too
<dhess>
the advice when I set it up was not to exceed about 6 drives per zvol
<johnw>
you're not in the Bay Area by any chance, are you?
<dhess>
I am in SF. I came to your Nix talk last year :)
<johnw>
you interested in a 4U soundproofing enclosure?
<johnw>
I have this monster machine, and the enclosure is a good one, but I don't know what to do with it now
<johnw>
it weighs 80 pounds
<dhess>
Not particularly, my server is in a little server room so it's quiet enough
<johnw>
ah
<dhess>
thanks though
<johnw>
it's free, if that matters
<johnw>
I just need help getting it out of the house physically
<dhess>
nah I'm good, thank you for the offer.
<dhess>
I may have a friend who would want it. I'll ask him next time I see him
<dhess>
also in SF
<johnw>
cool, I also have a 2U machine inside it, with the 12 bays
<johnw>
it's all aging hardware, but it's core i7 and 16G RAM
<johnw>
if he's willing to come get it, he can have it, I'm just going to keep the hard drives
<dhess>
I'll ask him next time we chat
<johnw>
thanks
<johnw>
I'd rather give it to him than a rando from Craigslist, which is what I'll try next
<dhess>
hehe
<dhess>
anyway this machine I have is a 4U with 36 bays(!)
<johnw>
whoa!
<dhess>
I only use 14 but when it's time to make an offsite backup, those other bays are very handy :)
<johnw>
do you have a rack?
<dhess>
yes! ;)
<johnw>
ah, ok
<dhess>
my special little room
<dhess>
once you go rack you never go back
<johnw>
well, now i'm downgrading to this 4 bay thunderbolt enclosure
<johnw>
it's quite speedy at least
<johnw>
just running this 12-bay server was costing up to $30/month
<dhess>
how large is your array?
<johnw>
so instead I'm now using backblaze b2 offsite storage, and this 4-bay array
<johnw>
it's 2 8TB drives, and 2 3TB drives, in striped mirror configuration
<dhess>
have you measured the RAM footprint of ZFS on Mac for that array?
<johnw>
no
<johnw>
i hook it up to my laptop, to move the memory pressure off my desktop
<dhess>
For my purposes it's just a single 4TB drive for iTunes, so it should be fine RAM-wise
<johnw>
yes
<johnw>
the Mac drive I'm replacing with this new 4-drive array is an old 4TB MyBook
<dhess>
Are you doing Time Machine backups to a zpool?
<johnw>
no, I use rsync and a utility I wrote called pushme
<johnw>
there's no form of continuous backup occurring anymore
<johnw>
I used to use CrashPlan for that, but then they disappeared
<johnw>
so I always run the risk of losing the current day's work if it's not been pushed to GitHub yet
<dhess>
I'll bet Time Machine on zfs doesn't work anyway. There's something about Time Machine and hard links or something
<johnw>
and in this library I'm exploring Conal's idea of transpiling via CCCs in Coq
<johnw>
clever: on darwin?
<dhess>
I should do something similar with Coq
<johnw>
work sidetracked me since the summer, but I want to get back to that work this year
<clever>
johnw: nixos/linux
<johnw>
clever: ah, cool
<johnw>
dhess: we need another Nix meetup in SF
<dhess>
clever: I addressed this to LnL earlier but now that you're around, have you seen any cases of Hydra remote builders getting stuck while downloading packages from a binary cache?
<dhess>
I have jobs that occasionally just hang in the middle of a download
<dhess>
johnw: I went to Gabriel's talk in .. December I think? It was good
<dhess>
or do you mean a different org?
<johnw>
ah yeah, I had to miss th
<johnw>
at one
<clever>
dhess: i dont think ive seen that, but i have seen it deadlock if localhost is in the slave list
<dhess>
clever: yeah this is not a localhost case. Oh well. I will dig around
<dhess>
johnw: do you ever make it to ZuriHac? I'm going this year, yay!
<johnw>
no
<dhess>
oh well, it's pretty far after all :)
<johnw>
yeah, hard to justify that one
<LnL>
must have missed that, but yes I think I’ve also seen that
<dhess>
LnL: any ideas ?
<dhess>
this happens on both NixOS remote builders and on Macs
<LnL>
are they all running 1.12?
<dhess>
none of them are, actually
<dhess>
and they are all dedicated remote builders. They don't do anything else
<dhess>
I do use them with interactive nix-builds sometimes, in addition to running Hydra jobs, but they're just builders, in any case.
<LnL>
I know there have been a bunch of improvements for the interaction with hydra
<dhess>
I haven't upgraded my Hydra in quite awhile. I guess I will try that and see if it gets better.
<LnL>
the nixos.org instance also uses unstable everywhere afaik
<dhess>
usually I can just kill the jobs and restart them and it's fine. Occasionally they get stuck, though.
<dhess>
LnL: oh you mean in 1.12. I can try that as well
<dhess>
I was running 1.12 on my Macs but it breaks cabal's Nix support
<dhess>
so I had to go back to 1.11
<LnL>
huh
<johnw>
what is up with Nix tonight
<johnw>
several different operations are all just locking up
<johnw>
they C-c fine, but they just won't finish
<johnw>
it's happening with nix-env --leq, nix-store --verify, and nix-copy-closure
<johnw>
and on two different machines
<johnw>
did I mess up by putting SageMath in /nix/store, with its 100k files?
<johnw>
i wonder how scalable the store is...
<johnw>
at least the nix-daemon is using cpu
philr has joined #nix-darwin
<dhess>
LnL: ghc-8.4 fails to build on macOS, it says, "checking version of gcc... configure: error: Need at least gcc version 4.4 (4.7+ recommended)"
<dhess>
does macOS stdenv only provide clang?
<johnw>
correct
<dhess>
how do the other GHC derivations address this?
<johnw>
good question
<johnw>
can't ghc build with clang?
<dhess>
I thought it could actually
<dhess>
it checks for clang and finds it
<LnL>
gcc is probably added to the expression somewhere
<dhess>
it's not a big deal, just wondered if it was something obvious
<johnw>
my fully GC'd /nix/store is still 49.86G
<LnL>
I recently found some because of overrideCC stdenv gcc7
<LnL>
issues*
philr has quit [Ping timeout: 248 seconds]
philr has joined #nix-darwin
<LnL>
so... who wants to help out with ZHF when we branch off?
<LnL>
pretty please :D
* copumpkin
will help as much as he can :)
<shlevy>
LnL: I think I can volunteer a bunch of nix mac users to do grunt work, if there is any
<shlevy>
e.g. stuff that doesn't require significant knowledge of the internal details of nixpkgs or anything, but just can build stuff and read error messages
<LnL>
cool, a decent amount of the failures are usually clang related and not necessarily darwin specific
<copumpkin>
I think I found a bug in our version of clang btw, but have yet to reduce it
<LnL>
we should probably also mark some stuff as broken
<copumpkin>
the short of it is that I have a snippet of code (from libsnark) that says `try { ... } catch (...) { /* don't fail */ }`
<copumpkin>
and that crashes the program saying uncaught exception
<LnL>
guess I'll do the same thing for 5.9 so it doesn't disappear again
* LnL
opens opensource.apple.com in dispair
<LnL>
do you think the swift thing is feasible as a replacement?
<copumpkin>
for CF?
<copumpkin>
if we can get its allocator not to rely on swift, then yes
<copumpkin>
otherwise it tangles everything up
<copumpkin>
it might be possible with some hackery to make it depend on something already in libSystem, but it's pretty awkward
Sonarpulse has quit [Ping timeout: 255 seconds]
<cbarrett>
hello friends
<LnL>
hey
<cbarrett>
how goes LnL
<LnL>
I wrote the most horrible code today...
<cbarrett>
oh dear
<gchristensen>
the most horrible? I don't know, I've written some pretty horrible code ... :D
philr has quit [Ping timeout: 256 seconds]
<LnL>
it's some weird bootstrapping stuff with badly modelled dependencies
Sonarpulse has joined #nix-darwin
<cbarrett>
nice
<gchristensen>
ouch
<cransom>
would anyone have suggestions for an osx vendor (for remote builds)?
<cbarrett>
I've had good experiences with bitrise
<cbarrett>
but It really depends on what your needs are I think
<cransom>
the only foreseeable need is just to run nix for a hydra remote.
<niksnut>
cransom: macstadium
<cransom>
niksnut: thanks!
periklis has joined #nix-darwin
periklis has quit [Ping timeout: 255 seconds]
<dhess>
hmm that's an interesting idea
hamishmack has quit [Quit: hamishmack]
hamishmack has joined #nix-darwin
hamishmack has quit [Quit: hamishmack]
<johnw>
niksnut: nice to see you here
hamishmack has joined #nix-darwin
<cbarrett>
anyone interested helping / chatting about getting cocoapods pods to build under nix?
<cbarrett>
I've been messing around with their Ruby libraries to read their various files, i think I have a pretty good handle on it
<cbarrett>
But it's a tricky thing to get onesself oriented with though.
<dhess>
cbarrett: I haven't reached that point yet where I need them but will in the not-to-distant future. Are Cocoapods still the way to go? There are a few competitors now, especially for Swift libraries, are there not?
<cbarrett>
there are
<cbarrett>
however my use is completely "selfish" :)
<cbarrett>
in that I'm so sick of Xcode builds at $WORK taking forever
<contrapumpkin>
the thing about cocoapods is that we don't have a good answer for building cocoa stuff in general under nix yet
<contrapumpkin>
we have a working xcbuild that works on most stuff, but not a valid nib/xib compiler
<contrapumpkin>
and there's not a good way for us to get one at a publicly available URL
<contrapumpkin>
we can make some of this stuff depend on Xcode being present but that gets pretty awkward
<contrapumpkin>
anyway, I'd definitely like to find a principled way to do all this stuff to get pureish builds of Cocoa stuff
<dhess>
contrapumpkin: I just put 2+2 together yesterday, you were an early iOS jailbreaker, yes?
<cbarrett>
contrapumpkin: I don't think the xib compiler is a necessary first step, there are plenty of pods that don't need it
<contrapumpkin>
dhess: yeah
<cbarrett>
I don't think it would be too hard to hack up a compiler, as long as you write to a format that can be read by NSKeyedArchiver I think you're OK
<dhess>
contrapumpkin: that's awesome. I thought that GitHub avatar looked familiar :)
<contrapumpkin>
but it only speaks iOS classes right now
<contrapumpkin>
dhess: you were also involved? :D
<dhess>
nope, just an admirer.
<dhess>
wow that xib2nib compiler is a *lot* more complicated than I would have guessed
<dhess>
kind of scary all of the .cpp files in there
<dhess>
one per class I guess?
<dhess>
jesus
<dhess>
it does look like maybe it could be auto-generated from docs
<dhess>
eh maybe not. Anyway, what a mess
<cbarrett>
Seems standard for C++ :P
<dhess>
lol true
<contrapumpkin>
yeah, I'm not actually sure how involved the conversion is from xib to nib
<contrapumpkin>
it seems kind of mechanical, but there might be a few special cases
<dhess>
I would be fine with something that required Xcode to be present. It's not ideal but it's better than not being able to use Nix
<dhess>
contrapumpkin: exactly
<dhess>
there are some comments in there that look like a human wrote them
<contrapumpkin>
but yeah, we could make some sort of special case in ~/.config/nixpkgs.nix or something to let you point it at xcode
<dhess>
but most of it looks like boilerplate
<cbarrett>
It'd be nice to be able to build without Xcode IMO
<contrapumpkin>
definitely agreed :)
<contrapumpkin>
we're not going to get xcode on hydra boxes
<dhess>
cbarrett: it would, but as an end goal more than a first pass
<cbarrett>
i don't think Xcode is actually helpful for building pods
<cbarrett>
Their native build file is not an Xcode project at all
<contrapumpkin>
I guess I just associated cocoapods with GUI apps with xibs
<dhess>
actually is Xcode downloadable now without a dev account or App Store id?
<contrapumpkin>
dhess: not taht I know of
<cbarrett>
dhess: it is not, and you have to do a EULA agreement -- with sudo
<contrapumpkin>
but anyway, if we have non-xibby cocoapods we should definitely be able to pull it off
<dhess>
cbarrett: oh yeah, that ugly bit.
<cbarrett>
contrapumpkin: yeah!
Sonarpulse has quit [Ping timeout: 276 seconds]
<cbarrett>
I have to leave for home now but we should chat about this another time. I'd done a fair bit of digging around in the guts of CocoaPods last month, have some scripts that dump useful things in a format suitable for awk munging :)
<contrapumpkin>
that'd be awesome :)
<contrapumpkin>
cocoapod2nix :D
<dhess>
hheeheheh
<dhess>
god imagine that Hydra could cache them, too
<dhess>
contrapumpkin: have you had a chance to see Levin's MacOS and iOS Internals yet?
<contrapumpkin>
oh no, I forgot to order the books
<contrapumpkin>
was planning to once he released the latest
<dhess>
I bought them but they're on the slow boat. From order time to delivery it's going to end up being about a month!
<contrapumpkin>
wow :)
<contrapumpkin>
he's self-publishing, right?
<cbarrett>
I've already got a project name in mind for it, following on from bundix -- sputnix :)
<dhess>
but they are available on Amazon now, in any case
<contrapumpkin>
cbarrett: hah, nice
<dhess>
I'm not sure whether it's self-published, strictly speaking, but it's definitely not a mainstream publisher
<dhess>
cbarrett: err you might wanna Urban Dictionary that one first
<cbarrett>
?
<dhess>
Always google your project names these days.