<sphalerite>
aminechikhaoui: no I mean the details for the build products
<sphalerite>
ipxe, initrd and bzImage
goibhniu has joined #nixos-dev
<sphalerite>
Oh, is this maybe because they're symlinks rather than being actual files?
<domenkozar>
well
<domenkozar>
hmmmz
<domenkozar>
ok
<domenkozar>
time to utilize 10 hetzner servers and finally fix this ldflags madness
<domenkozar>
btw I always forget
<domenkozar>
can we have bash 4 features in stdnev?
<domenkozar>
iirc not due to macos
<infinisil>
Macs don't have bash 4?
<infinisil>
And isn't the only impure one /bin/sh? Bash comes from nixpkgs afaik
<domenkozar>
yeah
<domenkozar>
right we fixed nix-shell to use nixpkgs bash instead of some magic
ixxie has quit [Quit: leaving]
ixxie has joined #nixos-dev
ixxie has quit [Client Quit]
ixxie has joined #nixos-dev
orivej has joined #nixos-dev
<angerman>
domenkozar: do we know which route we are going to take by now?
<domenkozar>
no
<domenkozar>
but I think whatever we do, it shouldn't be haskell hack
<angerman>
that's unfortunate :(
<domenkozar>
but rather stdenv one
<domenkozar>
if any
<domenkozar>
angerman: if Dezgeg suggestion can work
<angerman>
even if it's just looking for `lib*.{so,a,dylib}` in `lib` is good
<domenkozar>
we can quickly test on 10 hetzner machines today
<domenkozar>
I would prefer a solution that makes binutils setup-hook idempotent
<angerman>
your "today" :-)
<angerman>
my "today" is about to end :-)
<domenkozar>
well given this is a monthly saga, a day doesn't mean much anymore :D
<angerman>
I've been running some performance measurements on packet.net/aws... and I must say, I'm slightly disappointed in aws ec2
<domenkozar>
well, they trade latest and greatest for pricing
<angerman>
domenkozar: actually... pricing isn't even that much in their favour.
<angerman>
domenkozar: *if* I get a spot instance on packet, they are cheaper.
<domenkozar>
oh spot stuff
<domenkozar>
yes
<angerman>
getting aws spot instance is just ... well it seems silly not to.
<domenkozar>
depends
<angerman>
well, yes I was unable to get a c4.2xlarge on aws as a spot instance. But all the others, were not problem. And also for the duration of 2-4hs
<angerman>
domenkozar: no it's just nix build slaves. So if they die in the middle, I would just restart them. Worst case I lost a built package.
<angerman>
LnL: *IF* your workload is perfectly parallizable :(
<domenkozar>
angerman: well if you use something like cachix to save your binaries
<domenkozar>
you don't lose much indeed
<LnL>
sure, or you have enough packages
<angerman>
LnL: I'm basically looking at this funnel like workload, where it initially parallezises just fine, but has this linear dependency chain at the end.
<LnL>
ah :/
<angerman>
domenkozar: well the cache is currenly my local box.
<domenkozar>
oh you just use it as remote builders?
<domenkozar>
sure
<angerman>
But what's *really* annoying (and maybe cachex could help here) is that local -> packet.net (Amsterdam) @ 1mb/s; local -> packet.net (Tokyo) @ 2.5mb/s; local -> ec2 (Singapore) @ 25mb/s
<angerman>
and I can only get packet spot instances in amsterdam.
<domenkozar>
cachix is in AWS US east
<domenkozar>
so you'd probably get more
<domenkozar>
but note that nix uses xz
<domenkozar>
which caps throughput to 2-4MB/s
<domenkozar>
when compressing
<domenkozar>
not sure how well that parallelizes
<domenkozar>
if at all in nix
<angerman>
domenkozar: well. I'm basically uploading existing artifacts to a cold remote builder.
<angerman>
so they are essentially all pre-existing.
<angerman>
which is also why I can get 25mb/s to aws.
<domenkozar>
right
<domenkozar>
I haven't really benchmarked cachix world-wide, but I plan to do that soon
<angerman>
there's always cloudFront or fastly :p
<domenkozar>
well yes, that would be the end game :)
<angerman>
I would already be perfectly happy if I could have nix dump the artifacts onto an S3 bucket and use that as needed.
<angerman>
the annoying thing with fastly is their lack of free ssl though
<angerman>
another thing remote builder could optimize is: if we know we need to copy more to the server, keep copying while building expressions, and not have [copy] [build] [copy] sequenced.
<domenkozar>
well for that it needs stateful dependency tracking
<domenkozar>
but yeah
<angerman>
thinking about deps, I've just got an idea how to speed up haskell packages.
<angerman>
splitting tests from their packages, and running them speculatively as separeate deps would allow you to build more of the dependencies while waiting for the test to complete.
ixxie has quit [Ping timeout: 244 seconds]
<domenkozar>
well and currently we build twice
<domenkozar>
with and without profiling
<angerman>
I don't. But that' due to other reasons.
FRidh has quit [Quit: Konversation terminated!]
ixxie has joined #nixos-dev
ixxie has quit [Ping timeout: 244 seconds]
ekleog has quit [Quit: WeeChat 2.0]
ekleog has joined #nixos-dev
orivej has quit [Ping timeout: 268 seconds]
FRidh has joined #nixos-dev
Mic92 has joined #nixos-dev
goibhniu has quit [Ping timeout: 256 seconds]
<sphalerite>
I want to add a method to FSAccessor in nix/src/libutil/fs-accessor.hh for deeply resolving symlinks. I'm not sure how to return the various possible results though — there's either A, a final store path, B, a final path outside the store, or C, a loop. Anybody who knows C++ and nix's C++ style who could suggest the best way to do this?
Cale has quit [Remote host closed the connection]
orivej has joined #nixos-dev
Sonarpulse has joined #nixos-dev
orivej has quit [Ping timeout: 244 seconds]
orivej has joined #nixos-dev
<domenkozar>
ever tried to be hypnotised?
<domenkozar>
I think watching nix-bulid with 10 hetzner servers is working