<gchristensen>
yeah, this seems like a bad bug in Stack
<yegortimoshenko>
it's almost every tool, tbh
<yegortimoshenko>
cabal.project, mix, rebar3, even yarn
<yegortimoshenko>
clojure's deps.edn
<domenkozar>
anyone seeing more "error 7 while decompressing xz" lately?
<clever>
domenkozar: ive sometimes seen that with nix 1.11 when curl fails to download, so xz receives an early eof
<domenkozar>
Using nix 2.1.1
init_6 has quit [Ping timeout: 252 seconds]
<domenkozar>
could be the new fastly?
<clever>
maybe, check what -vvv says, and if you can reproduce it
<domenkozar>
could be my connection, will see if it persists
<cransom>
wd
<cransom>
oops. pardon me.
<clever>
domenkozar: is your fiber still glued into its socket?
<domenkozar>
:D
<domenkozar>
I don't want to check those spagetti
<clever>
for me, one end of the fiber is at the top of a pole, and the other is inside the house, so nobody should ever be touching either end
<domenkozar>
here the box is in middle of garage
<domenkozar>
and it has no doors
<clever>
:'(
<gchristensen>
andi-: I think one reason chromim takes so long sometimes is all the tests
<gchristensen>
when tests run, context switches go way up
<domenkozar>
gchristensen: fastly seems cool, but do we get any benefits from switching over?
<gchristensen>
domenkozar: we don't pay for it, it is sponsored :)
<domenkozar>
isn't cloudfront free?
<domenkozar>
well it makes S3 cheaper
<gchristensen>
no, we pay a lot for bandwidth out
<domenkozar>
afaik S3 bandwidth out is like 0.09$
<domenkozar>
while with cloudfront it's 0.085$
<gchristensen>
assuming an effective cache rate, fastly should be quite a bit cheaper for us
<andi->
gchristensen: even the "normal" compilation seems slower on the box.. Yesterday when the epyc machine was full of "stale" jobs and basically just building chromium it was also incredible slow.. I think my 6y old desktop did better :/
<gchristensen>
andi-: hrm, what time?
<domenkozar>
gchristensen: but still need to pay for traffic S3<->fastly?
<domenkozar>
which was the cost for cloudfront before
<andi->
gchristensen: must read backlog. In public transit right now.. Will ping you later about it.
<gchristensen>
domenkozar: yes, but should still be much lower since they will fetch much less frequently than users fetched
<domenkozar>
gchristensen: so we'll save on better caching?
<gchristensen>
no, we're saving because the CDN part is free
<gchristensen>
so we will save on every cache hit
<domenkozar>
ah :)
<yegortimoshenko>
fwiw, wasabi is $5/tb/mo and free egress :-)
<domenkozar>
I wouldn't trust wasabi with a stick :)
<domenkozar>
they have some weird stuff going on
<gchristensen>
same, that rate isn't making them money and isn't going to last
<yegortimoshenko>
perhaps
<domenkozar>
they are either cheating
<gchristensen>
it would cost significant money and time to move the binary cache, and moving to a budget provider seems like a bad choice
<domenkozar>
or they are reducing redundancy
<domenkozar>
or they'll go bankrupt
<domenkozar>
all of which is a no-no
<ekleog>
so if I understand now, the cache is 1. built on hydra at hydra.nixos.org, 2. stored on S3 at cache.nixos.org, 3. cached by cloudflare, 4. cached by fastly, 5. delivered to the users? :D
<ekleog>
3 cache layers, we'll soon be able to overcome x86 in number of caches
<ekleog>
s/x86/recent intel processors/
<aminechikhaoui>
no, builds are done in hydra, pushed to s3 and fastly is used as a CDN for the s3 bucket
<gchristensen>
no more #3
<domenkozar>
so is it now
<domenkozar>
cloudfront -> fastly
<domenkozar>
or s3 -> fastly? :)
<yegortimoshenko>
s3 -> fastly
<domenkozar>
using cloudfront could still save 5%? :)
<ekleog>
if fastly mostly hits, it'd double the price
<ekleog>
because data would need to be fetched first from s3 then from cloudfront
<yegortimoshenko>
cloudfront is free, you only pay for egress, so it wouldn't be double the price
<ekleog>
well, you'd pay egress from s3 and egress from cloudfront instead of just egress from s3
<domenkozar>
cloudfront model is that you get overall discount on egress
<ekleog>
assuming fastly hits as much as cloudfront, at least
<yegortimoshenko>
egress inside aws is free, too
<domenkozar>
ekleog: no, s3<>cloudfront is free
<domenkozar>
it's withing aws
<ekleog>
oh, didn't know
<domenkozar>
s3<>fastly is 0.09$
<domenkozar>
s3<>coudfront<>fastly is 0.085$
<yegortimoshenko>
domenkozar: yup, you're right. if there was cloudfront between s3 and fastly, it would be $0.085 not 0.09
<ekleog>
so yeah, using cloudfront could still save a bit?
<domenkozar>
niksnut: in other news, you got it right a decade before haskell :-)
<LnL>
oh I should watch that, I generally like his talks
<LnL>
btw, the hercules is pretty exciting :)
primeos has quit [Ping timeout: 252 seconds]
primeos has joined #nixos-dev
<domenkozar>
LnL: thanks :)
<LnL>
I understand people are not happy about the decision but I thought the project was basically dead
<domenkozar>
we got lots of positive private feedback
<domenkozar>
but very few public :)
<LnL>
and a ci system that isn't made to build _all of nixpkgs_ is kind of missing
<yegortimoshenko>
people kinda use buildkite for that for now
<domenkozar>
it's vaporware atm so I think once folk see that we're onto something they'll be onboard :)
<domenkozar>
turns out building a CI is tons of work
<clever>
another haskell guy i know claims he can rewrite hydra in a day, lol
<clever>
i doubt he can
<domenkozar>
probably I'm really bad at haskell working on this since April
<domenkozar>
but happy to hire him for a day :D
<LnL>
clever: including the language extensions? :D
<domenkozar>
clever: oh I see the catch, his daily rate is $1M
<simpson>
The core-most functionality of Hydra, and the stuff that people generally want for themselves, could probably be written in a day.
<domenkozar>
I'll be quiet :)
<simpson>
But what Hydra does for the Nix community at large probably takes as long to write as Hydra has taken.
<clever>
simpson: yeah
<Synthetica>
But that's the same as saying you could rewrite anything in a day, just because you could get some of the core functionality down
<yegortimoshenko>
simpson: you can write `nix-build` git hook in a day
<yegortimoshenko>
github webhook*
<domenkozar>
nix build --builders
<domenkozar>
done
<simpson>
Synthetica: There are some things that *can* be written in a day, and basic invocation of Nix is one of them.
<infinisil>
Coding Minecraft In 24 Hours Amazing Speedcoding!
<domenkozar>
simpson: we have for example evaluation errors per attribute
<yegortimoshenko>
:-)
<domenkozar>
that was a few days :-)
<clever>
infinisil: how much research did he have to do prior to that though :P
<clever>
semi-related, ive had bugs before, that i could swear ive already fixed, yet i couldnt find any trace of the code on any machine, so i just retyped the entire fix from memory
<clever>
and i have to wonder, did i fix it in a dream? lol
<infinisil>
Hehe
<LnL>
lol
<simpson>
I know and hate that feeling.
<simpson>
Because I can never remember what the actual fix is!
<clever>
the freaky thing is that i had every line of the fix memorized
<clever>
yet it was just missing on all copies of the source
<simpson>
Oh, huh. That's starting to sound like the Tetris Effect.
<LnL>
I usually have the reverse, vaguely remember fixing having fixed something before, but when I look at the code I find horrible code with an obvious bug
<clever>
simpson: yeah
<LnL>
your own code from a year ago is the worst :p
<clever>
LnL: i see that every time i open my config.nix lol
<clever>
nix code from the early days, when i was still learning nix
<manveru>
domenkozar: i'm building my own CI for Nix atm too :P
<yegortimoshenko>
manveru: is it public? :-)
<manveru>
the public version is pretty old unfortunately, as i had to tweak it a lot to work in our infrastructure
<manveru>
but i shall make a clean version this week :)
<yegortimoshenko>
i've had https://github.com/serokell/derivery but it's essentially glorified `nix-build` that listens for github webhook events and posts logs to gist, it's pretty bad. hercules sounds way more integrated/interesting
<manveru>
yeah
<manveru>
mine handles PR status and distributes builds and updates the binary cache
<manveru>
so the thing i'm working on atm is a good UI
<yegortimoshenko>
does it stream logs? :-) i found that suprisingly hard to do, and weirdly enough, i haven't found any standalone log streaming services or applications to
<manveru>
yeah, what's so hard about it?
<LnL>
streaming logs is annoying
<manveru>
you basically just write to websocket... it's not super performant, but good enough for my use
<manveru>
granted, that's the point where i had to give up on crystal, because it was just impossible to implement there without hitting segfaults :P
<manveru>
but the go rewrite wasn't hard
<manveru>
anw, i think this is the wrong channel
<manveru>
yegortimoshenko: i plan to present everything at NixCon and try to get it polished until then
<yegortimoshenko>
cool! i'll watch in on yt right after then :-)
<manveru>
don't really want to compete with Herkules, but given our IT departments fear of anything not hosted inside our DCs, i doubt we'd ever be able to use it
<LnL>
how do you guys handle caching vs trust and development builds?
Lisanna has quit [Quit: Lisanna]
jtojnar has quit [Remote host closed the connection]
jtojnar has joined #nixos-dev
matthewbauer has joined #nixos-dev
matthewbauer has quit [Ping timeout: 264 seconds]
jtojnar has quit [Read error: Connection reset by peer]
matthewbauer has joined #nixos-dev
matthewbauer has quit [Ping timeout: 240 seconds]
Lisanna has joined #nixos-dev
Lisanna has quit [Client Quit]
<andi->
I spent a bit of time reading (&greping) hydra and nix code to figure out how the build timeout is transmitted from hydra to the daemon. I found the wopBuildDerivation code in the nix repo and what looks like the counter-part in hydras build-remote.cc. The protocol seems to have a mismatch there. Hydra sends the silenTime & buildtimeout while the daemon expects the BuildMode... Am I missing another layer?
<andi->
ahh, it talks to nix-store --serve.. disregard my comment then.. sometimes writing things down makes you realize what is actually happening..
<LnL>
are you sure it's that operation?
<andi->
it wasn't.. it is in a different part of the nix repo.. I found this comment and the protocol also matches: case cmdBuildDerivation: { /* Used by hydra-queue-runner. */
<LnL>
ah, that's serve-protocol.cc I think
<andi->
nix-store.cc ;-)
<LnL>
after it receives the commands, but that's not what goes over ssh I think
<LnL>
or am I misremembering
<andi->
the linked hydra code is the part that goes via SSH as far as I understood (doesn't have to but can go through SSH). I am now tracing the calls within the nix repo from nix-serve.. It's kinda fun to read that stuff finally..
Lisanna has joined #nixos-dev
<gchristensen>
speaking of hydra questions, when using a "Git checkout" input source, does the checkout have a .git dir?
<yegortimoshenko>
don't see anything that would remove it, and last time i used hydra i think it was there, but i might misremembering things.
<gchristensen>
hmm only if deepclone is set
<andi->
awwhh. I think I might know why the timeouts aren't working... the store connection is opened (to the local daemon) whenever nix-store --serve is being invoked. Only after the store connection is open it reads the jobs/ops from stdin (SSH). Since the options (including timeouts) are only sent on initial openStore() the received values never make it to the actual build daemon...
<andi->
so sending options a 2nd time would be a hack around that..
<LnL>
gchristensen: no, not in my experience
<gchristensen>
yeah, same, though it does if deepClone is specified