<clever>
infinisil: ive looked into that before, and the biggest problem is the narinfo files, you have the hash of the build directions, not the hash of the result
<clever>
infinisil: so you cant just rely on ipfs's key/value store being hash(value) = value
<infinisil>
Yeah, you'd still need a lookup from <drv hash> -> <build content hash> in nixos.org
<infinisil>
Which actually also gets you the trust base
<clever>
my basic idea, is to just shove the ipfs hash of the .nar.xz, into the .narinfo
<clever>
but dont actually upload any objects to ipfs
<clever>
that would be a very trivial patch to add to hydra
<infinisil>
Ah yeah that would work
<clever>
then other people can upload .nar.xz files to their local ipfs node, as they feel like it
<infinisil>
Yeah that's a pretty nice idea, nix should then get an option useIPFSWhenAvailable or so
<infinisil>
I feel like it's almost worth learning C++ just to help with Nix development
<clever>
join the impure!
<clever>
join us!
<LnL>
I thought people already implemented parts of that
<LnL>
the way metadata propagates just doesn't scale for the amount of files that get generated
pie__ has quit [Ping timeout: 244 seconds]
<infinisil>
LnL: Not sure I get what you're saying
<LnL>
don't know the details, but there is (or was) some kind of performance problem
<gchristensen>
(it is very close to my time to go to sleep)
<gchristensen>
maybe cache-s3 is not very efficient :)
<clever>
`We have noted that the uploads/downloads of the caches, done by cache-s3, are much slower than what we would get with a simple aws s3 cp, often by almost a factor of 8/10`
<{^_^}>
jtojnar: 1 hour, 15 minutes ago <gchristensen> done
<Profpatsch>
clever: We already have stability issues with Cloud providers that are paid to do their jobs, if we add ipfs and torrents to the mix it’s going to be hell.
<Profpatsch>
especially if we try some fancy splitting the load between the three.
Sigyn has quit [Quit: People always have such a hard time believing that robots could do bad things.]
Sigyn has joined #nixos-dev
johanot has quit [Quit: WeeChat 2.4]
<Guest21659>
gchristensen: we could, I have not used it but you can define declarative jobsets
<gchristensen>
yeah, that would work for jobsets but the whole thing would be nicer :P
Guest21659 is now known as LnL
<manveru>
looks like b4acd9772975d549f491d48debb679cf4ec78133 broke the nevow python lib, which is required for tahoe-lafs :(
<manveru>
because nix-locate says it's in twisted, but that's already a dependency
drakonis has quit [Ping timeout: 252 seconds]
drakonis has joined #nixos-dev
<manveru>
ah, so seems like it needs to be added to checkInputs as well now
drakonis has quit [Read error: Connection reset by peer]
drakonis has joined #nixos-dev
<catern>
clever: for the most part the abstraction of a CDN is exactly the abstraction we need for the binary cache, modulo the fact that it requires you to use HTTP, which is not a constraint we have... hmm, are there any CDNs that say "hey we offer HTTP, but if you use [whatever protocol] directly, it'll be even faster"?
<gchristensen>
HTTP is a pretty decent protocol for it, I think the thing people miss is being able to have a loose, local cache
<emily>
just shell out to $BROWSER to download things from the cache. What could go wrong?
<ekleog>
catern: the abstraction we need for the binary cache is “fetch me this content-addressed file”, which is not exactly CDNs
<ekleog>
(not sure whether NARs are currently content-addressed but they definitely should be)
<catern>
ekleog: that's fair, we have more information, which the CDN abstraction throws away
ajs124 has joined #nixos-dev
<ekleog>
and hydra should (and AFAIR already does) make available a mapping from derivation hash to content-addressed hash
disasm has quit [Quit: WeeChat 2.0]
<catern>
ekleog: oh wait no you tricked me, it's not actually content-addressed :)
<catern>
yeah so I stand by what I said, the CDN abstraction is exactly what we want
<ekleog>
well I was still typing :p
<ekleog>
issue with the CDN abstraction is it requires HTTP and it requires a single actor for hosting a binary cache
<ekleog>
while the only “single actor” requirement we actually have is for hosting the derivation hash -> content-address mapping
<catern>
we want "here's the hash of a derivation, if you know that derivation and you have a build output for it, give me it"; it doesn't lose much information to reduce that to "here's a string, if you know that string and have something stored for it, give me it"
<ekleog>
here you're merging two independent tasks together, the derivation hash -> content-address, and the content-address -> content tasks
disasm has joined #nixos-dev
<catern>
yeah but that's because I'm describing the function we actually want to compute :)
<ekleog>
yes, but we don't have to compute it all with a single protocol
<catern>
the derivation hash -> content-address function isn't useful on its own, nor is the content-address -> content tasks function
<clever>
there is also a hash over the un-compressed form, which nix uses for validation
<clever>
FileSize: 930620
<clever>
NarSize: 5510000
<clever>
and then these are used to tell you how big the download will be, and how much it will use on-disk after unpacking
<catern>
(also I misspoke, I meant we go from out-path hash -> content, not derivation hash -> content)
<clever>
but, $out hash, is based on the hash of the derivation
<catern>
yeah, but it's not exactly the hash of the derivation
<catern>
just being more precise
disasm has quit [Quit: WeeChat 2.0]
drakonis1 has joined #nixos-dev
ajs124 has left #nixos-dev [#nixos-dev]
ajs124 has joined #nixos-dev
ajs124 has left #nixos-dev ["Machine going to sleep"]
ajs124 has joined #nixos-dev
<ajs124>
what's the most convinient way to bisect a kernel on nixos?
ajs124 has left #nixos-dev [#nixos-dev]
orivej has quit [Ping timeout: 244 seconds]
ajs124 has joined #nixos-dev
<catern>
so currently nix-shell, if TMPDIR is unset, sets TMPDIR=$XDG_RUNTIME_DIR; this is fairly annoying to me because XDG_RUNTIME_DIR is usually quite small (limited to 100MB on my system) and I run some processes inside nix-shell which can take up a fair amount of space (several GB)
<catern>
does anyone see a reason why to do this?
<catern>
On that note: nix-shell: don't use XDG_RUNTIME_DIR if TMPDIR is unset https://git.io/fjepw
<jtojnar>
can we temporarily forbid r-ryantm from sending pull requests? otherwise it will open one for every GNOME app
drakonis1 has quit [Quit: WeeChat 2.3]
drakonis_ has joined #nixos-dev
drakonis has quit [Ping timeout: 246 seconds]
drakonis1 has joined #nixos-dev
drakonis has joined #nixos-dev
drakonis has quit [Read error: Connection reset by peer]
drakonis2 has joined #nixos-dev
drakonis_ has quit [Ping timeout: 240 seconds]
drakonis1 has quit [Ping timeout: 245 seconds]
orivej has joined #nixos-dev
drakonis2 has quit [Ping timeout: 240 seconds]
drakonis2 has joined #nixos-dev
drakonis2 has quit [Read error: Connection reset by peer]
drakonis2 has joined #nixos-dev
drakonis2 has quit [Read error: Connection reset by peer]
drakonis2 has joined #nixos-dev
drakonis2 has quit [Read error: Connection reset by peer]
drakonis2 has joined #nixos-dev
niksnut has quit [Ping timeout: 244 seconds]
niksnut has joined #nixos-dev
drakonis has joined #nixos-dev
drakonis2 has quit [Ping timeout: 244 seconds]
drakonis_ has joined #nixos-dev
drakonis has quit [Ping timeout: 240 seconds]
drakonis has joined #nixos-dev
drakonis_ has quit [Ping timeout: 252 seconds]
drakonis has quit [Read error: Connection reset by peer]
drakonis has joined #nixos-dev
drakonis has quit [Read error: Connection reset by peer]
drakonis has joined #nixos-dev
drakonis has quit [Read error: Connection reset by peer]
drakonis has joined #nixos-dev
drakonis has quit [Read error: Connection reset by peer]