orivej has quit [Quit: No Ping reply in 180 seconds.]
orivej has joined #nixos-dev
alp has quit [Ping timeout: 272 seconds]
justanotheruser has quit [Quit: WeeChat 2.7.1]
justanotheruser has joined #nixos-dev
justanotheruser has quit [Ping timeout: 272 seconds]
justanotheruser has joined #nixos-dev
drakonis1 has quit [Quit: WeeChat 2.8]
cole-h has quit [Quit: Goodbye]
orivej has quit [Ping timeout: 265 seconds]
orivej has joined #nixos-dev
alp has joined #nixos-dev
ckauhaus has joined #nixos-dev
orivej has quit [Ping timeout: 256 seconds]
orivej_ has joined #nixos-dev
orivej has joined #nixos-dev
orivej_ has quit [Ping timeout: 272 seconds]
orivej has quit [Ping timeout: 260 seconds]
orivej_ has joined #nixos-dev
orivej_ has quit [Ping timeout: 272 seconds]
abbe_ has joined #nixos-dev
vdemeester_ has joined #nixos-dev
kalbasit_ has joined #nixos-dev
Haskellfant has joined #nixos-dev
infinisi1 has joined #nixos-dev
Jackneill has quit [Read error: Connection reset by peer]
Jackneill has joined #nixos-dev
michaelpj has quit [*.net *.split]
rycee has quit [*.net *.split]
thefloweringash has quit [*.net *.split]
danielrf[m] has quit [*.net *.split]
puzzlewolf has quit [*.net *.split]
ky0ko has quit [*.net *.split]
infinisil has quit [*.net *.split]
vdemeester has quit [*.net *.split]
kalbasit has quit [*.net *.split]
cocreature has quit [*.net *.split]
abbe has quit [*.net *.split]
Haskellfant is now known as cocreature
vdemeester_ is now known as vdemeester
kalbasit_ is now known as kalbasit
rycee has joined #nixos-dev
thefloweringash has joined #nixos-dev
ky0ko has joined #nixos-dev
danielrf[m] has joined #nixos-dev
puzzlewolf has joined #nixos-dev
michaelpj has joined #nixos-dev
__monty__ has joined #nixos-dev
alp has quit [Remote host closed the connection]
alp has joined #nixos-dev
FRidh has joined #nixos-dev
orivej has joined #nixos-dev
<domenkozar[m]>
Ericson2314: I really don't understand the motivation for IPFS
<domenkozar[m]>
the amount of work needed to make that happen is insane
<domenkozar[m]>
I know that filecoin sits on shitton of money to spread around, but it's probably a good idea to explain the motivation for all that complexity
noonien has joined #nixos-dev
<infinisi1>
qyliss: Hey, which survey software would you have recommended?
infinisi1 is now known as infinisil
drakonis_ has joined #nixos-dev
drakonis has quit [Ping timeout: 246 seconds]
drakonis has joined #nixos-dev
orivej has quit [Ping timeout: 246 seconds]
orivej has joined #nixos-dev
drakonis_ has quit [Ping timeout: 260 seconds]
drakonis has quit [Ping timeout: 272 seconds]
drakonis_ has joined #nixos-dev
drakonis has joined #nixos-dev
drakonis1 has joined #nixos-dev
drakonis has quit [Read error: Connection reset by peer]
drakonis has joined #nixos-dev
drakonis_ has quit [Ping timeout: 260 seconds]
drakonis1 has quit [Ping timeout: 272 seconds]
orivej has quit [Ping timeout: 246 seconds]
orivej has joined #nixos-dev
<gchristensen>
w.r.t. that hydra db weirdness niksnut thinks there is a bug in hpostgres
<LnL>
as in the perl library?
<gchristensen>
just postgres*
<gchristensen>
select * from jobs where jobset_id = 83 and name = 'nixpkgs.gnome3.gsettings_desktop_schemas.x86_64-linux'; -> 1 row: nixos | release-19.09 | nixpkgs.gnome3.gsettings_desktop_schemas.x86_64-linux | 83 ... select * from jobs where project = 'nixos' and jobset = 'release-19.09' and jobset_id = 83 and name = 'nixpkgs.gnome3.gsettings_desktop_schemas.x86_64-linux'; -> 0 rows
<LnL>
hmm did that change on the hydra instance recently?
<gchristensen>
not recently
<LnL>
wonder if that could be reproduced
<LnL>
import the dump with the exact same postgres as what's running
<LnL>
no idea if that's possible but maybe the index got out of sync with the actual table content somehow
<gchristensen>
even reindexing didn't work
<LnL>
ah already tried that
<gchristensen>
but yeah we have a snapshot of the system
<niksnut>
reindexing worked, after deleting the duplicates
<cole-h>
domenkozar[m]: Also, "burned by forgetting to update a hash of a Fixed Output Derivations" -> "burned by forgetting to update a hash of a Fixed Output Derivation" under "Nix: Re-running Fixed Output Derivations - At the Right Time" in "Tutorials & Resources"
<cole-h>
(No plural "derivation")
<andi->
hu, what is https://github.com/NixOS/hydra-ant-logger for? Is that logging for ant so hydra understands it? Is it the other way around? Just discovered that in the `hydra` project on hydra.
<domenkozar[m]>
Ericson2314: do you want me to include IPFS to nixos weekly?
<Ericson2314>
domenkozar: oh great idea! Thant would wonderful, thanks!
<timokau[m]>
Ericson2314: as a heads-up, the cert on the obsidian systems homepage has expired 4 days ago
<domenkozar[m]>
thanks everyone!
<Ericson2314>
timokau: you are the second person to tell me :) Should be fixed soon. Thanks though!
<drakonis1>
that's fantastic.
drakonis has quit [Ping timeout: 272 seconds]
<gchristensen>
Ericson2314: are you thinking the ipfs support would be a daemon front-end to ipfs, or inherent to nix itself?
justanotheruser has joined #nixos-dev
<timokau[m]>
Ericson2314: Oh well, better twice than not at all. Very excited to follow your progress :)
<gchristensen>
yeah!
<Ericson2314>
gchristensen: the nix daemon would talk to the ipfs daemon. But there is a question of e.g. should we be making a store plugin or putting it in nix proper
<FRidh>
lots of good news and tutorials this week
<gchristensen>
FRidh: so much!
<gchristensen>
Ericson2314: cool
<Ericson2314>
:) thanks you all!
<Valodim>
hoo boy. ipfs *seems* like it should be a great match for nix with its content addressing scheme, but I share domenkozar's concerns about benefit/complexity tradeoff
<clever>
Valodim: one problem i can forsee, what if version X of openssh has a security problem
<clever>
Valodim: what if i query ipfs, to see who has a copy of it?
<Valodim>
hmm. that's a rather specific concern
<clever>
there are also other problems
<clever>
for example, to get the most out of ipfs, you should share everything you have fetched from the cache
<clever>
but now you need 2 copies unpacked in /nix/store/, and the .nar.xz
<gchristensen>
plausible deniability and non-enumeration is important from a security perspective
<clever>
this project, lets you mount a .nar to /nix/store/foo
<clever>
with some tweaks, it could mount a .nar.xz
<clever>
then your nix store is just always compressed, and ipfs can share it as-is
<Valodim>
I think my biggest concern is what battles the nixos community wants to get involved in, and whether it wants to make "the decentralized web" one of them
<gchristensen>
Valodim: indeed, and as a plugin it can be a subset of the community that is interested in that
<clever>
gchristensen: yeah, there are also privacy concerns with the main ipfs daemon
<clever>
gchristensen: on first startup, it creates a permanent pub/priv keypair
<gchristensen>
yeah
<clever>
gchristensen: and on every future startup, it reports your public ip, and public key to the dht
<clever>
if i know your pubkey, i can track your ip
<gchristensen>
I *still* have ipfs nodes trying to connect to my IP, 1 year after I started up IPFS for about 15 minutes
<clever>
i mean more about knowing where you are at all times, if you leave ipfs on
<gchristensen>
right true
<Valodim>
gchristensen: if it was as clear cut as that, I wouldn't be worried :)
<clever>
the other issue, is the mix of content addressed vs not in nix
<gchristensen>
right, the split of input-addressed and output-addressed data
<clever>
this text file, says what the hash of .nar.xz is, the hash of the .nar within it, and the $out it unpacks to
<gchristensen>
I'm looking forward to seeing Ericson2314's team's solutions :)
<clever>
my solution, has been to just compute the ipfs merkle hash of the .nar.xz, and add it to the above .narinfo
<clever>
so cache.nixos.org acts as the index for input-addressed stuff
<clever>
and ipfs acts as the backing store for the content addressed stuff
<clever>
and cache.nixos.org is also the http seed, to re-populate ipfs when nobody has it
<Ericson2314>
clever: in what we have proposed, the ipfs daemon store is completely separate for the nix store
<Ericson2314>
which should help with the information leak
<clever>
Ericson2314: so you need 2 copies of the nix store?
<Ericson2314>
Yes
<Ericson2314>
well, of any content-addressed stuff
<Ericson2314>
if it's a legacy derivation, it's not going in IPFS
<clever>
Ericson2314: youll still have some information leaks, if you download openssh X, you will likely be keeping a copy in ipfs for sharing
<clever>
and youll advertise that fact in the DHT
<Ericson2314>
yes
<clever>
and now an attacker has a handy index of every IP that may be running a broken version
<Ericson2314>
don't use ipfs then :D
<clever>
Ericson2314: have you seen my narfuse idea?
<Ericson2314>
for me the important ting is that nobody unintentionally uses it
<gchristensen>
clever: this is an interesting reason to consider building it so the Nix <-> IPFS gateway is a separate daemon, so it can be run in a central spot which doesn't necessarily run what is in the cache
drakonis_ has quit [Read error: Connection reset by peer]
drakonis2 has joined #nixos-dev
alp has joined #nixos-dev
drakonis has quit [Ping timeout: 252 seconds]
alp has quit [Ping timeout: 272 seconds]
alp has joined #nixos-dev
<Mic92>
Ericson2314: I thought about using ipfs as decentral cache for files that we right now use `requireFile` for. Do you think one can use IPLD to bridge between our sha256sums that we use in requireFile and ipfs hashes?
<Ericson2314>
Mic92: are those flat hashes, nar hashes, or both?
<Ericson2314>
I'm hoping to move us towards git hashes (and git to move beyond sha1!)
<Ericson2314>
so both sides speak the same hashes and we don't have indirection which is both slow and makes GCing/pinning much harder
<Mic92>
Ericson2314: It's the outputHash of a file
<Mic92>
in the nix store
<Ericson2314>
Mic92: oh right that is always failing fixed ouput
<Ericson2314>
so that could be "flat" or "recursive"
<Ericson2314>
Mic92: yeah so phase 1 of our project is for things like that
<Mic92>
Ericson2314: Ok. So one could make requireFile automatically fallback to ipfs?
<Ericson2314>
Mic92: IPFS will be a type of remote store
<Ericson2314>
so anything with a hash IPFS can understand it can fall back on
<Ericson2314>
no hashed mirrors hacks
<Ericson2314>
(specifically, an IPFS daemon's is the remote store, rather than the network as a whole)
<Ericson2314>
* (specifically, an IPFS daemon is the remote store, rather than the network as a whole)
<Mic92>
Ericson2314: ok, but I could teach ipfs these hashes? My intention is to stay compatible with the hashes we use in nixpkgs so it is still usable for other users.
<Ericson2314>
Mic92: well that's why i went with git, it's not the native thing for nix or IPFS
<Ericson2314>
so it's a good meet in the middle for v1
<Ericson2314>
Perhaps latter we can say IPFS should know about nar hashes
<Ericson2314>
though I kinda dislike nar as a format as it's so bad for dedup
<Mic92>
Ericson2314: it's probably useful for migration.
<Ericson2314>
Agreed
<hyperfekt>
i feel like an argument could be made for nix to switch to unixfs-v2 in the long term
<Mic92>
The last time someone tries ipfs as a binary cache for nix, performance was terrible. I hope they fix it.
<Ericson2314>
With both sides now supporting more than one format (not just nars for us), both sides' codebases will have an "open mind" so to speak (abstractions), and we can see how things play out
<hyperfekt>
Mic92: i was told that the latest release of the go client, 0.5, fixes most of that?
<Ericson2314>
hyperfekt: unixfs-v2 does support all sorts of file system attributes and things we don't want though
<Ericson2314>
though I suppose we could just error if anything isn't something we can/want to reproduce in the build sandbox
__monty__ has quit [Quit: leaving]
<hyperfekt>
Ericson2314: yeah, i suppose it is already technically possible to refer to something that doesn't represent a valid nar. there's just no tool doing so that's why it doesn't come up, and i don't see why that needs to be different with unixfs-v2
<Ericson2314>
yeah dangling hashes are always fine
<hyperfekt>
but yeah, the IPLD effort makes all of this implementation-level concerns
<Ericson2314>
hyperfekt: other thing is we consume so much data from git
<Ericson2314>
and I since I use nix as much as a developer as distro person
<Ericson2314>
I am perhaps biased thinking about CI use-cases where i just wanna build fresh git tree hashes non-stop
<Ericson2314>
i.e. tons of data (since new PRs are popping up) and it's already in that format
<hyperfekt>
yeah, it would be great to use an IPLD storage agent for git. i remember the failed attempt to integrate incremental git fetching into nix
<Ericson2314>
right yeah we ought to be able to do all the awesome incremental stuff this way
<hyperfekt>
i will be following your efforts very closely :D
<hyperfekt>
i looked at IPLD only incidentally because i want to share content-defined chunks of files
<hyperfekt>
but i'm a big fan of all things distributed
Jackneill has quit [Ping timeout: 240 seconds]
Jackneill has joined #nixos-dev
orivej has quit [Ping timeout: 258 seconds]
orivej has joined #nixos-dev
<Ericson2314>
hyperfekt: :) glad to hear it!
<colemickens>
hyperfekt: I am super interested in ipfs and adjacent stuff for that sort of thing too. hyperdrive-daemon, #datrs, and asuran have all been on my radar recently, and unix v2 in ipfs, but I feel like I've been waiting forever for it.
<colemickens>
just randomly mentioning them. I feel like if I squint, some sort of asuran ideas, combined with hyperdrive-daemon (which appears to be dat v3 sort of)... I should write a random dump gist of my thoughts about this stuff sometime
<hyperfekt>
i feel like there is a lot of potential to get distributed data off the ground in standardizing on one highly flexible protocol with the experiences gathered by all these projects, but it needs to be done well
mmlb has joined #nixos-dev
<Ericson2314>
does "highly flexible" mean "all the features" or "smallest building block"?
<hyperfekt>
Ericson2314: the latter. the goal should be to pool efforts and reduce fragmentation in the parts where behavior can be shared.
orivej has quit [Quit: No Ping reply in 180 seconds.]
orivej has joined #nixos-dev
cole-h_ has joined #nixos-dev
cole-h has quit [Ping timeout: 256 seconds]
orivej has quit [Ping timeout: 265 seconds]
orivej_ has joined #nixos-dev
orivej_ has quit [Quit: No Ping reply in 180 seconds.]
orivej has joined #nixos-dev
justanotheruser has quit [Ping timeout: 260 seconds]
justanotheruser has joined #nixos-dev
cole-h_ is now known as cole-h
orivej_ has joined #nixos-dev
orivej has quit [Ping timeout: 264 seconds]
orivej_ has quit [Ping timeout: 265 seconds]
orivej has joined #nixos-dev
justanotheruser has quit [Ping timeout: 272 seconds]