jtojnar has joined joined #nixos-dev
Sonarpulse has quit [(Ping timeout: 240 seconds)]
stites[m] has joined joined #nixos-dev
pstn has joined joined #nixos-dev
hedning[m] has joined joined #nixos-dev
copumpkin has joined joined #nixos-dev
nocent has joined joined #nixos-dev
hl has joined joined #nixos-dev
sphalerite has joined joined #nixos-dev
regnat[m] has joined joined #nixos-dev
grahamc has joined joined #nixos-dev
olejorgenb[m] has joined joined #nixos-dev
rycee has joined joined #nixos-dev
moredread[m] has joined joined #nixos-dev
FRidh[m] has joined joined #nixos-dev
peterhoeg has joined joined #nixos-dev
mbrgm has quit [(Ping timeout: 248 seconds)]
mbrgm has joined joined #nixos-dev
orivej has quit [(Ping timeout: 246 seconds)]
_ts_ has joined joined #nixos-dev
disasm has joined joined #nixos-dev
FRidh has quit [(Quit: Konversation terminated!)]
goibhniu has quit [(Ping timeout: 240 seconds)]
ocharles has quit [(Ping timeout: 255 seconds)]
simpson has quit [(Ping timeout: 255 seconds)]
jtojnar has quit [(Ping timeout: 264 seconds)]
ocharles has joined joined #nixos-dev
simpson has joined joined #nixos-dev
tv has quit [(Ping timeout: 268 seconds)]
tv has joined joined #nixos-dev
vcunat has joined joined #nixos-dev
goibhniu has joined joined #nixos-dev
mbrgm has quit [(Read error: Connection reset by peer)]
mbrgm has joined joined #nixos-dev
<LnL> having a flag like jq -r for nix-instantiate would be nice
__Sander__ has joined joined #nixos-dev
<Mic92> rycee: some naming conventions written in stone, would be nice
<Mic92> fetchNodeModules looks saner to me then this node-packages-v6.nix monster in nixpkgs: https://github.com/NixOS/nixpkgs/pull/31137/files/cc04a30b6e14c4a6b2353766c8e22a6e96d30e76#diff-e4f5f72b1be5f9524ba0c733eb0bfb12
JosW has joined joined #nixos-dev
bgamari has quit [(Quit: ZNC - http://znc.in)]
bgamari has joined joined #nixos-dev
bgamari has quit [(Ping timeout: 240 seconds)]
bgamari has joined joined #nixos-dev
dos4gw has joined joined #nixos-dev
<copumpkin> niksnut: does the channel server keep around old channel URLs like https://d3g5gsiof5omrk.cloudfront.net/nixpkgs/nixpkgs-18.03pre119383.eafd703a63/nixexprs.tar.xz ?
<copumpkin> or do those get collected after a while?
FRidh has joined joined #nixos-dev
orivej has joined joined #nixos-dev
<globin> copumpkin: AFAIK nothing on s3 is ever collected
<niksnut> yeah, currently nothing is deleted
<niksnut> before I moved releases to S3, I did delete them fairly regularly
<niksnut> which may happen again in the future, so it's best not to rely on them
jtojnar has joined joined #nixos-dev
<copumpkin> niksnut: I guess I can just use a nixpkgs github tarball link for the same effect
<niksnut> yeah
pbogdan has left #nixos-dev ["ERC (IRC client for Emacs 25.3.1)"]
pbogdan has joined joined #nixos-dev
seppellll has joined joined #nixos-dev
FRidh has quit [(Quit: Konversation terminated!)]
<mguex_> Who here feels appointed to operate a community operated binary cache (i.e. not AW$)? I started the project with fpletz but there hasn't been much progress since a few weeks. I myself do not want to take the lead, but I am happy to hand over the project.
mguex_ is now known as mguex
<mguex> If you have questions, feel free to /query
<fadenb> mguex: I believe nobody will commit to something like this without more information upfront. What I would be interested in is the amount of traffic required for such a cache
<cransom> mguex: how would a community cache work? and do you have a proposal or rfc?
<mguex> fadenb: sure
<mguex> cransom: We have a machine that fetches all (binary) outputs and narinfos from cache.nixos.org and makes them available through http(s) and rsync.
<mguex> We obtain the list of nars by querying hydra jobsets and retain `n` evaluations (currently 9 per channel/release)
<mguex> That amounts to ~ 350 GiB
<MoreTea> this will probably become viable once we can sign per-nar in nix 1.12
<mguex> MoreTea: It's already viable: Hydra signs the .nar's, so as long as you have the pubkey in your nix.conf it does not matter where you get your nars from
<mguex> trafficwise: Frankly I have no idea. only niksnut knows what is going on on S3/AWS. The goal is to have a "core" infra with 3-5 nodes that redirect requests "cache.nixos.community" to mirrors close to the requesting IP (this is done through mirrorbits software)
<mguex> You can find the sourcecode @ https://github.com/NixIPFS/nixipfs-scripts and the result @ rsync rsync://cache.rrza.de
<mguex> I will publish the current nix config for the exporter machine soon to https://github.com/NixIPFS/infrastructure
<MoreTea> oh, nice.
<mguex> binary-cache-public-keys = hydra.nixos.org-1:CNHJZBh9K4tP3EKF6FkkgeVYsS3ohTl+oS0Qa8bezVs=
<mguex> MoreTea: ^ is all that you need in nix.conf
<mguex> fadenb: I am searching for people who are willing to operate the core infrastructure machines. As I said, these machines only distribute the traffic and do not need to serve themselves. This can be done by machines "downstream", i.e. in universities
<copumpkin> niksnut: seeing a hash change with 1.12 :O
<copumpkin> is that expected?
<copumpkin> just recent version of 1.12 too
<copumpkin> like I evaluate the output path of something in 1.12 before updating and it produced one value, and after updating it produced another value
Sonarpulse has joined joined #nixos-dev
<niksnut> what hash?
<niksnut> .drvs can easily change
<copumpkin> the actual output hash, the nix-store -r
<copumpkin> trying to figure it out because it's behaving super strangely now
<copumpkin> basically ran a nix-build on something with 1.11, it gave me hash X. Then I ran again with nixUnstable and gave me hash X again, then I nix-channel --updated and updated unstable and it gave me hash Y
<niksnut> something that can be reproduced easily?
<copumpkin> well, this is what's confusing now
<copumpkin> because after I exited the nix-shell with nixUnstable producing hash Y
<copumpkin> now 1.1 gives me Y too
<copumpkin> but I still have the .drv for X so I didn't just imagine it
<copumpkin> I don't know how easy to repro it is yet
<copumpkin> still trying to figure out what's going on
<copumpkin> what's the easiest way to go from a list of .drvs to a list of their output paths without actually realizing them?
__Sander__ has quit [(Quit: Konversation terminated!)]
<clever> copumpkin: lib.inNixShell
<clever> copumpkin: that returns true for both nix-shell based evals, and nix-build's ran inside nix-shell
<copumpkin> hmm, what's that in response to?
<clever> /home/clever/apps/nixpkgs/lib/trivial.nix: inNixShell = builtins.getEnv "IN_NIX_SHELL" != "";
<copumpkin> yeah I know about it, but trying to tie it back to my question
<clever> 2017-11-07 12:51:53 < copumpkin> because after I exited the nix-shell with nixUnstable producing hash Y
<copumpkin> still not tying it back other than a mention of nix-shell
<copumpkin> I'm getting nix-build producing different outputs on the exact same input
<copumpkin> based on the version of nix
<copumpkin> and seemingly some state
<clever> that version of nix may rely on the state of IN_NIX_SHELL
<copumpkin> because after I run the newest nixUnstable, it always produces the hash
<clever> try re-running both nix-build under nix-shell
<copumpkin> it wasn't always in nix-shell
<copumpkin> that was just one test
<clever> ah
<copumpkin> but I also was doing it with full explicit path
<copumpkin> all very weird
<clever> another thing that can do it, is the result symlinks and src = ./.;
<copumpkin> nah, not doing any of that here
<copumpkin> it seems to be literally just the nix version changing
<copumpkin> niksnut: how did filterSource need fixing?
<clever> i did see something about how indentation works inside '' strings, but i'm not sure when that takes effect
<copumpkin> or was it sooner?
<clever> copumpkin: i think it was just changed from using a struct FilterFromExpr to inlining the entire thing on the new line 1025
<copumpkin> yeah the change doesn't look too deep
<copumpkin> hmm
<copumpkin> :/
<vcunat> Mic92: I was cancelling evals on Hydra due to imminent staging merge (and larger rebuilds on master that were pushed in parallel)
<vcunat> mguex: I think I can operate such a machine, starting within a few months. It's been in my long-term plans and now I have the machine itself. That is, assuming I can get the necessary stuff into an armv7 container.
<vcunat> I probably don't understand your distribute vs. serve distinction, but I have 100Mbps symmetric connection, and I think can sustain lots of traffic.
<vcunat> ANN: glibc-2.26 is in master, bringing perhaps more breakages than anticipated https://github.com/NixOS/nixpkgs/commit/6ffafc78fbcde4d
<vcunat> In particular, if you have an idea how to fix tcp_wrappers, that would be great, as it's a blocker for both channels. I took a few approaches already, but the link always fails, even if I explicitly add -lnsl - and libnsl.so (from ${glibc}) does list the "missing" symbol: 00000000000048a0 T yp_get_default_domain
goibhniu has quit [(Ping timeout: 268 seconds)]
<Dezgeg> it's a versioned symbol only apparently: yp_get_default_domain@GLIBC_2.2.5
<copumpkin> okay, somehow fetchFromGitHub started returning a path called "source" instead of the old name
<copumpkin> anyone have any idea how that would happen? I haven't updated nixpkgs which is the only thing I can think of that would affect fetchFromGitHub's name
_ts_ has quit [(Remote host closed the connection)]
vcunat has quit [(Ping timeout: 250 seconds)]
<copumpkin> never mind
<copumpkin> turns out the -source thing broke everything for me :)
<copumpkin> niksnut: thanks a lot :P
<copumpkin> the whole "the hash didn't change but the store path did" thing is what bit me, fwiw
<copumpkin> because when I'm building images it actually can matter
vcunat has joined joined #nixos-dev
<Sonarpulse> friendly reminder I'm looking to merge https://github.com/NixOS/nixpkgs/pull/30549 tomorrow :)
romildo has joined joined #nixos-dev
<vcunat> Sounds OK at a higher level.
<copumpkin> what was the final outcome of the "how best to pin nixpkgs" discussion?
<copumpkin> at nixcon
<copumpkin> I think someone said that Tekmo had a good solution
romildo has quit [(Quit: Leaving)]
JosW has quit [(Quit: Konversation terminated!)]
orivej_ has joined joined #nixos-dev
orivej has quit [(Ping timeout: 250 seconds)]
_ts_ has joined joined #nixos-dev
Sonarpulse has quit [(Ping timeout: 240 seconds)]
Sonarpulse has joined joined #nixos-dev
<MoreTea> copumpkin: I think it was comparing the version iof Nix, using builtins.fetchurl with sha256 if >= 1.12, otherwise the trick of importing stuff from nix's closure
<copumpkin> yeah
<copumpkin> thanks :)
<copumpkin> so what controls the store path?
<copumpkin> I'm trying to figure out why different nix versions have different store paths in that solution
<clever> copumpkin: diff the 2 derivation trees until you find the edge between same and different
<clever> copumpkin: which is tricky, because you need to compare the $out of each derivation in the --query --deriver tree
<copumpkin> well, it has no real dependencies
<clever> that makes the diff simpler
<copumpkin> yeah, so the output hash is the same
<copumpkin> but I don't know what else to diff
<clever> try comparing the input .drv files?
<copumpkin> they're different in a bunch of ways, unfortunately
<copumpkin> this wasn't really the case with the import fetchFromGitHub approach
<copumpkin> that's why I'm trying to figure out what actually goes into the store path hash
<copumpkin> as opposed to the output hash
<copumpkin> since the content is identical
<copumpkin> know where that logic is in the nix source? or is it documented anywhere?
<copumpkin> would be nice to have a summary page of "what goes into all the flavors of hash in nix land"
<copumpkin> .narinfo, .nar, .drv paths, their associated output paths, etc.
<copumpkin> nice spec above that in a giant comment
vcunat has quit [(Ping timeout: 246 seconds)]
<copumpkin> oh it's obvious now why the hash is different
* copumpkin feels silly
<copumpkin> because fetchNixpkgs doesn't return a fixed-output derivation
<LnL> it doesn't?
<copumpkin> not really
<copumpkin> in 1.12 it's not even a real derivation
<copumpkin> and in 1.11 and earlier, it's a normal derivation depending on a fixed-output derivation
<copumpkin> the unpacker
<LnL> oh right it's the unpack part
<copumpkin> which screws me
<LnL> <nix/fetchurl.nix> is right?
<copumpkin> hmm
<copumpkin> yeah
<copumpkin> it even has an unpack parameters
<copumpkin> but the unpack parameter only unpacks nars
* copumpkin shakes his fist angrily
<LnL> odd, maybe that was added before the tar/gzip stuff?
<copumpkin> probably
<copumpkin> so I don't think I can use fetchNixpkgs for now
<copumpkin> I'll do something simpler I guess
<LnL> I knew about the builtin fetchurl but had no idea you could reference coreutils and stuff without nixpkgs
<copumpkin> oh wait, I can actually
<copumpkin> oh I have a very sneaky plan
<copumpkin> maybe I shouldn't though
<copumpkin> yeah it's ugly, and requires a hash of the unpacked and packed stuff, maybe not
<LnL> what about fetchTarball + fixed output drv
<copumpkin> actually yes
<copumpkin> I'll do this
<copumpkin> I can't use fetchTarball yet
<LnL> why?
<copumpkin> in 1.11 it didn't support a sha256
<LnL> sure, but I was thinking to use the one without hash
<copumpkin> I can't use that for various other reasons
<LnL> the fixed output drv would avoid evaluating the src every time
<copumpkin> I'm gonna try something evil
Sonarpulse has quit [(Ping timeout: 248 seconds)]
seppellll has quit [(Ping timeout: 250 seconds)]