<tilpner>
Right now the size difference is acceptable, but it won't be with a much larger amount of users
<Mic92_>
right
<Mic92_>
and we don't even have auto-generated stuff so far.
<tilpner>
There will always be a need for dynamic linking of user repos
<infinisil>
Hmm..
<Mic92_>
At the moment I see nur-combined as a backup, when upstream disappears.
<tilpner>
One of the proclaimed usecases of the NUR is "Automatic generated package sets (i.e. generate packages sets from PyPi or CPAN)"
<infinisil>
Ah yeah, that would be a bit too large possibly
<infinisil>
for nur-combined
<Mic92_>
This was also only an idea of mine, it is not written in stone though.
<tilpner>
Exactly. So there will be size-limits, and repos above that size will be linked dynamically
<infinisil>
Alright thanks
<infinisil>
I'm currently thinking how we could deeply integrate nur with nixpkgs/NixOS
<tilpner>
Add a module that enables the NUR
<tilpner>
Then build a search UI for NUR content
<infinisil>
Ah yeah, the module for nixos modules
<tilpner>
No, not that
<tilpner>
Just for pkgs
<tilpner>
It's better than nothing
<infinisil>
Enable?
<infinisil>
I'm thinking of just having a `pkgs.nur` attribute for accessing all of NUR
<tilpner>
some.option.path.nur.enable = true; => NUR is overlayed
<infinisil>
I think the namespacing of nur is something one would want to keep
<Mic92_>
where could I upload a packages.json for the search?
<tilpner>
It's probably not okay to have in nixpkgs-default
<infinisil>
tilpner: That's where I'm going for though, because then NUR can serve as the default way to declare third party stuff
<tilpner>
infinisil - Doesn't that hit all sorts of hydra policy walls?
<infinisil>
Don't let hydra build it, there's a way to make it ignore it
<tilpner>
If you can sneak IFD in... do it!
<infinisil>
tilpner: You mean to IFD the NUR repo?
<tilpner>
Yes
<infinisil>
That's what I'm worried about, beceause people will want to have reproducible configs
<tilpner>
You're not to pull nur-combined into nixpkgs, are you?
<infinisil>
Nah :P
<tilpner>
*not going to pull
<infinisil>
There needs to be a way to have a pinned nur, that's not too annoying..
<tilpner>
Well, the usual pinning via json files will work
<infinisil>
Ah! You mean to put the NUR nix files in nixpkgs and call the repos from that? :O
<infinisil>
Nice
<infinisil>
Wait, but then the json files still need to be updated
<tilpner>
But if you pin NUR in nixpkgs, how will NUR packages reach older releases without constant backporting?
<tilpner>
(Or maybe automated backporting is okay?)
<infinisil>
Yeah.. darn
<infinisil>
How about this: We put a `nur = import (fetchGit "<master NUR>");` into nixpkgs
<infinisil>
And we set up some infrastructure to have repo revisions
<infinisil>
Users can then either pin the repo revision which will leave them with the pinned one, or just use the latest one
<infinisil>
(revision being an integer 0, 1, ...)
<infinisil>
The integer would get increased every time a repo gets updated by the nur bot
<infinisil>
Maybe even a *single* such integer for all of nur, increasing every time the bot does any change
<infinisil>
Oh and forgot about this: Old revisions would need to be kept around
<infinisil>
the git sha's I mean
<infinisil>
Can't ever delete them, because people in the future need to get older ones
<tilpner>
So there would constantly be new nur sequential-id:nur-master-sha pairs PR'd to nixpkgs?
<tilpner>
It might be easier to just have users look up the sha themselves, and provide a function to get a pinned NUR from that sha
<infinisil>
Nope, nixpkgs uses IFD with the master NUR, always fetch the latest version
<tilpner>
Yeah, I got that part. Not the one after it though
<infinisil>
but the latest version will contain something like [<shaversion1>, <shaversion2>, ...] for every repo
<infinisil>
On updates, this list gets extended on master
<infinisil>
But no sha's will ever get removed
<infinisil>
Users can then pin nixpkgs by specifying the version (0, 1, 2, ...) of the repo they want
<infinisil>
s/nixpkgs/nur
<infinisil>
tilpner: That doesn't seem to bad eh!
<tilpner>
Yes, sounds better than how I interpreted it
<tilpner>
But I worry about accumulating hashes forever
<infinisil>
Let's say we have 100 repos, each updating their repository daily, and we update NUR daily. This means 100 new git hashes every day. At about 50 chars = 50 bytes for every git hash, this means 5KB per day, 1.8MB per year
<infinisil>
And there might be more repos, and much less updates
<infinisil>
(probably a lot less updates)
<infinisil>
Although, that doesn't include the git object data size
<infinisil>
Oh and we need the git hashes and the sha256's for the content
<infinisil>
I'll look into it some more
<tilpner>
File size is just one aspect. After deciding on a possible file structure for such a hash colletion, I would like to generate 5 years of worst-case content and measure Nix parse time and memory usage
<tilpner>
A system evaluation takes enough memory already, let's not make it much worse
<infinisil>
Yeah
<infinisil>
This is the only way I see NUR working in nixpkgs though
<tilpner>
I do like your intention of making each repo pinnable individually (right?)
<infinisil>
Yeah, that would be neat
<tilpner>
But on the other hand, it could lead to a lot of effort when I have 20 pinned repos
<tilpner>
I would need to find the newest version for each of them, and update them one-by-one
<infinisil>
Either pinning all of NUr or only specific repos would be best
<infinisil>
tilpner: Oh how about this: Provide a *latest* version, and provide a set of "LTS" versions that will be pinnable
<infinisil>
Then we could e.g. only pin every 10th update or so, but always have the latest version unpinned
<tilpner>
No, don't like it
<infinisil>
Would solve the problem of too big of a nix evaluation
<tilpner>
If your previous suggestion is feasible, it could be applied to pinning of individual repos, and at the same time pinning of the entire thing
<tilpner>
Being able to pin to every version is greatly preferable to pinning to arbitrary intervals
<tilpner>
The nix evaluation problem could be solved with more IFD
<infinisil>
Hmm yeah
<tilpner>
...and segmentation into per-repo hash collections
<infinisil>
How about it also enabling you to pin repos at specific hashes yourself
<infinisil>
In addition to the simple integer lts ones
<tilpner>
Still sounds arbitrary. How would you choose those "lts" versions, what would support be (backporting?)
<tilpner>
But I need to sleep now. Let's discuss this some other time :)