<flokli>
From my experience, and as explained in the description, it caused more problems than it fixed IMHO
<niksnut>
so we went from a situation where we had to tool created for the purpose of managing patches (git), to having to do it by hand
<niksnut>
that seems objectively a step back
<flokli>
Mostly "Obsolete patches accidentially got re-introduced, maintaining things in the custom fork required write access to that repo, syncing back and forth with a corresponding nixpkgs PR, and the patches themselves became very unmanage-able as well. For example, some of the patches were partial reverts of earlier patches, not improving in getting the total patch count down"
<flokli>
by maintaining branches, you keep history of the upstream you're tracking, but loose history on how our distro-specific patches evolve over time
<niksnut>
I think people were doing it wrong then
<niksnut>
e.g. you can rebase on top of a new release and drop/squash commits
<flokli>
yeah, but it didn't work. And there were only few who could push to the fork at all, and a PR process there wouldn't work
<niksnut>
why not?
<flokli>
https://github.com/NixOS/nixpkgs/pull/94354 adds a short liner of documentation on how to handle updates - you can still handle all the rebasing inside git, from a systemd worktree
<flokli>
The patch-based workflow is not too bad, and was much more enjoyable for the recent bumps I did. All the rebasing can still happen inside git, and we have proper history on how our distro-specific patches evolve over time.
<niksnut>
it sounds like a lot more work
<niksnut>
for instance, if I want to hack on our systemd, in the old situation I would just do a git clone
<flokli>
now it's a git clone, and git am *.patch
<niksnut>
now I have to do a git clone of the upstream tree, manually apply the patches, and then figure out a way to get a diff for my own changes that doesn't include the previous patches
<niksnut>
well, if you're using git am, then it's basically a fork but we're not publishing it
<flokli>
it's a set of distro-specific patches, shipped next to the package derivation itself, which doesn't seem too bad
alp has quit [Ping timeout: 256 seconds]
alp has joined #nixos-dev
<worldofpeace>
niksnut: that's a valid point, it is more steps. But in almost every distro I've explored inside their package expressions, the one's that had the patches shipped with package expression itself were more clearly understood and accessible. And we don't need to have people to manage traffic to yet another repo, and anyone who wants to operate on systemd can do much easier.
<worldofpeace>
I'm not sure I can remember if there's a tool that makes this workflow a lot easier with handling a bunch patches
<hexa->
quilt *duck*
<hexa->
(ducking because I'm not sure if it makes it "easier" per se")
alp has quit [Ping timeout: 264 seconds]
alp has joined #nixos-dev
<nbp>
What about storing the fork of systemd as another root of nixpkgs repository?
<nbp>
That bad thing is that users of Nixpkgs will have to clone systemd even if they never make use of it … the good thing is that people ho have access to Nixpkgs would also have access to the systemd fork.
<niksnut>
the ultimate monorepo - include all 3rd party source trees :-)
<qyliss>
worldofpeace: quilt or git-series might be of interest
<qyliss>
Probably git-series more so, although I'm not currently too familiar with either of them.
<worldofpeace>
qyliss: yeah, I think edef mentioned git-series to me before
<worldofpeace>
I did try the latter and, idk, I'm not sure I understood it correctly 🤣
<worldofpeace>
haven't heard of quilt though
<Valodim>
ooh, lucky :)
<Valodim>
(quilt is how debian manages its patch stacks :)
<Mic92>
Can a monorepo include itself?
<worldofpeace>
Valodim: ahh, then it all makes sense
<Mic92>
flokli: how many patches could you upstream since you switch away from the fork?
<nbp>
Mic92: or do you mean, should we still call it a monorepo when it starts depending on itself?
<edef>
niksnut: i do that for a subset of 3rd party source trees
<Mic92>
I was also more comfortable with the git-based approach, but I have to agree that for one without access to NixOS/systemd its actually more complicated to use our fork over patches.
alp has quit [Remote host closed the connection]
<worldofpeace>
one of the first things I had a hard time getting my head wrapped around when I first needed to modify systemd was NixOS's fork
<worldofpeace>
like, I don't think "how to update systemd" was documented anywhere in the nixpkgs manual. So I had to figure out there's a fork and people with permissions to the fork... and the branches... ugh. It's more straightforward to a contributor point of view for patches to just be in nixpkgs and to just update the src etc. etc.
<Mic92>
37
<worldofpeace>
I'm actually thinking that it would be good documentation to have in the manual on how to update a series of patches in nixpkgs. 90% of the packages I maintain actually have a lot of patches and I notice sometimes contributors aren't sure how to regenerate the patch, or they might do it a different way
<edef>
git am / git format-patch
<edef>
that's all you actually need
<edef>
i *feel* like anyone who knows how to work with git should know how to use those two, but github is rather aggressively focused on preventing you from learning about them
<qyliss>
Yeah, so we should have documentation for them.
<worldofpeace>
edef: we already have some stuff that caters to people like this with git
<edef>
i do think we should have docs about it, i'm just annoyed we have to document it separately from the general patch submission workflow
<edef>
because otherwise it's just "the patch submission workflow, but with git add rather than git send-email at the end"
evanjs has quit [Read error: Connection reset by peer]
evanjs has joined #nixos-dev
evanjs has quit [Read error: Connection reset by peer]
evanjs has joined #nixos-dev
justanotheruser has joined #nixos-dev
AlwaysLivid has joined #nixos-dev
orivej has joined #nixos-dev
drakonis has joined #nixos-dev
orivej has quit [Remote host closed the connection]
orivej has joined #nixos-dev
orivej has quit [Quit: No Ping reply in 180 seconds.]
orivej has joined #nixos-dev
ericnoan has joined #nixos-dev
drakonis1 has joined #nixos-dev
orivej has quit [Remote host closed the connection]
orivej has joined #nixos-dev
orivej has quit [Remote host closed the connection]
orivej has joined #nixos-dev
<infinisil>
Profpatsch: Hey, I know you like having good and generic library functions. I was just looking into adding one for doing generic folds
<infinisil>
To nixpkgs that is
justanotheruser has quit [Ping timeout: 246 seconds]
<infinisil>
And I'm not entirely sure how it would be done ideally
<infinisil>
One idea is something like `genericFold :: (Attrs -> Bool) -> (Attrs of b -> b) -> (non-Attrs -> b) -> Attrs -> b`
orivej has quit [Ping timeout: 246 seconds]
<infinisil>
First argument is the condition under which to recurse into attrs, second argument is how to merge an attribute set of result types into a result type, third argument is how to produce a result from non-attrs
<infinisil>
I think this might be fine. The only problem is that it doesn't work for lists. And making it work for lists makes it a bit nasty
<Profpatsch>
Well, nested attribute sets, which is a different beast alltogether
<infinisil>
Yeah it would have to be recursive
<infinisil>
> genericFold = cond: onAttr: onElse: let go = value: if isAttrs value && cond value then onAttr (mapAttrs (name: go) value) else onElse value; in go;
<{^_^}>
error: syntax error, unexpected ';', at (string):121:180
<infinisil>
> genericFold = cond: onAttr: onElse: let go = value: if isAttrs value && cond value then onAttr (mapAttrs (name: go) value) else onElse value; in go
<{^_^}>
error: undefined variable 'isAttrs' at (string):121:65
<Profpatsch>
We already have some functions like that, right? They are not very principled though
<infinisil>
> genericFold = cond: onAttr: onElse: let go = value: if lib.isAttrs value && cond value then onAttr (lib.mapAttrs (name: go) value) else onElse value; in go
<{^_^}>
genericFold defined
<infinisil>
Profpatsch: Yeah they're kind of all over the place, and I think specifically it can't do the above transformation
<infinisil>
s/it/they
<infinisil>
That there is my current implementation attempt, trying to test it now
<Profpatsch>
Before you put it in the lib, there should be multiple use-cases though
<Profpatsch>
otherwise a local let is probably better
<infinisil>
I think a lot of functions could be rewritten with that
drakonis1 has quit [Quit: WeeChat 2.8]
<infinisil>
> genericFold (x: true) (attrs: foldl (a: b: a + b) 0 (attrValues attrs)) (value: 1) { x = { y = 10; z = 20; }; bar = 30; }
<{^_^}>
3
<infinisil>
Example, this counts how many non-attrset values there are
<Profpatsch>
Maybe? I would call it something more specific though since it’s only working on recursive attrsets
<infinisil>
Yeah
<infinisil>
Maybe `genericAttrsFold` or `foldAttrs` (but we have that already), or `genericFoldAttrs` (but then it's confusing with foldAttrs which does something pretty much unrelated)
<infinisil>
Ugh
<infinisil>
I'm just gonna start my own standard library lol
<infinisil>
Maybe this could even be a builtin hmmm
<infinisil>
Since many functions could be reimplemented using that
<Profpatsch>
Is there a related language that we could mirror?
<Profpatsch>
Maybe the Racket stdlib?
<Profpatsch>
Clojure is even closer
<infinisil>
Was half joking about the new standard library btw :P
<Profpatsch>
And pretty well-designed from what I remember
<Profpatsch>
“only half-joking” :)
drakonis1 has joined #nixos-dev
<infinisil>
With the advent of flakes, this might actually be a good idea
<infinisil>
Would allow us to cleanly reimplement lib out-of-tree
<Profpatsch>
I don’t think it’s tied to flakes particularly
<Profpatsch>
niv already provides that feature
<infinisil>
Can't use niv in nixpkgs itself though
<Profpatsch>
Or just fetchTarball
<Profpatsch>
nixpkgs probably shouldn’t switch to an out-of-band librarie
<Profpatsch>
*y
<infinisil>
I wish it would, eventually. Or rather, I wish nixpkgs would be split up into more parts generally
<Profpatsch>
I don’t.
drakonis has quit [Ping timeout: 265 seconds]
<infinisil>
As previously mentioned: lib, stdenv, pkgs, nixos
<infinisil>
Would be a pretty clean separation, and flakes could tie them all together without problems
<Profpatsch>
[citation required]
<Profpatsch>
argh, [citation needed], can’t even get my memes right :)
<infinisil>
Yeah I don't have any research supporting this, but I believe!
<infinisil>
Especially stdenv separation would be interesting, because it would probably obviate the need for staging branches
<infinisil>
Maybe not actually
<infinisil>
But at least people could do changes to stdenv more freely
FRidh has quit [Ping timeout: 256 seconds]
FRidh has joined #nixos-dev
cole-h has joined #nixos-dev
<infinisil>
> genericFold = cond: onAttr: onElse: let go = path: value: if isAttrs value && cond value then onAttr (mapAttrs (name: go (path ++ [name])) value) else onElse path value; in go [];
<{^_^}>
error: syntax error, unexpected ';', at (string):121:181
<infinisil>
> genericFold = cond: onAttr: onElse: let go = path: value: if isAttrs value && cond value then onAttr (mapAttrs (name: go (path ++ [name])) value) else onElse path value; in go []
<{^_^}>
error: undefined variable 'isAttrs' at (string):121:63
<infinisil>
> genericFold = cond: onAttr: onElse: let go = path: value: if lib.isAttrs value && cond value then onAttr (lib.mapAttrs (name: go (path ++ [name])) value) else onElse path value; in go []