worldofpeace_ changed the topic of #nixos-dev to: #nixos-dev NixOS Development (#nixos for questions) | NixOS stable: 20.03 ✨ https://discourse.nixos.org/t/nixos-20-03-release/6785 | https://hydra.nixos.org/jobset/nixos/trunk-combined https://channels.nix.gsc.io/graph.html | https://r13y.com | 19.09 RMs: disasm, sphalerite; 20.03: worldofpeace, disasm | https://logs.nix.samueldr.com/nixos-dev
rajivr has joined #nixos-dev
jonringer has quit [Remote host closed the connection]
ris has quit [Ping timeout: 246 seconds]
ris has joined #nixos-dev
orivej has joined #nixos-dev
orivej has quit [Ping timeout: 260 seconds]
orivej has joined #nixos-dev
orivej has quit [Quit: No Ping reply in 180 seconds.]
orivej has joined #nixos-dev
kalbasit has joined #nixos-dev
orivej has quit [Remote host closed the connection]
orivej has joined #nixos-dev
orivej has quit [Remote host closed the connection]
orivej has joined #nixos-dev
orivej has quit [Ping timeout: 256 seconds]
orivej has joined #nixos-dev
orivej_ has joined #nixos-dev
orivej has quit [Ping timeout: 240 seconds]
orivej_ has quit [Quit: No Ping reply in 180 seconds.]
orivej has joined #nixos-dev
drakonis has quit [Quit: WeeChat 2.8]
drakonis has joined #nixos-dev
kalbasit has quit [Ping timeout: 265 seconds]
orivej has quit [Ping timeout: 256 seconds]
<srhb> I feel like I'm seeing a higher number of irreproducible test failures on hydra than usual, or is it just me? Example: https://hydra.nixos.org/build/124713670 https://hydra.nixos.org/build/124713670/nixlog/4
FRidh has joined #nixos-dev
<Mic92> srhb: this was because of a systemd bug with newer kernels I think.
<Mic92> we reverted the last kernel and re-introduced it in staging together with a systemd upgrade
<Mic92> At least the first link
<srhb> Oh, must have failed to reproduce correctly then, my bad. :)
<srhb> Yup, I fatfingered a copy paste of a sha1. Derpderp :)
phreedom has quit [Ping timeout: 240 seconds]
alp has joined #nixos-dev
tilpner has joined #nixos-dev
orivej has joined #nixos-dev
drakonis has quit [Read error: Connection reset by peer]
orivej has quit [Ping timeout: 256 seconds]
orivej has joined #nixos-dev
orivej has quit [Ping timeout: 260 seconds]
alp has quit [Ping timeout: 246 seconds]
justanotheruser has quit [Ping timeout: 260 seconds]
FRidh has quit [Ping timeout: 240 seconds]
FRidh has joined #nixos-dev
orivej has joined #nixos-dev
orivej has quit [Ping timeout: 256 seconds]
orivej has joined #nixos-dev
<flokli> niksnut: domenkozar[m]: can we archive github.com/nixos/systemd?
orivej has quit [Ping timeout: 240 seconds]
<niksnut> aren't we using it anymore?
<niksnut> I don't understand why we went back to juggling a bunch of patches in nixpkgs
<flokli> niksnut: no, we went to juggling patches. Let me look for the long description
<flokli> look in the description of https://github.com/NixOS/nixpkgs/pull/85334, and the linked issue
<{^_^}> #85334 (by flokli, 16 weeks ago, merged): systemd: 243.7 -> 245
<niksnut> *went back
alp has joined #nixos-dev
<niksnut> that's what we used to do before we had our own systemd fork
<niksnut> the systemd fork was introduced expressly to fix that situation
<niksnut> I don't understand the motivation given in #85334
<{^_^}> https://github.com/NixOS/nixpkgs/pull/85334 (by flokli, 16 weeks ago, merged): systemd: 243.7 -> 245
<flokli> From my experience, and as explained in the description, it caused more problems than it fixed IMHO
<niksnut> so we went from a situation where we had to tool created for the purpose of managing patches (git), to having to do it by hand
<niksnut> that seems objectively a step back
<flokli> Mostly "Obsolete patches accidentially got re-introduced, maintaining things in the custom fork required write access to that repo, syncing back and forth with a corresponding nixpkgs PR, and the patches themselves became very unmanage-able as well. For example, some of the patches were partial reverts of earlier patches, not improving in getting the total patch count down"
<flokli> by maintaining branches, you keep history of the upstream you're tracking, but loose history on how our distro-specific patches evolve over time
<niksnut> I think people were doing it wrong then
<niksnut> e.g. you can rebase on top of a new release and drop/squash commits
<flokli> yeah, but it didn't work. And there were only few who could push to the fork at all, and a PR process there wouldn't work
<niksnut> why not?
<flokli> https://github.com/NixOS/nixpkgs/pull/94354 adds a short liner of documentation on how to handle updates - you can still handle all the rebasing inside git, from a systemd worktree
<{^_^}> #94354 (by flokli, 5 days ago, open): systemd: 245.6 -> 246
<flokli> it's just that we "persist" these patches next to the systemd derivation, instead of our fork
<NinjaTrappeur> You need to rebase the existing distro-specific commits for a systemd bump, not only introduce new ones.
<NinjaTrappeur> (regarding the PR-based flow)
<niksnut> ideally we would be handling all patches of git-based projects in this way
<niksnut> because that's proper version management
<flokli> ideally, we get the count of distro-specific patches down over time
<flokli> and avoid having to manage too many of these alltogether
<niksnut> but that's not happening, I still see 18 patches on systemd
<flokli> I sat down with upstream and spoke about each and every one of them
<flokli> https://github.com/NixOS/nixpkgs/issues/80038 tracks this, and most of them can be upstreamed in one way or another
<{^_^}> #80038 (by flokli, 24 weeks ago, open): systemd: improve our downstream patch situation
<flokli> The patch-based workflow is not too bad, and was much more enjoyable for the recent bumps I did. All the rebasing can still happen inside git, and we have proper history on how our distro-specific patches evolve over time.
<niksnut> it sounds like a lot more work
<niksnut> for instance, if I want to hack on our systemd, in the old situation I would just do a git clone
<flokli> now it's a git clone, and git am *.patch
<niksnut> now I have to do a git clone of the upstream tree, manually apply the patches, and then figure out a way to get a diff for my own changes that doesn't include the previous patches
<niksnut> well, if you're using git am, then it's basically a fork but we're not publishing it
<flokli> it's a set of distro-specific patches, shipped next to the package derivation itself, which doesn't seem too bad
alp has quit [Ping timeout: 256 seconds]
alp has joined #nixos-dev
<worldofpeace> niksnut: that's a valid point, it is more steps. But in almost every distro I've explored inside their package expressions, the one's that had the patches shipped with package expression itself were more clearly understood and accessible. And we don't need to have people to manage traffic to yet another repo, and anyone who wants to operate on systemd can do much easier.
<worldofpeace> I'm not sure I can remember if there's a tool that makes this workflow a lot easier with handling a bunch patches
<hexa-> quilt *duck*
<hexa-> (ducking because I'm not sure if it makes it "easier" per se")
alp has quit [Ping timeout: 264 seconds]
alp has joined #nixos-dev
<nbp> What about storing the fork of systemd as another root of nixpkgs repository?
<nbp> That bad thing is that users of Nixpkgs will have to clone systemd even if they never make use of it … the good thing is that people ho have access to Nixpkgs would also have access to the systemd fork.
<niksnut> the ultimate monorepo - include all 3rd party source trees :-)
<qyliss> worldofpeace: quilt or git-series might be of interest
<qyliss> Probably git-series more so, although I'm not currently too familiar with either of them.
<worldofpeace> qyliss: yeah, I think edef mentioned git-series to me before
<worldofpeace> I did try the latter and, idk, I'm not sure I understood it correctly 🤣
<worldofpeace> haven't heard of quilt though
<Valodim> ooh, lucky :)
<Valodim> (quilt is how debian manages its patch stacks :)
<Mic92> Can a monorepo include itself?
<worldofpeace> Valodim: ahh, then it all makes sense
<Mic92> flokli: how many patches could you upstream since you switch away from the fork?
<worldofpeace> quilt seems kinda just fine
worldofpeace_ has joined #nixos-dev
<flokli> Mic92: let's carry the discussion over to https://github.com/NixOS/nixpkgs/issues/80038 - and I really need to add my notes...
<{^_^}> #80038 (by flokli, 24 weeks ago, open): systemd: improve our downstream patch situation
<nbp> Mic92: "Can a monorepo include itself?" yes
<nbp> Mic92: or do you mean, should we still call it a monorepo when it starts depending on itself?
<edef> niksnut: i do that for a subset of 3rd party source trees
<Mic92> I was also more comfortable with the git-based approach, but I have to agree that for one without access to NixOS/systemd its actually more complicated to use our fork over patches.
alp has quit [Remote host closed the connection]
<worldofpeace> one of the first things I had a hard time getting my head wrapped around when I first needed to modify systemd was NixOS's fork
<worldofpeace> like, I don't think "how to update systemd" was documented anywhere in the nixpkgs manual. So I had to figure out there's a fork and people with permissions to the fork... and the branches... ugh. It's more straightforward to a contributor point of view for patches to just be in nixpkgs and to just update the src etc. etc.
<Mic92> 37
<worldofpeace> I'm actually thinking that it would be good documentation to have in the manual on how to update a series of patches in nixpkgs. 90% of the packages I maintain actually have a lot of patches and I notice sometimes contributors aren't sure how to regenerate the patch, or they might do it a different way
<edef> git am / git format-patch
<edef> that's all you actually need
<edef> i *feel* like anyone who knows how to work with git should know how to use those two, but github is rather aggressively focused on preventing you from learning about them
<qyliss> Yeah, so we should have documentation for them.
<worldofpeace> edef: we already have some stuff that caters to people like this with git
<edef> i do think we should have docs about it, i'm just annoyed we have to document it separately from the general patch submission workflow
<edef> because otherwise it's just "the patch submission workflow, but with git add rather than git send-email at the end"
worldofpeace_ has quit [Quit: worldofpeace_]
<worldofpeace> edef: ahh, true
evanjs has quit [Read error: Connection reset by peer]
evanjs has joined #nixos-dev
evanjs has quit [Read error: Connection reset by peer]
evanjs has joined #nixos-dev
justanotheruser has joined #nixos-dev
AlwaysLivid has joined #nixos-dev
orivej has joined #nixos-dev
drakonis has joined #nixos-dev
orivej has quit [Remote host closed the connection]
orivej has joined #nixos-dev
orivej has quit [Quit: No Ping reply in 180 seconds.]
orivej has joined #nixos-dev
ericnoan has joined #nixos-dev
drakonis1 has joined #nixos-dev
orivej has quit [Remote host closed the connection]
orivej has joined #nixos-dev
orivej has quit [Remote host closed the connection]
orivej has joined #nixos-dev
<infinisil> Profpatsch: Hey, I know you like having good and generic library functions. I was just looking into adding one for doing generic folds
<infinisil> To nixpkgs that is
justanotheruser has quit [Ping timeout: 246 seconds]
<infinisil> And I'm not entirely sure how it would be done ideally
<infinisil> One idea is something like `genericFold :: (Attrs -> Bool) -> (Attrs of b -> b) -> (non-Attrs -> b) -> Attrs -> b`
orivej has quit [Ping timeout: 246 seconds]
<infinisil> First argument is the condition under which to recurse into attrs, second argument is how to merge an attribute set of result types into a result type, third argument is how to produce a result from non-attrs
<infinisil> I think this might be fine. The only problem is that it doesn't work for lists. And making it work for lists makes it a bit nasty
<Profpatsch> infinisil: what’s a generic fold?
<infinisil> It's related yeah
<Profpatsch> Specialized to attrsets?
<Profpatsch> And //?
<infinisil> Profpatsch: Specialized to attrsets maybe. But not for //
<Profpatsch> Rust has a way of giving an Item type for Iterators
<infinisil> It's not really iterating
<Profpatsch> And for a HashMap that’s (key, val)
<infinisil> Like, catamorphism i think
<infinisil> Lemme write an example
<Profpatsch> It really depends on the use-case, yeah
<infinisil> Motivation comes from bqv wanting, I quote: "{ a = { b = 1; }; c = 2; } to something like { "a/b" = 1; c = 2; }"
<infinisil> In this case, it would be `b ~ Attrs`
<Profpatsch> Since nix has a very simple type like NixType = Attrs | List | Int | …
<Profpatsch> I don’t think it makes much sense to have a generic fold function, when there’s only two ways to create structure
<infinisil> I guess it's not a generic fold, but attribute specific
<Profpatsch> yeah, sounds like you want foldl but for attributes
<Profpatsch> Well, nested attribute sets, which is a different beast alltogether
<infinisil> Yeah it would have to be recursive
<infinisil> > genericFold = cond: onAttr: onElse: let go = value: if isAttrs value && cond value then onAttr (mapAttrs (name: go) value) else onElse value; in go;
<{^_^}> error: syntax error, unexpected ';', at (string):121:180
<infinisil> > genericFold = cond: onAttr: onElse: let go = value: if isAttrs value && cond value then onAttr (mapAttrs (name: go) value) else onElse value; in go
<{^_^}> error: undefined variable 'isAttrs' at (string):121:65
<Profpatsch> We already have some functions like that, right? They are not very principled though
<infinisil> > genericFold = cond: onAttr: onElse: let go = value: if lib.isAttrs value && cond value then onAttr (lib.mapAttrs (name: go) value) else onElse value; in go
<{^_^}> genericFold defined
<infinisil> Profpatsch: Yeah they're kind of all over the place, and I think specifically it can't do the above transformation
<infinisil> s/it/they
<infinisil> That there is my current implementation attempt, trying to test it now
<Profpatsch> Before you put it in the lib, there should be multiple use-cases though
<Profpatsch> otherwise a local let is probably better
<infinisil> I think a lot of functions could be rewritten with that
drakonis1 has quit [Quit: WeeChat 2.8]
<infinisil> > genericFold (x: true) (attrs: foldl (a: b: a + b) 0 (attrValues attrs)) (value: 1) { x = { y = 10; z = 20; }; bar = 30; }
<{^_^}> 3
<infinisil> Example, this counts how many non-attrset values there are
<Profpatsch> Maybe? I would call it something more specific though since it’s only working on recursive attrsets
<infinisil> Yeah
<infinisil> Maybe `genericAttrsFold` or `foldAttrs` (but we have that already), or `genericFoldAttrs` (but then it's confusing with foldAttrs which does something pretty much unrelated)
<infinisil> Ugh
<infinisil> I'm just gonna start my own standard library lol
<infinisil> Maybe this could even be a builtin hmmm
<infinisil> Since many functions could be reimplemented using that
<Profpatsch> Is there a related language that we could mirror?
<Profpatsch> Maybe the Racket stdlib?
<Profpatsch> Clojure is even closer
<infinisil> Was half joking about the new standard library btw :P
<Profpatsch> And pretty well-designed from what I remember
<Profpatsch> “only half-joking” :)
drakonis1 has joined #nixos-dev
<infinisil> With the advent of flakes, this might actually be a good idea
<infinisil> Would allow us to cleanly reimplement lib out-of-tree
<Profpatsch> I don’t think it’s tied to flakes particularly
<Profpatsch> niv already provides that feature
<infinisil> Can't use niv in nixpkgs itself though
<Profpatsch> Or just fetchTarball
<Profpatsch> nixpkgs probably shouldn’t switch to an out-of-band librarie
<Profpatsch> *y
<infinisil> I wish it would, eventually. Or rather, I wish nixpkgs would be split up into more parts generally
<Profpatsch> I don’t.
drakonis has quit [Ping timeout: 265 seconds]
<infinisil> As previously mentioned: lib, stdenv, pkgs, nixos
<infinisil> Would be a pretty clean separation, and flakes could tie them all together without problems
<Profpatsch> [citation required]
<Profpatsch> argh, [citation needed], can’t even get my memes right :)
<infinisil> Yeah I don't have any research supporting this, but I believe!
<infinisil> Especially stdenv separation would be interesting, because it would probably obviate the need for staging branches
<infinisil> Maybe not actually
<infinisil> But at least people could do changes to stdenv more freely
FRidh has quit [Ping timeout: 256 seconds]
FRidh has joined #nixos-dev
cole-h has joined #nixos-dev
<infinisil> > genericFold = cond: onAttr: onElse: let go = path: value: if isAttrs value && cond value then onAttr (mapAttrs (name: go (path ++ [name])) value) else onElse path value; in go [];
<{^_^}> error: syntax error, unexpected ';', at (string):121:181
<infinisil> > genericFold = cond: onAttr: onElse: let go = path: value: if isAttrs value && cond value then onAttr (mapAttrs (name: go (path ++ [name])) value) else onElse path value; in go []
<{^_^}> error: undefined variable 'isAttrs' at (string):121:63
<infinisil> > genericFold = cond: onAttr: onElse: let go = path: value: if lib.isAttrs value && cond value then onAttr (lib.mapAttrs (name: go (path ++ [name])) value) else onElse path value; in go []
<{^_^}> genericFold defined
<infinisil> > doit = attrs: listToAttrs (genericFold (x: true) (attrs: concatLists (attrValues attrs)) (path: value: [ (nameValuePair (concatStringsSep "/" path) value) ])
<{^_^}> error: syntax error, unexpected ';', expecting ')', at (string):84:159
<infinisil> > doit = attrs: listToAttrs (genericFold (x: true) (attrs: concatLists (attrValues attrs)) (path: value: [ (nameValuePair (concatStringsSep "/" path) value) ]) attrs)
<{^_^}> error: undefined variable 'listToAttrs' at (string):84:16
<infinisil> > doit = attrs: lib.listToAttrs (genericFold (x: true) (attrs: lib.concatLists (lib.attrValues attrs)) (path: value: [ (lib.nameValuePair (lib.concatStringsSep "/" path) value) ]) attrs)
<{^_^}> doit defined
<infinisil> > doit { x = { y = "Y"; z = "Z"; b = { c = "C"; d = "D"; }; }; a = "A"; }
<{^_^}> { a = "A"; "x/b/c" = "C"; "x/b/d" = "D"; "x/y" = "Y"; "x/z" = "Z"; }
<cole-h> 👀
<infinisil> Profpatsch: ^ version of genericFold that also exposes the attribute path
AlwaysLivid has quit [Ping timeout: 246 seconds]
AlwaysLivid has joined #nixos-dev
justanotheruser has joined #nixos-dev
alp has joined #nixos-dev
AlwaysLivid has quit [Ping timeout: 256 seconds]
AlwaysLivid has joined #nixos-dev
drakonis1 has quit [Ping timeout: 264 seconds]
drakonis has joined #nixos-dev
alp has quit [Ping timeout: 264 seconds]
alp has joined #nixos-dev
FRidh has quit [Quit: Konversation terminated!]
alp has quit [Ping timeout: 240 seconds]
alp has joined #nixos-dev
alp has quit [Client Quit]
rajivr has quit [Quit: Connection closed for inactivity]
<matthewbauer> Does anyone here know how to write zsh completions? I was trying to convert https://github.com/NixOS/nix/blob/master/misc/bash/completion.sh to zsh, but realized I have no idea what i'm doing :)
<srhb> matthewbauer: In case you didn't know, one already exists
<matthewbauer> srhb: i don't think it supports the new completion system
<srhb> Maybe it suffices, or can serve as inspiration :)
<srhb> Ma27 has an open PR that supports 3.0 partially, not sure if it uses that.
<matthewbauer> srhb: i guess the issue is it's not dynamic - you have to hardcode all of your commands ahead of time
<srhb> I didn't check the code, you may very well be right :)
<matthewbauer> i'd like to make it autogenerate like how the bash completions , but that turns out to be much harder in zsh than bash apparentl
* srhb nods
<matthewbauer> i guess https://github.com/spwhitt/nix-zsh-completions/pull/34 would be enough though
<{^_^}> spwhitt/nix-zsh-completions#34 (by Ma27, 2 weeks ago, open): [WIP] Nix 3.0 (+flakes) support
<ma27[m]> what do you mean with autogenerate? This completionbasically parses the output of `nix --help`
<ma27[m]> ah I see :)
<matthewbauer> yeah if you do `NIX_GET_COMPLETIONS=1 nix add` you can see what the output looks like
Jackneilll has joined #nixos-dev
Jackneill has quit [Ping timeout: 260 seconds]
AlwaysLivid has quit [Read error: Connection reset by peer]
AlwaysLivid has joined #nixos-dev
AlwaysLivid has quit [Remote host closed the connection]
kalbasit has joined #nixos-dev
kalbasit has quit [Ping timeout: 264 seconds]
drakonis has quit [Quit: WeeChat 2.8]
drakonis has joined #nixos-dev
drakonis has quit [Quit: WeeChat 2.8]