<{^_^}>
#75584 (by Infinisil, 7 weeks ago, open): Configuration file formats for JSON, INI, YAML and TOML
Jackneill has quit [Ping timeout: 240 seconds]
Jackneill has joined #nixos-dev
<niksnut>
worldofpeace: thanks
<worldofpeace>
niksnut: totally, thank you too 👍️
Jackneill has quit [Ping timeout: 252 seconds]
tilpner has joined #nixos-dev
<infinisil>
Profpatsch: Will do, not ready yet, needs docs and tests
<infinisil>
But I should have time to get to it now
Jackneill has joined #nixos-dev
FRidh has joined #nixos-dev
psyanticy has joined #nixos-dev
<worldofpeace>
andi-: the thing is, what they mention there about things not being verified before being merged to master is not nixpkgs future
<gchristensen>
😻 worldofpeace 😻 good GRIEF you put exactly what I was thinking in words better than I could have.
<andi->
worldofpeace: yeah, I agree. Never intended to support any other practice :)
<andi->
It is just that I've seen it and it will happen again ;) To some degree all those people that just cherry-pick to stable are doing that..
<andi->
(without creating a PR, building it, running tests, …)
<worldofpeace>
gchristensen: I'm just seeing NixOS had a past of basically not good practices
<gchristensen>
you can't know for certain that they aren't building and running tests, just that they aren't doing a PR. I would of course prefer a PR.
<worldofpeace>
nixpkgs is not some rinky dink packages archive where anyone can dump their stuff
<andi->
gchristensen: given the time delta between "merged to master" and "backported" I have a pretty good feeling in many cases.
<gchristensen>
nixos' past definitely was a product of the time, and the size of the community :P
<gchristensen>
andi-: good point
<worldofpeace>
we have people to support
<gchristensen>
+1
<worldofpeace>
so it's more we really can't do any of that anymore
<gchristensen>
we need to be striving to be better pretty much 100% of the time
<gchristensen>
y'all are really wonderful, and I'm glad we're in community together
<worldofpeace>
I really really want people to take NixOS seriously, and I don't think there should or have to be an RFC about this thing. I don't want to tell someone "well this is the future", but I don't see any real way of stopping things if that's what they want.
<gchristensen>
there is no need for an RFC, because this has been the policy forever
<gchristensen>
maybe there could be an RFC to solidify it, but not for this merge
<worldofpeace>
I meant the core fundimental disagreement that I identified gchristensen
<gchristensen>
oh maybe I misunderstood
<worldofpeace>
but I'm also worried, I'm not sure how this community is/could handle such disagreements.
kreisys has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
justan0theruser has quit [Ping timeout: 265 seconds]
<gchristensen>
worldofpeace: for context, we have dealt with disagreements with oxij several times in the past, and this won't be the last one :P btw, don't feel you need to reply to messages like this. I can surely write something -- I usually do :). or, we could write one together. they can be tiring
<worldofpeace>
gchristensen: yesssssss, I still haven't replied to the SLIM stuff
* elvishjerricco
is out of the loop
<elvishjerricco>
tl;dr?
<gchristensen>
oxij had some pretty sharp words which in general, a lot of contributors disagree are true currently, and more importantly disagree that it isn't NixOS's future
<tilpner>
Perhaps if nixpkgs policies were collected in a single place, so that they can be easily referenced, would help here
<gchristensen>
it would
<tilpner>
Maybe they are in the nixpkgs manual, and I just don't know it
<tilpner>
But it's often unclear what the policy on anything is, which is expected of a not-really-centrally-coordinated project
<das_j>
hmm, I have a package (kimai) which expects a config file in its root. If I use kimai.overrideDerivation and place the config file there, does this mean that two kimais with different configs can use the same base kimai package? or is the base package always rebuilt?
<worldofpeace>
tilpner: I've had that discussion in length publicly
<tilpner>
worldofpeace: I realise it's much easier said to "just write down the policies", than actually doing and taking into account all the edge cases :)
<gchristensen>
way way harder
<worldofpeace>
tilpner: totally, I realized it would be easier to have a discussion thread than actually do anything productive just yet 😜
<tilpner>
Speaking of uncertainty about policies: #79262 #79265
<tilpner>
I really would like to be able to label my PRs sometimes, like this I have to worry about finding the right people to poke instead of them finding them by label
Cale has quit [Ping timeout: 245 seconds]
<tilpner>
I was unsure whether this should have a backport, since it's said to not be exploitable in the current 19.09 version, but I'm too scared to leave my system security to a unrelated change in EOF handling
<tilpner>
Do I ping people to get this merged quickly, do I mention the PR a bunch, or do I wait for someone to find it?
<gchristensen>
great questions, tilpner
<tilpner>
I know it seems impatient, the PR has only been open for an hour (but I wonder if patience is appropriate for a potential security fix)
<tilpner>
So, uhh, that worked
<Taneb>
Sometimes impatience is a virtue
<tilpner>
<3 gchristensen (And yes, I am using pwfeedback because my keyboard is terrible)
<infinisil>
I'm afraid that will be very bad for participation
<mkaito>
I actually like that too
<__monty__>
infinisil: Not really cause you can't participate through either.
<infinisil>
That interface doesn't make people want to contribute
<infinisil>
Other than people who have used it already
<mkaito>
if you're looking to attract people, github is literally the only game in town
<infinisil>
__monty__: Oh and hyperkitty has a reply button that opens a mailto:
<mkaito>
so archaic :P
<__monty__>
I disagree. The hyperkitty interface looks like google-groups which instantly screams to me "this is a terrible, useless UI."
<__monty__>
I've never participated in mailing lists and I still prefer the plaintext threaded archive.
<mkaito>
if we're going to be like that, I prefer mutt.
<__monty__>
At least you can find and read discussions in that without more than half your screen being covered in useless menus you never use.
<drakonis>
hyperkitty is baarely usable
<__monty__>
Maybe we should move this discussion to -chat? I have serious doubts about the fancier version's accessibility for people who actually need accessibility.
<infinisil>
Seems relevant for -dev, because it would be nice to switch to something better for RFC discussions
<mkaito>
I'm incredibly jaded about this kind of thing, hence my usual "kids don't like it if it doesn't have emoji" comment. Form over function is more prevalent the younger your target audience.
<mkaito>
and I'm not even 40, why do I talk like I'm 70
<__monty__>
If you want a fancy interface combined with a mailing list I think forum software that semi-supports email interaction is the way to go. Even though it has its own shortcomings. At least it'll actually stimulate participation from people who wouldn't bother with a mailing list.
<__monty__>
mkaito: The comment doesn't even make much sense to me because unicode has tons of emoji : p
<__monty__>
Or rather 👅
<infinisil>
How about discourse?
<infinisil>
I don't see a reason why this couldn't be used for rfc discussions too
<drakonis>
loomio looks like a discourse fork
<infinisil>
Though not sure if discourse is any better than github
<mkaito>
I haven't used discourse a lot, but when I have, I hated every moment.
<mkaito>
but then, I'm on IRC, I'm not your target audience here :P
<__monty__>
mkaito: You're not confusing discourse with discord or disqus, are you? Both are common confusions afais.
<gchristensen>
honestly a collaborative document editor might be the thing, similar to a google-doc
<mkaito>
__monty__: no
<__monty__>
Those are *really* hard for discussion imo.
<drakonis>
run your own editor instance instead of relying on someone else's
<__monty__>
I don't think github's bad for the RFC editing part? It's only bad for the discussion part.
<mkaito>
maybe this is an unpopular opinion, but while I see the appeal of discourse and github to attract people into the project, maybe things like RFC could live on a simple mailing list. Much bigger projects run like that, and they do just fine.
<infinisil>
__monty__: Yeah mostly discussion only
<infinisil>
Maybe we should look around what other projects use for RFC-like discussions
<infinisil>
And ask them whether it works for them or what the problems are
<__monty__>
I have heard about epic google-docs synthesized by a huge ungoverned community. I've just never experienced them making collaboration pleasant and clear.
<gchristensen>
how does the rust community handle this?
<__monty__>
The discussion living right next to the PR is pretty convenient imo. How many people will see an RFC and then dig through email archives to find discussion?
<niksnut>
I think in rust rfcs they're more proactive in hiding outdated comments
<niksnut>
to unclutter the discussion
<infinisil>
+1 to that
<infinisil>
I wish RFC authors could do that in GitHub
<__monty__>
They could if those comments were in "reviews" I guess.
<gchristensen>
niksnut: hmm good to know! yes, that would be good
orivej has joined #nixos-dev
<qyliss>
__monty__: you can participate through hyperkitty
<qyliss>
Rust RFCs are even harder to follow than ours
<qyliss>
__monty__: the RFC would presumably be posted to the mailing list, not as a PR
<mkaito>
the usual code review workflow over email is to post a thread with a patchset, discuss that patch set, and new revisions of the same patch set are posted as new email threads, maybe with a link to the archive of the previous discussion. this would probably also apply to an RFC discussion workflow to some extent.
ixxie has quit [Ping timeout: 260 seconds]
_ris has joined #nixos-dev
phreedom has quit [Remote host closed the connection]
phreedom has joined #nixos-dev
<__monty__>
qyliss: A reply-to button isn't participation.
<__monty__>
The RFC being in a git repo is a good thing imo.
<qyliss>
they'd still end up in the git repo
<qyliss>
Am I wrong in thinking that you can post to Hyperkitty mailing lists from the web UI if you're signed in?
<__monty__>
Fwiw, I think haskell standards discussion moved from a mailing list *to* the Rust RFC repo model to make it *less* closed-doors.
<qyliss>
well then I hope they enjoy not being able to follow their discussions
<qyliss>
(I just checked and indeed you can participate entirely from Hyperkitty without ever opening a mail client)
<__monty__>
I follow a couple NixOS RFCs and github's good at notifying about new comments.
<__monty__>
The only thing it's bad at is reading the entire thread.
<qyliss>
But not where in the discussion those comments are
<__monty__>
ime
<qyliss>
So if somebody comments "I agree with this", what are they talking about? Who knows?
ixxie has joined #nixos-dev
<__monty__>
That problem would happen with any non-threaded discussion platform *including* mailing lists.
<qyliss>
Mailing lists are threaded
<qyliss>
there are no "non-threaded" mailing lists because that isn't how mail works
<__monty__>
Mailing lists are as threaded as a github PR "comments" thread.
<qyliss>
What? No they're not.
<__monty__>
It's one giant thread. That's not a *threaded* discussion.
<mkaito>
what? no
<mkaito>
that's not how email works
<qyliss>
Have you ever used a mailing list?
<__monty__>
Email threads that split the discussion are always worse imo.
<andi->
I like(d) those. Alot better to observe the conversation. You can also reply to some specific comment that isn't the latest anymore and anyone wiht a reasonable mail client is able to see the relationship (besides proper quoting being used)
<__monty__>
I've never liked it in mailing list archives.
<__monty__>
Old reddit's threading is first-in-class imo.
<andi->
I hate old and new reddit *shrug*
<andi->
I guess we can just say everyone has another taste :)
<gchristensen>
+1
<__monty__>
We're not talking about reddit though. Just it's display of threaded discussion.
<andi->
__monty__: yeah, the clicking there drives me mad. I barely ever used it but I don't like their model of interleaving messages with structure of the conversation. That is what makes me stick with neomutt so much. I have a view of the structure and then I can opt-in to the content.
<aanderse>
so a couple days now i notice i run "make" in nixpkgs/nixos/doc/manual and i get an error
<__monty__>
but old reddit didn't have "see more" just "comment below threshold."
<aanderse>
is it just me?
<infinisil>
aanderse: unclean git tree?
<infinisil>
I think I've had trouble in that case before
<infinisil>
git clean -f -x -d cleans it up, removing all untracked files
<infinisil>
(replace -f with -n to see what would be removed)
<aanderse>
infinisil: ok, running a hard reset onto upstream/master then running that command
<aanderse>
will see what happens
ixxie has quit [Ping timeout: 265 seconds]
<aanderse>
infinisil++
<{^_^}>
infinisil's karma got increased to 206
<infinisil>
Seems to have worked :D
<aanderse>
mhm, thank you
FRidh has quit [Remote host closed the connection]
joko has quit [Quit: WeeChat 2.6]
<gchristensen>
I've taken to writing some personal docs whenever I learn something that took more than a few minutes to figure out
<FRidh>
Could someone check the merge commit 0be87c79797a5fa384fbc356c74ed54f9f7829ea on master and the two commits leading to it. I think its corrected now, just want to be sure. Note I am available now for some time but I'll look at the IRC log :) I recall the branches got mixed up once before
<mkaito>
IRC log tl;dr: muh mailing lists, github not so good for long discussions :P
<FRidh>
ok, I think it is not corrected now, as it seems it reverted master instead of staging-next. sigh. Could you have a look aszlig ?
drakonis has quit [Quit: WeeChat 2.6]
<FRidh>
it's resolved now
ixxie has quit [Ping timeout: 260 seconds]
<gchristensen>
FRidh++ thank you
<{^_^}>
FRidh's karma got increased to 16
<aanderse>
if i have a zip file, which contains a zip file, which contains a zip file which has the "source" for a derivation... is there any way to extract that which is less horrible than the obvious unpackPhase override?
<clever>
aanderse: plan b: complain to the author, loudly
<aanderse>
clever: larry ellison does not hear my complaints about his company :(
<samueldr>
I believe in nature, those are called "warning signs"
<aanderse>
ha ha ha
<aanderse>
i'll go with the horrible unpackPhase override then
janneke has quit [Quit: janneke quits Mes'sing]
<infinisil>
For the future I'd like derivation sources to be self-contained
janneke has joined #nixos-dev
<infinisil>
Instead of having to define unpackPhase within the derivation that wants to use the source, this should be attached to the fetcher derivation directly
janneke has quit [Read error: Connection reset by peer]
janneke_ has joined #nixos-dev
<infinisil>
An idea for the interface: Every source derivation should have $out/bin/create-source, which when called will produce the unpacked source in cwd
<qyliss>
Oh yes this would be great
<qyliss>
I've thought about just using an output
janneke_ is now known as janneke
<qyliss>
But the important thing is just that there's literally any way to do it
<qyliss>
Because it's really helpful for AGPL compliance
<qyliss>
e.g. on https://spectrum-os.org/lists/archives/ I have to provide the link in the bottom to the source code, and doing that right now with Nix is pretty annoying.
<infinisil>
Problem with putting the source in a derivation is that it would produce unpacked derivations, which could be rather big and more than double the necessary space in the store for the source
<infinisil>
This is one of the reasons why I think such a binary would be better
<qyliss>
I think edef has been maybe doing some work or at least thinking here too
<qyliss>
But yeah, that makes sense
<qyliss>
I'd like a helper function that would create the source and then tar it back up again, in that case
<gchristensen>
infinisil: I don't think fixed output derivations can have a closure
<gchristensen>
and I think it is preferrable to have hashes that presumably match what upstream would provide (even if we put it in to a different base)
<infinisil>
gchristensen: It wouldn't be a fixed-output derivation
<qyliss>
There's nothing stopping you attaching a derivation with a script to the FOD in passthru, though, that depends on the source one and unpacks it.
<gchristensen>
ah
<infinisil>
It would depend on the fixed-output derivation doing the fetching
<qyliss>
oh yes please no more FODs
<gchristensen>
so it'd just be a decompressor then
<gchristensen>
not a thing doing the fetcher
<qyliss>
It should apply the patches too IMO
<infinisil>
Kind of, but the important part is that `fetchurl` and such should do the whole thing in one swoop
<gchristensen>
I think you're both talkin about different things, then?
<qyliss>
the whole patchPhase
<qyliss>
we might be :/
<infinisil>
Oh yeah patches too good point
<qyliss>
Patches are the important thing for my use case
<gchristensen>
qyliss is talking about provenance I think, and infinisil is talking about simplifying unpack phase
<qyliss>
I think both are complimentary
<gchristensen>
interesting
<infinisil>
Okay maybe not fetchurl for backwards compat, but my idea is to be able to do `${fetchUrl' { url = "..."; sha256 = "..."; unpack = "tar.gz"; }}/bin/create-source`
<infinisil>
And anything that represents a source should adhere to such an interface
<infinisil>
multiple sources, local sources, patched sources
<qyliss>
I like it
<gchristensen>
seems cool to not need decompression tools in the build closure
<qyliss>
the create-source script would have them in its closure though, right?
<infinisil>
Multiple sources could just be `${combineSources { subdir1 = fetchUrl' { ... }; subdir2 = fetchUrl' { ... }; }}/bin/create-source`
<qyliss>
and that would be run in the build closure
<qyliss>
so they'd still end up there unless i'm mistaken
<infinisil>
qyliss: Yeah
<gchristensen>
mmm depends, but I was thinking they'd not be in $PATH and instead explicitly referred to in the unpacker
<arcnmx>
> It should apply the patches too IMO
<{^_^}>
undefined variable 'It' at (string):277:1
<gchristensen>
but yeah still in the build closure
<infinisil>
Ah yeah
<arcnmx>
yeah, the way that patches are applied as part of the derivation's build process has always been a weird thing to me
<gchristensen>
ehh some patches depend on $out. some complicated stuff happens at patch time
<gchristensen>
and this would nerf SOURCE_DATE_EPOCH discovery, since the patched files would all be 1 second past epoch
<qyliss>
tbh I think stuff that depends on $out might be better suited for configurePhase even if it is modifying the sources
<arcnmx>
though I guess the "applied" line becomes hazy if you're talking about a script as part of $src that does indeep apply it, but at least having them bundled as inputs to the src seems like a good start
<gchristensen>
interesting
<qyliss>
since it's really doing the job of a configure tsep
<qyliss>
step
<infinisil>
gchristensen: qyliss: Nothing prevents the create-source script from using $out
<infinisil>
It gets run within the builder after all
<qyliss>
oh that's true
<gchristensen>
heh nice
<qyliss>
although I think it would not be ideal
<infinisil>
And SOURCE_DATE_EPOCH could be adjusted too with it
<qyliss>
Because then the source would have to know about the outputs of the derivation using it
<qyliss>
Which doesn't sound good at all
<gchristensen>
I think this idea might have merit :). I think a risk with this idea is making it more complicated to follow and understand how a build works.
<infinisil>
Well it kind of makes sense
<gchristensen>
it is kind of nice that $src is inert
<infinisil>
Tbh I think it would make it easier to understand
<qyliss>
src packages are often seperate in other package managers, fwiw
<infinisil>
Who knows how unpackPhase works even and what special cases it supports?
<infinisil>
Why does sourceRoot need the "source" prefix?
<qyliss>
our src packages are currently pretty meaningless, since they don't get you source you can build to get the right result
<qyliss>
infinisil: it's not too hard to read fwiw
<infinisil>
Yeah, when I do `nix-build -A hello.src` I kind of wish it would always just be a folder I can cd into
<infinisil>
Currently I often need to run <unpacker> first on it
<qyliss>
I mostly just wish it had its patches
<qyliss>
your idea wouldn't give us that, though, would it
<infinisil>
It totally would!
<gchristensen>
I don't mean this as a "go away" comment, I think the idea has merit: I think an RFC would be really great for this, to get a lot of people considering the impact it'll have and different ways it might be useful
<qyliss>
you'd have to nix-build -A hello.src && result/bin/unpack-src
<qyliss>
It's already getting a bit hard to keep track of on IRC
<infinisil>
qyliss: There could be a convenience attribute for actually building a derivation containing the final source
<infinisil>
E.g. `nix-build -A hello.src.processed`
<qyliss>
that makes sense
<gchristensen>
I have often wondered what it would look like to have each stdenv.mkDerivation phase be a different derivation
<gchristensen>
like, how annoying is it to have your 30min build fail at preInstall
<infinisil>
Hm yeah
<infinisil>
gchristensen: How about a development mode that does that, disabled by default
<gchristensen>
that'd be nice
<infinisil>
Although, that might give problems with self-referential packages
<infinisil>
Patching $out into the source..
<gchristensen>
:P
<gchristensen>
we have the technology
<infinisil>
We have?
<gchristensen>
sure! patching sed s/$predrvout/$out/
<gchristensen>
the limitations Nix enforces gives us incredible leeway to make bad and mediocre ideas work :P
<infinisil>
Oh yeah I guess that would work
<__monty__>
"Nix turns bad practices into OK practices. Tell your friends!" : )
<edef>
honestly, i think we overuse being able to use $out in patchPhase
<infinisil>
(probably have to watch out that all $out's are the same length though, to not mess up binaries
<edef>
i'd *like* to run everything before configurePhase in its own drv
<edef>
we already have things that are one source package and multiple binary packages
<edef>
cf, wireguard and wireguard-tools
manveru has quit [Ping timeout: 246 seconds]
<edef>
and a variety of build systems work great with unpacked source trees already as well
<edef>
immutable, unpacked*
<edef>
i'm fairly sure cmake is fine with immutable source trees, and meson definitely is
<infinisil>
edef: A big problem with this is the size of it though. Then there's 1 derivation with the compressed source and 1 with the unpacked source, potentially much bigger
<infinisil>
I'm thinking of e.g. nerdfonts which is like 2GB
<infinisil>
Extreme example, but it essentially means all sources now take double the space in your /nix/store
<edef>
mm
<infinisil>
Also, writing/unpacking the sources to /nix/store is potentially much slower than /tmp in which builds are done
<edef>
that's imo a deficiency in the nix implementation
<infinisil>
/tmp is often on tmpfs's
<qyliss>
not on NixOS
<qyliss>
And we recommend against changing that I'm pretty sure
<infinisil>
We do?
<infinisil>
boot.tmpOnTmpfs
<qyliss>
Defaults to false
* arcnmx
sets that ><
<arcnmx>
no slow tmp allowed o:
<qyliss>
there have been discussions about enabling it by default but the consensus IIRC seems to be that it's a bad idea
<edef>
oh? i've not seen those
* infinisil
tries to find the discussion
<qyliss>
I don't recall the exact details
<edef>
i definitely run a tmpfs myself
psyanticy has quit [Quit: Connection closed for inactivity]
<qyliss>
But I think the idea is that tmpfs memory in out of reach to the OOM killer
<niksnut>
I've removed the "blocked" tag - easy fix
<niksnut>
gchristensen: on the subject of changing the schema, I also seem to remember that some queries would be sped up a lot if we denormalize the DB
<niksnut>
for example, by adding a jobset field to jobsetevalmembers
<niksnut>
no wait, adding a "jobname" field to jobsetevalmembers
<gchristensen>
hmm interesting
<disasm>
FYI, I'm pushing a number of issues to the next milestone. If you feel you can complete something before Monday, please don't be discouraged and go ahead and merge. Just trying to make this milestone easier to track what still needs done.
<gchristensen>
disasm++ thanks!
<{^_^}>
disasm's karma got increased to 14
<niksnut>
currently, if you want the "foo" job from an eval, you have to join all the jobsetevalmembers with the builds table to get the one named "foo"
<gchristensen>
right, and I believe jobsetevalmembers is a huge table
<niksnut>
so that basically requires looking at all the builds in the eval
<niksnut>
should have used datalog...
<gchristensen>
hehe
<gchristensen>
yeah, jobsetevalmembers is a big table: 1,175,908,938 rows
<clever>
off the top of my head, that is an eval+buildid pair table, listing every buildid in every eval
<{^_^}>
#78670 (by tfc, 1 week ago, open): nixosTests.installer: Port installer and ZFS test to python
<worldofpeace>
I'm thinking any tests that aren't ported will just get marked as broken, same like with packages last release. But I expect make-test.nix to use the python driver, and keep like make-test-perl.nix, and mention this in the release notes that it's planned to be removed 20.09.
<worldofpeace>
surprised it hasn't got more attention
<samueldr>
I think an issue is that it abused of a particular implementation behaviour of nix-env, to remove results
<samueldr>
that makes it non-trivial to support well
jonge[m] has joined #nixos-dev
<jonge[m]>
hey samueldr clever do you have an idea what could be wrong here? i ported the installer tests and one of them sometimes succeeds sometimes doesn't on some missing /dev/block/xx:yy device in the `bootctl` step of the installation: https://github.com/NixOS/nixpkgs/pull/78670
<{^_^}>
#78670 (by tfc, 1 week ago, open): nixosTests.installer: Port installer and ZFS test to python
<{^_^}>
#68897 (by edolstra, 20 weeks ago, open): Flake support
<simpson>
Whoo! And then there will be a party.
<gchristensen>
qyliss: ^
bennofs has quit [Remote host closed the connection]
bennofs has joined #nixos-dev
<infinisil>
niksnut: Hm is it a good idea to merge this just before the branchoff? I think merging it right *after* would be better to allow for some stabilization time