<zybell_>
thoughtpolice:thats an instance of the plugin breakage I wrote on #nixos about. You can ofc try to paper it over, but it won't hold for long and the bugs will be harder to fix as more versions are affected in the future.
<thoughtpolice>
I'm not sure at all what you're referring to -- what's papered over? The current solution or the proposed one?
<thoughtpolice>
shlevy: Hmmm, how did you build a bootstrap Nix on RISC-V? You said you did it for Fedora, right? Where did you get boost_context from?
ma27 has quit [Ping timeout: 256 seconds]
<Dezgeg>
a guess: an older version of Nix that had bundled boost
<thoughtpolice>
Assuming boost_context was also patched for RISC-V. That's really the bigger question (since we use coroutines now in libnixutil) I suppose.
<thoughtpolice>
Err, oh. I see what you mean, before we even used coroutine, which was actually only added recently, I think.
<zybell_>
Ok, slowly: If you have plugins(any sort of plugins),you need a meeting point where plugin packages meet their hosts. Traditionally a dir in /usr/lib serves that purpose. Because of the structure of nix that is not possible. You try to use attributes as meeting point. But only for postgresql. And for versions,not APIs. As I think more about the solution 'paper over' may be false, because it could be *part of* a solution. But it should
<zybell_>
apply more generally to more pkgs.
<thoughtpolice>
Yes, we all know that. If the complaint is "I fundamentally don't like <design decision>", that's fine but it's literally 100% out of the scope of the issue in question.
<thoughtpolice>
I'm not asking "What problems are you thinking about today", I'm asking whether people who literally use the current PostgreSQL infrastructure have inputs on what would be a strict enhancement.
<simpson>
thoughtpolice: Whatever doesn't break Hydra gets my vote~
<thoughtpolice>
Mmm, does our Hydra use any PostgreSQL plugins at all, I wonder?
<thoughtpolice>
simpson: Doesn't look like it does, so in theory it shouldn't impact it at all. Really the proposed API is a pretty minor change overall, I think. And it could be made backwards compatible with a little effort too...
<thoughtpolice>
Well, backwards compat I'd say is mandatory for at least 1 release.
<zybell_>
Not so strict:In the future all plugins go to one meeting point, whereas now a individual solution is found. That individuality accidentally avoids API mixmatch breakage. Something that in the future explicitly must be provided for. You did the first steps for that, but I think you aren't wholly there.
<simpson>
thoughtpolice: Yeah, I'm not offended by the suggested change at all, but I don't use Pg directly, so I'm more concerned with stuff like (my) Hydra.
<MichaelRaskin>
I use PostgreSQL, but with no plugins.
<thoughtpolice>
simpson: Yeah, that's understandable, I have my own too. I'm pretty sure quite a lot of people actually *use* the postgresql module but I'm not sure how many use the extensions. I found this a surprising oversight.
<thoughtpolice>
Then again we only have like 5 or 6 of them (even less until I added a few)! so maybe a bit self-fulfilling, there
<thoughtpolice>
MichaelRaskin: https://github.com/NixOS/nixpkgs/issues/38469 is a silly bug which seems to indicate to me the extensions aren't used much, except maybe outside postgis.
<zybell_>
Is postgresql *as a* extension (guile-squeek) orthogonal to this?
<MichaelRaskin>
thoughtpolice: probably so
<simpson>
zybell_: Yes, the issue is clearly and explicitly about extensions to Pg.
<MichaelRaskin>
thoughtpolice: postgresqlXPackages and optCall of the plugin list sound like a reasonable plan in general
<thoughtpolice>
MichaelRaskin: I was wondering if I should make a new module attribute or override the old one with something like optCall, yeah.
<thoughtpolice>
I suppose I can add a deprecation to the old way, in any case.
<zybell_>
simpson: I thought it better to think about extensions more generally and do a 'extension consumer' and 'extension provider' API and make postgresql a shining example for both.
<zybell_>
It would even make the life of extension writers(to pg)easier.
<simpson>
zybell_: That sounds terribly overengineered TBH.
<zybell_>
lears on #nixos has a similiar problem with xmonad now. That is not overengineered. It is needed,NOW!
<simpson>
Why should similar problems have identical solutions? XMonad, for example, only has a single configuration file and then is recompiled, right? That's a very different model from Pg loading libraries at runtime, isn't it?
<zybell_>
I think every day at least one instance of that problem turns up (in several disguises)
<simpson>
Well, I still think that your idea is bad. You should prove me wrong by writing some Nix.
<MichaelRaskin>
… which will not get adopted anyway, because even good ideas don't actually get adopted, but could theoretically convince a slight majority of Nixpkgs developers.
<zybell_>
XMonad uses extensions. I didn't see anything about recompile. (Could it be someone compiled in dlopen()?)
<simpson>
Oh, okay, TIL.
<gchristensen>
btw zybell_, you seem new to the nixos community -- how did you find us?
<gchristensen>
it is cool to see people jump right in like you have
<zybell_>
simpson:Just to be sure:what do you think my idea was?
<simpson>
zybell_: I don't know. Why don't you tell us?
<simpson>
Or better, can you show us? Maybe just a mockup of what you'd like the top-level Nix API to look like for this?
<zybell_>
gchristensen:You seemed to have an awful lot of IRC-channels for an nix(=nothing)OS. I thought:funny.
kmicklas has joined #nixos-dev
<simpson>
samueldr: I don't have Nix commit bit, and I haven't tested your PR, but I applaud the amount of testing you've done for a single-character fix.
<samueldr>
it's not for the fix... it's for it not to break in the future :)
<samueldr>
I'm more concerned about the *quality* of the test, though it tests properly, I don't know if everything I added is sane with regards to the tests
zybell_ has quit [Ping timeout: 264 seconds]
zybell_ has joined #nixos-dev
<zybell_>
Not to keep you in suspense: I am in the process of thinking up a solution to a problem I'm clearly seeing, using too the (often unwitting) input from that great community. It has to do with all programs linking against libdl, that is: every program in NixOS.
<gchristensen>
not every program uses libdl though?
<zybell_>
gchristensen:every!program(nsswitch)
<gchristensen>
ah
<zybell_>
You can imagine my unwillingness to propose a half thougt idea if every program is affected. But on the other side I criticize half ideas in this area.
<zybell_>
I only want the other half thought too. Implementation is another matter.
<simpson>
If we don't see your problem, then maybe you haven't shown us. Do you have a reproducible test case for this problem, or a clear description?
<zybell_>
The clear description is: package xy cant find plugin-yz.so: No such file or ... strace shows it doesnt look into the dir where it is installed. Why? As plugin it is not a builInput. Compiling a package with all possible(!) runtime plugins is impossible.
<zybell_>
*buildInput
<simpson>
Right. At runtime, XY must be passed a capability for the YZ plugin, typically via env.
<zybell_>
Adding all possible plugindirs in /store to LD_LIBRARY_PATH doesn't scale, that is: it is possible atm with the few packages in NixOS, but on debian-scale numbers the env is too small. Except you wrap every prog into a script that sets LD_LIBRARY_PATH. That doesn't work with shebang lines, where you need another wrapper. And you have to *modify* the shebang-lines!
<MichaelRaskin>
We already hav eto modify shebang lines, though
<zybell_>
On build time ok,but runtime?
<gchristensen>
all possible?
<MichaelRaskin>
Well we do enough wrapping-equivalent stuff when preparing environments, that aso rewrapping/rewriting-shebangs would fit just fine.
<gchristensen>
we only do specific possible, not all possible
<MichaelRaskin>
Re: debian-scale: Nixpkgs is actually lagre enough that it is tricky to define what does it mean for it to have more or less packages than Debian
<zybell_>
How do you know that emacs-15.20 does *not* provide a plugin for postgresql-23.09?
<MichaelRaskin>
I know a much better thing
<gchristensen>
zybell_: sometimes, based on questions like that, I get the impression you've not actually used nixos
<MichaelRaskin>
If the user is actually using Nix and getting some benefits from it, they want control over what command-line programs are even allowed to be plugins for PostgreSQL
<simpson>
zybell_: I'm willing to entertain the question, but you breezed right past my point that emacs would need to have a dynamic capability (paths into the Nix store are capabilities!) passed to it. That's the only way that we can preserve the Nix security model, y'know? Like you say, build inputs are for build time.
<zybell_>
gchristensen:You may have misunderstood me. The question is not Can an arbitrary package provide a plugin (and make the host load it)? But Can the user configure a plugin to be loaded from a foreign package when the config of the loading prog doesn't allow that package to be specified?
<simpson>
No, that's a defect in the 'loading prog'. We could carry a patch to fix it, but ideally upstream would fix it instead.
<gchristensen>
or something like emacsWithPackages maybe
<simpson>
thoughtpolice: Nice, the package-sets.nix is definitely great.
<thoughtpolice>
I'll probably have to do some extra nonsense to make it support version range exclusions (e.g. foo-x.y.z doesn't build with PostgreSQL 11, or whatever). haven't tested prior to postgres96 yet
<zybell_>
simpson:For that dynamic capability Thoughtpolice proposed a nix-attribute,by package and version. By package is clear because you dont want plugins written for other sw, but in the case of libpq.so, which can be source to many language bindings, even that fails. And version when you mean differently stable parts of an API is not a good idea IMHO, but as clearly I confess not to have a better atm.
<MichaelRaskin>
Well, there are packages where versions mean something — Linux kernel stable branches, PostgreSQL.
<zybell_>
If nix-attributes could work like symlinks, I would feel better with that approach.
<zybell_>
MichaelRaskin: A plugin that uses only the stable API, that is works with *every* version, how can it declare that?
<MichaelRaskin>
In our case it is still linked against libpq.so by full path, so it does depend on a version
<MichaelRaskin>
Nix does more rebulds than one could expect, and dependency replacement is a topic which is still in discussion…
<zybell_>
Ok, AFK
pie__ has joined #nixos-dev
pie_ has quit [Ping timeout: 276 seconds]
<thoughtpolice>
gchristensen: Ping, if you're around
<gchristensen>
somewhat
<thoughtpolice>
gchristensen: So I pulled the 2.0-maintenance branch of Nix and built with make && make install. First off, thank you because the daemon works perfectly. :) I was wondering if there's any post-setup needed, since I get a warning on nix-repl:
<thoughtpolice>
warning: the group 'nixbld' specified in 'build-users-group' does not exist
<gchristensen>
nice!
<thoughtpolice>
Also, I was wondering about the nix.conf file, and how to set up .profile appropriately
<thoughtpolice>
Normally the binary installer updates .profile -- but make install won't, it seems, right?
<gchristensen>
hmm
<thoughtpolice>
(ISTR also that Nix recently was updated so that it would use dynamically allocated build users/groups, but maybe that's not in the 2.0 branch)
<shlevy>
thoughtpolice: It's not on master even, just an experimental branch
<thoughtpolice>
Oh, my mistake.
<gchristensen>
I don't know what `make install` does I do: nix-build ./release.nix -A binaryTarball.x86_64-linux, extract the result, and run the installer inside it
<thoughtpolice>
Ah, I have to bootstrap from 'make'. I'm running this build on RISC-V ;) So I'll probably just have to do a little extra post setup myself, nbd
<thoughtpolice>
thanks gchristensen -- I'll hopefully get to a rebuild with Nix itself quick enough. Unless shlevy has already done it and wants to tell me how... :)
<shlevy>
thoughtpolice: I pinged you in #riscv... I made an installer tarball already!
<shlevy>
thoughtpolice: If you have a good way for me to give you a 145M tarball you can test it ;)
<thoughtpolice>
Oh, I missed that. Sorry!
<shlevy>
But anyway "make install" behaves normally (i.e. installing into /usr or /usr/local or whatever) with Nix, it's just normal autotools
<shlevy>
But by default it points to /nix
<shlevy>
(as in, installed in /usr, but with store at /nix)
<thoughtpolice>
Right, but there's nothing to do group setup, etc. Fair enough -- not the hardest fix anyway
<shlevy>
Oh, yeah
<Dezgeg>
don't run as root, no need to setup groups then
<shlevy>
You don't want to run as root actually
<shlevy>
we don't have seccomp on risc-v
<shlevy>
So we can't prevent setuid binaries
<Dezgeg>
so it would have been beneficial to do an nosuid bind mount instead of seccomp :P
<thoughtpolice>
I'll admit I find that to be low on the list of things I should really be worrying about right now. :P
<shlevy>
:D
<shlevy>
Fair
<thoughtpolice>
A fix for another day. Also yes I do have somewhere you can put that tarball -- throw me an ssh key and I can make you an account
<thoughtpolice>
shlevy: I've already got a build myself like I said in my own Fedora VM -- is there any branch you were using of nixpkgs to bootstrap the tarball itself?
<thoughtpolice>
I didn't even see riscv64-linux in lib/systems.nix for example for meta.platforms
<shlevy>
thoughtpolice: on staging cross-compilation to RISC-V works fine
<thoughtpolice>
Ahhhhh
<shlevy>
I haven't published anything that tries to get a native stdenv
<thoughtpolice>
too many branches already ;_;
<thoughtpolice>
(personally anyway, I'm juggling a bunch)
<shlevy>
staging will be very helpful once your'e ready to start on a native stdenv, as it has the right binutils and glibc :)
<shlevy>
thoughtpolice: have to head out soon, let me know when I can start the upload
<thoughtpolice>
Is there any ETA on staging
<thoughtpolice>
shlevy: I'm making an account now, one second
<shlevy>
thoughtpolice: Nothing concrete, sadly. I'm hopeful https://github.com/NixOS/rfcs/pull/26 can resolve it btu currently waiting on feedback from vcunat
<MichaelRaskin>
gchristensen: re: your float twits: I think using these floats migh make a person a better programmer of numerical things… They _are_ good enough to describe realistic periods in seconds, though.
pie__ has quit [Ping timeout: 255 seconds]
<gchristensen>
MichaelRaskin: hmm I didn't know you were a twit.
<MichaelRaskin>
I don't actually have an account, or I would probably reply there
pie_ has joined #nixos-dev
<gchristensen>
oh, cool. you're saying the limitation is a good thing?
<MichaelRaskin>
There are a few feeds that my spider regularly grabs (yours is not in that list), and ther are a few that I sometimes look at
<gchristensen>
good idea, it would be bad for my ego. *redirects some of this to -chat*
<MichaelRaskin>
Well, you want either to pay real attention to make sure the pitfalls have the standard configuration, or the pitfalls should be made a bit larger so that people notice them in advance
<MichaelRaskin>
So the precision bottoms out around one part per million