drakonis has quit [Read error: Connection reset by peer]
init_6 has joined #nixos-dev
<samueldr>
I may need an adult^W someone with github orgs admin rights to enable github actions for mobile-nixos, or else I don't know why they are not running in mobile-nixos#112
<samueldr>
I've been able to get the errors by evaluating using strace on the hydra evaluator
<samueldr>
the warning is benign though
<samueldr>
(my build machine filled up and the eval aborted after 140 minutes due to / being full... so I won't have another eval done before way later tomorrow)
<FRidh>
thank you. I won't give that a try on my machine :)
cole-h has quit [Quit: Goodbye]
init_6 has quit [Ping timeout: 250 seconds]
orivej has joined #nixos-dev
init_6 has joined #nixos-dev
teto has joined #nixos-dev
__monty__ has joined #nixos-dev
<jtojnar>
samueldr it took exactly an hour on my laptop too
<jtojnar>
how do you find the error message in the strace log?
misuzu has quit [Remote host closed the connection]
Jackneill has joined #nixos-dev
misuzu has joined #nixos-dev
orivej has quit [Ping timeout: 250 seconds]
Jackneill has quit [Ping timeout: 265 seconds]
ixxie has joined #nixos-dev
ixxie has quit [Ping timeout: 264 seconds]
lassulus_ has joined #nixos-dev
lassulus has quit [Ping timeout: 256 seconds]
lassulus_ is now known as lassulus
lassulus_ has joined #nixos-dev
lassulus has quit [Ping timeout: 252 seconds]
lassulus_ is now known as lassulus
WilliButz has quit [Remote host closed the connection]
WilliButz has joined #nixos-dev
lassulus_ has joined #nixos-dev
<pie_[bnc]>
the 19.09 release notes have this hidden gem in them
<pie_[bnc]>
19.09 release notes:
<pie_[bnc]>
" We now install the sysctl snippets shipped with systemd.
<pie_[bnc]>
This also configures the kernel to pass core dumps to systemd-coredump, and restricts the SysRq key combinations to the sync command only. These sysctl snippets can be found in /etc/sysctl.d/50-*.conf, and overridden via boot.kernel.sysctl (which will place the parameters in /etc/sysctl.d/60-nixos.conf). "
<pie_[bnc]>
do we really want to disable all sysrq by default
<pie_[bnc]>
who will i have to die on this hill against
<pie_[bnc]>
sysrq working has been a base rescure assumption for linuxen for ages and should not be disabled by default
<pie_[bnc]>
if you want to lock down your machine you can do so
<pie_[bnc]>
gettingyour system into some sort of failed or DOSes state is easier than some random person walking up to it and resetting it, unless you like to leave your laptop running unattended at conferences
<flokli>
I think there's good reasons most distros disable it by default, and don't see why NixOS should use other defaults here. If you want sysrq on a laptop running fedora/Ubuntu/.., it's the same thing.
<MichaelRaskin>
The only reason other distros disable it by default is because they blindly accept systemd default change here
<pie_[bnc]>
if you disagree then add a default entry to configuration.nix when the the nixos system is regenerated and dont add it into the "oh heres some extra unimportant release notes section"
<pie_[bnc]>
sorry, im getting a bit fired up on the last part there
<pie_[bnc]>
ill retract that
* Emantor
is with flokli here.
<Emantor>
Its even easier to enable in NixOS for all your machines because of config sharing.
<pie_[bnc]>
by which i mena "unimportant release notes"
<pie_[bnc]>
Emantor: you enable it after youve screwed yourself over by not having it
<pie_[bnc]>
all sane sysadmins ASSUME they have sysrq unless they have it pulled out from under them by systemd
<Emantor>
Thats a one time cost. And in this case fine to make since most users dont need it/dont know it exists.
<pie_[bnc]>
then add it to configuration.nix
<pie_[bnc]>
by which i mean the default generated one
<pie_[bnc]>
and if you want to cargo cult reasoning, why not just say "well the kernel enables it
<flokli>
I don't think distros have just disabled sysrq "because it was pulled out from under them by systemd". I assume they did before, too
<pie_[bnc]>
nope, last i checked it works on ubuntu, though it has been several years since i used ubuntu tried
<flokli>
And the current behaviour is pretty much how it's on all other distros as well. So just locally enable it if it's an "attended box", but keep the default secure.
<pie_[bnc]>
why is sysrq a security issue
andi- has quit [Remote host closed the connection]
<pie_[bnc]>
worst you can do is reboot
<flokli>
Did you read the article?
<pie_[bnc]>
which may be a problem under some threat models sure
<flokli>
You can still reboot. But you can't dump register values.
<pie_[bnc]>
all sysrq are disabled by default
<pie_[bnc]>
or whatever 16 sets but i couldnt reboot, so let me check
<MichaelRaskin>
sync-only?
<pie_[bnc]>
im certain im not going to change what you think anyway
<MichaelRaskin>
I think current Debian enables sysrq
<pie_[bnc]>
ok yeah if you are insistent on disabling register dumps, which i admit is not something even the average sysadmin needs, enabling most "safe" sysrq should be fine
<flokli>
What I think doesn't really matter that much. But what a lot of distros use might be that way due to good reasons.
<flokli>
So if we want to alter from community-agreed defaults, we need to have very good reasons.
andi- has joined #nixos-dev
<pie_[bnc]>
the alteration is the disabling and you seem to have accepted that without much issue
<flokli>
And for now, all I heard was "didn't read the release notes, and suddenly sysrq didn't work on my laptop"
<pie_[bnc]>
the reasoning isnt hard to accept but security fearmongering~
<Emantor>
MichaelRaskin: current debian has kernel.sysrq = 438
<pie_[bnc]>
fwiw i did read the release notes but you dont remember every detail
<pie_[bnc]>
so i am certainly partly at fault here
<MichaelRaskin>
438 sounds like disabling the most risky stuff only
<flokli>
Please open a nixpkgs issue, and hear other peoples opinion on that. IRC probably is a bit lossy. Also, it might make sense to do some git archaeology on systemd about how these values were chose.
<MichaelRaskin>
flokli: a community consensus not shared by default Debian install is not a consensus
<pie_[bnc]>
hell, i didnt even *know* you could disable individual sysrrq and i assumed theyre all enabled because -h worked, so id been saying wrong things all night
<Emantor>
MichaelRaskin: this enables everything except process dumps and killing processes.
<pie_[bnc]>
Emantor: thats better than me asking on irc
<pie_[bnc]>
flokli: to be honest, i might be phrasing things more strongly than needed here out of frustration, in case it sounds like im being agressive towards you, i dont mean it, i dont care that that this got pulled in without much thought because systemd, ijust thnk it should be changed
<pie_[bnc]>
flokli: sure ill open an issue, i expect it to turn into a big bikeshed
<pie_[bnc]>
i was going to open an issue anyway till i found the git blame for what caused this and figured it would be easier to bring it up
<Emantor>
The debian default looks good.
<flokli>
pie_[bnc]: well, it came around quite aggressive. I can understand it was frustrating it suddenly didn't work, but we should see how the defaults in other distros developed, and then come to an informed conclusions
<flokli>
"meh, it broke my laptop, let's revert" and some rants on systemd don't really help there.
<flokli>
So happy to continue the discussion in an issue
<pie_[bnc]>
stuff like this happens because everyone just does whatever systemd gives them
* Emantor
can queue the next bikeshed discussion about CONFIG_RANDOM_TRUST_CPU…
<pie_[bnc]>
not that anyone has the capacity to audit every little detail
<MichaelRaskin>
Well, this commit comes with no reference why is this exactly the safe options.
<pie_[bnc]>
disable everything is pretty safe
<pie_[bnc]>
but so is turning off your computer
<pie_[bnc]>
'x0
<pie_[bnc]>
* 'x)
<MichaelRaskin>
Actually no, there are cases where sending E before S reduces the risk of data loss
<flokli>
Heh
<pie_[bnc]>
im not sure i can add anything constructive here other than googling up other peoples upstream threads so im going to take a break
<MichaelRaskin>
I mean, after https://github.com/NixOS/nixpkgs/issues/36247 my assumption «sometimes weird changes in systemd leak downstream into NixOS, happens, need a manual override»
<{^_^}>
#36247 (by yesbox, 2 years ago, closed): Mosh/tmux/screen broken in NixOS 18.04
<flokli>
pie_[bnc]: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/194676 has many references to other distros, and how they set these values. If there's something more permissive/useful that still is secure, we should probably just PR this to systemd.
<pie_[bnc]>
honest question, you think they care?
<flokli>
but killing screensavers also sounds scary, so sysqr+f should probably be disabled too
<flokli>
if we provide a good justification why, I think so.
<pie_[bnc]>
they have their safe defaultts and they will proably keep them because they are upstream and downstreams probably use it to lock down their platforms
<pie_[bnc]>
but...sure why not give it a shot i guess
<flokli>
I did some PRs to systemd already, and some of them got accepted.
julm has quit [Quit: rebuild]
<flokli>
So if we come up with a better default that makes sense across all distros, getting it into there sounds like the right path
<pie_[bnc]>
if you are ok with working on that that would be good
<MichaelRaskin>
Given Debian and Fedora disagree…
<pie_[bnc]>
open source culture demands that since im clamoring i fix it but im pretty sure im not going to try that today
<pie_[bnc]>
youre much better qualified to submit and argue for a pr, i feel like id just get shot down
<pie_[bnc]>
i wonder if debian or fedora checked with upstream about a change
init_6 has quit [Ping timeout: 260 seconds]
<flokli>
I don't know either. As I said, let's open an issue in the nixpkgs repo, do some archeology, and if there's a safe, more useful default to set, we can PR that to systemd.
<yorick>
I've never gotten anything done by opening a nixpkgs issue
<yorick>
PRs have slightly better luck
<flokli>
if I run into something I can't quickly fix, I open an issue, so others know about it, might help me in fixing it if they're affected too.
<flokli>
I might end up fixing it by myself, but might have collected some feedback from the community in the meantime.
<pie_[bnc]>
anyway yeah, im going to force myself to make breakfast now
<flokli>
pie_[bnc]: bon appetit :-)
<pie_[bnc]>
systemd could accept a pr, or it might not, i can understand their reasoning for the default, but that doesnt mean i think its justified for *us*, they are a library (uhh?) we are a distro
<pie_[bnc]>
(breakfast pie_[bnc]! breakfast!)
<pie_[bnc]>
but we can still try a pr ofc
<Emantor>
pie_[bnc]: The maintainer comment was during the vote on the default init system preference in debian.
<pie_[bnc]>
what i actually meant is just because they set that config option doesnt mean people necessarily pull it in as is?
<pie_[bnc]>
flokli: i would like to once again apoligize but i still feel largely justified in my fervor
<pie_[bnc]>
which is atypical i have to say
<flokli>
pie_[bnc]: I'm composing a reply to the issue
<pie_[bnc]>
and i think my reasoning is that sysrq is your last line for rescuing a system before a hard reboot, unless its not the processing thats locking up and you happen to have ssh in
<pie_[bnc]>
it would be less of a problem if it was common knowledge that sysrq exists but is disabled by default
<pie_[bnc]>
but maybe a decade is indeed a gray beard in average computing
<pie_[bnc]>
and im not one to be about "tradition" but i dont like losing control of a system that would be rescuable you just cant reach the stuff behind the fence
<pie_[bnc]>
"muh uptime"? which is only important because i have lots of long running sessions with accumulated state...but maybe thats my fault for expecting desktops to be reliable
<pie_[bnc]>
which they are not and indeed i keep finding novel ways like the previous, to clean the slate
<pie_[bnc]>
but i havent had a kernel hang for months so barring this my uptime was good
<pie_[bnc]>
so rephrase, does "but muh debugging" warrant a change
<pie_[bnc]>
do you want a secure system, or an engineerable system?
<pie_[bnc]>
its not like we have built in configuration profiiles
* pie_[bnc]
wouldnt mind a poke all the things switch which enables debugging by default etc
<flokli>
pie_[bnc]: how would you have used sysrq, if it was fully enabled?
<pie_[bnc]>
(better enableDebugging is on the bucket project list)
<flokli>
invoked kgdb?
<pie_[bnc]>
probably would have tried -r and -v
<pie_[bnc]>
ive certainly learned some things today
<MichaelRaskin>
And sometimes -k is pretty useful
<pie_[bnc]>
you know sometihng we forgot to chant today?
<pie_[bnc]>
"what is your threat model"
<pie_[bnc]>
as a software engineer i feel like people dont place enough emphasis on the inspectability of systems :( (nixos wil let us be better if we want to though)
<flokli>
cole-h: I can't reopen there, but gchristensen probably can.
FRidh has quit [Quit: Konversation terminated!]
<cole-h>
gchristensen++ flokli++
<cole-h>
Oh right, bot's dead again
<gchristensen>
ehhh
<gchristensen>
whyyyy
<gchristensen>
I fixed you
orivej has joined #nixos-dev
<{^_^}>
flokli's karma got increased to 8
<{^_^}>
flokli's karma got increased to 9, gchristensen's karma got increased to 242
<{^_^}>
nixos-homepage#372 (by das-g, 19 hours ago, open): "Nix expression" on package search results links to wrong URLs
<gchristensen>
huh
<drakonis>
yowza.
<julm>
gchristensen: thanks for bringing sanoid/syncoid to my attention the other day, I'm trying its recent implementation in NixOS, works great so far
phreedom has quit [Remote host closed the connection]
<gchristensen>
nice! cool :D
phreedom has joined #nixos-dev
<julm>
I don't know how it compares to znapzend or else, but so far it's enough do-one-thing-but-do-it-well for me
<niksnut>
flokli: isn't that an ofborg issue?
ixxie has quit [Ping timeout: 258 seconds]
<gchristensen>
even if it is, pushing to master and breaking all the PRs is pretty rough
<flokli>
even if it is, if it goes via a PR, we see it breaks ofborg tooling
<flokli>
and in the current case, I think it's `nix-instantiate --arg nixpkgs "{ outPath=./.; revCount=999999; shortRev=\"ofborg\"; }" ./pkgs/top-level/release.nix -A tarball` which got broken
<flokli>
yup.
<niksnut>
yes, that needs to pass a 'rev' argument
<gchristensen>
right, so let's fix ofborg and then merge the PR so we don't break all the PRs
<flokli>
I think we would need to keep the API to some extend. Ofborg should still work on the still-supported branches.
<Emantor>
Huh, won't this break older PRs not containing the commit?
<niksnut>
hydra is the API
<gchristensen>
having PRs validated for evaluation is a major benefit to everybody, and showing some courtesy to it is showing courtesy to all the other contributors and reviewers, even if it doesn't perfectly match the API for hydra
<gchristensen>
so let's fix ofborg, but let's not break the PR checks first
<gchristensen>
what is the fix that is needed?
<gchristensen>
and what is the correct attrset to send?
<gchristensen>
flokli: can you send a Pr adding `rev = ...`?
<flokli>
gchristensen: I'd expect this to then break PRs for 19.09 or 20.03
<flokli>
like Emantor also wrote
<gchristensen>
that passing `rev` would break them?
<gchristensen>
hydra has always passed `rev` afaik, or was there something else?
* flokli
shrugs
<gchristensen>
I mean, what is the concern? that ofborg passing `rev` would break it?
<gchristensen>
or maybe Emantor you can clarify?
<flokli>
after taking another look, passing in rev for commits before that commit should be fine too
<gchristensen>
I guess we can just ignore it then
<gchristensen>
ah okay
<flokli>
But I wonder why it suddenly became mandatory - we default to a lot of zeroes in many places
<gchristensen>
I think it just started being used by the eval itself
<gchristensen>
we can pass all zeros for ofborg, too
<flokli>
yeah, I'll file a PR for that
<flokli>
but TBH, things like this should go via PRs, so we can catch it.
<gchristensen>
yeah definitely
<flokli>
and shortRev can probably also be removed, if the full revision now is mandatory.
<gchristensen>
we might as well keep it, to stick with what hydra passes
<MichaelRaskin>
Should shortRev be a prefix of rev then?
<niksnut>
Ericson2314: don't know what behaviour you have in mind
<MichaelRaskin>
After module system, that claim «not a general-purpose language» is hard to believe
<niksnut>
afaik, with fetchGit ./., indexed but uncommitted files are visible
ixxie has joined #nixos-dev
<niksnut>
MichaelRaskin: while obviously useful, the module system is also an example of something that should have been a language feature
<Ericson2314>
oh hmm it already does `fetchGit ./.`
<niksnut>
and once implemented as a library feature, it gets so many ad hoc features, it's impossible to ever turn it into a language feature (at least not in a backwards-compatible way)
<Ericson2314>
Odd becauses I am pretty sure I had to git clean to fix a build once
<Ericson2314>
(a nix build)
<Ericson2314>
well, I'll look into it
<Ericson2314>
niksnut: while you are here, had a chance to look at those PRs? :) Mind you I would love to keep on cranking them out at this rate for a month or so, if that can go a long gway to answering's shea's question of "how can nix be a nicer library to use?"
<niksnut>
no
<Ericson2314>
Do we have any sort of guidance at what sorts of changes the other people with commit access can review, so I am not just dumping 50 PRs on you?
<samueldr>
while you're on, in case you missed it, niksnut, at your own convenience if you can look at and maybe guide towards the issue for https://github.com/NixOS/hydra/issues/728 it would be helpful as channels are blocking without clear information about the error
<samueldr>
(and I *do* mean at your own convenience)
<Ericson2314>
For stuff like window support or meson, that is a big deal of course, and same for user-facing changes. But there is a bunch of other stuff that is minor pure refactoring
<niksnut>
usually "unexpected EOF reading a line
<niksnut>
" means OOM from a child process
<niksnut>
or some other crash
<samueldr>
please read the issue, within the strace output the error can be seen as being "written" by a process, and then "read" by another
<samueldr>
and they are not OOM, both occasions they were tangible errors we had to act on
<niksnut>
Ericson2314: regarding "nix as a library": we haven't even decided whether we want to stick with C++ or move a bit more towards Rust, so it's hard to commit to a stable API at the moment
<Ericson2314>
niksnut: I absolutely don't want to commit to a stable API yet either
<niksnut>
ok :-)
<Ericson2314>
And all the C++ refactors I've done I am definitely mentally writing rust and then converting it :)
<Ericson2314>
so we are ready to stick in C++ (not trying to be horribly unideomatic, e.g. doing the more std::copy after all) or switch to Rust
<pie_[bnc]>
I think hiding things behind opaque language features is counterproductive, but I'm not in a position to get into this. <niksnut> MichaelRaskin: while obviously useful, the module system is also an example of something that should have been a language feature
<pie_[bnc]>
If you want to do funcitonal engineering you make things composable, and opqaue things are not. Unless you already have some sort of formalization that you know absolutely perfectly covers some sort of algebra.
<MichaelRaskin>
My opinion that module system is just the largest example, and even mkDerivation already does quite a bit of computation
<pie_[bnc]>
Similar to why people should engineer applications as libraries with a frontend.
<MichaelRaskin>
I would of course like the move to doing more things outside the module system and only use it for final composition if at all, but I have ulterior motives for config generators to be easier to reuse outside NixOS
<Ericson2314>
MichaelRaskin: pie_[bnc] A lot of what I am interested in with content adressable derivations and ret-cont is taking the language out of the hot path, especially with flakes it is nice if the basic derivation language is pretty powerful
<pie_[bnc]>
As opposed to putting everything in a single executable.
<Ericson2314>
Related to refactoring, is questions about how we can have as little code in the daemon as possible: e..g no libnixexpr or flakes stuff etc
<niksnut>
for a DSL, having functions is probably a mistake because it allows everybody to define different abstractions (so it becomes hard to read other people's nix expressions)
<pie_[bnc]>
Ericson2314: I quite follow but if the base derivation stuff ends up sufficiently coherent and powerful to hold everything together with whatever you put on top of it, sure why not?
<pie_[bnc]>
*I don't quite
<Ericson2314>
niksnut: meson removing functions makes me puke but that is the reason and I do understand and sympathize
<MichaelRaskin>
I have a suspicion that local abstractions are how Nixpkgs survives being permanently bottlenecked about everything
<pie_[bnc]>
Ericson2314: but thats like saying ok here is a "typed assembly language" you can generate it with whatever you want.
<niksnut>
Ericson2314: until very recently the daemon didn't link against libnixexpr etc. since it doesn't do any evaluation
<pie_[bnc]>
this sounds ripe for fundamental disagreement :D <niksnut> for a DSL, having functions is probably a mistake because it allows everybody to define different abstractions (so it becomes hard to read other people's nix expressions)
<pie_[bnc]>
MichaelRaskin: can you expand?
<Ericson2314>
niksnut: I was guessing that was something you *wouldn't* like meson, but maybe I was wrong. (still the lack of scopes and overliance on mutation is no good.) What changes so libexpr was needed?
<Ericson2314>
MichaelRaskin: I agree, which is why I almost want something likes flakes to be more a part of nixpkgs than nix!
<MichaelRaskin>
pie_[bnc]: each language ecosystem defines its own set of abstractions… I am not sure any of options in a no-abstraction DSL sound appealing
<Ericson2314>
There's <basic trusted building thing DAGS | tons and tons of abstractions so we don't go insane | package all the things >
<Ericson2314>
as a 3-layer stack
<pie_[bnc]>
MichaelRaskin: Now I cant tell if you;re agreeing or disagreeing.
<MichaelRaskin>
Ericson2314: I am just scared by scopedImport existing
<simpson>
pie_[bnc]: A module system need not be much more than what we have: A calling convention, a rule for composition, and a loader which can assemble modules into a single bigger module.
<niksnut>
Ericson2314: nothing really changed except that nix-daemon is not a separate executable anymore. it still doesn't use any libexpr though
<pie_[bnc]>
MichaelRaskin: also, yeah more consistent conventions would be great. And I think we lose a lot to (seemingly?) not having anyone putting serious effort into what consistent abstractions should be here.
<puck>
MichaelRaskin: dw, it's perfectly emulatable with only `import` and a local store :)
<simpson>
It is *very* tempting to just hack out whatever is needed in glue code, and so that module signature and calling convention need to be frozen, documented, enforced.
<pie_[bnc]>
MichaelRaskin: For example I have work on trying to factor out the various withPackages impementations in nixpkgs
<Ericson2314>
niksnut: if we could enforce some things don't need the stuff they are linked with ,taht would be nice
<niksnut>
Ericson2314: regarding meson, yeah not having functions sounds like a good thing. Even GNU make has a function mechanism and it leads to unreadable code.
<MichaelRaskin>
pie_[bnc]: if build*Package, withPackages etc. needed to be built-in abstractions, we would just not get off the ground packaging anything.
<Ericson2314>
niksnut: OK cool. Well this is good to know, helps me answer the question of whether to do windows support before or after 100% rust
<Ericson2314>
pie_[bnc]: I think humans just aren't disciplined enough, which is why we either need types
<pie_[bnc]>
simpson: sure, but thats bad engineering discipline, and the only way I can think of off the top of my head is to have a decent stdlib, which people have told me they dont want - or at least that was my impression. - furthered by emphases that nix shouldnt be a general purpose language. and you also cannot enforce structures because nix does not have a type system.
<Ericson2314>
you just don't get deep layers of abstractions without types, or if you do, they are impossible to refactor
<yorick>
the module system, implemented in nix, is neither fast or easy to understand
<pie_[bnc]>
Ericson2314: see what i just said
<pie_[bnc]>
my engneering principles are consistency and compositionality
<Ericson2314>
so you get one shot to make it perfect --- 2nd system helll
<pie_[bnc]>
one of the ways to enforce consistency is by reusing data types, and welll im not really sure what to say for the second part
<pie_[bnc]>
Ericson2314: we are probably agreeing with eachother but i dont see where that leaves us
<MichaelRaskin>
yorick: a part of not easy to understand seems to be that its aim is more or less emulating global writeable state with just enough constraints for convergence
<Ericson2314>
:)
<yorick>
MichaelRaskin: yes, but you could write this in C++ with more clarity and efficiency
<yorick>
and possibly should
<simpson>
pie_[bnc]: I don't buy criticisms that rely on the assumption that somebody, somewhere, is good at writing code. In particular, in this instance, I don't think that it's right to assume that there exists somebody who will refactor ad-hoc module systems. We should focus on making Nix the right language for doing what it does, not on making Nix support arbitrary abstracta.
<Ericson2314>
yorick: but if we have two different repos one definiing the abstraactions one using them, written in two different langauges
<pie_[bnc]>
I suppose I would not be against better primitives and I might be arguing about where the line is drawn.
<Ericson2314>
the debug cycle has gone to shit and we've lost 100x productivity
<simpson>
We have two pretty good modular calling conventions right now, and we could enshrine one of them.
<yorick>
Ericson2314: the module system api doesn't change more than once a yaer
<pie_[bnc]>
simpson: so this is about about moving language into library because the language in inadequate
<Ericson2314>
yorick: and it only grows and grows, no good. Change is overdue
<pie_[bnc]>
simpson: I did not mean to imply someone is good at writing code, they are but I'm not
<Ericson2314>
We have serious problems of ambient authority with the module system, and it's unclear how to address it
<simpson>
pie_[bnc]: But we gotta keep evenflow, and also move library into language because the library is crowded and heterogenous.
<pie_[bnc]>
I think we have some major development culture problems which end up being technical problems because we cant compensate.
<simpson>
julm: The other is nixpkgs' convention for calling parts of the tree with auto-guessed arguments.
<julm>
callPackage ?
<pie_[bnc]>
(I have to say the nix community is amazing and I love it, but there are some big problems I think)
<pie_[bnc]>
re this whole discussion
<MichaelRaskin>
Reaching decisions is hard everywhere; Nix* is pretty good at working together despite unresolved disagreements, though.
<Ericson2314>
niksnut: We agree on what the problems are, but I think designing a DSL which solves them i much harder than you do.
<pie_[bnc]>
The adhoc development is really great, and it seems like some solid bases have developed - I'm quite new the > 10 run considered - but it looks like theres more designs that should start crystallizing out of the things in nixpkgs
<Ericson2314>
niksnut: but 1) I am curious whether you thought of datalog when writing theses DSLs and 2) for the record my Nix PRs are supposed to not hinge on my spicy opinons--I am not trying to show-horn those in :)
<MichaelRaskin>
With no abstraction, I expect cross-compiling to become prohibitively expensive and LibreOffice package a larger mess than it is; with _some_ limited abstraction kind of sufficient for build*Packages I expect Nixpkgs to become a Turing tarpit
<pie_[bnc]>
maybe im bringing my personal preferences in here too much but a recurring small annoyance of mine is that we dont have an stdlib wrapper for adding pre and post hooks to phases, and people always override them
<pie_[bnc]>
this is something that should become an encoded convention, or a programmatic change to stdenv
<Ericson2314>
pie_[bnc]: the structured attrs was supposed to help with that, becausese there actually is a lists we can append to if it becomes a bash array
<Ericson2314>
ok brb have a phone call
<pie_[bnc]>
Ericson2314: cool, not to say progress isnt being made, just hasnt trickled down to me yet :)
<niksnut>
Ericson2314: yes, CUE is a good inspiration
<pie_[bnc]>
(on that note , i acknowledge ther there *IS* a lot of good background work being done)
<pie_[bnc]>
I think nix's genius is in bringing the compositionality of funcitonal programming to system configuration. I would be reluctant to detract from that basis.
<pie_[bnc]>
*the compositionality of functional programming, and in fact a proper programming language, to system configuration
<pie_[bnc]>
as opposed to the n+1th crippled programming language for system configuration
<pie_[bnc]>
I am *not* against better stdlib, and more efficient primitives that fit the structures we need
<pie_[bnc]>
am I ranting unnecessarily? :D
<simpson>
I think Nix's genius is in being a nice compiler target. I can imagine a future where a reasonable nice language (not like they exist, but *imagine*) is used to perform high-level actions, and emits Nix expressions as part of a package-management task.
<simpson>
pie_[bnc]: A little? Like, a lot of this is handled directly by the RFC process, and while I could agree that that process is slow, it does at least exist.
<pie_[bnc]>
simpson: which might be another way to phrase what I imagine Ericson2314 meant by better derivation stuff
<niksnut>
.drvs are basically the low-level target
<pie_[bnc]>
right
<ma27[m]>
completely unrelated: did anybody of you successfully ran the kernel selftests on NixOS?
<niksnut>
Nix ended up as a function language because 1) that's what we did in Utrecht; 2) I didn't really know what the appropriate abstractions for the domain should be
<pie_[bnc]>
maybe we should be arguing about drv primitives ;P nixpkgs has too much inertia to port to any major changes anyway right? :PP
<Profpatsch>
niksnut: not just drvs, nix expressions are a compile target on their own right by now.
<niksnut>
so being a function language is useful because it allows quick iteration (e.g. building the module system)
<Profpatsch>
2) is fantastic because without that nixpkgs wouldn’t be a thing.
<pie_[bnc]>
niksnut: I would tentatively raise that functinalness is what let it get this far
<MichaelRaskin>
And cross-compilation
<pie_[bnc]>
right
<pie_[bnc]>
^ re: iteration
<niksnut>
but now that we have a better understanding, it's worth investigating DSLs that don't suffer from the lisp curse
<MichaelRaskin>
Module system is closer to means than to ends
<niksnut>
(and that don't require GBs of memory to evaluate...)
<simpson>
niksnut: To follow that line of thought, I would imagine that a high-level environment would be chimeric, composed of multiple languages, and that instantiation is only one sort of action that's possible. So I would argue that a mid-level language *without* abstractive macros or eDSLs is required anyway, as a sort of linguafranca.
<Profpatsch>
niksnut: have you looked at dhall-nix? You can use dhall as typed language and “FFI” into nix.
<simpson>
And to dovetail with pie_[bnc]'s thought, I would suggest that maybe it is appropriate for Nix to not gain those features.
<MichaelRaskin>
I am not sure Nix _requires_ that much RAM to evaluate, maybe transpiling the library functions into something JIT-table would keep evaluation reasonably efficient
<Profpatsch>
It’s a bit shitty in that it generates nix files that are put in the store and uses IfD, but that’s an implementation detail I guess
<Profpatsch>
you can put closures from dhall to nix and the other way around and that’s fantastic.
<pie_[bnc]>
simpson: not sure i agree in this specific case that nix is an ideal compile target
<niksnut>
MichaelRaskin: JITing might make evaluation faster but would make it use *more* memory
<MichaelRaskin>
I have an impression that some things are kept that would be cheaper to recompute with fast enough evaluation
<yorick>
MichaelRaskin: partial evaluation and deduplication would probably help more
<simpson>
pie_[bnc]: I will double-down with clearer words: Nix probably ought *not* to have macros or eDSLs.
<pie_[bnc]>
would be nice if at the end of the day we could have a good demo or something and choose if we want to go with the result.
<pie_[bnc]>
do things even need to be huge transitions
<infinisil>
Regarding the module system: I think the basic idea of it is actually pretty simple and straight forward: It's a "smart" attribute set: attribute paths can influence other attribute paths
<pie_[bnc]>
meh. I have to go get my glasses that I left at my friends house. o/
<infinisil>
In controlled ways
<pie_[bnc]>
maybe we really need a more core nixpkgs that we can have a bitmore agility with
<pie_[bnc]>
so many directions to go in...
* pie_[bnc]
is just wondering if there's things that could be done so that it's less of a big deal to figure this out
<MichaelRaskin>
Problem is: however core we go, we probably already need some Perl modules, some Pythone packages, some Rust, and preferably all that sould be cross-compilable
<pie_[bnc]>
musing; at the end of the day nix is just a compiler for attrsets isnt it?
<niksnut>
a compiler for a graph of .drvs
<pie_[bnc]>
does anyone have an associative array abstract machine yet? :P
<simpson>
IMO that's enough; that is a thing. So the next thing would be to write tools which produce (graphs of) .drvs.
<pie_[bnc]>
(side question; why indeed is nix so amenable to its domain of application)
<Profpatsch>
simpson: such as guix? :P
<MichaelRaskin>
pie_[bnc]: it has build isolation, and stdenv examples are bundled.
<simpson>
Profpatsch: like go2nix and friends.
<MichaelRaskin>
Basically, once you have functional abstractions, the rest will be built somehow
<adisbladis>
I find those tools universally terrible from a UX perspective.
<Profpatsch>
simpson: As I said, I consider nix expressions to be a perfectly fine compilation target
ixxie has quit [Ping timeout: 252 seconds]
<niksnut>
MichaelRaskin: but the goal is to get rid of functional abstractions...
<Profpatsch>
niksnut: hello functional, I’m dad
<niksnut>
although admittedly modules/configs are a kind of functions in disguise
<MichaelRaskin>
Is the goal to rid of _all_ Turing-complete abstractions?
<Profpatsch>
Can you emulate functions with let-bindings, imports and interpolations? Asking for a friend
<MichaelRaskin>
pie_[bnc]: if you want _asbtract_ machine, Abstract State Machine formalism might be one of examples
<adisbladis>
If that was an explicit goal I'd start looking into stop contributing to nixpkgs and move to Guix.
<MichaelRaskin>
Profpatsch: depends on what imports even are
<Profpatsch>
MichaelRaskin: nix imports in that case.
<Profpatsch>
Oh, and IFD!
<Profpatsch>
That must make it possible, surely.
<niksnut>
probably Turing-completeness isn't really the issue, any abstraction mechanism potentially makes it hard to understand other people's code
<MichaelRaskin>
Profpatsch: right now imports are usually returning functions anyway
<Profpatsch>
MichaelRaskin: But if niksnut took functions away from us, we would search for ways to circumvent that restriction.
<MichaelRaskin>
niksnut: well, without abstraction I expect Nix either to become a pure compilation target from a language that has some, or become unbearable, say for cross
<Profpatsch>
(see e.g. bazel, which is usually a dance of working around arbitrary restrictions)
<niksnut>
Profpatsch: yeah it's really hard to prevent functions (see: the C preprocessor or C++ templates)
<MichaelRaskin>
Profpatsch: but the imports might end up nerfed then
<Profpatsch>
niksnut: I’d like to mention the idea of restricted nix (with different file ending and less features) that I brought up in a discussion a while ago.
<bhipple>
The fact that everything in nix is a function is its killer feature IMO. I see some talk of trying to create config DSLs or whatever and it honestly terrifies me.
<MichaelRaskin>
adisbladis: I am not even sure we won't find enough people valuing cross-compilation, or TeXLive, to just fork and keep the good old stuff
<niksnut>
Profpatsch: .nicl?
<bhipple>
I think some of us may have spent too long with it to realize how nice it is and a dose of yaml+jina from the ansible/kubernetes/etc. world would do us well to remember how nice it is to have functions ;)
<Profpatsch>
Then you can take away all abstraction stuff (and make it fast) and still fall back on general recursion when calling these files.
<Profpatsch>
I wouldn’t take away functions though, just letrec.
<simpson>
bhipple: OTOH sometimes it is better to write a one-liner of jq and load JSON into Nix, rather than write twenty lines of Nix.
<Profpatsch>
And keep a non-recursive let.
<Profpatsch>
And disallow the y-combinator of course.
<niksnut>
bhipple: a yaml file is verbose, but anybody can understand it
<bhipple>
That's true, but I'm thinking of things like the overlay mechanism vs. trying to do something like that in a system like Anaconda (shudder)
<MichaelRaskin>
YAML is not something that can be understood out of context
<MichaelRaskin>
RubyGems found this the hard way
<bhipple>
niksnut: I agree. It is verbose, and anyone can understand it. Until you add Jinja templates, because it's not expressive enough to deduplicate. And then add Jinja macro helpers, because the simple for loops in the Jinja templates weren't enough. And then add libraries of Jinja macros, because you have them all over the place
<bhipple>
see: Kubernetes and Helm
<bhipple>
Now you're basically writing pseudo python libraries, but without all the tools
<pie_[bnc]>
right <MichaelRaskin> Basically, once you have functional abstractions, the rest will be built somehow
<simpson>
bhipple: Sure. But careful not to forget the main plague of Django templates before Jinja2, and Cheetah before Django: They were not fast.
<pie_[bnc]>
hence the vehemence :D<niksnut> MichaelRaskin: but the goal is to get rid of functional abstractions...
<simpson>
Folks are now generating C extension modules or CPython bytecode in order to gain more speed, and this tradition has continued in spite of JITs existing. The lesson I learned is that a JIT needs to not just exist, but needs to be part of the reference implementation.
<MichaelRaskin>
I would be all for a restructuring where hello/default.nix does not define a function but just a plain data structure, though
<simpson>
...But OTOH I don't know how far down Nix ought to scale.
<niksnut>
MichaelRaskin: +1
<pie_[bnc]>
nixpkgs has become large and now we need to learn to engineer in the large
<MichaelRaskin>
And if a typical package was a plain no-abstraction data structure, maybe we could JIT all the functions (as there would be few)
<MichaelRaskin>
Note I am not asking for gcc to be defined as a plain data structure
<Profpatsch>
that’s the hot stuff
<MichaelRaskin>
I would be already happy with hello
<pie_[bnc]>
MichaelRaskin: im not really making much of an argument here but the mkderivation argument set is that
<pie_[bnc]>
except
<pie_[bnc]>
yeah idk where that gets us, nevermind
<Profpatsch>
It even has types!
<niksnut>
Profpatsch: does it do recursion? :-)
<Profpatsch>
I would hope so!
<MichaelRaskin>
I wonder if it would be useful if simple dependency lists were by-name, and complicated cases with necessity of overrides could put a function (pkgs: …) for some build input
<niksnut>
that's something of a datapoint in this debate - for instance, somebody reading that file would have to expend brain cycles understanding what mergeShallowConcatLists does :-)
<MichaelRaskin>
Code is read more often than written, but patched without reading even more often!
<pie_[bnc]>
ehh we dont want people that dont know nix writing packaging or ci anyway ;P
<pie_[bnc]>
(jk)
mdash has joined #nixos-dev
<pie_[bnc]>
if its useful, put it in stdlib
<pie_[bnc]>
as a haskell neophyte, i have a hunch a problem with functional languages indeed the power of abstraction cripples you until yuo have some sense of the stdlib
<MichaelRaskin>
useful — to whom, and in _a_ stdlib or in _the_ stdlib
<pie_[bnc]>
because the content is in the concepts
<pie_[bnc]>
MichaelRaskin: batteries included *shrug*
<simpson>
If it's useful, write it. If it's *common*, move it up to shared modules. (If writing the useful things is too difficult, then add abstraction!)
<pie_[bnc]>
having it in stlibbeats everyone having their own implementaiton
<niksnut>
add abstraction and write an ICFP paper about it
<pie_[bnc]>
whoa whoa whoa hold up xD
<pie_[bnc]>
on a semi serious note i have a thread on the discourse somewhere calling for more of precisely that kind of design for nixpkgs application space
<pie_[bnc]>
im wont to have my head too far in the clouds though
<pie_[bnc]>
and to be fair, in this specific case, given that nix's fundamental data structure is attrsets, having a good vocabulary for it makes sense
<pie_[bnc]>
you dont *have* to use spaceFoldingHilbertIterationNSeqConvolutionMerge though
tokudan has quit [Remote host closed the connection]
tokudan has joined #nixos-dev
tokudan has quit [Remote host closed the connection]
tokudan has joined #nixos-dev
lovesegfault has quit [Quit: WeeChat 2.7.1]
zarel has quit [Ping timeout: 265 seconds]
zarel has joined #nixos-dev
Ox4A6F has quit [Read error: Connection timed out]
Ox4A6F has joined #nixos-dev
codyopel has quit [Ping timeout: 246 seconds]
codyopel has joined #nixos-dev
<Ericson2314>
hi
<Ericson2314>
simpson: yes I do agree on the compile target
orivej has quit [Ping timeout: 256 seconds]
<Ericson2314>
I also think it's a great away to put-off the politics of what the nix language shoud look like
<Ericson2314>
I am personally strongly pro functional + pro types rather than DSL straightjackets as a way to promote uniformity, but as a first step, let's just make the drv language as good as possible so both camps can demonstrate their views, and then we can decide for nixpkgs with more evidence
ekleog has quit [Quit: WeeChat 2.6]
ekleog has joined #nixos-dev
<pie_[bnc]>
if the drvs are good enough it doesnt matter anyway ;PPP
<pie_[bnc]>
just write raw drvs!!!
<Ericson2314>
pie_[bnc]: yeah I think this is the only way to make progress while we have this dispute