<gchristensen>
ran out of inodes. GC'd 1709973 store paths and still at 2% free inodes... so GCing more.
orivej has quit [Ping timeout: 245 seconds]
valebes has quit [Ping timeout: 244 seconds]
valebes has joined #nixos-dev
<disasm>
LnL: I'm looking at https://github.com/NixOS/nixpkgs/pull/61553. It looks pretty straight forward but hasn't been touched since May. Is this something we want to get in 19.09?
<LnL>
disasm: I have used it a few times so should be ready to go, but I agree that adding extra stuff like this to the cc-wrappers isn't great
ddima has quit [Quit: Lost terminal]
valebes has quit [Ping timeout: 245 seconds]
valebes has joined #nixos-dev
valebes has quit [Quit: Quit]
<marek>
LnL: about the gitlab service, the problem is it is kinda messy, the service itself will fail to run if it cannot reach and valide the connection to the master node
<marek>
LnL: and that service module is only about the service running with a specific configuration, I was thinking about what tests I could write but could not come up with any
<infinisil>
tilpner: Something we can't do with nested sets though is the callPackage thing
<infinisil>
As in, you'll have to use { stdenv }: stdenv.gccNoLibs { ... }
<infinisil>
Instead of { gccStdenvNoLibs }: gccStdenvNoLibs { ... }
<infinisil>
And this would change how things need to be overridden
<tilpner>
Right. It's unreasonable anyway, even if callPackage worked
<infinisil>
What's unreasonable
<infinisil>
?
<tilpner>
I imagine these sorts of breaking tree-wide changes will not be accepted into any used branch of nixpkgs
<tilpner>
They'll take forever to implement, and would need an RFC
<infinisil>
I think with some automated tools to make the job easier and a proper deprecation story (hello rfcs#33), it might just work (with an RFC of course)
<tilpner>
It's similar to the "central stdenv -> buildCPackage" and "separate config and dependency calls" proposals in terms of feasability
<infinisil>
tilpner: I sure do like those proposals
<infinisil>
tilpner: Oh actually regarding the "having related attrs together in all-packages", because all-packages.nix is just one overlay of many, we can just have a different overlay for manually-declared attrs
<infinisil>
Which won't be defined in the auto-calling all-packages.nix flat directory
<infinisil>
This also makes the transition story rather easy
<tilpner>
Lots of disjoint overlays (sorted!) sound good too, if that's what you mean
<tilpner>
Realistically, the best chance for a lot of these proposals is a "we need to re-do it anyway" event
<infinisil>
I'm thinking about moving all single packages from all-packages.nix to a flat directory tree that gets turned into an overlay by autocalling all files
<tilpner>
I don't know the status on eelcos new Nix concept (the mystery gist thing), but if it materialises, that might be such an event
<tilpner>
But I don't think I could push through my "separate config and dependency calls" before that, so...
<infinisil>
tilpner: Do you have a link for that?
<tilpner>
No, it's just an issue I ran into, I have no proposal for it
<infinisil>
I see, I'm wondering how it would be implemented
<tilpner>
You could™ just change the arity of each "package" from 1 to 2
<tilpner>
But that felt like a lot of work, so I stopped considering it
<tilpner>
:/
<infinisil>
And have callPackage call both arguments?
<tilpner>
The problem is that callPackage auto-filled configuration arguments, which are not meant to be auto-filled from the dependency set
<tilpner>
They could be auto-filled from one of the configuration sets we have
<infinisil>
Hm yeah
<tilpner>
It doesn't really matter how they're separated, it could be a nested attrset with helpers
<infinisil>
How about moving the configuration inside the derivation
<tilpner>
But then how do you override it?
<infinisil>
something like `passthru.modifyConfig = newConfig: ...`
<infinisil>
Which produces a new derivation but with a changed config
<tilpner>
Maybe, if we can come up with functions that abstract over this
<tilpner>
We wouldn't want every configurable package to implement it slightly differently
<tilpner>
And even if it works, we'd be adding yet another override mechanism
<infinisil>
Probably just a `{ stdenv, makeConfigurable }: makeConfigurable ({ enableFoo ? true }: stdenv.mkDerivation { ... })` would work
<infinisil>
s/just//
<infinisil>
Yeah we do have a bunch of override mechanisms already..
<tilpner>
It still sounds like a good approach, and much more feasible to implement
<infinisil>
Tbh ideally we'd be using the module system for configurations
<tilpner>
Ehh
<infinisil>
This would give us type checking, documentation, and easy integration into nixos modules
<infinisil>
Related is #56227
<tilpner>
The module system brings a lot, but also has the "you can't have multiple of something unless you specfically allow that" and performance issues
<tilpner>
Oh, maybe I misunderstood what you meant
<infinisil>
tilpner: How do you mean "you can't have multiple"?
<infinisil>
It sure would be slower than just a function though
<infinisil>
But ignoring that, using the module system (or anything similar like the nixlang++ thing) would be nice
<tilpner>
I thought you suggested adding modules for packages, not shared things that could be configurable among all packages
<tilpner>
And then you would get the nixpkgs equivalent of "how do I run this service twice with different settings without duplication or altering the module"
<infinisil>
Ah, I did suggest the former, but the latter is good for global configurations
<infinisil>
tilpner: I don't think that's a problem with packages, because all you can configure is some options for how to build them and you'd get a single derivation as a result, which you can put multiple times wherever you want
<tilpner>
Oh!
<infinisil>
NixOS has the service duplication problem because all options are global
<tilpner>
I assumed there would be a single call to lib.evalModules
<tilpner>
But there isn't, right?
<infinisil>
If the module system were used for package config's then yeah we'd call lib.evalModules
<tilpner>
So callPackage would now call evalModules at some point?
<tilpner>
That would be the correct system, if not for performance :/
<infinisil>
Yeah
<infinisil>
I'm actually not sure how bad it would be, because there would only be 2 modules, one for defining the options, and one that the user supplies
<infinisil>
(It won't be faster than what we have now for sure though)
<tilpner>
Yeah, callPackage doesn't do a lot
<tilpner>
And considering how slow evals can be already, it's a hard sell for purity
<infinisil>
tilpner: Oh another idea: How about we use the `config` set for all configuration, so e.g. the configuration for `pkgs.hello` would be in `config.pkgs.hello`
<infinisil>
niksnut: Yeah we linked to it earlier too, I like the idea, but it will be hard to switch to it given nixpkgs size
<tilpner>
We were talking about that too. Is that more of a 2 year, or a 5 year plan?
<infinisil>
Maybe we should adapt semver for nixpkgs :P
<infinisil>
(so we can signal such a change being incompatible)
<tilpner>
Ehh
<niksnut>
not semver, just create a new language / nixpkgs-ng repo
<tilpner>
It's an option to not update e.g. your config file library for a while
<tilpner>
It's not really an option to not update nixpkgs for that same while
<tilpner>
That's what I'm hoping for, niksnut. For nixpkgs-ng, there can be attempts to address all the other shortcomings of the current structure, for little cost (because everything needs to be ported anyway)
<infinisil>
We could use Rust for a new Nix version :D
<niksnut>
yes
<clever>
one thing ive thought of, could you transpile nix to c?
<clever>
or just directly write a nix to llvm-ir compiler, and let llvm finish the job to native
<tilpner>
For JIT compilation, or by Hydra?
<clever>
tilpner: to make all nix evals faster
<tilpner>
So every eval generates and compiles C?
<clever>
`nix-channel --update` (or its replacement) would just compile the new nixpkgs (binary cache also)
<niksnut>
that's the guix approach
<niksnut>
but it adds a lot of complexity
<niksnut>
for nix-ng, it would be better to get rid of things that make evaluation slow
<niksnut>
(laziness, functions, ...)
<clever>
ah, ive not looked into how guix does anything, i use too much closed-source junk
<infinisil>
niksnut: Huh, nix without lazyness or functions? How does that work?
<niksnut>
this would also improve maintainability of packages (you don't have to deal with whatever crazy abstractions other people come up with)
<tilpner>
What does guix do exactly? Compile guile to something?
<niksnut>
infinisil: hopefully modules is enough (basically modules are a templating mechanism)
<niksnut>
I mean, you need *some* laziness
<__monty__>
How does laziness in particular affect performance? I'm even more skeptical about functions affecting performance.
<infinisil>
Hm..
<tilpner>
I guess you could serialise a fully-evaled nixpkgs to disk, but that's probably not going to help
<tilpner>
So I'm curious what optimisation guix does
<niksnut>
alternatively, it might help to ban recursion rather than functions in general
<niksnut>
and the language should be such that a full garbage collector is unnecessary
<__monty__>
So no more folds, map, etc.?
<niksnut>
right
<infinisil>
niksnut: Ah so it's not that you can't abstract with functions anymore, but rather it works based on configuration/modules which just encode dependencies between values (therefore if you change a value representing an argument, the value representing the result can change)
<infinisil>
That sound about right?
<niksnut>
nix is a DSL, so those functions shouldn't be written *in* the language anyway
<__monty__>
Sounds like you want to replace nix with YAML...
<niksnut>
YAML is a bit too far on the other side of the expressiveness spectrum
<clever>
i think having some memoization in the modules framework might help
<clever>
ive found memory usage and eval time to get horid, if you have 110 evals of nixos in a single eval
<clever>
(10 nixos containers each, on 10 nixops machines)
<niksnut>
there is builtins.memoise, but for NixOS modules you probably want incremental evaluation
<yorick>
I'm mostly using nix as a dsl that reads other package manager's lockfiles these days. not having functions would be... annoying
<clever>
if it could somehow reuse computation of the `options` tree between machines and containers
<infinisil>
I actually think you could still get functions back by using modules underneath
<niksnut>
yeah, it's really hard to prevent sneaky turing completeness
<qyliss>
If functions should be written in other languages, that's killing the chances of there ever being multiple implementations of Nix.
<niksnut>
e.g. the C preprocessor is turing complete
<hyperfekt>
niksnut: In the nixlang++ proposal, under the heading 'incoude mechanism', the option of using extends syntax to get the builder is weighed - how would one override the builder in that case? Would I extend the package definition and override the builder option?
<qyliss>
Would make it way more difficult for people to learn and meaningfully contribute too
<clever>
niksnut: did builtins.memoise ever get into master?
<qyliss>
There'd be no gradual progression of writing a single package to working on shared abstractions.
<niksnut>
clever: no
<niksnut>
qyliss: it makes it easies to contribute, because every nix package would have the same structure
<qyliss>
Makes it easy to contribute simple package changes, sure
<qyliss>
Doesn't make it easy to contribute to more complex stuff like language builders
<niksnut>
you can factor out commonality into config modules
<infinisil>
qyliss: Yeah ^^ think writing NixOS modules with options like `patches = mkOption { ... }`
<__monty__>
niksnut: qyliss is rightly saying that anything that's "been done before" would be easy but anything past that would require mucking about in nix's C/perl/whatever code.
<infinisil>
But fun small/efficient functions i guess such modules would be a bit overkill
<infinisil>
s/fun/for
<infinisil>
(unless the overhead can be optimized away, which doesn't sound entirely unreasonable)
<__monty__>
niksnut: Also, I guess nothing can stop you from writing an EDSL in nix that'd implement your vision, as a POC. If it were the other way 'round, that'd be impossible : >
<niksnut>
that's kind of the point though
<niksnut>
the fact that you *can* write an EDSL in Nix means it's too powerful
<yorick>
clever: we're also seeing that particular slowness, yeah
<clever>
niksnut: oh, i have an idea, right now, import will cache the top-level value, but in a lot of nixos modules, that is just { config, lib, ... }: stuff, so if you import the module twice, you still have to run the function twice
<clever>
niksnut: what if import automatically called memoise on all functions it returns?
<niksnut>
hyperfekt: probably you should ignore that section, since having an all-packages.nix is old-style thinking
<clever>
although, the config tree likely wont match 100% ever, so that may not help
<clever>
and it would force all of config to not be lazy?
<niksnut>
probably it would be better to have bazel-style dependencies
<niksnut>
e.g. deps = [ ":libfoo" ];
<niksnut>
clever: right, the issue is that config depends on everything
ris has joined #nixos-dev
<niksnut>
that's why you'd need incremental evaluation
<niksnut>
e.g. when you have a 100 NixOS machines that have an identical set of modules except for their configuration.nix
<qyliss>
I think that actually, I can withdraw my criticism of not having functions, because for most cases where complex functions might be useful, it's probably better to just generate Nix expressions.
<niksnut>
so you'd calculate the configuration for the base modules and then incrementally apply configuration.nix
<qyliss>
(e.g. there's no reason to parse a lockfile in Nix, when you can have a tool like Carnix or whatever that generates a Nix version)
<niksnut>
well, generation is worse than functions
<niksnut>
because now users can't modify a nix file anymore and need to figure out how to run the generator
<qyliss>
How do you envision, say, loading dependencies from a Gemfile?
<niksnut>
probably this should be a built-in function
<niksnut>
like builtins.fromTOML, but doing a bit more
<qyliss>
You're going to have to grow a hell of a lot of built-in functions
<niksnut>
yes
<qyliss>
Note that in this case, it would involve actually running a Ruby interpreter
<qyliss>
Is Nix going to link against or shell out to Ruby for this?
<niksnut>
ideally not
<qyliss>
The Gemfile could also be arbitarily slow / memory-hungry, etc.
<qyliss>
That doesn't matter if you generate Nix from it.
<qyliss>
It matters a lot if you evaluate it every time.
<puck>
you could maybe have nix call out a derivation that gets run at evaluation time? (iirc this can already be done?)
<qyliss>
(Gemfiles are also totally impure, of course)
<yorick>
puck: yeah, import-from-derivation works
<yorick>
but hydra disallows it
<puck>
weird
<qyliss>
rightly so
<yorick>
it's idiologically shady
<puck>
i guess
<yorick>
e*, also, it causes builds during evaluation, which is the worst
<niksnut>
IFD is a good point, in a more constrained language it would probably be easier to have an IFD-like feature because it would only be used in certain places
<puck>
yeah, that's the disadvantage
<qyliss>
My experience with language package managers in Nixpkgs makes me think that the only sensible way to deal with them is to generate some well-known, pure code that can be used at evaluation time.
<qyliss>
Because source files often don't purely express their dependencies, and depend on the network and stuff
<qyliss>
So without generating our own, basically lockfiles, we can't have purity.
<puck>
alternatively you could have a "core" lib that is still written in fully-featured nix, and have the derivations use the limited nix DSL thing
<qyliss>
And once we're doing that we might as well be generating Nix that's easy to evaluate.
<yorick>
qyliss: reading lockfiles works very well for us for npm, yarn, mix, stack already
<qyliss>
What if the lockfiles don't contain checksums?
<qyliss>
(Cargo until recently didn't have this, Bundler doesn't and I'd be shocked if it ever did)
<yorick>
package managers are moving in a direction that contains checksums
<qyliss>
We can't rely on that, though
<qyliss>
If we do rely on it, then it becomes impossible to _ever_ integrate a language that doesn't do it.
<qyliss>
There would just be no way to use programs written in that language on NixOS, unless we generate code.
<qyliss>
And that would be an awful experience.
<qyliss>
yorick: in what context are you reading these lockfiles btw? nodePackages certainly doesn't, IIRC?
<clever>
qyliss: if you assume the lock file is reproducible without hashes, then you could just fetch what it defines, in a fixed-output drv, and then hash the entire mess
<clever>
qyliss: i believe cargo can already do that on nixpkgs
<qyliss>
That's a reproducibility nightmare that we should be moving _away_ from
<clever>
but you dont have per-dep caching of source, and if any dep changes, the entire thing is broken
<qyliss>
Because it's never reproducible without hashes
orivej has quit [Ping timeout: 246 seconds]
<clever>
it assumes they are at least using git revs
<qyliss>
Now all of a sudden arbitrary dependency upgrades have the potential to break hashes.
<clever>
yeah, its better to have one fixed-output per dep, and to use an existing primitive like fetchurl or github
<qyliss>
yep
<clever>
a few months ago, i tried writing a gclient2nix, and it was turning into a nightmare
<qyliss>
But how do you do that based on a lockfile that doesn't contain hashes?
<qyliss>
You either manually find all the hashes, or you automate it and generate code. You end up with the same thing.
<clever>
basically, every repo defines its own deps, and you have to recursively look in each dep for a file defining more deps
<qyliss>
I don't think there's another way.
<clever>
and there are post-fetch hooks, that i think get ran after you fetch everything, that can mutate the source
orivej has joined #nixos-dev
<clever>
and the whole tool assumes you keep the .git in every single dep
<clever>
after a successful run, it creates a 30gig mess of source
<clever>
and running it in a pure sandbox is difficult, because the post-fetch tools re-fetch an unpatched gclient
<qyliss>
We're still seeing fallout from buildRustPackage, btw, months after I thought I'd fixed them all.
<qyliss>
This month there was another checksum that had to be updated but had gone unnoticed because of caching.
<qyliss>
If it's possible to support every language package manager purely in Nix, without using custom fixed output derivations, that would be amazing.
orivej has quit [Ping timeout: 245 seconds]
<qyliss>
But I don't think it is. And if it's not, we're going to be stuck generating code.
<edef>
Mic92: i'm working on reworking the git-series packaging
<edef>
Mic92: right now i have the version in nixpkgs segfaulting
<edef>
Mic92: it does not segfault with non-vendored libgit2
<Mic92>
edef: I see.
<edef>
Mic92: opening a PR to git-series for the OpenSSL upgrade seems pointless given how long your PR has languished
<edef>
Mic92: and i assume Josh is more likely to take emailed patches
<Mic92>
edef: I forgot, that I have opened a PR to this project.
<Mic92>
Ah, two years ago.
elvishjerricco has quit [Ping timeout: 252 seconds]
evanjs has quit [Ping timeout: 245 seconds]
<Mic92>
... no suprise.
<edef>
:D:
<edef>
so i'm hoping the email helps there
<edef>
because this situation is a little silly
lightbulbjim has quit [Ping timeout: 252 seconds]
c00w has quit [Ping timeout: 252 seconds]
elvishjerricco has joined #nixos-dev
tv has quit [Ping timeout: 245 seconds]
c00w has joined #nixos-dev
lightbulbjim has joined #nixos-dev
lassulus has quit [Ping timeout: 258 seconds]
zimbatm has quit [Ping timeout: 252 seconds]
georgyo_ has quit [Ping timeout: 250 seconds]
georgyo_ has joined #nixos-dev
zimbatm has joined #nixos-dev
<thoughtpolice>
emily: Hm, I think #67827 might have timed out on Darwin too, but I'm not sure? The build report has no real info it seems which is bizarre.
<samueldr>
thoughtpolice: something failed to build, a dependency to both glasgow and migen
<samueldr>
when dependencies are built, the log is not split per derivation with the way ofborg is built
<thoughtpolice>
I bet I know why
<samueldr>
searching for error: often helps, but not here, though error (without colon)
<samueldr>
makeinfo: warning: error loading ./Config: Can't locate ./Config at /nix/store/8k7bzklf54s54xlpr9zmaryjy3077k07-texinfo-6.5/bin/makeinfo line 395.
<thoughtpolice>
Yeah, nextpnr. Dunno how I missed that. I wonder why aarch64 passed.
<thoughtpolice>
Maybe I got a dinky x86 machine but one of the good newer ARM machines?
<samueldr>
it always runs on a many-cored ARM server for aarch64 builds
<samueldr>
so yeah, x86 builds can timeout while the aarch64 build succeeds on the ofborg infra
<thoughtpolice>
What a world to live in. Well it's probably all fine, then.
<thoughtpolice>
We could pick a few obvious changes in from that PR and let Hydra churn them out for borg to use. But I think it's probably just as well to merge it all.
<samueldr>
it didn't time out on darwin though
<samueldr>
(I don't know if it should have succeeded or not)
<samueldr>
it may as well be an existing build issue in a dependency
<thoughtpolice>
Oh interesting, nextpnr failed there too, but for another reason
<thoughtpolice>
Ah, that's an existing failure we didn't catch.
<thoughtpolice>
Hydra hasn't been building it either, must have missed that before.
<thoughtpolice>
Trellis is fine, however.
<samueldr>
right, if it's not a regression on Darwin, but existing the PR (imo) can move on
<samueldr>
(ideally a fix be done in another :))
<thoughtpolice>
Right
evanjs has joined #nixos-dev
evanjs has quit [Ping timeout: 245 seconds]
evanjs has joined #nixos-dev
tv has joined #nixos-dev
lassulus has joined #nixos-dev
__monty__ has quit [Quit: leaving]
<samueldr>
adisbladis: re: 67355 it's good to be merged?
<adisbladis>
samueldr: I just pushed the changelog entry a few minutes ago, was waiting for borg checks before pushing the button :)
<samueldr>
good, then carry on :)
<samueldr>
it was since you pushed the changelog entry that I asked
<ashkitten>
if i'm specifying a specific version of a dependency (openssl_1_0_2) for a package, should i do that in the package's default.nix or in all-packages.nix as an argument to callPackage
<infinisil>
ashkitten: I wanna say the former, that's what I've been doing and makes sense to do for the future if we want to get rid of the all-packages.nix thing, but the latter has better behavior for overriding dependencies
<ashkitten>
sure
<ashkitten>
i'm gonna go ahead with default.nix
abbradar has quit [Ping timeout: 248 seconds]
abbradar has joined #nixos-dev
<worldofpeace>
adisbladis: In the case of different display managers we do actually influence what the default session is. In the case of lightdm we set user-session= with what's detected as defaultSessionName.
globin has quit [Ping timeout: 252 seconds]
globin has joined #nixos-dev
<worldofpeace>
Do you recall which combinations resulted in this?
domenkozar[m] has quit [Ping timeout: 252 seconds]