<gchristensen>
samueldr: okay I think I'm bringing up a big-parallel-only machine
<samueldr>
nice
<samueldr>
and the others would have lost their big-parallel tag, right?
<gchristensen>
yeah
<samueldr>
let's see in 2-3 days what the state of all this is :)
<gchristensen>
okay there machines have big-parallel for aarch64
<gchristensen>
aa59f714.packethost.net 6e0512f0.packethost.net 89aa8aae.packethost.net the first two do 2 builds at a time, the third does 30 at a time.
<gchristensen>
the third one I'm fixing to not have big-parallel now
<gchristensen>
okay the third one is no longer big-parallel
<gchristensen>
samueldr: the next step, though, is properly tagging software as big parallel ... :)
<samueldr>
yep
<samueldr>
I probably can use the hydra scraping stuff I already have to figure out what timed out on aarch64 in a few evals
<gchristensen>
cool
drakonis has quit [Quit: WeeChat 2.7.1]
v0|d has joined #nixos-dev
erictapen has quit [Ping timeout: 264 seconds]
<ryantm>
I am looking for testers/guinea pigs for new single-package update features in nixpkgs-update and the instructions at https://github.com/ryantm/nixpkgs-update
<ryantm>
Please send me your feedback it is still rough around the edges!
<cole-h>
What does "based on information from ryantm" mean in step 2?
<ryantm>
cole-h: I removed that, it is just an optional parameter that changes the PR message, which doesn't matter in this basic case.
<ryantm>
Thanks for looking at it!
<cole-h>
Cool, was confused :)
<cole-h>
First bug report: using the exact example command from a checkout gives me "Couldn't find derivation file."
lovesegfault has joined #nixos-dev
<ryantm>
Maybe your checkout of nixpkgs doesn't have postman v 7.20.0?
<ryantm>
needs better error message!
<cole-h>
It does -- I `git log --grep="postman"` and it shows your bot's PR merged
<cole-h>
Also, viewing the derivation shows it's at 7.20.0
<cole-h>
Ah, I see my problem
<cole-h>
I have a customized NIX_PATH
<cole-h>
"error: file 'cole/nix/sources.nix' was not found in the Nix search path" shows up in the strace
<cole-h>
Where `NIX_PATH=cole=~/.config/nixpkgs:nixpgs=~/workspace/vcs/nixpkgs`
<ryantm>
It should be setting NIX_PATH to the current directory instead of that though.
<cole-h>
Right, that's the problem: I have an overlay that refers to this custom path. I bet if I were to remove all instances of that, things would be hunky-dory
<cole-h>
OK, so the real problem was this:
<cole-h>
1) I have things that refer to <cole/....> in my overlays/etc; 2) I set nixpkgs-overlays=/var/empty using direnv in my nixpkgs checkout so that I don't conflate my overlayed packages with the ones built from the checkout
<cole-h>
(AKA not on you)
<ryantm>
Maybe it would be better to put nixpkgs= on the front of the NIX_PATH instead of overriding it.
andi- has quit [Ping timeout: 272 seconds]
<cole-h>
Aha, there we go. Removing my NIX_PATH stuff cleaned it up. Working flawlessly now!
<cole-h>
Well, at least not barfing, yet
<cole-h>
ryantm: Maybe the "updating vulnerability database" step of a single update could be optional behind a flag or something?
andi- has joined #nixos-dev
<cole-h>
ryantm: Also the "build yourself" stuff appears to be hardcoded to r-ryantm's nixpkgs. No suggestions on what should be done with that, though.
<ryantm>
Yeah, probably should put CVE reporting behind a flag just because the initialization is so slow. If you run it again though, it is very fast.
<ryantm>
Maybe if it just said that it was slow that would cover it. It's a nice feature to have.
<cole-h>
416M of database might suck for those with slow download speeds
<gchristensen>
jtojnar: not sure if you saw it, but only 2 machines are big-parallel for aarch64 now, and they both have lots o cores and run 2 jobs at a time, so hopefully that clears up some of the slowness / brittleness. maybe we should do the same for kvm needing jobs?
<jtojnar>
yeah, I try to read everything 🤣️
<jtojnar>
is there a way to see how many jobs are before that one in queue?
<clever>
/nix/store/5zlkyh74w48n3rgd8fd6paxyxdh16zia-nixos-system-zookeeper1-20.03pre-git/sw/bin/ls: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /nix/store/6m2k8kx8h216jlx9dg3lp4m90bz05yck-glibc-2.30/lib/ld-linux-x86-64.so.2, for GNU/Linux 2.6.32, not stripped
<clever>
and yep, its an x86 nixos build
<gchristensen>
wtf
<clever>
kafka # /nix/store/c5695j6l8ligwzxq3c5v3ayphp9g0al6-apache-kafka-2.12-2.3.1/bin/.kafka-topics.sh-wrapped: line 17: /nix/store/qcbf8dv0kcgs5lkfbs4f2vwszqsfsfbp-coreutils-8.31/bin/dirname: cannot execute binary file: Exec format error
<clever>
which then runs a non-x86 binary??
<clever>
/nix/store/qcbf8dv0kcgs5lkfbs4f2vwszqsfsfbp-coreutils-8.31/bin/dirname: ELF 64-bit LSB executable, ARM aarch64, version 1 (SYSV), dynamically linked, interpreter /nix/store/2vn35phq7d4q3q6pi6jipxy1h1x3346s-glibc-2.30/lib/ld-linux-aarch64.so.1, for GNU/Linux 2.6.32, not stripped
<clever>
yep
<clever>
its an x86 vm, running x86 nixos, testing an aarch64 binary
<clever>
error: a 'aarch64-linux' with features {} is required to build '/nix/store/5pxx80knsxki1w3k13ip0aw5wwrj2mpv-python3-3.7.7-env.drv', but I am a 'x86_64-linux' with features {benchmark, big-parallel, kvm, nixos-test}
<{^_^}>
#84620 (by cleverca22, 8 seconds ago, open): nixos: kafka test: fix building for other arches
FRidh2 has joined #nixos-dev
FRidh has quit [Ping timeout: 265 seconds]
<worldofpeace>
gchristensen: are there docs on maintainers-list.nix in the nixpkgs manual?
johnny101m has joined #nixos-dev
<worldofpeace>
other than just stuff about the meta attr, which I see in there
FRidh2 has quit [Remote host closed the connection]
johnny101m2 has quit [Ping timeout: 265 seconds]
<gchristensen>
worldofpeace: I don't think so
<gchristensen>
but there shoul be :)
<gchristensen>
clever: eww :)
cole-h has joined #nixos-dev
<worldofpeace>
Jan Tojnar: I've noticed with 3.36 that log out isn't available with my single user
<jtojnar>
worldofpeace that is the correct behaviour
<jtojnar>
though I have a bug where sometimes it is available
<jtojnar>
and another bug that acountservice reports nonsense
<worldofpeace>
Yeah, I've now seen that bug
<worldofpeace>
maybe the point release will have it fixed
<worldofpeace>
Though I do always get log-out in the x11 session
<gchristensen>
worldofpeace: do you have a plan on merging staging-20.03 to release-20.03? I'm wondering if we should do it ~today, to make tomorrow's meeting more successful
<domenkozar[m]>
dunno, github and my editor preview these
<gchristensen>
cool, is there a way to hint to github or whatever that they're python?
<lovesegfault>
shows Core, date , hash, libxml, pcre, Phar, Reflectionm SPL, standard, and XML
<etu>
lovesegfault: This works in my checkout: "nix-build --expr 'with import ./. {}; php74.withExtensions (e: with e; [ curl json ])'"
<gchristensen>
or rather, hint to github/etc. that they're nix
<lovesegfault>
etu: `php -m` still comes up without the modules for me
<ma27[m]>
etu: can confirm that, `nix-build -E` is fine, but `nix-shell -p` isn't
<lovesegfault>
Maybe my way of checking is wrong and php -m doesn't do what I expect?
<etu>
lovesegfault: since it's a build you have to run "./result/bin/php -m"
<etu>
ma27[m]: hmmm, any idea why?
<ma27[m]>
lovesegfault: I'm not (yet) sure why, but it works fine if you don't use `nix-shell -p` :)
<lovesegfault>
Oh, right, that did work!
<ma27[m]>
no, unfortunately
* etu
has no idea either :/
<lovesegfault>
I came across this because arcanist is now broken for me
<ma27[m]>
but `php` in the `nix-shell -p` expr isn't wrapped
<worldofpeace>
Jan Tojnar: I've gotten three elementary ecosystem devs to consider nix, two of which have expressions in their repos :D
<lovesegfault>
let me see if I can come up with a fix
<etu>
ma27[m]: hmm, it is since it get all the default modules?
<jtojnar>
worldofpeace++++
<etu>
ma27[m]: oh, I ran from another nixpkgs
<MichaelRaskin>
worldofpeace: so the plan for a friendly NixOS desktop edition is to convert Elementary to a NixOS edition?
<etu>
ma27[m]: yep, php74 in nix-shell seems to be php74base
<ma27[m]>
I checked with `-I nixpkgs=$(pwd)` though
<ma27[m]>
or is this not enough as well?
<etu>
env NIX_PATH=nixpkgs=./. nix-shell -p php74
<etu>
I did reproduce it with that
<worldofpeace>
MichaelRaskin: haha, I don't think elementaryOS could stand our 6 month releases
<lovesegfault>
Eh, what's the right way to specify php with extensions in buildInputs/wrapping?
<etu>
lovesegfault: withExtensions
<worldofpeace>
MichaelRaskin: but a pantheon NixOS edition would work well
<domenkozar[m]>
gchristensen: don
<domenkozar[m]>
e
<gchristensen>
that's the good stuff. thanks!
<etu>
lovesegfault: Should work to put this in a buildInputs list: (php.withExtensions (e: with e; [ curl json ]))
<worldofpeace>
MichaelRaskin: nixos could be user friendly if we had GUI's, then we could actually compete
<MichaelRaskin>
Last time a person who wanted a configuration GUI showed up, it turned out that their needs would also be a challenge to support in GUI, though
<lovesegfault>
the wrapper is created correctly, but in post-install fixup it patches the interpreter in all the php files to a php without the extensions
<etu>
ma27[m]: I don't get how nix-shell picks up the wrong attribute though :/
<lovesegfault>
Where can I read the default postInstall for php?
<ma27[m]>
me neither.... currently busy with other stuff though.... unless you find a solution, I can take a look tomorrow :)
<worldofpeace>
MichaelRaskin: Yeah, the issue is ongoing configuration, using it when it's installed. But a configuration editor program is basically what's needed
<manveru>
haven't done much on it in the meanwhile... waiting for some UX improvments in calamares
<worldofpeace>
manveru: thank you. what was the motivation for calamares? last time I checked, that one is easily transplantable
<worldofpeace>
* for using
<manveru>
mainly because they have a nice partitioning step
<manveru>
it's the part i hate most about installing nixos :)
<worldofpeace>
manveru: I have to agree, partitioning in there is pretty nice
<manveru>
but i don't really have anything to try this installer on... it works in qemu but that doesn't really say much
<manveru>
so for now it's on ice until someone is interested enough in helping to contact :)
<worldofpeace>
manveru: haha, I see what you mean. c++ hacking required?
<worldofpeace>
I'd like to help with something like this for 20.09
<manveru>
no, it just has some python
<manveru>
you can write plugins in both c++ and python, so it hasn't been too hard
<manveru>
but the package|preset|de|features (however you wanna call it) selection step just didn't allow obvious multiple selections, you have to use shift+click and there's no way to indicate that
<manveru>
no idea if that's still the case... should probably check :)
<worldofpeace>
manveru: cool, python is fine with me. I see the readme says "PythonQt deprecated", I guess that's not for plugins
<manveru>
not sure, haven't used that part
<manveru>
i think it was for rendering with python
<manveru>
just gathering all the selections from the dialogs and writing them into a nix file
<gchristensen>
w.r.t. https://github.com/NixOS/nixops/pull/1277 I'm looking for reviews, but it doesn't have to be super nit-picky but like ... do I sound coherent? :P
<{^_^}>
nixops#1277 (by grahamc, 52 minutes ago, open): Docs: Update authoring.rst after a few more implementations.
<gchristensen>
that means the queue runner restarted
<lovesegfault>
running steps is all over the place over the last 7 days
zarel has joined #nixos-dev
<gchristensen>
yeah thats normal
<gchristensen>
afaik, once it restarts, it has to load up all the builds to determine an ordering for the steps. so over the next bit, it'll keep loading jobs and do a better job scheduling
<lovesegfault>
Ah, I see
<samueldr>
gchristensen: that "runnable jobs per machine type" graphs is the one where we want to see downward trends, to know it's not actually getting backlogged, right?
<lovesegfault>
o hydra.nixos.org it shows >200k queued builds
<gchristensen>
yep
<gchristensen>
samueldr: exactly
<samueldr>
that darwin hump though
<gchristensen>
but after a queue runner restart, it takes a bit to let that graph settle out to be able to get meaningful data from it
<gchristensen>
all of these numbers are unreliable until the queue runner has loaded everything
<lovesegfault>
Why was it restarted?
tv has joined #nixos-dev
<gchristensen>
presumably because it was changed
<lovesegfault>
Ah
<gchristensen>
or some other operational need mandated it
<mkg20001>
worldofpeace: manveru : The calamares installer looks pretty interesting, the elementary one sadly is still wip. I'll take a look at both later soon.
<mkg20001>
What I need for the installer is some simple fake DE like ubiquity that boots quickly, elementary's installer and ubiquity has that one. Additionally the calamares feature selection just didn't quite cut it.
<manveru>
mkg20001: i use i3 for the wm, just starts calamares in fullscreen but you can open terminals and stuff as needed (if you know the keyboard combos :P)
<worldofpeace>
mkg20001: I think ubiquity and elementary's installer just ship a session (xsession or gnome-session)
<mkg20001>
worldofpeace: I've looked through ubiquity's code and seen some cli that sets the background and some really basic stuff that uses python-pam directly. Elementary's seems to boot a gnome session
<worldofpeace>
mkg20001: you could obviously tell I perfer the latter, but I guess we need something decoupled from gnome
<gchristensen>
jtojnar: nice work in 83c2ad80ca8c
<mkg20001>
The i3 solution sounds good. I guess something like that is good.
<worldofpeace>
(Well, to be clear there's two things here. Installation in a live environment and an installer session. ) I believe the iso would have a grub item to select the latter
<gchristensen>
oh hrm I'm seeing actually this was just a reformat
<gchristensen>
I really liked the wording around thumbs-down's and stuff :)
<manveru>
mkg20001: can't say i'm in love with adding a json layer on top of nixos config... now you have to learn both :P
<mkg20001>
manveru: It's not really adding it on top, it's something that can be optionally used and makes it easier for scripts to mess with the config. There's a key-value module which allows setting keys like in the config. The .json will be useful later when building a GUI to configure nixOS.
tv has quit [Ping timeout: 256 seconds]
<manveru>
mkg20001: yeah, i guess it's the best thing we have until someone builds a jq equivalent for nix...
tv has joined #nixos-dev
<mkg20001>
manveru: Since nix is a programming language and not just a simple format, that would likely involve parsing the .nix file, messing with the AST and then turning it into a string again, if we're talking about an utility that allows editing any value from any nix file. That's more than a tad more complicated. Unless there's something I missed.
<manveru>
no, that's what i mean :)
<manveru>
it's possible in theory... but probably not gonna happen
<jtojnar>
worldofpeace reviewing the session settings now
<mkg20001>
manveru: Also another reason for why conf-tool exists in the first place was that there is stuff like "adding a user" that seems like it's just "adding the user" but then there's a bunch of stuff like adding the user to a few groups like audio, video, etc to make things "just work ™︎", but unsure if that's something for the module system, so we could abstract that away using the module system and then have the tool ju
<mkg20001>
modify the keys that use this system
<manveru>
mkg20001: so yeah, i'd be ok with adding this conf-tool and just generating json from the installer, it's a lot nicer than the nix templating i have right now... but i really would like an option to "eject" the config for power users after installation
<mkg20001>
manveru: conf-tool builds .nix files after every change, just removing the .json and edding those directly should do the trick
<manveru>
sounds good then :)
<mkg20001>
editing*
<mkg20001>
Unsure though if they should be merged. Right now every module creates it's own .nix file, but they could be merged aswell intro one conf-tool.nix. What do you think manveru ?
<mkg20001>
If you want to try the tool: Add meros channel (https://nix.mercode.org/dev/meros/) and run meros.conf-tool. It only modifies conf-tool/ and conf-tool.json in /etc/nixos unless you run "init" (init is not required for any of the commands to work, it's mainly for creating the default.nix and stuff)
<Irenes[m]>
hey, so I have a question
<Irenes[m]>
right now, the way nixpkgs is implemented, imports cannot depend on config
<Irenes[m]>
if you try to, you get an infinite loop
<Irenes[m]>
was that a principled decision, or just the way things happen to be?
<Irenes[m]>
samueldr mentioned that infinisil might have context
<Irenes[m]>
since options can depend on config, you can get essentially the same end result by importing everything you would even think about importing but making the contents of each file conditional
<Irenes[m]>
but that's tedious and requires a lot of boilerplate
<Irenes[m]>
and it results in having to load all that code even when it's not needed, which may have performance implications
<Irenes[m]>
also the follow-up question is that if I were to look into changing it, is that a thing that people would want?
<infinisil>
I'm pretty sure that it's just not possible to implement this
<Irenes[m]>
because it won't converge to a fixed point?
<infinisil>
E.g. think about what should happen if you conditionally import a module, but within the module it turns the condition off
<Irenes[m]>
sure, but we can have config options that control other config options; what makes that conflict different?
<MichaelRaskin>
infinisil: that would count as conflicting assignment
<infinisil>
With this, all the options defined in the module will lead to infinite recursion
<infinisil>
And at least NixOS needs to be able to list all options I guess
<infinisil>
Oh and the module system needs to know all options in order to evaluate all assignments to them
<Irenes[m]>
that's a good point about needing to be able to list all the options, but it does seem like a separate one. we could imagine a primitive that runs in a special "always include" mode, for the purpose of generating the list of options.
<Irenes[m]>
hm
<Irenes[m]>
although if it's really true that module resolution needs to run in the always-include mode at least once per invocation, that substantially reduces the performance benefit
<Irenes[m]>
all right. well I'm going to poke into it and convince myself of what you said, I guess.
<Irenes[m]>
I'll report back, even if it's just to say that you're right.
<infinisil>
The problem with running it in a different mode is that it requires changes in the modules to use it
<infinisil>
But I guess that might not be too bad
<MichaelRaskin>
I think the hard part is that what-forces-load-of-what is not completely trivial
<Irenes[m]>
sure
<MichaelRaskin>
So one needs to look at all the modules and record that.
<Irenes[m]>
yes, certainly
<MichaelRaskin>
(and trying to get acceptance for such a cross-cutting change which also includes a non-trivial change to the core risks ending up as a redesign of the entire modules system)
<infinisil>
I actually have some notes on a reachitecture of the module system in a mostly backwards compatible way that allows modules to be conditionally excluded
<Irenes[m]>
yeahhhh I don't think it would be worth it if, for example, it wound up that any module that sets config options in other modules needs to explicitly record where they came from
<infinisil>
Essentially, modules get separated into components (a component consists of multiple modules), which are activated based on assigned option paths
<Irenes[m]>
interesting
<infinisil>
And this is done iteratively. So if the first set of components assign options that belong to components that aren't included yet, this is repeated with them included
<Irenes[m]>
right, yeah
<infinisil>
Until the set of components doesn't change anymore
<Irenes[m]>
just in the spirit of thinking through the limitations
<Irenes[m]>
one thing this does not allow you to do is dependency injection (but for modules)
<Irenes[m]>
which is fine, I think
<Irenes[m]>
we don't have that now and I don't see much need for it; dependency injection can happen at the package level
<Irenes[m]>
like if you had a bunch of components which were stuff needed for particular hardware platforms
<Irenes[m]>
... well, I can't think of a scenario where you would actually be substituting one for another
<Irenes[m]>
so I think that's fine
<infinisil>
Well you can do that already with disabledModules
<Irenes[m]>
oh, hmm
<Irenes[m]>
right, I see. I was about to think out loud about the modules for X display manager / window manager / etc. I gather that's how they do it already.
<infinisil>
With disabledModules? I don't think that's used for anything in nixpkgs
<Irenes[m]>
ah okay
<Irenes[m]>
well, given that X works as-is, I am prepared to stipulate that dependency injection for modules can't be that important. I think the X stuff is pretty close to the most messy thing this needs to deal with.
<Irenes[m]>
another thing to think through with components is how they would interact with overlays
<Irenes[m]>
overlays are... beneath this layer, right? really just about packages
<Irenes[m]>
let me check my understanding
<Irenes[m]>
yes, beneath this layer. they won't interact much. good.