gchristensen changed the topic of #nixos-dev to: NixOS Development (#nixos for questions) | https://hydra.nixos.org/jobset/nixos/trunk-combined https://channels.nix.gsc.io/graph.html | 18.09 release managers: vcunat and samueldr | https://logs.nix.samueldr.com/nixos-dev
jtojnar has quit [Remote host closed the connection]
Drakonis has joined #nixos-dev
jtojnar has joined #nixos-dev
jtojnar has quit [Remote host closed the connection]
jtojnar has joined #nixos-dev
lassulus_ has joined #nixos-dev
lassulus has quit [Ping timeout: 244 seconds]
lassulus_ is now known as lassulus
jtojnar has quit [Quit: jtojnar]
jtojnar has joined #nixos-dev
Drakonis has quit [Remote host closed the connection]
Drakonis has joined #nixos-dev
orivej has quit [Ping timeout: 240 seconds]
Drakonis has quit [Remote host closed the connection]
sphalerite_ is now known as sphalerite
Ericson2314 has joined #nixos-dev
orivej has joined #nixos-dev
page has joined #nixos-dev
goibhniu has joined #nixos-dev
Lisanna has joined #nixos-dev
CrystalGamma[m] has joined #nixos-dev
<domenkozar> two items for this week
<domenkozar> hmmz
<domenkozar> August is a busy month as always :)
<goibhniu> Hi Lisanna, would you mind if I /msg you? (It's about NixCon)
<Lisanna> Oh, uhh, sure
<domenkozar> could someone proof read?
<domenkozar> 2little2late
<ekleog> domenkozar: missing a point at the second bullet point
<ekleog> otherwise, lgtm :)
<domenkozar> point, you mean dot?
<domenkozar> here we go:
<ekleog> indeed :5
<ekleog> :)*
<ekleog> removing the dot of the first bullet point is another solution indeed
<domenkozar> thanks for spotting that :)
<ekleog> you're welcome :) but more than that, thank you for handling NixOS Weekly! (as well as all the rest you do for NixOS :p)
<domenkozar> thanks :)
ldlework has quit [Quit: co'o ro do]
NinjaTrappeur has joined #nixos-dev
<niksnut> closure size of nixUnstable on 18.03: 109 MiB; on master: 470 MiB
<infinisil> niksnut: i think the hackage-packages.nix updates are a big part of that
ldlework has joined #nixos-dev
ldlework has quit [Client Quit]
ldlework has joined #nixos-dev
<niksnut> infinisil: closure size of nixUnstable, not size of nixpkgs
<Mic92> Is it due changes in curl?
<niksnut> mainly, it appears to be due to the fact that openssl now has binutils and glibc.dev in its closure (and binutils is > 200 MiB now)
<niksnut> lib/libcrypto.so.1.0.0: …...........compiler: /nix/store/9z650yzczx5zyhcw5n36ya6x4jq436mr-gcc-wrapper-7.3.0/bin/gcc -I. -…
<Mic92> But was nix not depending on openssl before?
<Mic92> ok. so openssl could be source/binary patched to not include this string.
<{^_^}> #45460 (by edolstra, 27 seconds ago, open): OpenSSL closure size increase
<niksnut> Mic92: right
<niksnut> not sure why that string is there now
<niksnut> another issue is that nix now has an unnecessary dependency on icu via boost
<Mic92> afaik openssl allows to introspect some information like this at runtime
<Mic92> OPENSSL_CFLAGS
<niksnut> oh, this is probably caused by $CC now being an absolute path
<infinisil> Ahh I see
<Mic92> I suppose grepping over the system closure will reveal more occations of this.
init_6 has joined #nixos-dev
<niksnut> it's already reverted on staging
__Sander__ has joined #nixos-dev
joepie91___ has joined #nixos-dev
joepie91___ is now known as joepie91
<infinisil> Another instance of a PR adding loads of options that probably won't get used by 99.99% of people: #45412
<{^_^}> https://github.com/NixOS/nixpkgs/pull/45412 (by costrouc, 1 day ago, open): minecraft-server: 1.12.2 -> 1.13.0 + service refactor
* gchristensen leaves a comment
<infinisil> Maybe this single-config-option approach I've talked about would work well here too
<CrystalGamma[m]> I don't know if anyone noticed, but commit 1dc0404d6efb27407ce7716a14610b5f9bbf115a (according to my bisections) broke the musl stdenv (on staging), but I'm not sure how (grepping and find -type l -exec readlink {} ; didn't turn up any reference and when building with -vvv it also mentions all the paths it says must not be referenced as 'unreferenced input's) …
<CrystalGamma[m]> that's the 'turn $CC into an absolute path' commit AFAICS …
<globin> CrystalGamma[m]: that should be reverted on staging
<CrystalGamma[m]> it is? I guess I haven't pulled in the last 24h …
<ekleog> infinisil: oh, the one-config-option vs. multiple-config-option eternal debate?
* ekleog would hope this would be great to solve by a RFC, but in the current state of RFCs… :(
<gchristensen> we're outside of the realm of preference, we're in the realm of it is close-to-or-is causing problems
<ekleog> yeah, I'm completely in favor of using a configFile = "…" + validator approach over using nixos config options too when possible, would it be only to reduce the learning curve of NixOS… but apparently not everyone agrees
<ekleog> (though I must plead guilty to having already done similar huge-module-for-a-config-file modules… :/)
<gchristensen> I think those are fine, too, but they don't need to live in nixpkgs/nixos
lopsided98 has quit [Ping timeout: 260 seconds]
lopsided98 has joined #nixos-dev
orivej has quit [Ping timeout: 264 seconds]
<infinisil> ekleog: What I'm talking about is to use a single option that transforms a nix value into the config file
<infinisil> So e.g. `config.foo.bar = 10;` could be turned into an ini file like `[foo]\nbar=10` (or whatever format the config needs to be in)
<infinisil> This has the advantage that you can set defaults that can still be overridden, you can merge values from different modules, and there is no big evaluation time increase, as it's only a single option
<infinisil> Disadvantage being that you lose the type checking for every single option and that the user will have to look up in the packages docs what options are valid. Could be extended with a build-time check if the package supports it though
<clever> infinisil: the systemd service files do similar under .serviceConfig
<clever> infinisil: i have gotten the case wrong on those before, and it just silently didnt work
<clever> it also means you cant mkForce the attributes of serviceConfig, because they are not options
<ekleog> infinisil: oh, that's a third option indeed :) though I'm not really a big fan, pretty much like the validator option (though it'd likely require some pretty heavy changes to nixos to be able to validate at build-time?)
<infinisil> I'm pretty sure mkForce still works even when it's not an option
<clever> infinisil: ive seen users having problems with serviceConfig.ExecStart = mkForce ...;
<infinisil> Couldn't that have been because it needs to have an empty string to override it? `mkForce ["" "/foo/bin/bar"]`
<infinisil> Because it's a list probably
<infinisil> ekleog: Well it's the best thing to imo do while the module system is so slow
<clever> posibly, and thats just getting ugly :P
<ekleog> infinisil: if the module system was able to add tests at the end of the build script for the system derivation, then the option could just be a plain old list validated at compile-time by some program-specific stuff
<ekleog> s/list/string/
<ekleog> my point for using just strings is, it makes coming to nixos and converting examples found on the web to nixos much easier
<infinisil> ekleog: That's already possible, just make the config file derivation error out when building if it's invalid
<ekleog> oh indeed
* ekleog didn't think it through
<infinisil> ekleog: I mean, it's still possible to just use a config verbatim, both variants can exist (but the verbatim config will of course override the nix-value-as-config)
<infinisil> verbatim just isn't very composable, doesn't allow overriding and you might make syntax errors
orivej has joined #nixos-dev
<ekleog> syntax errors would be caught by the config file derivation erroring out
<infinisil> Unfortunately most packages don't support something like that
<ekleog> running the package and killing it just afterwise wouldn't make the trick?
* ekleog didn't try yet, but most packages tend to fail to start when the config is wrong
<infinisil> Hmm.. If it errors out then that could work yeah
<infinisil> Maybe it's a bit weird to run a package you usually run on the target host on the build host now
<ekleog> indeed
<ekleog> I still think that'd make NixOS less frightening, esp. if we could get some proper templating support to the config files :)
<infinisil> But yeah, I think I'll add that to my ZNC changes :D. Works with both verbatim and nix-value-as-config ways
<infinisil> Templating why? Isn't that what options allow you to do already?
<ekleog> for repetitions and things like having one's own stuff :)
<ekleog> like, I've got a dict of machines -> public key for my VPN, being able to iterate over it
<ekleog> but I can't expect the module writer to have it done
<ekleog> actually, I think I'm more or less advocating for much fewer options
* infinisil doesn't quiet get it
<ekleog> basically, I feel like a NixOS system shouldn't be much different to configure than a normal system with salt|ansible|other, for people to grasp it more easily :)
<ekleog> thus thinking it'd be better to have much fewer options, even maybe the bare minimum, and push most of the current module options into NUR
<ekleog> now, that's an extremist point of view that I'm even exaggerating here, but the idea is here, even though I don't even remotely hope for it to ever come true ^^
Lisanna has quit [Quit: Lisanna]
Lisanna_ has joined #nixos-dev
<ekleog> yeah, saw that go through recently, I don't necessarily agree it's a good think to force an expert system on every single user of NixOS (well, I like it, but have heard it to be a burden to adoption: it requires learning something that is not linux in the sense people are used to)
<manveru> but i agree that always having an option to override the actual config with a file/string of your choosing would be nice...
<cransom> i tend to like (and also try to write) modules that try and do fancy things that think they are doing the right thing for my use case, but if that's completely wrong, they take a configFile and assert that if you defined a config and configFile, it's an error as configFile will override the fancier options.
<manveru> jup
<manveru> one issue with config files is that they get outdated if you don't keep them in sync with your package... plus most people only want to tweak one or two options anyway, if any
<infinisil> Should it error when you have both a config and configFile defined?
<manveru> well, that might break other modules that set options on other modules when you enable them
<cransom> i think it should. if one completely overrides the other, it's an invalid config.
<infinisil> Hmm actually in my case it's pretty hard to detect whether the user actually assigned config
<ekleog> manveru: yeah, I'm not thinking it's not useful, just that maybe it should be in NixES -- Nix Expert System -- and be kept decoupled from raw NixOS :)
<cransom> manveru: it may, but you already decided to step outside of the system a bit so it's on you to make sure it stays up to date.
<infinisil> I'll just note in the description that configFile overrides this
<ekleog> would solve teh issue of derive time, bwt
__Sander__ has quit [Quit: Konversation terminated!]
<manveru> :)
<manveru> i think there's also a lot of people that would rather see all config files replaced with dhall...
<ekleog> methinks that's not the kind of people we need help to trick into nixos :°
<infinisil> ekleog: Ohh, a problem with the running the package with the config as a config validation
<infinisil> Is that the package might fail because it's run in the sandbox
<ekleog> oh indeed :(
<infinisil> I've now implemented such a thing with ZNC, and it does indeed work, but I can't be sure that it's gonna work for all config values
<ekleog> and patching the package to get a validator by just commenting out everything past the config parsing isn't going to scale well :°
<infinisil> Another advantage of this nix-value-as-config approach is that you could validate the value through some assertions, therefore checking the config right during evaluation
<infinisil> However that doesn't work when you allow `extraConfig` verbatim values in this nix value (which I'm doing with znc)
<infinisil> Because you can't validate stuff over strings without ugly parsers :/
pie_ has quit [Ping timeout: 264 seconds]
orivej has quit [Ping timeout: 265 seconds]
init_6 has quit [Ping timeout: 272 seconds]
lassulus has quit [*.net *.split]
aminechikhaoui has quit [*.net *.split]
rsa has quit [*.net *.split]
noefk has quit [Ping timeout: 240 seconds]
noefk has joined #nixos-dev
nwspk has joined #nixos-dev
ChanServ has quit [*.net *.split]
ChanServ has joined #nixos-dev
orivej has joined #nixos-dev
lassulus has joined #nixos-dev
lassulus has quit [Client Quit]
orivej has quit [Ping timeout: 240 seconds]
lassulus has joined #nixos-dev
lassulus has quit [Client Quit]
lassulus has joined #nixos-dev
lassulus has quit [Client Quit]
lassulus has joined #nixos-dev
pie_ has joined #nixos-dev
pie_ has quit [Remote host closed the connection]
pie_ has joined #nixos-dev
pie_ has quit [Remote host closed the connection]
pie_ has joined #nixos-dev
pie_ has quit [Remote host closed the connection]
pie_ has joined #nixos-dev
pie_ has quit [Remote host closed the connection]
lassulus has quit [Quit: WeeChat 2.0]
lassulus has joined #nixos-dev
aminechikhaoui has joined #nixos-dev
<niksnut> C++17 structured bindings ftw, it's almost becoming a decent language
lassulus has quit [Quit: WeeChat 2.0]
<gchristensen> oh!
lassulus has joined #nixos-dev
<gchristensen> I dunno niksnut someone in #nixos-chat is working on a Nix parser =)
<gchristensen> (in Rust)
<niksnut> awesome :-)
<LnL> oh wow, that's a thing now
<gchristensen> oh wow, structured bindings _are_ nice
<LnL> the tie thing from before looked extremely weird to me :p
<LnL> don't think I ever used it
<infinisil> I don't think #45478 is a good idea
<{^_^}> https://github.com/NixOS/nixpkgs/pull/45478 (by tycho01, 51 minutes ago, open): update NixOS fdisk instructions based on my flow (18.03)
<infinisil> I don't see the reason to switch to gdisk, fdisk supports GPT already
<gchristensen> the deletion of 'r' is a good change though :)
<samueldr> hmm, but "m" isn't "exit expert mode" ?
<gchristensen> hmm thinking a different one
<samueldr> m print this menu
<samueldr> r return to main menu
<clever> i was helping him in #nixos when he installed, he was following the uefi directions, but still had an MBR partition table
<gchristensen> I was thinking 'f'
<clever> and he didnt even need uefi
<srhb> fdisk ends up defaulting MBR in some conditions still, right?
<clever> srhb: he was installing over manjaro, and didnt wipe the partition tables
<srhb> And also there are cases with weird terminals where it's really bad at showing the partition type list, compared to gdisk
<srhb> clever: Ah, okay, hard to guard against that, gdisk would do the same.
disasm has quit [Ping timeout: 260 seconds]
<clever> checked my irc scrollback, it was fzer0 i was helping, but maybe tycho is his github handle?
<samueldr> right now the status quo is better than exchanging four quarters for a dollar of still dubiously "right" instructions... I should dig back my notes for the installation instructions
<clever> he also had a lot of trouble editing partition tables while things where mounted, so the tables didnt update
<clever> mounting boot to /mnt and root to /mnt/boot
<clever> and then trying to umount /mnt when /mnt/boot was still mounted
<samueldr> (one of the idea was to be descriptive instead of prescriptive; explain the expected results, and then show an example to achieve, and completely separate UEFI and MBR schemes)
<clever> and he kept spell-checking umount to unmount, then couldnt figure out why it said command-not-found
<infinisil> samueldr: +1 to that
<clever> ive been wanting a good decision tree website to help guide you thru partitioning nixos
<clever> because there are just so many options
<samueldr> clever: cleverly done it could be distro-agnostic
<samueldr> (or agnostic-enough)
<gchristensen> the Nix ecosystem could use a little less cleverness when it comes to docs :P
<clever> you can use gpt or mbr tables if you are booting legacy, but a lot of people have trouble with legacy on gpt
<samueldr> and yes, the main stumbling block I faced was paralysis in front of that tree :/ I had only my own opinionated knowledge :)
<clever> and id say there is no point to ever use mbr now
<infinisil> clever: What kinda constraints does NixOS partitioning have anyways?
<infinisil> I don't think there's anything you can apply to all instances
<clever> infinisil: most of them come from the motherboard firmware
<samueldr> infinisil: what I found out is there's a couple assumptions that nixos makes where it's easier if you go with the flow
<clever> if you want uefi boot, then you must use GPT, with a fat32 /boot/
<samueldr> (ESP at /boot)
<infinisil> Ah right, UEFI vs BIOS
<clever> if you want luks, you must have a /boot seperate from /
<clever> if you want legacy on gpt, you must have a bios boot partition, 1mb, not formated, not mounted
<infinisil> Yeah that would be nice to get some good interactive thing for
<clever> then you need to combine the above
<samueldr> gchristensen: though that tree would be out-of-docs :) the docs being descriptive with a bit of prescriptivism with a well-known example
<clever> infinisil: and then if you want zfs+swap+luks, you need to decide if hibernation is important to you
<gchristensen> domenkozar: https://github.com/grahamc/test/pull/3/files .travis.yml can support nix versions 1.11.16 through 2.0.4 thanks to niksnut :)
<clever> infinisil: and then decide if you want a keyfile or lvm
<infinisil> Ah right, so not just partitioning, but filesystem management in general
<infinisil> nixpart something something aszlig
<clever> infinisil: this script will partition, format, and install nixos: https://github.com/cleverca22/nix-tests/blob/master/kexec/justdoit.nix#L48
<clever> infinisil: luks and uefi are optional, and seperate options
<clever> and the size of /boot and swap can be adjusted, along with the zfs pool name
<infinisil> Yeah, have seen that script many times :P
<clever> infinisil: i also have something new ive been working on...
* samueldr wonders if clever is stateless
<samueldr> ;)
<clever> infinisil: this script will generate an AMI within aws, that has zfs based rootfs
<infinisil> It's getting a bit tight to do with bash hah
<infinisil> Whoa, setting up a zpool in a nix-build
<clever> runInLinuxVM had to be heavily coerced into supporting zfs, lol
<infinisil> Damn yeah
<clever> the rootModules option doesnt support append, so adding zfs removes 9plan, and then /nix/store vanishes, lol
<clever> and it technically doesnt support out-of-tree modules at all, but luckily still works if you override the kernel with an aggregateModules
pbogdan has quit [Ping timeout: 256 seconds]
pbogdan has joined #nixos-dev
pbogdan has quit [Ping timeout: 272 seconds]
pbogdan has joined #nixos-dev
kalbasit[m] has joined #nixos-dev
pbogdan has quit [Ping timeout: 264 seconds]
pbogdan has joined #nixos-dev
<aszlig> infinisil: oh yeah, i need to get this higher on my todo list...
<aszlig> infinisil: done :-D
<aszlig> probably it won't land in 18.09 though, as i don't want to hurry it in half-assed
<aszlig> but it really needs to be done very soon, because nixpart 0.4 is a big mess
ChanServ has quit [*.net *.split]
<clever> aszlig: i noticed a lack of zfs support in the examples infinisil linked
<aszlig> clever: yeah, the problem is that blivet doesn't yet support zfs
ChanServ has joined #nixos-dev
<aszlig> i also looked for better alternatives to blivet but haven't found anything that would be better suited, especially when it comes to representing store as a tree
<aszlig> for example debian has partman, which is a mixture of c and shell scripts and it's quite stateful in the whole process, so you have states between every single stage
init_6 has joined #nixos-dev
<aszlig> while for blivet it's just build device tree based on current system information -> do the changes needed to get to the new device tree
<aszlig> between those states you just work on pure data structures
<clever> aszlig: have you seen justdoit and my more recent zfs image scripts? (both linked above)
<aszlig> clever: i've seen that, but as said, this is inherently stateful and more like partman
<aszlig> i mean, sure if you already know what hardware and what your storage layout is, then it's fine
<aszlig> but it doesn't help if you want to do this generally for a variety of hardware
<clever> aszlig: it could be modified to take those via a module, and then you just nix-build to generate the script on-demand
<aszlig> eg. if you want to support NVDIMM you need to change the script
<clever> it could be tied into the nixos modules, and you just nix-build '<nixpkgs/nixos>' -A config.system.partitionscript -I nixos-config=./configuration.nix
<clever> and have enough templating (like stage-1.sh) to allow modules to insert and extend things
<aszlig> clever: yeah, but again, you still have a partition script, which is the goal to get rid of :-)
<aszlig> clever: i'm not saying such a thing shouldn't be possible
<clever> ah, and the above doesnt deal with changes at all
<clever> the above would be more suitable for parsing a configuration.nix, generating a partition&format script, running it, then mounting things and doing nixos-install
<aszlig> "the above" as in?
<aszlig> you mean config.system.partitionscript?
<clever> what i was just explaining
<clever> yeah
<aszlig> i mean, with the blivet-approach you can even do both... like adding scripts to specific device tree nodes
<aszlig> and you get the advantage of specifying the device tree declaratively... which is especially useful if you have a heterogenous set of hardware
<aszlig> there was also a request for supporting NILFS a while back
<aszlig> so both ZFS and NILFS would be nice... and while the latter shouldn't be a lot of effort i'm not sure about zfs
<clever> aszlig: i did find a surprised in boot.growPartition, its hard-coded to look at fileSystems."/".device
<clever> which is "tank/root" ...
<clever> zfs breaks a lot of standard assumptions
<aszlig> mhm, that boot.growPartition option looks more like an ad-hoc solution
<clever> TMPDIR=/run sh $(type -P growpart) "/dev/xvda" "3"
<clever> i had to copy a chunk of it into postDeviceCommands, and also enable the broken option to get the binaries in the initrd
<clever> oh, and for extra fun, zfs mounts the FS within postDeviceCommands!
<clever> so i had to use mkBefore to expand it before importing
<clever> and mkAfter to tell zfs to resize properly
<clever> and mkMerge, so i could use before and after in the same config entry!
<aszlig> hmhm...
<aszlig> this brings me to another interesting idea...
<aszlig> when blivet executes the actions to be performed it's basically some kind of a monad, so we could in thery take this and turn it into a shell script for mounting file systems or even systemd mount units
<aszlig> clever: hm, i added your gists/repos as notes for the todo...
<clever> there are also some issues i found in nixpkgs, that i plan to PR soon-ish
<clever> line 26 says that / is found by label, and will autoResize
<clever> line 82 is needed for the ec2 keypair from an AMI to make its way into /root/.ssh/authorized_keys
<clever> so i must include this file (or copy/paste a lot of it) for an AMI to have any chance of being usable
<clever> and line 26 conflicts with zfs
<clever> i also have no clue what lines 85-121 are for, some kind of union mount between the store and another disk, ive never seen that in use on aws before
<aszlig> mhm, ideally we shouldn't have any "autoResize" parts in the fileSystems submodule
<aszlig> and more like with https://github.com/NixOS/nixpart/wiki/Device-tree-representation#ext234 -> partition.root.grow
<clever> aszlig: oh, and this may also be of interest: https://github.com/cleverca22/nixos-configs/blob/master/iscsi-boot.nix
<clever> this allows you to define fileSystems."/".iscsi = { enable = true; host = "192.168.2.11"; lun = "iqn.2015-01.com.example:san.img"; };
<clever> and then it just mounts the rootfs over iscsi
<aszlig> clever: ah, iscsi is supported by blivet at least :-)
init_6 has quit []
<aszlig> clever: anyway, right now the goal is to first tie in the new nixpart into the module system without breaking the status quo so that users can play around with it... adding zfs support would be something that'd need to be added to blivet then
<clever> yeah
<aszlig> clever: blivet is a fedora project, so i highly doubt zfs support is on their roadmap
<aszlig> i also found this: https://github.com/Antergos/TODO/blob/master/Cnchi/README.md - no idea what cnchi is though
<aszlig> -> Use blivet instead of pyparted
<aszlig> -> Add ZFS support
<aszlig> ah, it's the antergos installer
<aszlig> yeah
<aszlig> looking through that code, it looks very similar to blivet without a device tree
<aszlig> so probably switching to blivet might be still on their roadmap
<aszlig> and they don't support ZFS either
<aszlig> ah, wrong, they do