2018-02-04

<clever> ive also discovered some weird things
<clever> ivanivan: so its in here
<clever> nix-repl> config._module.args.pkgs.linuxPackages.kernel
<clever> ivanivan: under config._module.args you can find every argument that is passed to configuration.nix
<clever> michaelpj: runCommand takes a set, which can contain anything mkDerivation accepts
<clever> michaelpj: runCommand
<clever> ivanivan: pkgs is rather hard to access from the nixos config, one min
<clever> ah, but no meta possible with the raw one
<clever> michaelpj: all writeScriptBin does is set 4 arguments for writeTextFile
<clever> and :r to reload everything after making changes
<clever> and you can :b any of those attributes to build them, then poke around with ls
<clever> how i found the last one
<clever> all of the kernel modules
<clever> nix-repl> config.system.modulesTree
<clever> nixos will then package that script (and all its deps) into an initrd
<clever> the bash script inside stage1, that contains paths to the kernel modules in the initrd
<clever> nix-repl> config.system.build.bootStage1
<clever> nvidia drivers, built against the kernel
<clever> nix-repl> config.boot.kernelPackages.nvidia_x11_beta
<clever> my current kernel
<clever> nix-repl> config.boot.kernelPackages.kernel
<clever> «derivation /nix/store/sc196z0sb5vhq16wvz7695ilvk1b9imq-linux-4.9.75.drv»
<clever> and then eval things like config.boot
<clever> ivanivan: nix-repl '<nixpkgs/nixos>'
<clever> ivanivan: pkgs.makeModulesClosure is used to merge all the modules together into 1 directory tree, and it runs the bash script i linked above, which does depmod
<clever> ivanivan: all making sense?
<clever> boot.kernelPackages = pkgs.linuxPackages; will then change both the kernel, and the module packages together
<clever> and if you do boot.extraModulePackages = [ config.boot.kernelPackages.foo ]; you will always get the module for the right kernel
<clever> ivanivan: and then nixpkgs will auto-build it against the right kernel
<clever> ivanivan: you need to ensure the kernel module is inside pkgs.linuxPackages.foo
<clever> ivanivan: yeah, nix will do it for you at a later point in time
<clever> ivanivan: depmod has to be ran after nix has merged every package that provides modules
<clever> ottidmes: add in the kexec tools, and your crypto tools, and it could then kexec over to any kernel it can find
<clever> ottidmes: it currently boots into a very very basic shell, that is implemented in haskell
<clever> ottidmes: this is an "OS" based on a custom init binary (written in haskell) booted inside the initrd under linux
<clever> ottidmes: do you know haskell any?
<clever> ottidmes: i should look into that issue when i get a chance and finish my idea up
<clever> then it kexec's your choice of OS
<clever> ottidmes: and if you do connect, you get a boot menu (or shell) over ssh
<clever> ottidmes: my idea, was to make a very thin "OS", that runs sshd, and if you dont connect within 30 seconds, it boots the default config via kexec
<clever> ottidmes: because the grub generation list for emergency rollbacks, is 100% useless on a headless server
<clever> ottidmes: i had an idea a few months ago, about making a kexec based bootloader, for servers
<clever> ottidmes: hmm, yeah, you could, and that reminds me
<clever> ottidmes: the problem, is that nixos has to modify the bootloader config every single time you `nixos-rebuild switch`, so it has to re-generate that, and re-flash the PBA
<clever> ottidmes: at a glance, i think thats just making a /boot containing your kernel, initrd, and bootloader, formated with ext4, and gzip'd
<clever> ottidmes: you generally put /boot on that unencrypted partition, and then it gets the kernel/initrd from that
<clever> ottidmes: i think this option allows the legacy grub stub (between sector 1 and partition 1) to ask for a password and decrypt /boot/
<clever> ottidmes: https://nixos.org/nixos/options.html#boot.loader.grub.enablecrypt
<clever> ottidmes: enless your using luks support within grub, the /boot needs to be in cleartext somewhere
<clever> the method i gave above has minimal impact on what nix is doing to save you, and can be undone easily when your finished
<clever> but this variant will be the original rootfs, without any mounts ontop of it
<clever> viric: the kernel is smart enough to know, and just reuse the existing open handle
<clever> viric: mount the entire device again, elsewhere, for example, mount -v /dev/sda1 /mnt/
<clever> viric: ah
<clever> viric: what do you want to do?
<clever> ottidmes: and if you always mount that unlock partition to /boot, nixos-rebuild can update it properly
<clever> ottidmes: you can use configuration.nix to modify the scripts in stage1
<clever> ambro718: leave the overlay in the public location, and make a symlink under $HOME that points to it
<clever> ambro718: you can just make a symlink pointing to that main location
<clever> ambro718: you put them into the directory this line looks at
<clever> you also need to make sure the /boot partition is available at /boot inside the chroot
<clever> ij: switch and test involve interacting with systemd and starting services up, which wont work under chroot
<clever> ij: only "nixos-rebuild boot" and "nixos-rebuild build" would work in there

2018-02-03

<clever> kini: yep
<clever> kini: nix-prefetch-url just downloads it with curl, then calls nix-store, which needs $NIX_REMOTE set right
<clever> kini: you have to tell nix that its in multi-user mode now, and how to contact something that does have write
<clever> kini: this error is caused by not setting `export NIX_REMOTE=daemon`
<clever> 2018-02-02 22:35:32 < kini> and when I try to run THAT in the nix-shell, I get "error: Nix database directory ‘/nix/var/nix/db’ is not writable: Permission denied"
<clever> dhess: https://groups.google.com/forum/#!topic/nix-devel/zDQH7Ulotoc was recently mentioned, nixwrt
<clever> samueldr: i have since put in a nixos router, with gigabit cards, and i idle in #nixos-on-your-router
<clever> the router would have become a bottleneck
<clever> the new internet downloads at 300mbit
<clever> and it only had 100mbit cards
<clever> 600kbyte/sec would tax the cpu at about 70%
<clever> i retired it when i upgraded to fiber
<clever> it was an x86 machine with i think 128mb of ram
<clever> i once ran linux from scratch on my router
<clever> yep
<clever> lord|: there is also a script that uses mount namespaces to fake having /nix/
<clever> so root can just make you an empty /nix/ and chown it over, and then you can install without ever gaining root
<clever> lord|: if you already have write access to /nix/, it wont ask for sudo
<clever> pretty much everything else should just work though
<clever> lord|: so you cant just install the nix variant of virtualbox
<clever> lord|: opengl and things needing kernel stuff are the only issues
<clever> captive portals are an equal pain in the ass
<clever> so it ignores the vars set on nix-env
<clever> nix-daemon is the one responsible for doing the downloads
<clever> yeah
<clever> Environment="LD_LIBRARY_PATH=/nix/store/697dh7g650f1qd4aqpkn3zing0g1qgx7-libX11-1.6.5/lib:/nix/store/w7nbvpgaqfvdw3i1383agl5yw72fc8g1-libXext-1.3.3/lib:/run/opengl-driver/lib"
<clever> [Service]
<clever> whatever process started nix-daemon
<clever> kini: are $SSL_CERT_FILE, $https_proxy, $CURL_CA_BUNDLE are all set and exported in the context of the nix-daemon?

2018-02-02

<clever> infinisil: but bash will neuter itself if argv[0] == sh
<clever> ah
<clever> sh i think
<clever> infinisil: which is just the absolute path to a shell
<clever> infinisil: there is also ${stdenv.shell}
<clever> lejonet: the default kernel has 32bit support
<clever> identical to mine
<clever> lejonet: what does md5sum say about it?
<clever> lejonet: that binary is valid on my 64bit machine
<clever> BusyBox v1.23.2 () multi-call binary.
<clever> [clever@amd-nixos:~/Downloads]$ /nix/store/lan2w3ab1mvpxj3ppiw2sizh8i7rpz7s-busybox
<clever> lejonet: and if you run it manually?
<clever> lejonet: what does "file" say about the binary?
<clever> freeman42x: i dont think the makefile includes an install rule, so `make install` just fails
<clever> freeman42x: now you need to add mesa to line 8 and the inputs
<clever> mesa.out 0 s /nix/store/nppbk91gbdnys2950grrl9zbpssx4iyg-mesa-17.0.3/include/GL/glx.h
<clever> [clever@amd-nixos:~/Downloads]$ nix-locate GL/glx.h
<clever> freeman42x: lists dont use ,
<clever> freeman42x: add xlibs to line 8, and xlibs.libX11 to the buildInputs
<clever> xlibs.libX11.dev 3,118 r /nix/store/2802m84rk70vac9n6wz6d8cgzzzk2600-libX11-1.6.5-dev/include/X11/cursorfont.h
<clever> [clever@amd-nixos:~/Downloads]$ nix-locate cursorfont.h
<clever> freeman42x: postPatch = "cd src";
<clever> freeman42x: oh wait no
<clever> freeman42x: try postUnpack = "cd src";
<clever> freeman42x: needs an unzip on line 8
<clever> freeman42x: simplest way is to just wait for it to fail, then it tells you the right one
<clever> freeman42x: then add buildInputs = [ unzip ];
<clever> freeman42x: oh, thats not even a fetchurl
<clever> freeman42x: try sticking to fetchurl, and just add unzip to the buildInputs of the derivation using it
<clever> freeman42x: that lets fetchurl and stdenv unpack it
<clever> freeman42x: or, just add unzip to the buildInputs
<clever> elvishjerricco: yeah
<clever> elvishjerricco: and post a static check, like travis does
<clever> elvishjerricco: it will then lookup the org/user owning that repo in github_authorization
<clever> elvishjerricco: for github status, when a build passes, it will find the input of the given name (cardano in the case of line 80), and lookup where on github it came from, and what rev it was in the eval
<clever> that will do it
<clever> hyper_ch2: does the new myconfig.nix have the same imports as the original configuration.nix?
<clever> hyper_ch2: with --show-trace ?
<clever> hyper_ch2: yeah, that pulls in konsole and a ton of crap
<clever> dang
<clever> i last used that /boot variant to move /nix from the / dataset to the /nix dataset
<clever> having a backup installer in /boot would let you fix it
<clever> that does basically the same thing, but via the bootloader
<clever> hyper_ch2: https://github.com/cleverca22/nixos-configs/blob/master/rescue_boot.nix may also be of interest to you
<clever> hyper_ch2: how big is your /boot/ ?
<clever> so you can just set initialPasswords and ssh keys on whatever you want
<clever> hyper_ch2: any config you put into this area will become active once the kexec boots up
<clever> Rovanion: build nix from source, and configure the prefix, storepath, and varpath to all live within $HOME
<clever> Rovanion: its also a config option, and they could have just turned it off
<clever> Rovanion: yes, but you either need to use user namespaces, or recompile everything from scratch
<clever> i think i used the CLI args to configure it
<clever> last time i used it, yes
<clever> yes
<clever> only hackage has really good support for that kind of thing in nix
<clever> nope
<clever> and there is no python variant of it in nixpkgs
<clever> ldlework: at one point, libgumbo was an alias for gumbo, but now only libgumbo resolves
<clever> /home/clever/apps/nixpkgs/pkgs/top-level/aliases.nix: libgumbo = gumbo; # added 2018-01-21
<clever> /home/clever/apps/nixpkgs/pkgs/top-level/all-packages.nix: gumbo = callPackage ../development/libraries/gumbo { };
<clever> ldlework: first, search the rest of nixpkgs for the string gumbo, and find out where its used and how
<clever> ldlework: wrap the entire call to withPackages in ( and )
<clever> dont know then
<clever> iqubic: are you on a channel?
<clever> iqubic: depends heavily on your download speed
<clever> depends on what you may want to keep secret within the nix store
<clever> fresheye1: one issue they have, is that the guest can read the entire nix store, and i'm not entirely sure how solid root is in the guest
<clever> dtz: sonarpulse may know more
<clever> so you gain nothing there
<clever> and the nixos cache is already on S3
<clever> some CI tools do have caching, but they just upload the files to a place like S3
<clever> eceuv: the queue times on hydra can be much shorter, even with the paid travis plan, ive found myself waiting hours for a job to even start
<clever> eceuv: hydra can serve the resulting binaries to you and speed up your own local builds
<clever> eceuv: hydra keeps a cache of all of your depdencies, so you dont haev to wait for travis to redownload them every time
<clever> but if the input is another derivation, you already locked the hash of things in with a fixed-output derivation, so it doesnt really work there
<clever> dhess: and most fo that filtering is to prevent the hash from changing
<clever> dhess: cleanSource just uses filterSource behind the scenes, and filterSource is meant to filter things as they get copied from the mutable filesystem to the immutable nix store
<clever> dhess: nix does it by operating on a secondary store within the tmpdir
<clever> dhess: you may want to look at how nix's tests test filterSource, it runs nix inside nix
<clever> i think it would be best to just have the input data not be in a tar
<clever> ah
<clever> dhess: why do you want to run cleanSource on it?
<clever> dhess: if its a tar, clean it up in postUnpack
<clever> dhess: generally, you shouldnt be using cleanSource on a tar, only on a working directory
<clever> dhess: runtime dependencies
<clever> dhess: the things imported by filterSource and cleanSource must have no dependencies
<clever> dhess: cleanSource doesnt understand how to unpack things
<clever> :D
<clever> iqubic: yeah
<clever> samueldr: it might be that the ld.so path is wrong, and the bash script assumes the error is due to noexec

2018-02-01

<clever> on both cases, the config has to get into there somehow
<clever> nix.buildMachines just configures /etc/nix/machines for you
<clever> hakujin: yeah
<clever> hakujin: is /etc/nix/machines configured?
<clever> Shados: i generally use tab completion in nix-repl

2018-01-31

<clever> ij: thats likely giving you a shell suitable for building terraform, not using it
<clever> joepie91: some quick math confirmed how big the framebuffer was, then i used MTD to convert a chunk of video ram into a block device, mkswap, double the ram!
<clever> joepie91: the gpu also had 64mb of ram, and it was used as a server
<clever> joepie91: i'm also reminded of a cheap trick i used, back when i had 64mb of ram
<clever> joepie91: i cant remember who brought it up last time in #nixos, but they mentioned the switch
<clever> joepie91: chromium is the only place i noticed that tooltip issue
<clever> joepie91: i had to comment that out to even get graphics
<clever> 52 #"amdgpu.dpm=0"
<clever> joepie91: and changing the power management options breaks the gpu entirely on my end
<clever> but it still locks X up solid every now and then
<clever> i have since switched to the amd-gpu driver, and most of those issues have gone away
<clever> joepie91: i have even seen my windows wallpaper, slightly corrupted, upon logging into nixos!!
<clever> so my tooltips always rendered as garbage on the first drawing
<clever> joepie91: oh, that reminds me, for the longest time, ive found evidence of my GPU using tooltip textures before they where initialized
<clever> unknown, i wasnt verifying the files often enough when they did become corrupt
<clever> so bit-rot wont be an issue, but while moving, it may become corrupt
<clever> viric: zfs would prevent the issues on-disk, but in my case, problems could still happen in ram and over the nfs layers
<clever> ij: yeah
<clever> viric: it was next to the other one in my bookmarks, lol
<clever> ij: there is a post hook you can set, which can just rename whatever it wants
<clever> yet nobody knows what the utility even does
<clever> the talk at 58 minutes in, he starts diving into obscure utilities that are installed on every mac
<clever> viric: there is also https://www.youtube.com/watch?v=8Gp-RXCLO2M#t=58m10s
<clever> i use ZFS and its a must now
<clever> which cached his modified content, and served it out to end-users repeatedly
<clever> he even got internal requests between 2 google servers
<clever> yeah
<clever> and if the right bit flips, you connect to him instead, and he has a perfectly valid ssl cert
<clever> and got valid ssl certs for each of them
<clever> the guy took a number of major domains, computed every 1 and 2 bit flip variant that was still ascii, and registered all of them
<clever> viric: there is also a defcon talk i saw, that focused on bit-flips in domain names in ram
<clever> but thats after years of the files roaming from system to system
<clever> it was several files, and 1 or 2 flipped bits each
<clever> yeah, i ran it thru | hexdump -D and then diff'd
<clever> and i wound up with duplicates, that didnt match
<clever> viric: ive noticed similar, the files had move thru many computers, OS's, filesystems, and network protocols
<clever> so the map can change during boot
<clever> which unloads things like filesystem drivers that where helping with bootloading
<clever> the firmware will release a large chunk of ram when the OS says the bootoader phase is done
<clever> EFI also adds fun additions to that
<clever> yeah, thats the real bug
<clever> rom's can be slow to read, so the bios copies it to ram, then flags it as read-only
<clever> remapped to a rom, possibly a rom shadow
<clever> viric: the fun part, is that 4 machines, and qemu, all fail the same test, at the same address
<clever> viric: the changes in nixpkgs broke it, and memtest lacks testing in nixos
<clever> viric: if memtest is compiled with certain hardening flags, it gives a false positive during one of the tests
<clever> viric: we where floating the idea of a bios level malware that had infested us, lol
<clever> then i had a friend running nixos half way across the continent run a memtest, his ram is also bad! ......
<clever> ran a memtest on a laptop from 3 or 4 generations ago, that also has bad ram
<clever> swapped all the hardware over to that motherboard, still bad....
<clever> i swapped the ram out with another machine, still bad, ok, its the motherboard?
<clever> my main nixos desktop had a weird fault, and suspecting ram, i ran memtest, which confirmed bad ram
<clever> that bug nearly convinced me i was insane :P
<clever> i once found a bug in memtest, due to gcc hardening flags in the nixos build
<clever> but the entire source is on github
<clever> the builds of it require a one-time payment
<clever> ive also noticed synergy doing weird grey-zone stuff
<clever> grub and systemd generate the names differently, and ive always been a grub guy
<clever> but systemd-boot lacks such an option
<clever> ottidmes: that sets an upper limit on how many generations are in the bootloader
<clever> ottidmes: https://nixos.org/nixos/options.html#boot.loader.grub.configurationlimit
<clever> ottidmes: there is also a nixos option that limits how many times it will happen, on grub
<clever> ottidmes: each generation has a unique initrd, so it has to re-make each of them
<clever> so nix cant cache things
<clever> and it has to happen after the nix builds, so the secrets stay out of /nix/store/
<clever> ottidmes: that message is from cpio, as it generates an initrd on the fly, containing your secrets
<clever> ottidmes: or other secret related things in the initrd?
<clever> ottidmes: is boot.loader.grub.extraInitrd set?
<clever> ottidmes: aha, one minute
<clever> ottidmes: is there an execve within logfiles.2046 ?
<clever> ottidmes: then `grep --color '2 blocks' logfiles*`
<clever> ottidmes: strace -o logfiles -ff -s 5000 nixos-rebuild switch
<clever> ottidmes: ok, time to go overkill!
<clever> ottidmes: yep
<clever> nixer: https://github.com/NixOS/nixpkgs/blob/master/maintainers/scripts/copy-tarballs.pl i think this manages creating the cache
<clever> nixer: it only contains tar's that this script finds, and only for certain licenses
<clever> nixer: yeah
<clever> ottidmes: what if you have `journalctl -f` running in another terminal during the nixos rebuild?
<clever> nixer: i think thats the tarballs.nixos.org cache
<clever> what bootloader do you have enabled?
<clever> ottidmes: does the message happen every time you nixos-rebuild?
<clever> ottidmes: the importantance of the config depends on if that log shows the message or not
<clever> ottidmes: run nix-store -l on that path, does that show the same message?
<clever> ottidmes: and gist your entire configuration.nix
<clever> ottidmes: can you screenshot the terminal when it happens?
<clever> yep
<clever> you can see the priority if you just echo $PATH
<clever> so that would explain the shadowing
<clever> oh, and that one also has priority over nixos
<clever> so you will want to remove that before you continue
<clever> philipwhite: you installed vim as root with nix-env
<clever> philipwhite: what does "which vim" alone say?
<clever> philipwhite: everything ive checked says that vim is building with python by default, how are you checking to see if it has python support?
<clever> philipwhite: aha, found the definition of customize: https://github.com/NixOS/nixpkgs/blob/master/pkgs/misc/vim-plugins/vim-utils.nix#L328
<clever> philipwhite: this confirms that true is the default, and that false is causing it to rebuild
<clever> philipwhite: if i'm reading the expressions right, all 3 of those options default to on
<clever> philipwhite: can you gist all of the nix files you have modified?
<clever> cinimod_: nix-env -q shows the names
<clever> cinimod_: -e only works on names, not attribute paths
<clever> cinimod_: only one of those can be installed at once
<clever> :q!