2018-10-28

<clever> nixpkgs master is never rebased, would break too much
<clever> feep: which repo?
<clever> jophish: i'm thinking, dont change too much at once
<clever> jophish: or just run `systemctl status > /dev/ttyS0`
<clever> jophish: next thing 'm thinking of is maybe a systemd timer to do something on /dev/ttyS0 at regular intervals?
<clever> does anything appear after the last thing in the console? when you left it idling
<clever> using that, you can read the logs on the SD card
<clever> jophish: journalctl --root=/path
<clever> one min
<clever> jophish: and it likely lacks anything fancy like acpi to signal a clean shutdown?
<clever> jophish: does it have a usb host interface?
<clever> jophish: is the tx line wired up right, what happens when you mash enter?
<clever> jophish: is it hanging, or just not running a serial getty?
<clever> jophish: nice
<clever> dont ask me why the dir has to be so complex :P
<clever> nikivi: its somewhere in /private/var/folders/5l/5s7qw5ld1xxglm0094_hyh_c0000gn/T/
<clever> nikivi: apple has to be different :P
<clever> feep: oooo!, that makes a lot more sense!
<clever> nikivi: and then compare the procs using cpu in `top` or `htop` with the owners of the nix-build-* dirs
<clever> nikivi: `ls -ltrh /tmp/`, the names of all packages building are in ah you already found that dir
<clever> roots channels are also the default for other users
<clever> you removed your own read access
<clever> feep: you wanted to add the 1000 bit, not make it only 1000
<clever> feep: ls -lh /nix/var/nix/profiles/per-user/root/channels
<clever> without the : at the end, oops
<clever> feep: ls -lh /nix/var/nix/profiles/per-user/root/channels/nixos:
<clever> feep: what does ls -lh say about that path?
<clever> thats the channel
<clever> or wait no
<clever> feep: your ~/.nix-profile isnt setup right
<clever> feep: what was the error?
<clever> with import <nixpkgs> {}; callPackage ./otherfile.nix {}
<clever> if the file starts with { stdenv }: then it must be loaded with callPackage, from another file
<clever> then just run nix-build
<clever> feep: create a file called default.nix, with the following contents: with import <nixpkgs> {}; stdenv.mkDerivation { name = "name"; src = ./.; }
<clever> > stdenv.mkDerivation { name = "name"; src = ./.; }
<clever> > baseNameOf (builtins.toString ./.)
<clever> > ./.
<clever> steveeJ: enless you flip on the removable flag ing rub
<clever> steveeJ: efi usually needs special efi vars to boot, and no other machine will know to boot it
<clever> steveeJ: though you could do both legacy and efi at once
<clever> steveeJ: and make sure it boots via legacy, not efi
<clever> steveeJ: just install to that usb device as normal
<clever> and resolv.conf becomes static as well, so your dns cant be based on dhcp
<clever> jophish: openssh refuses to use a world-readable private key
<clever> jophish: i could make /etc/ a static symlink to a storepath, but then its a bit more tricky to build up /etc, and every /etc file is world readable
<clever> jophish: yeah
<clever> jophish: for setup-etc.pl and managing /etc/
<clever> sadly, not-os depends on perl at runtime
<clever> jophish: that can happen if you hit the max-jobs on nix.conf, or are depending on something another nix cmd was already building
<clever> i did have my hydra building that on an rpi, but it hasnt been booting reliably lately
<clever> jophish: i think thats a native arm build, not a cross
<clever> jophish: yep
<clever> jophish: console=tty0 is still present, so the true cause isnt visible
<clever> the tricky part, is getting host deps in the right spots, so you can copy files and assemble the image without an arm device
<clever> jophish: using a combination of localSystem and something else, you can rig up cross-compiling
<clever> jophish: possibly, i have some basic support for it in there, but havent gotten it fully working due to various bugs
<clever> ah
<clever> jophish: i'm not sure the machine actually has v7 support
<clever> dramforever: since nix accepts that
<clever> dramforever: you could just use base64 everywhere?
<clever> yeah, when you duplicate console=, the printk's go to all, but /dev/tty and the output your expecting only goes to the last
<clever> jophish: so most of those others arent helping any, and the tty0 line may send the stage-1 elsewhere
<clever> jophish: it looks like only ttyS0 and ttyS1 are working
<clever> i suspect console=tty0 is directing the stage-1 banner to the hdmi output, if it even has drivers
<clever> try removing the ones at the end, one at a time
<clever> jophish: oh, your console='s
<clever> but somehow, lvm is showing up in dmesg?
<clever> it should at least print a stage-1 banner before failing
<clever> yeah
<clever> dramforever: and nix itself accepts both base64 and base32
<clever> dramforever: the nix-hash program can convert between hash types
<clever> oh, somehow didnt link it
<clever> jophish: everything prior to that is just setting env vars and testing for files in the initrd, nothing that should hang
<clever> jophish: line 67 of the above file prints the stage1 header
<clever> jophish: my only thoughts are to add a -v to the #! line, and maybe set -x, then boot that new image
<clever> jophish: i dont see any obvious signs its actually running stage1 or stage2, yet udev somehow loaded
<clever> not sure on what one, anything obvious if you add -vvvv ?
<clever> ekleog: try turning off auto-optimize and seeing if that helps
<clever> ekleog: is auto-optimize on?
<clever> can you test builtins.toFile in `nix repl` ?
<clever> nix should build it when you refer to <nix/unpack-channel.nix>, which nix-channel does internally
<clever> ekleog: is this on master?
<clever> *looks*
<clever> does it exist on the host?
<clever> and does --update fail or suceed?
<clever> ekleog: which user are you on?
<clever> ekleog: and does nix-channel --list show it as having a channel?
<clever> i hate those other ones, they make a total mess of your downloads folder
<clever> jophish: ah yeah, all of the unpack stuff assumes its a tar/zip with a single rootdir
<clever> jophish: yeah
<clever> so it instead unpacks, and operates on the hash of the files within
<clever> the reason it auto-unpacks, is because a lot of libraries for generating a zip include wildly varying timestamps, and then the hash of your zip keeps changing, so nix cant work
<clever> its a bit poorly named, fetchzip is just a modified form of fetchurl, that has added zip support, and will auto-unpack
<clever> fetchzip can do that then
<clever> jophish: does it have to be unpacked in the store, or just at build time?
<clever> dramforever: ive also memorized how it works, and never knew it was in the docs, lol

2018-10-27

<clever> thats better then any monitor i have can do!
<clever> i dont think the above is documented fully, but ive not re-read the docs in a while
<clever> file:///path/to/dir i believe will generate a pile of .nar.xz and .narinfo, that could then be rsync'd to an http server and act as a binary cache
<clever> s3://bucket will work for amazon S3 buckets
<clever> ssh://host and ssh-ng://host will work over ssh to a remote nix
<clever> local?root=/mnt/ will make it emulate a chroot before working on a localstore (it just prefixes all paths with /mnt/)
<clever> daemon is for connecting to nix-daemon via /nix/var/nix/daemon-socket/socket
<clever> local tells it to just open /nix/store/ directly
<clever> the store-uri for --from and --to supports several schemes
<clever> not really, but you can `nix copy --help` to get most of the args
<clever> --no-check-sigs do not require that paths are signed by trusted keys
<clever> i believe that will just copy everything in /mnt/nix/store to /nix/store/
<clever> pareidolia: nix copy --from local?root=/mnt/ --all
<clever> but being integrated, i dont really push it that hard
<clever> integrated intel is the most stable one i have currently
<clever> __red__: amd is where i have that bug that crashes on unplug
<clever> but nix copy and nix-copy-closure properly merge things
<clever> yeah, rsync would mess up the db
<clever> or nix-copy-closure --all
<clever> pareidolia: nix copy --all
<clever> is the problem specific to a port, a monitor, or where you lay it out on the multi-desktop setup?
<clever> are you able to swap which port each monitor is on?
<clever> does it persist even if you switch to older generations via the grub menu?
<clever> ah, that is strange
<clever> it behaves more like i was yanking out a hard-drive, lol
<clever> but if i use xrandr (or friends) to disable the monitor, i can safely unplug it
<clever> __red__: if i unplug a monitor without de-activating it, xorg crashes hard
<clever> __red__: gpu drivers are very hit and miss
<clever> then it parses it as json, and obeys the action key
<clever> and the very first thing it does is check if the msg begins with "@nix "
<clever> but then handleJSONLogMessage will handle it internally, and do something
<clever> if it returns true, the message is not printed to the user
<clever> Lisanna: and handleJSONLogMessage will try to parse every single message to see if it contains the lines you mentioned
<clever> Lisanna: this code handles the combined stdout+stderr+NIX_LOG_FD output (NIX_LOG_FD is where the status's go)
<clever> Lisanna: was just reading that ~5 hours ago
<clever> you can
<clever> i'm also looking into the plasma failure
<clever> sounds like a task for #nixos-dev
<clever> correct, only happens if something within the channel's tested set fails
<clever> schmittlauch[m]: looks like plasma and some random installer failures
<clever> andi-: does that test reveal any behaviour differences between user@host and ssh-ng://user@host in the machines file?
<clever> nice
<clever> then you could test it in a pure manner, and hydra can keep testing it in the future
<clever> andi-: it may also be possible to rig up a nixos-test using the existing qemu framework, to spin up a hydra and build slave, with the sortaSlow derivation as a job
<clever> andi-: though i currently lack access to the hydra i reproduced it on, the power blipped at home, and that box never came back online
<clever> andi-: its just things like builder@amd.localnet x86_64-linux /root/.ssh/nix 8 1
<clever> andi-: nope, one sec
<clever> arianvp: you can rig it up so ssh shares a single tcp socket (and auth session) between many ssh shells
<clever> arianvp: an ssh master connection can also help
<clever> andi-: that would likely simplify this, lol: [clever@system76:~]$ ssh ext.earthtools.ca -A -t 'ssh c2d -t "screen -xRRU"'
<clever> arianvp: ah
<clever> oops, andi-
<clever> arianvp: which program is that flag for?
<clever> (until you manually remove them, of course)
<clever> so you can do fun things like `ssh laptop -A 'ssh-add'` to copy the laptop keys into the gpg-agent, and they stay there, forever
<clever> ive also noticed, gpg-agent saves all added keys to its internal keyring, and they arent lost at shutdown
<clever> though i'm using normal secrets in the agent, not PKCS11
<clever> i also prefer the gpg-agent for ssh
<clever> are there any obvious errors if you strace both the ssh-agent process and the ssh-add proc?
<clever> i see
<clever> ah
<clever> arianvp: that action would likely require copying the secret out of the yubikey and into the ssh-agent, and then all security is gone and somebody could just clone the yubikey
<clever> bbl
<clever> hyper_ch: it can re-sync the hardware-configuration.nix with the currently mounted FS's
<clever> hyper_ch: because nixos-generate-config will overwrite hardware-configuration.nix
<clever> hyper_ch: i'm guessing you just set the fsType = "bind";
<clever> hyper_ch: both files use the identical syntax
<clever> hyper_ch: the same as in hardware-configuration
<clever> hyper_ch: but you probably want to do it in configuration.nix
<clever> pie_: do the naughty and just edit the nix store? lol, and then later `nix-store --verify --check-contents --repair` to fix it
<clever> pie_: that will build something in /mnt/nix/store and deal with all the chroots for you
<clever> pie_: NIX_REMOTE=local?root=/mnt/ nix-build ...
<clever> pie_: the new version will be at a new path
<clever> pie_: simplest thing is to just edit the nix expression and run nix-build on it
<clever> the install script at https://nixos.org/nix/install.sh would install nix and configure all of that automatically
<clever> Mo0O: ahh, that would explain some things
<clever> Mo0O: how did you install nix?
<clever> Mo0O: nix-channel --add https://nixos.org/channels/nixpkgs-unstable nixpkgs ; nix-channel --update
<clever> Mo0O: what does nix-channel --list and `echo $NIX_PATH` report?
<clever> feep: this returns true if x.y exists in the set
<clever> > { x.y = 2; } ? x.y
<clever> overrideAttrs is usually better
<clever> feep: nixpkgs.overlays = [ (import ./foo.nix) ];
<clever> aki_k: nixpkgs.config is in the page you linked, and its content is the same as config.nix, which comes from the nixpkgs manual
<clever> aki_k: nixpkgs.config.packageOverrides = pkgs: { sbt = pkgs.sbt.override { jre = pkgs.jre10; }; };
<clever> aki_k: try doing sbt = pkgs.sbt.override { jre = pkgs.jre10; }; in your packageOverrides
<clever> aki_k: ah, thats the purity of nix, sbt doesnt care what you installed, it will use the java that sbt was pointed at to begin with
<clever> aki_k: that result symlink sounds like the product of you running `nixos-rebuild build`, and its only updated when you run that
<clever> some of us are at nixcon and paying attention!
<clever> gchristensen: done!
<clever> dramforever: so it wont look like a proxy to users
<clever> dramforever: nginx can also be configured as a reverse proxy, and can probably cache
<clever> dramforever: so its slow for the first hit, and then caches for the future
<clever> dramforever: ah, you could also just run something like a caching squid proxy
<clever> or just run it directly on the http server
<clever> dramforever: you can then rsync that dir up
<clever> dramforever: `nix copy --to file:///local/dir` will create a directory with the same layout as a binary cache
<clever> ah
<clever> the PR i linked is just a handy diff, showing how it differs from a normal hydra setup
<clever> dramforever: here is the same config for the official hydra: https://github.com/NixOS/nixos-org-configurations/blob/master/delft/hydra.nix#L37
<clever> dramforever: hydra.nixos.org is likely using very similiar config
<clever> dramforever: that PR adds support to the iohk hydra, and the comments explain what setup has to be done manually

2018-10-26

<clever> __red__: as long as your not doing an efi install, you can just do a normal fdisk, mkfs, mount, nixos-generate-config, and nixos-install
<clever> Lisanna: *looks*
<clever> catern: thats in the nixpkgs manual
<clever> so you can do it either way
<clever> catern: -chardev should support both pty's and unix sockets
<clever> catern: add the 1st file above to imports, and then use the 2nd one to create the hvc device
<clever> catern: the nixos test framework adds a 2nd serial port, that goes directly to a root shell
<clever> catern: ahh
<clever> ah
<clever> catern: ah
<clever> catern: then using either ^a or ^[ (i forget) you can toggle between qemu and the serialport
<clever> catern: if you instead do `-serial mon:stdio` then it will multiplx you into both stdio and the qemu monitor
<clever> arianvp: using src = ./. at any point? does it have a node_modules left-over from testing?
<clever> avn: youll want to filter those out
<clever> avn: `nix path-info --all` will list every path that is valid, then you just remove those from the dir listing and you have the garbage
<clever> avn: have you seen `nix path-info --all` ?
<clever> avn: ah, are you more wanting to see what paths are still valid?
<clever> avn: oh, and i think domen make an improved NAR parser
<clever> avn: have you seen nix-store --restore ?
<clever> haitlah: hydra handles building each step seperately, and uploading every product to s3, so your compile-time things are also in the cache
<clever> haitlah: that might be your only choice, just make sure no secrets are in the store
<clever> catern: oh, and line 94 of the same file
<clever> catern: oh, also
<clever> catern: youll need to set users.users.root.initialPassword
<clever> catern: it has no default login password
<clever> catern: that script was to make qemu emulate a system with or without nvme, and/or uefi
<clever> haitlah: you could also just `nix copy --all` to upload the entire store
<clever> haitlah: you need t know what derivations you want in the cache, and upload them there, and its tricky to know what was compile-time only, which is where hydra helps out
<clever> catern: you can also use qemu-vm.nix as an example, and then make your own scripts like https://github.com/cleverca22/nix-tests/blob/master/kexec/simple-test.nix#L26
<clever> catern: maybe, not entirely sure
<clever> haitlah: ah, then you just want to `aws configure` and then `nix copy --to 's3://bucket?secret-key=...&region=...' /nix/store/foo`
<clever> catern: within that {} you could also override the kernel, but you would have to do a full kernel build for each run
<clever> catern: yep
<clever> haitlah: it also supports things like `nix copy --to s3://bucket/ /nix/store/path`
<clever> the nixos test framework also uses this to get things like profiling data out of the guest
<clever> shared*
<clever> so you can use SHARED_DIR to configure where /tmp/share goes on the host
<clever> xchg is also in that tmpdir, but not configurable
<clever> and TMPDIR is the result of mktemp -d
<clever> catern: if SHARED_DIR isnt set, it will use $TMPDIR/xchg
<clever> catern: yeah
<clever> catern: qemu-vm.nix is covering everything that nixos usually asserts on, which is why {} is "valid"
<clever> catern: and line 98 is the other end for 501-506
<clever> catern: line 501-506 is an example of how 9plan stuff is done from the nixos side
<clever> qemu-vm.nix overwrites the config for things like the rootfs
<clever> just {} would be a valid configuration.nix
<clever> you likely dont need any
<clever> this config controls what subset of .ko files get copied to the initrd
<clever> catern: nix just builds ALL the modules!
<clever> there is no clear pattern between module name and CONFIG_ param
<clever> catern: the kernel doesnt make that so easy
<clever> catern: like so
<clever> [root@system76:/etc/nixos]# nix eval -f '<nixpkgs/nixos>' -I nixos-config=./configuration.nix config.boot.initrd.availableKernelModules
<clever> that is what tells nixos what to include in the initrd
<clever> catern: eval the config.boot.initrd.availableKernelModules attr
<clever> that can work, as long as you know what all is needed
<clever> then you change the kernel, and they cant load...
<clever> catern: and those modules are baked into the initrd it builds
<clever> catern: one tricky part though, is that nixos needs a decent number of modules to boot (in qemu-vm.nix for example, it needs 9plan to mount the store)
<clever> catern: for for the kernel, youll have to mess with line 109 in that file
<clever> catern: if your configuration.nix has virtualisation.graphics = false; then it will link stdio to the serial port
<clever> catern: the only issue, is that this specifies a -kernel based on the boot.kernelPackages defined within it
<clever> catern: so you just want: nix-build '<nixpkgs/nixos>' -I nixos-config=./configurationn.nix -A vm
<clever> catern: build-vm just tells nixos-rebuild to build a certain attr of <nixpkgs/nixos>
<clever> catern: have you seen nixos-rebuild build-vm?
<clever> ah
<clever> Lisanna: not-os is one example
<clever> Lisanna: you are able to just run the module framework against non-nixos things
<clever> arianvp: i havent looked into the yarnNix attribute yet
<clever> ah
<clever> arianvp: you would need to run that part manually, and add its result to nixpkgs
<clever> arianvp: yarn2nix still needs IFD, even with a yarn.lock
<clever> arianvp: *looks*
<clever> arianvp: sure
<clever> yeah
<clever> if outputHashMode="recursive" then $out can be a dir, and its a hash over the NAR of that dir
<clever> if outputHashMode="flat"; then $out must be a single file, and outputHash is its hash
<clever> gleber_: then the $out will depend entirely on those 3 attrs, and the name attr
<clever> gleber_: if you define the 3 above attrs, any derivation can become fixed-output
<clever> gleber_: define a derivation with an, outputHashAlgo, outputHash, and outputHashMode ("flat" or "recursive")
<clever> gleber_: fixed-output derivations can do that
<clever> gleber_: so the derivation still has a system set on it, but its declaring that the linux and darwin builds have identical output, and it can mix&match them via the binary cache
<clever> gleber_: and system is just set to whatever pkgs its currently in
<clever> gleber_: all it really does is define the output hash, the same as fetchurl
<clever> gleber_: there is a font derivation in nixpkgs that does that
<clever> and then i wind up with 20 in /tmp/