2018-03-15

<clever> `RISC-V Linux User Emulation`
<clever> boomshroom: oh, look at the most recent commit in this dir: https://github.com/riscv/riscv-qemu/tree/riscv-all/linux-user
<clever> boomshroom: unknown
<clever> boomshroom: https://github.com/riscv/riscv-qemu is also of interest
<clever> you could also fork my repo, and file a PR to add the qemu-user's
<clever> yep
<clever> boomshroom: but you can use mkForce to wipe out the values ive set
<clever> boomshroom: the type is listOf str, so nix will automatically append each value together
<clever> jtojnar: i just started a nix build that appears to involve 546 derivations...
<clever> boomshroom: it also patches nix to even support that option
<clever> boomshroom: yep
<clever> foldingcookie: things may have changed since that commit
<clever> !tofu
<clever> foldingcookie: launching X manually
<clever> foldingcookie: yep, systemd doesnt allow that
<clever> aborsu: i have it working on my system
<clever> and you can be sure the prompt is coming from the system
<clever> systemd will then re-spawn the login prompt
<clever> with the right key sequence, you can kill every process on the current tty
<clever> srhb: but thats where the sak comes in, its a lot like ctrl+alt+del on windows
<clever> srhb: and yeah, you could just run a fake terminal, and fake the login prompt
<clever> foldingcookie: Xorg needs root to access most gpu's
<clever> foldingcookie: systemd doesnt really like that
<clever> foldingcookie: services.xserver.enable = true;
<clever> foldingcookie: it generates a bash script, that runs the finished nixos build under qemu
<clever> foldingcookie: there is also the fun `nixos-rebuild build-vm`
<clever> boomshroom: simply add qemu.nix to the imports of your configuration.nix, and set qemu-user.arm = true; and it configures everything!
<clever> ssh keypair or go home
<clever> i prefer disabling password auth entirely
<clever> ah, it might be a side-effect of how the interactive auth works
<clever> to remind you which password you should be inputing
<clever> ive found that if you get the password wrong a few times, ssh will start showing the user@host
<clever> exec "/nix/store/qgrslxc2flrc5yr47fdv0w5mk7sfx5ql-openjdk-8u172b02-jre/bin/java" -jar "${extraFlagsArray[@]}" "$@"
<clever> -r-xr-xr-x 1 root root 188 Dec 31 1969 /nix/store/d45qjlxala1yrisjj1wn70q9wificfgf-java-wrapped
<clever> this derivation produced the following outputs: out -> /nix/store/d45qjlxala1yrisjj1wn70q9wificfgf-java-wrapped
<clever> nix-repl> :b runCommand "java-wrapped" { buildInputs = [ makeWrapper ]; } "makeWrapper ${jre}/bin/java $out --add-flags -jar"
<clever> boomshroom: runCommand "java-wrapped" { buildInputs = [ makeWrapper ]; } "makeWrapper ${java}/bin/java $out --add-flags -jar"
<clever> for example?
<clever> boomshroom: the other just renames the program to .foo-wrapped, and then calls the 1st
<clever> boomshroom: there is both makeWrapper and wrapProgram, i can never remember which is which, but one takes a pair of source and destination, so destination becomes a bash script that runs source
<clever> nope
<clever> tilpner: you just need to be carefull when the foreign binary is something ldd can be ran on
<clever> thats using a dynamic bash, so ldd returns bash's deps, not the foreign binary deps, but in java's case, ldd doesnt make sense
<clever> tilpner: the writeShellScriptBin in your paste
<clever> and suddenlllllllly, you have x86 libraries in your arm initrd!
<clever> then the x86 ld.so parses those magic vars, and dumps qemu's deps!!
<clever> binfmt-misc then runs qemu against that ld.so
<clever> tilpner: ldd just sets some magic variables, and runs the local ld.so against the file in question
<clever> tilpner: you must also avoid dynamically linked binaries if your emulating another platform, due to how ldd works
<clever> hmmm, now to wrangle some c++ types!
<clever> src/libstore/derivations.cc:59:36: error: conversion from 'nix::Setting<std::__cxx11::list<std::__cxx11::basic_string<char> > >' to non-scalar type 'nix::StringSet {aka std::set<std::__cxx11::basic_string<char> >}' requested
<clever> yeah
<clever> tilpner: boot.binfmtMiscRegistrations didnt exist when i wrote that
<clever> it looks like the Settings class has been entirely rewritten since i made this patch
<clever> boomshroom: nope, thats built into the kernel
<clever> hmmm
<clever> src/libstore/derivations.cc:59:62: error: 'class nix::Settings' has no member named 'get'; did you mean 'set'?
<clever> but nixos now has a boot.binfmtMiscRegistrations to handle things for you
<clever> boomshroom: line 31-40 generates a bash script to configure it
<clever> boomshroom: this is the old code for handling it: https://github.com/cleverca22/nix-misc/blob/master/qemu-user.nix#L8
<clever> boomshroom: you also need to know what bits to match against
<clever> boomshroom: i'm writing a nixos module to fully automate it
<clever> symphorien: yep
<clever> boomshroom: the qemu-user expressions i wrote include a register script written in bash
<clever> boomshroom: ive even used it to run a closed-source x86 program on a raspberry pi once
<clever> boomshroom: yeah, but performance will be worse for the build
<clever> boomshroom: i'm in the middle of writing that module right now
<clever> boomshroom: and now i can run arm binaries!
<clever> boomshroom: all i did was set qemu-user.arm = true; https://gist.github.com/cleverca22/ac838bd5c8781cb01768e5f55a3c5496
<clever> boomshroom: i just set qemu-user.arm = true; on my laptop, its doing a nixos-rebuild switch
<clever> lets see, boot.binfmtMiscRegistrations
<clever> qemu-user can trick the native toolchain
<clever> i should finish my qemu module
<clever> if you dont, it fails like this
<clever> error: a 'armv7l-linux' is required to build '/nix/store/vpl6gfsq8q6rvb5c1983vi65b67bs51q-bootstrap-tools.drv', but I am a 'x86_64-linux'
<clever> but if you configure a build slave in /etc/nix/machines, it can just build them
<clever> not that i'm aware of
<clever> boomshroom: it is currently downloading armv7l builds of everything from the binary cache, and sources for some
<clever> boomshroom: i aimed nixops at this file, https://gist.github.com/98f4e81bfc9750b3484d5b477c1e2d11 and am now running `nixops deploy --build-only`
<clever> tfc[m]: interactive and non-interactive shells source different files
<clever> tfc[m]: i think you need to add the nix logic to .bashrc
<clever> tfc[m]: what OS is the remote machine?
<clever> boomshroom: ive also used it to refer to files within nixpkgs in a pure manner, tarball = pkgs.callPackage (pkgs.path + "/nixos/lib/make-system-tarball.nix") {
<clever> boomshroom: so you could import (pkgs.path) { new config };
<clever> boomshroom: pkgs.path is the path the pkgs set came from
<clever> tfc[m]: `strace -ff -o logfiles -e execve nix copy --to ...` can you gist the logfiles this generates?
<clever> tfc[m]: and it works when you just `ssh bla` ?
<clever> tfc[m]: and yeah, id also consider that a bug in nix, an issue should be filed
<clever> tfc[m]: https://gist.github.com/cleverca22/07dc4f902b5f6d998965887ff861f406 for example, line 1 is what it matches on from the cli (ssh example.com), then line 2 overwrites the host it actually connects to (optional, but handy for aliases), and line 3 changes the default username
<clever> tfc[m]: you can specify a default username in ~/.ssh/config
<clever> boomshroom: this is where nix imports nixpkgs: https://github.com/NixOS/nixpkgs/blob/master/nixos/modules/misc/nixpkgs.nix#L64
<clever> nixpkgs.config sets the config arg, but crossSystem is a sibling of config, not a child
<clever> which gives me an idea...
<clever> and it will farm it out to a machine that can run the right toolchain
<clever> boomshroom: i'm not sure how to set the arch for nixos cross, but for nixos native, you can just do nixpkgs.system = "i686-linux"; for example
<clever> boomshroom: for native or cross compiling?
<clever> boomshroom: the nixos side of things works on pretty much any arch linux supports, natively and cross, the issue is more about building the nixpkgs side of things
<clever> they insist on natively built things
<clever> and tools like nix-env dont play nicely with cross-compiling currently
<clever> native compiling is simpler, but it can also be cross-compiled with some effort
<clever> boomshroom: it also runs on 32bit x86, 32bit arm, and 64bit arm
<clever> the above opens the default.nix for the hello package in $EDITOR
<clever> try: nix edit -f '<nixpkgs>' hello
<clever> nix edit has been handy
<clever> i havent looked into how 2.0 works in depth yet
<clever> but nix-env is the only tool to ever use this layout, so its rather useless the instant you want to just build or shell
<clever> the only purpose for channels vs channels_root, is that only root can write to the channels_root directory, so each user can maintain control over their group of channels, but can also share with others
<clever> and it has no depth limit, so ~/.nix-defexpr/a/b/c/d/e/f/g/h/i/j/k/l/m/n/o/p/q/r/s/t/u/v/w/x/y/z/default.nix becomes a channel called z!
<clever> if a directory contains a default.nix, it wont be recursed into
<clever> in both cases, the <name> becomes what you refer to it as in nix-env
<clever> it accepts both <name>.nix and <name>/default.nix
<clever> depends on if you want to store a collection of channel-like things or not, and if they have any files they also need
<clever> infinisil: yep
<clever> nix.nixPath i think it was
<clever> you can also set it in configuration.nix
<clever> electroc1t: <nixos> and <nixpkgs> map to the same path by default
<clever> electroc1t: thats the nixos channel on root, which has a nixpkgs symlink to .
<clever> tilpner: but your calling it with {} already, so it cant
<clever> tilpner: when you use nix-env --arg config '{allowUnfree = true;}' it will pass it to the function it loaded
<clever> though that breaks nix-env -iA nixpkgs.hello --arg config ...
<clever> nix-env expects it to return a function
<clever> tilpner: i think you need to drop the {}
<clever> tilpner: this creates a foo channel, so i can nix-env -iA foo.hello
<clever> import /home/clever/apps/nixpkgs
<clever> [clever@amd-nixos:~]$ cat .nix-defexpr/test/foo/default.nix
<clever> i recently made a PR to split rocksdb up, the static and dynamic libraries where in the same output as the headers!
<clever> probably
<clever> Slabity: i think
<clever> Slabity: nix-env -i -E '{ ... }: with import <nixpkgs> {}; hello.overrideDerivation (drv: { src = fetchurl { ... }; })'
<clever> but if you instead set programs.man.enable = false;, it omits all the man pages
<clever> leotaku: this augments systemPackages, so it installs the .man of every package
<clever> leotaku: one min
<clever> infinisil: meta.outputsToInstall
<clever> joko: the kernel packages are complex, because nixpkgs needs to generate a set of module packages around the kernel
<clever> joko: boot.kernelPackages = (import <nixpkgs2>{}).linuxPackages; would be a way to mix the new nixpkgs into an old nixos
<clever> gchristensen: do you have access?, can you try restarting the build? https://hydra.nixos.org/build/70719143
<clever> but the build log shows no errors, and appears to be passing
<clever> qtquickcontrols2 isnt building
<clever> Avaq: the graphical iso, ova image, and plasma5 tests have been failing because of a dependency
<clever> Avaq: click on the hydra job for it
<clever> gchristensen: downloads from cache.nixos.org without trouble
<clever> there, that disables nix.conf!
<clever> NIX_CONF_DIR=/home/clever NIX_REMOTE=local?root=/tmp/fakestore nix-store -r /nix/store/bm7pb1s7rx1ad80706b5xqrznq7fgpgx-gcc-7.3.0
<clever> my caching is setup too well
<clever> and the desktop is stealing a copy off the router!
<clever> gchristensen: heh, it snuck over to my desktop and stole a copy from that machine, lol
<clever> gchristensen: running NIX_REMOTE=local?root=/tmp/fakestore nix-store -r /nix/store/bm7pb1s7rx1ad80706b5xqrznq7fgpgx-gcc-7.3.0
<clever> and desktop
<clever> gchristensen: that storepath is already valid in my laptop
<clever> foldingcookie: and once it has done that, it has to check every .narinfo file on cache.nixos.org for the outputs your missing, to figure out what it can download
<clever> foldingcookie: it has to evaluate every package in nixpkgs, which requires reading all the default.nix files, and running all the expressions, on your machine
<clever> !-A
<clever> which may be handy for more complex things when you cant just copy/paste the whole command
<clever> nh2[m]: though .overrideDerivation can modify it, the way you linked
<clever> nh2[m]: no real difference
<clever> mguex: and then opening a unix socket inside the sandbox, and forwarding things both ways, along the nix protocol
<clever> mguex: there will need to be support for multiple bi-directional channels, similar to how the stdout is forwarded to the client
<clever> robstr_: strange
<clever> robstr_: nothing at all?
<clever> robstr_: does dmesg say anything?
<clever> nh2[m]: yeah, that should work
<clever> nh2[m]: i dont think there is, but you can use something like the wrapped i gave, to just make a symlink at any path pointing to any path
<clever> nh2[m]: in a derivation?
<clever> Lisanna: ah, then without the quotes is best
<clever> Lisanna: also, why do you want that basename?
<clever> nix-repl> baseNameOf ./math.nix
<clever> "math.nix"
<clever> nix-repl> baseNameOf "${./math.nix}"
<clever> "jylc9ipi7icxdidi2payk0vpj1ibh5b2-math.nix"
<clever> infinisil: and each item is installed as a seperate entry, so i can still update just one
<clever> infinisil: i find that handy, i have a custom set of packages in my overrides, and i just nix-env -iA nixos.mystuff which is a set
<clever> infinisil: ive used it recently, and have even written a weechat service for nixos, but i still use irssi as my primary client
<clever> robstr: if you already navigate irc with commands, it should be easy, its just /join, /part, /connect, /server stuff
<clever> robstr: i use irssi, its curses based
<clever> ah yeah
<clever> yegor]: ah, if your in a shellHook you can just use $(pwd)
<clever> yegor]: what do you plan to do with that path?
<clever> yegor]: why do you need the working directory?
<clever> ahhh, right
<clever> yegor]: builtins.toString ./.
<clever> and with the sheer size, you have even less reason to share a nibble
<clever> LnL: for ipv6, you flip it backwards, and each nibble (4 bits) is a segment, less need to share a dns server
<clever> LnL: for ipv4, you flip the ip backwards, and each byte is a seperate segment, so you have to delegate the "subdomains" and if 2 people share a byte, somebody has to store both sets of answers
<clever> LnL: yes
<clever> lol
<clever> lol
<clever> infinisil: correct
<clever> i wrote a c++ prog you run via IFD that handles it
<clever> srk: so you have to expand it until the total length is right
<clever> srk: the :: in the middle expands to a non-fixed number of 0's
<clever> have fun doing that in nix :P
<clever> infinisil: 2607:f8b0:4006:819::200e has to be turned into e.0.0.2.0.0.0.0.0.0.0.0.0.0.0.0.9.1.8.0.6.0.0.4.0.b.8.f.7.0.6.2.ip6.arpa
<clever> infinisil: try a PTR record
<clever> for both v4 and v6
<clever> these scripts deal with creating forward and reverse dns records
<clever> infinisil: oh, one min
<clever> electroc1t: not directly, the file on 20-28 isnt expandable, but you can use services.xserver.displayManager.sessionCommands = "${xorg.xrdb}/bin/xrdb -merge ${./yourfile}"; to just inject more things in
<clever> infinisil: just an example value
<clever> the syncthing module can also be updated to chmod it to a value you enter into the config
<clever> systemd.services.syncthing.preStart = "chmod 777 /var/lib/syncthing";
<clever> and it gets ran as root, so it can chmod things
<clever> tilpner: you can set this from configuration.nix, and it will be sanely merged in
<clever> tilpner: systemd.services.<name>.preStart
<clever> tilpner: you can add a prestart script to the service
<clever> silly perl and its weird if statements
<clever> ahh
<clever> just that createHome is set to true
<clever> there is no check in there to see if it doesnt exist
<clever> tilpner: what is that path?
<clever> you can also replace <nixpkgs> with a fetchurl against a given revision, so it ignores $NIX_PATH
<clever> tobiasBora: then it obeys ./config.nix and ignores ~/.config/nixpkgs/config.nix entirely
<clever> tobiasBora: its generally better to just do import <nixpkgs> { config = import ./config.nix; }

2018-03-14

<clever> Orbstheorem[m]: and if you run nix-shell against the .drv file, you will get into an env thats fairly similar, just missing a few env vars
<clever> Orbstheorem[m]: if you build with --keep-failed, it will leave the /tmp/nix-build directory behind, so you can inspect what remains
<clever> Slabity: builtins.fetchTarball doesnt need a hash
<clever> ottidmes: heh
<clever> woffs: there was a bug in nix that allowed it, i suspect Slabity's problem is because it was fixed
<clever> Slabity: nope
<clever> Slabity: yep
<clever> Slabity: nix-instantiate --eval '<nixpkgs>' -A lib.nixpkgsVersion --arg overlays '[]'
<clever> Slabity: nix-instantiate --eval '<nixpkgs>' -A lib.nixpkgsVersion
<clever> Slabity: what does `type nix-build` say?
<clever> gchristensen: the `--arg config {}` disables config.nix
<clever> slabity[m]: what about something simple like `nix-build '<nixpkgs>' -A hello --arg config '{}'`
<clever> slabity[m]: or just picking an older generation in the grub menu
<clever> slabity[m]: what does `sudo nix-channel --list` report?
<clever> Ralith: you can just nix.package = pkgs.nixUnstable; in configuration.nix
<clever> jtojnar: that will create a new store in the chroot directory, with the complete closure
<clever> jtojnar: with nix2, you can `nix copy --no-check-sigs --to local?root=/home/clever/chroot/ /nix/store/foo`
<clever> tobiasBora: if you remove that, and create the /home/$X/ it would work without them, i think
<clever> tobiasBora: i suspect that the NIX_REMOTE=local?root= uses namespaces to get the job done
<clever> tobiasBora: you may need namespace support to build it, and i suspect proot makes it worse
<clever> tobiasBora: download nixpkgs somehow (git clone, or nix-channel), and then point NIX_PATH at it
<clever> tobiasBora: you still need to setup NIX_PATH via nix-channel
<clever> tobiasBora: thats a nix 2.0 command
<clever> krey: it stops garbage collection after it deletes 5gig of garbage
<clever> krey: it wont help in your case, but i also prefer this variant: [root@amd-nixos:~]# nix-collect-garbage --max-freed 5g
<clever> krey: -d with root will delete more
<clever> krey: with or without root?
<clever> krey: to start with, try just a normal `nix-collect-garbage` and see how much you regain
<clever> tobiasBora: yeah, i didnt think these 2 modes could be mixed at first
<clever> and it expects you to chroot into rootfs (or copy it over to /home/$X/nix/store)
<clever> it puts the final files in /home/clever/rootfs/home/$X/nix/store
<clever> NIX_REMOTE=local?root=/home/clever/rootfs/ NIX_CONF_DIR=/home/$X/etc/nix NIX_LOG_DIR=/home/$X/nix/var/log/nix NIX_STORE=/home/$X/nix/store NIX_STATE_DIR=/home/$X/nix/var nix-build -E 'with import <nixpkgs> {}; nix.override { storeDir = "/home/'$X'/nix/store"; stateDir = "/home/'$X'/nix/var"; confDir = "/home/'$X'/etc"; }'
<clever> i am now generating a store, for an example home, that doesnt exist
<clever> building '/home/tobias/nix/store/14dp8ydgcxzfsa3k2pihjsw9lr4sc6pf-bash-4.4.tar.gz.drv'...
<clever> why?
<clever> i think i can do the entire thing with nix, on any machine with nix
<clever> i think i have a fix for that too
<clever> is that you have to replicate the $HOME of the target
<clever> the only remaining issue with the above, which i might be able to solve
<clever> tobiasBora: also, i think debian can be fixed with: sudo sysctl -w kernel.unprivileged_userns_clone=1
<clever> tobiasBora: it is now building the world, at this new path
<clever> building '/home/clever/nix/store/40brlg8zm8r6z5m3cnppaicrqwzhvs5f-binutils-2.28.1.tar.bz2.drv'...
<clever> tobiasBora: this changes where nix searches for nix.conf
<clever> [clever@system76:~]$ NIX_CONF_DIR=/home/clever/etc/nix NIX_LOG_DIR=/home/clever/nix/var/log/nix NIX_STORE=/home/clever/nix/store NIX_STATE_DIR=/home/clever/nix/var nix-build -E 'with import <nixpkgs> {}; nix.override { storeDir = "/home/clever/nix/store"; stateDir = "/home/clever/nix/var"; confDir = "/home/clever/etc"; }'
<clever> tobiasBora: and now my host nix.conf breaks it
<clever> error: opening file '/etc/nix/secret-key-file': Permission denied
<clever> tobiasBora: this version, will compile a glibc&nix, that expect the store to be at /home/clever/nix/store, so proot isnt required
<clever> [clever@system76:~]$ NIX_LOG_DIR=/home/clever/nix/var/log/nix NIX_STORE=/home/clever/nix/store NIX_STATE_DIR=/home/clever/nix/var nix-build -E 'with import <nixpkgs> {}; nix.override { storeDir = "/home/clever/nix/store"; stateDir = "/home/clever/nix/var"; confDir = "/home/clever/etc"; }'
<clever> roconnor: undefined reference is a link-time failure, you need to pass -lsodium to the linker
<clever> tobiasBora: not sure yet, still testing things
<clever> roconnor: no, the stdenv handles that
<clever> tobiasBora: with NIX_REMOTE=local?root, it basicaly fakes a chroot, during the nix build, so the build expects that path to become / at runtime, requiring proot
<clever> roconnor: libsodium, the stdenv handles the rest for you
<clever> the NIX_REMOTE one still needs proot to function
<clever> tobiasBora: or this, not sure yet
<clever> [clever@system76:~]$ NIX_REMOTE=local?root=/home/clever/nix/store nix-build -E 'with import <nixpkgs> {}; nix.override { storeDir = "/home/clever/nix/store"; stateDir = "/home/clever/nix/var"; confDir = "/home/clever/etc"; }'
<clever> tobiasBora: something like this i think
<clever> [clever@system76:~]$ NIX_STORE=/home/clever/nix/store nix-build -E 'with import <nixpkgs> {}; nix.override { storeDir = "/home/clever/nix/store"; stateDir = "/home/clever/nix/var"; confDir = "/home/clever/etc"; }'
<clever> manveru: it has to clone the given ref (maybe the default one), then checkout the commit you want, and deal with all the history
<clever> tobiasBora: and some of those args may need tweaking, i havent used them in a while
<clever> tobiasBora: i mis-typed NIX_REMOTE as NIX_STORE
<clever> manveru: git doesnt allow you to fetch a specific revision, so it has to fetch a specific branch, and then work its way backwards in history
<clever> tobiasBora: the NIX_REMOTE part will put everything, including glibc, into that new path
<clever> tobiasBora: yep
<clever> tobiasBora: combine it with NIX_STORE=local?root=/home/clever/nix/store and you can build it in that path
<clever> tobiasBora: that will generate a nix that expects the store to be at that path
<clever> tobiasBora: nix.override { storeDir = "/home/clever/nix/store"; stateDir = "/home/clever/nix/var"; confDir = "/home/clever/etc"; }
<clever> srhb: i have no idea who so is, but my tab-completion likes him, lol
<clever> so: i think overrideAttrs shouldnt break passthru
<clever> tobiasBora: and for the paranoid admins, you could maybe use https://directory.fsf.org/wiki/Pam_chroot
<clever> also, https://nixos.org/nix/install doesnt really need root, if the admin simply does `mkdir /nix && chown tobias /nix`, you can install with zero sudo access
<clever> proot or namespaces get around that issue
<clever> so every user has to have their own custom build
<clever> and it wont work in /home/tobias/nix
<clever> the problem there, is that the compile only works in /home/clever/nix
<clever> and the user runs that to enter the root