2018-05-26

<clever> ldlework: and if you try wireshark on that port?
<clever> ldlework: is it giving an error?
<clever> ldlework: you just pick networkboot from f12, and type justdoit
<clever> ldlework: with things like my justdoit script, you can automate it further
<clever> ldlework: ive done exactly that with 2 netbooks and a router
<clever> ldlework: once you do install to the laptop, you can just set the targetenv to none, and give nixops an IP
<clever> i cant find a simple example
<clever> ldlework: it can be configured from nixops, check the nixops manual
<clever> ldlework: the aws firewall
<clever> ldlework: https://xkcd.com/1988/
<clever> ldlework: i would just skip docker entirely
<clever> ldlework: that can happen if you didnt allow port 22 in the firewall
<clever> nice
<clever> ldlework: its colliding with the ec2 host
<clever> ldlework: change the name of your key
<clever> let me look over it closer
<clever> ah
<clever> everything looks normal to me
<clever> ldlework: what is the error?
<clever> ldlework: none involves managing systems that already run nixos, and nixops doesnt care where the machine exists
<clever> ldlework: ive only used nixops with the none and aws backends
<clever> yeah
<clever> and if its a tar file, nix cant know the dir within the tar
<clever> it cant be, because it depends on whatever directory the unpackPhase created
<clever> and exactly that is all over the source stack2nix generates
<clever> 8077 postUnpack = "sourceRoot+=/cborg; echo source root reset to $sourceRoot";
<clever> so you can postUnpack = ''sourceRoot="$sourceRoot/foo"'';
<clever> the unpackPhase sets the sourceRoot to the source/ i'm guessing
<clever> ah, then you want to append to sourceRoot, one min
<clever> just src = builtins.fetchGit /media/projects/foo is enough to set the sourceRoot
<clever> jtojnar: the stdenv will auto-detect the sourceRoot for you
<clever> jtojnar: why are you trying to access its .name?, it doesnt have one
<clever> { outPath = "/nix/store/0s7gcniafv8i90p1ph26h7jqh1ycs4ll-source"; rev = "2982847b329505b41d3a038e1ff626a8b874920e"; revCount = 14130; shortRev = "2982847"; }
<clever> it already has a working outPath on it
<clever> "/nix/store/0s7gcniafv8i90p1ph26h7jqh1ycs4ll-source"
<clever> nix-repl> "${builtins.fetchGit ./iohk/cardano-develop}"
<clever> gchristensen: i dont think it can, and the name is already source
<clever> jtojnar: can you gist the whole nix expression?
<clever> jtojnar: what about just src = builtins.fetchGit /media/projects/foo;
<clever> which manages instances of AccelClass*
<clever> and kvm is configured within kvm_accel_class_init
<clever> gchristensen: it looks like qemu can only ever get kvm support via opening /dev/kvm
<clever> now i want to read the qemu source, lol
<clever> gchristensen: ah
<clever> id have to check the qemu source to confirm things
<clever> angerman: i think gchristensen mentioned yesterday that qemu uses hyperkit when you tell it to use kvm
<clever> das_j: did you also set enableParallelBuilding?
<clever> angerman: but that requires installing docker, while https://github.com/cleverca22/not-os/blob/master/linux-build-slave.nix can give a linux build slave with just a single nix-build command
<clever> das_j: you also have to set enableParallelBuilding = true; on the derivation
<clever> eacameron: but darwin nixpkgs has clang by default
<clever> eacameron: the linux kernel expressions in nixpkgs try to run gcc
<clever> eacameron: linux cant cross-compile from mac, it uses gcc, but darwin has clang
<clever> scription?
<clever> eacameron: the above expression is cross-compiling libc and gcc...

2018-05-25

<clever> eacameron: i think this is cross-compiling hello-world from darwin->linux
<clever> nix-repl> :b (import <nixpkgs> { crossSystem = { arch = "x86_64"; config = "x86_64-unknown-linux-gnu"; }; }).hello
<clever> eacameron: yep
<clever> eacameron: then you can now also set it up as a build slave using the right priv key
<clever> > xorg.libX11
<clever> eacameron: haskell might be harder, thats in angerman's territory
<clever> eacameron: oh, hmmm
<clever> eacameron: that should be equaly simple, once this works
<clever> eacameron: all of not-os, cross-built from x86 to arm
<clever> eacameron: its currently building...
<clever> eacameron: cross-compiling would also allow that
<clever> eacameron: yep, just nix-copy-closure it about
<clever> eacameron: then any key that works for the local machine, also wors for the vm
<clever> eacameron: you can also --arg sshKeyFile ~/.ssh/authorized_keys
<clever> ah
<clever> eacameron: ?
<clever> eacameron: try adding -vvvv to the ssh
<clever> eacameron: check the bash history to see what cmd you used
<clever> eacameron: did you give it a pub or priv key with --arg?, it needs to be the pub one
<clever> eacameron: you can now `ssh root@localhost -p 2222`
<clever> eacameron: testing the cross-compile stuff now...
<clever> eacameron: i suspect i can improve that with Sonarpulse's cross-compile stuff
<clever> eacameron: yeah
<clever> eacameron: yeah, thats a limitation it has right now, you need a linux slave to bootstrap it
<clever> __monty__: i tend to keep things pretty secure, so havent had any issues like that
<clever> __monty__: i tend to have one pair per machine
<clever> eacameron: but i could improve it to do what you thought
<clever> eacameron: some things in nixpkgs changed recently that break not-os easily
<clever> eacameron: its to lock in the nixpkgs rev
<clever> eacameron: it has to be ran in a checkout of not-os
<clever> eacameron: look at line 2 of the file
<clever> eacameron: keeping the readfile there allows you to just --arg sshKeyFile ~/.ssh/id_rsa.pub and it works
<clever> eacameron: the "${ could go, it was orignally inserted with other keys, that then got deleted
<clever> sqzlh: you need to run stack2nix in the root of your project, and it will generate a default.nix file
<clever> look into cabal2nix
<clever> sqzlh: id recomend just using cabal, and not stack, when on nix
<clever> eacameron: it now works fully
<clever> eacameron: it was lacking a `set -e`, so it kept going
<clever> eacameron: but it is now working
<clever> on mac, it fails
<clever> on linux, it just makes a sparse file that occupies zero space
<clever> eacameron: the `truncate -s 50g` command to make a dummy file for the rootfs
<clever> eacameron: boooo, support sparse files already!!!, booo
<clever> eacameron: truncate: failed to truncate 'rootdisk.img' at 53687091200 bytes: No space left on device
<clever> eacameron: hmmm, and darwin doesnt even have the truncate binary!
<clever> sqzlh: list based things should append most of the time
<clever> sqzlh: nix.nixPath = [ ... ];
<clever> eacameron: next big thing, is to try to cross-compile from mac to linux, an entire os
<clever> eacameron: i recently played with a few random platforms using things like (import <nixpkgs> { crossSystem = (import <nixpkgs/lib>).systems.examples.muslpi; }).hello
<clever> ooo, tests!
<clever> yeah
<clever> eacameron: then you wont need a linux slave to bootstrap
<clever> eacameron: next fun thing id want to look into is getting it to cross-compile the whole thing from darwin
<clever> eacameron: fix pushed
<clever> lol
<clever> i'll patch it to run at bootup
<clever> error: Package ‘e2fsprogs-1.44.1’ in /nix/store/7y6iyld88gq3c9xlyqn6wisy2i6im6nz-c29d2fde74d.tar.gz/pkgs/tools/filesystems/e2fsprogs/default.nix:37 is not supported on ‘x86_64-apple-darwin’, refusing to evaluate.
<clever> eacameron: oh, dang, its using mkfs.ext4, which doesnt work on darwin!
<clever> eacameron: and pushed
<clever> eacameron: done
<clever> MichaelRaskin: but yours went up
<clever> infinisil: was that a bug? lol
<clever> eacameron: the bot doesnt even care if the "user" exists
<clever> c++
<clever> eacameron: adjusting the script...
<clever> eacameron: or format it from the host side, before booting qemu
<clever> eacameron: line 150 will need to mount something else, and handle formatting it if its blank, then add a new drive to the qemu args
<clever> eacameron: line 150 mounts a tmpfs that will eventually become / and 160 creates a /nix/.overlay-store/rw that the overlay fs will use for writing to
<clever> oh, and by default, the root disk is a tmpfs, so it can only store half of that in the nix store
<clever> maybe also add memory ? 512 to the args
<clever> eacameron: yep
<clever> eacameron: id do { sshKeyFile }:
<clever> eacameron: easily
<clever> eacameron: i just went with the defaults that ssh-keygen had created ages ago
<clever> judson: the hash at the end is a chrome extension id
<clever> eacameron: yeah
<clever> srhb: ive also found ones in there for apps that i installed in chrome
<clever> eacameron: dont have access to a native mac where vm extensions would work properly
<clever> eacameron: it took 22 seconds to boot on a virtualized mac, vs 11 seconds to boot on a native linux host
<clever> doesnt require docker to be installed
<clever> eacameron: you can also use nix remote builders with the qemu script i linked above
<clever> i just never switched my network over to them
<clever> judson: i did :P
<clever> judson: yes
<clever> eacameron: if you run nix-build on this, you get a bash script that runs qemu, start that, and you have an ssh server on 127.0.0.1:2222 that gives you a linux build slave
<clever> eacameron: yesterday, i threw together a derivation that allows building linux things on darwin
<clever> eacameron: lol
<clever> and the lan.reverse below allows me to do reverse dns on 192.168.2.*
<clever> judson: that creates my own private TLD, .localnet, which is resolvable within my house
<clever> judson: one min
<clever> and it doesnt even tell you, it just silently never runs the job
<clever> gchristensen: the same problem also exists within hydra
<clever> and if it needs no features, that string is empty
<clever> gchristensen: beyond that, you could have a string that contains a fragment of the error, and insert it into a second one
<clever> gchristensen: i think concatStringsSep can be used in the 1 feature case as well
<clever> so you cant get the kvm feature on any aws machine
<clever> samueldr: most hosting providers like aws dont enable it on the host
<clever> eacameron: you can just lie about your slave supporting kvm, thats simpler
<clever> gchristensen: ah yeah, thats a simple change
<clever> eacameron: nested visualization doesnt work, but qemu will sanely fall back to software emulation (which is also a lot slower)
<clever> gchristensen: why wont it merge?
<clever> gchristensen: ?
<clever> builder@system76.localnet armv6l-linux,armv7l-linux,x86_64-linux,i686-linux /etc/nixos/keys/distro 4 1 big-parallel,nixos-test,kvm
<clever> eacameron: you need to tell nix what build slave has kvm support
<clever> gchristensen: the error nix gives should show that feature too
<clever> eacameron: `nix show-derivation /nix/store/qd29ichvjw3sfp9ygvv8sxk8kfqh8l3h-nixos-boot-disk.drv` what is listed under the required system features?
<clever> yeah, it should only be used when testing, when you want to force rebuilds
<clever> yep
<clever> eacameron: it just means you get a different .drv every time, so it never hits a binary cache
<clever> eacameron: its an eval-time thing, so its not really impure
<clever> ah, my build slave is turned off
<clever> error: a 'x86_64-darwin' is required to build '/nix/store/rgx06v6ix9h93rxla9r92gfc6v2cp27v-foo.drv', but I am a 'x86_64-linux'
<clever> it will now use the darwin build of bash to write the file
<clever> [clever@amd-nixos:~]$ nix-build -E 'with import <nixpkgs> { system = "x86_64-darwin"; }; writeText "foo" builtins.currentTime'
<clever> eacameron: oops, just writeText
<clever> eacameron: nix-build -E 'with import <nixpkgs> { system = "x86_64-darwin"; }; writeTextFile "foo" builtins.currentTime'
<clever> symphorien: set `preferLocalBuild = true;` in your derivation
<clever> symphorien: one min
<clever> eacameron: or the override LnL just gave
<clever> eacameron: nix-build -E 'with import <nixpkgs> { system = "x86_64-darwin"; }; enableDebugging hello'
<clever> one min
<clever> eacameron: nix-build '<nixpkgs>' -A hello --argstr system x86_64-darwin
<clever> sqzlh: any time nix is downloading something, it has to keep the entire thing in ram, until the dl is done
<clever> sqzlh: what command did you run?
<clever> ldlework: it will then boot into nixos on a ramdisk, ssh back in, run justdoit
<clever> ldlework: in that example, you build a special tar, upload it to literally any linux machine, then untar and run /kexec_nixos
<clever> look at the session.md in the same dir
<clever> the use-case in that git repo is doing kexec
<clever> sort of
<clever> snapshot support
<clever> dedup
<clever> compression
<clever> all data is immutable once on disk
<clever> it hashes all data stored so any corruption can be idenfied
<clever> the filesystem
<clever> so you can just boot the iso, login as root, and run justdoit
<clever> so if you include that module in your installer ISO, then its pre-installed on the ISO
<clever> that nixos module puts the compiled copy of the script into systemPackages
<clever> ldlework: justdoit
<clever> ldlework: you may also want to look at my justdoit.nix
<clever> yeah
<clever> ldlework: i always set uid = 1000; on myself in the config, for nfs reasons
<clever> ldlework: looks like a start
<clever> why does it have to be a nix-shell on the remote server?
<clever> and you cant run the nix-shell locally or eval the nix expressions remotely?
<clever> what exactly are you trying to do?
<clever> and you cant use the nix expressions normally?
<clever> Myrl-saki: why do you need to copy them?
<clever> Myrl-saki: what do you need to copy?
<clever> ldlework: sort of like https://github.com/NixOS/nixos-hardware ?
<clever> ldlework: configuration.nix and everything in the imports list, is treated as normal nixos modules
<clever> ldlework: you can also still make another file, that sets common groups of options
<clever> ldlework: yep
<clever> ldlework: thats how every single nixos service works
<clever> ldlework: its better to use options then to try and pass params to the module yourself
<clever> ldlework: look at the vim.nix in my nixos-configs repo
<clever> ldlework: create custom options
<clever> and one of those starting files is in the imports of configuration.nix, which is purely whats needed to boot and nothing else
<clever> ldlework: i just keep everything in a single repo, and have several starting files, like amd-nixos.nix and laptop.nix
<clever> it uses the same version of nixpkgs as pkgs, rather then the one in NIX_PATH
<clever> "
<clever> Myrl-saki: pkgs.path + "/pkgs/development/haskell-modules/configuration-ghcjs.nix
<clever> ldlework: only my router is on there in full, under deployments
<clever> probably
<clever> ldlework: https://github.com/cleverca22/nixos-configs is most of it
<clever> Myrl-saki: // can be used with package sets, after you supply the self argument
<clever> yeah, that looks like fixpoint stuff
<clever> ?
<clever> Myrl-saki: combine that, with the attrs in a derivation with the same name, basically
<clever> Myrl-saki: oh, also: nix show-derivation /nix/store/5bk1ddvdzxhixj3lhlaps2yvzhhgzg4h-blaze-builder-0.4.0.2-r1.cabal.drv
<clever> Myrl-saki: was going to mostly make educated guesses based on the contents

2018-05-24

<clever> bemeurer: yeah, you need nonfreeLicensing = true; as well
<clever> Myrl-saki: and its contents?
<clever> bemeurer: testing things on my end, it looks like it needs ffmpeg-full.override { nvenc = true; nonfreeLicensing = true; }
<clever> Myrl-saki: paste the full path of the drv file
<clever> bemeurer: i dont see an error in that gist
<clever> bemeurer: and it doesnt need cuda to be installed, it will automatically download it
<clever> bemeurer: nix-build -E 'with import <nixpkgs> {}; ffmpeg-full.override { nvenc = true; }'
<clever> bemeurer: you will need to write an override for ffmpeg that adds cuda support
<clever> bemeurer: it will never pick up libraries you have installed when rebuilding
<clever> bemeurer: nix doesnt work like that
<clever> bemeurer: nix-build '<nixpkgs>' -A ffmpeg-full
<clever> oh, didnt that break in nix 2.0?
<clever> bemeurer: nix-env should clearly say that
<clever> bemeurer: thats strange
<clever> bemeurer: and nix-env -q shows?
<clever> bemeurer: try adding -v
<clever> bemeurer: i think you want nix-env -iA nixos.cudatoolkit
<clever> infinisil: it may need to render errors better
<clever> > throw "custom error"
<clever> > cuda
<clever> Myrl-saki: thats one way
<clever> Myrl-saki: pong
<clever> Myrl-saki: it both makes it easyer to read, and also harder to parse wrong
<clever> foo = with pkgs; [ a b c ]; rather then with pkgs; { foo = [ a b c ];
<clever> other then `with lib;` i keep my with statements as narrow as possible
<clever> infinisil: let has higher scope, so `with pkgs;` doesnt overwrite things on you
<clever> be careful when writing identity :P
<clever> > a:a
<clever> > a: a
<clever> URI
<clever> close!
<clever> Myrl-saki: can you guess why what i just did happened?
<clever> > a:b
<clever> `let foo = bar; in` == `with rec{ foo = bar; };` essentially
<clever> i think let is the only place it works
<clever> Myrl-saki: something fun i did 7 months ago: https://gist.github.com/cleverca22/60c46368afb999c16224fc17af5c538d
<clever> Myrl-saki: yeah, thats my guess too
<clever> Myrl-saki: grep nixpkgs to see how hackage-packages.nix is meant to be loaded
<clever> Myrl-saki: and what arguments does nixpkgs/pkgs/development/haskell-modules/hackage-packages.nix expect?
<clever> Myrl-saki: what is the type of ((import <nixpkgs> {}).callPackage ./pkgs/ghcjs {})
<clever> infinisil: but scopedImport cant
<clever> infinisil: behind the scenes, import can cache the value that results from importing a file
<clever> infinisil: i believe scopedImport breaks caching in nix
<clever> Myrl-saki: it wont need to inspect the types of both params to know which append to use
<clever> Myrl-saki: you want overrideCabal
<clever> Myrl-saki: can you gist the file?
<clever> Peaker: if it dies for any other reason, nix will delete all output it had created
<clever> Peaker: only if a build exits with code 0 will nix keep its results
<clever> Peaker: the network namespace also stops you from doing network access
<clever> Peaker: its using mount, user, and network namespaces, which act mostly like containers
<clever> Peaker: with the sandbox on, you cant even search /nix/store/*/include/
<clever> Peaker: reading things like /usr/include (if it exists) and /etc are still an issue, but the nix sandbox can do that (it uses namespacing)
<clever> Peaker: and the builds are ran under a dedicated user like nixbld1, that has limited write access
<clever> Peaker: only if your very naughty and search /nix/store/*/include/ will you ever find something you didnt ask for
<clever> Peaker: the paths are hashes under /nix/store/, and if you dont ask for something, you dont get its hash
<clever> elvishjerricco: inherit (pkgs) foo bar baz; doesnt work on a list
<clever> elvishjerricco: how would you do: with pkgs; [ foo bar baz ]; then?
<clever> __monty__: ive had issues in the past, where i had to compile a whole haskell program, just to read 1 field in a dhall file, lol
<clever> __monty__: that looks useful
<clever> __monty__: link?
<clever> ldlework: yeah
<clever> ldlework: it starts with an empty home
<clever> ldlework: i just have nixos configure most things properly, system wide
<clever> ldlework: i dont use home-manager
<clever> ldlework: yeah