2017-07-01

<clever> that should get the exact same pkgs as configuration.nix
<clever> how are you loading configuration.nix?, that should be handled by nixos
<clever> how are you importing things?
<clever> so you just need to control whatever nixpkgs= in $NIX_PATH points to
<clever> which always starts with <nixpkgs
<clever> so it will depend on where nixos-rebuild looks for nixos
<clever> nixos will always grab that pkgs from ../../.. (the same tree nixos is in)
<clever> ah
<clever> the one in shell.nix can be an example, but you would need to use grep to check for the entry existing
<clever> only option there is to write your own nix expression that modifies ~/.nix-channel further
<clever> release.xml was docs for the admins, on how to fork and make a release channel
<clever> and then it just stays like that
<clever> when they fork nixpkgs to make the nixos-17.03 channel, they manualy change the default
<clever> yep!
<clever> oh, i think i know whats going on
<clever> the release notes do make mention of it
<clever> nixos/doc/manual/development/releases.xml: <literal>system.defaultChannel</literal> attribute in
<clever> i have no idea why its hidden
<clever> but because it has internal=true, its hidden from the docs
<clever> the same way as any other option
<clever> but that will only trigger once, when /root/.nix-channels is missing
<clever> tilpner: this would be the best way to configure the channels: https://github.com/NixOS/nixpkgs/blob/master/nixos/modules/programs/shell.nix#L43
<clever> --add doesnt touch defexpr, its --update that handles that
<clever> ah
<clever> tilpner: ~/.nix-channels is just a config file for nix-channel and nothing else, nix-env reads ~/.nix-defexpr/

2017-06-30

<clever> TCP is sort of 2 uni-directional connections, that happen to share the same source/dest port
<clever> if that symlink ever gets modified or deleted, nix will detect it and clean things up automatically
<clever> which is also what the 'result' links from nix-build are
<clever> Infinisil: this will create an indirect root to a given storepath
<clever> Infinisil: nix-store -r /nix/store/foo --add-root ~/configfile --indirect
<clever> and will not allow you to break its target
<clever> but if the symlink is in the nix store, nix keeps that backref in the db.sqlite
<clever> and yeah, roots then keep that place alive
<clever> nix prevents such a problem by just forcing an application to put all of its junk in a single place
<clever> static analasys of the code can compute such things
<clever> Infinisil: it works by measuring the pointer difference between the parent and an element, then just subtracting it from every pointer!
<clever> Infinisil: i have noticed, the linux kernel uses a weird macro, to get the address of a parent struct
<clever> yeah, if you already have it, then nothing really happens
<clever> ah
<clever> there is no way for your nix to know which channel or commit it came from
<clever> without even having a copy of the nix expressions that made it
<clever> that lets you fetch anything from the binary cache
<clever> Infinisil: another thing, try running this: `nix-store -r /nix/store/m1031rczja4jbnrikkk71yzg6051sn61-git-2.12.2`
<clever> and at this point, it has no clue what nix file it came from
<clever> then when nix wants to download from the binary cache, it will take the hash from the $out path, and query cache.nixos.org/<hash>.narinfo
<clever> that drv will also reference the drv of the buildtime deps
<clever> Infinisil: it will then generate a .drv file that serializes the map, and the $out path, and store the drv at hash(value)=value in /nix/store/foo.drv
<clever> Infinisil: it will then hash that map, to compute the $out path
<clever> Infinisil: when you eval builtins.derivation { ... } in nix, it will flatten the attribute set down to a string=string map
<clever> nix has no clue what channel a given derivation came from
<clever> so now, the first 2 levels only have 2 character filenames, and act as an index
<clever> make the following path, xs/7p/3w2iqvv0y1d9r79wb6jrp4pm1h93.narinfo
<clever> Infinisil: rather then have a single directory with just xs7p3w2iqvv0y1d9r79wb6jrp4pm1h93.narinfo
<clever> Infinisil: the only thing i can think of, is to do what git does
<clever> its a cache of everything that hydra has ever built, since the project began, with zero garbage collection
<clever> the issue right now, is that cache.nixos.org isnt for any 1 channel
<clever> but the end-user would need to download that directory listing entirely, to map foo.narinfo to the hash of foo.narinfo
<clever> you could do it 100% over ipfs
<clever> and then posted that directory on IPNS
<clever> if you stored the list of narinfo files in IPFS
<clever> imagine having to download the directory listing for cache.nixos.org, before you can download 1 file
<clever> but i think the directory is single blob of name + ipfshash pairs
<clever> so it maps to a directory in the IPFS space, where everything is hash(value)=value
<clever> IPNS is a pubkey = ipfshash + signature
<clever> but there would be performance costs there
<clever> yeah, you could potentialy use IPNS to store the narinfo files
<clever> but the system would need to allow multiple values under a key
<clever> so even if its not a trusted map, you can identify false entries
<clever> the blobs contain signatures made with the nixos keypair
<clever> the only real goal of cache.nixos.org at that point, is to be a map from xs7p3w2iqvv0y1d9r79wb6jrp4pm1h93 to a 591 byte blob
<clever> id say this would work, if you continue to use the .narinfo on cache.nixos.org
<clever> the signature and hash of the file are in the narinfo, so you can still verify its not been tampered with
<clever> if an IPFS field was added, you could download the .nar.xz from ipfs instead
<clever> try running curl on that URL
<clever> when nix wants to install /nix/store/xs7p3w2iqvv0y1d9r79wb6jrp4pm1h93-hello-2.10
<clever> yeah, thats my basic idea
<clever> if nix stored the original signature, his only option is to remove the signature, and then i can just search for anything not signed by nixos
<clever> at this point, how do i detect it??
<clever> and then update db.sqlite to have the right hash
<clever> if an attacker gets root on my machine, and knows how nix works, he can trojan any binary in /nix/store/
<clever> this also opens up a new security option
<clever> and then return the original signature
<clever> in theory, nix could save the signature to the db.sqlite when it downloads things
<clever> simpson: for example, if my hydra downloads something from cache.nixos.org, and then my hydra re-serves it, the nixos signature is lost
<clever> simpson: another area i think nix needs work in, is how signatures are stored client side
<clever> the problem, is a key=value db, that doesnt rely on the key being a hash of the value
<clever> simpson: in theory, the narinfo files could also be published over ipfs, and the signatures they already contain will maintain the trust
<clever> then the narinfo on cache.nixos.org says where to download it normally, the IPFS hash, and the signature
<clever> and in my idea, hydra.nixos.org will do the builds, sign them as safe, and compute the IPFS hash
<clever> you need the binary to be signed by a trusted authority
<clever> yeah, just because i build a binary, doesnt mean i havent packed it with trojans
<clever> so you still need the central cache, but it only acts as an index
<clever> Infinisil: the ideas i have, are to store the IPFS hash in the narinfo on cache.nixos.org
<clever> yeah, it would still need a generic key=value db, and the existing narinfo's can handle that
<clever> Infinisil: but nix stores data as hash(buildscript)=output
<clever> Infinisil: IPFS stores data as hash(value)=value
<clever> i currently have 4 i think, over a range of models
<clever> this chart lets you identify exactly which model you have
<clever> i'm running my own binary cache to solve that, but its fairly behind right now
<clever> Temp4598349088: but you would still need an ARM build server to actualy do the compiles
<clever> Temp4598349088: so you can deploy a pre-built programs into an existing install
<clever> Temp4598349088: nixops will ssh into an existing nixos machine, and copy the missing files into /nix/store/
<clever> Temp4598349088: nix will keep a local cache of everything in /nix/store/
<clever> Infinisil: you would need an ARM based machine, not x86
<clever> Temp49873267: try running 'env' first, and see what vars are available
<clever> plymouth comes up after grub
<clever> rebuild boot also does that, without making it active on the running system
<clever> Temp49873267: the reboot in the above command will then immediately restart, and undo it
<clever> Temp49873267: the 'test' action will activate it without making it the default
<clever> so the env method becomes pure if ran thru nix, and just works if not
<clever> but i think the fixup phase also patches #!/usr/bin/env bash
<clever> and in some cases, you can #!/bin/bash, and then let the fixup phase patch it for you
<clever> oh god no!
<clever> gchristensen: i have tried to install nixos with a 32bit kernel and zfs root before, and nixos-install would NEVER finish
<clever> noobly_: you are supposed to be running nix-build on a nix file, and then the nix file runs gcc for you
<clever> DavidEGrayson: i believe i found the problem
<clever> error: syntax error, unexpected ')', at /home/clever/iohk/iohk-nixops/hello_world.c:1:10
<clever> $ nix-build hello_world.c
<clever> DavidEGrayson: a pre-written filter that excludes .git, .svn, CVS, .hg, .o, .so, and the result symlink
<clever> DavidEGrayson: there is also lib.cleanSource

2017-06-29

<clever> ah
<clever> Infinisil: and maybe pipe it thru jq
<clever> Infinisil: i usualy just toJson it
<clever> bennofs: on my router (16.03pre76756.885acea) it still works
<clever> bennofs: strange, i'm seeing that here too, but i remember it doing something diff before
<clever> my ISP just decided to replace all the phone lines with fiber
<clever> but me, i'm in the middle of nowhere, lol
<clever> but i think mine is more abnormal, the google fiber guy is in the middle of a major city near a backbone
<clever> Infinisil: and i know somebody else that has google fiber, he gets 670mbit down, 860mbit up!
<clever> but nix relies purely on a single dns entry, cache.nixos.org, and AWS cloudfront to do all mirroring and load balancing
<clever> most package managers setup normal http servers with either round-robin dns, http redirects, or a mirror list right in the client
<clever> 370mbit downstream
<clever> same problem here, my internet is too fast and nothing can keep up, lol
<clever> bernas: thats just normal network issues with cloudfront, happens on and off all the time
<clever> and if a is in the inputs of c, it will behave like b was also in the inputs
<clever> Infinisil: so you can for example have package a, that propagates package b
<clever> Infinisil: and when the stdenv finds that in any input, it will iterate over its contents, and behave as if they where also in the inputs
<clever> Infinisil: propagatedBuildInputs are put into $out/nix-support/propagated-build-inputs
<clever> so libraries go into buildInputs, but build-time tools like cmake go in nativeBuildInputs
<clever> Infinisil: when cross-compiling, buildInputs will be for the target, and nativeBuildInputs for the host
<clever> which both configures and installs in the same step
<clever> nixos automaticaly does that if you set hardware.pulseaudio.enable = true;
<clever> ah
<clever> Turion: what do you see in cat /proc/asound/cards
<clever> so your never going to have it secure
<clever> the only way you can make it work is by turning off the sandbox and just reading from /home/foo/bar, but the password will still be in the .hex in $out
<clever> that would be an impurity, and nix doesnt allow them
<clever> asking for the pw on stdin?
<clever> why?
<clever> Infinisil: stdin is connected to /dev/null, try checking "ls -l /proc/self/fd"
<clever> nix will enforce the output having the same result every time, via that hash
<clever> tsmeets: as long as you know what the hash of the final $out will be, you are free to do whatever you want with the network access
<clever> and fetch.nix in the same dir
<clever> tsmeets: look at the factorio downloader
<clever> tsmeets: that can easily be done
<clever> tsmeets: is the command pure or not? (does it use the network)
<clever> ah
<clever> sphalerite: it is currently 3:27 am here
<clever> so you can FOO=bar nix-shell ....
<clever> Infinisil: by default, nix-shell lets env vars leak in
<clever> Infinisil: the .drv file still gets made
<clever> its normaly the wrong solution, but you need the password hidden, and are deleting it asap anyways
<clever> its also not sandboxed, so you can run pass and sudo teensy-loader right from nix-shell
<clever> just rm -rf
<clever> so you dont have to deal with nix when cleaning the files up
<clever> the stuff built under nix-shell never goes into the store
<clever> i would do something like nix-shell --run "something" and then have the something do everything, it can be impure
<clever> i have used the teensy before
<clever> i would just only build it under nix-shell then
<clever> how long does the build take?
<clever> there is no safe way to pass them into a build
<clever> nix doesnt really deal with passwords very well
<clever> ah yeah, the keyboard you where flashing earlier?
<clever> ah
<clever> Infinisil: did you write this firmware?
<clever> Infinisil: and then talk to it over serial after flashing, to set a pw
<clever> Infinisil: also, i would rather rewrite this code to store the password in the teensy's eeprom
<clever> then grep the logfiles it generates for the hash of the storepath your trying to delete
<clever> sudo strace -o logfiles -ff -s 5000 nix-store --delete $drv
<clever> or maybe
<clever> sudo strace -o logfiles -ff nix-store --delete $drv
<clever> and why does the error say bin when the gist said hex?
<clever> was that in the code or just the set -x output?
<clever> why did that have single quotes?
<clever> its bash that expands it, when passing it to ls
<clever> ls doesnt expand * either
<clever> shouldnt
<clever> why didnt it work?
<clever> it should work the same with just -w $result/hex/*
<clever> 1, dont use ls $result/hex/*
<clever> try not using export on that variable
<clever> yeah, it will check the open file handles, and env variables
<clever> to start with, what is the exact --delete command you are running?
<clever> Infinisil: it can also check env variables and many other places for liveness
<clever> Infinisil: never use ignore liveness
<clever> zarkone: builtins.getEnv
<clever> wine doesnt do driver level things
<clever> probably doesnt
<clever> granting normal users access to the usb nodes of a keyboard would allow keyloggers and break keyboard grabs
<clever> and you need windows-only software to hear the low battery beep
<clever> but not the battery level
<clever> i am able to detect every button and switch, even power
<clever> i have used libusb before to try and get more features out of my wireless headset
<clever> libusb permissions are missing a bit on nixos
<clever> strace it?
<clever> run it as root?
<clever> and it will just call dsl_destroy_snapshot behind the scenes when you try to rmdir
<clever> so the /.zfs/snapshot directory is more like /proc, its not the real references
<clever> correction, that commit changed the default of that option from 0 to 1
<clever> Infinisil: that option appears to have been added in the same commit as zfs allow
<clever> MODULE_PARM_DESC(zfs_admin_snapshot, "Enable mkdir/rmdir/mv in .zfs/snapshot");
<clever> Infinisil: this works to delete a snapshot
<clever> [root@amd-nixos:~]# rmdir /.zfs/snapshot/zfs-auto-snap_hourly-2017-06-28-23h00
<clever> i believe rm on a directory in snapshots will delete a snapshot
<clever> Infinisil: you know about /.zfs/ ?
<clever> ah
<clever> when you do have access to /dev/zfs
<clever> something like zfs allow
<clever> Infinisil: if you run the command with strace, can you see what fails?
<clever> yeah
<clever> i can probably also delete any filesystem or mount things anywhere, so keep that in mind
<clever> if i chown it to myself, i can run "zpool status" without root
<clever> crw------- 1 root root 10, 58 Jun 14 22:02 /dev/zfs
<clever> Infinisil: i think the bulk of it relies on +w to this device
<clever> open("/dev/zfs", O_RDWR) = -1 EACCES (Permission denied)
<clever> you need to know the internals some and make an educated guess
<clever> yeah, not very well
<clever> this explains what every field is
<clever> hits : ARC reads per second
<clever> [root@amd-nixos:~]# arcstat.py -v
<clever> so its fairly idle
<clever> you had 22 hits to the cache, and 0 misses, on that second sample
<clever> ah, the miss and hits columns
<clever> my stats are during an ncdu run over 1tb of data
<clever> your system is probably idle, so its not missing the arc
<clever> so it allocates more of the arc to recent, rather then freq
<clever> and if that ghost list gets a hit, it knows that the MRU half of the arc is too small
<clever> when things expire off the most-recently-used list and vanish from the arc, it keeps the id of the block in a ghost list
<clever> mrug : MRU Ghost List hits per second
<clever> c is the goal size for the arc, and arcsz is the current size
<clever> 23:14:36 1.0K 205 586 74 37 372 213 53 785 758M 75 24 17 260 754 649 82 136 74 739 758M
<clever> 23:13:46 2.7K 475 1.7K 78 113 813 933 4 1.8K 1.4G 75 24 22 588 2.1K 1.4K 77 418 78 801 1.4G
<clever> time read dmis dhit dh% mrug mru mfu mfug mread c ph% pm% mm% miss hits mhit mh% mmis hit% eskip arcsz
<clever> including on-demand read misses, hit rate, arc size, and more
<clever> Infinisil: every 10 seconds, it will show a crap-ton of starts
<clever> Infinisil: oh, this is another one
<clever> arcstat.py -f time,read,dmis,dhit,dh%,mrug,mru,mfu,mfug,mread,c,ph%,pm%,mm%,miss,hits,mhit,mh%,mmis,hit%,eskip,arcsz 10
<clever> naspool/root compressratio 1.95x -
<clever> it may not have been on at all times
<clever> naspool/nas compressratio 1.06x -
<clever> ncdu -x /nas/
<clever> so, why am i using 1.05 tb!
<clever> naspool/nas@zfs-auto-snap_hourly-2017-06-28-23h00 0 1.05T 0
<clever> but it doesnt update when things expire
<clever> i think the used field, is the difference between that snapshot, and the next one
<clever> Infinisil: this reveals another column, that gives a better view into the disk usage
<clever> [root@nas:~]# zfs list -t snapshot -r naspool/nas -o name,used,refer,written
<clever> oh there is another cmd that isnt well documented
<clever> i have sometimes cleaned things up, but had to delete months worth of backups to reclaim any space
<clever> yeah
<clever> i think i left a linux kernel build in /tmp/nix-build-linux-4.9.25.drv-0
<clever> hmmm, between april and may, my rootfs went up by 5gig
<clever> and the systemd journal
<clever> if i diff 2 previous snapshots, i can see changes in a /tmp/nix-build
<clever> there is a "zfs diff"
<clever> lvm internally does that, when modifying the lvm metadata
<clever> so both systems can survive loosing a single drive
<clever> nas is a raidz1
<clever> amd is an SSD mirror
<clever> and they already have redundancy
<clever> dont yet have the disk space on a single machine that can take it all
<clever> each with its own limit
<clever> then zfs does snapshots every 15min, 1h, 1day, 1 week, and 1 month
<clever> i generally set com.sun:auto-snapshot=true on my filesystems, and services.zfs.autoSnapshot.enable=true;
<clever> then i can choose all 3 seperately at a later point
<clever> adisbladis: i usualy just make seperate volumes for / /nix and /home
<clever> so i can exclude it from snapshots
<clever> gchristensen: i generally want to put /nix/ on its own volume
<clever> thats pretty neat
<clever> oh, i think i saw you mention that earlier
<clever> in any other case, it loads ~/.config/nixpkgs/config.nix by default
<clever> when nixos loads a copy of nixpkgs to pass to configuration.nix, it will override the config to be the value of nixpkgs.config
<clever> it only ever reads ~/.config/nixpkgs/config.nix
<clever> nix-env will never read configuration.nix
<clever> noobly: you usualy dont want to use sudo nix-env
<clever> acording to that link, it will only be an issue if i manualy run systemd-resolve, while an attacker can control the dns replies