2019-02-21

<clever> so i can just ~/clevervim/bin/vim on any box, and get my exact config
<clever> its also very simple to de-nixos that file, and then just build vim with nix-build
<clever> this creates a bash alias instead
<clever> eyJhb: only improvement i can see at a glance, is your shell script for vi, https://github.com/cleverca22/nixos-configs/blob/master/vim.nix#L67-L73
<clever> everything looks fairly normal
<clever> eyJhb: do you have any weird things in your fileSystems. config?
<clever> eyJhb: yeah, they should perform identially
<clever> i'm runnign nixos-unstable
<clever> and if you logout from everything, it will destroy it
<clever> eyJhb: that unit is responsible for creating this tmpfs, if you login via any method (graphical, or ssh)
<clever> tmpfs on /run/user/1000 type tmpfs (rw,nosuid,nodev,relatime,size=3284088k,mode=700,uid=1000,gid=100)
<clever> eyJhb: yep
<clever> user-runtime-dir@1000.service loaded active exited /run/user/1000 mount wrapper
<clever> eyJhb: i use slim, and havent had any problems, despite slim being "deprecated"
<clever> ah, its user-runtime-dir, thats why i didnt see user-dir on my box
<clever> eyJhb: i dont see a user-dir service on my box
<clever> eyJhb: then you likely just need to set the right restart conditions on the service
<clever> eyJhb: does systemd register it as failed in `systemctl status`? you could change the restart conditions to have systemd itself restart on failure
<clever> so if you use 1 passphrase on both the zfs and swap, it just unlocks both at once
<clever> but nixos has recently gained the ability to temporarily save your passphrase to an env var, and use it on all disks
<clever> and i didnt like that, so my zfs is on lvm (for swap), and lvm is on luks
<clever> previously, nixos required some fairly ugly hacks (luks key for swap+root on a 3rd partition, that is luks'd with a passphrase)
<clever> lol
<clever> i try to match the pool name and hostname up
<clever> only in example commands :P
<clever> just `zpool add tank cache /dev/nvme1n0p1` and your done
<clever> but under linux, its just a plain old nvme device
<clever> ar1a: yep
<clever> the drivers just refuse to work in such a situation
<clever> so you cant harm performance, and then claim optane modules make your system slower
<clever> basically, it will only work, if you can afford the overhead of a cache
<clever> but, that special sauce, is also locked to certain intel CPU's
<clever> due to a lack of zfs and cache-device support in windows, they have custom stuff that will add a caching layer to ntfs
<clever> the optane drivers on windows, are just plain whack though
<clever> the nas, has an optane module, which uses the nvme protocol, but is supposed to be even faster
<clever> nvme0n1 259:0 0 13.4G 0 disk
<clever> and the desktop
<clever> nvme0n1 259:0 0 477G 0 disk
<clever> ar1a: that would be my main laptop storage
<clever> nvme0n1 259:0 0 465.8G 0 disk
<clever> i was surprised how easily skyrim just worked
<clever> and for many, it wont even start
<clever> and then for others, its almost unplayable
<clever> proton works great for some games, practically seamless
<clever> ajs124: there is also an flock bug when dealing with nfs
<clever> lol
<clever> i have so much free space, that steam overflows an int, and thinks i have 0 bytes free!
<clever> ZFS, ~8tb total size
<clever> dont trust the avail, i had to set a quota due to steam bugs
<clever> nas:/nas 3.6T 3.6T 84G 98% /nas
<clever> Filesystem Size Used Avail Use% Mounted on
<clever> i havent subscribed to pass either
<clever> i suspect it may be due to them using things like libmpv for playback
<clever> GPL-2.0!?
<clever> ar1a: i was also surprised to discover, the plex native frontend, is on github
<clever> ar1a: i recently set this up on a random box near my TV, and now the plex frontend runs on bootup, fullscreen, in a tv-friendly mode
<clever> ar1a: have you used plex?
<clever> cant really think of anything else to test
<clever> yep, it is claiming to be subscribed to 224.0.0.251
<clever> ar1a: what does `ip maddr` report, for the machine that isnt receiving 5353 stuff
<clever> and it looks like you where right, 224.0.0.0 is multicast, and 224.0.0.251 is the multicast dns IP
<clever> i believe 1 is the default
<clever> though i cant ping 224.0.0.251, because i havent set sysctl net.ipv4.icmp_echo_ignore_broadcasts
<clever> ar1a: are any of the machines running with a static ip config? rather then dhcp?
<clever> ar1a: your sure both machines are on the same subnet, and same AP, and same essid?
<clever> but that will also depend on the type of crypto the wifi is using
<clever> oh, maybe your switch is to blame?
<clever> its like broadcast, but the switch can inteligently filter things
<clever> multicast address to be specific
<clever> are the packets leaving x? do they pass thru the router? do they reach the target?
<clever> a common thing when debugging networking isuses, just tcpdump all the machines!
<clever> yeah
<clever> ar1a: do you see packets leaving your machien, on 5353?
<clever> ar1a: the nas.local was also failing, but when i added publish.address = true; to it, then it started to work
<clever> and in my case, the testing is happening between 2 nixos machines
<clever> ar1a: only thing that comes to mind i the uppercase letters in the hostname
<clever> ar1a: https://gist.github.com/c882cdb1eac9a7ba10d0c13e61406f19 how does yours differ?
<clever> ar1a: check the execstart script, and youll find a path to the config file
<clever> [root@system76:~]# cat /etc/systemd/system/avahi-daemon.service
<clever> ar1a: on both ends?
<clever> so the problem, is that the remote avahi, was configured to keep its IP a secret
<clever> ar1a: avahi.publish.addresses = true; on the laptop, and suddenly it can answer!
<clever> publish-addresses=${yesNo (publish.userServices || publish.addresses)}
<clever> ar1a: i see this in my avahi config
<clever> publish-addresses=no
<clever> so i have to use the ISP wifi, which lacks v6, to convince it i'm canadian
<clever> netflix sees canada on v4, america on v6, and thinks i'm trying to bypass region locking, so it refuses to play anything
<clever> my ipv6 tunnel, registers as an american proxy
<clever> which ties into a second problem
<clever> and then a second 192.168.2.0/24, that is only for tv and netflix
<clever> a 10.67.15.1 for the pppoe link between routers
<clever> the real 192.168.2.0/24
<clever> that also means i essentially have 3 private subnets
<clever> so the isp router still works (but is double-nated)
<clever> and for extra fun, i run my own pppoe-server, on vlan 34!
<clever> then i run a 2nd router, also on vlan 34
<clever> and then due to the isp changes, it does pppoe over vlan 34, while keeping the tv service working, on vlan 35
<clever> so, i mis-configured the isp router to use pppoe
<clever> except, the tv service only works on the isp router
<clever> same
<clever> because the router also does 802.1q vlan tagging, on the upstream link
<clever> i suspect the ISP modified the firmware
<clever> the website for the router claims that feature doesnt exist, so there is no need to turn it off
<clever> my ISP wifi sandboxes everything, and there is no off switch!
<clever> you can run that on a hotel wifi, and see every iphone, as it connects! :D
<clever> ar1a: `avahi-browse -a` is another handy debug thing, it will show devices as they come&go
<clever> the avahi-daemon within the laptop, isnt responding
<clever> but the laptop is choosing to not answer
<clever> ar1a: i can confirm that the laptop is even receiving the queries about itself!
<clever> recvmsg(11, {msg_name={sa_family=AF_INET, sin_port=htons(5353), sin_addr=inet_addr("192.168.2.15")}, msg_namelen=16, msg_iov=[{iov_base="\0\0\0\0\0\1\0\0\0\0\0\0\10system76\5local\0\0\1\0\1"
<clever> [root@system76:~]# strace -p 2723
<clever> thats the nas...
<clever> what the......
<clever> 01:20:21.101267 IP 192.168.2.11.5353 > 224.0.0.251.5353: 0 AAAA (QM)? retracker.local. (33)
<clever> the laptop interface is even receiving them
<clever> [root@system76:~]# tcpdump -i wlp3s0 -p -n port 5353
<clever> i can also confirm, it is sending packets out, when socat tries to query
<clever> [root@amd-nixos:~]$ tcpdump -i enp3s0 -p -n port 5353
<clever> since socat is directly talking to avahi, it doesnt really matter that its in a shell
<clever> ah, but that part wont really matter much
<clever> i havent done any of this in a nix-shell
<clever> i can see that it gets the request, and then starts sending UDP packets out port 5353, on every interface on the box
<clever> stracing avahi now, while running the above
<clever> ar1a: if you fully reboot (and ruin your uptime once more) does that start working better?
<clever> ar1a: i can reproduce the error with just this
<clever> time (echo "RESOLVE-HOSTNAME-IPV6 system76.local" ; sleep 6) | socat stdio unix:/var/run/avahi-daemon/socket
<clever> i think there was a power blip ~34 days ago, lol
<clever> 34 days on the router
<clever> 34 days on the nas
<clever> 61 days on the laptop
<clever> ar1a: 31 days on the desktop where i'm debugging
<clever> ar1a: looking like it, investigating that end now...
<clever> ar1a: and avahi timed out ~5 seconds later
<clever> [pid 2593] <... read resumed> "-15 Timeout reached\n", 4096) = 20
<clever> and asked avahi what the answer is
<clever> [pid 2593] write(19, "RESOLVE-HOSTNAME-IPV6 system76.local\n", 37) = 37
<clever> then it connected to avahi
<clever> [pid 2593] connect(19, {sa_family=AF_UNIX, sun_path="/var/run/avahi-daemon/socket"}, 110) = 0
<clever> ar1a: ok, so the first thing nscdi did, was check /etc/hosts, and then look for any systemd containers with a matching name, lol
<clever> [pid 2593] openat(AT_FDCWD, "/run/systemd/machines/system76.local", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory
<clever> DigitalKiwi: not yet
<clever> [root@amd-nixos:~]$ strace -p 2561 -ff -e 'trace=!epoll_ctl',futex,epoll_wait -s 300
<clever> stracing nscd now...
<clever> ar1a: and i can confirm, nscd has loaded it correctly
<clever> [root@amd-nixos:/etc]$ cat /proc/2561/maps | grep nss_mdns
<clever> 7f939e048000-7f939e04b000 r-xp 00000000 00:19 1667262 /nix/store/9m5gn9wqybdrfsjhfx4s7qa8mcfz6fks-nss-mdns-0.10/lib/libnss_mdns.so.2
<clever> i think the only way for it to work, is via nscd
<clever> and the nssModules are added to the LD_LIBRARY_PATH of nscd itself
<clever> but its not finding it anywhere
<clever> ar1a: its searching $LD_LIBRARY_PATH, and then the rpath of ping
<clever> ar1a: and the option to enable .local, adds nssmdns to the nssModules...
<clever> and ping is trying to load the libnss_mdns library, to add .local support to itself
<clever> openat(AT_FDCWD, "/run/opengl-driver/lib/libnss_mdns.so.2", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
<clever> ar1a: hmmm, if i stop nscd, then ping fails much faster...
<clever> it doesnt show any .local's, but it can see the laptop
<clever> + enp3s0 IPv4 system76 [a0:af:bd:82:39:0d] _workstation._tcp local
<clever> + enp3s0 IPv4 system76 _ssh._tcp local
<clever> ar1a: `avahi-browse -at` can find system76, so avahi itself is working
<clever> ar1a: given what youve said, i'm not sure any of my machines have working .local, since i rarely reboot and have switched on nearly all of them, let me see if i can fix it locally....
<clever> nimblepoultry: <nixpkgs/nixos/modules/profiles/hardened.nix> or (pkgs.path + "/nixos/modules/profiles/hardened.nix")
<clever> ar1a: looks like mine is also broken, done many switch's
<clever> ping: system76.local: System error
<clever> ar1a: what about nscd.service?
<clever> ar1a: does it recover if you manually restart avahi.service in systemd?
<clever> ar1a: the major difference, is that nixos-unstable wont update, if things like grub are broken, so it is far less likely to brick the entire machine
<clever> ar1a: its not really a downgrade, nixpkgs-unstable and nixos-unstable both follow master, and there is only a 2 hour difference between the versions
<clever> ar1a: just re-run nix-channel --add, with the name set to nixos
<clever> ar1a: if you want the nixos unstable channel, then you must use nixos-unstable
<clever> ar1a: it is not safe to run nixos from the nixpkgs channels, and can potentially brick the machine
<clever> ar1a: what does `sudo nix-channel --list` report?
<clever> slack1256: if you nix-channel --update all the machines at the same time, then they will usually match up, you could also use things like nixops to manage things more centrally
<clever> slack1256: nix-serve just shares whatever is currently in /nix/store/, and doesnt really care about nixpkgs versions
<clever> may need to update the package to use ncurses
<clever> :S
<clever> ,locate libcurses.so
<clever> DanC: ahh
<clever> DanC: opam using the ld from $PATH or its own ld?
<clever> DanC: did you `-p ncurses` ?
<clever> -rw-r--r-- 1 root root 50 Mar 17 2017 /etc/nix/signing.pub
<clever> -rw------- 1 root root 94 Mar 17 2017 /etc/nix/signing.sec
<clever> slack1256: i just stick the keys into /etc/nix/
<clever> mek42_laptop: at lower levels, some FS's can corrupt some storepaths, leaving random (recently edited) files truncated, `nix-store --verify --check-contents` will find that
<clever> mek42_laptop: at the top-level, none of the changes will happen, the entire nix-env operation is atomic
<clever> DigitalKiwi: another package appears to be doing a --replace, to just delete that dependency
<clever> pkgs/development/python-modules/cheroot/default.nix: --replace "'setuptools_scm_git_archive>=1.0'," "" \
<clever> infinisil: yeah, lol
<clever> infinisil: the whole point of the uid-map, is that it keeps them static for you, so you dont have to set things in configuration.nix
<clever> infinisil: if you had assigned that from the first deployment, it would always have that uid, and the problem wouldnt exist
<clever> infinisil: update-users-groups.pl would need to be updated, to support forcing the uid change
<clever> infinisil: which means recursively searching every mounted FS
<clever> infinisil: the problem, is that you must find every file owned by that user, and fix the uid
<clever> and its only a one-time thing, for when the uid defined in the nixos module is changed
<clever> infinisil: the uid-map file sovles that, by remembering the uid even if you delete a user
<clever> infinisil: in the past, nixos would forget the uid when you delete a user from the cfg, and it may wind up with a different uid when you re-enable, causing similar breakage
<clever> infinisil: i would just do the standard linux solution, manually edit /etc/passwd to sync it over, manually chown the home dir, and then add a nixos twist on the end, fix /var/lib/nixos/uid-map
<clever> infinisil: thats to prevent the horid mess that happens when the uid changes, and the files arent chown'd over
<clever> yokyolu: i think hackage2nix turns a hackage snapshot into a haskellPackages, but callHackage lets you just directly load a given version of a given package
<clever> DigitalKiwi: looks like python is trying to download dependencies at build-time, you may need to supply it with more buildInputs
<clever> DigitalKiwi: what error did it fail with?
<clever> DigitalKiwi: nothing at a glance, does it build and run?
<clever> slack1256: if you just want to see if it builds, use nix-build
<clever> slack1256: why are you using nix-env to test it?
<clever> slack1256: what are you trying to do exactly?
<clever> slack1256: so you need to drop the nixos. prefix
<clever> slack1256: but when using -f default.nix, it has no channels loaded
<clever> slack1256: the nixos. in -iA, refers to a channel called nixos
<clever> DigitalKiwi: 90% of the time, it can just go into buildInputs, propagated is mainly only used in python and some other special setups
<clever> ah, then youll need to add that to package.json
<clever> bsima: did you set a name when calling it?
<clever> bsima: gulp is a build-tool that expects node_modules to already be setup, so you need npm (or yarn) to create node_modules
<clever> that step needs network, but you can then add yarn.lock to the git repo
<clever> bsima: if you run `yarn` without any args, it will create a yarn.lock
<clever> https://github.com/input-output-hk/daedalus/blob/develop/yarn2nix.nix is a more complex example of yarn2nix
<clever> bsima: yeah
<clever> bsima: you may want to use yarn2nix instead
<clever> yep, nix.buildCores
<clever> bsima: usually best to start from source
<clever> cthachuk: gcc is also in the stdenv by default, you dont need to list that
<clever> cthachuk: buildInputs = [ (boost.override { enableStatic = true; }) cmake gcc ];
<clever> your welcome :)
<clever> thomasd: looks like it is
<clever> thomasd: then there is no problem?
<clever> thomasd: ah, then youll want something more like `env | grep --color julia`
<clever> thomasd: yeah, you can `nix-shell default.nix -A parallel-julia.env` and then `ghc-pkg list`
<clever> thomasd: your original msg says 1.0.3 is the worng one, so which derivation is using itt?
<clever> thomasd: yeah, or its name, from the .drv it started building
<clever> thomasd: which derivation is getting a 1.0.3 of julia?
<clever> thomasd: can you pastebin your current nix?

2019-02-20

<clever> srk: i started from the wrong end, i learned the c code within the ghc rts, before learning the language, lol
<clever> reallymemorable: so i had to deal with files getting corrupt, and restarting over&over for days
<clever> reallymemorable: the crazy thing i did, that really hammered LFS in, was trying to install it on a laptop with a failing harddrive
<clever> reallymemorable: ive been using linux since ~2005, and i learned the most when i installed http://www.linuxfromscratch.org/
<clever> and the ssh-agent also has to remain running
<clever> the socat is required for the nixbld users to talk to ssh-agent
<clever> yeah
<clever> only time ive passed thru NYC was for connecting flights
<clever> :D
<clever> UserKnownHostsFile /dev/null
<clever> StrictHostKeyChecking=no
<clever> reallymemorable: add these 2 lines to the /tmp/ssh-config
<clever> reallymemorable: yeah
<clever> reallymemorable: try switching back to ssh-config-file=/tmp/ssh-config, and also add nix.sandboxPaths = [ "/tmp/ssh-config" ]; to the configuration.nix, and do another nixos-rebuild
<clever> ,libraries cthachuk
<clever> nope
<clever> reallymemorable: try setting nix.useSandbox = false; in the host configuration.nix, and nixos-rebuild switch to apply it
<clever> reallymemorable: what path is it showing in the error?
<clever> so its not using the new path
<clever> reallymemorable: you have ssh-config-file= in there twice
<clever> reallymemorable: with the new path?
<clever> yeah
<clever> yeah, make that ssh-config-file=/nix/store/something
<clever> and edit it before you run that cmd
<clever> reallymemorable: echo the current value, then just copy/paste it to a new `export NIX_PATH=....`
<clever> the agent handles that
<clever> they shouldnt have access to the keys
<clever> reallymemorable: try `nix-store --add-fixed sha256 /tmp/ssh-config`, and then replace the /tmp/ssh-config in $NIX_PATH with the path it returned
<clever> reallymemorable: that looks like a different error, its not complaining about ssh-auth-sock
<clever> reallymemorable: can you pastebin the whole output on that terminal, including the error msg?
<clever> reallymemorable: i dont think it would
<clever> reallymemorable: EC2 doesnt provide access to the GPU console, so you need to run a vnc server, like https://gist.github.com/cleverca22/32f71c41c8e37df57c0018ba6e07816f
<clever> asymmetric: ah, yeah, that should also work
<clever> asymmetric: (import (fetchFromGitHub { .... }) {}).foo
<clever> asymmetric: (import (fetchFromGitHub { .... })).foo
<clever> DigitalKiwi: have you read the nixpkgs section on writing packages?
<clever> can you pastebin the whole output on that terminal, including the error msg?
<clever> reallymemorable: did you do the export NIX_PATH i gave above, in the same terminal that you ran the haskell code in?
<clever> nixos on EC2 will still have the sandbox enabled, and have the same issues
<clever> the previous pastes show that you already have the code, so you can just `grep -r --color NIX_PATH`
<clever> does NIX_PATH appear anywhere in the code?
<clever> link?
<clever> is the haskell code your running on github?
<clever> is the socat still running?
<clever> and then run the haskell code
<clever> so you would replace step 3, with the above export line
<clever> your haskell code is doing nix-build behind the scenes, in some area you cant modify
<clever> then try to build the thing again
<clever> reallymemorable: export NIX_PATH=ssh-auth-sock=/tmp/hax:ssh-config-file=/tmp/ssh-config:$NIX_PATH
<clever> reallymemorable: ssh-auth-sock and ssh-config arent in there
<clever> reallymemorable: what does `echo $NIX_PATH` report?
<clever> reallymemorable: what was the end-result when using socat and /tmp/hax?, that should still work
<clever> selfsymmetric-mu: i try to never have channels on the user, root is the only source of channels
<clever> reallymemorable: what part of the GUI do you want to use?
<clever> selfsymmetric-mu: it will warn you if you have 2 nixos channels, every single time you use nix-env
<clever> selfsymmetric-mu: do you have a `with pkgs;` near the systemPackages?
<clever> yeah, thats where libredirect can help
<clever> catern: i just added /etc/protocols to extra-sandbox-paths to solve my issues
<clever> catern: libredirect lets you patch such things at runtime
<clever> catern: libredirect or qemu
<clever> catern: nope
<clever> yeah
<clever> or add ssh-auth-sock=/tmp/hax:ssh-config-file=/tmp/ssh-config to your $NIX_PATH variable
<clever> use a second terminal, with `-I ssh-auth-sock=/tmp/hax -I ssh-config-file=/tmp/ssh-config` in the nix-build command