2019-01-29

<clever> _deepfire: that reminds me, some build systems depend on the order that readdir() returns files in
<clever> :'(
<clever> _deepfire: this is the 2nd last line of stack2nix output, overriding the file that fixes everything
<clever> compilerConfig = self: super: {};
<clever> _deepfire: i had to add mtl, stm, text, and parsec to make it work
<clever> and then its somewhat random which version gets picked
<clever> stack2nix undoes the fix, and then defines duplicates that arent null'd out
<clever> _deepfire: so you wind up with duplicate packages, and sometimes nix-shell grabs the wrong one, resulting in errors like expected Data.String, got Data.String
<clever> of note, stack2nix undoes the compiler-configuration
<clever> and thats what should be null
<clever> _deepfire: i think you want to check what ghc-pkg list gives, when using just the bare ghc derivation
<clever> _deepfire: yeah, but it also shouldnt be baked into the compiler
<clever> it needs all of the deps of cabal, to be able to run Setup.hs in packages
<clever> _deepfire: of note, the only reason nix has those boot packages baked into the compiler, is for Cabal, and running Setup.hs
<clever> _deepfire: ahh
<clever> and if you just want the package, you dont have to do anything, ghc already has it
<clever> _deepfire: only nix-tools can overwrite those with new versions
<clever> _deepfire: the packages set to null like base, are baked into the ghc package itself as boot packages
<clever> nope
<clever> iqubic: ive found that pavucontrol volume levels just stick for me
<clever> what was the issue?
<clever> yeah
<clever> and daemon.config doesnt take a string, but a { flat-volumes = "no"; }
<clever> iqubic: look on the nixos option search, what options does pulse have?
<clever> iqubic: what doc did you find flat-volume in?
<clever> iqubic: which file does it belong in?
<clever> iqubic: look in /etc/pulse/, which file did flat-volume wind up in?
<clever> iqubic: add --verbose after --start
<clever> elvishjerricco: works great for some games
<clever> yeah
<clever> since it aint windows anymore
<clever> samueldr: yeah, thats what my guess is

2019-01-28

<clever> samueldr: those are windows terms for the roots in the registery
<clever> samueldr: i see HKLM and HKCU in that VDF file
<clever> yeah
<clever> i also see these symlinks, but those point into the dir your nuking
<clever> lrwxrwxrwx 1 clever users 29 Jan 27 21:08 /home/clever/.steampid -> /home/clever/.steam/steam.pid
<clever> lrwxrwxrwx 1 clever users 31 Jan 27 21:08 /home/clever/.steampath -> /home/clever/.steam/sdk32/steam
<clever> elvishjerricco: yeah
<clever> i'm guessing that behind the scenes, steam saves things to the register, and they needed "a registery" on linux, to save to
<clever> at a glance, i think its emulating the windows register system
<clever> registry.vdf is about the only thing that could pose an issue
<clever> elvishjerricco: that only has symlinks to the other, and some very small files
<clever> elvishjerricco: i think the bulk of it is in ~/.local/share/Steam
<clever> jackjennings: this is a list of extra fields, that are passed to all nixos modules in the deployment
<clever> ToxicFrog: you would need to edit stage-2-init.sh in nixpkgs, and add a new option
<clever> Baughn: what about a memtest86 on the host?
<clever> Baughn: thats pretty bad
<clever> then the binary will be in the working directory for the bulk of the phases
<clever> greatscottttt: src = ./foo; where foo is a directory containing the ELF binary
<clever> _deepfire: nix-store --query --roots will be of use, to see if a given path has roots or not, and which ones
<clever> _deepfire: it might be simpler to just root the immediate deps of a given .drv, and not be recursive in any way
<clever> that will draw the line for you, only the immediate deps of that shell
<clever> all paths in env vars are "in use"
<clever> _deepfire: one trick ive used, just leave a nix-shell open
<clever> because the gcc source is in the build-time dep tree
<clever> _deepfire: also, where do you draw the line, do you keep the gcc source code?
<clever> but you wont have any way to prune the roots automatically, thats handled by hydra-update-gc-roots.service, which uses the postgres DB
<clever> on its own, you can (ab)use that binary without any hydra setup
<clever> it expects a path within the magic gcroots folder, since its not making indirect roots
<clever> [root@nas:~]# ls -l /nix/var/nix/gcroots/hydra/
<clever> hydra-eval-jobs takes a release.nix, and spits out json describing every drv in the file, and creates the roots for you
<clever> _deepfire: the `hydra-eval-jobs` binary does that, and warns you if you forgot the dir: warning: `--gc-roots-dir' not specified
<clever> _deepfire: the only other option, is to do what hydra does, traverse the build-time graph yourself, and add an indirect root for every drv
<clever> _deepfire: i think you want `nix-instantiate --add-root result --indrect` in combination with 2 nix.conf fields, keep-env-derivations and gc-check-reachability
<clever> and builtins.fetchTarball gets you the same effect
<clever> ah
<clever> bgamari: one trick i use, to make sure i didnt miss any <nixpkgs> in the codebase, is to just unset NIX_PATH, but that makes it imposible
<clever> bgamari: also, i try to avoid `(import <nixpkgs>{}).fetchurl` whenever possible
<clever> bgamari: so if you give it a different url, with the same contents, the name can sometimes differ, and then $out differs
<clever> bgamari: ah, that can do it, the $out is based on both name and hash
<clever> bgamari: can you reproduce it with a simple example.nix?
<clever> bgamari: builtins.fetchurl (when the hash is missing) will have to re-query every hour (configurable in nix.conf), and if the e-tag differs, re-download
<clever> bgamari: pkgs.fetchurl computes the $out based on the hash you declared, and will check /nix/store for it, and should only download once
<clever> bgamari: builtins.fetchurl or pkgs.fetchurl?
<clever> domenkozar: https://gist.github.com/cleverca22/43344a86c25f833e3b804788a8c0cd92 am i digging too deep into the nix source when optimizing? lol
<clever> so your code could load a script at runtime, compile it to native, and then run it, without ever writing to disk
<clever> you can then either emit an executable, emit llvm bitcode, or directly write compiled code into ram and get a function-pointer you could FFI into
<clever> and then using standard haskell parsing libraries, to turn your language into an ast, into llvm, and then into binary code
<clever> the above, is using the llvm libraries in haskell
<clever> ive read thru the entire http://www.stephendiehl.com/llvm/ guide, but not actually done any of the steps
<clever> Church_: nice
<clever> i used that virtualbox guide, to temporarily run my physical install, under xen, passthru'd to the card it was already working on
<clever> Church_: i only sleep when you least expect it :P
<clever> so you can migrate things
<clever> and it explains how to undo it, to make a psysical install "work" under a vm
<clever> its sort of like baking the right drivers into the initrd
<clever> it explains what windows does, that locks an install to a specific motherboard
<clever> https://www.virtualbox.org/wiki/Migrate_Windows is also something i had briefly experimented with
<clever> windows!
<clever> and after expanding the image, it just refused to work ever again
<clever> and i didnt have room to install a single game
<clever> the other anoying fact, is that once i got the gpu fully working, i realized i only gave it a 15gig disk image
<clever> the gpu is just the weird duckling on the pci bus
<clever> _deepfire: oddly, i had far far more success, with usb, ethernet, and soundcard passthru
<clever> looking-glass just gets rid of the need for a 2nd monitor, but you still need 2 GPU's
<clever> yeah, with the mess i was doing, i would have needed a 2nd gpu, dedicated to linux, with its own monitors
<clever> i did mess with some utils, that try to save the gpu state, and restore it, but it didnt work
<clever> _deepfire: i think that reset, is half done in the card itself (when the reset line on the pci slot is driven), and half done in the bios
<clever> and now, your linux host, has to live with the uptime stability of windows
<clever> so, you must reboot the host to fix the gpu state
<clever> so the windows drivers refuse to work on the next boot (of the guest)
<clever> wedens: the worst part, is that if windows is shut down, it doesnt reset the state of the GPU
<clever> and the drivers are happy
<clever> and because linux has never touched it, the GPU is in the same state it would be in, when chain-loading win7 from grub
<clever> then, via ssh, i can boot win7, and grant it control of the GPU
<clever> single gpu, single monitor at the time
<clever> which results in zero video output when linux boots
<clever> so, i had to 100% blacklist the gpu on the linux commandline
<clever> and the linux gpu drivers mess with the gpu enough to break the windows drivers
<clever> the main issue, is the my motherboard doesnt have per-slot reset lines, so i cant "reboot" the video card only
<clever> wedens: before i switched to nixos, i had briefly gotten pci-passthru (but it depended on a monitor) to work on gentoo, with a win7 guest (using xen, rather then kvm)
<clever> anything unity based should be perfect
<clever> neither does borderlands 2, for an unknown reason
<clever> .net based games (space engineers) dont work at all
<clever> it will clearly say "python3 not found" in stdout, if your behind
<clever> Church-: it does need python3, nixpkgs was fixed for that a few months back
<clever> dont have any other windows-only things installed currently
<clever> Church-: megaton rainfall starts, but ive not tried to play yet, and the window size gets wonky if it looses focus
<clever> Church-: stationeers only has 2 bugs, that are present on all linux distros (one font missing, minor, and a mouse moving glitch, patch available, but not easy on nixos)
<clever> Church-: works great for some games
<clever> PeterHK: i think if you just add a derivation with $out/etc/{passwd,shadow,group} to the contents, you can skip runAsRoot entirely
<clever> so just not using runAsRoot would be the best option
<clever> if you fake having kvm support, the image build will be fairly slow
<clever> PeterHK: it brings qemu in automatically, so you dont have to add it anywhere
<clever> PeterHK: or you can do things without runAsRoot
<clever> PeterHK: you can either fake it by setting `system-features = kvm` in nix.conv (via the nix.extraConfig option i think)
<clever> PeterHK: runAsRoot requires spinning up an entire qemu VM, and without kvm, the performance is going to suck massively
<clever> PeterHK: does dockerTools.buildImage have runAsRoot set?
<clever> petersjt014: also, wpad, the stuff to auto-configure proxy's on LAN's, uses JS
<clever> PeterHK: does /dev/kvm exist?
<clever> man/info/docs take up space, and if you dont need them, you can turn them off
<clever> petersjt014: did you disable man/info/documentation?
<clever> rules*
<clever> and then an admin can just write whatever logic they want to make up the ruls
<clever> petersjt014: yeah, it has JS functions, that are ran with your current login details, and return a bool to allow/deny access
<clever> and polkit is what allows you to do root-like things (rebooting) if you have physical access (logged in on a terminal or X11)
<clever> petersjt014: polkit uses JS to define the rules of who can do what
<clever> petersjt014: spidermonkey is part of policykit
<clever> only lost about 20gig of data at the time
<clever> one day, i was doing that while an xfs was mounted on the lv, and i suspect it ignored lvm via caching, and it corrupted an ext3 partition
<clever> petersjt014: so all writes to disk are done by pvmove, and there should be zero risk of corruption
<clever> petersjt014: purely by reading that, i can find the fragments of a given lv, and then call pvmove, to safely move the fragments around to make them be in order
<clever> petersjt014: the backup files in /etc/lvm/backup/tank are a textual form of the lvm metadata, which fully describes what byte ranges you need to piece together, to make every LV
<clever> Yaniel: many years ago, i made a php script, to defrag lvm
<clever> likely 32bit and 64bit mono's
<clever> steam depends on the other mono too
<clever> steam depends on one mono
<clever> [root@amd-nixos:~]$ nix why-depends /run/current-system /nix/store/fiim5f606lwjk9w0i8agc8148rrvwmpm-mono-5.16.0.220/
<clever> in my case, its dominated by 2 copies of mono, and 12gig in total
<clever> 608M /nix/store/v8cbpm2m7ivgqxk53dsbdjwclm9hinwl-mono-5.16.0.220
<clever> 683M /nix/store/fiim5f606lwjk9w0i8agc8148rrvwmpm-mono-5.16.0.220
<clever> 12G total
<clever> this uses du, so its a bit slower, but it will tell you the top 5 storepaths in the current nixos build
<clever> [root@amd-nixos:~]$ du -h --max=0 -c $(nix-store -qR /run/current-system) | sort -h | tail -n5
<clever> nix-store -qR /nix/store/foo, will tell you the full closure, without any duplicates
<clever> petersjt014: this will tell you the size (in bytes) of a given path, without having to run things like du
<clever> petersjt014: nix-store --query --size /nix/store/foo/
<clever> do they have the change?
<clever> then just look in there, at the files the PR modifies
<clever> that tells you where nixpkgs is
<clever> nix-instantiate --find-file nixpkgs
<clever> there was a talk at nixcon2018, about improving the network stuff in nixos
<clever> pbb: one thing to note, auto-upgrade doesnt work when nixops is at play
<clever> so i had to just turn off radvd, or i have broken v6 on all machines
<clever> i turned on v6 forwarding, but the router doesnt route anymore
<clever> ive actually had ipv6 problems lately
<clever> ah
<clever> in my case, the router needs 2 IP's, WAN ip (via dhcp) and LAN ip (static, it IS the dhcp server!)
<clever> ah
<clever> why is dhcpcd messing with the static ip of an iface it doesnt manage?
<clever> so i lost all control of the router
<clever> when it failed, it deleted the internal LAN ip!!
<clever> that reminds me, when my isp was down a month ago, i told dhcpcd to renew the lease with upstream
<clever> the journal should say more
<clever> restarting network-setup.service may restart other units
<clever> so you have momentary packet loss
<clever> pbb: sounds like its bringing the entire network down, then back up
<clever> pbb: is ssh using v4 or v6? does it recover or stay broken?
<clever> Ankhers: idris likely needs to obey the NIX_CFLAGS vars
<clever> thats just how compiling works in nix, all compiling must be done under nix-shell or nix-build
<clever> if you are doing*
<clever> Ankhers: header related things, then you must be in nix-shell
<clever> Ankhers: are you in nix-shell?

2019-01-27

<clever> > pkgsi686Linux.stdenv.is64bit
<clever> git blame says they did alter the hash
<clever> tilpner: checking git blame, this is the commit that changes the lua-mode version, and it was bad to begin with
<clever> commit b801736b75877c573ed667dbafcf62569c54c024 (HEAD, refs/bisect/bad)
<clever> Author: Michael Alan Dorman <mdorman@ironicdesign.com>
<clever> but that looks like it isnt it
<clever> 635114b889be9f99d4bd05bdb95b949a7e41ec67 is the first bad commit
<clever> Bisecting: 148 revisions left to test after this (roughly 7 steps)
<clever> i did a grep, and pasted the answer above, about 10mins ago
<clever> 2019-01-27 18:08:32 < clever> pkgs/applications/editors/emacs-modes/melpa-generated.nix: lua-mode = callPackage ({ fetchFromGitHub, fetchurl, lib, melpaBuild }:
<clever> github search doesnt handle massive auto-generated files
<clever> Bisecting: 4572 revisions left to test after this (roughly 12 steps)
<clever> could try a bisect, lol
<clever> kyren: but from 18.03, it works
<clever> kyren: when i build from nixos-unstable, it fails
<clever> i think the next thing to check is git blame, and see who last touched lua-mode
<clever> including all history of what you had in them
<clever> yeah, it can also destroy your nix-env profiles
<clever> then decide if you want to keep x or not, and modify it to not depend on foo
<clever> then nix why-depends, to figure out how x depends on foo
<clever> that will tell you why its live
<clever> nix-store --query --roots /nix/store/foo/
<clever> nix-store --query --roots /nix/store//
<clever> it even even make the machine un-bootable
<clever> which makes it rather hard to do anything
<clever> in my case, the target of /run/current-system vanished
<clever> kyren: thats how i bricked my nixos machine one day :P
<clever> kyren: with rm or nix-store --delete?
<clever> building '/nix/store/iw8ci8rpyfjgaavxf9x0wvr2yim3y3i8-source.drv' on 'ssh://clever@192.168.2.167'...
<clever> [clever@amd-nixos:~/apps/nixpkgs]$ nix-build '<nixpkgs>' -A emacsPackagesNg.lua-mode.src --argstr system x86_64-darwin
<clever> but it only says it, on the machine that doesnt have that path
<clever> the error says which path you want to delete
<clever> hash mismatch in fixed-output derivation '/nix/store/7kw98xzwm0l6dqqmk32x8xpyc82f03y1-source':
<clever> kyren: which is why i started with a nix-store --delete
<clever> fails
<clever> tails for me on linux
<clever> i think this will reproduce the problem
<clever> nix-build '<nixpkgs>' -A emacsPackagesNg.lua-mode.src
<clever> tilpner: which os are you on?
<clever> fetchFromGitHub, which is fetchzip based
<clever> pkgs/applications/editors/emacs-modes/melpa-generated.nix: lua-mode = callPackage ({ fetchFromGitHub, fetchurl, lib, melpaBuild }:
<clever> kyren: run `nix show-derivation /nix/store/foo.drv` on the .drv for downloading that tar (the drv that failed)
<clever> kyren: i dont see any files that conflict by name
<clever> kyren: does nixpkgs use fetchurl or fetchzip?
<clever> kyren: so weird things can happen if the source has both Makefile and makefile
<clever> kyren: on darwin, the filesystem isnt case sensitive
<clever> kyren: --delete wont ruin the rest of your cache, and tells you if its in use or not
<clever> kyren: depends on if something is using it or not
<clever> kyren: try nix-store --delete on the path, and then see if it fails on both
<clever> kyren: if the given path is already in /nix/store/ it wont download it, so ti wont notice the hash mismatch
<clever> fresheyeball: it only ever runs the string you give it
<clever> fresheyeball: runCommand never runs phases
<clever> fresheyeball: if you move that configurePhase into the string on lines 18-37, it should also work
<clever> and see what its doing just before it fails
<clever> fresheyeball: run it under `strace -e trace=open -f`
<clever> fresheyeball: your /etc/protocols is missing again?
<clever> fresheyeball: haskell code?
<clever> neeasade: you can only uninstall by name, nix-env -q lists the names
<clever> (different sides of the same coin)
<clever> the pros, is that it wont leave a copy of the full tar in /nix/store (until the next gc), causing you to spend $tar_size + $fontsize to build it
<clever> the cons (relative to fetchurl+runcommand) is that it has to re-download each time you change the blacklist, or if you want to build a different subset on the same machine
<clever> ixxie: the above will download a given url, unpack it to $out, then delete something, the sha256 is over whatever remains after deletion
<clever> fetchzip { url = "..."; sha256 = "..."; extraPostFetch = "cd $out ; rm things"; }
<clever> so, you could do 100% of this in fetchzip, without the runCommand
<clever> ixxie: and fetchzip accepts a extraPostFetch
<clever> ixxie: the confusingly named fetchzip, is just a variant of fetchurl that unpacks to $out, rather then making $out a tar (or zip)
<clever> ixxie: oh, another crazy idea
<clever> with recursive hashing, its the hash over `nix-store --dump $out` (which produces a binary file, think of it like tar -c $out`
<clever> with flat hashing, outputHash is the raw hash of that file
<clever> when outputHashMode = "flat" $out must be a single file
<clever> when outputHashMode = "recursive";, $out can be a file, dir, or symlink
<clever> at the benefit that when the font isnt changing, hydra doesnt uselessly rebuild it
<clever> but now you have 5 hashes to maintain
<clever> you could make 4 derivations, that each install a certain subset, and each has its own hash
<clever> and you need to confirm it produces the same result on both linux and darwin
<clever> the only downside, is that you must update 2 hashes on every update
<clever> it doesnt matter what version of tar and cp you use, the fonts should be identical
<clever> 2: it can reuse the product between stdenv changes that normally trigger a mass-rebuild
<clever> 1: darwin&linux can share the product, so hydra only has to build it once
<clever> this gives 2 critical advantages
<clever> then, your $out depends only on the name&hash
<clever> if you set those 3 attributes, then you are giving nix a promise, that the hash of your $out will be the one you gave
<clever> outputHashMode = "recursive"; outputHashAlgo = "sha256"; outputHash = "HASH HERE";
<clever> related,
<clever> then it will fail
<clever> i set vim to replace mode, and 0 out a few digits of the hash
<clever> and if it never tries to download, it wont notice the hash is wrong
<clever> and it wont try to download
<clever> if the $out for a given hash exists, it uses it
<clever> or did you reuse an old one?
<clever> is the sha256 correct?
<clever> another trick i saw fonts doing, but its optional, make it a fixed-output derivation
<clever> since runCommand omits gcc and other things
<clever> if you use runCommand, then hydra can potentially build it faster
<clever> but fonts dont need gcc!
<clever> stdenv.mkDerivation gives you a copy of gcc
<clever> oh, another major difference
<clever> i just prefer using runCommand if its not actually building something
<clever> but its mostly personal preference as well
<clever> and it wont try to patch it either (via $patches) and other things
<clever> so you can skip the dontBuild part
<clever> runCommand doesnt bother with all the phases
<clever> meta can still be added to it, runCommand "name" { meta = ...; } ...
<clever> that will do the same basic thing
<clever> let src = ...; in runCommand "name" {} ''unpackFile ${src} ; cd $sourceRoot ; ls''
<clever> ixxie: you may also want to switch this over to runCommand
<clever> those are the worst, then you have to sift all the garbage out from your downloads folder!
<clever> when the tar just barfs all over your working dir when you unpack it
<clever> its only for tar's that have no root dir
<clever> it wont spam up your nixos generations
<clever> i would also recomend `nixos-rebuild build` (likely doesnt need root) for testing
<clever> if you remove sourceRoot=, then it will work better
<clever> ixxie: cu12-1.9 is what was made when the tar was unpacked