2018-07-21

<clever> d1rewolf: i dont really use the nix-env search, i just tab-complete in nix repl
<clever> infinisil: it also happens with dynamic ones
<clever> so when you try to lookup .foo.bar, it will find foo in the hash table, then use its index to search the set
<clever> tilpner: all strings being used as keys in sets are put into a global hash table, so the keys in the set are actually integers
<clever> d1rewolf: only nix-env supports installing a set, nix config files need you to give a list of packages
<clever> d1rewolf: jetbrains is not of type package, it is of type set
<clever> d1rewolf: run `nix repl '<nixpkgs>'` then eval `jetbrains` inside there
<clever> d1rewolf: jetbrains is a set of packages, if you tell nix-env to install jetbrains, it will instead install everything inside jetbrains
<clever> tilpner: a set is a strict list of keys and values, and all keys must be known before any can be accessed
<clever> tilpner: dont think that is possible
<clever> d1rewolf: when you ask nix-env to search, it may not show all attributes within jetbrains, so jetbrains.jdk wont appear
<clever> d1rewolf: nix-env doesnt recurse into everyting on its own
<clever> you can also do [ pkgs.android-studio ];
<clever> do you have a with pkgs; nearby?
<clever> hmmm, and android-studio is the attr path
<clever> d1rewolf: the nix files need attribute paths, not names
<clever> rotaerk: i think nixon is special, us president
<clever> america wont let nix have the doman :P
<clever> that reminds me, nixcon.com cant be registered, because of its edit distance from nixon.com
<clever> --ignore-liveness still wont corrupt the store, but it can delete important paths that leave the machine unusable
<clever> and nix-channel was refusing to download the new tar until an hour had passed
<clever> its far more likely, that you messed up the tar on your first test, and ~/.cache/nix/tarballs was caching the contents of the tar for 1 hour
<clever> thats the whole point of nix
<clever> zgrep: and because of how nix works, deleting things like that is pointless, nix will perfectly recreate the exact same files at the same paths every time
<clever> it can delete the current nix-env profile, and basically uninstall ALL programs at once
<clever> if you --ignore-liveness, you risk seriously breaking the machine
<clever> so it will always create the same contents
<clever> the caching is based on a hash of the build directions
<clever> what do you get when you ls ~/.nix-defexpr/channels/ ?
<clever> shouldnt
<clever> zgrep: works as expected on this end, one symlink to the dir, and all the files inside that
<clever> lrwxrwxrwx 1 root root 53 Dec 31 1969 /home/clever/.nix-defexpr/channels/znur -> /nix/store/4xj01s8kciim4kaigyd1yk95n9pndfcn-znur/znur
<clever> what is the URL to your channel?
<clever> the entire tar is inside a single root directory
<clever> zgrep: look closer at the output of the command you gave, that default.nix is inside a directory
<clever> zgrep: how does the layout in there differ?
<clever> zgrep: what if you look in ~/.nix-defexpr/
<clever> not actually nixpkgs, but nixos, hmmm
<clever> nixos-18.09pre145679.dae9cf6106d/nixos/release-small.nix
<clever> in the tar itself
<clever> zgrep: the nixpkgs channel has a nixpkgs subdir
<clever> iqubic: thats what device is
<clever> i dont see an option for that
<clever> swapDevices = [ { device = "/dev/disk/by-uuid/68bce3d2-cfc5-4400-ad8c-ddb751441beb"; } ];

2018-07-20

<clever> changing the id of a user/group can cause problems, so nixos avoids doing it for existing users
<clever> it may need an mkForce, and it would only work if set prior to installing
<clever> maurer: yeah, that does make more sense
<clever> johnw: or just link it against libnixexpr
<clever> maurer: i had an idea a few days ago, could nix be compiled to llvm?
<clever> and matrix is striking again!
<clever> infinisil: and i dont think a PR will be accepted with storepaths in it, so just manually paste the fetchFromGitHub in for now?
<clever> infinisil: that will get rid of src.nix and the need to override the src
<clever> infinisil: cabal2nix https://github.com/diku-dk/futhark --revision v0.6.2
<clever> infinisil: if you run cabal2nix with a git url, it will create the right src= for you
<clever> infinisil: those 2
<clever> ah
<clever> ,find <utils.bash>
<clever> infinisil: youll also want to read <utils.bash> and <add-flags.sh>
<clever> infinisil: id say use it
<clever> since none of the cross-compile stuff was set, it was broken
<clever> how it decides between NIX_BUILD_LDFLAGS, NIX_LDFLAGS, and NIX_TARGET_LDFLAGS
<clever> that controls which env vars gcc reads
<clever> infinisil: what if you export NIX_CC_WRAPPER_x86_64_unknown_linux_gnu_TARGET_HOST=1 ?
<clever> infinisil: manually run gcc with args that are close enough
<clever> infinisil: it will only give output when gcc is ran
<clever> infinisil: try setting `export NIX_DEBUG=1` and then compare how it behaves under nix-shell to outside
<clever> infinisil: thats the raw gcc
<clever> > gcc.cc
<clever> and when you use it outside of nix, those flags are missing
<clever> based on the state of cross-compile flags
<clever> infinisil: there is a accumulateRoles function that generates the var NIX_CFLAGS_COMPILE
<clever> infinisil: i think i see the problem
<clever> id say make your own wrapper around gcc, that inserts the right flags
<clever> is that possible?
<clever> make sure that "runtime" happens inside another derivation
<clever> put opencl into the propagated inputs for the compiler, and use the compiler from another derivation
<clever> this seems like a job for propagatedBuildInputs
<clever> try dropping the space in here?
<clever> --set NIX_CFLAGS_COMPILE "-I ${opencl-headers}/include" \
<clever> infinisil: oh, your wrapping some kind of compiler
<clever> infinisil: those are the correct names, and both happen to be perl and openssl
<clever> pkgs/top-level/perl-packages.nix: NIX_CFLAGS_COMPILE = "-I${pkgs.openssl.dev}/include";
<clever> pkgs/top-level/perl-packages.nix: NIX_CFLAGS_LINK = "-L${pkgs.openssl.out}/lib -lcrypto";
<clever> infinisil: let me confirm the names
<clever> yeah, seperate one for compile and link
<clever> infinisil: then you need to find the derivation that includes the libs, and add it to buildInputs
<clever> and we are getting matrixed again
<clever> infinisil: is opencl in buildInputs?, does its lib exist in ${opencl}/lib/ ?
<clever> dhess, jgt: its safer to push just the closure of the program, rather then the entire nixops deployment, and to also never give the package build the secrets
<clever> jgt: i found some "secrets" in the closure of hydra
<clever> /nix/store/d3aa78jf7pm9lk6ad67x4f21a068dxn6-python-2.7.15/lib/python2.7/distutils/tests/test_config.py:password:secret
<clever> [clever@amd-nixos:~/nixpkgs]$ grep -r secret $(nix-store -qR /nix/store/awdgjdg1lcpm27xkf68aph52a10hvyfg-hydra-2017-11-21)
<clever> jgt: you can also grep those paths for secrets
<clever> jgt: run `nix-store -r /nix/store/0ipns30slkihw012867q3fxr6ksyachw-nixops-machines.drv` and then repeat -qR on the output it gave
<clever> jgt: thats the build-time dependencies, not the runtime ones
<clever> infinisil: that is how buildInputs are injected into it
<clever> infinisil: the gcc wrapper used by nix already obeys $NIX_COMPILE_CFLAGS
<clever> infinisil: the problem is packages trying to avoid conflicts, which nix solves, and now they make nix harder to use :P
<clever> infinisil: i think you need to compute -I${opencl}/include/CL in nix, and pass that in however you want, usually NIX_COMPILE_CFLAGS i think
<clever> jgt: for example: du -hc --max-depth=0 $(nix-store -qR /nix/store/awdgjdg1lcpm27xkf68aph52a10hvyfg-hydra-2017-11-21) | sort -h
<clever> jgt: give it the path to your output
<clever> jgt: nix-store -qR
<clever> but only certain binaries support it
<clever> NIX_SHOW_STATS=1 will tell you where all the memory is going, on a type basis
<clever> ,profiling
<clever> 32gig...
<clever> bennofs: i just ran it with 700 tabs open in chrome ....
<clever> bennofs: the bulk of the runtime was waiting for nix-daemon to do IO
<clever> Elapsed (wall clock) time (h:mm:ss or m:ss): 12:25.14
<clever> Maximum resident set size (kbytes): 7782192
<clever> bennofs: aha, and it finished
<clever> bennofs: 7.4gig used
<clever> and yeah, its up to 4.8gig of ram
<clever> bennofs: all i did was run this in the root dir of a nixpkgs checkout, and its streaming all kinds of json at me
<clever> clever 30494 22.0 14.2 5297164 4683932 pts/7 Sl+ 17:37 1:39 ./hydra/bin/hydra-eval-jobs -I . pkgs/top-level/release.nix
<clever> bennofs: only paths listed in -I are allowed, to improve purity
<clever> bennofs: you need to include -I NIXPKGS_CHECKOUT_DIR
<clever> release.nix also has some special nix code to reduce memory usage
<clever> and the job name for each .drv
<clever> all it does is eval the nix expressions and spit out json containing all of the .drv paths
<clever> bennofs: just call it with the right flags and it will always produce the right output
<clever> bennofs: the whole point of the hydra-eval-jobs binary is to isolate the eval from the environment
<clever> bennofs: hydra-eval-jobs can be used without configuring or installing hydra: https://github.com/input-output-hk/cardano-sl/blob/develop/scripts/ci/check-hydra.sh
<clever> bennofs: you can either run the indexer against the release.nix file, or directly run hydra-eval-job against release.nix
<clever> d1rewolf__: what file did you set allowUnfree in?
<clever> d1rewolf__: nix-env obeys the per-user nixpkgs config, not configuration.nix
<clever> bennofs: it has some perl based scripts in it
<clever> d1rewolf__: the attribute paths are google-chrome for the pre-compiled blob and chromium for the opensource one (also precompiled, but by hydra)
<clever> or -A out -A doc
<clever> sphalerite mentioned the all attribute, so try -A all
<clever> a.b is an attribute path
<clever> > ({ a = { b = 2; }; }).a.b
<clever> a path you form from multiple attributes
<clever> fresheyeball: it then reads attributes from the file it loaded
<clever> fresheyeball: -A takes an attribute path
<clever> fresheyeball: then use nix-build -A doc
<clever> fresheyeball: what command do you run to build the package?
<clever> fresheyeball: nix must always build all outputs at the same time
<clever> fresheyeball: try adding .doc to the attr path
<clever> booglewoogle: does it behave differently if you run `sudo -i` then seperately run `nixos-rebuild switch`
<clever> sphalerite: ah maybe, the above have different timestamps
<clever> lrwxrwxrwx 1 clever users 58 Jul 19 17:12 result -> /nix/store/adf048j1l1adwx35rd3acpalmjbkv1mk-openssl-1.0.2o
<clever> lrwxrwxrwx 1 clever users 62 Jul 19 17:11 result-dev -> /nix/store/q239yikz665n4a5rff7rg2vc7jpay6xb-openssl-1.0.2o-dev
<clever> manveru: the symlinks have a recent timestamp, seperate from the unix 1 from the store
<clever> fresheyeball: if you point nix-build at a derivation with multiple outputs, it will create a result and a result-doc symlink
<clever> manveru: i dont want recursion and i want it sorted by lastmod
<clever> fresheyeball: ls -ltrh
<clever> booglewoogle: sudo changes env vars
<clever> native windows steam*
<clever> Acou_Bass: that reminds me, one game i installed with wine steam (and native windows wine) left linux binaries in its install directory!
<clever> srk: i later discovered it had native linux under steam, and that plays with no issues at all
<clever> srk: i tried to play a game under wine a few days ago, and the input lag made it imposible to play
<clever> fresheyeball: you may need an `mkdir $doc` to fake it sooner on
<clever> d1rewolf_: but then it will try to download when you run it, and things will be a lot slower
<clever> d1rewolf_: you can, with `nix-build '<nixpkgs>' -A i3blocks --no-out-link`
<clever> ocharles_: but it runs your deployment files thru <nixops/eval-machines.nix> with some complex args
<clever> ocharles_: nix-instantiate --xml i think
<clever> > builtins.functionArgs ({ a, b ? "hasDefault" }: b)
<clever> manveru: i dont think there is, only to check if it has a default
<clever> d1rewolf_: some scripts i made, though these are all inlined: https://github.com/cleverca22/nixos-configs/blob/master/util.nix
<clever> because its meant for keys that are never meant to enter the store
<clever> ocharles_: i think nixops is doing some funny things, and wont build the derivation
<clever> d1rewolf_: youll need to write a nix file to package that shellscript, and apply sed to it
<clever> d1rewolf_: if that string is within a nix file, it will automatically expand to /nix/store/0zxf7wj1sj10zj74kck0faqfzk8z7gi3-i3blocks-1.4/libexec/i3blocks
<clever> d1rewolf_: run something like sed over the script to insert ${pkgs.i3blocks}/libexec/i3blocks into the script
<clever> the /etc stuff has text and source, but deployment.keys decided to rename it
<clever> ocharles_: ah, yeah, its keyFile
<clever> hmmm, *checks docs*
<clever> ocharles_: deployment.keys.<name>.source
<clever> ocharles_: i think you want .source
<clever> i dont think it will
<clever> and pass the derivation the storepath of the snapshot
<clever> each time you deploy, nix will snapshot that directory and store it in the local /nix/store/
<clever> and then modifying it to take a snapshot of the keybase files
<clever> ocharles_: id recomend passing "${/keybase}" to the dhallToNix script as well
<clever> yeah, since your using deployment.keys, it will copy to the specified path on the server, outside the store
<clever> ocharles_: the output of that will land in /nix/store/ and be readable to all users on the machine
<clever> ocharles_: can you gist your current nix file?
<clever> ocharles_: when done correctly, its read directly by the python code in nixops
<clever> ocharles_: nixops deployment.keys are not read by the nixbld group
<clever> coconnor: i believe its pavucontrol
<clever> coconnor: if you check pavucontrol, do you see several streams on the playback tab?
<clever> yeah
<clever> and a trojaned copy of the source will always live in a different directory and be ignored
<clever> mightybyte: so somebody wanting an un-trojaned copy, will be using the hash of the unmodified source
<clever> mightybyte: the hash of that storepath is based on the contents of the directory
<clever> mightybyte: because a malicous user can just give the daemon a trojan'd copy of sudo, and wait for it to be ran as root
<clever> mightybyte: but then the rules of the store cant easily be enforced, and a multi-user setup would become a security problem
<clever> mightybyte: only if /nix/store is user writeable can you run without the daemon, and then nix-build just spawns the ssh directly as your user
<clever> and either way, things get cached in /nix/store and become world readable
<clever> theres not really a way to give just one user access to the cache
<clever> dedicated just to the cache use
<clever> mightybyte: i would just generate a new private as root
<clever> mightybyte: each user or machine has their own private for accessing the cache, and then the cache has a whitelist of publics that can read it
<clever> nh2: makeFlags i think
<clever> ah, then id have an ssh key just for the cache
<clever> mightybyte: i generally give root its own key for build slaves, that cant access non-buildslave machines
<clever> mightybyte: root can read other peoples files, so you can symlink the .ssh/id_rsa also
<clever> mightybyte: just give it user@host ?
<clever> booglewoogle: if you put `foo = pkgs.hello;` into the derivation for a shell.nix, then $foo will contain /nix/store/hash-hello-1.2.3/
<clever> RetardedOnion: and once that is configured, i can just nix-env -iA nixos.manymans to install one
<clever> RetardedOnion: line 13 and 20 are 2 old buildEnv's i have used in the past
<clever> RetardedOnion: create a package override containing that buildEnv
<clever> RetardedOnion: this excludes a few things that cant be installed, then generates a list with everything else
<clever> nix-repl> buildEnv { name = "all-the-kde"; paths = [ (builtins.attrValues (kdeApplications // { callPackage = null; newScope = null; override = null; overrideDerivation = null; packages = null; overrideScope = null; recurseForDerivations=true; })) ]; ignoreCollisions = true; }
<clever> hmmm, needs to be filtered, it has a few non-package things
<clever> RetardedOnion: buildEnv { name = "all-the-kde"; paths = [ (builtins.attrValues kdeApplications) ]; ignoreCollisions = true; }
<clever> RetardedOnion: id recomend writing a buildEnv expression in a package override, that merges them, and then install that
<clever> d1rewolf: in what context is it not being set?
<clever> xinming: not sure if it will help or not, but have you seen perl.stdenv.mkDerivation ?
<clever> xinming: ah, perl and libssl?
<clever> mjhoy: yeah, just run nix-env -i /nix/store/path-nix
<clever> xinming: sure
<clever> xinming: pong
<clever> mjhoy: and it should no longer garbage collect your things
<clever> mjhoy: /nix/var/nix/profiles/per-user/mjhoy/profile
<clever> mjhoy: yes, and it should end in profile
<clever> mjhoy: it should look similar to the one i pasted above
<clever> mjhoy: manually create .nix-profile with `ln -sv`
<clever> mjhoy: maybe
<clever> mjhoy: you need to fix the old ~/.nix-profile before you can install anything
<clever> mjhoy: the target may already exist from the old install
<clever> mjhoy: try deleting .nix-profile and then creating a new symlink like above
<clever> mjhoy: it sounds more like you manually created the symlink and broke things
<clever> mjhoy: if it does not, then GC WILL eat everything
<clever> mjhoy: .nix-profile MUST point to a subdirectory of /nix/var/nix/profiles/
<clever> lrwxrwxrwx 1 clever users 45 Oct 11 2015 /home/clever/.nix-profile -> /nix/var/nix/profiles/per-user/clever/profile
<clever> mjhoy: thats not right!!
<clever> mjhoy: where does ~/.nix-profile point when you ls -l ?

2018-07-19

<clever> and i need to get off to bed, goodnight
<clever> boomshroom: oh wow, nice find with that site
<clever> boomshroom: what happens if you run evtest (from the evtest package) as root?
<clever> xinming: heading off to bed now, goodnight
<clever> xinming: try adding preConfigure = "export LD_LIBRARY_PATH=${openssl.out}/lib"; and see if that makes a difference
<clever> xinming: that also works
<clever> xinming: can you gist the current nix expression?
<clever> xinming: what happens if you set LD_LIBRARY_PATH=${openssl.out}/lib/ ?
<clever> and why is it also looking in glibc??
<clever> ./logfiles.5427:openat(AT_FDCWD, "/nix/store/l0adcz8y05vajfi483a7qv31i03rspqq-glibc-2.27/lib/libssl.so", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
<clever> xinming: also, why is that working?, are you not on nixos?
<clever> xinming: you need to fix perl to not be stupid
<clever> xinming: bingo, theres the problem
<clever> ./logfiles.5379:openat(AT_FDCWD, "/usr/lib64/libssl.so.1.0.0", O_RDONLY|O_CLOEXEC) = 3
<clever> xinming: i do see it loading a few child processes, can you make another set of logs with `strace -ff -o logfiles -s 500` ?
<clever> xinming: thats weird, i see no attempt at opening it, it just assumes its not present
<clever> imalison: so the automatic loading from NIX_PATH and $HOME no longer work
<clever> imalison: line 68 imports nixpkgs, and 70 passes it nixpkgs.config and nixpkgs.overlays
<clever> imalison: nixos forces nixpkgs to only use nixpkgs.overlays, and to ignore NIX_PATH
<clever> yes
<clever> nixos only uses those, and nothing else
<clever> imalison: nixpkgs.overlays in configuration.nix
<clever> can you pastebin all of the strace output?
<clever> but youve got a non-standard tool
<clever> nix-shell/nix-build should already put them into scope for all of the standard tools
<clever> xinming: what is it reading and writing?
<clever> xinming: what does it do shortly before it fails?
<clever> xinming: run it under strace and read the log
<clever> xinming: how does it search for libraries? does it just assume its in /usr/lib/ ?
<clever> xinming: throw strace at it?
<clever> xinming: is it using dlopen()?, how does it search for libs?
<clever> xinming: is the error happening at runtime or compiletime?, and what compiler is being used?
<clever> xinming: propagateBuildInputs are not available at runtime
<clever> boomshroom: the above may help
<clever> boomshroom: check lsusb to find its vid:pid
<clever> boomshroom: a udev rule can automate that
<clever> boomshroom: you need to chown the device nodes in /dev/bus/usb/
<clever> code cant lie
<clever> read the code, not the comments
<clever> they omited the fact that --optimize is valid
<clever> hyper_ch: docs can lie :P
<clever> hyper_ch: the manpage says s, the source says either!
<clever> op = opOptimise;
<clever> else if (*arg == "--optimise" || *arg == "--optimize")
<clever> / also lets you search in less, and chrome
<clever> / lets you search in a man page
<clever> to the source!
<clever> optimize also works....
<clever> nix-store --optimise
<clever> to the man page!
<clever> the hardlinks are costing you 4mb
<clever> ah, then your just not saving much with it
<clever> hyper_ch: nix-store --optimize
<clever> hyper_ch: swap helps there
<clever> hyper_ch: but qemu tries to allocate it all at once and can fail before zfs has a chance to move out of the way
<clever> hyper_ch: it defaults to 50% of the ram, and it will release some under memory pressure
<clever> hyper_ch: much more readable
<clever> hyper_ch: "zfs.zfs_arc_max=${toString (6 * 1024 * 1024 * 1024)}"
<clever> hyper_ch: you can also use math for the max arc size
<clever> hyper_ch: ah, oops
<clever> hyper_ch: you could also rollback
<clever> hyper_ch: i just never shutdown :P
<clever> hyper_ch: if your fine with manually importing the pool every time
<clever> hyper_ch: out of ideas then
<clever> hyper_ch: now that you know the right command, reboot and type it in as fast as possible
<clever> hyper_ch: also note that there may be a seperate issue, where it doesnt work until the uptime is 2 minutes, and your too slow to catch it