2017-07-23

<clever> your /lib/x86_64-linux-gnu also has to be removed
<clever> wrapProgram $out/bin/foo --unset LD_LIBRARY_PATH
<clever> so you need to create a wrapper script using wrapProgram in your nix expression
<clever> the problem is that the python in nix is obeying env vars you have set
<clever> kiloreux: what if you only remove it for the python process, using a wrapper script?
<clever> kiloreux: removing /srv from LD_LIBRARY_PATH may fix it
<clever> kiloreux: looks like you have the wrong zlib in your LD_LIBRARY_PATH
<clever> yeah, that might work better,given that cmakeDir is unusable
<clever> Infinisil: you would need to set dontUseCmakeBuildDir and then do the build dir yourself in preConfigure if you want to keep it
<clever> Infinisil: yeah, that will probably break things
<clever> mpickering: every attribute in a derivation becomes an env variable during the build
<clever> yeah

2017-07-22

<clever> and it will then do an out-of-tree build (the .o's are seperate from the src)
<clever> when you run cmake, you give it a path (relative or absolute) to the dir where the source and CMakeLists.txt exists
<clever> ah
<clever> LnL: yeah, i think thats making bash do regex?
<clever> i have no idea how, but that sed-like function is implemented in pure bash
<clever> yeah, its also harder to insert storepaths with a .patch
<clever> its mostly just preference
<clever> yeah
<clever> the patch would fail to apply, and draw attention to what needs to be fixed
<clever> sed may just silently ignore things if upstream changes something, and then you get odd errors
<clever> a patch is more likely to fail in a safe way
<clever> avn: that usualy leaves the ssh privatekeys in the nix store, and visible on your binary cache for the world to see
<clever> sheenobu: then you need to disable that, and copy the source in during postUnpack
<clever> mpickering: and now you have a second derivation, being used as the source for the first
<clever> mpickering: you just do src = fetchgitPrivate { ... };
<clever> and only if it runs on the local machine (build slaves will break it)
<clever> just during fetchgitPrivate
<clever> so ssh-agent thinks socat (running as root) is doing the requests, and allows it
<clever> and root is exempt from that safety
<clever> the socat in my gist will proxy it over
<clever> so the git running as nixbld1 cant use it
<clever> sheenobu: but the ssh-agent in the ssh package, actively rejects any connection coming from the "wrong" user
<clever> sheenobu: fetchgitPrivate expects <ssh-auth-sock> to point to a unix socket for an ssh agent
<clever> mpickering: fetchgit and fetchgitPrivate auto-generate such a derivation
<clever> mpickering: all network access must be done in a seperate derivation, that declares the output hash ahead of time
<clever> cherrybl0ss0m_: and then in github, create a personal access token that has repo:status
<clever> cherrybl0ss0m_: create an xml section called <github_authorization> that contains "orgname = token <sha1>"
<clever> sheenobu: i just use an android device for netflix
<clever> cherrybl0ss0m_: https://github.com/cleverca22/hydra-configs in here are the declarative configs for everything on my hydra, half of them have pull-request support setup
<clever> cherrybl0ss0m_: yes
<clever> obviously, thats for the chromium package (not google-chrome or firefox)
<clever> chromium.enableWideVine = true; has worked for me in the past
<clever> netflix uses widevine on chrome and something else on firefox
<clever> netflix doesnt use flash
<clever> ive just not turned flash on
<clever> cwre: and yeah, adobe is free to break it by just deleting the tar, and nix will demand you get the exact right version
<clever> M and P to adjust the sort order
<clever> also, check 'top' and see what is using the cpu and ram
<clever> cwre: i think your setting nixpkgs.config.config.firefox
<clever> cwre: then it has to go under nixpkgs.config.firefox.enableAdobeFlash = true;
<clever> cwre: how is firefox installed?
<clever> cwre: firefox.enableAdobeFlash = true; in whatever config.nix your firefox is based around
<clever> yep
<clever> yeah, the fixupPhase runs it over $out/bin/
<clever> mudri: the buildEnv would atomicaly replace the last buildEnv you had by the same name
<clever> mudri: then it lands in ~/.nix-profile/bin/ which is already in PATH
<clever> mudri: for PATH, i would just create a buildEnv based derivation, and nix-env -iA it
<clever> rvolosatovs: change the {path}: to just path:
<clever> sheenobu: and then it runs that busybox on this script, which uses busybox to unpack the tar, and patchelf to fix everything to refer to its new $out
<clever> sheenobu: this is a bare busybox binary, staticly linked, not even in a tar, and a tar containing glibc + gcc + patchelf
<clever> sheenobu: nixpkgs uses this to bootstrap everything: https://github.com/NixOS/nixpkgs/blob/master/pkgs/stdenv/linux/bootstrap-files/i686.nix
<clever> sheenobu: in the same directory is a txt showing the output you can expect
<clever> sheenobu: this is a bare-bones derivation that doesnt use the stdenv directly
<clever> rvolosatovs: this is a path, a division operator, an addition operator, and a string
<clever> error: syntax error, unexpected '+', at (string):1:11
<clever> nix-repl> ./result/ + "/bar"
<clever> can you paste the exact code you tried?
<clever> an unquoted path has several special properties, and you can concat more to it and keep those
<clever> rvolosatovs: just do (./foo + "/bar")
<clever> note the type code on line 30
<clever> silver_hook: and if you want to make GPT boot via legacy: https://gist.github.com/cleverca22/d332976c361dcee9d9a4032c84d9cca6
<clever> just make sure /boot is outside the luks
<clever> and thats pretty much it
<clever> and this tells it to search for a pool called laptop (it looks in lvm automatically) and mount that as /
<clever> fileSystems."/" = { device = "laptop/root"; fsType = "zfs"; };
<clever> silver_hook: this tells nixos to open the luks device at /dev/sda3 before scanning for lvm devices
<clever> silver_hook: boot.initrd.luks.devices = [ { name = "root"; device = "/dev/sda3"; preLVM = true; } ];
<clever> silver_hook: let me see
<clever> avn: none of the machines have swap on zfs
<clever> and the desktop has 64gig of NVME swap
<clever> but the desktop, it just hangs, it doesnt even try to use the swap
<clever> sure, it gets io bound in swap all the time, but it recovers
<clever> silver_hook: oddly, the laptop with 2gig of ram performs "better" then the desktop with 16gig of ram
<clever> it also has snapshots, so i have undo for almost any file
<clever> i now have the option to use zfs send to backup the entire system
<clever> so the lvm just splits the luks up into zfs and swap
<clever> and zfs for zfs
<clever> silver_hook: luks for encryption, lvm because i know swap on zfs is bad, and i didnt want 2 luks partitions
<clever> silver_hook: my laptop is booting with zfs on lvm on luks
<clever> mpickering: you would need to update the packages config to list all of the packages you want, and then nixos-rebuild to update the docs
<clever> nixos yes, nix no
<clever> silver_hook: nixos needs to generate the grub.conf far more often, you can usualy get away with just putting the other distro into grub.conf via the extraConf option in nixos
<clever> some, but version numbers vary so much that it doesnt always work right
<clever> comparing the version numbers would be more tricky
<clever> i think
<clever> foo = if (old.foo.name == "foo-1.2.3") then old.foo else old.foo.overrideAttrs (drv: { src = ...; });
<clever> nope
<clever> it should
<clever> but you could put an if statement into it, that checks the version of the old package
<clever> the override/overlay will have priority, and will have the final say in what is used
<clever> rvolosatovs: you can do that with either normal packageOverrides, or overlays
<clever> yeah
<clever> alunduil: nixos test framework
<clever> bennofs: you might be able to use libredirect (from nixpkgs) to redirect /proc/self/exe to a shell script
<clever> cwre: so the default kernel uses this set of patches
<clever> and line 12284 generates that over the linux_4_9 attribute
<clever> so nixos uses this alias as a default
<clever> cwre: finding a link...
<clever> which would be what linuxPackages is an alias to
<clever> cwre: probably the one that your using as a default
<clever> cwre: and then use that kernel via its matching linuxPackages set
<clever> cwre: you want to modify one of the kernel attributes near my link to refer to your new patch
<clever> cwre: i think you need to reference the patch elsewhere
<clever> /home/clever/apps/nixpkgs/pkgs/top-level/all-packages.nix: kernelPatches.p9_fixes
<clever> only real issue, is that it only protects against accidental deletions, since the backups are on the local disk
<clever> you now get automatic backups every 15mins, hour, day, week, and month, each one with its own max# that expires automatically
<clever> yegortimoshenko: https://nixos.org/nixos/options.html#zfs.autosna and set a flag on the filesystem with "zfs set"
<clever> yegortimoshenko: zfs snapshots are great to have a rolling backup system

2017-07-21

<clever> yeah
<clever> ottidmes: at a glance, i dont think the cpu supports pagefault detection
<clever> ottidmes: that will depend heavily on how reliable the unbricking method is
<clever> but thats typically handled by the existing kernel
<clever> if the cpu has an MMU, they you should be able to set a pagefault handler that catches access to unmapped memory
<clever> celph_: which pebble?
<clever> celph_: though it still relies on ulimit being set as normal
<clever> celph_: i recently did systemd.coredump.enable = true; and have found it handy in catching things
<clever> tilpner: yep
<clever> and then apply the overlay to that nixpkgs
<clever> tilpner: internally, it will re-import its own chosen nixpkgs version, and not force the config
<clever> tilpner: if you import the default.nix from this "overlay", it wont actualy be used as an overlay
<clever> tilpner: oh, i think i know
<clever> tilpner: all of that is inside a copy of nixpkgs-channels from revision ed07, which isnt mentioned in any of the fetchurl or fetchgit calls
<clever> tilpner: can you add the entire trace as another file in the previous gist?
<clever> tilpner: also, useSandbox is a nix option, not a nixpkgs option, so it does nothing on lin e13
<clever> tilpner: that should give a backtrace to where it came in from
<clever> tilpner: ah, try putting a syntax error into your config.nix and then run it with --show-trace
<clever> tilpner: can you link a gist with the file?
<clever> tilpner: which argument?
<clever> ottidmes: so you can boot the vm once, then test a single function on 1000's of different bits of data
<clever> ottidmes: and then each clone of the vm gets different input data
<clever> ottidmes: triforce works by forking the entire bloody qemu process, to create clones of the vm
<clever> ottidmes: there is even a variant that is embeded into a modified qemu, that can fuzz kernels
<clever> tilpner: nix-build --arg config '{}'
<clever> joepie91: but then nobody else benefits! lol
<clever> joepie91: and i have considered setting loose ~30 bots in this channel named after nix functions, just so i can tab-complete things like callPackage in here :P
<clever> joepie91: my irc client is good at completing with just 2 letters most of the time
<clever> ottidmes: so you dont even need support for custom ISO's
<clever> ottidmes: the directory above that file, is a kexec trick i made, that can hijack any existing linux machine
<clever> ottidmes: so you could create an ISO that has this module included, boot the ISO, ssh in, and run justdoit, and your done
<clever> ottidmes: when ran, that will format /dev/sda, and install nixos
<clever> ottidmes: this is a custom nixos module, that pre-installs a script called justdoit
<clever> ah
<clever> joepie91: ah, its the default value?
<clever> ottidmes: and its up to you to deal with the hardware layer (or creation of VM's within control panels)
<clever> ottidmes: if you set the targetEnv = "none"; in nixops, it will just ssh into an existing nixos machine and manage it
<clever> ottidmes: i have considered using nixops just to manage my laptops
<clever> ottidmes: i have also written a kexec based tool that works similiarly to nixos-infect
<clever> cwre: and nix's closures can basicaly do the same thing, just write the right expression s and it does it for you
<clever> cwre: without it depending on any part of the host
<clever> cwre: with LFS, they need a toolchain at a non-standard path, that can be transplanted from the host to the guest
<clever> cwre: yeah
<clever> cwre: nix simplifies it with storepaths, but does the same basic process
<clever> cwre: and then repeat it all, but against / this time, to make the real install
<clever> cwre: then you can mount that entire thing into a chroot, and be 100% isolated from the original host toolchain
<clever> cwre: so it uses /tools/lib and /tools/bin/ and even /tools/lib/ld.so
<clever> cwre: the basics of LFS, is building an entire toolchain (bash, gcc, glibc), that is rooted to /tools/
<clever> lol
<clever> cwre: having done LFS, i'm able to read and understand the stdenv bootstrap in nixpkgs
<clever> cwre: i have also ran linux from scratch on my router, so yes
<clever> i was mostly gentoo based prior to discovering nixos
<clever> i have nixos on my desktop, 2 netbooks, 1 laptop, the router, and the NAS
<clever> simpson: pretty much any haskell program in existance is calling this code on a regular basis: https://github.com/ghc/ghc/blob/master/rts/sm/MBlock.c#L106
<clever> simpson: the GHC RTS has a hefty amount of C in it...
<clever> joepie91: i find i dont even look hard enough for such tooling
<clever> mbrock: i just skipped cabal and stack, and went directly to haskellPackages.ghcWithPackages, and it works fine
<clever> joepie91: this script also breaks if any of the cargo deps happen to use a git submodule
<clever> yeah
<clever> so the fetch and build always use the same lock file
<clever> joepie91: nix will re-do the fetch job if the lock file has been modified
<clever> but its trying to do network again afterwards
<clever> then cargo should just run without network
<clever> and then you provide the sha256 over that entire set of deps
<clever> joepie91: nixpkgs uses the lock file to pre-fetch things in a fixed-output derivation when it has network
<clever> joepie91: except, nixpkgs has done some trickery to provide it everything pre-fetched
<clever> joepie91: ah, for context: this line is trying to network IO when nix has disabled the network: https://github.com/NixOS/nixpkgs/blob/master/pkgs/build-support/rust/default.nix#L86
<clever> joepie91: do you know much about how 'cargo fetch' runs?
<clever> id have to read more of the expressions to know what exactly is going on
<clever> you may need to apply an override to pkgs.firefox-nightly-bin then
<clever> Infinisil: the overlay is already setup, no need to use mozillaPkgs
<clever> Infinisil: i think its better to just do pkgs.firefox-nightly-bin
<clever> Infinisil: yeah

2017-07-20

<clever> profile can also source bashrc
<clever> ottidmes: and bashrc will source profile in some cases
<clever> ottidmes: interactiveShellInit lands in /etc/bashrc, shellInit lands in /etc/profile
<clever> gchristensen: sure
<clever> just 13
<clever> overlay*
<clever> your already applying an override, so just delete line 13 of the gist, and use pkgs.firefox-nightly-bin
<clever> config isnt an argument
<clever> ah, i see
<clever> so you can just pkgs.firefox-nightly-bin
<clever> though, the overlay your adding puts that nightly into the main pkgs set anyways
<clever> then it wont read any config.nix, and it will just work
<clever> cwre: you probably want mozillaPkgs = import mozillaPkgsDir { config = { allowUnfree = true; }; };
<clever> cwre: line 13 imports a new copy of nixpkgs, as root, so it reads roots config.nix file
<clever> cwre: what about the /root/.config/nixpkgs/config.nix?
<clever> though that instance is only used for a fetchFromGitHyb
<clever> cwre: line 5 overrides the config, so it doesnt read any config.nix files
<clever> cwre: can you gist that file?
<clever> cwre: what command is causing the unfree error?
<clever> cwre: what is it set to?
<clever> cwre: it shouldnt be
<clever> cwre: is $NIXPKGS_CONFIG set to anything?
<clever> cwre: oops, ^^^
<clever> celph: does /etc/nix/nixpkgs-config.nix exist?
<clever> gchristensen: pong
<clever> sphalerite[m]: yeah, wider hardware support may bloat it more
<clever> depends on the goals
<clever> sphalerite[m]: yeah, but the i3 is probably optional, if you wanted it to only display a pdf and do nothing else
<clever> sphalerite[m]: its about 40mb
<clever> though that currently lacks X support
<clever> sphalerite[m]: heh, sounds like something i would do with not-os
<clever> boomshroom: i think you can also run this with a normal nix, that was compiled against /nix/store/
<clever> boomshroom: an example i had made a made nearly a year ago
<clever> 2016-08-19 05:25:17< clever> [clever@amd-nixos:~]$ NIX_LOG_DIR=/home/clever/nix2/var/log NIX_REMOTE= NIX_STORE=/home/clever/nix2/store NIX_STATE_DIR=/home/clever/nix2/var/nix/db nix-build '<nixpkgs>' -A hello -Q -j8 --option build-use-sandbox false --option build-users-group ""
<clever> boomshroom: there is also an env variable that overrides the store location
<clever> boomshroom: try unsetting $NIX_REMOTE
<clever> and also, the hash in /nix/store/<hash> is only the first 160 bits of the sha256
<clever> i believe the <hash> on line 154 is the sha256 from the fetchurl, and the hash of this entire string is what goes into /nix/store/<hash>-name
<clever> wait no, that part is excluded for fixed-output ones
<clever> and the "/nix/store" string is in that outer hash
<clever> yeah
<clever> a character is prepended to the hash algo
<clever> taktoa: slightly different rules for that
<clever> line 104 has a giant comment explaining things
<clever> which is just several of those functions jammed into one
<clever> taktoa: it appears to use makeOutputPath with the output name and that hash
<clever> note the if statement on line 338
<clever> *looks*
<clever> boomshroom: yeah, you can just make a dummy home folder, chown it, and cd over
<clever> michaelpj: yeah
<clever> boomshroom: and if its in your home dir, you need to use the same home everywhere
<clever> boomshroom: yes
<clever> michaelpj: so if it exists, the one in home is ignored
<clever> michaelpj: that is a default that has higher priority then ~/.nixpkgs/config.nix
<clever> /etc/nix/nixpkgs-config.nix
<clever> [clever@amd-nixos:~]$ echo $NIXPKGS_CONFIG
<clever> but you can do nixpkgs.config = import /etc/nix/nixpkgs-config.nix; to shortcut that
<clever> nixos will only ever obey nixpkgs.config
<clever> nix-env and nix-build ignore the overrides in configuration.nix
<clever> spinus: nope
<clever> boomshroom: yeah, nix-copy-closure
<clever> michaelpj: that sounds like how i would set them up
<clever> boomshroom: nixops only helps if you want nixos, but if you just want nix on another distro, nixops wont help any
<clever> michaelpj: can you gist both examples?
<clever> GPT uses more then 1 sector
<clever> thats enough code to let it read /boot, and then it can do things right
<clever> grub's kernel, and fs driver (ext4 for example) get concat'd together, and then throw into "free" space between sector 0 and partition 1
<clever> that sounds like grub's stage 1.5 on MBR
<clever> lol
<clever> the userland translates things to a simpler form
<clever> there is basicaly no lvm support in the kernel, same for luks
<clever> yeah
<clever> i mainly booted my gentoo without an initrd
<clever> deltasquared: ive previously used linuxfromscratch and gentoo, nixos install wasnt that complex
<clever> deltasquared: oh rather, kexec is the only way to bypass that, and my kexec trick relies on the MBR being obeyed
<clever> deltasquared: yeah, kexec requires that the host still runs the boot sector in the MBR
<clever> but with any decent datacenter, you can remotely wipe and try again