2017-05-09

<clever> tywkeene: have you tried just using chromium instead?, its a nearly identical browser
<clever> tywkeene: i can only see .deb files for chrome 50, 51, 52, and 53
<clever> tywkeene: if i open the dir in the error: http://repo.fdzh.org/chrome/deb/pool/main/g/google-chrome-stable
<clever> tywkeene: what url is it showing as 404'ing?
<clever> heh, yeah that works
<clever> domenkozar: and i dont have the exact url for the luks page memorized
<clever> domenkozar: though search engines cant index the archive
<clever> domenkozar: that could work
<clever> gchristensen: yeah, but having any kind of access to the archive, that new users cant stumble upon, would allow moving the data to better docs
<clever> last i looked, the zfs and luks pages where still accurate
<clever> any backup or archive of the wiki that somebody could dig info out of?
<clever> :)
<clever> ah
<clever> simpson: trying to enforce sandboxing outside of the builds sorta?
<clever> simpson: signing can do the same thing, verify who the sender is, but not hide the content of the message from others
<clever> gchristensen: are we talking about signing or full encryption of things in the store?
<clever> Svarog: i think the customized vim is just a vim shell-script that runs the real vim, so everything else in $out/bin/ is missing
<clever> why is that part of vim!?
<clever> uhhh, what? lol
<clever> i'm used to redirects forcing ssl
<clever> ah
<clever> yeah, i can load things without ssl
<clever> i think it just lacks a redirect entirely now
<clever> niksnut: http://nixos.org is a redirect to http://nixos.org
<clever> sphalerite: my hydra-module is where gc-check-reachability came from
<clever> https is up now
<clever> niksnut: oops, wrong tab-complete, ^^^
<clever> nixy: connection refussed on nixos.org!
<clever> sphalerite: one of these i think: gc-keep-outputs, gc-keep-derivations, gc-check-reachability
<clever> nixy: that feels unrelated, the linode page i read just does the old curl|sh and then forcibly gives you nixos-install via nix-env
<clever> ive used the linode guide on the wiki to do similiar things on other providers and local systems: https://nixos.org/wiki/Install_NixOS_on_Linode
<clever> domenkozar: yeah, nixUnstable.perl-bindings on nixpkgs master is correct
<clever> domenkozar: but setting NIX_STORE_DIR in the env hydra runs under should temporarily fix it
<clever> domenkozar: the default in here is broken: /nix/store/lpa61xrasw7xvv72s46k3zlw26crfvv9-nix-perl-1.12pre5308_2f21d522/lib/perl5/site_perl/5.22.3/x86_64-linux-thread-multi/Nix/Config.pm
<clever> $storeDir = $ENV{"NIX_STORE_DIR"} || "";
<clever> domenkozar: yeah
<clever> domenkozar: the code says the error should contain /nix/store/, but it doesnt!
<clever> 269 gone($c, "Path " . $path . " is no longer available.") unless isValidPath($path);
<clever> 267 $path = $Nix::Config::storeDir . "/$path";
<clever> domenkozar: oh!
<clever> domenkozar: id say its either the missing /nix/store in the path, or the isValidPath api is broken
<clever> ok, so isValidPath should have returned true
<clever> domenkozar: what does "nix-store --query --hash /nix/store/ny5yph7mj2c09z2998pgbdaxz78is4ac-cardano-sl-explorer-1.0.0" say?
<clever> gone($c, "Path " . $path . " is no longer available.") unless isValidPath($path);
<clever> i'm fetching a copy of the hydra source to check it as well
<clever> odd
<clever> domenkozar: is the path in the nix store as well?
<clever> domenkozar: hydra uses 0-byte files in /nix/var/nix/gcroots/hydra that have the same name as entries in /nix/store/
<clever> domenkozar: hydra uses its own roots, seperate from normal nix roots
<clever> unicode quotations dont always survive
<clever> domenkozar: what is it doing or not doing?
<clever> nix-store --add-root result --indirect /nix/store/foo
<clever> gchristensen: i'm also interested in flagging a service to restart after standby
<clever> havent read its source so i cant say for sure
<clever> and some clients may warn you about servers that still allow it
<clever> ssl 2.0 has some major security problems with it, so pretty much all servers refuse ssl 2.0 now
<clever> ah
<clever> one step in glibc needs several gig of ram, and fights over gcc's for it
<clever> and even that has issues sometimes
<clever> 4 cores on my pi, 1 derivation with make -j4
<clever> viric: moar swap

2017-05-08

<clever> the service has to be enabled
<clever> Filystyn: teamviewer only works if you run a daemon as root, so you cant just run it under a nix-shell
<clever> neat
<clever> Infinisil: this will run "shutdown -r +5" every hour on the hour
<clever> nice
<clever> so you need to comment it out after your done testing in build-vm
<clever> so the config becomes invalid when you try to use test/switch/boot/build
<clever> samae: it has to go into configuration.nix, and sadly, its only valid when using build-vm
<clever> heh, i once asked somebody for the nixos-version output, and he cut the hash off 1 character away from it being unique
<clever> but i can see the value when you want to open the link up a year later
<clever> ah, i often try to fight github whenever it does that, makes the links much longer
<clever> turning the sandboxing on prevents that, and makes such bugs fail much sooner
<clever> also, if you dont build with sandboxing, you can read things you dont technicaly depend on, and create fun to find bugs
<clever> and then things will break if i dont somehow supply that later on
<clever> but by applying unsafeDiscardStringContext, i can reference something without depending on it
<clever> uncompressed, and compressed
<clever> so if nix was left alone, it would ship 2 copies of the os every time you nix-copy-closure
<clever> nh2: via line 94
<clever> nh2: but line 115, the kernelParams includes a reference to the final rootfs, unpacked
<clever> nh2: with that line in not-os, line 114 copies the entire rootfs to $out
<clever> if you set virtualisation.graphics = false; in configuration.nix, it will use a serial console on the tty, rather then a gui console
<clever> ah
<clever> and you where editing the wrong one?
<clever> ah
<clever> domenkozar: ah, both visible and !internal did it?
<clever> and that lets you cheat in some fun ways
<clever> nh2: but!, this string has no context
<clever> "/nix/store/rkvwvi007k7w8lp4cc0n10yhlz5xjfmk-hello-2.10"
<clever> nix-repl> builtins.unsafeDiscardStringContext hello
<clever> so fragments of the string still have the context, and pass on the dependancy
<clever> and line 1657 then makes a new string, with that context
<clever> line 1653 will extract the context from the input string
<clever> and this works even if you mess with the string via builtins.substring
<clever> nh2: and if that string winds up as the input to stdenv.mkDerivation, the newly made derivation will "magically" depend on hello being built
<clever> nh2: so the string "/nix/store/rkvwvi007k7w8lp4cc0n10yhlz5xjfmk-hello-2.10" has some invisible state, that says it depends on the hello derivation
<clever> or output paths i think
<clever> nh2: every string in nix, has a list of .drv paths behind it, what that string depends on
<clever> nh2: have you seen how context works on strings in nix?
<clever> never touched the stuff :P
<clever> i saw an open PR for it
<clever> and why i get horid performance when i do input = ./5gig-file.pdf;
<clever> which is also why it warns about things over 256mb in size
<clever> domenkozar: and uses that hash as a string to embed into the .drv
<clever> domenkozar: it loads the entire path into ram, serializing it into a NAR bytestring, then hashes it, and unpacks it to /nix/store
<clever> domenkozar: i'm pretty sure nix-instantiate does the copy
<clever> so it is pure, after the copy is done, and depends on the hash(file contents), rather then what value it happens to have when the build reads it
<clever> and next time you start a build, it copies it again, and gets a different storepath, which causes all the things depending on it to change paths
<clever> so if the contents of the file changes mid-way thru a build, it keeps using the old version
<clever> nh2: "${./foo}" copying files around, is to lock them in at a given value
<clever> Unode: nix will ignore timestamps when hashing, but .nfs files could probably mess with things
<clever> so the substring stuff on line 1653 right below, will copy to the store
<clever> yeah, the default is true
<clever> but ${} must call it with true
<clever> so the builtin toString calls it with false there
<clever> oh!, the last argument is copyToStore
<clever> it is noted in the source code
<clever> but only if its being treated like a string
<clever> so toString does something different from just inserting it into the middle of a string
<clever> toString ./notes.txt == "/home/clever/apps/nixos-installer/installer-gui/notes.txt"
<clever> domenkozar: but if you run toString on a real path, it turns into an absolute path, not in the store
<clever> i dont know if toString being different is a bug or not
<clever> so toString does something very difference from just treating it as a string
<clever> "${./notes.txt}" == "/nix/store/55j24v9qwdarikv7kd3lc0pvxdr9r2y8-notes.txt"
<clever> toString ./notes.txt == "/home/clever/apps/nixos-installer/installer-gui/notes.txt"
<clever> and i can see where the confusion comes from
<clever> that casts it to a string
<clever> nh2: when inside a string i believe
<clever> i usualy "${./foo}" to force it to be a string and in the store
<clever> so it depends on what you apply to the string
<clever> but there are some methods that will coerce it to the original path
<clever> bb4kk3r, nh2: ah, if you do it inside a string, it will import the path to /nix/store, based on the hash of the contents
<clever> bb4kk3r: both hydra and nix-build will refuse to build it for any other platform
<clever> bitonic: how is the file being referenced? example?
<clever> its not exactly going to parse as a simple attribute
<clever> if the user later adds (import nixos-unstable {}).foo to systemPackages
<clever> and also the potential for full expressions in the file
<clever> but i needed a way to parse configuration.nix if i wanted to edit the config again later
<clever> with that ui, you can lookup any nixos option in a tree, and edit the resulting configuration.nix
<clever> domenkozar: and this is what i made after digging around in the nixos options a few months back: https://www.youtube.com/watch?v=rIdPKzYTN-w
<clever> i just ran nix-repl '<nixpkgs/nixos>'
<clever> yeah, then just config={}; and it should lock it down
<clever> from before the PR had been accepted
<clever> there was a callPackage override in my config.nix, that entirely replaced it
<clever> i have run into problems before where i was trying to debug a package in nixpkgs, and nothing i changed had any effect
<clever> domenkozar: depends, setting config={}; will make it a lot more stable and reproducable, but also cause overrides to not work for people that dont notice it
<clever> bb4kk3r: lib.platforms has a bunch of templates you can use
<clever> bb4kk3r: meta.platforms
<clever> domenkozar: the config argument for nixpkgs isnt being set, so this will load ~/.nixpkgs/config.nix and behave differently for every developer
<clever> domenkozar: i also noticed a minor problem here, https://github.com/NixOS/nixops/blob/master/doc/manual/default.nix#L5
<clever> true
<clever> nix-repl> options._module.args.internal
<clever> yep
<clever> heh, 12 days ago it was added!
<clever> oh, i dont remember there being that one
<clever> dont think thats the area i'm thinking of
<clever> let me see where that went
<clever> domenkozar: hmmm, i think we would need to look at how nixos options are generated
<clever> jophish: then a process like nix-serve wont have read access to the secret key
<clever> jophish: another extension to the idea i had, is for nix-daemon to sign things at build time, rather then when nix-serve is trying to serve them
<clever> gchristensen: second, you could verify that paths in /nix/store where signed by cache.nixos.org, without having to re-download things, and exclude 95% of the storepaths that you may want to audit
<clever> gchristensen: a few things, first, nix-serve/nix-copy-closure would be able to reuse the original cache.nixos.org-1 signatures
<clever> gchristensen: what do you think of keeping the signatures after downloading from the cache?
<clever> gchristensen: modifying nix to keep the signatures in the db.sqlite after downloading from the binary cache
<clever> gchristensen: that reminds me, there is an idea ive been wanting to make a PR for
<clever> some things like the profiles currently lack enable/disable options: https://github.com/NixOS/nixpkgs/tree/master/nixos/modules/profiles
<clever> domenkozar: yep, but you can also use imports
<clever> tommyangelo: and i feel backing up home is much safer
<clever> so there is no performance difference
<clever> and in the hasAttr case, line 12 just directly copies the function reference
<clever> this for example
<clever> so the lib variant works on older copies of nix as well
<clever> that calls the builtin for you
<clever> the one in lib is turned into an if statement
<clever> but then its later found to be a performance bottleneck, so it gets moved to a builtin
<clever> ertes-w: some stuff is initialy added in lib, written purely in nix
<clever> in theory, you could run luksipc from the install cd, mount the luks'd root, then fix the configuration.nix and re-run nixos-install (which is just a script to nixos-rebuild under a chroot)
<clever> tommyangelo: there is http://johannes-bauer.com/linux/luksipc/ but ive never used it, and getting it to boot afterwards with nixos in the mix may be tricky
<clever> i think
<clever> postInstall = ''wrapProgram $out/bin/tmux --add-flags "-f <config>"'';
<clever> wrapProgram $out/bin/tmux --append "-f <config" i think
<clever> tmux = tmux.overrideDerivation ({old: { postInstall = "...."; });
<clever> it sounds like it doesnt have a -f flag, so such a wrapper wont work
<clever> 2017-05-08 00:50:38 < rcschm> no.
<clever> 2017-05-08 00:50:33 < clever> does tmux have a commandline argument that can change the config path?
<clever> does tmux have a commandline argument that can change the config path?
<clever> oh
<clever> the above is for replacing ~/.nixpkgs/config.nix
<clever> rcschm: it will obey $NIXPKGS_CONFIG if that is set
<clever> thats nixpkgs-unstable, for use on other distros
<clever> hyper_ch: nixos-unstable hasnt updated for 4 days
<clever> hyper_ch: are you using master or a channel?
<clever> hyper_ch: evening
<clever> simukis: nix-store --delete
<clever> you want to use packageOverrides
<clever> and maybe not even those, callPackage has a ref to the original pkgs
<clever> so its not impacting the deps inside it, only the things that refer to it later on (the few things you callPackage)
<clever> but you did that after everything inside referenced stdenv
<clever> you overwrite the stdenv attribute on the pkgs attrset
<clever> simukis: ah, i see why
<clever> then the change isnt impacting a depdency of the shell
<clever> because its doing stdenv = { mkDerivation = ...; }; and now everything else is missing from the stdenv
<clever> simukis: that would also overwrite the entire stdenv and break everything
<clever> stdenv is the main stdenv from nixpkgs
<clever> no
<clever> cant see any cause
<clever> which libraries is it stripping?
<clever> simukis: can you gist your nix expression?
<clever> earldouglas: maybe something with pygmentize and hnix
<clever> and it should mass-rebuild
<clever> find the glibc package in nixpkgs, and add foo="bar"; to the derivation
<clever> no help from the cache
<clever> ah, if its /share/nix, then it will have to recompile everything always
<clever> nsswitch.conf configures how to convert uid->name and also domain->ip
<clever> Unode: is the problem on a nixos machine, or a centos with /nix on nfs?
<clever> Unode: oh, what is in /etc/nsswitch.conf ?
<clever> changing it to a different release will definitely change it
<clever> Unode: updating or downgrading the channel should force it to get a new glibc
<clever> simukis: dontStrip = true;
<clever> so nearly everything failed to compile
<clever> turns out, the inodes are 64bits wide, and the 32bit userland only partialy supports it
<clever> Unode: i had some very nasty surprises when i tried to run a 32bit os on a 64bit nfs server
<clever> yeah, just thinking m4 can have other weird bugs
<clever> ive found that m4 spits out errors when pkg-config is missing, yet still makes a semi-usable configuration script

2017-05-07

<clever> sphalerite: stage2 remounts it read-only here, but if some naughty python was ran before that point, it could mess things up
<clever> and if python was being naughty at one of those times, it could try to re-optimize pyc files
<clever> but the chroot from the installer lacks that, and during early boot it may be missing
<clever> the store is normaly bind mounted read-only, so not even root can write to it
<clever> adding --repair to the previous command should re-fetch it from the cache
<clever> it has a version without _
<clever> sphalerite: the binary cache copy of that path lacks a _sysconfigdata.cpython-35.pyc file
<clever> gchristensen: nix-store --verify --check-contents says what?
<clever> maybe something got ran as root at the wrong time, and it was able to mess with the store contents
<clever> strange then
<clever> the above python may go away
<clever> this will delete 1 byte worth of garbage, and all invalid files
<clever> sphalerite: something else you can do, nix-collect-garbage --max-freed 1
<clever> that is the key question
<clever> but because the build isnt done, you can +w it, and modify
<clever> write a file to $out that is a dup of something security related, and -w it, then sleep until optimize has hardlinked it up
<clever> which just now gives me an idea for an exploit
<clever> and i think --optimize might try to optimize invalid directories that "dont exist"
<clever> nix will leave the $out of failed builds behind for you to investiate
<clever> only block level dedup as far as i know
<clever> and the hash table has to stay in ram
<clever> while zfs dedup, hashes every block as its being written, and looks it up in a hash table, to see if it has to write, or increment a refcnt
<clever> and if 2 files have the same hash, they will share the inode and data
<clever> --optimize will hash the entire file, and hardlink it to /nix/store/.links/<hash>
<clever> yeah
<clever> sphalerite: --optimize and dedup do the same basic thing, but dedup costs a lot of ram, while --optimize is almost free
<clever> so encryption after compression is fine, your already leaking size anyways
<clever> yeah
<clever> but for other filesystems, the difference usualy goes the other way
<clever> /dev/sda1 on / type xfs (rw,noatime,attr2,inode64,noquota)
<clever> 4.0K -rw-r--r-- 1 clever clever 3.3K Feb 12 2016 btrfs
<clever> obadz: this file is using 4.5kb on-disk due to zfs encryption, but contains 6.5kb of data
<clever> obadz: ls -lhs will show the on-disk size, and the size of the contents
<clever> obadz: 4.5K -rwxrwxr-- 1 clever users 6.5K Apr 20 11:27 top_block.py
<clever> but other then that, you just need to try and see what fails
<clever> sophiag: it may say in the git commit for the package
<clever> sophiag: you can set an ignore broken flag, and then see the true failure
<clever> sophiag: usualy a package is marked as broken so you know it fails before the build even starts
<clever> sophiag: vim user here
<clever> jeremejevs: the example in irc uses the dependencies from the target nixpkgs, while the override doesnt
<clever> gchristensen: pong
<clever> fuzzy-id needed networking.enableB43Firmware

2017-05-06

<clever> yeah, that should also affect the perl code
<clever> it uses perl to read the host /etc/nix/nix.conf and override things
<clever> dash: oh, the nixos-install script adds its own --option binary-caches
<clever> ah, i would expect it to obey that one
<clever> dash: how did you configure the binary-caches?
<clever> it was an old server out of an xray machine that got upgraded
<clever> yep
<clever> this is what i have in my router
<clever> i have dual-rank modules, so it maxes out at 8gig
<clever> but it turns out that it only supports 16gig if you use single-rank modules
<clever> i tried putting 16gig of ram into my router last week