2020-07-23

<clever> and unzip adds an unpack hook, to enable zip support
<clever> CMCDragonkai_: the trick, is that fetchzip just adds unzip to the nativeBuildInputs, and runs the normal unpackPhase function
<clever> CMCDragonkai_: pkgs.fetchzip can fetch tar files
<clever> your welcome
<clever> yeah, it was added relatively recently
<clever> ilios: you need to comment it out for nixos-option to parse it fully
<clever> ilios: and does the option exist in 19.09?
<clever> ilios: what version of nixpkgs are you using?
<clever> colemickens: another sign, the append-secrets path you verified, was not the one that failed
<clever> colemickens: yeah, it has to run the append-initrd secrets, for every generation, to copy the secrets from not /nix/store to /boot
<clever> colemickens: that will do it!
<clever> colemickens: what does uname -a return? and `ls -l result` vs `ls -l /nix/var/nix/profiles/system` ?
<clever> duairc: id keep them as seperate functions, defining each sub-tree
<clever> fresheyeball: yeah
<clever> fresheyeball: pass them to nixpkgs when you imported it, as normal

2020-07-22

<clever> duairc: why is it having to set options in so many places for each thing in the list?
<clever> you must clearly return something that doesnt contain .foo or .bar
<clever> duairc: that is where your recursion comes from
<clever> duairc: and it cant know what config.foos is, until it knows that doFoo cant return { foos = ...; }
<clever> duairc: to figure out what attributes `doFoo` and `doBar` return, it must first know what config.foos and config.bars are
<clever> colemickens: what command did you run?
<clever> fresheyeball: you need to use pkgsMusl for everything, replace every instance of pkgs with it
<clever> and if your expr is written the wrong way, it has to eval it fully to find the answer
<clever> duairc: nix needs to know which side returns .a and which returns .b, to be able to avoid infinite recursion
<clever> duairc: and that is possibly your problem
<clever> duairc: why do you even need the mkMerge? why not just put everything as simply config = { a = {}; b = {}; }; ?
<clever> duairc: can you pastebin the module?
<clever> colemickens: then having aarch64 binaries is weird, what command did you run?
<clever> fresheyeball: now we have an error, but its not obvious what it means
<clever> colemickens: if you dont force the cpu type, it defaults to whatever cpu the nix-build binary was built for
<clever> fresheyeball: and with `-j 1 --option cores 1` ?
<clever> i think
<clever> fresheyeball: it was building 2 things in parallel and the errors are jumbled together, try again with `-j 1`
<clever> pkgsCross.musl64.pkgsStatic may also help
<clever> it contains the entire pkgs tree, cross-compiled to various targets
<clever> > pkgsCross
<clever> fresheyeball: pkgsCross.musl64
<clever> fresheyeball: it sounds like the musl target may work better
<clever> fresheyeball: and if you paste that error into google?
<clever> fresheyeball: seen nh2's guide?
<clever> colemickens: try doing `nix-store --verify --check-contents` ?
<clever> colemickens: are you doing any cross compiling?
<clever> fresheyeball: you probably want to delete that script
<clever> fresheyeball: then its in the $out, as the #! of a script
<clever> fresheyeball: what did why-depends say?
<clever> colemickens: and are you on an aarch64 machine?
<clever> fresheyeball: either use sed to modify it, or delete that file in postInstall
<clever> fresheyeball: dont include the path to nodejs in your $out
<clever> colemickens: run file on that bash, what does it say?
<clever> fresheyeball: see also, nix why-depends
<clever> colemickens: and the #! is pointing to a normal bash?
<clever> colemickens: what are the contents of that file?
<clever> now its closer to the number given by `nix path-info -Ss`
<clever> 8.8G total
<clever> 542M /nix/store/v8yd6i0jly06xl0lly90i9jfc1bqdlv1-firmware-linux-nonfree-2020-05-19
<clever> 413M /nix/store/fsyxzww54jn4zslflfslc19r61hz97aw-mesa-20.0.7-drivers
<clever> [clever@amd-nixos:~/apps/nixpkgs]$ du --apparent-size -hc --max=0 $(nix-store -qR /run/current-system) | sort -h | tail
<clever> and du is showing on-disk usage, which is upset by the zfs compression i have
<clever> 6.0G total
<clever> 253M /nix/store/mfy3sn7vjdnb17y5ry6hcnsjbr8gwf6j-paper-icon-theme-2018-06-24
<clever> 318M /nix/store/v8yd6i0jly06xl0lly90i9jfc1bqdlv1-firmware-linux-nonfree-2020-05-19
<clever> du -hc --max=0 $(nix-store -qR /run/current-system) | sort -h | tail
<clever> you need to use the right flags with path-info
<clever> $ nix path-info -Ss /run/current-system
<clever> /nix/store/974fn8h449qmy8z7v24lqgvvq98bph5n-nixos-system-amd-nixos-20.09pre230335.9480bae3370 45424 9456416728
<clever> its like running `cat foo | cat | cat`
<clever> those tools both do the same thing
<clever> it only shows build-time when ran on a .drv file
<clever> fresheyeball: nix path-info and nix-store -qR
<clever> nilsirl[m]: boot.kernelPackages = pkgs.linuxPackagesFor pkgs.custom-kernel;
<clever> nilsirl[m]: nothing will actually use custom-kernel until you change boot.kernelPackages
<clever> nilsirl[m]: nixpkgs.config.packageOverrides = super: { .... }
<clever> nilsirl[m]: the first codeblock there is showing you exactly what to set it to
<clever> so if multiple devices share a pw, it only asks once
<clever> sreybastien[m]: stage-1 will retry the same pw on every other device
<clever> sreybastien[m]: if your mounting things in the initrd (like the rootfs), then it only has to ask once, in stage-1
<clever> manveru: `tcpdump -i lo -A port 8200` may still reveal more info, what text is it sending?
<clever> ah, yeah, for that you would want iptables
<clever> manveru: tcpdump -i lo
<clever> that may work?, but might re-eval a lot...
<clever> infinisil: what happens if you use .extend to mutate the config entry, like pkgs.config ....
<clever> and build a nixos around it
<clever> that will reuse the pkgs tree, with whatever overlays it already has
<clever> eval = import (pkgs.path + "/nixos") { configuration = { imports = [ userconfig ]; nixpkgs.pkgs = pkgs; }; }
<clever> and the fix-point and lazyless never eval the 2nd instance
<clever> it wont eval the pkgs tree, because nixpkgs.pkgs overrides the default
<clever> infinisil: you would need to import (pkgs.path + "/nixos") to eval nixos, and then within that, set nixpkgs.pkgs = pkgs;
<clever> so you just need pkgs.path to get the modules
<clever> infinisil: you could also set the nixos option nixpkgs.pkgs to force it to use a pre-overlaid tree
<clever> lovesegfault: yeah, that cant really accept a pre-overlaid nixpkgs without major changes
<clever> lovesegfault: nope, but you can use myPkgs.extend second_overlay
<clever> you may need to include the full network config in configuration.nix
<clever> check the docs for linode
<clever> diegosaa: do you know if linode does dhcp, or needs a static ip?
<clever> ,paste diegosaa
<clever> what about `ifconfig -a` ?
<clever> the network is totally dead
<clever> ah, so the last link in the chain is the serial console
<clever> ah, what does `tty` return?
<clever> then how did you ssh into it?
<clever> did you ssh with v4 or v6?
<clever> what does `ip route` report?
<clever> sounds like the gateway and dns are both down
<clever> diegosaa: can you ping 8.8.8.8?
<clever> diegosaa: can you ping google.com?
<clever> you can do that short-term, to make nixos-rebuild work again
<clever> it usually looks like this
<clever> [clever@amd-nixos:~]$ echo $NIX_PATH
<clever> /home/clever/.nix-defexpr/channels:nixpkgs=/nix/var/nix/profiles/per-user/root/channels/nixos:nixos-config=/etc/nixos/configuration.nix:/nix/var/nix/profiles/per-user/root/channels
<clever> is your .bashrc or .bash_profile setting it?
<clever> what does `echo $NIX_PATH` say?
<clever> then complain that nixpkgs=.... isnt a valid variable
<clever> it will insert the value of the variable
<clever> jgart[m]: also, `unset $NIX_PATH` wont work, the $ causes it to unset something else
<clever> jgart[m]: are you using screen or tmux?
<clever> simplest is to just relaunch the shell

2020-07-21

<clever> {^_^}: lol
<clever> numkem: i also use nixops for some machines, look at deployments/house.nix
<clever> so anything using secrets gets rebuilt
<clever> then when i build the same expr locally, outside of hydra, i have a secrets.nix in my local dir
<clever> so load-secrets.nix supplies dummy defaults, enough to make it build
<clever> hydra never gets a secrets.nix
<clever> nope, the reverse
<clever> look at load-secrets.nix in my nixos-configs
<clever> and rebuild a few configs with secrets
<clever> and its pre-built, so the switch goes faster
<clever> then i can check hydra, to see if updating will even work, or if the build will fail
<clever> correct
<clever> (import <nixpkgs/nixos> { configuration = ./configuration.nix; }).config.system.build.virtualBoxOVA
<clever> check the nixos-configs example i linked above
<clever> it can take a raw path
<clever> you can override that with { configuration = something; }
<clever> numkem: <nixpkgs/nixos/default.nix> will expect your configuration.nix to be at <nixos-config> by default
<clever> numkem: open up `nix repl -I nixpkgs=/home/numkem/src/nixpkgs/nixos` and then try to just eval `<nixpkgs/nixos/modules/virtualisation/virtualbox-image.nix>`
<clever> yeah, that should work
<clever> 2020-07-21 17:51:11 < clever> the nixos is missing
<clever> 2020-07-21 17:51:02 < clever> numkem: nixpkgs/nixos/modules
<clever> numkem: and what path did you use for it?
<clever> numkem: its possible that virtualbox-demo.nix has a different name in your version, check what the nixos/release.nix is doing
<clever> numkem: ls -l /home/numkem/src/nixpkgs/nixos/nixos/modules ?
<clever> numkem: that doesnt look like the root of nixpkgs
<clever> numkem: why is there a nixos at the end of that -I?
<clever> numkem: this configures hydra to map <nixpkgs> to the latest nixos-20.03
<clever> you can also use this, if you already have nixpkgs imported somehow
<clever> > pkgs.path
<clever> that returns a path
<clever> but you can also do `nixpkgs + "/nixos/modules/installer/virtualbox-demo.nix"` and then set nixpkgs to an expr
<clever> numkem: if you make an input named nixpkgs in hydra, then <nixpkgs> maps to it
<clever> numkem: you need to somehow give it the path to a full nixpkgs, either with a derivation or a searchpath
<clever> numkem: and is -I nixpkgs= pointing to the root of nixpkgs?
<clever> numkem: can you paste your expr?
<clever> numkem: no, that shouldnt make any diff
<clever> the nixos is missing
<clever> numkem: nixpkgs/nixos/modules
<clever> numkem: until it is fixed, you must do: (import <nixpkgs/nixos> { configuration = { imports = [ <nixpkgs/nixos/modules/installer/virtualbox-demo.nix> ./configuration.nix ]; }; }).config.system.build.virtualBoxOVA
<clever> id say thats a bug in the release.nix, and it should be fixed
<clever> numkem: but ova ignores it!
<clever> 195 modules = [ versionModule ./modules/installer/virtualbox-demo.nix ];
<clever> 189 ova = forMatchingSystems [ "x86_64-linux" ] (system:
<clever> netboot stuff, will include configuration into the list
<clever> 39 makeModules = module: rest: [ configuration versionModule module rest ];
<clever> 105 modules = makeModules module {};
<clever> 101 makeNetboot = { module, system, ... }:
<clever> numkem: ah, release.nix doesnt respect that for ova
<clever> numkem: what expr are you using? what is the error? what is it not doing?
<clever> na, this one is more to do with how --arg is implemented
<clever> so you can easily miss mistakes when they are named wrong
<clever> numkem: yep, also any unused --arg's are silently discarded
<clever> numkem: each thing in -A can apply the --arg again, https://gist.github.com/cleverca22/240c21dab528666c85a95c4edfdeae58
<clever> numkem: but it can do a few more complex things too...
<clever> numkem: nix-build will basically import the given path, then run it with an attrset containing every --arg, then use -A to select an attr
<clever> (import <nixpkgs/nixos/release.nix> { configuration = ./configuration.nix; }).ova.x86_64-linux
<clever> (import <nixpkgs/nixos/release.nix> { configuration = ./configuration.nix; }).ova
<clever> the release.nix does some of that for you
<clever> nix-build '<nixpkgs/nixos/release.nix>' -A ova.x86_64-linux --arg configuration ./configuration.nix
<clever> ah, that config.system is a nixos option, that any attr can be shoved into
<clever> hydra is basically just a way to automate nix-build against release.nix files
<clever> config.system is the cpu to build for
<clever> numkem: hydraJob just strips un-needed attributes off a thing, to reduce memory usage of hydra
<clever> Guest60204: because 19.09 is a snapshot of what nixpkgs was like on sept of 2019
<clever> virus_dave: ldd forces it to use the right ld-linux, you need to use patchelf to fix it
<clever> eacameron: its been called buildInputs since before the cross stuff got added
<clever> eacameron: yeah
<clever> immae: nativeBuildInputs is for build-time tools like pkg-config
<clever> immae: the buildInputs are meant for libraries for the target, and in some configs are omited from $PATH
<clever> xwvvvvwx: you need to edit $out, not the real /usr/bin of the host
<clever> alexfmpe: do you have a shell.nix?
<clever> alexfmpe: then its directly in $PATH, what args did you run nix-shell with?
<clever> alexfmpe: what does `type git` return?
<clever> freezeboy: i'm also using this to build it without launching the shell
<clever> [clever@amd-nixos:~/apps/nixpkgs-hasura]$ nix-build -E 'with import ./. {}; python3.withPackages (ps: with ps; [ pyrealsense2 ])'
<clever> freezeboy: -I nixpkgs=/path/to/nixpkgs
<clever> which pr?
<clever> ah, in your fork?
<clever> was it added to things recently?
<clever> error: undefined variable 'pyrealsense2' at (string):1:64
<clever> freezeboy: what value did you give to withPackages?
<clever> lol
<clever> my personal hydra is stuck on a version that randomly segfaults when serving http requests, lol
<clever> ivan: ive also run into that and it doesnt look easy without diving head-first into flakes
<clever> $ nix-instantiate --eval -E '{ a=42; }' --json
<clever> {"a":42}

2020-07-20

<clever> M0-[m]: and i dont think the wrapGAppsHook is going to really work at all in the fhs env
<clever> M0-[m]: what happens if you try buildInputs ?
<clever> M0-[m]: first problem i see is that your not using extraPkgs right, it must do `pkgs.gmp` not just `gmp`, that will break 32bit stuff
<clever> M0-[m]: it might be buildInputs
<clever> Graypup_: pkgs.path is one option
<clever> Graypup_: use `path_to_pkgs + "/pkgs/build-support/docker"`
<clever> M0-[m]: extraPkgs is part of the top level function, so you can only change it with .override
<clever> 1 { appimageTools, buildFHSUserEnv, extraPkgs ? pkgs: [] }:
<clever> i dont think anything overrideAttrs acts on, can be a function
<clever> M0-[m]: matrix will pastebin it automatically, and only send a link to irc
<clever> yurb: that wont delete anything from the store, at least on its own
<clever> yurb: if you use nix-diff, you should be able to identify the root cause
<clever> Rian[m]: yeah
<clever> Rian[m]: yeah
<clever> Rian[m]: it would be better to fix the service definition, so it restarts correctly
<clever> zanc: ah, seems to be missing, its basically the same as mkForce, just at a different priority
<clever> zanc: it should be in the nixos manual
<clever> zanc: it sets the value at the default priority, so it wont conflict when you set it in another module without any priority
<clever> energizer: nix-bundle also relies on linux namespacing, so it wouldnt work on a mac
<clever> dang!
<clever> with quotes, oops
<clever> stateDir = ../../path/to/real/state/
<clever> what if you use the old .. exploit? lol
<clever> 8 workingDir = "/var/lib/" + cfg.stateDir;
<clever> 46 "--storage.tsdb.path=${workingDir}/data/"
<clever> ah, not exactly the right option...
<clever> services.prometheus.pushgateway.stateDir
<clever> Directory below /var/lib to store metrics.
<clever> hexa-: and why is it a bind mount?
<clever> just set mountpoint=legacy on the dataset, and fileSystems to mount it as usual
<clever> you can still do it with fileSystems
<clever> hexa-: and then you just need to ensure the filesystem is defined in fileSystems
<clever> nixpkgs/nixos/modules/services/databases/mysql.nix: unitConfig.RequiresMountsFor = "${cfg.dataDir}";
<clever> Example: { RequiresMountsFor = "/data"; }
<clever> systemd.services.<name>.unitConfig
<clever> finding an example...
<clever> hexa-: sounds like you want requires mounts for, not after
<clever> hexa-: just define the after in configuration.nix
<clever> adyatlov: you can also `nix-shell -A pkgsCross.avr.avrlibc` then `unpackPhase` to get an unpacked copy
<clever> and then you can just edit it directly
<clever> adyatlov: and vim can edit tarballs easily, just adjust it to `-o avr-libc.tar.gz` so vim knows what type to expect
<clever> adyatlov: ah, if you want the src, add .src to the path
<clever> samueldr: the bash-completion thing might be fixed, but now hydra is failing to eval due to segfaults! lol
<clever> adyatlov: pkgsCross.avr.avrlibc i believe
<clever> adyatlov: just run nix-build on the right attr path, and the result symlink will keep things alive
<clever> unclechu: you could also pull in a sources.nix, if your willing to accept the script being a combination of 3 files
<clever> unclechu: you can also use builtins.fetchTarball with a sha256
<clever> and it would still get pretty ugly, having to put the entire nix expr on one line
<clever> it would only work if it accepts -E
<clever> it tends to have worse performance, because it has to download the deps when you try to run it
<clever> yeah, those ones cant easily use niv, i tend to avoid them
<clever> thats exactly what shell.nix is for
<clever> unclechu: that goes into your shell.nix file
<clever> unclechu: then you can just do `(import ./nix/sources.nix).nixpkgs` to get the path to nixpkgs
<clever> unclechu: basically, you just `nix-env -iA nixos.niv` then `niv init` and `niv add nixos/nixpkgs`
<clever> > pkgs.niv.meta.description
<clever> > pkgs.niv
<clever> unclechu: using something like niv, you can easily specify sha256's and then be protected from that
<clever> unclechu: yeah, the attacker would need to control a CA or be within github's CDN
<clever> unclechu: when used with -I, the https in the url will be respected, so the attacker will need to break https first
<clever> i'll bump to master and see what happens...
<clever> samueldr: bash-completion fails to cross-compile due to errors in its python based tests
<clever> samueldr: has the bash completion thing been fixed yet?
<clever> yeah, it does have a musl target as well
<clever> currently, it fails due to bash completion not building
<clever> hanetzer: the build from a month ago, is using a nixpkgs that did sucessfully cross-compile
<clever> hanetzer: https://hydra.angeldsis.com/job/things/rpi-open-firmware/arm7.diskImage/all this was building a full SD image for the rpi, based around my open firmware
<clever> yeah, recent master fails for me
<clever> hanetzer: i have had some success in cross-compiling nixos to my rpi3, but a surprising number of #!'s reference the x86 bash, yet somehow still work

2020-07-19

<clever> hexa-: you can also use this syntax to provide a default when the attr is missing
<clever> > ({}).foo or "fallback"
<clever> superbaloo: ah yeah, i think its a fixup hook that makes nix-support
<clever> immae: one sec
<clever> ive also done experimenting, i made an initrd with a single static elf and nothing else
<clever> have you seen the existing nixos initrd stuff? using ldd to copy just whats used
<clever> ah
<clever> it will only be available in other derivations, at build time
<clever> so you wont have cryptsetup available at runtime
<clever> also, propagated inputs dont survive at runtime
<clever> and knowing with storepath each maps to
<clever> making the automatic stuff smarter, would involve recursively discovering the libraries needed by every static lib
<clever> or overwrite it with just 1 path
<clever> ah, then you can just delete the file in $out/nix-support, in a postInstall hook
<clever> but if the derivation is providing a static lib, it needs to propagate everything the static lib depends on
<clever> yeah
<clever> superbaloo: only the final step in the build, the executable does linking, and that needs curl to be propagated
<clever> superbaloo: usually, static libraries arent linked at all, its just an ar archive containing every .o you just built, but not your dependencies
<clever> superbaloo: double-check with objdump, did it actually embed curl into its output library?
<clever> why does it need curl to build, but shouldnt propagate curl further?
<clever> superbaloo: so you would need to remove curl from the normal buildInputs
<clever> superbaloo: youll notice in the show-derivation, that buildInputs is entirely empty
<clever> superbaloo: i think something is detecting that your doing static builds, and it moves all of the buildInputs to propagatedBuildInputs