2020-01-21

<clever> Raito_Bezarius: yep
<clever> no timestamp issues
<clever> Raito_Bezarius: one handy trick, you can run hydra-eval-jobset to force an eval, and see the errors in realtime
<clever> Raito_Bezarius: the hash-hash-source is going to be a purity problem, try adding name = "source"; to cleanSource
<clever> Raito_Bezarius: yeah, thats usually best
<clever> Raito_Bezarius: hydra-eval-jobset will generate an attribute set for each input, using the code i previously linked
<clever> Raito_Bezarius: what about trackSrc.outPath?
<clever> Raito_Bezarius: both
<clever> Raito_Bezarius: you can freely use builtins.trace within a release.nix
<clever> digash: your welcome
<clever> digash: try -A hello.meta
<clever> digash: your irc client is broken, i am saying dash dash x m l
<clever> digash: why did --xml turn into mxml?
<clever> digash: can you screenshot the terminal?
<clever> digash: exactly as i typed it?
<clever> digash: `nix-instantiate --xml '<nixpkgs>' -A hello --eval --strict`
<clever> digash: you can also `nix-instantiate --xml '<nixpkgs>' -A hello --eval --strict`
<clever> digash: callPackage automates that
<clever> digash: the file defines a function, you must pass it the arguments it needs before it can produce the data your requesting
<clever> ,callPackage digash
<clever> kraem: so if the directions havent changed, it will just reuse the previous build
<clever> kraem: nix will hash all of the build directions to compute what $out should be
<clever> kraem: there is also `nix-build --check` to force a rebuild
<clever> kraem: why do you want to run the build again?
<clever> kraem: oops, yeah
<clever> knupfer: what does -q --roots say as root?
<clever> Raito_Bezarius: it tried to treat your string as a hashmap, and just exploded
<clever> Raito_Bezarius: the perl that parses the json has very little typechecking
<clever> Raito_Bezarius: inputs.nixpkgs must be a set, not a string
<clever> nixpkgs = "git://github.com/NixOS/nixpkgs.git ${nixpkgs-version}";
<clever> inputs = {
<clever> Raito_Bezarius: oh, i see your problem
<clever> Raito_Bezarius: try doing `jq < /nix/store/thatfile` ?
<clever> Raito_Bezarius: go to the build page in hydra, and on the details tab, youll find the $out for it
<clever> Raito_Bezarius: what about `journalctl -f -u hydra-notify` ?
<clever> what do you see?
<clever> Raito_Bezarius: if you check `journalctl -f -u hydra-queue-runner`, you should see info about .jobsets:jobsets
<clever> Raito_Bezarius: hydra will then parse that, and dynamically create/disable more jobsets, which themselves need to eval
<clever> Raito_Bezarius: once .jobsets has done an eval, it will create a job, that just must then successfully build, creating a spec.json
<clever> Raito_Bezarius: if you `strace -f -e execve hydra-eval-jobset <project> <jobset>` then you can see the full translation, as it executes `hydra-eval-jobs`
<clever> or for string type inputs, --argstr
<clever> Raito_Bezarius: this loop, then converts each input, into a --arg
<clever> and the plugin can only return certain keys, its not an inteligent translation from hashmap -> set
<clever> ah, it also explicitely tells you the type of the input
<clever> outPath gets added somewhere along the line
<clever> Raito_Bezarius: it will be this perl hashmap, converted into a nix set
<clever> what exactly is within that set, depends on the type of the input
<clever> so you can import <nixpkgs>
<clever> it will also generate -I foo=/bar, for each
<clever> with names matching the inputs
<clever> it generates one --arg for each input
<clever> the perl script hydra-eval-jobsets, will generate a `--arg nixpkgs '{....}'` as it evals release.nix
<clever> it can also be inputs hydra is internally generating, it seems
<clever> Raito_Bezarius: hydra considers this a no-op, and its producing the exact same store-paths as before, so it didnt even create a new eval#
<clever> Raito_Bezarius: for each eval, hydra will list what inputs it passed to the default.nix when creating that eval
<clever> Raito_Bezarius: oh, and also...
<clever> yeah
<clever> hydra then adds 1 more input ontop of that, declInput, which is where hydra got that spec.json from
<clever> what inputs to fetch, and what nix file to eval
<clever> spec.json has all of the config for the jobset called .jobsets
<clever> though ive not played with that aspect
<clever> which lets you skip the src input entirely, if default.nix and spec.json are in the same one
<clever> declInput is the repo spec.json came from
<clever> Raito_Bezarius: as for why it took 5mins to fail, thats probably the timeout within git, for git://
<clever> Raito_Bezarius: and i got some ideas from nixdroid, on how to make the resulting json, without nixpkgs
<clever> Raito_Bezarius: this repo contains spec.json files for many projects, and code shared between them
<clever> Raito_Bezarius: if you want to see some other examples for declarative jobs, https://github.com/cleverca22/hydra-configs/tree/master/things
<clever> Raito_Bezarius: `sudo -u hydra -i` will cd into the new $HOME for you
<clever> ah
<clever> Raito_Bezarius: since the evaluator runs as `hydra` and just runs `git fetch`, it just needs keys in $HOME for `hydra`
<clever> Raito_Bezarius: so the problem, is that hydra-eval-jobset was unable to fetch the input named src, so it couldnt even get your default.nix
<clever> hence, "connection timed out"
<clever> gitlab probably has that port blocked in the firewall
<clever> Raito_Bezarius: git:// connects over a git specific protocol, on a different port
<clever> Raito_Bezarius: try using ssh:// instead of git://
<clever> Raito_Bezarius: is it a public or private gitlab repo?
<clever> Raito_Bezarius: can you pastebin the spec.json file?
<clever> Raito_Bezarius: and can you pastebin the nix file that `.jobsets` is running?
<clever> Raito_Bezarius: what happens when you run `hydra-eval-jobset track .jobsets` ?
<clever> Raito_Bezarius: which page is the error actually on?
<clever> Raito_Bezarius: the eval of .jobsets should be quick then
<clever> its only purpose, is to create a json file, that describes your other jobsets
<clever> the release.nix within .jobsets should be a fast one
<clever> Raito_Bezarius: the spec.json stuff just creates a jobset called .jobsets
<clever> Raito_Bezarius: yes
<clever> what does it output?
<clever> Raito_Bezarius: as the hydra user, run hydra-eval-jobset, and give it a project and jobset name
<clever> [hydra@nas:~]$ hydra-eval-jobset hydra-example example-unstable-small
<clever> Raito_Bezarius: are you doing IFD?
<clever> Raito_Bezarius: where exactly are you seeing the error message? what lines are around it?
<clever> Raito_Bezarius: `Connection timed out` means that the ip isnt responding, its not an authentication issue

2020-01-20

<clever> aswanson: ah, so you didnt actually update the channels
<clever> aswanson: and i think https://nixos.org/nixos/options.html is based on nixos-unstable
<clever> aswanson: when did you last nix-channel --update?
<clever> adding pkgconfig to the inputs should automate that
<clever> siers: if you only want a single shell script, pkgs.writeScriptBin "name" ''contents''
<clever> CRTified: this would setup port forwarding, 8080 to 80 in the guest: https://github.com/input-output-hk/iohk-ops/blob/master/modules/test-hydra-locally.nix#L13-L20
<clever> CRTified: it has outgoing access only by default
<clever> CRTified: it should have network by default, do you see a device in `ifconfig` ?

2020-01-19

<clever> iqubic: if its an fhs wrapper, then that will break it entirely
<clever> iqubic: check what discord prints to stdout when you click the link
<clever> noonien: $CC, $LD, and $OBJCOPY always point to the target tools

2020-01-18

<clever> for when you mistype ls :P
<clever> > pkgs.sl.meta.description
<clever> o1lo01ol1o: you could just put s3:// into the prefix option i linked, but then you rely on the end-user being able to load s3:// url's in a browser
<clever> o1lo01ol1o: for the iohk hydra, ive got everything going almost directly from s3
<clever> o1lo01ol1o: hydra should still be generating links to those, and the bucket privs should be setup to allow reading any file
<clever> DigitalKiwi: i feel like an alias may lead to more accidents, when you run it on a machine without the alias
<clever> DigitalKiwi: not that i know of, but ive never had an accident with it, i just always `gist -p`
<clever> o1lo01ol1o: it also depends on luck, and if you deleted something that somebody needed
<clever> o1lo01ol1o: yeah
<clever> o1lo01ol1o: S3 is nice, because you can just forget about it and never GC
<clever> o1lo01ol1o: any time you switch the backend of a cache url, or garbage collect, you will have to repeat that whole mess
<clever> o1lo01ol1o: `rm ~/.cache/nix/binary-cache-v6.sqlite*` as root, on every machine that is having problems
<clever> o1lo01ol1o: and if you start deleting things from the cache, it doesnt recover for a month, enless you delete the cache of the cache
<clever> o1lo01ol1o: nix uses these files, to track what nar's exist on a given cache
<clever> /root/.cache/nix/binary-cache-v6.sqlite /root/.cache/nix/binary-cache-v6.sqlite-journal
<clever> o1lo01ol1o: emptying the bucket and GC'ing hydra itself, will only make things worse, because more things will be missing
<clever> o1lo01ol1o: when you moved it, files that where on hydra "disappeared" because they are not in the s3 bucket
<clever> DigitalKiwi: one thing i like about gist, is that you can just `git clone` a gist, to see all of the files and history
<clever> you can also use `gist -u <url> file.nix` to update an existing gist
<clever> > pkgs.gist.meta.description
<clever> > pkgs.gist
<clever> i tend to just run `gist -p file.nix`
<clever> zeta_0: both ghcid and ghci are for haskell files
<clever> zeta_0: it will run `cabal repl`, and do a `:r` every time your source changes, thats all it does
<clever> zeta_0: basically, just run `ghcid -c "cabal repl"`, thats it
<clever> zeta_0: personally, i just use bare ghcid and thats it
<clever> zeta_0: that wouldnt cause the error on line 83
<clever> zeta_0: read line 83 of the paste
<clever> kreisys: and the error matches the one i pasted above?
<clever> kreisys: if you -qR the offending path, what does it say?
<clever> kreisys: fixed-output might work, as long as it has no "runtime" deps, according to `nix-store -qR`
<clever> kreisys: whatever you put into path= must be a plain directory, not a derivation with dependencies
<clever> kreisys: throw EvalError(format("string '%1%' cannot refer to other paths, at %2%") % path % *attr.pos);
<clever> kreisys: thats the path = origSrc; part
<clever> and it sorta made the impure method fatal, by accident!
<clever> thats exactly the problem that switching cleanSource to .path is meant to fix
<clever> kreisys: because it will always create a clean /nix/store/hash1-hash2-source, and use a dirty hash2 in the name!
<clever> kreisys: if you use the default name, then it will rebuild just as much
<clever> kreisys: or the error msg should be improved, to say that you should give a pure name, rather then letting the impure hash leak thru in the name
<clever> kreisys: id say this is a bug in nixpkgs, it should no-context the name
<clever> kreisys: try adding a name="something"; to your lib.cleanSource call
<clever> } else if (attr.name == state.sName)
<clever> name = state.forceStringNoCtx(*attr.value, *attr.pos);
<clever> kreisys: aha!!, column 67, is the name=
<clever> 80 outPath = builtins.path { filter = filter'; path = origSrc; name = name'; };
<clever> :D
<clever> Ashy: that will just build the whole system from that directory, and then the new nixos-rebuild command will be in $PATH
<clever> Ashy: nixos-rebuild -I nixpkgs=/path/to/nixpkgs test
<clever> inferencerules: git bisect!
<clever> can you paste the exact error msg?
<clever> kreisys: what error does hydra give if you try to filter a storepath?
<clever> so they will just copy a subset of the store path, and create another entry in the store, and the string depends on the files it did copy
<clever> filterSource and lib.cleanSource act at eval time
<clever> (if done in a normal derivation)
<clever> so it will want to re-apply "removing some files" and rebuild anything down-stream
<clever> if the 1st path has changed, it doesnt know what impact that might have on your transformation
<clever> purity?
<clever> hydra-eval-jobs always acts on the copy in the nix store
<clever> kreisys: and then if the storepaths are different from last time, it runs hydra-eval-jobs to eval it, and turn the nix exprs into a json blob listing all jobs
<clever> kreisys: hydra-eval-jobset will fetch all sources, then copy them into /nix/store/
<clever> inferencerules: thats a bug in why-depends, youll need to use `nix-store -q --tree` and follow the line up from the problem package
<clever> kreisys: thats why i avoid keeping source in the root of the repo, and then i can filter off just ./foo with lib.cleanSource
<clever> 2020-01-17 21:34:21 < clever> Raito_Bezarius: so i always do src = ./foo; rather then ./.
<clever> 2020-01-17 21:34:11 < clever> Raito_Bezarius: this is also why i tend to put the sources somewhere other then the root of the project
<clever> but, the string itself still changes with the source, and messes with your own hashes
<clever> and wont include it in the sandbox
<clever> nix wont know about you depending on that thing, so it wont copy it to the build machines
<clever> kreisys: and then niv's sha256 wont be right, so you have to abandon niv's help
<clever> kreisys: you would need to use a postUnpack in something like fetchzip, to edit it before the hashing
<clever> Ashy: that seems like a simple enough PR that would get merged
<clever> inferencerules: nix-store -qR <thatdrv> | grep gnome-desktop, would find the 2nd drv you need to give why-depends
<clever> inferencerules: `nix-rebuild dry-run` will show you the final drv you use as the 1st arg to why-depends
<clever> inferencerules: and maybe nix why-depends on the .drv files
<clever> inferencerules: nix-store -q --tree, on the final thing your building
<clever> but the official channels lack that
<clever> if the nixexprs.tar.xz for a channel, contained a .update-on-nixos-rebuild file, then it also gets updated
<clever> 209 if [ -e "$channelpath/.update-on-nixos-rebuild" ]; then
<clever> Ashy: it also checks each channel for a special file
<clever> Ashy: `nixos-rebuild --upgrade` will only `nix-channel --update nixos`, which restricts it to one channel
<clever> hpfr[m]: systemd then promptly dropped into emergency mode, because all of the nas based fstab entries couldnt be found, and it wasnt possible to recover without a reboot
<clever> hpfr[m]: related, i accidentally ran the nas activate script, on my laptop, and because both are nixos, /etc/NIXOS existed, and it happily steam-roled all of my state :P
<clever> hpfr[m]: its more to prevent accidents, rather then stop intentional actions
<clever> hpfr[m]: /etc/NIXOS_LUSTRATE helps to clean the gentoo out of your machine on first boot, which makes it much smoother of a transition
<clever> hpfr[m]: i have intentionally side-stepped that, to convert a gentoo into nixos
<clever> hpfr[m]: it stops something like nixos-rebuild from making a total mess all over an ubuntu machine
<clever> hpfr[m]: its required, thats how shell scripts know your actually on nixos
<clever> when did i break 300? lol
<clever> it would only differ outside of nixops, if you `nix-build ~/nixpkgs/nixos` and dont change the search path
<clever> ${<nixpkgs>} would likely work the same, in this exact case
<clever> yeah
<clever> it also intentionally breaks nixos-rebuild, so you cant un-deploy nixops
<clever> and then adjust NIX_PATH to obey that
<clever> aanderse: this will take whatever nixpkgs is being used to build the machine (see above to adjust), and put a copy of it at /run/current-system/nixpkgs on the remote machine
<clever> one sec
<clever> aanderse: this forces it to ignore the channels and use that specific rev
<clever> # nixops modify -d house deployments/house.nix -I nixpkgs=https://github.com/nixos/nixpkgs/archive/dae9cf6106d.tar.gz
<clever> aanderse: still do
<clever> if you try to filter the result of a filter, it will just work
<clever> lib.cleanSource and friends, can compose together normally
<clever> Raito_Bezarius: so i always do src = ./foo; rather then ./.
<clever> Raito_Bezarius: this is also why i tend to put the sources somewhere other then the root of the project
<clever> Raito_Bezarius: lib.cleanSourceWith
<clever> Raito_Bezarius: find the .drv for 2 different builds, and run nix-diff
<clever> lib.cleanSource ./.; will remove common mistakes from the source
<clever> Raito_Bezarius: if your doing ./. and . changes (for example, due to a result symlink), it will want to build it again
<clever> in this case, you can just blindly `chown +w -R $out` each time you copy something
<clever> because the 1st item created read-only dirs in the way
<clever> you should see, it copied 1 item successfully, then it failed on a 2nd item
<clever> maybe also add an `echo copying $i`
<clever> Raito_Bezarius: change it to `cp -rv` and youll see what is happening
<clever> Raito_Bezarius: or use a flag to cp, to tell it to not preserve that mode bit
<clever> Raito_Bezarius: when cp copies things from /nix/store, the copy inherits the read-only flag, you need to `chmod +w -R` after you copy things
<clever> Raito_Bezarius: can you pastebin the entire expr?

2020-01-17

<clever> noonien: if your running nix-shell on the rpi, and doing arm development, it will want a native built copy, not a cross-built copy
<clever> dsx: the attributes passed to pkgs.substituteAll
<clever> notgne2: and then check the build log for the 32bit system, is it creating similar files? did they wind up in the $dev?
<clever> notgne2: what files are in the $out ?
<clever> notgne2: nix-build '<nixpkgs>' -A llvmPackages.compiler-rt.dev
<clever> notgne2: try with nix-build -A instead?
<clever> when fetching from the binary cache, it can skip un-needed (or un-requested) outputs
<clever> :b llvmPackages.compiler-rt.dev
<clever> notgne2: you need to build .dev too
<clever> notgne2: you should build the 64bit version, and see what is supposed to be in $out
<clever> notgne2: its only generating a $dev and not a $out
<clever> notgne2: then you should see the real error more clearly
<clever> notgne2: run `nix-store -r /nix/store/54fhgwxkvjp4bk9chfpj2mklc67lh9yv-compiler-rt-7.1.0.drv` to repeat that build
<clever> notgne2: how did it fail?
<clever> notgne2: for the cmd in the wiki, you want `--argstr system i686-linux`
<clever> just use i686-linux instead of x86_64-linux in the build command
<clever> 64bit x86 can run 32bit x86 just fine
<clever> for 32bit x86, you dont need a cross compiler
<clever> lovesegfault: my trick is `ls -ltrh /tmp/`
<clever> thats just patchelf'ing a ghc i believe
<clever> a derivation has to opt-in
<clever> its off by default
<clever> same answer then
<clever> on the builder
<clever> then it will use whatever nproc reports, if enableParallelBuilding is set on the drv
<clever> lovesegfault: what does `nix show-config | grep cores` report?
<clever> lovesegfault: for most nixos systems, i dont, hydra is just configured to build from the channels, so things are pre-built, and i know if its going to work or not before i `nix-channel --update`
<clever> lovesegfault: yep
<clever> lovesegfault: then nix why-depends ./result /nix/store/ghc
<clever> lovesegfault: `du -hc --max=0 $(nix-store -qR ./result) | sort -h`
<clever> thats the main way i use it
<clever> lovesegfault: nix why-depends /nix/store/foo /nix/store/ghc
<clever> lovesegfault: you can give why-depends store paths

2020-01-16

<clever> [clever@amd-nixos:~]$ head apps/nixpkgs/nixos/default.nix -n30
<clever> }:
<clever> , system ? builtins.currentSystem
<clever> { configuration ? import ./lib/from-env.nix "NIXOS_CONFIG" <nixos-config>
<clever> lovesegfault: when you import <nixpkgs/nixos> you can pass it a system= arg
<clever> although, considering the error, its likely because i enabled `repeat=2` on the build machine, rather then the inputs
<clever> lovesegfault: https://hydra.angeldsis.com/build/98715#tabs-summary this is the first build to fail, click on the link under "changes"
<clever> lovesegfault: on hydra, you can see that my router stopped building ~2 days ago
<clever> lovesegfault: a full hydra, would give you more though...
<clever> lovesegfault: if you run `nix-build release.nix` it will be dumb, and just build every attribute of the set, which is what you kinda want here
<clever> lovesegfault: thats basically what my release.nix automates
<clever> have you seen my release.nix?
<clever> ah, hmmm
<clever> which pulls in the entry-point for that machine, based on the hostname
<clever> lovesegfault: configuration.nix should have basically just 1 line in it, { imports = [ ./nix-config/whatever.nix ]; }
<clever> lovesegfault: then your update procedure is just `git pull && nixos-rebuild switch`
<clever> lovesegfault: i would put everything into a single directory, like /etc/nixos/nix-config, and then use an activation script to basically do `ln -sv $HOME/.config/nixpkgs ${./nixpkgs-config}`
<clever> lovesegfault: yep
<clever> always helps to have an extra set of eyes, that assume different things
<clever> najtra: but your pastebin says its in firmware-firmware/vendor
<clever> najtra: the -I is pointing to firmware-firmware/legacy/vendor
<clever> najtra: found the problem
<clever> __monty__: the 1st pastebin shows that its being passed directly to cc
<clever> najtra: what is the full path to flash.h? `find ~/repo/firmware-firmware -name flash.h` ?
<clever> najtra: brb
<clever> najtra: what are the exact args your using?
<clever> dminuoso_: then you have time to swap cables
<clever> dminuoso_: you can possibly tell grub to load the kernel, but not boot it right away
<clever> dminuoso_: i was also asking around in #coreboot a few days ago, and found https://github.com/avivgr/teensy_debugdev
<clever> dminuoso_: ehci debug also only works on the "first" port of a given controller
<clever> philipp[m]2: probably for purity, to catch such leaks
<clever> dminuoso_: whichever one is last, is used as the input when reading /dev/console
<clever> dminuoso_: if you supply multiple console= lines, it will print logs to each one
<clever> philipp[m]2: /usr/bin/env doesnt exist at build time, you have to run the patchShebangs function on the file
<clever> dminuoso_: try without quiet?
<clever> dminuoso_: and earlycon doesnt work?
<clever> dminuoso_: grub may stop printing when it boots something

2020-01-15

<clever> butterthebuddha: to add things to -I and -L, even if the build system overwrites CFLAGS
<clever> ack!
<clever> gnidorah: if your in the maintainer list, hydra.nixos.org should email you when they fail to build