2019-11-16

<clever> making each one use its own channels would have just added to the complexity
<clever> with a single rootfs
<clever> lovesegfault: at one time, i ran both armv6l and x86_64 on the same SD card
<clever> ah
<clever> srhb: the entire nixos-unstable channel is now hanging on 6 aarch64 builds
<clever> it came out of an xray machine, lol
<clever> the router has a dual-socket motherboard, with 8gig of ecc ram
<clever> one is in the nas, the other is in the router!
<clever> lovesegfault: and i technically have 2 hydras, lol
<clever> and now i can see, that unstable broke all of my graphical setups!
<clever> and unstable
<clever> which means hydra can now pre-build it against 19.09
<clever> lovesegfault: that allows things to still build, even if the secrets are missing
<clever> lovesegfault: look at load-secrets.nix in my repo
<clever> lovesegfault: because i had to deploy while at nixcon, lol
<clever> lovesegfault: there is an uncommited change, to swap my router over to a vpn ip
<clever> lovesegfault: and because i avoid using nixops features, i'm able to still build the config outside nixops
<clever> lovesegfault: instead of a configuration.nix file, i now just have lines 10-13 and 14-17, plus a 7-9 shared between both
<clever> lovesegfault: https://github.com/cleverca22/nixos-configs/blob/master/deployments/house.nix is how i manage the router+nas
<clever> but the desktop/laptop are still normal nixos
<clever> srhb: i have been switching over to nixops, the nas and router are now managed via that
<clever> srhb: its more that i need to edit and test things on one end, and then i forget to push
<clever> and i have to commit and pull/push to sync them up every now and then
<clever> so each machines copies of nixos-configs begin to diverge
<clever> and i do also have the issue that i dont push often enough
<clever> lovesegfault: thats solved by just putting it all into a single git repo, so you can share a file between them
<clever> any machine that should be doing that duty, just has ./media-server.nix added to its ${hostname}.nix file
<clever> so the hdmi out behaves like a dumb device, like a chromecast
<clever> take media-center.nix for example, that deals with running plex on bootup, and configuring auto-login
<clever> lovesegfault: these are all semi-optional things, that are grouped up for easy on/off
<clever> and then nas.nix (in git) deals with grouping stuff
<clever> so i fill that file in once at install time, with only 1 custom line
<clever> and configuration.nix only has stuff tied to that hardware
<clever> the nas just has imports = [ ./nixos-config/nas.nix ]; in its configuration.nix
<clever> thats the other major thing, i dont try to automate things like that
<clever> but i dont maintain the list as hard as i should if that was a real threat
<clever> in theory, that would prevent you from pulling off a mitm attack on my 1st session, so you cant ever mitm me on those hosts
<clever> core.nix also has ssh host keys for everything i ssh into commonly
<clever> but the vim config itself is in its own vim.nix module
<clever> lovesegfault: core.nix is stuff i want basically anywhere, like my vim config, and other stuff
<clever> lovesegfault: i have 2 rough classes, and some core files
<clever> lovesegfault: check my nixos-configs repo for an example
<clever> lovesegfault: yep
<clever> gwen: the issue with python stuff, is that you tend to have to wrap the final script, that is using the python library, that is using the qt libraries
<clever> your first point! :D
<clever> lovesegfault++
<clever> but, closed source things like teamspeak arent built by hydra
<clever> hydra should email everybody listed in the maintainer field
<clever> this reveals which PR its in
<clever> likely due to changes with less
<clever> and that fails now
<clever> its probably trying to pipe the EULA thru less
<clever> lovesegfault: aha!
<clever> + fakeLess = writeShellScriptBin "less" "cat";
<clever> teamspeak-client: fix stuck build
<clever> commit 7ddc19ba8cb57c5941827e0bd40d4418fb6d8d2c
<clever> The merge base 8a18c9f26148d1bcfcd9ba711d56e8bdb80899a1 is bad.
<clever> This means the bug has been fixed between 8a18c9f26148d1bcfcd9ba711d56e8bdb80899a1 and [44d9a86f41fc03fa37338f49778624d3122a757d].
<clever> maybe it was forked during a problem, and then unstable fixed it
<clever> its broken on both the top and bottom rev
<clever> hmmm
<clever> but limiting myself to revs in that list, i avoid rebuilds
<clever> my bisect has some perf issues, building all of QT
<clever> lovesegfault: also of use, this has the full history of the 19.09 channel
<clever> the executable is a shell script, that will ask you to agree to the EULA, then unpack the source
<clever> lovesegfault: that file
<clever> > teamspeak_client.meta.position
<clever> 34 version = "3.3.0";
<clever> seems pretty basic
<clever> 53 echo -e 'q\ny' | sh -xe $src
<clever> that did they do, that shouldnt happen
<clever> lovesegfault: oh, weird, teamspeak is only broken on 19.09
<clever> lovesegfault: and there goes the -small channel!
<clever> wait... now teamspeak works again....
<clever> but a bisect points to a commit entirely unrelated to teamspeak
<clever> whatever auto-answers it is now broken
<clever> the build just hangs with a EULA in the log output
<clever> a more anoying problem, is building the teamspeak client
<clever> lovesegfault: a recent failure of qtwebengine building has stopped me from building plex-media-player, but everything else works fine
<clever> lovesegfault: the scripts only care that the tested jobset is passing, which grabs a subset of important packages
<clever> so some things may slip thru
<clever> lovesegfault: -small also tests fewer things
<clever> lovesegfault: the -small channels dont want for hydra to build everything, so they update faster (but then you dont have good coverage on the cache)
<clever> lovesegfault: the main channels wait for hydra to finish building everything, plus testing a few select things (the tested job)
<clever> lovesegfault: https://hydra.nixos.org/eval/1554789 15 jobs still in the queue
<clever> lovesegfault: nixos-unstable wont update until all tests pass, and hydra has tried every single job
<clever> lovesegfault: 12 days ago
<clever> xd1le: they get built before all phases
<clever> currently running 4.19 on nixos-unstable
<clever> Nov 01 17:01:22 amd-nixos kernel: Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl and seccomp
<clever> Nov 01 17:01:22 amd-nixos kernel: Spectre V2 : Mitigation: Full AMD retpoline
<clever> phyfey[m]: i see evidence of some patches already being active on my current machine
<clever> Nov 01 17:01:22 amd-nixos kernel: Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization
<clever> most of the time, yes
<clever> slack1256: just slap a ./rescue_boot.nix into your imports list
<clever> slack1256: at the cost of ~300mb on your /boot partition, this will add a rescue env to the grub menu
<clever> slack1256: one min
<clever> slack1256: there is no rescue mode installed by default, you would need to boot from the original install iso
<clever> ebzzry: it uses nix-locate and nix-index
<clever> > lib.makeLibraryPath [ stdenv.cc.cc xlibs.libSM ]
<clever> ,locate libSM.so.6
<clever> ebzzry: a better option is to patchelf the binary in the derivation that compiled it
<clever> ebzzry: LD_LIBRARY_PATH = "${stdenv.cc.cc.lib}/lib"; would temporarily work around the issue
<clever> ebzzry: sounds like something went wrong with the RPATH of the binary, it should have gcc in there already
<clever> ebzzry: was it compiled by nix?
<clever> ebzzry: when you run what?
<clever> ebzzry: it should already be inside a nix-shell, what is failing?
<clever> ebzzry: what are you trying to do?
<clever> ebzzry: its in the search path for the linker by default, no need to do anything special
<clever> ebzzry: stdenv.cc.cc.lib
<clever> so you can escape the tracking metadata, and experiment with setting it again
<clever> systemd-run lets you spawn new things from pid1, and they arent tagged to your user
<clever> nope
<clever> testing things, requires a shell, that has not yet signed in!
<clever> and sudo cant clear that flag
<clever> when you login, pam needs a special flag to set who initially signed in
<clever> i think it was pam related issues
<clever> ah, ive messed with systemd-run before
<clever> slack1256: you can also tweak the nicelevel of an entire autogroup, the reddit thread explains that
<clever> ive not really played with cgroups yet
<clever> so `make -j300` gets an equal share to the browser
<clever> slack1256: and each autogroup gets its own fair share, and then nice levels only work within that autogroup
<clever> slack1256: basically, every time you call setsid (like, opening a new shell in ssh or a term emulator), it creates a new autogroup
<clever> slack1256: one minute
<clever> lordcirth__: for x in ${toString listOfSources}; do cp $x . ; done
<clever> nix will execute whatever the derivation depends on
<clever> lordcirth__: just use them in other derivations
<clever> gchristensen: the teamspeak client build also hangs now, bisection pointed to a weird unrelated thing

2019-11-15

<clever> lordcirth__: or mapAttrs
<clever> lordcirth__: probably the map function
<clever> dest= i think it was, it should be in the nixops manual
<clever> dminuoso: configure nixops to copy them to a dir that isnt a tmpfs
<clever> boolman: you want just `rm result`, no `-rf` needed, no `/` needed
<clever> magneticduck: remember to mount /mnt/boot too
<clever> magneticduck: just mount things to the right places and re-run nixos-install
<clever> magneticduck: yeah
<clever> you may need to disable the CSM
<clever> magneticduck: just tell the firmware to boot the usb stick in efi mode
<clever> manveru: and line 39 is pulling in the source-pins of iohk-nix, so you also get all of those
<clever> manveru: with this, you can just nix-build release.nix -A source-pins, to pre-download everything it hould need at IFD time
<clever> so you can recursively go down the pins
<clever> and to make a link to your deps's source pin derivations
<clever> manveru: then you can follow those to see the code behind the pin
<clever> manveru: my prefered way, is to generate symlinks, like ln -sv ${fetchurl ...} reponame
<clever> manveru: as for making hydra cache things properly, you just need to generate a derivation that depends on the same fetchurl calls, and expose it in release.nix
<clever> pkgs.fetchzip forces you to supply a sha256, and makes it simpler to cache
<clever> another thing, switching from fetchTarball to pkgs.fetchzip would help
<clever> yeah
<clever> no need to check for changes, if you know the content is unchanging
<clever> manveru: if a sha256 is present, i think it just ignores the ttl entirely, and uses the $out if it exists in /nix/store/
<clever> manveru: the ttl mainly controls re-checking http for changes, when you already have the file
<clever> manveru: far fewer steps
<clever> manveru: but if you use a binary cache, it will stream it thru a decompressor, and deserialized into /nix/store
<clever> manveru: then more cpu churn, as its serialized, hashed, and sent to nix-daemon, and deserialized back into /nix/store/
<clever> manveru: then you have a cpu delay to uncompress and untar it to a tmp folder, as your current user
<clever> manveru: when using things like fetchTarball against github, you first have a slow delay at github, generating the tar and streaming it over
<clever> manveru: if you provide a sha256 ahead of time, it can also search the binary cache for things, and fetch it in a faster manner
<clever> manveru: the file and unpacked links, point into the nix store, but arent GC roots, so nix can GC things, and they just act as a shortcut to save time
<clever> manveru: the .info files contain metadata, like when it was last checked (to control how often it rechecsk), and the etags (to help with cache control headers)
<clever> manveru: when nix is fetching things at eval time, it manages some symlinks in ~/.cache/nix/tarballs/
<clever> and thats a major bottleneck
<clever> the main problem i ran into, is that builtins.fetchTarball, without a sha256, cant use the binary cache
<clever> and then it can get things from there after a gc
<clever> substituters = http://nas.localnet:8081/ file:///tmp/cache
<clever> manveru: then using `nix copy --to file:///tmp/cache /nix/store/foo` to copy the things it built to a dummy cache
<clever> manveru: main thing i was doing, is building with -Q, and noting everything it build during eval
<clever> partially my fault, i added it to the source, and not the docs
<clever> manveru: its not in the docs, i keep checking for it in the source
<clever> ,profiling manveru
<clever> callHackageDirect will just fetchurl the given version, but needs a sha256
<clever> so its limited to what was in the hackage snapshot, when the tarball was made
<clever> callHackage is a helper around callCabal2nix, using a tarball of cabalfiles and sha256's
<clever> and callHackageDirect
<clever> that will call cabal2nix for you, look for name.cabal in ./path, then callPackage the generated file with {}
<clever> kenran: you can also use self.callCabal2nix "name" ./path {}
<clever> kenran: yes
<clever> kenran: haskellPackages = super.haskellPackages.override { ghc = something; };
<clever> and ive been using nix for years
<clever> i still havent read it
<clever> that makes even less sense to me, lol
<clever> dminuoso: there is also a `nix-copy-closure` command you could check the man page of
<clever> dminuoso: not sure, i havent really seen it used much before nix
<clever> dminuoso: the nixos build for the target machine, and everything it depends on
<clever> niso: the status plugin is optional, its just to report pass/fail back to the vcs system
<clever> so if you just duplicate that plugin, and switch it to the branches api, your done
<clever> niso: in this case, it fetches all pulls, and generates json
<clever> niso: the first plugin to return non-undefined will terminate the search, and whatever it returned is used
<clever> niso: hydra will call fetchInput for every plugin, and pass it the value and type
<clever> niso: this adds an input to the jobset, of type githubpulls, which will fetch a list of all PR's, and pass it to toxvpn/default.nix (line 6) as a json file
<clever> niso: yeah, one second
<clever> niso: that bash still runs in the nix sandbox, and has to follow all the normal rules when building things
<clever> niso: there is no real way to get the list of branches from this job
<clever> niso: i would generate the json using nix, things like map and listToAttrs
<clever> niso: currently, you need to hard-code the list of branches into a nix file, and use declarative jobsets to generate the jobsets
<clever> kenran: the only real difference, is that you can have several of them, and you use the 2nd argument to refer to other packages+overrides, rather then rec{
<clever> kenran: overlays are list a list of functions, that take both the previous package set, and the final package set, and return the top-level attrs to overwrite
<clever> kenran: packageOverrides just takes a previous version of the package set, and returns the top-level attrs to overwrite
<clever> `man nix-hash`
<clever> oops
<clever> lordcirth__: nix will accept both
<clever> lordcirth__: base32 vs base64
<clever> which you can now nix copy over
<clever> exarkun: i can confirm the above nix-build generates the same $out
<clever> exarkun: and confirm it has the same output as line 58 of the `show-derivation` gist
<clever> exarkun: try this on an x86 machine
<clever> nix-build -E 'with import <nixpkgs> {}; fetchzip { url = "https://github.com/raspberrypi/linux/archive/raspberrypi-kernel_1.20180619-1.tar.gz"; sha256 = "0yccz8j3vrzv6h23b7yn7dx84kkzq3dmicm3shhz18nkpyyq71ch"; name = "source"; }'
<clever> exarkun: the name must also be set to source, or it wont line up
<clever> so you must unpack the file, and then add the directory
<clever> exarkun: this was made by pkgs.fetchzip (which auto-unpacks), not pkgs.fetchurl
<clever> exarkun: you must unpack the tar.gz, before you add it to the nix store
<clever> "outputHashMode": "recursive",
<clever> exarkun: to start, with, run nix show-derivation on that source.drv, and pastebin the contents
<clever> > hello.src
<clever> exarkun: run `nix show-derivation` on it
<clever> exarkun: /nix/store/45r705j2ig7227nmbx8l50wpiww87992-source.drv
<clever> exarkun: thats a fixed-output derivation
<clever> exarkun: where do you see the file being added to the nix store?
<clever> exarkun: ?
<clever> lordcirth__: sounds likely
<clever> lordcirth__: ive not used nuget, so i'm not sure what you can do to solve that
<clever> exarkun: those should behave the same way
<clever> zeta_0: i dont use emacs
<clever> exarkun: did you give file:// an absolute path?
<clever> exarkun: try it as root
<clever> exarkun: are you doing the --from as root?
<clever> exarkun: nope
<clever> what is the error and can you pastebin the code?
<clever> lordcirth__: you probably need a recursive set, or a let block
<clever> lordcirth__: Baughn is also on irc
<clever> lordcirth__: also, check the callPackage for eventstore, in all-packages.nix
<clever> lordcirth__: that only works if the program tries to run Nuget directly
<clever> ./build
<clever> cp ${foo} something/bar/nuget.exe
<clever> let foo = fetchurl {...}; in stdenv.mkDerivation { ... buildPhase = ''
<clever> lordcirth__: use fetchurl to grab a copy using nix, and then in buildPhase copy it to the right place before you ./build
<clever> lordcirth__: if it cant find nuget.exe, it will try to download it, but nix wont allow that
<clever> if [ ! -f "$nugetExe" ]; then
<clever> buildDir="$rootDir/_build"
<clever> nugetExe="$toolsDir/NuGet/$nugetVersion/nuget.exe"
<clever> toolsDir="$buildDir/tools"
<clever> rootDir=$(dirname $0)
<clever> lordcirth__: yep
<clever> lordcirth__: try buildPhase = "./build"; ?
<clever> what are the contents of build?
<clever> what does the new version use in its place?
<clever> did the old version use it?
<clever> lordcirth__: yeah
<clever> lordcirth__: oh, and `make` in the middle
<clever> lordcirth__: phases hasnt been set, so it will do the usual configure/build/install stuff, which includes running ./configure and then running `make install`
<clever> lordcirth__: that file is the one that builds it
<clever> that also works
<clever> inkbottle: and its easy enough to enable docker in nixos
<clever> inkbottle: a docker is likely the simplest way to get a debian like env
<clever> inkbottle: all i can think of then is to try using yarn2nix instead of plain npm
<clever> inkbottle: does ` pkg-config --cflags --libs libusb-1.0` work in the nix-shell?
<clever> inkbottle: maybe npm isnt obeying pkgconfig?
<clever> inkbottle: its in libusb1.dev
<clever> $ ls result-dev/lib/pkgconfig/
<clever> libusb-1.0.pc
<clever> inkbottle: pkgconfig must also be in the nix-shell args, it wont work when globally installed
<clever> inkbottle: so you must use pkgconfig to find the path
<clever> inkbottle: but libusb deviced to break things, and hid the files under include/libusb-1.0
<clever> inkbottle: nix added the include dir to the -I path
<clever> $ ls result-dev/include/libusb-1.0/libusb.h
<clever> result-dev/include/libusb-1.0/libusb.h
<clever> $ nix-build '<nixpkgs>' -A libusb1.dev
<clever> ,libraries inkbottle
<clever> locate will just blindly search /nix/store and can find things nix-shell isnt making available
<clever> inkbottle: this is how i confirm what is inside a package
<clever> $ ls result/lib/ -lh
<clever> $ nix-build '<nixpkgs>' -A libusb1
<clever> inkbottle: libusb1 has libusb-1.0.so
<clever> inkbottle: libusb only has libusb-0.1.so
<clever> CMCDragonkai: i think so
<clever> inkbottle: it will only be found if you tell nix-shell to provide it
<clever> CMCDragonkai: these 2 i think
<clever> src/config/config_crypto.c:#if defined ( CRYPTO_PUBKEY_RSA ) && defined ( CRYPTO_DIGEST_SHA256 )
<clever> enabledFestures
<clever> with the rest of the cfg flags
<clever> sha256 is likely what you want enabled
<clever> CMCDragonkai: youll need to use the x509 command to see which algo is being used, then enable the right compile time flag
<clever> One example is SHA1 certificates which is not supported (they are considered broken anyway so upgrade your certificate)
<clever> youll need to check with the ipxe devs maybe
<clever> not sure then
<clever> CMCDragonkai: the pastebin also looks good