2019-12-16

<clever> and once you get self and super the right way around, time_1_9_2 doesnt exist on my nixpkgs
<clever> nix-shell -p '(haskellPackages.override { overrides = self: super: { time_1_9_2 = haskell.lib.dontCheck super.time_1_9_2; }; }).ghcWithPackages (ps: with ps; [ time_1_9_2 typed-process ])'
<clever> i think that would do the same thing
<clever> nix-shell -p '(haskellPackages.override { overrides = super: self: { time_1_9_2 = haskell.lib.dontCheck super.time_1_9_2; }; }).ghcWithPackages (ps: with ps; [ time_1_9_2 typed-process ])'
<clever> bahamas: such as...
<clever> bahamas: basically, just shove that entire string into -p
<clever> bahamas: you must wrap line 15 in ( and )
<clever> bahamas: nix doesnt use , to seperate list items, so [ function value ] is a list with 2 elements, not 1, which can lead to confusing errors
<clever> MichaelRaskin: currently, i'm deaing with FPU exceptions on bootup
<clever> MichaelRaskin: maybe i just didnt read the docs back then, lol
<clever> pie_: the scripts are more just a way to load the modules, and configure other stuff to mount the real root
<clever> bahamas: you must create another derivation, and add ghcWithPackages to the buildInputs of it
<clever> bahamas: thats giving you a shell suitable for building ghcWithPackages, not using the result of a built ghcWithPackages
<clever> pie_: so you can do things like add zfs to the initrd, to deal with root on zfs, without rebuilding a new kernel
<clever> pie_: the main job of the initrd, is to let you customize which modules get loaded at boot, without having to compile the kernel again
<clever> pie_: back when i was playing around with initrd's in gentoo
<clever> pie_: i know this, because i manually reverse engineered, and then implemented firmware handling, in bash, to get wifi in the initrd, lol
<clever> pie_: lfs?
<clever> pie_: udev is then responsible for finding the file, and given the kernel a copy
<clever> pie_: when a driver wants to load firmware, it tosses the filename down to udev
<clever> nixos/modules/services/hardware/udev.nix: echo -n "${config.hardware.firmware}/lib/firmware" > /sys/module/firmware_class/parameters/path
<clever> pie_: one of the modprobe or udev config files says to look there
<clever> lrwxrwxrwx 1 root root 111 Dec 31 1969 /run/current-system/firmware/iwlwifi-1000-5.ucode -> /nix/store/pxdhpx9mllm8rsmi2y21qbzpyaawiw33-firmware-linux-nonfree-2019-10-22/lib/firmware/iwlwifi-1000-5.ucode
<clever> lrwxrwxrwx 1 root root 111 Dec 31 1969 /run/current-system/firmware/iwlwifi-1000-3.ucode -> /nix/store/pxdhpx9mllm8rsmi2y21qbzpyaawiw33-firmware-linux-nonfree-2019-10-22/lib/firmware/iwlwifi-1000-3.ucode
<clever> [clever@amd-nixos:~/apps/nixpkgs-hasura]$ ls /run/current-system/firmware/iwlwifi-* -l
<clever> pie_: ls /run/current-system/firmware
<clever> DigitalKiwi: in my case, its having a panic, on bootup, because pid 1 failed with SIGILL
<clever> DigitalKiwi: are you sure you want mine? lol, it cant even printf!
<clever> 1) SIGHUP 2) SIGINT 3) SIGQUIT 4) SIGILL 5) SIGTRAP
<clever> [ 3.416539] pc : [<00010bec>] lr : [<00012458>] psr: 80000010
<clever> [ 3.396318] potentially unexpected fatal signal 4.
<clever> 10bec: ed2d8b04 vpush {d8-d9}
<clever> 10be8: e92d4ff0 push {r4, r5, r6, r7, r8, r9, sl, fp, lr}
<clever> 00010be8 <printf_core>:
<clever> pie_: and thats why i never reboot! lol
<clever> lovesegfault: it cant
<clever> ,callPackage lovesegfault
<clever> jwaksbaum[m]: what was the issue?
<clever> it was named before fetchGit existed
<clever> CMCDragonkai: i think its for all 3 of the impure fetch functions, fetchurl, fetchTarball, and fetchGit
<clever> [nix-shell:~/apps/kernel/build]$ ls ~/.cache/nix/tarballs/
<clever> [nix-shell:~/apps/kernel/build]$ ls ~/.cache/nix/gitv2/
<clever> thats a bug, ignore it
<clever> CMCDragonkai: the above flag, changes it to 1 minute
<clever> CMCDragonkai: by default, nix will cache the result of the impure fetches for 1 hour, and not check again until that hour has expired
<clever> CMCDragonkai: what if you `--option tarball-ttl 60` ?
<clever> CMCDragonkai: but if your in a ci context, shouldnt you have an env var that says which rev to use?
<clever> CMCDragonkai: there is a cache to speed things up
<clever> lovesegfault: i try to avoid python :P
<clever> CMCDragonkai: not really, i think builtins.fetchGit replaced it anyways
<clever> pie__: its been like this for years
<clever> and if i restart pulse, chrome looses audio capture entirely
<clever> and pulse lost the ability to see the headset
<clever> i tried to replug the headset to fix a weird latency thing it tends to get after a lot of uptime
<clever> i need to restart chrome tonight...
<clever> pie__: thatll do it!
<clever> jwaksbaum[m]: can you pastebin the entire configuration.nix file?
<clever> jwaksbaum[m]: try `nix-build tiny.nix -A kernel` ?
<clever> jwaksbaum[m]: just the kernel?
<clever> jwaksbaum[m]: oh, which kernel package do you actually want to build?
<clever> the attr in question works on unstable
<clever> «derivation /nix/store/g190nly76iwc6fzvf1xaww2adqmk7bl8-amdgpu-pro-17.40-4.19.87.drv»
<clever> nix-repl> linuxPackages.amdgpu-pro
<clever> nix-repl> linuxPackages.kernel.dev
<clever> «derivation /nix/store/x17x6xj4bfdqj06p39bxlclyk8j2xxx6-linux-4.19.87.drv»
<clever> jwaksbaum[m]: *looks*
<clever> CMCDragonkai: fetchgitPrivate never documented how to use the ssh-auth-sock, and this is the only way i found that works
<clever> CMCDragonkai: socat acting as a proxy, will blame everything on root, and ssh-agent has an exception to let root in, so `sudo ssh foo` can use your agent, even though root isnt you
<clever> CMCDragonkai: if ssh-agent detects you connecting from the "wrong" user, it thinks its malicious, and refuses to do anything
<clever> CMCDragonkai: every time something connects to it, socat will open a socket to $SSH_AUTH_SOCK, and forward the traffic over
<clever> CMCDragonkai: /tmp/hax is a unix socket, that socat will listen on
<clever> and depending on what the key can do, they could still wreak havoc :P
<clever> CMCDragonkai: nix-build -E 'with import <nixpkgs> {}; runCommand "name" {} "ssh -O IdentityAgent=/tmp/hax ssh user@host rm -rf /"'
<clever> any attacker could still connect to the agent, and make use of it, until you boot them out, but they cant copy the secret itself
<clever> yeah, fetchgitPrivate supports an ssh agent, so you can keep the key itself secure within the agent
<clever> CMCDragonkai: boom, your now screwed
<clever> CMCDragonkai: nix-build -E 'with import <nixpkgs> {}; runCommand "name" {} "cat /run/keys/ohgodno"'
<clever> CMCDragonkai: and why cant you use builtins.fetchGit?
<clever> CMCDragonkai: sandboxPaths exposes it to EVERY SINGLE BUILD
<clever> CMCDragonkai: nixops will copy keys there, but no build within the machine can read it
<clever> CMCDragonkai: any method not using an agent, involves the keys being world-readable in /nix/store/
<clever> CMCDragonkai: youll either need to continue using builtins.fetchgit, or re-add fetchgitPrivate and run socat like in my gist
<clever> CMCDragonkai: root being a requirement actually makes some of it simpler
<clever> CMCDragonkai: you want pkgs.fetchgitPrivate, which needs some funky stuff with your ssh agent, and then any derivation downloading something can make use of your keys
<clever> jared-w: checking nixpkgs...
<clever> jwaksbaum[m]: its trying to read kernel.dev, to get the headers
<clever> 14 kernelDir = if libsOnly then null else kernel.dev;
<clever> jwaksbaum[m]: lets start by readiing line 14 of amdgpu-pro/default.nix
<clever> jwaksbaum[m]: sure, let me read the paste above...
<clever> jared-w: i got bored one day, and read the docs for every single acme package :P
<clever> jared-w: i think this one is of more use
<clever> > haskellPackages.acme-realworld.meta.description
<clever> i suspect acme-everything has a special exception to ignore it
<clever> even some acme things are in there
<clever> yes
<clever> Orbstheorem: i believe so
<clever> jared-w: or for virtualbox, it could just be a slight variant of the installer iso
<clever> jared-w: my basic plan with the 1189 issue, is that you would have a pre-built kexec tar, and nixops will just upload it to a target running any version of linux
<clever> jared-w: and thats where https://github.com/NixOS/nixops/issues/1189 comes in!
<clever> gchristensen: and github wont let us sweep that under the rug! lol
<clever> angerman: i believe the nixops virtualbox backend starts with a vbox app file, that has a base nixos pre-installed
<clever> angerman: and it can then use that remote machine, to build itself
<clever> angerman: but if your deploying with nixops, it will first create a linux machine, either in the cloud or virtualbox (depending on the backend of choice)
<clever> gchristensen: should i maybe edit the title of that now?
<clever> jared-w: https://github.com/NixOS/nixops/issues/984 is about fixing nixops to use the remote machine automatically
<clever> but it also has the bootstrap issue, of needing a linux build machine to make the inner nixos
<clever> jared-w: this will generate a bash script (darwin bash) that runs the darwin qemu, and then boots linux nixos within it
<clever> jared-w: if your nix install is setup in a single-user mode, nixops can automatically configure the remote machine to be the build machine
<clever> jared-w: nixops needs a linux build machine configured in /etc/nix/machines
<clever> jared-w: and thats basically what nixops does, nix-build + nix-copy-closure
<clever> jared-w: why does nixops not work on macos?
<clever> > builtins.getEnv
<clever> keithy[m]: have you tried turning on allowUnfree in configuration.nix?
<clever> keithy[m]: all list based options are additive, so nixos will add the lists together, rather then removing things

2019-12-15

<clever> mehlon: yeah, $out may work
<clever> lovesegfault: not yet, still stuck at vpush
<clever> lovesegfault: with which project?
<clever> mehlon: or wrapProgram + --run "cd foo"
<clever> mehlon: libredirect?
<clever> and it waited until you spoke up in the channel!
<clever> pie__: if something has changed, yes
<clever> pie__: not sure then
<clever> pie__: ps -eH x, look at the process tree, what is running?
<clever> pie__: try without RemainAfterExit, and you may need to systemctl stop it
<clever> pie__: also, because of RemainAfterExit and oneshot, it may not re-run after a rebuild
<clever> i dont know what SyslogIdentifier will do to it
<clever> pie__: normally, journalctl -f -u workaround.service
<clever> `_: if you use pkgs.fetchsvn, give it a name and sha256, it will cache the result and reuse it for the same name+sha256
<clever> pie__: yep
<clever> `_: thats probably part of it, git is everywhere now
<clever> lol
<clever> pie__: if you set .script, it will auto-generate a shell script for you, and put it into serviceConfig.ExecStart
<clever> `_: builtins.fetchGit is is mostly for private git repos
<clever> so they both slow down the build, and harm purity
<clever> `_: the builtin.fetch* functions are eval time fetches, only one can run at once, and sha256 is optional
<clever> `_: all of the pkgs.fetch* functions are build-time fetches, which need a sha256, and can be ran in parallel
<clever> `_: which forces you to fetch the entire source before you know the output path of anything
<clever> `_: because it happens at eval time, and nix will just hash the directory before it does the eval
<clever> `_: and if you fail to meet that promise, your build is considered as failing
<clever> `_: the sha256= is a promise that your build will always have the same result, and that promise is what gives you permission to access the network
<clever> `_: pkgs.fetchsvn will take a sha256, and only if the name= or sha256= changes, would it rebuild
<clever> `_: fetchGit behaves the same as src = ./foo;
<clever> `_: causing it to perform a new build
<clever> `_: builtins.fetchGit happens before the eval is finished, and if you change the source, the output path will be properly updated
<clever> `_: the filesystem access is also restricted
<clever> pie__: ive seen activation scripts entirely block a machine from booting
<clever> pie__: systemd oneshot services!
<clever> and with 1 line in your config, you can do the same!
<clever> 509 xkbOptions = "caps:shiftlock";
<clever> 508 xserver = {
<clever> so capslock does !@#$%^&*()_+ to the top row!
<clever> so now, the capslock key, affects numbers!
<clever> EdLin: i turned on shiftlock, instead of capslock
<clever> the idea was to have predictable latency, and high bandwidth, with linux support
<clever> EdLin: at one point, i was looking into a custom audio capture system, ethernet based at the time, that would allow for 32 channel audio capture, 24bit, 96khz
<clever> `_: my current issue, is that busybox crashes with SIGILL when ran as init in the initrd
<clever> pie__: its probably not under -u, since it doesnt have a unit name
<clever> `_: heh, funnily enough, i'm currently working on open source firmware, for the rpi!
<clever> pie__: the journal maybe
<clever> but steam and proton have mostly solved that
<clever> but i still had to use windows for games
<clever> around the age of xp, i began switching to linux
<clever> ive ran dos, win3.11, 95, 98, xp, 7, and 10 i think
<clever> EdLin: a malformed ping packet, would cause a bluescreen
<clever> EdLin: ive heard about the ping of death in win95
<clever> :D
<clever> ive been trying to solve this problem lately, i'm pretty sure i have the right arch for this build...
<clever> 1) SIGHUP 2) SIGINT 3) SIGQUIT 4) SIGILL 5) SIGTRAP
<clever> [ 6.699657] Kernel panic - not syncing: Attempted to kill init! exitcode=0x00000004
<clever> [ 6.544240] potentially unexpected fatal signal 4.
<clever> because i fix it when it breaks, and then it doesnt get into a stable channel!
<clever> then the stuff i care about remains in a working state
<clever> part of why i always run nixos-untable
<clever> and then it winds up in the next stable release, in a broken state
<clever> if nobody uses it in nixos-unstable, its unlikely to be fixed
<clever> other channels may not have the problem
<clever> youll need to set allowBroken = true, find the real problem, and then file a PR to nixpkgs to fix it (and mark it as not broken)
<clever> thats because they fail to compile, and marking it as broken stops you from wasting time building something that will fail
<clever> how do they fail?
<clever> `_: they seem to work just fine for me, how are you testing them?
<clever> `_: then it may need to be patched by the nix build process
<clever> `_: does the script detect if things are already checked out at a certain dir?
<clever> `_: yes, the network is disabled to make sure the builds are reproducable
<clever> `_: also, you want to use `nix-build file.nix -A package` for testing, nix-env isnt suited well for testing
<clever> `_: nix disables all network access during a build, you must use pkgs.fetchsvn to fetch from svn
<clever> i lack hydra control
<clever> ivan: i think you need a special flag to make systemd aware that nfs is using the network, and to umount those before shutting off the network
<clever> ivan: why is it hanging? you should probably fix that issue first
<clever> ivan: how do you shutdown the machine?
<clever> ivan: are you shutting down properly?
<clever> ivan: is /boot mounted correctly?
<clever> maybe?
<clever> ivan: system.copySystemConfiguration
<clever> lovesegfault: might be in libstore of nix itself
<clever> lovesegfault: and the thing its compressing isnt on disk, its a single char* in ram
<clever> lovesegfault: youll have to patch hydra-queue-runner
<clever> lovesegfault: xz i think
<clever> lovesegfault: its copying the build product from the build machine, to hydra, compressing, then uploading to an S3 bucket
<clever> wavirc22: you dont have to set the uid or gid of your thing, nixos will just auto-generate one on startup
<clever> before step 7, do step 6
<clever> 2019-12-15 01:50:51 < lovesegfault> 5. git add ./configuration.nix
<clever> 2019-12-15 01:51:10 < lovesegfault> 6. git commit -m "Hi adding my config woohoo! Git rocks!"
<clever> EdLin: did you commit yet?
<clever> EdLin: what did it say when you ran that command?
<clever> 2019-12-15 01:51:29 < lovesegfault> 7. git push -u origin master
<clever> and i feel that bloats things too much
<clever> some people just make everything on by default :P
<clever> its just a matter of enabling it
<clever> bash can do the same things
<clever> programs.ssh.startAgent
<clever> Whether to start the OpenSSH agent when you log in. The OpenSSH agent remembers private keys for you so that you don't have to type in passphrases every time you make an SSH connection. Use ssh-add to add a key to the agent.
<clever> and share it between every terminal
<clever> having an ssh agent configured, will automate the `eval "$(ssh-agent)"` step
<clever> and then test with `ssh git@github.com`
<clever> EdLin: does `ssh-add -l` show a key?
<clever> EdLin: that broke your connection to the agent, re-start the ssh-agent, and redo ssh-add
<clever> EdLin: then your ssh key isnt working yet
<clever> EdLin: what does `ssh git@github.com` report?
<clever> not https://
<clever> EdLin: you have to clone from git@github.com:owner/repo
<clever> it computes the public from the private
<clever> you only add the private key to the agent
<clever> then thats not precisely what you typed in!
<clever> EdLin: the error only happens if you use `-l`
<clever> EdLin: short term, try doing: eval "$(ssh-agent)" and then ssh-add
<clever> it just makes it simpler if you have a pw
<clever> an agent isnt needed
<clever> EdLin: what does `ssh git@github.com` report?
<clever> then an agent isnt running, and ssh will load ~/.ssh/id_rsa by default
<clever> EdLin: use `ssh-keygen` to create a keypair, with optional passphrase, then configure the public key in github
<clever> EdLin: did you tell github about your ssh keypair?
<clever> EdLin: did you create a repo in github yet?
<clever> [ 2.213178] ---[ end Kernel panic - not syncing: Attempted to kill init! exitcode=0x00002a00
<clever> samueldr: it works as expected on x86
<clever> samueldr: throwing together a script to build the x86 busybox, and see what it does in this situation!
<clever> samueldr: any guess as to what to do next? maybe test in qemu?
<clever> dansho: sounds perfect for your issue
<clever> [ 6.555003] Kernel panic - not syncing: Attempted to kill init! exitcode=0x00000004
<clever> so i have to bake it into the kernel
<clever> my bootloader lacks initrd support right now
<clever> [nix-shell:~/apps/kernel/build]$ time make $makeFlags zImage -j8 && time nix-build ~/apps/rpi-open-firmware/ -A helper
<clever> CONFIG_INITRAMFS_SOURCE="initrd.cpio"
<clever> samueldr: i have its exit code, so i could try `exit 42` lol
<clever> samueldr: i think this line, is why stdout/stderr dont work
<clever> samueldr: [ 5.479812] Warning: unable to open an initial console.
<clever> dansho: not sure
<clever> samueldr: ive got full uart output, let me pastebin
<clever> samueldr: it also failed with error -13, when i forgot to make it executable, so its definitely looking at it
<clever> samueldr: thats my /init
<clever> 59 text = ''
<clever> 60 #!${self.busybox}/bin/sh
<clever> 57 script = pkgs.writeTextFile {
<clever> samueldr: ive been using impure incremental builds to speed up testing
<clever> samueldr: i have the whole working dir from the kernel build and the .config
<clever> [ 6.554946] Kernel panic - not syncing: Attempted to kill init! exitcode=0x00000004
<clever> samueldr: if i put a #! script into the initrd, and use CONFIG_INITRAMFS_SOURCE to tack it on, i get:
<clever> samueldr: relatively, ive turned off random bits while debugging
<clever> an initrd with a busybox at /init also fails
<clever> samueldr: you able to help with some early arm boot issues?
<clever> [ 5.493390] Please append a correct "root=" boot option; here are the available partitions:
<clever> [ 5.485495] VFS: Cannot open root device "mmcblk0p2" or unknown-block(0,0): error -6
<clever> [ 5.479812] Warning: unable to open an initial console.
<clever> worldofpeace: the sd image being over-sized should be fixed now
<clever> my real problem right now, is that i dont have any serial output
<clever> [ 6.574990] Kernel panic - not syncing: Attempted to kill init! exitcode=0x00000004
<clever> so the rest of the kernel logs show timestamps in seconds
<clever> DigitalKiwi: it mainly just controls the scaling factor, between the arm timers and "real time"
<clever> its supposed to be set by the board firmware, before any os level stuff runs
<clever> DigitalKiwi: the arm assembly before linux, wasnt setting it, so it defaulted to 0
<clever> DigitalKiwi: the main timer freq in hz
<clever> DigitalKiwi: ive just solved that one today, lol
<clever> DigitalKiwi: [ 0.000000] Division by zero in kernel.
<clever> dansho: its passing special flags to clang, like -I flags
<clever> so /init gave no output
<clever> lovesegfault: linux failed to open /dev/console
<clever> [ 6.550101] Run /init as init process
<clever> [ 6.555014] Kernel panic - not syncing: Attempted to kill init! exitcode=0x00000004
<clever> [ 6.507919] Warning: unable to open an initial console.
<clever> dansho: read the clang shell script inside clang-wrapper
<clever> dansho: clangd may need the same wrapper as clang, check `which clang`
<clever> dansho: mkShell calls plain stdenv.mkDerivation
<clever> then it gives you clang automatically, without any clang in buildInputs
<clever> dansho: replace stdenv.mkDerivation with clangStdenv.mkDerivation
<clever> dansho: you must use clangStdenv if you want clang working
<clever> lovesegfault: you try getting 3 cross-compilers setup without nix :P
<clever> dansho: is clangd being ran under nix-shell with clangStdenv?
<clever> lovesegfault: i have a single nix-build command, that can build arm assembly, vc4 assembly, a linux kernel, and then assemble all of them into a single directory
<clever> nix, to the rescue!
<clever> so i cant reproduce their claims of it working :P
<clever> but the actual build of linux wasnt documented
<clever> lovesegfault: the dram bringup and loading of linux was previously solved
<clever> lovesegfault: my current problem, is that the mmc reader is failing, so it cant mount the rootfs
<clever> .... i'm not
<clever> lovesegfault: it "just works" if you use the closed-source blobs
<clever> this is the furthest ive gotten so far
<clever> [ 5.604569] Kernel panic - not syncing: VFS: Unable to mount root fs on unknown-block(0,0)
<clever> lovesegfault: ive also made major rpi3 progress here