2017-10-23
11:44
<
clever >
sphalerite: also, when nixos-rebuild tries to fetch the nix that obeys your configuration.nix nixpkgs.config, it wont include qemu-vm.nix, and then it throws errors that those options are invalid
11:41
<
clever >
seequ: that module is only in scope if you use build-vm, so all the docs will omit them
11:39
<
clever >
seequ: yes
11:36
<
clever >
its now 6 hours into a 3rd attempt
11:35
<
clever >
sphalerite: and after the 2nd failure, something else snuck into the queue long enough to keep the logs alove, ENOSPC
11:35
<
clever >
sphalerite: when hydra restarted the job, it destroyed all logs, so i had no way of finding the cause of the failure
11:35
<
clever >
sphalerite: ive blown 9h21mins building gcc, twice yesterday
11:06
<
clever >
EFI would likely work differently though
11:05
<
clever >
tanonym: as long as the nixos grub can read the ubuntu / to read out kernels
11:05
<
clever >
tanonym: that could also work, just dont give ubuntu a /boot partition, and leave it using the /boot directory of /
11:03
<
clever >
tanonym: either seperate /boot's or disable the script that manages grub config
11:02
<
clever >
tanonym: you will want to make sure they dont overwrite the nixos grub config when they do kernel updates
11:02
<
clever >
sphalerite: one of those, i dont use darwin much
11:01
<
clever >
etu: yeah, i can see efi being better, because you dont have to configure one as a master, and chainload the others
10:59
<
clever >
ive also heard that the apple uefi supports HPFS+ for the efi system part
10:58
<
clever >
so i had to entirely disable EFI booting to get legacy grub to even start
10:58
<
clever >
etu: its almost like the boot priority is within the CSM module, and CSM only runs if EFI fails
10:58
<
clever >
etu: if EFI is on, it always boots first, and it ignored the boot priority configured in the bios
10:57
<
clever >
etu: one other issue i ran into with my main desktop
10:57
<
clever >
but oddly, ive helped a number of users in here sucessfully install with efi
10:57
<
clever >
legacy boot partition on gpt, on every machine
10:56
<
clever >
etu: personally, ive never once gotten uefi to work
10:55
<
clever >
etu: so if you wipe the disk first, you can never remove that bootloader from the efi vars
10:55
<
clever >
etu: ive also heard of some, that dont let you delete a given bootloader if the UUID it refers to is missing from the disk
10:54
<
clever >
etu: i think it also depends on the UEFI implementation, some dont give you a choice between the registered bootloaders
10:35
<
clever >
seequ: lol
10:33
<
clever >
seequ: line 10-38 shows how it gets generated
10:32
<
clever >
seequ: its directly in the store, and only the systemd service file has the path
10:10
<
clever >
sphalerite: you can also trigger this with <break>o over the serial port
10:10
<
clever >
o will just cut power, without even doing a sync
10:10
<
clever >
sphalerite: any of these special letters can be printed there, and will take immediate effect
10:10
<
clever >
[1062186.379382] sysrq: SysRq : HELP : loglevel(0-9) reboot(b) crash(c) terminate-all-tasks(e) memory-full-oom-kill(f) kill-all-tasks(i) thaw-filesystems(j) sak(k) show-backtrace-all-active-cpus(l) show-memory-usage(m) nice-all-RT-tasks(n) poweroff(o) show-registers(p) show-all-timers(q) unraw(r) sync(s) show-task-states(t) unmount(u) force-fb(V) show-blocked-tasks(w) dump-ftrace-buffer(z)
10:10
<
clever >
[root@amd-nixos:~]# dmesg | tail
10:09
<
clever >
[root@amd-nixos:~]# echo h > /proc/sysrq-trigger
10:09
<
clever >
sphalerite: oh, now i remember how to power off and reboot from the shell
10:07
<
clever >
tanonym: /exit
10:06
<
clever >
yeah, that sounds more likely to do something
10:06
<
clever >
sphalerite: i dont see sync in the kernel flags
10:03
<
clever >
it would have to be patched
10:00
<
clever >
sphalerite: there is also something in stage-2 that can record the boot progress to that file
10:00
<
clever >
exec > >(tee -i /run/log/stage-2-init.log) 2>&1
09:58
<
clever >
stage-2 is also what mounts the root rw
09:57
<
clever >
sw/bin/init is raw systemd, and omits a lot of the bring-up stage-2 does
09:57
<
clever >
sphalerite: this one
09:57
<
clever >
[root@amd-nixos:~]# less /nix/var/nix/profiles/system/init
09:57
<
clever >
sphalerite: try /system/bin/init, no sw
09:56
<
clever >
sphalerite: are you using the nixos stage2?
09:55
<
clever >
Documentation/kdump/kdump.txt for further details.
09:55
<
clever >
a way to allow loading ring0 code at runtime, but then disable it for hardening
09:55
<
clever >
ah, kexec_load_disabled + modules_disabled
09:52
<
clever >
so it can inspect the ram nixos left behind
09:51
<
clever >
and boot the stock os as a crash kernel
09:51
<
clever >
sphalerite: for example, after you fail to boot whatever (as long as you can read the nixos FS), kexec over to the stock kernel & os
09:51
<
clever >
sphalerite: but having it in the nixos kernel may also be of use
09:50
<
clever >
sphalerite: oh, and is kexec available?
09:48
<
clever >
i can see it being useful for hardening, or to deal with broken bootloaders
09:47
<
clever >
sphalerite: and you can even tell the kernel to ignore params given by the bootloader
09:47
<
clever >
sphalerite: another potentially fun kernel option, you can embed kernel params into the kernel binary
09:41
<
clever >
in my case, i was using a packageOverride, so it was affecting the inputs of the existing linuxPackages sets
09:40
<
clever >
sphalerite: yeah
09:38
<
clever >
sphalerite: some, one min
09:33
<
clever >
sphalerite: nixos stage-2 will also try to remount it rw fairly early on
09:32
<
clever >
but given that it can find init, the root is mounting (possibly read-only)
09:32
<
clever >
sphalerite: there is also rootfstype, and rootflags you could play with
09:31
<
clever >
sphalerite: one sec
09:30
<
clever >
sphalerite: nice
07:44
<
clever >
Orbstheorem: and that e thing i think is only for grub
07:43
<
clever >
Orbstheorem: boot.kernelParams = [ "param1" ];
07:39
<
clever >
nix-repl> builtins.unsafeGetAttrPos "fetchurl" pkgs
07:39
<
clever >
{ column = 3; file = "/nix/store/q71jxrnm2a49s8hzfirm4ssri1x0lqqg-nixos-18.03pre118328.6d86fcb86d/nixos/pkgs/top-level/all-packages.nix"; line = 181; }
07:36
<
clever >
hyper_ch: nixos doesnt support dkms
07:33
<
clever >
nix-store -qR /run/current-system | grep zfs
07:32
<
clever >
hyper_ch: of the package or on-disk?
07:31
<
clever >
hyper_ch: hello
07:30
<
clever >
sphalerite: does lsmod show anything when running the original OS?
07:30
<
clever >
sphalerite: ah, that could become an issue
07:28
<
clever >
sphalerite: are you booting the same kernel it normally runs with?
07:05
<
clever >
yeah, a new /boot for nixos would be safer i believe
07:05
<
clever >
this code relies on grub, but it could probably be adapted to systemd-boot
07:05
<
clever >
fearlessKim[m]: it puts the entire nixos installer into /boot, at the cost of ~300mb
07:04
<
clever >
fearlessKim[m]: 512 or more
06:59
<
clever >
fearlessKim[m]: the rebuild will probably delete others, now that it see's the GC
06:54
<
clever >
fearlessKim[m]: try deleting one or 2 of those files, then redo nixos-rebuild boot
06:52
<
clever >
srhb: it might copy the new kernels in before deleting old kernels from /boot
06:47
<
clever >
fearlessKim[m]: garbage collection can help, but you will want to make it bigger
06:47
<
clever >
fearlessKim[m]: 96mb is pretty tiny, thats bound to run into issues
05:38
<
clever >
why are you trying to that? i seems like a fairly difficult thing to setup
05:38
<
clever >
but things like xinit and startx, skip the login screen
05:37
<
clever >
and the DM then drops root for its login page, and also for the user after they have logged in
05:36
<
clever >
ghostyyy: systemd runs the display manager as root
05:35
<
clever >
ghostyyy: there is still the problem that xorg needs root to access the gpu, and nixos lacks the setuid binary
05:10
<
clever >
fearlessKim[m]: there is also clangStdenv.mkDerivation
04:39
<
clever >
but i have been working on some experimental things, to build the entire nixops deployment under hydra
04:38
<
clever >
the simplest thing is to just build the large packages, load them up with callPackage and ensure you use the same nixpkgs and foo.nix in both hydra and nixops
04:38
<
clever >
hydra doesnt understand nixops
04:37
<
clever >
so now i have 32bit x86, 64bit x86, darwin, armv6, and armv7 builds of nix
04:37
<
clever >
dhess: with the right application of things like map, you can generate a variant of each job, for each platform
04:37
<
clever >
how you generate the set, and what platforms it contains, hydra doesnt care
04:36
<
clever >
and hydra will just build them
04:36
<
clever >
and hydra expects that nix expression to return a set of derivations
04:35
<
clever >
so hydra is building the same thing, against 2 different channels
04:34
<
clever >
and over here, it refers to nixos-unstable-small
04:34
<
clever >
in the example-master jobset, it refers to nixpkgs master
04:34
<
clever >
the <nixpkgs>
04:33
<
clever >
so line 3 refers to the nixpkgs defined in the inputs
04:33
<
clever >
every input is in NIX_PATH, and fed as an argument to the default.nix
04:33
<
clever >
the 'nix expression' line refers to the entry point in a given input (listed below)
04:32
<
clever >
this is an example hydra project
04:32
<
clever >
ah, i havent heard anything about fully automatic updating
04:32
<
clever >
and when it does nix-copy-closure to push the changes out, it will obey the remote nix.conf
04:32
<
clever >
dhess: behind the scenes, nixops just uses nix-build to build things on the deployer, so it obeys nix.conf as normal
04:31
<
clever >
dhess: it can easily pull from a hydra cache
04:30
<
clever >
services.hydra.enable = true; does most of the setup
04:28
<
clever >
useSubstitutes = true;
04:28
<
clever >
services = {
04:26
<
clever >
it can also do arm builds on raspberry pi's if you wanted
04:26
<
clever >
builder@192.168.2.126 armv6l-linux,armv7l-linux /etc/nixos/keys/distro 1 2 big-parallel
04:25
<
clever >
and you can set nix.buildMachines in configuration.nix to generate it
04:25
<
clever >
nixos and hydra just assume that file is at /etc/nix/machines
04:25
<
clever >
exactly the same file
04:24
<
clever >
an example config from /etc/nix/machines (nixos can generate this file)
04:24
<
clever >
builder@192.168.2.15 i686-linux,x86_64-linux /etc/nixos/keys/distro 3 4 big-parallel,kvm,nixos-test
04:24
<
clever >
and that the user it ssh's to can import unsigned storepaths (it either needs to be the single user, or be trusted in nix.conf)
04:23
<
clever >
all hydra requires is that "ssh user@darwin nix-store --version" can find nix-store, and that it can build things
04:19
<
clever >
dhess: yes
03:58
<
clever >
try removing that line and see if it still works
03:58
<
clever >
uhhh, yeah, thats not right
03:57
<
clever >
thats just going to waste disk space
03:57
<
clever >
build-dir isnt supposed to be installed
03:57
<
clever >
ah, thats ths shrink-rpath hook
03:55
<
clever >
viaken: run "nix-store -l /nix/store/hash-foo-1.2.3" to pull up the logs
03:53
<
clever >
viaken: can you pastebin the entire error?
03:52
<
clever >
what derivation is doing this?
03:51
<
clever >
its only used at compile time
03:51
<
clever >
thats a .o file, it cant be patcheld'd
03:48
<
clever >
viaken: what does "file" say about that file?
2017-10-22
20:48
<
clever >
sphalerite: dang!
20:47
<
clever >
sphalerite: also, what is the build-cores of each in nix.conf?
20:45
<
clever >
try setting it to null
20:44
<
clever >
sphalerite: the platform asserts stop you from reading super.gtk-sharp-2_0 in any way
20:38
<
clever >
kuznero: util-linux defaults to the .bin output
20:37
<
clever >
heading off to bed now
20:36
<
clever >
${libuuid.out}/lib/
20:35
<
clever >
and util-linux is the one that has libuuid.so
20:35
<
clever >
kuznero: libuuid is just an alias to util-linux
20:35
<
clever >
libuuid.out 0 s /nix/store/aswq968ln90nrykgi7902lg80vmibygf-util-linux-2.29.2/lib/libuuid.so
20:18
<
clever >
kuznero: by adding ${libunwind}/lib to the rpath list
20:17
<
clever >
kuznero: you have to add libunwind to the rpath
20:17
<
clever >
kuznero: buildInputs dont show up in the rpath when you patchelf
20:07
<
clever >
let the spam ensue!
20:06
<
clever >
sphalerite_: and they are back!
20:03
<
clever >
you need to open a shell with "sudo -i" then "nix-channel --list"
20:03
<
clever >
which breaks "sudo nix-channel --list"
20:03
<
clever >
sudo on darwin sets $HOME wrong
20:02
<
clever >
and it must start with the name of a channel from "nix-channel --list"
20:01
<
clever >
ldlework: you need -A
19:59
<
clever >
sphalerite_: one min
19:59
<
clever >
sphalerite_: what does /proc/cpuinfo say?
19:58
<
clever >
sphalerite_: it will dynamicaly adjust based on cpu load
19:55
<
clever >
sphalerite_: ah, it can go a almost twice as fast as an rpi
19:54
<
clever >
sphalerite_: how many mhz on the chromebook?
19:54
<
clever >
so you have to lock the entire x86 machine to one type
19:54
<
clever >
but the bigger issue is that the ELF header saying v6 or v7 is too complex for binfmt-misc
19:53
<
clever >
which qemu-wrap can set
19:53
<
clever >
i think there is a CLI arg you can pass to do that
19:52
<
clever >
and then things fail when moved to a real v6, that lacks v7 support
19:52
<
clever >
openssl (i think) will notice the v7 support when doing a v6 only build, and force v7 opcodes into the product
19:52
<
clever >
one minor problem ive run into, qemu emulates a v7 with v6 backwards compat
19:51
<
clever >
i use either "ps -eH x" or pstree for that kind of output, but they show the entire system
19:42
<
clever >
thats mostly IO bound
19:41
<
clever >
so it will be even to double, depending on how hyperthreading handles qemu-user
19:40
<
clever >
sphalerite_: also, how many cores in the chromebook vs laptop?
19:35
<
clever >
even if qemu is 50% the speed, you can still build at an effective 150%, by building across both machines with nix build slaves
19:33
<
clever >
and cluster that build!
19:33
<
clever >
sphalerite_: but you could now repeat the qemu-user stuff against all your x86 machines, and then add them all as build slaves
19:28
<
clever >
hmmm, chrome is still coredumping
19:28
<
clever >
Oct 22 16:26:20 amd-nixos systemd-coredump[22563]: Process 19452 (chromium) of user 1000 dumped core.
19:25
<
clever >
and because its staticly linked, it can probably just be (import <nix-misc> {}).qemu-user
19:25
<
clever >
sphalerite_: nix.sandboxPaths
19:25
<
clever >
build-sandbox-paths = ${toString cfg.sandboxPaths} /bin/sh=${sh} $(echo $extraPaths)
19:23
<
clever >
sphalerite_: the closure of qemu-user has to be added to build-sandbox-paths
19:23
<
clever >
sphalerite_: oh, right, nix sandbox, the qemu isnt present
19:22
<
clever >
thats better
19:22
<
clever >
Battery 0: design capacity 7800 mAh, last full capacity 3542 mAh = 45%
19:21
<
clever >
iqubic: i have 3
19:21
<
clever >
its % is better, but its capacity is much worse, lol
19:20
<
clever >
hmmm, this battery looks a bit better
19:20
<
clever >
Battery 0: design capacity 2400 mAh, last full capacity 1543 mAh = 64%
19:17
<
clever >
iqubic: thanks for reminding me to check, this battery is toast
19:17
<
clever >
Battery 0: design capacity 5200 mAh, last full capacity 1741 mAh = 33%
19:15
<
clever >
ldlework: so that kind of mess cant happen
19:15
<
clever >
ldlework: nix knows when the overrides have changed, and will use a different storepath for the overridden and non-overridden version
19:14
<
clever >
ldlework: why do you want to force a rebuild?
19:10
<
clever >
sphalerite_: i think updating to chrome 62 fixed the problem
19:04
<
clever >
seequ: it was nearly 10 minutes of people ping-timeouting
19:02
<
clever >
its a bouncer
19:01
<
clever >
iqubic: matrix got an exception to allow it
19:01
<
clever >
iqubic: those are all running on a single machine
18:53
<
clever >
adamt: looks like it
18:52
<
clever >
matrix is at it again...
18:46
<
clever >
MichaelRaskin: which isnt present in the version 60 i'm currently running
18:45
<
clever >
MichaelRaskin: so it can never even attempt to use enough ram to run
18:45
<
clever >
MichaelRaskin: in this case, i have 32gig of total ram, and 64gig of swap, but the process is setting a fairly low 2gig limit on itself
18:44
<
clever >
MichaelRaskin: and one of my extensions needs >2gig for a single tab
18:44
<
clever >
MichaelRaskin: the problem i ran into, is that chromium sets a data size limit of 2gig on every render process
18:43
<
clever >
sphalerite: cant think of any, see what happens if you just copy the pointers
18:37
<
clever >
Tekmo_: it may cause problems down the road, when the drv's it references get deleted
18:36
<
clever >
ehgads, i thought chrome was nearly done building when it claimed 700/800!!
18:36
<
clever >
[488/28739] STAMP obj/components/base32/base32.inputdeps.stamp
18:35
<
clever >
sphalerite: ah, nice, feel free to open a PR
18:34
<
clever >
Tekmo_: .drv files are supposed to be storepaths that have deps, and the hash of the contents (in non-fixed mode) matches the storepath
18:33
<
clever >
sphalerite: ah
18:33
<
clever >
Tekmo_: that could be a problem
18:32
<
clever >
Tekmo_: ah, but does it have the same path after being added back?
18:32
<
clever >
sphalerite: so map $HOME/nix to /nix, on systems where you lack root
18:31
<
clever >
sphalerite: how hard would it be to patch it to fake /nix instead?
18:31
<
clever >
sphalerite: ive been thinking, the fhs userenv stuff is able to use mount namespaces without root, to fake /lib and /bin...
18:28
<
clever >
and --add/--add-fixed only allow things that have no deps
18:27
<
clever >
Tekmo_: ah, the tricky part that the CLI api wont allow, the .drv must depend on the input .drv's
18:25
<
clever >
what about nix-store --add-fixed?
18:24
<
clever >
Tekmo_: it might be simpler to call builtins.derivation with the right args, and let nix generate it
18:20
<
clever >
it generally helps for name to contain the version, but you can also add a pname that lacks the version
18:19
<
clever >
jluttine: this is also a thing you can use
18:19
<
clever >
{ name = "package-name"; version = "1.2.3"; }
18:19
<
clever >
nix-repl> builtins.parseDrvName "package-name-1.2.3"
18:18
<
clever >
jluttine: some languages use pname for that
18:15
<
clever >
and 62 does have the fix
18:13
<
clever >
so, version 60.0.3112.90 is bork, nixpkgs master is at version 62.0.3202.62
18:01
<
clever >
sphalerite: but the interesting part, is that the comment perfectly describes my problem, and that whole if statement is missing from the version i'm currently running
18:01
<
clever >
sphalerite: so when it gets shifted by 33 bits, it doesnt overflow
18:01
<
clever >
sphalerite: i believe that forces gcc to treat the literal as a 64bit, even though 1 could fit into a 32bit
18:00
<
clever >
error: cloning builder process: Operation not permitted
17:55
<
clever >
sphalerite: and interestingly, lines 429-436 dont exist on the copy i'm running in nix
17:52
<
clever >
its been on my todo list for 2 years...
17:52
<
clever >
i still need to file a PR for that
17:52
<
clever >
that adds a new nix.conf field, a list of platforms the cpu "supports"
17:51
<
clever >
sphalerite: in nix-misc
17:49
<
clever >
sphalerite: the presense of /dev/kvm inside the nix sandbox is an impurity, that controled if the problem happened or not!
17:49
<
clever >
sphalerite: and because i have kvm enabled on my machine, i could never reproduce the problem
17:48
<
clever >
or rather, dies due to an illegal instruction
17:48
<
clever >
then cryptonite in haskell segfaults
17:48
<
clever >
sphalerite: so a qemu without kvm, lacks sse3 support by default
17:47
<
clever >
sphalerite: turns out, 64bit qemu emulates the very first 64bit cpu, one that lacks sse3 features that are now considered standard
17:45
<
clever >
sphalerite: ive even made an x86-64 qemu-user, on x86-64, and it was able to reproduce problems qemu-system-x86-64 had, without having to emulate the full kernel
17:45
<
clever >
sphalerite: and it sorta worked! (pulseaudio threw an exception though)
17:45
<
clever >
sphalerite: whats even more crazy, i just typed in the x86-64 arch one day, and built the same nix expression on a raspberry pi
17:43
<
clever >
but qemu-user just maps guest threads to host threads, and can use all the cores
17:43
<
clever >
sphalerite: also, qemu-system-arm doesnt have SMP support, so its limited to 1 core
17:43
<
clever >
sphalerite: yeah, because it wont need to emulate a kernel
17:26
<
clever >
sphalerite: aha, tracefile.25892:prlimit64(0, RLIMIT_DATA, {rlim_cur=2147483647, rlim_max=2147483647}, NULL) = 0
17:21
<
clever >
sphalerite: and it fails 100% of the time when trying to load a given page
17:20
<
clever >
sphalerite: found part of the issue, chromium is using ulimit to limit itself to 2gig of ram
17:19
<
clever >
iqubic: its linuxPackages at the 2nd one
17:18
<
clever >
iqubic: what line did you add to configuration.nix?
17:09
<
clever >
sphalerite: i'm currently fighting a very weird chromium bug, [pid 13045] mmap(0x9448d800000, 2097152, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = -1 ENOMEM (Cannot allocate memory)
17:02
<
clever >
JaakkoLuttinen[m: run "nix-store -q --tree" on the root .drv that nixos-rebuild prints
16:59
<
clever >
tobiasBora: nothing you install with nix-env can be found by the compiler
16:58
<
clever >
usbhid, and a driver for the usb controller i believe
16:58
<
clever >
ghostyyy: you need to include usb drivers in the initrd, boot.initrd.availableKernelModules
16:53
<
clever >
iqubic: boot.kernelPackages = pkgs.linuxPackages_4_12; for example