<samueldr>
iso image didn't cross-compile until really recently
<elvishjerricco>
Ah. Then I'll try your branch
<samueldr>
well, basically how many days since I changed cross-system
<samueldr>
I wanted a quick(er) ISO build for Tow-Boot
<samueldr>
so I yak shave'd myself into a corner
<samueldr>
and fixed cross-compilation for the iso
<elvishjerricco>
I wonder if I'm getting any inadvertent freebies with binfmt.emulatedSystems = ["aarch64-linux"]... Like maybe some of these builds *should* fail :P
<samueldr>
that's a possible thing I guess
<samueldr>
if qemu is given through to the sandbox
<samueldr>
but I don't use it
<elvishjerricco>
samueldr: Does qemu need to be given to the sandbox for the kernel to use it with binfmt?
<samueldr>
the kernel "just" executes with a given binary
<samueldr>
e.g. when you can ./thing.exe
<samueldr>
it passes it to wine or mono
<samueldr>
with a particular configured path
AmandaC has quit [Quit: WeeChat 3.1]
<samueldr>
and that path has to be accessible in the current... context?
<samueldr>
so the binfmt thing, when used for Nix builds, it adds paths to the sandbox
<elvishjerricco>
Oh
<elvishjerricco>
Yea I see that now: extra-sandbox-paths = /run/binfmt /nix/store/vgyjzpk541vxnhy36l7h3ld1m824cgpa-qemu-5.1.0
bennofs__ has joined #nixos-aarch64
AmandaC has joined #nixos-aarch64
bennofs_ has quit [Ping timeout: 268 seconds]
<elvishjerricco>
Well I managed to cross compile the iso. Whether or not it works, I have no idea
<samueldr>
heh, try booting it!
<samueldr>
(and you're back to square one of no working IBF on the CM4?)
<elvishjerricco>
Yea, I've paused trying to debug what's wrong with it until I get my serial cable (hopefully tomorrow)
<elvishjerricco>
But I did try booting this iso with qemu using `qemu-system-aarch64 -machine raspi3 -drive file=iso.qcow2,format=qcow2,snapshot=on -kernel tow-boot/tow-boot-rpi3.bin`
<samueldr>
no idea if that should work
<elvishjerricco>
I got through grub (keyboard didn't work though)
<elvishjerricco>
and now it's just sitting there
<samueldr>
oh, nice
<elvishjerricco>
still showing the last frame that grub put up "0s remaining."
<samueldr>
but still, no idea if that realy should work
<samueldr>
that sounds like something hung trying to continue with the boot
<elvishjerricco>
and spinning some cpu cores pretty hard
<samueldr>
oh, then maybe it's just slow
<samueldr>
oh right
<samueldr>
it is slow
<samueldr>
full blown emulation is slow
<elvishjerricco>
yea I was guessing it'll just be slow
<samueldr>
it's probably loading our quite big kerne
<samueldr>
kernel*
<elvishjerricco>
doesn't qemu have a JIT mode?
<samueldr>
not sure
<samueldr>
IIRC U-Boot's own bits for loading images in EFI mode are not exactly performant
<samueldr>
I figure my next attempt on the helios64 I'll try with angerman's kernel configs and with sd image
<samueldr>
or maybe just that kernel and the iso still, at first
<samueldr>
but not today
<samueldr>
though I did find two minor issues on Tow-Boot
AmandaC has quit [Quit: WeeChat 3.1]
<samueldr>
I don't know if mainline (with the assumed appropriate DTB built-into Tow-Boot) should be enough or not
<samueldr>
or maybe that DT is incomplete
<elvishjerricco>
Well I killed it so I could try again with `-serial stdio` but that didn't make any output.
AmandaC has joined #nixos-aarch64
Raito_Bezarius has quit [Ping timeout: 250 seconds]
Raito_Bezarius has joined #nixos-aarch64
<elvishjerricco>
samueldr: Shouldn't I have seen a bunch of stuff on stdout with `-serial stdio`?
<samueldr>
I don't know for sure
<samueldr>
I don't know what the "raspi3" machine actually does
<elvishjerricco>
Me either :P
<samueldr>
there is, though, a Tow-Boot build for qemu
<samueldr>
you just, somehow, don't have all the commits
<matthewcroughan>
well yeah because I started modifying it, so I'm just gonna need these changes.
<matthewcroughan>
like before you did this fix
<samueldr>
probably because you were using it from before 14 days ago
<matthewcroughan>
yup
<matthewcroughan>
disabling wpa_supplicant is absolutely brutal
<matthewcroughan>
I can't believe it depends on spidermonkey, what a world
<samueldr>
it's possible that it's one of _its_ dependency
<samueldr>
that actually depends on spidermonkey
<samueldr>
those are obviously not upstreamable fixes, they're workarounds
<gchristensen>
pcsclite -> polkit -> spidermonkey
<matthewcroughan>
As long as it compiles, I'm very very happy.
<matthewcroughan>
I just want a little pi to run docker and pihole that's built with nix ;~;
<matthewcroughan>
After this I will play around with Guix though to see what's up.
<matthewcroughan>
Probably more suited to the board I'm trying to use anyway. Nix is kind of a big thing, with systemd and all. I just love the tooling.
<matthewcroughan>
gchristensen: We can probably remove that dependency, and turn it into an option that can be enabled with an override?
<matthewcroughan>
samueldr: Yeah fair enough, I don't mean to suggest that systemd is the issue. Even I, a proponent of SystemD have allowed myself to be affected by the anti-systemd bias.
<clever>
if you run this, do you see some non-aarch64 paths in your closure?
<elvishjerricco>
That... could take a while to run :P
<samueldr>
IIRC we're not "pure" in that there may be parts of the toolchain
<elvishjerricco>
But I'll give it a shot
<clever>
elvishjerricco: you can run the same thing on the host that cross-built it, just use the right storepath
<samueldr>
but some of the previously obvious failures were fixed
<elvishjerricco>
Yea there are some paths here that don't have the aarch64-yadda suffix
<elvishjerricco>
like gcc
<clever>
and then `nix why-depends` to find the path between things
<clever>
its likely in the debug info of the aarch64 gcc
<clever>
something like that
<samueldr>
though we shouldn't have the issue where shell scripts use the x86_64 interpreter in their shebangs anymore for the "trivial" instances we had before
<matthewcroughan>
samueldr: I can't believe your nixpkgs worked out. Thanks so much.
<elvishjerricco>
So here's a stupid one liner to find all the non-cross builds in the closure of a derivation:
<elvishjerricco>
for p in $(nix path-info -r ./aarch64-cross-iso); do [ "$(nix-store -qd "$p")" != "unknown-deriver" ] && (nix show-derivation "$p" | jq '.[] | .inputSrcs[]' | grep "cross-file.conf" || echo FAIL "$p" 1>&2); done
<matthewcroughan>
colemickens: It could be that it works just fine, if I do *not* use your crosspkgs branch and just use 20.09, but disable polkit propely using those changes.
<samueldr>
if you look at the history, elvishjerricco, you'll see empty commits delimiting first the last commit found on an arbitrary master, then delimiting another bunch of WIP changes
<colemickens>
hm.
<elvishjerricco>
"Add support for Raspberry Pi 4 USB" uh... is that why usb didn't work on my nixos install? (The one that does boot; not the tow-boot one)
<matthewcroughan>
I just need to test that, but it's quite late here ;D
* colemickens
thinks about the "automated-known-good-arm-nixpkgs-revs" project again
<samueldr>
elvishjerricco: maybe
* colemickens
even bought flirc cases for an rpi3b+
<colemickens>
nvme not working can also be uas missing :D
<colemickens>
as I took a month or two to figure ut
<samueldr>
I'll have to take some time and upgrade my rk3399 builder, which uses an NVMe SSD
<samueldr>
last time I tried I ended up bricking it... temporarily at least
<matthewcroughan>
I've got some really weird interfaces on my orangepi..
<samueldr>
so I didn't end up taking the time to actually upgrade it
<matthewcroughan>
What on earth is sit0 and ip6tnl0, never seen those before.
* colemickens
wonders if he can build stage-1 mobile nixos yet
<matthewcroughan>
well, I'm wondering why I'm only now just seeing it
<samueldr>
colemickens: I'm confused
<matthewcroughan>
not like I've explicitly set this interface anywhere
<colemickens>
it's the whole "it re-evals the config for stage-1 manually and doesn't propagate the inputs arg" thing.
<samueldr>
oh right
<samueldr>
there's a PR for that now
<samueldr>
if you want to test the pair
<colemickens>
yeah, firing 'er up
<samueldr>
what are the chances I'll end up doing something wrong in my attempt at converting my pinebook pro setup into an ESP (still using extlinux for now) + LUKS rootfs?
orivej has quit [Ping timeout: 252 seconds]
<matthewcroughan>
Okay.. here we go
<matthewcroughan>
Trying to build a podman container to run at startup.
<matthewcroughan>
THE POWER
<matthewcroughan>
No more bitbake. I'm full.
<matthewcroughan>
Oh.. It's recompiling rust. Oh dear.. I might faint.
<samueldr>
reminder that there's basically no cache for armv7l, if you do native builds
<matthewcroughan>
Yeah that's alright. Though for some reason it's now including spidermonkey again.
<matthewcroughan>
That's a real problematic dependency.
<matthewcroughan>
OH crap.. I think Docker and Podman both have some link to spidermonkey.
<clever>
error: 1 dependencies of derivation '/nix/store/mkmrzh0996zwpjpkxkc971k8znib4rvx-nixos-sd-image-21.05.20210504.b2a67fa-armv7l-linux.img-armv7l-unknown-linux-gnueabihf.drv' failed to build
<clever>
that is the root drv your trying to build
<clever>
so you can `nix-store -q --tree` on that, to see the entire build-time dep tree
<clever>
then use `/spider` to search for that, and follow the line upwards
<clever>
and by memory, yeah, spidermonkey is the JS engine within polkit, because polkit rules are written in javascript!!
<matthewcroughan>
Here's an example of a buggy overlay I wrote. It should work, but it doesn't. The reason it doesn't seems to be because it evaluated `prev.ndi`'s `src` function, which is set to requireFile.
<matthewcroughan>
requireFile kills the build. But the point of this overlay was to override that behavior.
<matthewcroughan>
Alas. It builds both the overrideAttrs version and the previous version, because it has to, and consequently kills the build.
<matthewcroughan>
that's not gonna cross-compile it though.. is it?
<matthewcroughan>
I think it actually is O.o
<samueldr>
cool, so without any `console=` parameters, the kernel uses /chosen/stdout-path to determine the console to use, so by default on the pinebook pro that'll be the serial console
<samueldr>
not nice when you want to input the passphrase
<samueldr>
in other news, I'm not far from being able to actually start using UEFI on the PBP... once "known issues" are fixed
<clever>
[1/0/169 built, 0.0 MiB DL] waiting for a machine to build '/nix/store/6hwd71r02hlkk4d0aaypa6kavj71fyvp-zlib-1.2.11.drv'
<clever>
the new `nix build` api cant stop interrupting itself
<clever>
the last line of build log, is always getting covered up by complaints about the dep tree being wider then the build machine list
<samueldr>
in a pinch, to future-proof themselves, one can use the ESP as their /boot partition (as anyway NixOS assumes), but use extlinux.conf with it
<samueldr>
you might just need to set the legacy boot flag on it in some ways or other
cole-h has quit [Ping timeout: 260 seconds]
<matthewcroughan>
Wow, Docker takes absolutely forever to build :D
<matthewcroughan>
OH, nvm, it was using emulation O.o
<matthewcroughan>
I can't figure out how to do crossSystem there
<matthewcroughan>
virtualisation.docker.package = inputs.old.legacyPackages.armv7l-linux.docker_18_09; will not cross compile, I'm thinking I've one something wrong ;D
<clever>
because your flake file, said that legacyPackages.armv7l-linux is native
<matthewcroughan>
well typically, you'd say `nixpkgs.crossSystem = { system = "armv7l-linux"; };`
<clever>
that only impacts the nixpkgs nixos loaded, the { pkgs, config, ... }: one at the top
<matthewcroughan>
how can I have it impact this too?
<clever>
your flake is re-importing a different nixpkgs, with diff args
<matthewcroughan>
clever: what hardware are you doing all of this compilation on?
<clever>
a pi400
<matthewcroughan>
Lol, funny.
<matthewcroughan>
It's like time has looped back on itself.
<matthewcroughan>
I'm pretty sure people were doing compilation like this back in 1980
<matthewcroughan>
The tooling is better, but the principles are the same. Do it on your big fat-ass computer over there and pump it over to your cheap one
<clever>
yep
zupo has joined #nixos-aarch64
<n0emis[m]>
I'am even doing the compilations for my aarch64 devices on a server with binfmt...
<matthewcroughan>
n0emis[m]: binfmt is unbearable! :D
<clever>
[1/0/149 built, 57 copied (108.1 MiB)] building glibc-2.32-40 on ssh://pi@pi400 (round 1/2): patching file support/resolv_test.h
<clever>
progress is being made!
<matthewcroughan>
clever: actually, looks like something broke between that commit you're locked on and now
<matthewcroughan>
so if you did that darni
<matthewcroughan>
whoops, premature enter key
<matthewcroughan>
so if you did that compilation on nixos-unstable of today, or stable, you'd find git doesn't cross-compile because of perl532Packages.ModuleBuild
<domenkozar[m]>
clever: what's the situation with pi400 and NixOS?
<matthewcroughan>
where do overrides go again?
<matthewcroughan>
in overlays? or can they exist on their own?
<clever>
matthewcroughan: same as overrideAttrs, git = super.git.override { perlSupport = false; };
<clever>
domenkozar[m]: i would assume its the same as the pi4, but ive not checked it recently
<matthewcroughan>
I wish I could just blacklist Perl.
<samueldr>
domenkozar[m] / clever: needs a u-boot update, and probably adding the vendor dtb to the FIRMWARE partition
<samueldr>
as u-boot 2021.04 doesn't support the pi 400
<clever>
domenkozar[m]: the biggest differences from a driver perspective, is that the pwm audio is missing, and i think the wifi chipset might be a little different, and the vl805 eeprom is gone
<clever>
samueldr: ahh, what did they do to break uboot?
<samueldr>
nothing
<samueldr>
the vl805 eeprom being gone is not much of an issue since the newer rev of pi4 too lacks that eeprom
<matthewcroughan>
whether I cross compile or not, doesn't matter, this cli remains the same
<samueldr>
"can't see how i would have any effect" is totally different than "I see it having no effect"
<samueldr>
and it may not be an issue
<matthewcroughan>
Well, both are true.
<samueldr>
if the ecosystem handles it
<clever>
[1/0/2 built, 119 copied (1633.1/1633.6 MiB), 363.2 MiB DL] building go-1.16.4 (round 1/2) (buildPhase): Building Go toolchain1 using /nix/store/97pdgrxwgpnsmf4449cfr2jn0a19yl71-go-bootstrap/share/go.
<samueldr>
you can't see how *configuring Nixpkgs to do cross-compilation* would have any effect?
<matthewcroughan>
Oh, I can. I just cannot imagine how pkgsCross changes the CLI args of a go build.
<clever>
matthewcroughan: it might be using env vars
<matthewcroughan>
Would it not be better to build podman in accordance with the original makefile than to use buildGoModule?
<matthewcroughan>
Since the original makefile already has all the architectures accounted for, even ppc.
<samueldr>
the CLI args not changing is not necessarily an issue
<samueldr>
the compiler tooling might have the same name, and have been built differently in a way it knows the default target is now cross-compilation
<matthewcroughan>
Well, it errors with "error 1"
<matthewcroughan>
It's a very non descript error.
<clever>
[1/0/2 built, 119 copied (1633.1/1633.6 MiB), 363.2 MiB DL] building go-1.16.4 (round 1/2) (buildPhase): Building packages and commands for target, linux/arm64.
<matthewcroughan>
It reports that there's an error on the Makefile at L208
<samueldr>
if it runs with any sort of parallelism, it might as well be a few lines higher
<{^_^}>
golang/go#24433 (by pjebs, 3 years ago, closed): cmd/go: can't load package: build constraints exclude all Go files in
zupo has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
<samueldr>
but "Error 1" is make reporting that whatever it ran exited with exit code 1
<samueldr>
and it looks like, at a glance, that there is only one thing ran beforehand, no parallelism interleaving messages
<matthewcroughan>
> cgo is disabled when cross compiling.
<{^_^}>
error: syntax error, unexpected ')', expecting ID or OR_KW or DOLLAR_CURLY or '"', at (string):495:1
<matthewcroughan>
always do that by accident, sorry :P
<clever>
imports github.com/coreos/go-systemd/v22/internal/dlopen: build constraints exclude all Go files in /build/source/vendor/github.com/coreos/go-systemd/v22/internal/dlopen
* clever
opens grep.app
<matthewcroughan>
Yeah the cross compilation is not being set up properly.
<clever>
samueldr: by chance do you have @ powers in #nixos ?
<samueldr>
nope
<clever>
ah, gchristensen is awake
<clever>
building go a 3rd time!
<clever>
it seems `nix repl --option repeat 0` doesnt work
HenrikK has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
bennofs_ has quit [Read error: Connection reset by peer]
bennofs_ has joined #nixos-aarch64
alpernebbi has quit [Quit: alpernebbi]
<clever>
[1/0/129 built, 97 copied (202.9 MiB)] building gcc-10.2.0 on ssh://pi@pi400 (round 1/2): source root is gcc-10.2.0
<FantasyCookie17[>
My RockPro64 seems to just hang at `U-Boot TPL 2020.07 (Jul 06 2020 - 19:22:53)` after the first reboot after applying my config (the `matrix` config from https://codeberg.rog/FantasyCookie17/nixos-server-configs)… Does the baud rate constantly stay at 1.5M or is it expected to change? Note: I don't have the eMMC fix applied (yet, at least), but given it worked so far, and I didn't even get to the appropriate
<FantasyCookie17[>
error message, I assume that' not the issue here.
<FantasyCookie17[>
Nvm. It stopped hanging there; I got a bit further now…
<FantasyCookie17[>
I don't get any issues with hs200, though.
<clever>
build of '/nix/store/wr2g8mfx73gknyfw1c4hvgq9iry9fxkh-gcc-10.2.0.drv' on 'ssh://pi@pi400' failed: builder for '/nix/store/wr2g8mfx73gknyfw1c4hvgq9iry9fxkh-gcc-10.2.0.drv' failed with exit code 1
<clever>
samueldr: ack, something broke!
<clever>
builder for '/nix/store/wr2g8mfx73gknyfw1c4hvgq9iry9fxkh-gcc-10.2.0.drv' failed due to signal 11 (Segmentation fault)
<FantasyCookie17[>
Is there something I can do to fix this issue? Like, telling systemd to wait with mounting these or something?
<samueldr>
elvishjerricco: note that RX/TX sometimes is... "weirdly labeled"... never clear whether the cable labels "you should plug this cable to RX of the board" or "this is the RX of the adapter"
<samueldr>
elvishjerricco: so if you get no output from U-Boot, swap RX and TX as a first step
<clever>
i had come across a rather pesimistic post on the rpi forums, asking "how slow is go?"
<clever>
and after ipfs cross-compiled so easily, i gave their benchmark util a try with nix, so i would be using the exact same compiler on every target
<clever>
and 32bit is performing way way worse then 64bit
<FantasyCookie17[>
I heard Go's GC was poorly designed.
<clever>
FantasyCookie17[: i think the problem in this case, is using float64 based math on a 32bit build
<elvishjerricco>
samueldr: Back for a while. So how do I do the equivalent of `dtoverlay=dwc2,dr_mode=host` with tow-boot?
<FantasyCookie17[>
It doesn't free to the OS by default, only to itself, afaik, unless you set an environment variable, and it has rather poor performance.
Acou_Bass has quit [Read error: Connection reset by peer]
<samueldr>
it's meant to check cards you buy if they really are the size they say
<samueldr>
but also good enough to check whether they're broken or not
<elvishjerricco>
samueldr: So you're telling me to run `fdt addr ${fdtcontroladdr}` and => `fdt list /soc/usb@7e980000` in Linux, right?
<samueldr>
in u-boot console
<elvishjerricco>
oh
<samueldr>
:)
<elvishjerricco>
wait
<elvishjerricco>
so
<clever>
samueldr: let me make a backup with ddrescue first...
<elvishjerricco>
Which u-boot/dtb/config.txt combination should I be using for this?
<samueldr>
"correct" config.txt configuration for usb on CM4 according to the vendor, no dtb files
<samueldr>
with Tow-Boot
<clever>
elvishjerricco: my understanding is that you must have `dtoverlay=dwc2,dr_mode=host` (and the overlay may need to exist), or the usb controller will just hard fail when you access it
<clever>
plus what samueldr is saying, so linux knows to access it
<clever>
[12091731.197220] blk_update_request: I/O error, dev mmcblk0, sector 3553536 op 0x0:(READ) flags 0x80700 phys_seg 8 prio class 0
<samueldr>
clever: there won't be a linux for what I want to see
<clever>
or whatever os you plan to run below uboot
<samueldr>
I want to see if without a dtb file, and the proper config.txt configuration, the fdt as seen by U-Boot is valid
<elvishjerricco>
is ${fdtcontroladdr} supposed to be a variable I need to sub in?
<samueldr>
it's a variable already built into U-Boot
<matthewcroughan>
Welp, samueldr, thanks to you and a few others I got it all working
<samueldr>
elvishjerricco: you may be right that the overlay files need to be at the right place
<samueldr>
elvishjerricco: can you manage that too?
<matthewcroughan>
about 40 lines of nix, docker on startup, on an orange pi zero, running the latest kernel
<clever>
this dead SD card also explains the segfault gcc had earlier
orivej has joined #nixos-aarch64
cole-h has joined #nixos-aarch64
<elvishjerricco>
samueldr: Ok. Booted with `dtoverlay=dwc2,dr_mode=host`, with the overlay files present, and without the cm4 dtb, then running `fdt addr ${fdtcontroladdr}` followed by `fdt list /soc/usb@7e980000`, I get this: https://www.irccloud.com/pastebin/qerjroMe/
<elvishjerricco>
status = "okay";
<samueldr>
yep
<samueldr>
that's good news
<elvishjerricco>
What does that mean?
<samueldr>
status is a special flag in DTs
<samueldr>
"disabled" means it's disabled, "okay" means it's usable
<samueldr>
so for an OS (or for U-Boot) to use it, the status needs to be "okay"
<clever>
samueldr: do you know if the pi4 uboot in nixos has the dwc driver enabled? its possible to avoid it if you use xhci for both internal and external usb
<elvishjerricco>
So then why doesn't u-boot see it?
<samueldr>
dr_mode = "host";
<samueldr>
that is another thing the overlay overlaid
<samueldr>
elvishjerricco: can you re-try `usb start`?
<samueldr>
or did you already?
<elvishjerricco>
"No working controllers found"
<samueldr>
clever: why would you do that?
<samueldr>
elvishjerricco: hmmm
<matthewcroughan>
clever: did you ever get not-os booting on actual hardware?