<clever>
samueldr: do you happen to know why this image is missing from the latest master?
<samueldr>
it was never meant to stay
<samueldr>
(and anyway it hadn't been building for a few months)
<clever>
i noticed the failures
<samueldr>
the generic sd image is the only provided and supported image
<samueldr>
and now works just fine on the raspberry pi 4B
<samueldr>
oops, I see how it can be an issue for the 400
<clever>
does the generic one already have firmware?
<samueldr>
yes, same setup as usual
<samueldr>
FAT32 FIRMWARE partition for raspberry pi because it's too inconvenient not to ship it with
<samueldr>
because adding a partition after the fact is hella inconvenient
<clever>
is the firmware already on the partition, or is it just a blank partition?
<samueldr>
already configured to be bootable on the Pi 3 family (including +) and 4B
<clever>
nice
<samueldr>
400 and CM4 might not work since U-Boot 2021.04 doesn't have support for the CM4 and 400 yet
<clever>
the one i linked above, just boot-loops on a pi400, uboot cant find the sd card
<clever>
but on a pi4, it just worked
<samueldr>
given it was from january, it *may* have been 2021.01
<clever>
given that its from 2021-01, that would make sense
<clever>
yep
<samueldr>
if you update U-Boot to master and rebuild the generic sd image, it might just work
<samueldr>
and since there would only be u-boot to build, would be a quick native build on the rpi4
<samueldr>
well, slow because of I/O
<clever>
could i also just nix-build the new uboot, and copy it to the firmware partition?
<samueldr>
but mainly assembling the image
<samueldr>
oh
<samueldr>
I guess that would work
<samueldr>
*might* or may not need adding the dtb file to the root of the firmware partition
<clever>
the rest is already good enough, and nixos can deal with any updates
<samueldr>
for the 4B it was required at one point
<samueldr>
well, NixOS doesn't manage the partition at all
<samueldr>
the firmware partition
<clever>
the rootfs though
<samueldr>
(in the expected use case)
<samueldr>
yes
<clever>
yeah, firmware i update myself
<samueldr>
use the new_kernel variant though
<samueldr>
just in case there's a difference between 5.10 LTS and 5.11 (or 12)
<clever>
i was thinking just let nixos upgrade the jan image
<samueldr>
oh
<samueldr>
sure
<clever>
nixos-rebuild --upgrade for the rootfs, manual for uboot
<samueldr>
yep, should work too, probably, hopefully
<clever>
first thing, configuration.nix is missing
<clever>
as usual
<samueldr>
I guess I'll need to do a bit more testing, but I just tried using phosh on an x86_64 system and... I get a cursor in a black void!
<samueldr>
not optimal
<clever>
samueldr: nixos-generate-config, edit some, nixos-rebuild switch, and reboot, uboot failed to find most things, and booted the original generation!
<samueldr>
success!?
<clever>
not really
<clever>
the new generation cant be found by uboot
<samueldr>
I mean, it didn't brick!
<clever>
so it undoes all changes on each reboot
<clever>
its nearly as bad as the previous sd card :P
<clever>
one gc cycle, and it will be a brick
<samueldr>
there must be a configuration option missing or something
<clever>
[root@nixos:~]# ls -ltrh /boot/nixos/q95k3nasvk0j3891639da25378qj2zn0-linux-5.4.88-dtbs/broadcom/
<clever>
and pi4, is missing!
<clever>
so, each generation searches for dtb files in a unique dir, but the new generations lack a pi4 dtb file
cptchaos83 has joined #nixos-aarch64
<clever>
and uboot successfully does fallback, until a generation works
<clever>
kernelPackages is on the default value
<samueldr>
sounds about right
<clever>
so nixos-generate-config generates a cfg that fails to boot
<samueldr>
as it might if you had a too new laptop
<clever>
testing out linuxPackages_rpi4
orivej has quit [Ping timeout: 265 seconds]
<clever>
samueldr: heh, thats the exact storepath the original generation was on, guess thats the answer!
<clever>
uboot seems rather slow, it sits at `scanning mmc 0:2` for a while
<clever>
but it is booting the newest generation this time
<samueldr>
alright, so at least the phosh issue is not that it outright doesn't work, same nixpkgs revision it works with Mobile NixOS, so I guess I have some dreaded state breaking things
<clever>
i'm testing that linuxHeader thing i mentioned before
<clever>
samueldr: https://termbin.com/ram4 i believe this omits the patch only on native arm32, but includes the patch on cross-arm32
<clever>
the patch was originally meant to fix a cross-arm32 bug, and it also broke native arm32
<elvishjerricco>
Alright. Back on the cm4.
<elvishjerricco>
No idea what to look into next :P
<samueldr>
hmmmmmmmmmmmm... the "lockscreen" for phosh is dismissed whatever input I give it
<clever>
samueldr: found the cause of why dwc fails, the defconfig in tow-boot is only loading a dwc2 gadget mode driver
<clever>
its not capable of driving it in host mode
<samueldr>
I saw it was named gadget in the option, but I didn't check further, I assumed it was named that way because of legacy reasons
<samueldr>
interesting though
<samueldr>
since all CM4 boards are different, I assume CM4 users would want to customize the build for their CM4 use case
<samueldr>
not sure how to make this happen really...
<samueldr>
... and in that case why not U-Boot specialized for their use case?
<clever>
samueldr: the uboot has an xhci driver, which is the "better" interface
<clever>
aha, driver_check_compatible will compare each of_match in a given struct driver
<clever>
samueldr: do you know how you turn on a log category in uboot?
cole-h has quit [Ping timeout: 240 seconds]
<clever>
samueldr: everything i'm seeing, says that uboot is ignoring the dtb it got from "somewhere", and not even checking the compatible or enable keys
<clever>
ok, so its built with CONFIG_OF_BOARD, which means board_fdt_blob_setup will supply a dtb from the firmware, and thats what lands in ${fdtcontroladdr}
<clever>
and that dtb should be activating hw
<clever>
everything says it should be working
<clever>
yet its not printing any logs
<clever>
[root@nixos:~]# vcgencmd measure_temp
<clever>
temp=81.3'C
<clever>
[root@nixos:~]# vcgencmd get_throttled
<clever>
throttled=0x60000
<clever>
oh, thats why gcc took so long!
ajs124 has quit [Quit: Bridge terminating on SIGTERM]
ajs124 has joined #nixos-aarch64
zupo has joined #nixos-aarch64
zupo_ has joined #nixos-aarch64
zupo has quit [Ping timeout: 265 seconds]
HenrikK has joined #nixos-aarch64
HenrikK has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
jdnixx-M1 has quit [Quit: Bridge terminating on SIGTERM]
ky0ko has quit [Quit: Bridge terminating on SIGTERM]
ky0ko has joined #nixos-aarch64
jdnixx-M has joined #nixos-aarch64
HenrikK has joined #nixos-aarch64
dev_mohe has joined #nixos-aarch64
dev_mohe has quit [Client Quit]
HenrikK has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
<matthewcroughan>
clever: do you think not-os would work with Docker?
zupo_ has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
<matthewcroughan>
To clarify. I mean, do you think it could run Docker containers?
<matthewcroughan>
Maybe will have more luck with Podman. I'm anticipating issues with Dockerd not working.
monk has left #nixos-aarch64 ["Error from remote client"]
monk has joined #nixos-aarch64
orivej has quit [Ping timeout: 240 seconds]
ryantrinkle has joined #nixos-aarch64
monk has left #nixos-aarch64 ["Error from remote client"]
monk has joined #nixos-aarch64
cole-h has joined #nixos-aarch64
<clever>
-r-xr-xr-x 1 root root 127K Dec 31 1969 /nix/store/69yk3j89af7fqf5lkn9njlay7nqrj5mr-python3-3.8.9/lib/python3.8/lib-dynload/_ctypes.cpython-38-arm-linux-gnueabihf_failed.so
<clever>
what does the "failed" in here refer to?
<clever>
*** WARNING: renaming "_ctypes" since importing it failed: /nix/store/gydwsw4ffkfln89isj8zb6w319b4sa3a-libffi-3.3/lib/libffi.so.7: undefined symbol: ffi_prep_closure_loc
<clever>
aha
monk has left #nixos-aarch64 ["Error from remote client"]
<clever>
Following modules built successfully but were removed because they could not be imported:
<clever>
_ctypes
zupo has joined #nixos-aarch64
zupo_ has joined #nixos-aarch64
zupo has quit [Ping timeout: 260 seconds]
monk has joined #nixos-aarch64
<samueldr>
clever: so you're saying it does use the firmware provided DT, but for some reason the logs you activated (?) are not shown
<samueldr>
I assume logs you activated are in the non-OTG variant of the driver?
monk has left #nixos-aarch64 ["Error from remote client"]
monk has joined #nixos-aarch64
<clever>
samueldr: i added my own print statements to the probe code for dwc, and it doesnt run
<clever>
which implies that either its not loading the driver, or the uart isnt online yet
<clever>
i found mode pre-existing debug in the DT parser, where it finds drivers and activates them
<clever>
but i have no idea how to see that log
<clever>
samueldr: lists_bind_fdt() will iterate over every node in DT, report the name/compatible, if its disabled, and then try to load a matching driver
<clever>
but i'm not able to see those logs
<clever>
according to the source and readme, it should take the firmware provided DT (which i can read with the cmds you gave lastnight), and enable its own internal drivers
<samueldr>
yep, fdt addr ${fdtcontroladdr} is the right way to check the built-in/previous-stage firmware
<samueldr>
to me it really sounds more like the probe isn't being run than otherwise
ryantrinkle has quit [Ping timeout: 240 seconds]
<samueldr>
the driver is being built, right?
<clever>
yeah, i do see the driver being rebuilt, if i run buildPhase after editing it
<clever>
i need to switch over to netboot, so i dont have to swap cards constantly
dev_mohe has joined #nixos-aarch64
monk has left #nixos-aarch64 ["Error from remote client"]
<matthewcroughan>
clever: are there any examples of not-os being built for a raspberry pi?
bennofs__ has quit [Read error: Connection reset by peer]
<elvishjerricco>
clever: So you're saying I need to change this to have `TFTP_IP=192.168.2.15` `BOOT_ORDER=0x5241`?
<samueldr>
hey, just saying, I probably won't spend much time on Tow-Boot, U-Boot and NixOS on ARM things for the last half of this month since... well... Mobile NixOS needs to be worked on :)
<elvishjerricco>
and then flash the eeprom?
<clever>
your BOOT_ORDER must include a 2 somewhere
<clever>
where, depends on the priority you want
<clever>
your 0xf41 says to try sd, usb, then loop, until something works
<clever>
my 0x5241 says to try sd, usb, network, the 2nd usb, and then give up
<clever>
elvishjerricco: having TFTP_IP makes it simpler, you dont have to mess with the dhcp server then
<elvishjerricco>
Hm well I don't have the ability to connect this pi to my home network with ethernet, so I'd have to wire it straight to my desktop
<clever>
then you also need something else
<clever>
TFTP_IP and CLIENT_IP
<clever>
that lets you configure a static ip on the pi
<clever>
then throw a static on the nixos machine too
<clever>
or run a dhcp server on the desktop
<elvishjerricco>
and exclude the interface from networkmanager I'm sure
<clever>
another handy thing, if you put pieeprom.{udp,sig} on that tftp directory, the pi will reflash itself over the network
<elvishjerricco>
Oh, that is handy
<clever>
obviously, interrupting it will result in a soft-brick
<clever>
and you need recovery.bin on a uSD to unbrick it
* colemickens
might find a new home for his pbp now that he has a real laptop again
<elvishjerricco>
I'll take it :P
monk has left #nixos-aarch64 ["Error from remote client"]
monk has joined #nixos-aarch64
monk has left #nixos-aarch64 ["Error from remote client"]
monk has joined #nixos-aarch64
HenrikK has joined #nixos-aarch64
<elvishjerricco>
clever: What kind of permissions are needed on the services.tftpd.path directory?
<elvishjerricco>
I'm getting "File Not Found" from `curl tftp://127.0.0.1/config.txt`
zupo has joined #nixos-aarch64
<clever>
elvishjerricco: the tftp daemon needs to be able to read the files, and have +x on the parent dirs
<clever>
and it runs as nobody
<clever>
so o+x on dirs, and o+r on the files
monk has left #nixos-aarch64 ["Error from remote client"]
zupo_ has quit [Ping timeout: 265 seconds]
<samueldr>
use namei to list information about a tree
<elvishjerricco>
Ok, got it. `curl tftp://192.168.3.1/config.txt` now shows config.txt
<elvishjerricco>
Huh. I didn't manage to update it apparently...
<elvishjerricco>
d'oh
<elvishjerricco>
I used nix-build instead of nix build for the first time in forever and forgot that I have alias nix-build="nix-build --no-out-link"
<clever>
*doh*
<elvishjerricco>
There we go
monk has joined #nixos-aarch64
<samueldr>
now that's working against yourself efficiently
<elvishjerricco>
Ok, I can see that it's *trying* somewhat to netboot. I decided to try using a dhcp server instead of a static ip, but it's not working. The server keeps giving DHCPOFFER, but doesn't get anything back.
quinn has joined #nixos-aarch64
<clever>
elvishjerricco: does the dhcp server have a range and a static ip?
<clever>
elvishjerricco: line 77 says what interface to listen on, line 113 gives it an ip, dhcpd will lookup what ip is on that interface, to decide what config block (line 81) to follow
<clever>
line 87 then gives out ip's to clients
<elvishjerricco>
clever: Yea my config is actually just a stripped down version of that. And I've verified it works by booting the pi off a working sd card
<clever>
anything in the dhcp server logs? what is the full exchange in wireshark?
<elvishjerricco>
May 17 15:45:23 pyromancer dhcpd4[20093]: DHCPDISCOVER from dc:a6:32:fe:7a:33 via enp4s0
<elvishjerricco>
May 17 15:45:24 pyromancer dhcpd4[20093]: DHCPOFFER on 192.168.3.120 to dc:a6:32:fe:7a:33 via enp4s0
<elvishjerricco>
I just see this in the dhcp log
<elvishjerricco>
repeating, with the ip increasing by one each time
<elvishjerricco>
no idea how to wireshark :P
<clever>
double-check that the ethernet cable has all 8 wires, and no pins are damaged on any connectors
<elvishjerricco>
clever: I mean I was able to ssh over this connection when I was testing the dhcp with the working sd card
<elvishjerricco>
Do I just `wireshark -i enp4s0` or something?
<clever>
wireshark has a gui to select an interface
<elvishjerricco>
Guessing I need to run it as root?
<clever>
reading the git history can tell you when things changed
<elvishjerricco>
clever: would a different version not have this stage 2 uart bug?
<clever>
possibly, you could try older versions and see what works
superherointj has joined #nixos-aarch64
<elvishjerricco>
Don't know if this matters, but apparently `vcgencmd otp_dump | grep 17:` is supposed to yield `17:3020000a`
<elvishjerricco>
But I get `17:000048b0`
<clever>
*looks*
<elvishjerricco>
Apparently it should be `17:3020000a` after you boot with `program_usb_boot_mode=1` in config.txt. I've done that, but it's still `17:000048b0`.
<clever>
program_usb_boot_mode is pi3 only
<elvishjerricco>
Just wondering if this has anything to do with why u-boot wouldn't find the usb controller
<clever>
it has zero effect outside of the pi3
<elvishjerricco>
ah whoops
<elvishjerricco>
alright anyway, back to netboot
<clever>
i also managed to piss off one of the engineers on that subject :P
<elvishjerricco>
or... maybe I stop wasting time with that...
<clever>
there was a history of pi4's being mysteriously bricked
<clever>
and one day, somebody reported using program_usb_boot_mode=1 on a pi4, and then it was bricked
<clever>
a quick check of the boot rom disassembly, said that the bit that once did usb-boot, is now something else
<clever>
and if it doesnt match something, it dont boot!
<clever>
elvishjerricco: upon discovering such a fault, what would you do?
<samueldr>
whew
<elvishjerricco>
clever: so the device was perma-bricked?
<clever>
that was my theory
<elvishjerricco>
ouch
<clever>
and i quickly opened a ticket to report it as such
<clever>
>> It's a shame you didn't check that before spreading the fear, uncertainty and doubt.
<clever>
because i reported an issue, and didnt disassemble every piece of involved code on the spot
<clever>
i was accused of "spreading the fear, uncertainty and doubt"
<clever>
the engineer was giving very short and non-descript answers, like "Thank you for your concern, but program_usb_boot_mode does nothing on a Pi 4."
zupo has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
<elvishjerricco>
clever: I'm moving back to this channel. #raspberrypi people seem to have an irrational disdain for u-boot
<clever>
ive noticed the same on the rpi forums, for any bootloader other then stock
<clever>
that includes uefi
<clever>
and ive also helped people de-uboot their installs before, just to make dtoverlay work
<clever>
which reminds me, i found a new trick to make de-ubooting simpler
<samueldr>
that's quite ridiculous
<samueldr>
sure, U-Boot "makes things more complex"
<samueldr>
but only because the vendor does their own weird things
<clever>
the official firmware, also ignores the kernel= statement, when you netboot
<clever>
so it just silently doesnt load uboot anymore
<samueldr>
one thing I want to work on with Tow-Boot is allowing better support for vendor things like those at run-time, e.g. a "bios" menu for vendor shenanigans
<clever>
i had given them the idea to add a bios menu to the existing spi flash, they turned it down
<clever>
i considered adding one to the desktop app (rpi-imager), but upon reading its source, i can see why it doesnt
<clever>
the imager is as dumb as a sack of bricks
<samueldr>
clever: that's the reason I implemented the Tow-Boot menu with PDCurses
<samueldr>
so building rich TUIs is easy
<elvishjerricco>
clever: Are you certain that the firmware just ignores kernel=, or is that just an assumption based on my experimentation?
<clever>
the choice between allowing usb or not, isnt a client side choice
<clever>
there is a json file, that lists available .img files
<clever>
and the image with the config, was pre-generated, and put in a list
<clever>
adding a bios style boot-order menu to rpi-imager, would be a major overhaul
<samueldr>
one big issue is HID support,
<samueldr>
they kind of shot themselves in the foot with a small eeprom
<clever>
elvishjerricco: it appears to be ignoring the entire config.txt, the dwc overlay isnt loaded, uart_2ndstage doesnt work, kernel= doesnt work
<clever>
samueldr: a secondary problem, is that the initial binary you load, is limited to 128kb
<clever>
to get over that limit, you either need plugins and some kind of dynamic linker, or load a second executable and chainload
<samueldr>
meh, lots of platforms have similar issues
<samueldr>
not like it's insurmountable
<clever>
there are 2 pi specific things you could potentially add to tow-boot
<clever>
1: you can sniff the spi->start protocols, and know where the firmware booted from
<clever>
so uboot always boots from the right place
<clever>
2: you could have a gui to edit the bootconf.txt and boot-order in the SPI flash
<samueldr>
Tow-Boot will never "sniff" to know where to boot from
<samueldr>
and I'll explain
<samueldr>
it makes the boot process more inscrutable when you treat it as "just a bios"
<samueldr>
because now it matters where it is installed
<clever>
elvishjerricco: reading the kernel.pcap file first...
<samueldr>
the boot order is already part of the environment, so I "just" (hah) need to properly get things like `saveenv` working on all platforms in a sensible manner
<clever>
i can see its fetching start4.elf and config.txt from the serial# subdir
<samueldr>
then make a UI that "just" edits the boot_targets variable
<clever>
then it loads pieeprom.sig (not found)
<clever>
then it begins downloading the actual start4.elf
<clever>
then it grabs fixup4.dat
<superherointj>
samueldr, bought today a Raspberry Pi 4 to install NixOS. Can test Tow-Boot :)
<clever>
elvishjerricco: try deleting the [pi3] section entirely, and the [pi4] header, what happens then?
orivej has quit [Ping timeout: 252 seconds]
<elvishjerricco>
clever: those things had been commented out in the boots that I sent you
<elvishjerricco>
Though I guess I was assuming comments work...
<clever>
elvishjerricco: clone this repo, and try doing a bisect, use symlinks to wire in start4.elf and fixup4.dat, the other files dont matter
<clever>
boot each version, and see what it does
<elvishjerricco>
Oh wow this repo is going to take forever to clone
<elvishjerricco>
They ship kernel binaries in that thing
<clever>
yes
<clever>
they wiped the history at one point too
<elvishjerricco>
Yay...
<clever>
i dont know how they did it, but a clone stops at some point
<clever>
but the parent commit is still visible as a hash
<clever>
`git fetch origin <hash>` can pull it, and get more
<clever>
ive looked at the firmware in the original commit, and it doesnt even look like pi firmware
<clever>
it looks like it belongs in a cable box
<elvishjerricco>
I'm guessing I want to remove the kernel8.img copy while I'm bisecting this? So I know whether it's respecting kernel= i.e. the config.txt?
<clever>
yeah
<samueldr>
clever: highly likely they just started from an existing broadcom cable box devkit
<clever>
samueldr: yep
<clever>
samueldr: it still had support for loading games and such
<elvishjerricco>
Geez. 12% is 2.34GiB
<samueldr>
ooh
<clever>
[clever@amd-nixos:~/apps/rpi/firmware]$ du -h .git
<clever>
16G .git
<elvishjerricco>
clever: I bet we could get a better response on the rpi forums or something if we came up with a config.txt thing that has nothing to do with u-boot
<clever>
and its sad that we have to hide what we are trying to do
<elvishjerricco>
btw did you make any progress on tow-boot -> usb on your pi400?
<clever>
not since lastnight
<clever>
i'm stumbled, since all attempts to get logs out of it fail
<elvishjerricco>
yuck
<clever>
stumped*
<samueldr>
I'll try looking at it some during the week-end, if I remember