orivej has quit [Quit: No Ping reply in 180 seconds.]
orivej has joined #nixos-aarch64
orivej has quit [Ping timeout: 264 seconds]
orivej_ has joined #nixos-aarch64
alp has quit [Ping timeout: 272 seconds]
orivej_ has quit [Ping timeout: 246 seconds]
orivej has joined #nixos-aarch64
orivej has quit [Ping timeout: 246 seconds]
orivej has joined #nixos-aarch64
alp has joined #nixos-aarch64
hoverbear has joined #nixos-aarch64
<patagonicus>
My replacement HC2 arrived. Now I can try to boot NixOS on it. :)
AmandaC has quit [Quit: Toodles]
AmandaC has joined #nixos-aarch64
orivej has quit [Ping timeout: 265 seconds]
orivej has joined #nixos-aarch64
orivej has quit [Quit: No Ping reply in 180 seconds.]
orivej has joined #nixos-aarch64
zupo has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
orivej has quit [Quit: No Ping reply in 180 seconds.]
orivej has joined #nixos-aarch64
zupo has joined #nixos-aarch64
quinn has joined #nixos-aarch64
orivej has quit [Ping timeout: 258 seconds]
zupo has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
orivej has joined #nixos-aarch64
orivej has quit [Ping timeout: 258 seconds]
orivej has joined #nixos-aarch64
<patagonicus>
Anyone want to help me getting nixos booted on my odroid hc2? I have an armv7 image that runs on a RPi3, but I'm fighting with u-boot.
<patagonicus>
My idea was to use the one from the official ubuntu image, but it wants the initrd in the uImage format, but I only have it as gzip compressed cpio image (since the nixos image uses uboot to get to extlinux first) and I have no idea how to get it into that format.
<patagonicus>
https://lastlog.de/blog/posts/odroid_xu4_with_nixos.html there's also this article, but it's not the newest and doesn't seem to work anymore, i.e for "installing DRM crap" the link doesn't contain the files it references.
<patagonicus>
And the image of nixos 15.09 provided towards the bottom doesn't boot for me, which is really weird.
bdju has quit [Read error: Connection reset by peer]
<patagonicus>
Well, I guess technically that is for the xu4, but it's supposedly almost the same board as the hc2.
bdju has joined #nixos-aarch64
<srk>
tried building recent u-boot for it?
<patagonicus>
Well, there's an image for the xu3/4 in nixpkgs, but I don't know how to install those files. ^^'
<patagonicus>
Oh, actually, I think those match up with the files mentioned in the article.
<patagonicus>
Yep. Let me try to get that u-boot booting.
<patagonicus>
Luckily the HC2 only has SD, no eMMC, so I only have one thing to worry about. And it's easily swappable.
<patagonicus>
Yep, got U-Boot 2020.01 up. :)
<patagonicus>
So, where/how do I install extlinux so that u-boot finds it? I think once I get to extlinux I can manage the rest - at least I'm familiar with extlinux and initrds. :)
<patagonicus>
Ah, I guess uboot just searches for an extlinux.conf?
<patagonicus>
Hmm. Now it doesn't boot anymore. Maybe GPT was a bad idea.
<patagonicus>
That PR "just" gives me the bootloader and kernel, not a full SD image, does it?
alp has quit [Quit: Leaving]
<srk>
that's the job of sdImage, not sure how to dd the uboot image automatically though
<patagonicus>
I'll try rebasing the PR.
<srk>
you can grab sd-image-armv7l-multiplatform and add the correct uboot manually
<patagonicus>
So, change the image to the kernel for the xu4 and build that, flash onto sd-card, then flash u-boot from PR onto sd-card.
<srk>
don't bother with kernel I would say, mainline should work (depending on what peripherals you need..)
<patagonicus>
Ok
<patagonicus>
Interestingly the uboot source is cloned from some German IT software's server.
<patagonicus>
Ah, no, that's apparently just the main company behind uboot. TIL.
rajivr has quit [Quit: Connection closed for inactivity]
alp has joined #nixos-aarch64
<patagonicus>
srk: Hmm, how do I built the uboot from the PR? For the one in upstream nixpkgs I just added odroid-xu3-bootloader to the pkgs to install in my configuration.nix (and then extracted that from /nix/store from the resulting sd-image).
<patagonicus>
I only half understand what that does, but it seems to work (with XU4).
<srk>
e.g. you can just nix build nixpkgs.pkgsCross.armv7l-hf-multiplatform.ubootOdroidXU3
<patagonicus>
Hmm, that was what I had booted earlier, but once I added a partition using GPT it stopped responding at all.
<patagonicus>
Ah, yeah. I flashed u-boot again and it's complaining about invalid GPT headers, so I'm guessing the GPT headers overlap with where the image wants to be. Let's try again with MBR.
<patagonicus>
https://pastebin.com/izHvXHsx looks like it's not reading my extlinux.conf. Maybe it does need to be vfat? Or do I need a separate config for u-boot?
<patagonicus>
I did mark the partition as bootable.
<srk>
lol Pastebin.com is under heavy load right now :(
<patagonicus>
Oh, wait, now I can't remount the FS. Maybe I created the partition to much in the beginning, but I don't thinks.
<srk>
paste.rs is also good
<patagonicus>
Ooh, maybe the config needs to be in an extlinux dir, not the root of the fs. I created it again with a larger first sector and copied the config both to the root as well as in an extlinux dir and it showed up. :)
<patagonicus>
Yep, I think it's having the directory. Ok, I'll add the kernel, initrd and the root fs and see if it boots.
<patagonicus>
It's trying to load /extlinux/../nixos/dtbs/exynos5422-odroid.dtb and is failing.
<patagonicus>
I think I should eat something before continuing with this.
<srk>
hehe, I've just finished eating
<srk>
you can look at the sd-image...multi.iso contents or just copy these or copy uboot over the iso (with dd)
<patagonicus>
Hmm, I should have copied the files, but there's no exynos5422-odroid.dtb. There is plenty of files that start with that, but then have hc1 or xu4 before the .dtb.
<patagonicus>
But the extlinux config only specifies the part, not sure where uboot gets the filename from.
<patagonicus>
*specifies the path to the directory
alp has quit [Ping timeout: 272 seconds]
<srk>
try printenv in uboot shell
<patagonicus>
"board_name=odroid" I guess that's it.
<patagonicus>
The prompt is also ODROID-XU3.
<srk>
hmm, not sure, maybe board_name is part of exynos5422-odroid you can try changing with set board_name odroid-...
<srk>
and boot(ing), saveenv might work as well for setting that persistently. best to add support for it so it finds it without manual fixes tho
<patagonicus>
Oh, wow, I have a shell.
<patagonicus>
I copied the hc1 and just renamed it.
<patagonicus>
It seems to have booted just fine. Didn't see any obvious errors, but I'll need to check. But I really should eat something first. :D
<srk>
cool!
bdju has quit [Quit: leaving]
bdju has joined #nixos-aarch64
bennofs has joined #nixos-aarch64
bennofs_ has quit [Ping timeout: 258 seconds]
alp has joined #nixos-aarch64
<patagonicus>
So, turns out all you need to do is: create the generic armv7 sd image and flash it to an SD card, go to the dtbs dir and cp the odroidhc1 file to odroid and flash the the xu3 uboot from nixpkgs.
<patagonicus>
It's not a clean solution - for one it would probably better if we could tell uboot to look for the right file and also we don't even need the first partition with the firmware, but it works.
<patagonicus>
Would it be ok to add that to the NixOS wiki as a new page for the XU{3,4}/HC{1,2} and mark it as community supported?
<patagonicus>
Of course with a note saying that it's not perfect, but seems to work. :)
<patagonicus>
srk: do you have Paypal or something like that? I'd love to send you a few bucks for a coffee or a beer since you probably saved me a couple of days of frustration in just a few hours.
<samueldr>
patagonicus: the FAT32 partition is only for the raspberry pi family
<samueldr>
the way they boot would make it *extremely* annoying not to include their firmware on it, considered to only *mildly* annoying for other boards
<samueldr>
because you would need to somehow get the partition on the SD card
<patagonicus>
samueldr: Yeah, but I don't think we can build an image that boots the generic stuff and also on the XU4 family since they use different uboots.
<samueldr>
if the FAT32 partition labels were longer, it would be RASPBERRYPI-FIRMWARE
<patagonicus>
But I think I can make my own derivation that uses sd-image.nix to build an image just for the hc2.
<samueldr>
yeah, was just telling you about what the FAT32 partition is, so you can totally ignore it
<samueldr>
and disable it now!
<samueldr>
(I think?
<patagonicus>
^^
<patagonicus>
My NixOS is mounting it, but u-boot ignores it, so no harm done.
<samueldr>
exactly, only a useless vestigial appendage :)
<patagonicus>
I'm going to try for a full rebuild on the native machine first, then I can worry about building a new sd-image. I also want to see if the board can handle it.
<patagonicus>
I think once that's through I'll order the other two HC2s and the three HDDs for them, because that should be the only hurdle left. :)
<patagonicus>
The main annoyance is the board_name thing not lining up with the file names of the dtbs. I wonder why/how to fix that.
<patagonicus>
Making a copy of the dtb file works, but it feels wrong. And at that point we could also get rid of the other dtbs.
<patagonicus>
Hmm, I guess I could try to change the extlinux config to point to the file instead of just the directory.
<samueldr>
for mainline u-boot, it's literally a value compiled in
<samueldr>
(for many board families)
<samueldr>
(most?)
<patagonicus>
Ah. Hmm. Well, the u-boot I'm using comes as a binary blob from Hardkernel themselves. :/
<samueldr>
so you likely would want to figure out where it is, and patch it for the right name according to the mainline linux FDT filenames
<patagonicus>
xu4 and hc1/2 are pretty much the same, as far as I know. The xu4 just has more IO, but the hc1/2 have a SATA port built in (using up the USB3 port of the xu4 - but HDMI is also missing).
<patagonicus>
HC1 and HC2 should be really almost identical. Main difference is the size of the casing - the hc1 only fits 2.5" drives, the hc2 can do 2.5" and 3.5". But maybe they use different voltages, I think I read something somewhere.
<samueldr>
>> Unfortunately (as of XU3/4), the FBL (equivalent of u-boot’s SPL) must be signed to run on Exynos5 SoCs. As a result EXYNOS5 devices must explicitly reuse the pre-built binary FPL code from HardKernel and can only use u-boot only as a second stage bootloader. this is also true for the HC2.
<samueldr>
right, so it's as I thought
<patagonicus>
That explains why I get the uboot messages twice.
<patagonicus>
But as long as I can do what I want with the second stage that doesn't matter, does it?
<samueldr>
that "doesn't explain it" possibly; there may often be two components, sometimes three for u-boot
<samueldr>
so it's not like you run two u-boot
<samueldr>
but you are using the first component of u-boot from the blob
<samueldr>
basically, u-boot is too big to be able to be loaded by some (most?) SoCs
<patagonicus>
Ah, ok, I think I understand.
cole-h has joined #nixos-aarch64
<samueldr>
so they make a tiny-teeny minimal u-boot that then, itself, does the minimum work to load the real u-boot
<patagonicus>
Similar to the master boot record and the multiple stages of grub. At least grub legacy did, not sure what grub2 is doing, but I doubt it fits in the first sector. :D
<samueldr>
kind of yeah
<samueldr>
but with u-boot it's "more fun" like: you don't have RAM, only the CPU cache, so you also need to initialize the RAM, and possibly other hardware fun!
<patagonicus>
^^
zupo has joined #nixos-aarch64
<samueldr>
it looks like everything you need is in the "blog.pcfe.net" article I linked, albeit requiring some assembly (not the language)
<samueldr>
it's likely you can extrapolate from the xu3 expression to get yourself an hc2 build going
<samueldr>
you won't even need to symlink the .dtb files
<patagonicus>
"U-Boot 2019-01 has the necessary change. No more symlinking necessary." Hmm. But my u-boot identifies as 2020-01.
<jwaksbaum[m]>
<samueldr "basically, u-boot is too big to "> Could the teeny tiny uboot be put on the eeprom of a pi4? Or still not teeny tiny enough
<samueldr>
possibly, but it's not to our advantage as the firmware from the raspberry pi foundation does a lot of work
<samueldr>
so either it would need to be compatible with loading that firmware
<samueldr>
or do the same work
<patagonicus>
Ah, I see. So instead of the hardkernel u-boot-dtb.bin they use the fedora u-boot.bin, but keep the rest of the files.
<samueldr>
and *if* we fit a u-boot SPL with the pi firmware on there, the firmware files on FAT32 partition, which are an unsightly mess, would still need to exist :)
<samueldr>
the boot process of those boards don't map well to the raspberry pi boot process
<samueldr>
there is no other file than the u-boot SPL and the "real" u-boot
<samueldr>
well, there is, it's where the SPL is tacked on
<patagonicus>
Sorry, I meant the four files that the fusing writes to specific sectors.
<patagonicus>
I.e, before it's on the SD card.
<samueldr>
oh right, yeah
<patagonicus>
So, is there a nix package for a generic mainline armv7 uboot I could use?
<samueldr>
no, there is no generic u-boot
<samueldr>
but there is an expression that builds specific u-boots out of mainline
<samueldr>
patagonicus: check in pkgs/top-level/all-packages.nix, search for "Upstream U-Boots"
<samueldr>
the XU3 u-boot that is actually in use by the expression you linked earlier :)
<patagonicus>
Oh. That's why the the u-boot-dtb.in was under a different nix store path.
<patagonicus>
But then why does it try to load exynos5422-odroid.dtb which doesn't exist?
<samueldr>
the build broke from upstream changes I guess
<patagonicus>
Ah! There's hardware.deviceTree.name for specifying a specific file.
<samueldr>
an extremely new setting
<samueldr>
(I think?)
<samueldr>
or else, the u-boot part is
<patagonicus>
"5 days ago" :D
<samueldr>
yeah, flo//kli did this recently
<samueldr>
for a situation maybe not dissimilar
<patagonicus>
I'll let it rebuild my packages natively (I just did nixos-rebuild build, not boot/test/switch) and then I can add that setting.
<samueldr>
though for his situation it was a new board he ported
<samueldr>
oof, that build may take a hot minute or a thousand
<patagonicus>
So, only thing left to do is figure out how to install the bootloader blobs to the sd card image, I think.
<patagonicus>
Yeah, I know. :)
<patagonicus>
Still faster than the RPi3 I tried it on before (which died twice before I gave up).
<samueldr>
heh
<patagonicus>
Probably for the best, I should do other things at least on Saturday.
<patagonicus>
But this was less painful than I had feared. Don't think much is stopping me from setting up the glusterfs cluster I'm planning, other than getting some more hardware.
<samueldr>
if you have a "fast" aarch64 machine, an option is to use a VM with hw virtualisation, relatively minor hit in perf, but you can get better hardware crunching at it
<samueldr>
but the aarch64 hardware needs to support armv7l/armv8 32 bit
<samueldr>
which is not a given
<patagonicus>
Nope, only have the HC2 and an RPi3. I know there's some unofficial binary caches, but I'd rather not use those.
<samueldr>
perfectly understandable
<patagonicus>
Once I now the HC2 can rebuild everything I'll get two more, then I can at least do distributed builds for the future.
<samueldr>
which is a reason why I push hard in validating cross-compilation does not have regressions to get people started in a trustable manner :)
<patagonicus>
:)
orivej_ has quit [Ping timeout: 256 seconds]
orivej has joined #nixos-aarch64
minicom9 is now known as minicom
<patagonicus>
Hmm, looking at the instructions for other community supported boards, a few of them simply have things like "put the generic image on the SD card, then download the uboot for this board and dd it to the right sectors".
<samueldr>
yeah
<samueldr>
as long as your board boots the mainline kernel, it should be possible to do it
<patagonicus>
Also I wonder if current nixos-unstable will even build on armv7. Cross-compiling it doesn't work as something (systemd, I think) is pulling in efibootmgr. But I don't know yet if that also happens when building natively, I have a suspicion that it's the host system leaking into the build.
<THFKA4>
i have two HC1s running Nix, they mostly work. haven't updated in a few months though
<THFKA4>
won't be getting another arm7 device for sure though
<patagonicus>
THFKA4: "mostly"?
<THFKA4>
Rust bootstrap was broken for a while, and the vendor kernel uhh..freezes at boot unless I enable debug logging
<THFKA4>
the Rust issue has been fixed though
<patagonicus>
Huh, weird. Well, I don't think I'll need rust and I'm going with the mainline kernel for now.
<THFKA4>
hmmm would mainline even be able to init the USB -> SATA bridge?
<THFKA4>
i remember running into that issue when trying
<patagonicus>
Although it wasn't a great sign that my first unit broke (as in broken hardware) after two days. And it broke in a weird way, it hangs during the kernel boot, before it loads the initrd, I think.
<patagonicus>
Well, I'm currently using swap on a SATA SSD, so I think it works. :)
<patagonicus>
Maybe I should have move /tmp to the SSD before starting the recompile, but oh well.
<THFKA4>
haha alright. i'm curious to see what your experience with mainline is going to be
<THFKA4>
i think the CPU governor was also tuned by the vendor
<THFKA4>
and possibly gigabit
<patagonicus>
Looks like I'll need cpupower utils for checking that. I don't want to reflash the system, so that'll have to wait.
<patagonicus>
THFKA4: hmm. Sending /dev/zero via nc over TCP gives me ~50MB/s, so around 400MBit/s. It's definitely more than 100MBit, but not even close to full GBit, so not sure what that is about. But I'm also running nixos-rebuild, so maybe that's eating resources.
zupo has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
orivej has quit [Ping timeout: 240 seconds]
orivej_ has joined #nixos-aarch64
<THFKA4>
i don't remember the exact USB topology on that thing, SATA and Ethernet are both bridged through USB though
<patagonicus>
We'll see. I don't need super high performance, it's mostly for storing borg backups of various machines. Plus some Prometheus monitoring and maybe a few random things here and there.
orivej_ has quit [Ping timeout: 264 seconds]
orivej has joined #nixos-aarch64
alp has quit [Ping timeout: 272 seconds]
<thefloweringash>
Is there a representative target in nixpkgs for a simple system that I could add to hydra? I guess I can always use (nixos {}).config.system.build.toplevel
<samueldr>
to test cross-compilation?
<samueldr>
I'd think the sd image is representative enough, and a good baseline
<thefloweringash>
“Also I wonder if current nixos-unstable will even build on armv7” <- seeing if I can answer that
<samueldr>
maybe not, cross has been on sick leave for a small while
<thefloweringash>
Was thinking of keeping an eye on the native situation
<samueldr>
oh
<thefloweringash>
Urg, sdImage is a little huge =\
<patagonicus>
thefloweringash: Well, give me some time. :) Once I've rebuild my system at the old commit, I'm going to try at the current nixos-unstable channel.
<samueldr>
>:| I have two build outputs, same byte for byte, one fails to work, one works, so it must be something about the closure rather than the output, but I can't wrap my head around how runCommand would work while mkDerivation would fail
orivej has quit [Ping timeout: 265 seconds]
<samueldr>
AH
<samueldr>
found it
<samueldr>
/nix/store/8mmbkcnghwqix19rnnck4q1wq83ldw0j-boot-gui.mrb-aarch64-unknown-linux-gnu vs /nix/store/c78i7r0hq1mjqh70az2ris7lixwinawh-boot-gui.mrb
orivej has joined #nixos-aarch64
<samueldr>
mruby uses the filename, seemingly of the resolved link, to know whether it loads bytecode or ruby code
<samueldr>
whew, now it makes total sense and I hate this
<samueldr>
and that also explains why it didn't fail within qemu, it's all a native build in qemu!
orivej has quit [Ping timeout: 240 seconds]
orivej has joined #nixos-aarch64
lordcirth has quit [Remote host closed the connection]