<samueldr> hmmm
<samueldr> I may be accidentally causing issues with ofborg builds on the aarch64 community vm
<samueldr> or not
* samueldr should have read the failure log better
<samueldr> I thought it was being culled by some oom or other process (and looked at dmesg)
<samueldr> (I did have one msquashfs failure which couldn't be accounted for)
<samueldr> (and it uses all the cores when possible)
<samueldr> haha oh I almost did it
<samueldr> got two bugs to figure out, but I just booted the nixos installer iso as aarch64 from uefi
<samueldr> using this (and only this) on the SD card https://github.com/andreiw/RaspberryPiPkg
<samueldr> (the two bugs are trivial, misnamed files)
<samueldr> built using nix-build -A config.system.build.isoImage -I nixos-config=modules/installer/cd-dvd/installation-cd-minimal-new-kernel.nix in the <nixpkgs/nixos> folder
<samueldr> nbp: ^
<gchristensen> nice
<samueldr> it's 4.19 mainline, not sure if it should work on your device... though grub should show up
<samueldr> and nice to note: that UEFI thing for the raspberry pi seems good enough to test things out
<samueldr> not sure what its limitations are
<samueldr> ah, and about trust: DO NOT trust that image
<samueldr> sorry, forgot to preface that it is built on the community builder
<samueldr> (read the notes to know why not to trust it https://github.com/nix-community/aarch64-build-box )
<gchristensen> samueldr: how do I use it? :)
<samueldr> the iso image?
mog has quit [Ping timeout: 245 seconds]
<gchristensen> yea. dd to an sdcard and boot?
<samueldr> exactly
* gchristensen goes to get his odroid c2
<samueldr> but you need to have an SBC with UEFI
<samueldr> which I'm not sure the odroid-c2 does
<gchristensen> ah
<gchristensen> "U-Boot Now Supports UEFI on 32-bit and 64-bit ARM Platforms"? maybe?
<samueldr> yeah, but if you have u-boot, you already can boot the classic sd-image-aarch64
<samueldr> with the current setup, you lose functionality by going UEFI instead of u-boot,
<samueldr> (the way u-boot will load the right device tree for the generation)
mog has joined #nixos-aarch64
<gchristensen> oh yaeh
<samueldr> hmm, took a quick peek at the odroid-c2's u-boot, can't try anything blind easily
<samueldr> it needs some intermediary steps which don't look too fun
<samueldr> http://git.denx.de/?p=u-boot.git;a=blob;f=board/amlogic/odroid-c2/README;h=bed48c5728ba9e0200d45773e28a34e949069492;hb=0157013f4a4945bbdb70bb4d98d680e0845fd784#l37
<samueldr> if I'm (un)lucky, the amlogic board I ordered also will need the same stuff
<samueldr> I think it'll be like that
<samueldr> http://git.denx.de/?p=u-boot.git;a=blob;f=board/amlogic/libretech-cc/README;h=d007f58764d4facbb7b1c8cba0882a2c41e8c4ac;hb=0157013f4a4945bbdb70bb4d98d680e0845fd784#l38
<samueldr> similar, looks worse
<samueldr> it's not the board I ordered, but I think it's close
<gchristensen> hrm I can't get the serial console to work.
<samueldr> oh
<samueldr> opensuse, and for the board cousin to the one I ordered, seems to have a custom tool to build u-boot
<gchristensen> there we go... took reseating some things.
<gchristensen> oh I forgot I set this thing to only pxeboot
<samueldr> I think I'll look more often towards opensuse for ARM things
<samueldr> archlinuxarm, too, seems to be using meson-tools https://archlinuxarm.org/packages/aarch64/uboot-odroid-c2-mainline/files/PKGBUILD
<gchristensen> samueldr: is the sd card ext2?
<gchristensen> ext4*
<samueldr> a bit of empty space, one FAT32 partition, one ext4 partition
<samueldr> assuming you're takling sd-image-aarch64
<samueldr> hmm, no, it's the same as the iso, not sure in reality what it is... but I think a squashfs file on a 9660 FS
<gchristensen> I think my u-boot doesn't support uefi
<gchristensen> ah yep U-Boot 2015.01-00075-g73f44ad (Feb 23 2016 - 23:38:22)
<samueldr> yeah
<samueldr> odroid has an old crusty u-boot fork :(
<gchristensen> don't they all
<samueldr> though, I'm looking quickly at making the derivation for you with mainline u-boot
<gchristensen> :O
<samueldr> (and ANYWAYS I'll need 80% of the work for a board)
<gchristensen> does anyone have a u-boot-enabled device to send me the default boot config? :P
<samueldr> huh?
<samueldr> gchristensen what are you looking for?
<gchristensen> I think this device is only setup to ipxe :)
<samueldr> but uh, what are you looking for exactly?
<samueldr> IIRC, our u-boot configs are built into our u-boots, so they can't really be sent
<gchristensen> the bootcmd from `env print`
<samueldr> it's a russian nested doll
<gchristensen> bootargs=root=/dev/mmcblk0p2 rw init=/init rootwait console=ttyS0,115200 hdmimode=720p60hz hdmitx=cecf logo=osd1,loaded,0x3f800000,720p60hz androidboot.hardware=odroidc2 androidboot.serialno=${fbt_id#} androidboot.selinux=disabled
<gchristensen> bootcmd=tftpboot ipxe/undionly.kpxe
<gchristensen> is what I have
<samueldr> definitely not what upstream u-boot (thus ours) have
<samueldr> http://git.denx.de/?p=u-boot.git;a=blob;f=include/config_distro_bootcmd.h;h=5838eb347798e430e809e870c8112b985b151a8f;hb=HEAD
<samueldr> ours is derived from this mess of expansions
<samueldr> and most probably won't work with your downstream u-boot from o-droid :/
<gchristensen> :)
<samueldr> that mess will, among other features, make uefi boot work
<gchristensen> oh cool
<samueldr> AFAIUI it's all that's required
<samueldr> not ready to upstream because of the dirt around u-boot, and untested
<samueldr> it needs to be built on an aarch64 machine
* samueldr wonders how one could do that
<gchristensen> nix-build . -A ubootOdroidC2 --argstr system aarch64-linux... :
<gchristensen> )
<samueldr> oh
<samueldr> you might want instructions on "flashing"
<gchristensen> https://wiki.odroid.com/odroid-c2/software/building_u-boot so I found this, and $ ./sd_fusing.sh <device/path/of/your/card> and got the script
<gchristensen> getting it from the microsd to the mmc is the next challenge I think?
<samueldr> most boards will prefer SD over MMC I think when both have the bootloader
<gchristensen> cool
<clever> i noticed the uboot for the banana pi already has an spl on it, so less fusing is required
<gchristensen> is u-boot.gxbb equiv to u-boot.bin ?
<samueldr> most allwinners are built as one file here
<samueldr> gchristensen: no idea, I think so
<samueldr> the archlinuxarm sd_fusing puts it at 512*97
<samueldr> (I based the built off of archlinuxarm's
<samueldr> it's kinda annoying how hardkernel's seems to want to be in the mbr
<samueldr> can't do gpt I guess
<samueldr> (bs=1 count=442 is mbr, right?)
<clever> the MBR and x86 bootstub share the first 512 bytes
<clever> GPT also has a protective MBR in the first 512
<samueldr> hm, maybe there's hope
<samueldr> it would be nice to have a really universal image, UEFI + u-boot "fusable"
<clever> depending on what tool you use to partition the disk, it can either duplicate the tables to the protective mbr, or just create a single dummy partition, claiming the disk is 100% in use
<samueldr> with EBBR or u-boot on SPI it would be really "just dd it"
<gchristensen> U-Boot 2018.09 (Sep 10 2018 - 21:46:42 +0000) odroid-c2
<gchristensen> woot
<clever> nice
<clever> at a glance, i think uboot already has bananapi support, along with a pre-fused spl binary
<samueldr> gchristensen: yay!
<samueldr> +1 for sane upstream software (u-boot)
<gchristensen> only does that when I don't have the mmc installed
<gchristensen> so need to figure out how to copy it over :)
<samueldr> if you can boot *any distro*, you probably can dd if=/dev/zero of=/dev/mmcTHERIGHTONE bs=422 count=1
<clever> in the case of the banana pi, reading its defconfig...
<clever> 12 CONFIG_SPL_I2C_SUPPORT=y
<clever> 3 CONFIG_SPL=y
<clever> 14 # CONFIG_SPL_DOS_PARTITION is not set
<clever> 15 # CONFIG_SPL_EFI_PARTITION is not set
<clever> and others
<gchristensen> u-boot's movi command might do the trick
<clever> this would imply that the SPL does support mbr and efi, but the bananapi disables those portions in its defconfig
<clever> odroid_defconfig lacks CONFIG_SPL's though
<samueldr> yeah, amlogic boards don't have the same setup as allwinner boards for that thing
<samueldr> meson-tools is an open source re-implementation of the amlogic secret sauce
<samueldr> it uses blobby bits to make the thing work, AFAIUI
<samueldr> (with the words I used, you know I'm an expert ;))
<clever> after glancing at the odroid wiki gchristensen linked above, it looks like the bootloader is in the tarball with sd_fusing.sh
<clever> and reading the sd_fusing.sh, i can see it writes 442 bytes to the MBR (just enough to leave the tables intact i would guess)
<clever> then it writes 512 bytes to the sector immediately after it
<clever> and then the uboot to an offset of 97
<clever> if you decompile the 442 byte blob, you can likely change the offset for the 2nd part
<clever> and then also change the offset for the uboot as well
<samueldr> depends whether it's signed or not
<clever> just need to find a copy of bl1.bin.hardkernel
<samueldr> from memory, their exynos boards have a signed bootloader
<gchristensen> I don't mind its locations, but copying it to this mmc is being tricky
<samueldr> you didn't get a usb reader with it?
<clever> a: the signature would likely take up half of the space on the 442 byte blob, and b: if its that smart, it could support an FS
<samueldr> my odroid-c1's emmc came with a usb reader I think
<samueldr> or if it didn't I added it to the cart :/
<samueldr> clever: possibly the signature is after the 442 bytes
<samueldr> the blob is bigger, but yeah, might not be
<gchristensen> samueldr: mmm don't think so
<samueldr> :(
<gchristensen> I was thinking maybe I can load it in to memory and write it out again
<samueldr> there's a random individual online that says it's fine to plug the emmc while u-boot from sd is loaded
<samueldr> I wouldn't know for sure though
<clever> samueldr: where can i find a copy of bl1.bin.hardkernel ?
<clever> ah, i notice its 49kb
<clever> which is exactly where the other dd drops the uboot
<clever> and the reason for the 3 dd commands rather then 1, is to make a hole for the original MBR tables
<samueldr> yeah
<clever> grub-install handles that kind of mess for you
<clever> oh, grub-install also handles making parts of it relocatable
<clever> there are special offsets in the initial 442 byte blob, that define where to load the rest from
<clever> if we can identify those offsets, the same could be done to BL1
<gchristensen> ok found a solution to my problem
<gchristensen> mmc dev 1; mmc read ${loadaddr} 0 1000; mmc dev 0; mmc write ${loadaddr} 0 1000
<samueldr> oh, transplanting whatever is on sd into emmc
<samueldr> (up to 1000 bytes)
<clever> related, the compute module for the rpi has a different solution to this
<clever> it can load the bootcode.bin over USB, in a DFU type mode
<clever> and they supply a tool, that will give a bootcode.bin type blob, that turns the rpi into a mass-storage device
<clever> then its just a usb stick
<clever> exposing the emmc
<samueldr> u-boot supports such modes AFAIK, but don't know how to actually make it work
<samueldr> (modders in the nintendo switch community make use of that feature from u-boot)
<clever> ah
<clever> https://github.com/raspberrypi/usbboot the msd folder has the mass storage device
<clever> the rest is about booting over usb
<gchristensen> ok redid it with 100000 bytes samueldr :) should be sufficient
<samueldr> I don't know if the "bl1" would have changed in-between
<samueldr> maybe loadaddr isn't the right start location?
<samueldr> well
<samueldr> that's nice
<samueldr> didn't really realise the efi image would work there lol
<gchristensen> trying a boot now
<samueldr> you'll lose serial if you don't take a serial option
<samueldr> (and maybe the option doesn't set the right device)
<clever> [nix-shell:~/Downloads]$ objdump --target binary -D bl1.bin.hardkernel -m arm | less
<clever> thats enough to dump some of the opcodes
<samueldr> yeah, that's u-boot giving control to the kernel
<clever> and then the kernel is responsible for doing all further prints
<gchristensen> ot it
<clever> ok, the first bne opcode is doing a relative jump, of over 0x5b0000 bytes...
<clever> thats going to fail if the firmware only loaded the first 512 bytes
<gchristensen> ok, enough fun for one day. thank you for playing
<samueldr> since there was a lot of talk since, nbp: https://logs.nix.samueldr.com/nixos-aarch64/2018-11-21#1542762484-1542762063; if you `dd` the iso built from that POC commit to a usb drive, it probably will boot on your fun bit of hardware :)
orivej has quit [Ping timeout: 268 seconds]
lopsided98_ has quit [Ping timeout: 264 seconds]
lopsided98 has joined #nixos-aarch64
orivej has joined #nixos-aarch64
FRidh has joined #nixos-aarch64
<nbp> samueldr: Thanks, I will cherry-pick / checkout these changes and test them later today.
LnL has quit [Max SendQ exceeded]
LnL has joined #nixos-aarch64
<exarkun22> Hi. I'm not ussing AArch64 but I am using armv7l (on pi 3 b+) and having problems with the extlinux bootloader ... anyone mind if I ask some questions about why I'm failing to upgrade my kernel here?
arianvp has quit [Ping timeout: 244 seconds]
<exarkun22> oh hey look there's an unmounted bootable fat partition with an old kernel on it :/
arianvp has joined #nixos-aarch64
<exarkun22> cool, took bootable flag off that, put it on the good partition, all is well now
<exarkun22> (well, except that I still have no gpio access even after switching kernels...)
zimbatm has left #nixos-aarch64 [#nixos-aarch64]
orivej has quit [Ping timeout: 252 seconds]
<nbp> samueldr: Damn, the kernel compilation is taking a really long time :/
<nbp> samueldr: not sure I would be able to test it today.
<samueldr> not saying you should install from it, but you could verify it boots using the pre-built image, it most likely shouldn't cause any issues with regards to trust, hopefully
<nbp> samueldr: I will try… in the mean time there is still a bunch of tests running in the background, so no hurry to test yet.
<samueldr> I'm mostly curious, it'd be nice to add aarch64 uefi support that way with the sd-image-aarch64 image... which might be possible
<nbp> samueldr: If the compilation does not finish soon, I will tell you on Monday.
<gchristensen> nbp: I could set you up withaccess to the (probably much faster) builder
<nbp> gchristensen: It might just be that the box is doing something else at the same time.
<nbp> gchristensen: and I am not in a hurry either ;)
<gchristensen> are you using aarch64.nixos.community?
<nbp> gchristensen: I downloaded samueldr iso file, while building it.
<gchristensen> ah ok
<nbp> gchristensen: I will tests later, or Monday.
<nbp> using the linuxPackages_latest is not helping me, as I rebuilds the kernel.
<nbp> which makes me wonder whether the CONFIG_COMPAT flag is set and if I would have to recompile it anyway :/
<gchristensen> well, if you would like to have access to a powerful aarch64 builder to help you, send a PR https://github.com/nix-community/aarch64-build-box (note: the cautions in that document apply to the iso from samueldr, afair)
<nbp> gchristensen: Noted.
FRidh has quit [Quit: Konversation terminated!]
<samueldr> _latest is not strictly needed
<samueldr> it was for the raspberry pi to work, but it's not related to UEFI booting, just 4.14 being a bit long in the tooth
<nbp> samueldr: oh, good to know, I will try without it then.
andi- has quit [Ping timeout: 250 seconds]
<samueldr> do note that it *could* be needed for better support for stuff, but that's really depending on your board, no idea for its support on mainline
<samueldr> (e.g. the raspberry pi works without latest, but the video output doesn't once modesetting turns on on 4.14)
<nbp> oh great …’it recompiles again :/
andi- has joined #nixos-aarch64
<nbp> samueldr: With your iso, grubs works :)
<samueldr> that confirms that at least your UEFI is standard and working well
<samueldr> wondering though "grub works", how does nixos fare?
<nbp> samueldr: currently it is waiting on "A start job is running for udev Wai…e Initialization"
<samueldr> hm, so at least the kernel does boot, everything after is uh, out of my knowledge area :)
<nbp> samueldr: ok, it booted.
<nbp> samueldr: and give me a login prompt \o/
* nbp now I will have to find a good name for this box.
<nbp> samueldr: Thanks :)
<samueldr> you're welcome, have fun :)
<samueldr> they I guess this means we need to figure out a good way to make the image uefi-bootable :D
<samueldr> (while still working for those u-boot slammed right into the initial sectors)
<bennofs[m]> does the current sd-image have support for headless setup? is there some way to easily execute an initial setup script on first boot=
<samueldr> the sd-image acts pretty much like the x86_64 installer iso
<samueldr> in fact, it loads common a configuration
<samueldr> so, no, but if it was contributed to the x86_64 iso, it should be possible to make it work on aarch64 also
<nbp> :thinking: Technically, you could make an iso / sd_image which does the intallation for you.
<nbp> I guess the only trick would be to not loop, and automagically reboot on the newly created system.
<samueldr> right now, the main difference between sd-image and the iso installer image is that one does a squashfs on iso9660 + EFI support files and the other makes an ext4 partition + FAT32 support partition for raspberry pi, both possibly could be combined
<bennofs[m]> yeah but it's harder to do so on ARM if you don't have any other ARM machines lying around :)
<bennofs[m]> where can I find the newest aarch64 image on hydra? seems nixos/trunk-combined doesn't have the aarch64 image anymore and the 18.09 release branch never had a successful build of sd_image for aarch64
<samueldr> AFAIK there's no "newest" build available yet :/
<samueldr> I still bootstrap from the image found here :/ https://www.cs.helsinki.fi/u/tmtynkky/nixos-arm/installer/
<samueldr> (which is now a bit over two months old)
orivej has joined #nixos-aarch64
<gchristensen> nbp: there are at least a few implementations of that around, notably clever's and the one used for Packet.net's provisioning process :)
jabranham has joined #nixos-aarch64
<andi-> probabyl fits best here: While crosscompiling from x86_64-linux -> raspberryPi (not the aarch64 version, armv6 IIRC) I need a 32bit x86 compiler for luajit to build some tools it runs during compilation.. It seems like something like `buildPackages.pkgsi686Linux.stdenv.cc` will then be the cross-compiler for x86-64 -> armv6 again. I can not wrap around that issue.. Just adding `-m32` to the HOST_CFLAGS
<andi-> doesn't do the trick since then glibc would need the 32bit stubs which it doesn't have...
<bennofs[m]> I think we have a gcc-multilib? But that doesn't solve the general issue of course
<samueldr> andi-: you may want to ask on the discourse AFAIK the ones holding deep cross-compilation knowledge are more likely to read and reply there :/
<andi-> ok, I just saw that there are a few cases were we just create empty stub files.. I'll play around with that first :/
<andi-> there is the multilib stdenv... Yields same result.. seems like I really would need to gain access to `nixpkgsFun` to get another combination of *platform :/
<clever> andi-: note, that only the oldest rpi's are armv6, the middle generation are armv7, and the newest are armv6+armv7+aarch64
<andi-> I have an RPi1 compatible FPGA board so that is correct
<clever> ah
<andi-> it is just the closest match we have to that
<andi-> Compiling my little application on that board takes around 50 Minutes... I was hoping crosscompiling would reduce that a bit
<clever> ah
<clever> andi-: i'm thinking that the luajit stuff may be broken for cross-compiles
<clever> if its needing a 32bit x86->32bit arm, then its not correctly changing the bit size when cross-compiling
<andi-> That is why I wanted the 32bit compiler
<andi-> since with just -m32 it didn't properly link anymore and all the libs are not there that it tries to link against
<clever> pkgsi686Linux gives you a copy of nixpkgs, that is purely 32bit
<clever> not multilib, but pure 32
<andi-> let me pastebin my current WIP..
<andi-> the trace puts out: /nix/store/kwckpwrx8a15rdcda50wgc16sywrxn31-armv6l-unknown-linux-gnueabihf-stage-final-gcc-debug-wrapper-7.3.0
<andi-> which is an x86_64-linux based compiler for armv6l...
<andi-> and not at alll what I'd expect..
<clever> andi-: what happens if you just use pkgsi686Linux.pkgsCross ?
<andi-> (if you remove the -m32 attributes that version even starts compiling stuff but can't eecute it due to platform mismatch)
<andi-> clever: can try
<clever> that would likely use an x86-32 -> armv6 compiler
<clever> and never get 64bit stuff mixed in
<andi-> that looks good
<andi-> not exactly what I was aiming for but works
<andi-> now it just needs a working strip ~.~