<samueldr> <3 nix, ridiculously trivial to make builds when doing big changes
<samueldr> just building four system images for different systems, using different kernels without having to break a sweat
<gchristensen> :D
<samueldr> :/ I was about to update the pinned revision of the mobile-nixos, lo and behold: it's pinned to channel:nixpkgs-unstable
<samueldr> job's done!
<gchristensen> :)
<samueldr> :( I goofed and lost the exact config I had for u-boot and now having trouble to compile it, even on the same nixpkgs commig + u-boot commit... that's annoying
<samueldr> :/ turns out the exact config I had was the right one, but *somehow* the extraConfig thing in our u-boot build doesn't play well with the specific options
<samueldr> guess I'll need to take a look to see how it could affect the build
efraim has quit [Quit: http://quassel-irc.org - Chat comfortably. Anywhere.]
efraim has joined #nixos-aarch64
<sphalerite> samrose_: you can use nix-store -q --graph to dump the dependency graph of your drv and find out what depends on the jdk
samrose_ has quit [Ping timeout: 250 seconds]
samrose_ has joined #nixos-aarch64
samrose_ is now known as samrose
<samrose> opinions sought: would it be best for me to perform nixos-generate-config on the same SD card, and then edit my new configration.nix, then nixos-rebuild switch on the SD card, then dd this whole thing over to emmc?
<samrose> I did try running nixos-install at the mounted filesystem on emmc, and it completed successfully. I then dd'd the u-boot bin over, rebooted, but u-boot couldn't find the partition on the emmc
<samueldr> was there a gap in front of the first partition?
<samueldr> you might have overwritten parts of the partition
<samueldr> did you set the "bootable" flag on the partition holding the extlinux.conf files?
<samrose> samueldr: so typically nixos-install should be viable on these aarch64 boards it seems you are saying. ok. yes, I think your theory could be right I may have gotten my partitions wrong
<samueldr> there's nothing really "magic" about nixos-install, so yeah it should work
<samrose> I guess the installer knows about partitions (liek which one is bootable)
<samrose> ?
<samueldr> but since there's no magic, care has to be taken in cases like allwinner devices where some data may need to be placed at random offsets
<samueldr> the installer has no knowledge, it only asks nixos-rebuild to install the bootloader
<samueldr> the only special thing it does, iirc, other than setting up the chroot, is to ask for a root password
<samrose> so, when I dd'd u-boot bin over, I sent it over bs=1024 and seek=8
<samueldr> sounds right
<samrose> so that would be the extent of what I can controll it seems yes? as everything else is done arbitrarily by nixos-install?
<samueldr> nixos-rebuild doesn't know how to partition, and in our case, won't know about setting the bootable flag, these are operations you'll have to do
<samueldr> in the case of the sd image, here's where it's done, specifically where the first partition is defined https://github.com/NixOS/nixpkgs/blob/f072cfe1ebff79efaa409258a38646a62c94dbff/nixos/modules/installer/cd-dvd/sd-image.nix#L119
<samueldr> this starts it at 8M in, allowing ample space for bootloaders
<samrose> ah yes understood. So you are saying to do the same thing when I create partitions
<samrose> this probably is where I screwed this up
<samueldr> learning exercise :)
<samrose> definitely is :)
<samrose> samueldr: so I am looking at this https://github.com/NixOS/nixpkgs/blob/master/nixos/modules/installer/tools/nixos-install.sh#L131 and if I am following you correct, then in the case of aarch64 nixos-install script will know which partition I have marked as bootable and load bootloader there, yes?
<samrose> in our case I think it is boot.loader.generic-extlinux-compatible
<samueldr> no/yes, IF nixpkgs has further knowledge to install it, yes, but in the case of u-boot + extlinux, nixkgs has no idea how to install it
<samrose> samueldr: ok, so I just need to make the target partition I create bootable, with a boot flag
<samrose> and then dd the u-boot seek=8
<samueldr> you shouldn't need it
<samueldr> should work with only one, unless you're on a raspberry pi, which wants a FAT32 partition
<samueldr> (or a UEFI device)
<samrose> hmm ok so I need to make a single partition and make it bootable flagged and this is the target for nixos-install
<samrose> ths whole sd-image.nix is a pretty amazing piece of work. learning a lot just readign through it
<tilpner> You can generate a single-partition image with uboot already installed
<samrose> it worked!
<samrose> tilpner: I wondered about that
<samrose> tilpner: any examples of that?
<tilpner> samrose: This is what I use: https://gist.github.com/da93457bc73335ebca58559dd8e03c5e
<tilpner> But the whole approach kind of fails when you want disk encryption
<samrose> til[ner thank you and good to know on theencryption
<samrose> *tilpner
<samueldr> ah, I wondered if you meant "there's already everything to build a single-partition image in nixpkgs" or "it's possible to do it" :)
<samueldr> oh, guestfish is problematic if it was desired to upstream your work, (1) it's not built on hydra, but that's kinda not really a deep issue (2) it's x86_64 only, that's more of an issue
<samueldr> I think it's x86_64 only
<samueldr> I'd need to re-check for aarch64 native
<sphalerite> so firefox isn't in the binary cache because rustc timed out on hydra
<sphalerite> is there a way to stop this from having?
<sphalerite> happening*
<tilpner> samueldr: Might be possible with debugfs anyway
<samueldr> I think we just switched to lkl
<samueldr> IIRC this allows any FS supported by the kernel
<tilpner> Oh!
<tilpner> What do I search for to find out more?
<samueldr> nixos/lib/make-ext4-fs.nix, cptofs, from lkl
<samueldr> (it's merged)
<samueldr> that's the project https://github.com/libos-nuse/lkl-linux
<tilpner> Oh, I found that, but didn't realise it could be used like that
<tilpner> Thanks!
<samueldr> I didn't know about it before the PR
<lopsided98> I've been working trying to figure out how to build guestfish on Hydra and for other platforms
<lopsided98> I have a prototype flexible image builder module that uses it
<lopsided98> It supports other platforms, but upstream doesn't provide prebuilt supermin images for them
<tilpner> lopsided98: samueldr just pointed out that master now uses cptofs from lkl for copying into images. Can that replace your usage of guestfish?
worldofpeace has joined #nixos-aarch64
<lopsided98> Yeah, I didn't know about lkl, it might be interesting to write a guestfish like tool for it
<lopsided98> One of the things I'm trying to avoid is the need to copy an entire ext4 partition into the partitioned image
<lopsided98> I would rather add the files directly to the partitioned image
<tilpner> So you mkfs in-place and guestfish files into that?
<lopsided98> Yes, guestfish starts up a tiny VM and mounts the image within it
<lopsided98> guestfish requires KVM, which can be problematic on ARM devices
<samueldr> lopsided98: the main issue with guestfish+hydra (assuming x86_64) is the size of the VMs used to do the dirty work
<samueldr> and conversely, I'm not sure those VMs exist for aarch64
<lopsided98> samueldr: The VM image has a lot of free space, it is possible to shrink it
<samueldr> yeah, IIRC the download is relatively small, so it makes sense
<samueldr> maybe we can build them ourselves?
<samueldr> guestfish *can* build them from nothing IIRC
<lopsided98> It is normally stored as a sparse file, but that doesn't work with NAR
<samueldr> but yeah, lkl made this a non-issue for my use case, even if (not benchmarked) it would be slower, pragmatically it works
<lopsided98> The build process makes a huge number of FHS assumptions and requires a supermin backend for the package manager
<lopsided98> I was trying to build images in a debian VM (within Nix)
<lopsided98> but it might be easier and much more elegant to write tooling around lkl
<samueldr> wondering if the guestfish images could be custom nix-based vms, while still compatible -- though probably risky compatibility-wise
<lopsided98> If we are using KVM anyway to build images, we might as well use Nix's VM support rather than guestfish
<lopsided98> I'd rather not use KVM
<samueldr> I don't think we're using kvm
<samueldr> so yeah, guestfish or custom tooling are both off the table
<samueldr> (lkl is entirely userspace IIRC)
<lopsided98> lkl sounds promising, but AFAIK there is no tool based on it that can replace guestfish for my use cases
<lopsided98> I might try to write one though
<lopsided98> cptofs does most of what I want, but I also need partitioning and filesystem creation in place in a partition
worldofpeace has quit [Remote host closed the connection]
<gchristensen> from #worksonarm: http://cluster.bitscope.com/
<samueldr> weird site
<samueldr> no mention whether it's a thing for sale
<samueldr> just "look at this"
<samueldr> until you see the footer, where they say "bitscope blade is at the heart"
<samueldr> yeah, "until you see the footer"
<samueldr> but then it's not the 144 one :)
<gchristensen> I think they're "Coming Soon"
<samueldr> the blades look interesting, but I'd like to see something more like a generic "how to design" that kind of infra
<samueldr> e.g. how one should design the power distribution
<samueldr> most of those that do clusters lack details in that department
<gchristensen> yea
<samueldr> but do they tell you how much acrylic and bolts they're using
<gchristensen> step 1: hire an EE
<samueldr> heh