<wavirc22>
Is there a public hydra jobset that does cross compilation from x86_64 to aarch64? I am unable to build glibc and was hoping to compare it to a failing public jobset.
<samueldr>
AFAIK no, no public jobset
<samueldr>
though glibc failing to build is a known issue and fixed in staging
<clever>
samueldr: ive been making slow progress on the rpi...
<clever>
+ mountFS /dev/sda1 / defaults auto
<clever>
+ local 'optionsFiltered=defaults,'
<clever>
332 local optionsFiltered="$(IFS=,; for i in $options; do if [ "${i:0:2}" != "x-" ]; then echo -n $i,; fi; done)"
<clever>
its now hanging here, and dropbear implodes if i try to ssh in
<samueldr>
so weird
<clever>
very
<clever>
i thought the cross-compile was to blame, but a native compile had the exact same faults
<rajivr___>
@thefloweringash Thanks for the pointer
Acou_Bass has joined #nixos-aarch64
<rajivr___>
thefloweringash: Thanks again. :-) It looks like NixOS supports both `defconfig` builds with `buildLinux` and `.config` with `linuxManualConfig`.
<colemickens>
while I understand the desire to remove the rpi4 image entirely, I'm more confused the more i read this comment about hte RPI4 pull request
<colemickens>
What is an "installer image". To me, the rpi4 image is an "installed" image, and the rpi4-sd-card module *is* included as part of current-system on the running SD card image.
<colemickens>
I guess having any sd card image with hte installer moniker is confusing to me, it seems like a mismatch to what an installer iso means.
<colemickens>
And would agree that it would all be less confusing if it were in nixos-hardware or nixos-generators or something.
<samueldr>
yeah, I understand how weird it can be, but really, it's the installer *profile*, we don't have any kind of profile for use by end-users in nixpkgs AFAIK
<samueldr>
and in the end, it's simply due to a limitation of the platform, that you are running the installer from the media you "install to" (rebuild into)
<samueldr>
there is no built-in way for most raspberry pi to boot an installer media via USB, then install to SD
<samueldr>
(and other ARM boards have similar limitations too)
<colemickens>
I'm not even sure that would be desireable even if it were possible, imo.
<samueldr>
it's a lucky thing that the way nixos works allows us to have mostly unrelated systems, the first being the installer profile, the latter being the running system, in the same "lineage" of generations
<colemickens>
So if I boot the "installer" rpi4 image and do a nixos-install in-place, does it include enough rpi4 bits that the next generation works?
<samueldr>
I don't know if nixos-generate-config will generate the right bits
<colemickens>
I guess I never actually ran nixos-install, I just immediately wrote my own configuration.nix and my own rpi4-without-install-profile module so I didn't actually go the "prescribed" route of doing the install-in-place.
<samueldr>
oh right, that would be via a rebuild anyway, not nixos-install, but same idea
<colemickens>
anyway, I think it it had just been put in nixos-hardware from the get-go that the right sort of expectations would be in place. I guess I could go ahead and open a PR doing that to start setting a precedent going forward?
<samueldr>
and I think we need to have more profiles for boards in a repo, likely nixos-hardware, rather than having some vague pointers on the wiki
* colemickens
nods
* samueldr
hinks
<samueldr>
think*
<samueldr>
we probably should deprecate and sunset the "named" architectures
<samueldr>
especially those that I think are not really well used and tested
<samueldr>
like sheevaplug
<samueldr>
(I think)
<samueldr>
and maybe rename "raspberry pi" to armv6 where it makes sense, with appropriate aliasing
t184256 has left #nixos-aarch64 ["Error from remote client"]
t184256 has joined #nixos-aarch64
<samueldr>
the idea would be to reduce confusion in how nixpkgs "supports boards"
<samueldr>
not sure if that makes sense
<colemickens>
sorta. I still have clarifying questions, but not sure it makes sense to bug you with them unless it's something I think I can actually help out with.
<colemickens>
mostly around what role the sd-card image should play, if any, in an ideal world in your view
<samueldr>
what do you mean by "the sd-card"?
<samueldr>
in an ideal world view the sd card wouldn't exist and the firmware would boot the uefi iso on aarch64, like it can on boards that support EBBR :)
<colemickens>
Right, that's what I expected, just wanted to hear it explicitly I think. I agree that ideally the sd-card image goes away.
<samueldr>
reality makes it we'll have the *generic* SD image, where the mainline kernel is used, and is a bring-your-own-firmware affair
<colemickens>
I think my ideal is a nixos-hardware sd-card image so I can forego "installing" but given how it all works its almost semantics at that point
<samueldr>
yeah
<samueldr>
there's also the way nixos works with native/cross builds
<samueldr>
you can't easily bootstrap yourself a custom nixos image with your system pre-installed if you don't already have access to that platform
<samueldr>
that's where having the sd image really helps
* colemickens
nods
* colemickens
totally isn't looking around for the rpi4 image he built on his last packet machine
<samueldr>
it's also a reason I pushed through and figured out how to cross-compile a system image, while it doesn't solve the native build issue, you can generally bootstrap yourself at that point
<samueldr>
see the pinebook pro example, where it could be your first aarch64 board, no mainline support, so no generic sd image
<samueldr>
with cross compilation working, you can make yourself an image, boot from it, rebuild with an annoying long native kernel build, then you have your native system going
<colemickens>
So you're truly cross-compiling the initial base image even, no qemu?
<samueldr>
entirely true cross-compilation
<colemickens>
I have sort of avoided some learning in this area by booting packet machines.
<samueldr>
it's more about making it accessible to others, I could easily have built it in other ways :)
<colemickens>
samueldr: so in that pursuit, I should probably go read the wiki then?
<samueldr>
there's some outdated info, and no in-depth info about the whys and hows of things yet
<samueldr>
you probably should, but not sure how helpful it will be in the end
<samueldr>
especially thinking about the "FIRMWARE" partition thing that changed in 19.09 that has not been reviewed for the wiki
<thefloweringash>
ooh, did someone say sheevaplug?
<samueldr>
I knew someone had something to say about that
<samueldr>
thefloweringash: s/sheevaplug/armv5whateverflavour/g in nixpkgs, with proper aliasing seems appropriate?
<thefloweringash>
absolutely!
<samueldr>
I haven't actually verified if it's how I think
<thefloweringash>
the sheevaplug platform in nixpkgs has weird kernel config that prevents the firewall from starting, iirc
<thefloweringash>
the only thing that I had to add was `SERIAL_OF_PLATFORM`, otherwise it seems to panic on boot. I'm guessing nixos doesn't support booting without some kind of serial console
<samueldr>
might be udev
<thefloweringash>
(I went down the wrong path of configuring the lower level debug printer before realizing I was missing regular serial)
<samueldr>
I had troubles on one device with a... maybe broken... serial console when it was set as a console= param on the kernel cli
<samueldr>
(broken as in wrongly configured on the cellphone)
t184256 has left #nixos-aarch64 ["Disconnected: Replaced by new connection"]
t184256 has joined #nixos-aarch64
orivej has quit [Ping timeout: 265 seconds]
goibhniu has joined #nixos-aarch64
<goibhniu>
Perhaps the RPi is as good as it gets?
<goibhniu>
Hi, I'd like to set up a few projects (sensors, multimedia) on ARM boards and I'm wondering which ones have the best support. I've had some success with the RPi 3B but it seems very buggy e.g. bluetooth doesn't always work, and I can't get the analog audio output to work. I see that some of the Pine boards have mainline kernel support, can I expect them to work better or is there a particular board you'd recommend?
<srk>
hey, sometimes you need to tweak config.txt or device trees to get some features working
<goibhniu>
Thanks srk, I wonder if there's a board without such issues.
<srk>
hardly :) linux is catching up quite slowly when it comes to arm boards
<goibhniu>
Bluetooth works 3/4 boots on the current default kernel, and doesn't seem to work on the latest kernel, so I suspect support is always going to be a pain.
<srk>
it depends on the features you need, I mostly run headless so I don't really care about all the features
<goibhniu>
ah, so do you think that the Pine boards claiming to have mainline support probably doesn't indicate they'll be more reliable?
<srk>
my imx6 based novena can't do display nor audio with mainline, its like 4y old
<srk>
well rpis also have mainline support
<srk>
but does it mean 100% peripheral support? :) no
<srk>
what's your use-case btw?
<goibhniu>
ahh, fantastic, thanks! that clears everything up
<srk>
mainline support could mean anything from 'it boots' to 100% supported :)
<goibhniu>
well, I have a few projects in mind, so I was sceptical that I can depend on the RPi
<srk>
major PITA with RPis are the SD cards
<goibhniu>
The first project I tried was to make a bluetooth audio receiver, and neither bluetooth nor audio worked well :D
<srk>
:D
<srk>
I do have PHAT DAC lying around which I wanted to use with pi0
<srk>
now I'm going to do it with stm32 instead
<goibhniu>
interesting
<srk>
bluetooth audio is bad quality anyway, you could also do pulseaudio over TCP
<gchristensen>
it is?
<srk>
it is, due to codecs and compression
<srk>
well, it depends, afaik good quality codecs come with licensing fees
<clever>
if you use the hands free profile (mic + speaker), it is forced to crap quality and single-channel audio (in both directions)
<clever>
if you set it to the other profile, the mic is disabled
<srk>
and it's still compression so :) not sure about actual bandwidth of the BLE4 tho
<clever>
BLE is generally much lower bandwidth i think
<srk>
oh
<clever>
its meant to be very low power usage
<srk>
hm, I can build gcc 9.2 if I disable checks for libuv and gmp /o\
<srk>
but that sucks cause I can't use the outputs for cache
<srk>
also I need to fix three too large integeres in nixpgks to be able to eval
<srk>
like
<srk>
- maxHandle = 9223372036854775806;
<srk>
+ maxHandle = 2147483644;
<srk>
+ (assertRange "FwMark" 1 294967294)
<srk>
- (assertRange "FwMark" 1 4294967295)
<srk>
is 32bit considered obsolete? :D
<gchristensen>
you use orangefs?
<srk>
not at all but it's still loaded
<gchristensen>
oh
<srk>
maybe it's in the default modules list
<gchristensen>
all modules are parsed every time
<srk>
looks like a nix bug
<gchristensen>
in what way
<gchristensen>
?
<srk>
as it's happening only on armv7l
<gchristensen>
but not on i686?
<srk>
don't have any i686 to test
<gchristensen>
I'm not sure it is a Nix bug exactly, unless the expectation is Nix on a 32-only system should be able to manipulate 64 bit numbers
<gchristensen>
but there is an interesting qustion of "what now?"
<srk>
indeed
<srk>
like the maxHandle could be a string as well, but not the assertRange number
<gchristensen>
maxandle has math done on it
<srk>
that's weird since that int is within 2**32
<srk>
ah, ok
<srk>
maybe it excepts signed int there?
<gchristensen>
not sure
<srk>
not strongly typed enough
<gchristensen>
okay
<srk>
maybe nix can work with larger-than-platform ints but does bound checking incorrectly?
<srk>
or you need to use something like GMP for that?
<gchristensen>
I'm expecting that one day when samueldr has a big bucket of time fall out of the sky he'll send me a PR so we can add some armv7l builds ot hydra
<thefloweringash>
srk: I'm experimentally maintaining an armv7l binary cache for the three main channels on cachix
<srk>
gchristensen: that means you have some armv7 hw for hydra already?
<thefloweringash>
an armv7l vm on a bigger uglier aarch64
<gchristensen>
an armv7l vm on a bigger uglier aarch64 :)
<srk>
pretty cool, I've had troubles building nix due to libuv test failures
<srk>
haha
<srk>
lets see if I can build cachix :D
<gchristensen>
you don't need to build cachix to use it
<gchristensen>
to read anyway
<srk>
ah lol, that's haskell, no way :D
<srk>
I can do it on my x86 machine and just copy the output I guess
<thefloweringash>
cachix and ghc are in the list of things attempted, but it bails out early with an error about "Your linker is affected by binutils #16177"
<gchristensen>
thefloweringash: if you write the good stuff, send me a link to your branch? :)
<gchristensen>
very ideally, it would cross-build the VM it boots
<thefloweringash>
doesn't have monitoring, and the port forwarding is using `hostfwd` instead of a tap
<thefloweringash>
^ it does
<gchristensen>
hostfwd sounds like an improvement you could make to my code :P
<thefloweringash>
would have to check if the hostfwd can bind to the wg only interface
<gchristensen>
no need for the wg parts
<gchristensen>
the builders this runs on won't use wg
<thefloweringash>
I'm not seeing an obvious point in the nixos-org-configurations repo to drop my module in and start hacking
<thefloweringash>
or is that in the grahamc/packet-nix-builder repo?
<gchristensen>
well it is a bit awkward
<gchristensen>
but yeah, I'd basically want to extend packet-ni-xbuilder to do the same thing that macs/ directory does in nixos-org-configurations
<thefloweringash>
alright, thanks for the pointers, I'll see if I can find some time this weekend
<gchristensen>
thanks thefloweringash!
<gchristensen>
thefloweringash: I'll take it in basically whatever format you can give it to me
<gchristensen>
I just don't have time for the leg work :( :( :(
orivej has joined #nixos-aarch64
orivej has quit [Ping timeout: 265 seconds]
<samueldr>
thefloweringash: not much progress yet
<samueldr>
when I last touched it, armv7l was verified as booting, I was going to look at your "alex" build vm thing to understand it and then work on it
<samueldr>
in a nutshell: nothing more than you have done
<thefloweringash>
Fair enough
lovesegfault has joined #nixos-aarch64
<srk>
looks like I can eval sdImage for armv7l using nixpkgs.crossSystem iff I disable gpsd :D
<samueldr>
verified to build an sd image 5 days ago on staging
<samueldr>
should also build 19.09
<srk>
cool!
<srk>
I'm using like 5 days old nixpkgs
<samueldr>
stable will not cross-compile, glibc was broken for cross
<samueldr>
staging has the fix
<srk>
# `xterm` is being included even though this is GUI-less.
<srk>
looks like this is quite common, my headless setup is pulling a lot of X libs as well :(
<srk>
need to examine the deps when it's done building
<samueldr>
that one I think is fixed in unstable
<samueldr>
oh, I meant "unstable will not cross-compile", not stable, stable will
<srk>
ah, no problem, will try with both, need to gc first tho
<samueldr>
mostly want to save you some time
<samueldr>
it's annoying, waiting for a build only for it to fail later :)
<srk>
indeed :) especially on armv7! :D
ryantrinkle has quit [Ping timeout: 268 seconds]
v0|d has quit [Read error: Connection reset by peer]
ryantrinkle has joined #nixos-aarch64
cptchaos83 has quit [Ping timeout: 258 seconds]
cptchaos83 has joined #nixos-aarch64
t184256 has left #nixos-aarch64 ["Error from remote client"]
t184256 has joined #nixos-aarch64
sigtrm has quit [Read error: Connection reset by peer]
sigtrm has joined #nixos-aarch64
wavirc22 has quit [Ping timeout: 268 seconds]
wavirc22 has joined #nixos-aarch64
<bdesham>
does anyone know of a tutorial for building an AArch64 (Raspberry Pi) SD card image from an x86_64 Mac?
<bdesham>
I see the "build your own image" section in the "NixOS on ARM" article, but it seems like I'd need QEMU and it's not clear whether that's even possible to get running on Darwin
<wavirc22>
bdesham: I build an SD image inside a linux VM. The SD image is cross compiled on a x86_64 to the aarch64 target.
<bdesham>
wavirc22: ok, thanks. maybe I can use Docker or Vagrant to get Linux running without too much fuss
<samueldr>
clever: how'd you end up getting libgcc_s.so on staging?
<samueldr>
got it, stdenv.cc.cc, but in lib64
orivej has joined #nixos-aarch64
<samueldr>
hm, might not have it in the end
<samueldr>
overriding the build inputs to not require libunwind is okay in my use case, that removes the need for libgcc_S
<flokli>
samueldr: any idea what's preventing staging from being merged?
<samueldr>
not at all, I'm entirely removed from the process of getting staging in master
<samueldr>
for me, it's a black box, PR goes into staging -> [???] -> goes into staging-next -> [???] -> goes into master
<lovesegfault>
samueldr: lol, that's exactly my mental model too