<Raito_Bezarius>
Hi there, I'm trying to build the SD image for RPi 3, but it seems to be taking a super long time to perform the ext4 partition creation (I see it's stuck at cptofs), I'm using qemu to cross compile everything, it seems to work for everything else except the cptofs -t ext4 -i ext4-fs.img nix-path-registration /
<Raito_Bezarius>
I don't know, if it's related, but I have: "Unsupported ioctl: cmd=0xffffffff80200204" and some warning related to /etc/mtab
<lovesegfault>
Raito_Bezarius: If you give me the config I can build it for you in the aarch64 build box
<{^_^}>
#76780 (by RaitoBezarius, 1 minute ago, open): Raspberry Pi Image building hangs on `cptofs`
<Raito_Bezarius>
It has been running for more than 11 hours now
orivej has joined #nixos-aarch64
bennofs has quit [Remote host closed the connection]
bennofs has joined #nixos-aarch64
<grw>
Raito_Bezarius: seems strange, it definitely works on native aarch64. can you check iotop or something to see if anything is actually being transferred?
<Raito_Bezarius>
Sure
<grw>
my experience with qemu-aarch64-static is that everything is REALLY slow, but maybe others have better luck
<Raito_Bezarius>
It seems it's not doing any I/O
<Raito_Bezarius>
using iotop -o
<Raito_Bezarius>
I get it that it could be really slow, but it didn't get out from the syscall, that's super strange that a syscall can take more than 11 hours… (excluding any suspicion of deadlock, or something like this)
<grw>
sorry, didnt read you checked that. yeah, guess it is bug
<Raito_Bezarius>
Yeah, I straced + gdb attached it
<Raito_Bezarius>
At what level could be this a bug?
<Raito_Bezarius>
Nix, qemu or kernel :D?
<Raito_Bezarius>
(I'm using a LTS 4.19 kernel)
<Raito_Bezarius>
But I can give it a try on a 5.X once a certain i915 bug is fixed
<grw>
i dont have enough knowledge in this areas to know really
<grw>
if you just want to build image (and not find root cause of this issue) you could just build natively on rpi with a different distro
<Raito_Bezarius>
Yes, but it's going to be very slow :(
<Raito_Bezarius>
I expect it to be a lot slower than using my x86_64 machine
<grw>
in my experience qemu-aarch64-static was even slower but probably depends on hardware
<Raito_Bezarius>
Worst case, I could try to go for a distributed build on some Packet native AArch64 machine
<grw>
yep. if you dont mind disabling some non-entirely-necessary stuff you can also cross-compile
<Raito_Bezarius>
Ah, that could be an interesting option
<Raito_Bezarius>
Is there some docs on cross compilation?
<thefloweringash>
lopsided98: that fixed ssh, thanks!
zupo has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
zupo has joined #nixos-aarch64
Thra11 has joined #nixos-aarch64
Thra11 has quit [Client Quit]
Thra11 has joined #nixos-aarch64
thelg has joined #nixos-aarch64
thelg has left #nixos-aarch64 [#nixos-aarch64]
zupo has quit [Read error: Connection reset by peer]
zupo has joined #nixos-aarch64
bdju has quit [Read error: Connection reset by peer]
bdju has joined #nixos-aarch64
zupo has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
zupo has joined #nixos-aarch64
zupo has quit [Ping timeout: 258 seconds]
zupo has joined #nixos-aarch64
zupo has quit [Ping timeout: 268 seconds]
zupo has joined #nixos-aarch64
zarel_ has quit [Ping timeout: 260 seconds]
zarel has joined #nixos-aarch64
zupo has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
zupo has joined #nixos-aarch64
zupo has quit [Client Quit]
zupo has joined #nixos-aarch64
zupo has quit [Client Quit]
zupo has joined #nixos-aarch64
zupo_ has joined #nixos-aarch64
zupo has quit [Ping timeout: 268 seconds]
lovesegfault has joined #nixos-aarch64
lovesegfault has quit [Client Quit]
<Raito_Bezarius>
Gave a try to the example grw and got:
<Raito_Bezarius>
error: file 'nixos-config' was not found in the Nix search path (add it using $NIX_PATH or -I), at /nix/var/nix/profiles/per-user/root/channels/nixpkgs/nixos/default.nix:1:60
<Raito_Bezarius>
Unsure how to understand this error
<clever>
Raito_Bezarius: why are spaces present in that string? and nixos-config is missing
<samueldr>
could be non-nixos for the missing nixos-config
<Raito_Bezarius>
it is actually a non-nixos machine
<samueldr>
are you using the cross-system repo?
<samueldr>
if so, how are you building it?
<Raito_Bezarius>
I just copied most of your files and reused the ./build.sh script
<samueldr>
hmm
<Raito_Bezarius>
But it seems like it's running now for no reason…
<Raito_Bezarius>
I just rerun it without changing anything I believe…
<Raito_Bezarius>
earlier, grw mentioned:
<Raito_Bezarius>
15:24 <grw> yep. if you dont mind disabling some non-entirely-necessary stuff you can also cross-compile
<Raito_Bezarius>
what could I miss by only performing a cross-compilation rather than full blown emulation?
<samueldr>
the inputs are different, so the cache cannot be used, and when you end up building on-device (if you do) you end up not re-using the cross-compiled stuff
<samueldr>
(unless you configure it to use a build host that cross-compiles(
<samueldr>
that's the main drawback if we ignore the next one
<samueldr>
the next one is that cross-compilation is not on par with native compilation, so there are things that currently just won't build
<Raito_Bezarius>
Alright, I see
<Raito_Bezarius>
So fortunately, I could use it just as a bootstrapping image?
<samueldr>
that's a good first step yes
<samueldr>
but as you're targeting the raspberry pi 3, you could use the hydra-built image to bootstrap yourself
<samueldr>
unless you need to run armv7l
<Raito_Bezarius>
I don't think I need armv7l, but I need to bootstrap it unattended
<Raito_Bezarius>
Meaning I'd like to give it some WPA passphrase & some SSH key
<Raito_Bezarius>
So that I can continue the process over SSH
<samueldr>
ah, I figure that means you don't have USB/HDMI or Serial
<Raito_Bezarius>
I only have some HDMI monitor, but no keyboards
<Raito_Bezarius>
grw: Alright, I'll integrate them and will tell you, thanks
<Raito_Bezarius>
Meanwhile, I guess no one tried qemu-aarch64 on their system to see if the issue is local to me or across most of qemu-arm-static binaries?
<Raito_Bezarius>
I'm filing a bug on QEMU bug tracker regarding this, because I have no idea on how to debug this further
<samueldr>
I'm thinking that, though not 100% useful, there may be some value in figuring out an unattended boot-and-rebuild nixos thingy
<samueldr>
something that's not arm-specific, where you can place this system, an "impure config file", and a configuration.nix that will be built on boot
<samueldr>
impure config file would be used for things like wireless setup
<grw>
samueldr: would be nice but as you say, not useful if there is no networking before rebuild
<samueldr>
yeah, why it would likely need something impure to prepare
zarel has quit [Ping timeout: 260 seconds]
<grw>
i guess in this case you could just manually add the /etc/wpa_supplicant.conf after making sd image
<samueldr>
/etc/ doesn't exist after dd'ing the image!
<samueldr>
(though it should be fine to add it)
zarel has joined #nixos-aarch64
<Raito_Bezarius>
couldn't there be multiple stages?
<samueldr>
hm?
<Raito_Bezarius>
like a stage 1 config.nix which set up networking then a stage 2 which uses this networking to download packages?
zupo_ has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
<Raito_Bezarius>
(or maybe what I say is redundant with your ideas and I misunderstood)
<samueldr>
maybe, but at that point it might be easier to cheat for setting up the networking, as a rebuild could want to download or use the cache
<Raito_Bezarius>
Makes sense
<samueldr>
while if it's done through impure means, we already have assumptions made
zupo has joined #nixos-aarch64
<samueldr>
I guess it would be better if it could somehow be somwehat compatible with cloud-init and other server-y unattended stuff
<Raito_Bezarius>
there would be great value I believe in this kind of stuff
<Raito_Bezarius>
especially for nixops I guess
<Raito_Bezarius>
sometimes I wished I could just let the target machine build his own stuff rather than using my machine to build it then copy it
<Raito_Bezarius>
(of course, that's possible if NixOS is already set up once)
zupo has quit [Ping timeout: 240 seconds]
<grw>
fine if you have one machine, but having 100 machines build their own config seems wasteful
<Raito_Bezarius>
IMHO, it depends on the how much the set of configs share with each other
<Raito_Bezarius>
Of course, if you have 25 load balancers, 10 web servers, 5 PostgreSQL servers
<Raito_Bezarius>
Yeah I agree with you
<clever>
nixops has support to use the target as its own build machine
<clever>
its used to do x86 builds on darwin
<Raito_Bezarius>
clever: is there some docs for this? I looked for it and I didn't find it