ryantrinkle has quit [Ping timeout: 240 seconds]
ryantrinkle has joined #nixos-aarch64
h0m1 has quit [Ping timeout: 250 seconds]
h0m1 has joined #nixos-aarch64
ryantrinkle has quit [Ping timeout: 240 seconds]
lordcirth_ has quit [Remote host closed the connection]
lovesegfault has joined #nixos-aarch64
<lovesegfault> Can anyone reproduce this failure? https://hydra.nixos.org/build/108162498
<lovesegfault> cc. clever, samueldr, gchristensen
<lovesegfault> it works perfectly fine for me
<samueldr> how big is it?
<samueldr> hydra limits build artifacts to 2.0GiB
<samueldr> the build status is "it worked, but I won't accept it as it's too big"
<samueldr> and don't be fooled!
<samueldr> sd_image is compressed
<samueldr> what's too big is the ext4-fs.img step!
<samueldr> to fix, either add an additional compression/decompression cycle in the build (bleh)
<samueldr> or combine all steps in one derivation (blah)
<lovesegfault> samueldr: Those options make me sad :P
<samueldr> I think the cleaner option is one big derivation, as it won't shuttle bits uselessly through the CPU
<samueldr> but monoliths are not nice :/
<lovesegfault> samueldr: What's the deriv. path?
<samueldr> hm?
<samueldr> it's part of the stuff in <nixpkgs/nixos>
<lovesegfault> sorry, terminology confusion, where's the build?
<samueldr> nixos/modules/installer/cd-dvd/sd-image.nix / nixos/lib/make-ext4-fs.nix
<lovesegfault> looking
<lovesegfault> samueldr: hmmm
<lovesegfault> why don't we make `make-ext4` output a zstd or something
<lovesegfault> or lzo
<lovesegfault> should be enough
<samueldr> [00:37:09] <samueldr> to fix, either add an additional compression/decompression cycle in the build (bleh)
<lovesegfault> is the nix store optimized?
<samueldr> that's what I meant
<lovesegfault> samueldr: Yeah, I mean that the cost of compressing to decompress will be (much) smaller than with bz2
<samueldr> I never specified a specific scheme :)
<samueldr> the goal is to shave bytes off
<lovesegfault> I can write this if I get some help, I'm not 100% sure how to go about it
<lovesegfault> since I can't get SSH to work I'll do this instead :D
<samueldr> though, it might be preferrable to use something that's already a dependency in stdenv / of the sd image
<lovesegfault> true
<samueldr> not that much to do, other than ensuring $out is compressed, and that whatever consumes it decompresses it
<samueldr> though, make sure make-ext4-fs takes an argument like "compress"
<lovesegfault> samueldr: do I change the name to ext4-fs.img.XX?
<samueldr> since I believe it's not exclusively used by the ext4 image
<samueldr> it's a good idea, to be a good citizen
<samueldr> not exclusively used by the sd_image*
<lovesegfault> Also, I asked about optimizing the store that is put in the img
<samueldr> sorry, should be going to bed :/
* lovesegfault greps
<samueldr> I don't think so, and I don't know it could be
<lovesegfault> samueldr: No worries, I'll try to have a PR for you in the am
<samueldr> and it's likely not to have much deduplication that can be done
<lovesegfault> samueldr: got it
<lovesegfault> gchristensen: I'm working on fixing the Aarch64 test
zupo has joined #nixos-aarch64
<lovesegfault> samueldr: Got the compression working, now fixing the consumers
<lovesegfault> Checked with sha that the images are, otherwise, identical
zupo has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
zupo has joined #nixos-aarch64
zupo has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
t184256 has left #nixos-aarch64 ["Disconnected: Replaced by new connection"]
t184256 has joined #nixos-aarch64
<lovesegfault> samueldr: All fixed: https://github.com/NixOS/nixpkgs/pull/75592
<{^_^}> #75592 (by lovesegfault, 52 minutes ago, open): nixos/lib: compress make-ext4-fs with bzip2
<lovesegfault> gnight
zupo has joined #nixos-aarch64
zupo has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
t184256 has left #nixos-aarch64 ["Disconnected: Replaced by new connection"]
t184256 has joined #nixos-aarch64
lovesegfault has quit [Ping timeout: 246 seconds]
orivej has quit [Ping timeout: 265 seconds]
zupo has joined #nixos-aarch64
zupo has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
zupo has joined #nixos-aarch64
zupo has quit [Client Quit]
zupo has joined #nixos-aarch64
zupo has quit [Client Quit]
zupo has joined #nixos-aarch64
zupo has quit [Ping timeout: 276 seconds]
zupo has joined #nixos-aarch64
orivej has joined #nixos-aarch64
kcalvinalvnn has quit [Quit: ZNC 1.7.4 - https://znc.in]
kcalvinalvin has joined #nixos-aarch64
zupo has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
orivej has quit [Ping timeout: 265 seconds]
ryantrinkle has joined #nixos-aarch64
alienpirate5 has quit [Ping timeout: 250 seconds]
bdju has quit [Read error: Connection reset by peer]
bdju has joined #nixos-aarch64
betaboon has quit [Quit: ZNC - https://znc.in]
betaboon has joined #nixos-aarch64
h0m1 has quit [Quit: WeeChat 2.6]
h0m1 has joined #nixos-aarch64
betaboon has quit [Ping timeout: 240 seconds]
betaboon has joined #nixos-aarch64
ryantrinkle has quit [Ping timeout: 265 seconds]
ryantrinkle has joined #nixos-aarch64
ryantrinkle has quit [Ping timeout: 268 seconds]
<clever> 2019-12-13 14:46:55 < felix_> well, vc4 is a dual core arcompact and some vector or simd units
<clever> samueldr: turns out, we have been using the wrong name for those cores the entire time, lol
<samueldr> we?
<samueldr> :)
<clever> samueldr: the entire rpi community, lol
<samueldr> oh crap, that's the core from those that made starfox on SNES!
<clever> ?
<clever> ah
<samueldr> assuming arcompact is really ARC
<samueldr> (which AFAICT it is)
<clever> 2019-12-13 14:38:34 < felix_> clever: my guess for the isa of those blobs would be arcompact. broadcom uses quite a lot of synopsys stuff and iirc at least one of their ddr4 memory controllers has an arc core for memory training. at least i think i read soething like that a few days ago
<clever> samueldr: this guy is saying that synopsys made the arcompact core, and i have also seen synopsys before, in connection to the rpi usb 2.0 controller
<samueldr> yeah, synopsis _is_ those with the ARC IP
<clever> samueldr: also, i think the above "open source" laptop, includes the same ddr4 controller as the rpi4, and they are explaining the blob firmware the ddr4 controller needs
<samueldr> so that sounds about right
<samueldr> though maybe it's _only_ on the DDR4 PHY
<samueldr> >> processor in the Synopsys DDR4 PHY
<clever> the rpi4 has some fishy 21kb blobs, that include both binary data and strings mentioning ddr4 and addressing
<samueldr> so not sure that VC4 runs ARC ISA
<clever> 2019-12-13 15:01:01 < felix_> the memsys0x.bin files don't look like arcompact code; the hello.bin does though
<clever> samueldr: hello.bin was made by the vc4 cross-compiler in nixpkgs
<samueldr> oh
ryantrinkle has joined #nixos-aarch64
<clever> samueldr: downloads as VideoCoreIV-AG100-R%20(5).pdf
<samueldr> hahaha
<samueldr> so you already have it
<clever> yep
<clever> this is only the 3d rendering section of the chip
<clever> pixel/vertex shaders and such
<samueldr> didn't know if it would be useful or not for your use case
<clever> this region of the chip isnt turing complete
<clever> as far as i know, the rpi4 can run 5 different instruction sets, on 3 main core clusters
<clever> 4 core clusters*
<clever> the arm cores can run both arm32 and arm64
<samueldr> third party links link to the ARC product brochure :)
<clever> the dual-core vpu can run vc4 (aka arcompact?)
<clever> the qpu runs shaders (its basically a 192 core cpu)
<clever> and the ddr4 thing appears to have its own firmware?
<clever> samueldr: ah, i just skimmed over the wiki on there earlier
<samueldr> though it's possible that the DDR4 PHY *also* has its own ARC-based cpu
<clever> `2019-12-13 15:01:01 < felix_> the memsys0x.bin files don't look like arcompact code`
<clever> vc4 objdump also spits out garbage for it
<samueldr> at least for the reform computer you linked earlier their DDR4 PHY has one
<clever> samueldr: i suspect thats what the memsys files are, on the rpi4, firmware for the ddr4 phy
<clever> ILLEGAL LOAD/STORE EXCEPTION PC:
<clever> Setting all DACs to .75 VOH meas
<clever> desired_phy_odt_ohm=
<clever> desired_resistance_ohm=
<clever> Dram Timing
<clever> samueldr: this definitely looks like firmware for something
<clever> samueldr: also, large chunks of what is being said in the comments, have since changed
<clever> samueldr: the entire gfx pipeline now has a proper implementation in linux drivers
<samueldr> sure, I linked to it as corroboration for using ARC
<clever> yeah
<clever> > If you port that BCM21553 code back to the RPi, then you'll still be relying on the binary blob to control the non-3D hardware, but the entire OpenGL ES stack (everything from libGLESv2.so and the shader compiler to the register poking) will be on the ARM.
<{^_^}> error: syntax error, unexpected ',', expecting ')', at (string):271:47
<clever> thats the purpose of the race the rpi foundation started a few years ago
<clever> they open-sourced the android drivers for a BCM21553, which lacked the VPU cores
tilpner has quit [Quit: tilpner]
<clever> > I'm not certain about the details, but I'd expect the most likely conflicts are over interrupts (the V3D interrupts need to be directed to the ARM and not the VPU),
<{^_^}> error: syntax error, unexpected ',', expecting ')', at (string):271:34
<clever> samueldr: and thats something i personally was involved in fixing!
<clever> i was able to trivially route the irq to the arm, but it was also still routed to the vpu, causing the vpu firmware to crash
<clever> samueldr: so basically, the superfx chip from the snes, is now driving the entire rpi, lol
<samueldr> yeah
<clever> it sounds like they are basicaly another arm?
<clever> and are licensing the core IP
<samueldr> "another ARM" as in "another ARM limited licensing core IPs" and not "another ARM vendor" right?
<clever> another company, doing the same thing as arm
<samueldr> AFAICT they are not at all ARM
<samueldr> yeah
<clever> samueldr: the wiki page for ARC also mentions it has an optional usb host
<clever> i suspect thats the 2.0 controller in all pi's, which they they exposed to the arm
<clever> samueldr: they are also involved in coverity!
<clever> samueldr: crazy idea....
<clever> samueldr: could a snes emulator, on an rpi, use the vc4 to accelerate the superfx stuff, lol
<samueldr> hypervisor-like, that would be fun
<samueldr> though I'm not sure they actually share the ISA
<samueldr> might be only by lineage that they are related
<samueldr> and not actually technically related
<clever> yeah
zupo has joined #nixos-aarch64
zupo_ has joined #nixos-aarch64
zupo__ has joined #nixos-aarch64
zupo has quit [Ping timeout: 276 seconds]
zupo_ has quit [Ping timeout: 276 seconds]
mcfrank has joined #nixos-aarch64
orivej has joined #nixos-aarch64
lovesegfault has joined #nixos-aarch64
<lovesegfault> samueldr: Around?
<samueldr> lovesegfault: depends
<lovesegfault> samueldr: On what? :P
<samueldr> on the commitment of my being around means :)
<lovesegfault> samueldr: None, just asking if you had/have/will have time to look at my compression PR
betaboon has quit [Quit: ZNC - https://znc.in]
zupo has joined #nixos-aarch64
zupo__ has quit [Read error: Connection reset by peer]
zupo has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
<samueldr> lovesegfault: looked at it
<samueldr> I ased in #nixos-dev about the importance of keeping make-ext4-fs compatible
<lovesegfault> samueldr: Thanks!
<lovesegfault> I'll address those in a second
<samueldr> and really don't mind the optimization clues in the following comment, that's basically a brain dump of what's needed in a refactor here
<samueldr> we want it to build
<lovesegfault> Got it :)
<lovesegfault> I'm tempted to make the ZST changes in here too
<lovesegfault> the difference was day and night
<lovesegfault> took the community box forever to build with bz2
<samueldr> would this bring in new deps in the build closure?
<samueldr> and does zstd cross-compiles?
<samueldr> just started a cross-compile to see
<samueldr> oh, duh, it wouldn't need to cross-compile as it's a nativeBuildInput
<samueldr> but it does anyway
<samueldr> I don't know any reason not to go with zstd directly
<lovesegfault> samueldr: Alright, I'll merge the changes after I fi
<lovesegfault> *fix those comments
<lovesegfault> Woohoo :D
<lovesegfault> samueldr: All fixed!
<lovesegfault> And uses zstd now
<lovesegfault> Are we not allowed to break internal stuff?
<samueldr> I said internal~ish since it's technically usable from the outside
<samueldr> so I'm not sure :/
<lovesegfault> samueldr: But I don't see a way not to break it
<lovesegfault> if I make it toggable I still need to change the output path
<samueldr> hm?
<lovesegfault> to either $out/foo.img or $out/foo.img.zst
<lovesegfault> Or change $out's name depending on compression being true? Can I do that?
<samueldr> name = "ext4-fs.img${optionalString compress ".zstd"}";
<lovesegfault> Hmmm
<samueldr> (assuming something like let inherit (lib) optionalString; in...)
<samueldr> ah, I should have been clearer, the main generated image should still be bzip2
<samueldr> (xz would be fine too)
<samueldr> for maximum compatibility
<samueldr> while zstd is easy to get working everywhere, it's not generally as well-known
<samueldr> and we don't even have the guarantee that the end-user is using nixos!
<lovesegfault> Fair, I'll change that
<lovesegfault> I just want something faster than bz2 :P
<samueldr> keep it as bz2 since it's not needed for fixing the build
<samueldr> and it'll reduce the confusion of yet another change with compression
<lovesegfault> Fair, will be bundled with my change for toggling compression
<samueldr> thanks
<lovesegfault> samueldr: np, thanks for all the guidance :)
<clever> samueldr: figured out why memsys00.bin has the 32bit chunks backwards
<clever> samueldr: the vc4 firmware is treating it as a uint32_t[] when it copies it, rather then treating it as a uint8_t[]
<clever> samueldr: and i suspect its shoving it into a 32bit wide fifo, that stores it to ram on the other end, in the forwards order
<samueldr> oh, I hate endianness or ordering of bytes or whatever this one is
<clever> so, the vc4 and dram controllers have differing endiannesses
<clever> and the act of reading it from ram, passing it thru a 32bit bus, and writing it back to ram, swapps the bytes
<samueldr> everyone knows the right solution to this is having a pointer of 2n-1 bytes, pointing to the middle, and reading from either end of the mirrorred data
ryantrinkle has quit [Ping timeout: 245 seconds]
<lovesegfault> all done!
<{^_^}> #75592 (by lovesegfault, 15 hours ago, open): nixos: compress make-ext4-fs with zstd
<lovesegfault> also added a nixfmt pass
<t184256> nah, the only right way is the unicode way: extend the byte to a variable amount of octets, allow only some octet sequences to be valid bytearrays, introduce several poorly supported control sequences that reverse the direction and pollute the rest of the possible value spectrum with emoji
<clever> samueldr: and a little more inspection reveals, i think both vc4 and arm have the bytes "backwards" in ram
<clever> samueldr: so its actually the ddr4 controller thats the odd one out, for having its bytes "forwards"
<samueldr> :/ -1 on nixfmt especially since I don't think the formatting has been "decided" on still
<samueldr> (though I could be wrong)
<lovesegfault> samueldr: I just don't want to have to think about it, I'll revert it then :)
<samueldr> in theory nixfmt on existing nixfmt'd code is perfectly cromulent, but otherwise I'm (personally) not in it yet
<lovesegfault> samueldr: revertin
<lovesegfault> samueldr: lolwat
<samueldr> web development seemingly is impossible
<samueldr> and I'm 100% sour against all those bad platforms being badly made
<lovesegfault> I thought about doing it in Nix, but I don't like interpolating stuff like that
<samueldr> as a previous web dev, I see the errors and it's 100% avoidable and it's just bad and I'm disliking my tone right now
<lovesegfault> I have never written anything that runs in a browser, so idk :P
<samueldr> oh, and it's just a tip, don't mind me :)
<samueldr> in that case I don't think it needs to be changed
<lovesegfault> Sweet :D
<lovesegfault> gchristensen: See, the community box access is already paying off, one fix at a time
<samueldr> a thing that could have been done is to have, at the top, ${if compressImage then "img=temp.img" else "img=$out"} and refer only to $img (not ./$img)
<samueldr> that way uncompressed operations could be done directly to the store output
<samueldr> and thus, no bash if at the end, only an expansion on compression to compress to $out
<samueldr> but that's just _a_ way to do it, not a needed change
<lovesegfault> samueldr: That _is_ better, let me try that
<gchristensen> woot!
<samueldr> I have a partial refactor of image building done for mobile nixos
<flokli> urgh, went from bootloader code to scala and sbt. that escalated quickly… :facepalm:
<samueldr> so I had some parts re-thought of
<samueldr> flokli ?
<samueldr> that doesn't sound healthy
<flokli> samueldr: spinalhdl for the fpga part
<samueldr> ah
<flokli> > Basically, SBT use online repositories to download and cache your projects dependancies, this cache is located in your home/.ivy2 folder. The way to setup an internet free environnement is to copy this cache from an internet-full environnement where the cache was already filled once, and copy it on your internet less environnement.
<{^_^}> error: syntax error, unexpected ',', expecting ')', at (string):271:10
<flokli> yeah, sure.
<samueldr> hdl and fpga, raises your blood pressure last I heard
<flokli> samueldr: I'd say crappy toolchains do
<samueldr> aw
<samueldr> .....................................................W..WWWWWWWWWWWWWWWWWWWWWWWW.WWWWWWWWWWW..WW.WWW...WW..WWWWWWWerror: XDG_RUNTIME_DIR not set in the environment.
<samueldr> Xerror: XDG_RUNTIME_DIR not set in the environment.
<samueldr> oops
<samueldr> /nix/store/jsj5d138ih9q9avnr1l6w81qmxgrxmpr-mruby-sdl2/test/rect.rb:4: No available video device (SDL2::SDL2Error)
<flokli> bless you :-)
<samueldr> that's the actual error
<ToxicFrog> Hello, samueldr's cat
<samueldr> *that* is why my build fails :)
<samueldr> could we add a video card to the nix sandbox?
<flokli> samueldr: lol. why do you need a video card to build things?
<samueldr> that's the test suite :)
<samueldr> that's a joke, the solution is to actually configure SDL2 to not output graphically I think
<flokli> :-)
<flokli> I was just gonna say…
<lovesegfault> samueldr: testing changes
<samueldr> working on a tooling for boot selection for mobile-nixos
<lovesegfault> samueldr: Working, pushed
<samueldr> lovesegfault: I had left a review comment too :)
<samueldr> (trivial)
<lovesegfault> samueldr: Yep, fixed and pushed :D
* lovesegfault blows smoke away from commit pistols
<samueldr> my bet with SDL2 is that it will be able to render to KMS/DRM for those shiny new snapdragon where I couldn't get a framebuffer going quickly
<samueldr> in addition to framebuffer
<flokli> samueldr: when wayland in initramfs?
<flokli> :-D
<lovesegfault> flokli: lol
<samueldr> if all fits in 8MiB
<lovesegfault> I'm just excited for the systemd initramfs
<samueldr> (~20MiB more likely though)
<flokli> lovesegfault: yes please :-)
<samueldr> I have a requirement where stage-1 needs to fit in 8MiB and be useful, though it can lack features
<clever> samueldr: oh, i recently became familiar with the rpi hardware scaler, its responsible for hardware composition
<clever> samueldr: basically, you write a series of messages to special ram, that specify things like pixel format, physical address of bitmap, xywh coords to copy from, and xywh to paste into the framebuffer at
<samueldr> having DRM/KMS is also helpful for dogfooding the Mobile NixOS stage-1 (and stage-0?) features on my laptop/desktop one day
<clever> samueldr: and the hardware will then fetch lines of image data, scale, alpha blend, and stack it all up, then shove it into the hdmi or lcd fifo
<clever> so with that knowledge, you can now blast bitmaps out the rpi display, and also have the option of blasting multiple bitmaps
<clever> i think android typically did 3, status, app, and home/menubar, at one time
<clever> x11 can use it to render a mouse pointer
<clever> games could abuse it for sprites, of any size
<clever> with non-int scaling factors
<flokli> neat