<samueldr> clever: which kernel?
<samueldr> rpi-5.4.y?
<clever> and the gpio expander is a child of that, which is why it knows to use the firmware
<clever> comparing https://gist.github.com/samueldr/a2930d853ba1d6b98dd1c47facded2f9 and a copy of rpi-4.19.y i have locally
<samueldr> I'm curious how rpi-4.9.y compares in related parts
<samueldr> great, you do too :)
<clever> but ive also given filenames
<samueldr> oh, 4.19... that's 10 more
<samueldr> (I was just off)
<clever> 4.9 is what ive been using on my pi3 open firmware stuff
<clever> 4.19*
<clever> now your doing it to me :P
<samueldr> yeah, learned recently that they're moving to 5.4... in fact I think it's done
<clever> ive seen a lot of threads on the forum about 5.4 breaking things
<samueldr> that's kind of expected
<samueldr> when you don't track mainline, bumping the kernel version is bound to bring breakage!
<clever> i suspect that they are rebasing their changes onto mainline each time
<clever> and having to fix all the conflicts
<clever> and then fixing the fallout from the upstream changes
<clever> phandle = <0x06>;
<clever> mboxes = <0x1c>;
<clever> mailbox@7e00b880 {
<clever> reg = <0x7e00b880 0x40>;
<clever> phandle = <0x1c>;
<clever> so the handle in the mboxes field, points to that mailbox entry
<samueldr> clever: forgot to gist the dmesg actually https://gist.github.com/23f78e40c48cc9702f516eb5cf6d6ed5
<samueldr> in case it helps
<clever> 284 #define ARM_3_MAIL0_WRT HW_REGISTER_RW(ARM_SBM_OWN3+0x80) // .. 0x8C (4 locations)
<clever> yeah, that looks like the mailbox
<clever> and thats the stuff you can do over it
<clever> tag 1 is get firmware revision, it returns a 32bit int
<clever> tag 3 is undocumented, as usual :P
<clever> 0x00030046 get throttled
<clever> 0x00000003 get firmware hash
<clever> samueldr: i'm thinking you want to turn on the debug in the mailbox
<samueldr> totally makes sense with what I observed
<samueldr> how? and what?
<clever> drivers/mailbox/bcm2835-mailbox.c: { .compatible = "brcm,bcm2835-mbox", },
<clever> i thought i saw some debug, but i'm not finding it now
superherointj has quit [Quit: Leaving]
ryantrinkle has joined #nixos-aarch64
rajivr has joined #nixos-aarch64
h0m1 has quit [Ping timeout: 244 seconds]
<samueldr> clever: any more clue? it's okay if you haven't taken any more time off your schedule or can't :)
h0m1 has joined #nixos-aarch64
alp has quit [Ping timeout: 244 seconds]
<clever> samueldr: none yet, id need to flash some images to my own pi4 and poke around a lot
<clever> and i'm currently trying to figure out why the PLL fractional division isnt working
<clever> its basically ignoring the fractional component entirely
<samueldr> where would you suggest I get in touch with peeps about the problem?
<clever> either the github issues for the linux fork, or the troubleshooting subforum, or maybe the uboot irc
<samueldr> just asked on U-Boot's IRC, though I'm pretty sure it won't be their problem really
<samueldr> hmm, google might be giving me clues
* samueldr sleuths
patagonicus has quit [Quit: The Lounge - https://thelounge.chat]
patagonicus has joined #nixos-aarch64
knerten1 has joined #nixos-aarch64
justanotheruser has joined #nixos-aarch64
knerten2 has quit [Ping timeout: 244 seconds]
<samueldr> clever: "got it"
<samueldr> it's linked to the 1GB thing
<samueldr> adding total_mem=1024 to config.txt "solves" it
<clever> ah, i think i know whats wrong
<clever> the VPU can only ever read the lower 1gig of ram
<clever> so any DMA between arm and vpu, must use the lower 1gig of ram
<clever> and the mailbox involves putting a packet into physical ram, and then telling the vpu where
<clever> something in the firmware/mailbox driver must be confused about those limits
<samueldr> we pass cma=*M
<samueldr> but not the raspberry pi firmware
<clever> drivers/firmware/raspberrypi.c
<samueldr> (the kernel command line params differ!
<clever> void *data = kmalloc(sizeof(*header) + buf_size, GFP_KERNEL);
<clever> ret = rpi_firmware_property_list(fw, data, buf_size + sizeof(*header));
<clever> buf = dma_alloc_coherent(fw->cl.dev, PAGE_ALIGN(size), &bus_addr, GFP_ATOMIC);
<clever> samueldr: are you able to build the broken kernel with a small tweak to the source, and then boot it under u-boot?
<samueldr> what you ask is out of the realm of the possible things with Nix
<samueldr> clever: yes I can ;
<samueldr> ;)
<samueldr> and it's rather trivial, only takes a long time due to relying on Nix
<clever> do you see the dma_alloc_coherent in your copy of the source? within rpi_firmware_property_list()?
<samueldr> "long" would be ~35 minutes for the build, then time to flash
<samueldr> give me 1 minute first
<samueldr> testing without cma= to see
<clever> can you add a `dev_err(fw->cl.dev,"bus address 0x%x\n", bus_addr);` to it?
<samueldr> yep, without cma= it works
<clever> do the logs say where the CMA is allocated?
<clever> [ 0.000000] Reserved memory: created CMA memory pool at 0x000000002a400000, size 64 MiB
<clever> > 0x000000002a400000 / 1024 / 1024
<{^_^}> attempt to call something which is not a function but an integer, at (string):323:1
<clever> 676
<clever> thats within the lower 1gig, should be fine...
<samueldr> you can see the same logs I do
<clever> ah, thats from rpi4.okay!
<clever> [ 0.000000] cma: Reserved 32 MiB at 0x000000007e000000
<clever> > 0x000000007e000000 / 1024 / 1024
<clever> 2016
<{^_^}> attempt to call something which is not a function but an integer, at (string):323:1
<clever> and rpi4.fail put it over 2gig into ram, where it wont work
<clever> [ 0.000000] Reserved memory: bypass linux,cma node, using cmdline CMA params instead
<samueldr> not exactly over
<clever> nearly over
<samueldr> since that's a 2GiB raspberry pi
<samueldr> yeah
<clever> the DTS had a CMA pre-allocated for you
<clever> and your cmdline.txt cma overrode it, breaking things
<samueldr> ugh, cma= is going to be an ugly issue it seems
<samueldr> for the generic image
<clever> the dts is pre-reserving it at the right place, so youll need to somehow omit it on pi machines
<samueldr> we don't know about the pi
<samueldr> it's just a kernel that's booted through u-boot!
<clever> this tells you where its safe to put CMA, and how big it should be
<clever> and it can be within the lower 768mb of ram, gpu_mem=256?
<samueldr> gpu_mem not given
<clever> vcgencmd get_something gpu?
<clever> `vcgencmd commands` shows all commands
<clever> > raspberrypi-tools
<{^_^}> "<derivation /nix/store/xg118vx1s4l9iwsgqd54zb1yklbgc51w-raspberrypi-tools-2020-05-28.drv>"
<clever> samueldr: i would also file an issue to the rpi fork of linux, rpi_firmware_transaction() should assert that the address is within the lower 1gig, and give a better error
<samueldr> hmm, I don't have an ethernet cable that can reach the Pi :/
<samueldr> and vgencmd isn't in the default package set :)
<clever> a bigger and more anoying problem, vcdbg is also very helpful at times
<samueldr> way to set myself for failure :)
<clever> but its binary only, arm32
<clever> vcdbg is basically the same as `gdb /proc/kcore`
<clever> it knows a lot of juicy secrets
<Jake[m]> <samueldr "and vgencmd isn't in the default"> Wait isn't it? I feel like I remember using it on my pi.
<Jake[m]> Yeah clever mentioned it's under raspberrypi-tools. Sorry if I'm misunderstanding though.
<samueldr> Jake[m]: in the default installed package set lol
<clever> Jake[m]: vcgencmd is opensource, vcdbg is closed, and nix only provides it on arm32 via patchelf i believe
<clever> both are in the same derivation
<samueldr> clever: how bad of a workaround would be it to break the part of the code that parses the cma= argument?
<clever> pretty ugly, lol
<clever> but also very simple, one sec
<clever> kernel/dma/contiguous.c:early_param("cma", early_cma);
<samueldr> ooh
<samueldr> no
<clever> this says that when you find cma=, run early_cma
<samueldr> I think we have the solution
<clever> what if you change it to cma_dont ?
<clever> stupidly simple way of breaking things!
<samueldr> we should stop putting cma=!
<samueldr> we're not users!
<samueldr> we're distro maintainers!
orivej has joined #nixos-aarch64
<clever> the firmware is also putting cma into the dts for us, with the correct values for this board
<samueldr> yeah
<samueldr> what I assume happens (unverified) is that in order of precedence it goes "default, DT, cmdline"
<clever> sounds right
<samueldr> I was looking at percentage values
<samueldr> and I don't think we can use them :/
<samueldr> we'd be wasting so much space
<samueldr> I'd assume we'd want 32M on 512MB platforms, since that's what was there previously
<samueldr> and 512M is the Raspberry Pi 3B
<samueldr> so that'd mean 7%
<samueldr> which means 286M for the 4GiB raspberry pi 4
<samueldr> or the pinebook pro
<samueldr> and AFAIK it's really too much for what's needed
<clever> i'm not yet sure why you would ever need more then 32mb
<samueldr> I don't know really
<samueldr> but
<clever> id just leave it at 32mb until things break, lol
<clever> and test out hw decode on each model too
<samueldr> it's **really** funny
<samueldr> since I had a comment about that yesterday from an embedded dev about the PBP
<samueldr> they recommend 64M
<clever> it will likely vary a lot from platform to platform
<clever> based on what is doing dma, and the hw limitations
<samueldr> yep
<samueldr> that was my understanding of it
<clever> i havent confirmed yet if the pi framebuffer must be within the lower 1gig
<samueldr> it's awfully funny how in less than 24 hours two discussions end up at cma=
<clever> pi4 changes things up, and puts several peripherals on their own dedicated 64bit capable MMU
<clever> but the VPU firmware itself can only access the lower 1gig directly
<samueldr> I'll leave it at 32M, since it was the default we used, and seems to work
<samueldr> if it is an issue, a user (!!) can configure it
<clever> if the user sets cma=, it will likely break all mailbox calls
<samueldr> or we might finally end up working on applying device tree overlays (!)
<samueldr> I meant outside of raspberry pi land
<clever> yeah
<samueldr> just started a build
<samueldr> in ~40 minutes I will be able to check
<samueldr> it boots!
<clever> cma= gone from the base image now?
<samueldr> yes
<samueldr> well, once the PR goes in
<samueldr> 32M set as a kernel config option
<samueldr> so you can set it lower or higher via kernel command line as an end-user if needed
<clever> ah
<clever> so your setting it in the kernel config, instead of the args
<clever> and then dts can override it, to fix the pi
<clever> i also just made a major discovery on why i couldnt get DPI to function
<clever> i forgot to set the clock source mux, lol
<clever> the default (0) source is gnd
<clever> so, every time gnd has X pulses, dpi gets 1 pulse
<clever> .... gnd never changes state.....
<Ke> it's not really a problem to have too much cma at those levels, user pages and mostly filesystem pages can still use that region
<Ke> I would assume 1G cma would not be a problem
<clever> userpages and cache can easily be moved as needed
<Ke> never tried it though
<Ke> like assuming you have at least 1G non-cma, which I assume applies to most things nowadays, but not all rpis though
<Ke> well even most rpis
<samueldr> nah, most raspbery pis are from before the fourth!
<samueldr> this is the generic configuration that should work without issues on all aarch64 boards supported by mainline :)
<clever> prior to the pi4, there was a limit of 1gig in the design
<samueldr> and ideally even the ones not supported by mainline, only by changing the kernel package
<samueldr> so any configuration change has to have a limited scope and keep compatibility
<samueldr> it even could affect the armv7l config similarly!
cole-h has joined #nixos-aarch64
<clever> ok, so with the latest config, i have a DPI clock, but i dont get any hsync/vsync...
<samueldr> I'm soo glad to be done with the U-Boot change
<clever> whoa, i have a pixel clock!
<clever> DPI clock measured at 9600000
<clever> at the exact clock the software said its at!!
<clever> thats some decent progress, confirms the DPI is configured partially
<clever> woot!, i now have hsync and vsync!!
<clever> i just had to tell the PV which clock to mux on
<samueldr> neat!
<samueldr> how far are you still from video output?
<clever> its capable of generating the timing information now
<clever> so i could just plug in the numbers for a DPI display, or a VGA adapter, and the display would think its valid
<clever> but its not actually sending any valid pixels yet
<clever> i havent told it where to find image data in ram
<samueldr> that's kind of what I assumed was the state, neat
<clever> but this also means, ive done all of the "undocumented" steps
<clever> this blog documenting how things works, only covers the HVS, the one piece remaining
<clever> so it should be pretty easy from here
<samueldr> oh, before I forget clever++ for good advice and help figuring out the problems earlier
<{^_^}> clever's karma got increased to 0o773
<justanotheruser> I wonder how hard it would be to write a script that converts a LineageOS ROM source to a derivation. e.g. https://github.com/LineageOS/android_hardware_samsung
<clever> currently, its configured for a 10x10 pixel screen, with 1 hsync and 1 vsync
<clever> sure
<samueldr> justanotheruser: probably not worth it compared to using unpackbootimg from a prebuilt boot.img to get the info
<justanotheruser> or at least the nix-relevant bits
<samueldr> on a prebuilt boot.img*
<samueldr> then letting the user find the kernel repo I guess
<samueldr> though finding the kernel repo is kind of totally different than parsing the Android.bp files
<samueldr> I wouldn't like parsing Android.bp files for the values
<samueldr> but parsing the repo (as in repo, the git tooling from google) to get the kernel repo location? might be doable
<justanotheruser> ah, to write the Fairphone 3 config I was just stealing from the lineageOS ROM repo
<justanotheruser> didn't realize there was a more straightforward way
<samueldr> yeah, perfectly valid strategy
<samueldr> yeah, I really want to write some docs about porting, but it's... kind of hard making time for that when I also want to work on the other stuff that's missing
<samueldr> I need a team!
<clever> samueldr: oops!, i just noticed, ive configured it for an FPS of 19,760, lol
<samueldr> 19760 or 19.760?
<clever> 19760
<samueldr> that's a lot of frames
<samueldr> audible even!
<clever> they are only 10x10, with a pixel clock of 2.4mhz
<clever> math says...
<clever> > 2400000 / ((10+1) * (10+1))
<{^_^}> 19834
<clever> that many fps
<clever> 1 pixel worth of hsync in each scanline, and 1 scanline worth of vsync in each frame
<clever> samueldr: https://imgur.com/a/hRlrafA and the vsync/hsynce on a scope!
<clever> if you count it out, you can see 10 periods between each vsync, and vsync is 1 period long
<justanotheruser> samueldr: what is your end goal for mobile-nixos?
<samueldr> to delete the project because it's redundant because every platform are good citizens with mainline linux
<clever> lol
<justanotheruser> heh
<samueldr> to be more down to earth
<justanotheruser> It would be interesting to run nixos on my primary
<samueldr> provide a space to describe the specifics for bad citizens
<clever> even if mainline linux worked, you still have to deal with building that android boot image, adb, and acting like a phone at runtime
<samueldr> and all the workarounds
<samueldr> and in turn, "simply" compose that on top of NixOS
<samueldr> clever: yeah, but if it was only that, that'd be part of NixOS outright
<samueldr> the runtime parts, anything during stage-2, are expected to go straight into Nixpkgs
<clever> though that part could still be part of nixos proper?
<samueldr> except if they're workarounds
<samueldr> justanotheruser: not sure is you see what I mean?
<samueldr> Mobile NixOS describes the specifics to build stuff for those bad citizens, but anything runtime and non-specific is part of NixOS proper
<justanotheruser> bad citizens?
<samueldr> custom kernel trees, weird workarounds, "bad" boot chains
<clever> justanotheruser: phones that need a fork of linux, or lack source entirely
<samueldr> I really want to run NixOS proper on my phone too, that's my personal goal
<justanotheruser> I see
<samueldr> ideally before I "have" to get a job that's not giving me time to work on that, if that ever is needed
<justanotheruser> I think things will get interesting when phosh is merged
<samueldr> yep, though that's just a UI thing, there's still loads of work to make the modem parts and phone parts work
<samueldr> you're also likely to have a... forgot the name, but one of those newer wifi interfaces for qualcomm
<samueldr> which is a WIP about understanding it
<samueldr> qcacld I think
<justanotheruser> the fairphone community got it working in lineageOS https://github.com/WeAreFairphone/android_device_fairphone_FP3
<justanotheruser> (wifi / 4G)
<samueldr> yeah, but that's using the prebuilt binaries on an android-like system, well, android system!
<samueldr> there are userspace components that "drive" around the hardware
<justanotheruser> ahhh
<samueldr> which are built using bionic libc, and a bunch of other assumptions that don't hold true either on glibc-land or on NixOS
<Mic92> thefloweringash: mpv just uses the codec automatically
<samueldr> it's not an impossible issue I think, but basically where I was stuck at with work
<samueldr> but I had to start a yak shave to get working on that
<justanotheruser> I will probably have to wait until ubuntu touch ports FP3 to get the modem working
<justanotheruser> or you to solve it, unfortunately :P
<samueldr> AFAIK ubports (not ubuntu touch anymore) is android-like in that manner
<samueldr> I might have outdated information though
<samueldr> I don't know how modern ubports deal with that
<samueldr> wifi is also likely going to need all that on modern devices (like a fairphone 3)
<justanotheruser> oh wow, I didn't realize how deeply android is embedded in the ecosystem
<samueldr> android are quite far removed from GNU/Linux, sadly
<samueldr> and those userspace daemons (only binaries are made available) are a way from vendor to work around the "inconvenient truth" of FLOSS :(
<samueldr> the driver is fully FLOSS
<samueldr> you know, the bit that shoves commands it receives from a socket into the proper HW interface
<samueldr> commands that are in no way part of the driver
<samueldr> so in reality the "actual driver", the smarts, are in userspace
zupo has joined #nixos-aarch64
<samueldr> that "Android Dumps" group of projects on git.rip is a goldmine
<samueldr> there are equivalent dumps on github too
<samueldr> I'll need to package this https://git.rip/dumps/firmware_extractor
cole-h has quit [Quit: Goodbye]
zupo has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
zupo has joined #nixos-aarch64
alp has joined #nixos-aarch64
juliosueiras has quit [Ping timeout: 244 seconds]
justanotheruser has quit [Ping timeout: 244 seconds]
stiell has quit [Ping timeout: 244 seconds]
stiell has joined #nixos-aarch64
juliosueiras has joined #nixos-aarch64
lopsided98 has quit [Quit: No Ping reply in 180 seconds.]
lopsided98 has joined #nixos-aarch64
quinn has quit [Quit: ZNC 1.8.1 - https://znc.in]
theotherjimmy[m] has quit [Quit: Idle for 30+ days]
juliosueiras has quit [Ping timeout: 256 seconds]
elvishjerricco has quit [Ping timeout: 244 seconds]
elvishjerricco has joined #nixos-aarch64
noonien has joined #nixos-aarch64
alp has quit [Ping timeout: 244 seconds]
alp has joined #nixos-aarch64
alp has quit [Remote host closed the connection]
alp has joined #nixos-aarch64
zupo has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
alp has quit [Remote host closed the connection]
alp has joined #nixos-aarch64
Darkmatter66 has joined #nixos-aarch64
Darkmatter66_ has quit [Ping timeout: 240 seconds]
fooker has quit [Ping timeout: 272 seconds]
fooker has joined #nixos-aarch64
zupo has joined #nixos-aarch64
alp has quit [Ping timeout: 240 seconds]
mvnetbiz_ has joined #nixos-aarch64
orivej has quit [Ping timeout: 265 seconds]
noonien has quit [Quit: Connection closed for inactivity]
alp has joined #nixos-aarch64
superherointj has joined #nixos-aarch64
wavirc22 has quit [Ping timeout: 258 seconds]
juliosueiras has joined #nixos-aarch64
zupo has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
zupo has joined #nixos-aarch64
orivej has joined #nixos-aarch64
ryantrinkle has quit [Quit: Leaving.]
dtz has quit [Quit: Idle for 30+ days]
orivej has quit [Remote host closed the connection]
orivej has joined #nixos-aarch64
bennofs_ has joined #nixos-aarch64
bennofs has quit [Ping timeout: 258 seconds]
cole-h has joined #nixos-aarch64
orivej has quit [Ping timeout: 260 seconds]
alp has quit [Ping timeout: 244 seconds]
stiell has quit [Ping timeout: 240 seconds]
stiell has joined #nixos-aarch64
rajivr has quit [Quit: Connection closed for inactivity]
alp has joined #nixos-aarch64
<colemickens> sam/ueldr: you'd said something about missing apps on aarch64? don't we have firefox for aarch64 though?
<samueldr> colemickens: hm?
<samueldr> disregarding sometimes lack of binaries, yeah
<samueldr> ah, on the tip of unstable we don't have cached binaries
<samueldr> well
<samueldr> we didn't until ~1 or 2 this morning for firefox
<samueldr> there's still that noto font I need to check back on
<colemickens> Hm. Maybe my aarch64 build process is utilizing a builder I have access to more than I realize.
<colemickens> I feel like I would've noticed a firefox build queuing somewhere, but maybe I just never noticed?
<samueldr> you might have never noticed
<colemickens> Is it like Chromium builds?
<colemickens> Where sometimes the channel updates before the binary is up?
<samueldr> but yeah, firefox didn't build apparently I misread the log before going to bed
<colemickens> Or it feels like that sometimes?
<samueldr> the channel updates independently of aarch64's coverage
<colemickens> :/ oh
<samueldr> the channel waits only on a limited subset of aarch64
<samueldr> if that was fixed, I'm sure aarch64 wouldn't cause issues to update in tandem
<samueldr> my gut tells me we're losing between a half and one order of magnitude of build time
<colemickens> I was going to ask about how much it would cost to buy|rent enough aarch64 to close the gap, but I guess maybe paying someone to work on a new scheduler might be a better use of money
<samueldr> I think even reducing the shares could help
<samueldr> or uh
<samueldr> whatever is used to say how many builds in parallel
<samueldr> they're oversubscribed I think
tilpner has quit [Remote host closed the connection]
<samueldr> since firefox *can* build in under an hour
<samueldr> (on the hydra infra!)
<colemickens> samueldr: is it worth just having a special manual job for these exceptional builds that are specially scheduled?
tilpner has joined #nixos-aarch64
<samueldr> but generally fails to build from 4 to 10 hours depending on when it ends up timing out
<samueldr> we'd need a machine that's reserved for that I figure
<samueldr> and probably to hand-picking those
justanotheruser has joined #nixos-aarch64
heywoodlh has joined #nixos-aarch64
alp has quit [Ping timeout: 244 seconds]
heywoodlh1 has joined #nixos-aarch64
<clever> samueldr: there is a link to a static vcdbg in this thread, to solve the arm64 issues
heywoodlh has quit [Ping timeout: 246 seconds]
<samueldr> cool, though I don't have issues anymore :)
<clever> but it could still be of use for others, to package into nixpkgs
<clever> just shove it into the existing raspi-tools derivation
<clever> since its currently missing from 64bit builds
<samueldr> that's a static 32-bit build AFAICT
<clever> but as long as 32bit is enabled in the kernel, it solves the need for having a 32bit glibc
zupo has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
zupo has joined #nixos-aarch64
alp has joined #nixos-aarch64
<heywoodlh1> Hey all, has anyone attempted a port of NixOS mobile to the Razer phone?
justanotheruser has quit [Ping timeout: 240 seconds]
<samueldr> heywoodlh1: 1 or 2?
<heywoodlh1> 1
<samueldr> 2 I had a wee look early on, and it might present some structural differences in recent qualcomm hw
<samueldr> 1 I don't have the HW
<samueldr> though it should be entirely possible to do, just as for the 2 it is
<samueldr> razer are providing sources in a perfectly usable manner
<heywoodlh1> I have a 1. I've been screwing around today with getting it ported but I'm such a noob with nix-shell that I'm hitting some walls.
justanotheruser has joined #nixos-aarch64
<samueldr> sorry, I don't really have much time to expand on porting right now :)
<heywoodlh1> I'm definitely out of my element but it sounded like a really fun challenge.
<samueldr> are you a NixOS user otherwise?
<samueldr> I'm curious since I've seen non-NixOS users be interested in some of NixOS-using projects without themselves using NixOS, which is always a bit surprising
<heywoodlh1> I have been a pretty zealous Arch user for about 5 years but discovered NixOS and have been using it for a few months now. I absolutely love it.
<samueldr> nice
<samueldr> heywoodlh1: one thing that might help, I don't know *how much*, but at least understand the porting better, is to first port postmarketOS...
<samueldr> at least when I made my first port (what kicked off the project) that's how I got to grips with understanding things better
<samueldr> I had a solid proof of things that should be working
<samueldr> their porting docs are much better since they have many more users that (1) contributed to the docs (2) used the docs to make ports
<samueldr> though if you have precise questions, do ask
<heywoodlh1> samueldr: in regard to attempting to port it, I basically copied the devices/oneplus-oneplus3 folder to devices/razer-phone folder and then started modifying the nix scripts to match the configuration changes relevant to the Razer.
<samueldr> that's a valid approach
<heywoodlh1> I probably should have gone with google-walleye since it has the same SOC as the Razer phone. But whatevs.
justanotheruser has quit [Ping timeout: 240 seconds]
<samueldr> there's so few differences that you can compare the files and you should be good to go
<heywoodlh1> Anyway, I *think* I got to the point where the kernel source is downloading properly. But there's a piece I can't seem to figure out.
<samueldr> though with the kernel config you should probably compare against walleye's
<heywoodlh1> Yeah, that makes sense
<samueldr> heywoodlh1: is it "where do you get the config.aarch64 file"?
justanotheruser has joined #nixos-aarch64
<samueldr> it's a hard thing to answer fullly because... there is no universal answer
<heywoodlh1> samueldr: Well, that is something I was wondering about. What I did is grabbed the kernel config from a Razer kernel source.
<samueldr> in theory the truthier answer is to start off the devices _defconfig from the kernel source tree... which sometimes is hard to know which file it is!
<samueldr> right, sounds like you had the right idea
<heywoodlh1> Okay well that is good to know
<samueldr> you probably want to run something like `bin/kernel-normalize-config razer-cheryl` to "expand" the partial defconfig to a full kernel config
<samueldr> (mind your NIX_PATH)
<heywoodlh1> But I think the part I'm running into issues with is that I think the kernel starts building and then it fails because `sed` can't find a Makefile.
<samueldr> another source for a config.aarch64 is the "/proc/config.gz" from either a running system, or extracted from a kernel image from a boot.img
<samueldr> heywoodlh1: can you fork; branch-off; commit your work?
<samueldr> well, and push and share
<heywoodlh1> Yeah, let me do that.
<heywoodlh1> I haven't yet, but I planned on it if I got something working
<samueldr> thanks, that's going to make answering much easier
<samueldr> since I'll just run the build to see the actual problem
<heywoodlh1> I'm guessing I'll have to have my Razer rooted to actually look at /proc/config.gz?
<heywoodlh1> I'll get it forked and my changes pushed
<samueldr> no need to root
<samueldr> if you are not already, there are other methods that are less invasive
<samueldr> and "good" OEMs don't ship with /proc/config.gz enabled
<samueldr> though that doesn't mean the config is not embedded in the kernel, which is possible even without having /proc/config.gz
<heywoodlh1> Okay. I'll figure out how to get access to /proc/config.gz later
<samueldr> the defconfig, in theory, should be enough
<heywoodlh1> It's in there
<heywoodlh1> For some reason Freenode didn't register me properly with my nick, gonna hop off then back on
heywoodlh1 has quit [Quit: WeeChat 2.9]
<samueldr> n/p
heywoodlh has joined #nixos-aarch64
<heywoodlh> Hey, I'm back, not sure why I had the "1" after my nick.
<heywoodlh> samueldr: sorry in advance for my crappy nix edits, nix-shell is still a bit foreign to me :)
<samueldr> I'm curious, what and how are you using nix-shell for with Mobile NixOS?
<heywoodlh> Isn't nix-shell what is running the nix scripts?
<samueldr> nix-build isn't nix-shell
<heywoodlh> Ah
<samueldr> well... actually that's a lie but try not to think about it
<heywoodlh> Haha I'll trust you on that one
<samueldr> nix-shell opens a (bash) shell with some packages in the environment
<samueldr> thus the shell part of the name
<samueldr> nix-build just builds what is described by the nix expressions
<samueldr> right, so I see the issue
<samueldr> the tarball has a subfolder
<heywoodlh> I'm guessing the build should be coming out of that subfolder?
<samueldr> normally you'd use sourceRoot https://nixos.org/manual/nixpkgs/stable/#ssec-unpack-phase
<samueldr> but it looks like the configuration generation step of the kernel build doesn't honor it
<samueldr> (I know I said I didn't have much time earlier, but I didn't have much time for vague questions, this is a precise question and is fine)
<heywoodlh> Where would I define the sourceRoot?
<samueldr> it would normally be in the mkDerivation call, so overrideAttrs should work, but it didn't for me when I tried just now
<samueldr> looking into it a bit
<heywoodlh> Again, Nix syntax is very new to me, so the instructions in kernel/default.nix didn't seem super clear to me as where the actual build commands run
<samueldr> the kernel building tooling in NixOS has years of baggage
<samueldr> and here I dropped another layer on top of that baggage
<heywoodlh> Sure, makes sense.
<heywoodlh> You could revise that sentence to "computers have years of baggage" lol
<heywoodlh> Anyway, thanks for the assistance, I would love to run Nix on this Razer.
<heywoodlh> *NixOS
<samueldr> okay, so I have this fixed in an upcoming WIP commit,
<heywoodlh> Was it a problem with my syntax in kernel/default.nix?
<samueldr> no
<heywoodlh> That's good to know :D
<samueldr> if you look into the tarball, there is a subfolder named msm-4.4
bennofs_ has quit [Ping timeout: 244 seconds]
<samueldr> when nix extracts the tarball, it tries hard to prevent tarbombs
<heywoodlh> That makes sense
<samueldr> so it creates a folder named msm-4.4-7083.tar, in which there is the msm-4.4 folder
<samueldr> well, tarbombs are not really an issue, I might have also misdiagnosed that
<samueldr> so by default nix goes "one level" into the tar
<samueldr> well into the extracted folder
<samueldr> but here there's two
<samueldr> sourceRoot / setSourceRoot are generally used in those cases
<heywoodlh> Ah
<samueldr> but somewhere along the line overrideAttrs doesn't allow doing that
<samueldr> with that I was able to normalize the config
bennofs has joined #nixos-aarch64
<heywoodlh> samueldr: thanks! Those changes worked great. So what exactly does kernel-normalize-config do?
<heywoodlh> Oh, just read the comments in the script
<samueldr> takes a config file and runs it through "the default answers" in what would be a make menuconfig to expand it to a full config file
<samueldr> it's also useful when upgrading a kernel
<samueldr> or when changing forks for a device with multiple forks
<heywoodlh> Okay, cool.
<heywoodlh> I am definitely on a roll for stupid questions now, but I'm gonna ask anyway.
<samueldr> nah, they'd be stupid only if you'd asked them multiple times and getting answers :)
<heywoodlh> samueldr: appreciate it! Lol
<samueldr> otherwise it's just that, asking questions
<heywoodlh> So do I just use bin/menuconfig to actually flash the image to the device?
<samueldr> bin/menuconfig is used to call up the kernel's menuconfig utility to configure kernel options
<samueldr> (google for make menuconfig for more info)
<heywoodlh> Got it. Looking online now.
<samueldr> the installation instructions would look more like `fastboot flash boot result/boot.img` and something along the line
<samueldr> it depends a bit on whether the device is A/B capable
<heywoodlh> Ah yeah. When I flashed it with LineageOS it did require me to do something wierd with the A/B partitions.
<heywoodlh> Super confusing but I did it lol
<heywoodlh> What bootloader do you use/prefer to use with Nix?
<heywoodlh> to clarify, I meant what bootloader do you use with NixOS Mobile
<samueldr> on android-based devices we're mostly forced to use whatever is on the device!
<samueldr> so this is ABL or aboot
<samueldr> in practice the distinction changes nothing for you
<samueldr> ABL works pretty much exactly like aboot
<samueldr> this, in turn, boots the boot.img, which is a kernel+initramfs image
<samueldr> the trick we're doing here is that in the initramfs we're bundling a menu to select a generation to boot from
<samueldr> not implemented yet, it should also allow using kexec
<heywoodlh> Okay.
<heywoodlh> There were certain things in menuconfig that were blatantly obvious to select. Like Platform support for the SoC that's in the Razer. Any other suggestions on things I should probably make sure are selected in menuconfig?
<heywoodlh> Ah, great reference. Thanks!
<samueldr> :/ just looking in parallel and the normalization gave bad results with defaults
<heywoodlh> Is it pulling those defaults from the aarch64.conf file I added in there?
<samueldr> yes
<samueldr> looking at the OEM's boot.img to see if they have the config bundled in
<heywoodlh> Lol whoops. K I'm gonna see if I can figure out what some sane configs would be for aarch64.conf
<samueldr> and they do!
<samueldr> oops, that looks like it's for an older release
<samueldr> indeed, there's another branch on the android dumps project for that
<samueldr> these are stock image files
<heywoodlh> How can you find the config using the boot.img file?
<samueldr> you'll have to extrapolate from the info here
<samueldr> the Linux make config build file is what you want
<samueldr> the filename may differ
<heywoodlh> Man, this is awesome. Thanks for taking the time to answer questions!
<samueldr> I also check on my side because there may be unexpected differences!
<heywoodlh> That makes sense. I want to figure out if I can find the config
<samueldr> sure, please do :)
zupo has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
<samueldr> oh, forgot an important bit
<samueldr> nix-shell in the root of mobile-nixos repo
<samueldr> that'll give you the utilities needed
<samueldr> you probably also want to re-do the kernel/default.nix off of the walleye one
<samueldr> should be rather trivial, copy the src and setSourceRoot
<heywoodlh> That is great
<heywoodlh> That's super cool how you can spawn shell instances like that. I didn't realize that's what `nix-shell` did.
<samueldr> nix-shell -p hello --run hello
<heywoodlh> So, got the config
<heywoodlh> With that config will there be anything I need to change in menuconfig?
<samueldr> yes, the link from postmarketOS still applies
<samueldr> but you're starting with the actual config from that working boot.img rather than with an accidentally bad one
<heywoodlh> Haha yeah, I'm glad you caught that.
<heywoodlh> Are the aarch64.conf files in the current nixos-mobile repository after you applied the optimal configuration?
<heywoodlh> I.E. through menuconfig?
<samueldr> yes
<heywoodlh> Okay cool.
<samueldr> when there is a similar enough device (same major kernel version, same kind of SoC) you can generally diff two files to gauge better
<heywoodlh> That makes sense.
<samueldr> I'm preparing a couple of changes I know you'll need that you can't really know beforehand (except by looking at similar devices)
<heywoodlh> Okay, that's great.
<heywoodlh> I appreciate all the help
quinn has joined #nixos-aarch64
<samueldr> helping someone that hasn't gone through a port yet helps in knowing which parts I'll need to document
<heywoodlh> Yeah, thanks. This is definitely uncharted territory for me. But very interesting!
<heywoodlh> I would hope to be able to get more involved in the project so hopefully I am able to become a bit more competent
<samueldr> nice, let's hope it goes well for you :)
<samueldr> okay
<samueldr> hmmm
<samueldr> let me rewrite the history a bit
<samueldr> so this is just the kernel builder stuff from walleye https://github.com/NixOS/mobile-nixos/commit/0339a7ef451a3caaa4b2c3730502cb9af30c1c91
<samueldr> boot as recovery devices (any recent android devices) will need this patch https://github.com/NixOS/mobile-nixos/commit/7f1dd07128c54e790e6c46c5a14b597ab1ecb704
<samueldr> where I also change one (new) option in config.aarch64
<samueldr> this fixes an issue with the NixOS toolchain vs. expectations from the OEM kernel source https://github.com/NixOS/mobile-nixos/commit/b64db2ea7d9b0fc76ae506f0f8405d06ccf4a2ca
<samueldr> this is a fixup of the patch found in all qualcomm devices, it seems razer-cheryl had just enough changes the other patch wouldn't apply https://github.com/NixOS/mobile-nixos/commit/7508eab34d04702b3d35787f3ded9c8a72dfa8c8
<samueldr> these are the "you couldn't know" changes
<samueldr> everything else I believe can be handled through looking at the postmarketOS wiki, and comparing with walleye
<samueldr> you'll also have some changes to apply on devices/razer-cheryl/default.nix which I haven't done
<samueldr> mainly ab_partitions
<heywoodlh> Okay. I'm going through the PostMarket stuff now and then I'll look at those.
<samueldr> and mobile.usb.mode
taktoa[c] has quit [Ping timeout: 272 seconds]
<samueldr> I would also include the "quiet" kernel command line option since it's the same SoC, maybe there is the same quirk/bug there
<samueldr> somehow on walleye not having a quiet boot breaks the framebuffer :/
taktoa[c] has joined #nixos-aarch64
<samueldr> their source tree is a bit unusual, it needed some different changes compared to the "universally good" patches for two out of three of those
<samueldr> which means I'm vindicated! copying those patches per-device is mostly needed it looks like
<heywoodlh> I didn't do all of the optional configurations for now. I plan on coming back to those later.
<samueldr> I don't know them off the top of my head, but I think none of the optionals end up being needed to boot to the "desktop"
<heywoodlh> Alright, I think I applied all the changes you made to my local repo
zupo has joined #nixos-aarch64
<heywoodlh> samueldr: how do I deal with the A/B partitions on the Razer?
<heywoodlh> Or was that in one of your patches
<samueldr> with (only) my changes I was able to successfully build an android bootimg
<samueldr> though I don't know if it worked (obviously)
<samueldr> A/B is not dealt with yet
<samueldr> I still need to find the mechanism used to tell the bootloader that the boot was successful
<samueldr> right now after a number of boots on A, it goes to B, then the same number of boots goes into failure mode
<samueldr> IIRC
<heywoodlh> Okay. So how do I build an image?
<heywoodlh> If anything, I'll try what I have now and see if it works
<samueldr> nix-build --argstr device razer-cheryl -A build.default
<heywoodlh> Hmm. Ran into an issue that I'm not sure why it's happening
<heywoodlh> One sec
<samueldr> you will need to build against nixos-unstable
<samueldr> two issues
<samueldr> (1) LOCALVERSION has to be set to "" for the expectation of NixOS
<samueldr> and (2) the version number has to match the *kernel* version number
<samueldr> (for the expectations of NixOS)
<heywoodlh> Got it.
<heywoodlh> How do I build against nixos-unstable?
<samueldr> which can be passed using -I
<samueldr> it could be -I nixpkgs=$HOME/whatever/nixpkgs with a proper git checkout
<samueldr> or -I nixpkgs=channel:nixos-unstable
<samueldr> the latter being whatever the channels point to right now
<samueldr> and harder to test a specific commit
<heywoodlh> So if I already labeled my nixos channel to point to unstable that should be sufficient for this build, right?
<heywoodlh> Also, could I use LineageOS recovery to flash the image that I am building?
justanotheruser has quit [Ping timeout: 244 seconds]
<samueldr> yes, you'll see it anyway fail quite harshly if it's not building against unstable
<samueldr> and uh... not sure about linageos recovery, I've been using only fastboot
<samueldr> oh, important note
<samueldr> there are (some) brick chances, as in actual bricks, when playing about with custom roms
<samueldr> some phones are way worse than others
<samueldr> though I guess that since you're using LineageOS, you already know the usual warnings
<samueldr> AFAIK, unless your phone has a quirk that makes it prone to _actual_ bricks, at worse you can just reinstall using fastboot
<samueldr> and note that if you are using the phone as a daily driver, there is no guarantee for your data
<heywoodlh> Nah, it's my backup phone
<heywoodlh> If I don't supply a partition to flash to, should that work even with the confusing A/B setup on this phone
<samueldr> I believe so
<heywoodlh> I.E. to flash Lineage Recovery I did `fastboot flash boot recovery.img`
<samueldr> though you can also look under result
<samueldr> there will be a flash-critical.sh script
<samueldr> it should show the commands to flash both
<samueldr> if you have added the ab_partitions config to the device definition
<heywoodlh> Since I don't care about Lineage I think I might just flash both
<samueldr> anyway it's not like you can't flash it back
<heywoodlh> Dang one other build error
<samueldr> did you enable CONFIG_USB_G_ANDROID ?
<samueldr> if so disable it
<samueldr> G_ANDROID is for older android devices that are using android_usb
<samueldr> I'll be off for a bit, if it fails again try to look at the error message and guess what option you enabled that could be causing that
<heywoodlh> Hmm, no I don't see that option
<heywoodlh> Okay, I'll look around
<samueldr> right
<samueldr> also compare with my final config.aarch64 file
<samueldr> since that one *compiles*
<samueldr> it probably doesn't work in useful ways
<samueldr> but it compiles
<samueldr> so if you diff it you might be able to guess about which option around usb gadget mode caused the failure
justanotheruser has joined #nixos-aarch64
<heywoodlh> The traceback had a bunch of stuff about configfs.c so I copied all the stuff you had regarding CONFIGFS