ChanServ changed the topic of #robotnix to: Robotnix: https://github.com/danielfullmer/robotnix || Channel logs: https://logs.nix.samueldr.com/robotnix
matthewcroughan has joined #robotnix
<matthewcroughan> Hey ;D
<danielrf[m]> hello!
<matthewcroughan> I have to say, the promise of building AOSP from source via Robotnix and submitting device support via Nix code is much more promising than browsing the XDA forums.
<matthewcroughan> What a brilliant idea.
<danielrf[m]> Haha, thanks! I agree!
<matthewcroughan> And a custom hosts file.. My god. Who even needs Nix on Android when you can just build Android with nix? :D
<matthewcroughan> How would I go about getting this working for my device then? How much work does one need to do?
<danielrf[m]> which device do you have?
<matthewcroughan> Two. One is a OnePlus One, (very well supported everywhere, so I guess that one will be easier)
<matthewcroughan> The second, a Xiaomi Mi 9 Lite which is my main phone
<matthewcroughan> the Mi 9 Lite is codename Pyxix and available here: https://dl.twrp.me/pyxis/
<matthewcroughan> Pyxis*
<matthewcroughan> >The AOSP project requires at least 250GB free disk space
<matthewcroughan> Is that real? :O
<danielrf[m]> unfortunately yes
<matthewcroughan> can we use nix distributed builds? Does it utilise derivations a lot?
<danielrf[m]> I expect the oneplus one should work with the lineageos flavor in robotnix
<danielrf[m]> The pyxix doesn't seem to be officially supported by lineageos, so it would be a bit more involved to get that working, but hopefully not too bad
<danielrf[m]> As far as distributed builds, the main problem is that robotnix currently has one giant derivation that takes hours to build.
<danielrf[m]> So not split up in a way that could be distributed via Nix
<matthewcroughan> Do you think it's possible to get to the stage where the build is split up?
<danielrf[m]> That said, I have some ideas of ways to improve that: https://github.com/danielfullmer/robotnix/issues/25
<ajs124> slightly related, and because I was wondering that the other day, distcc in nix(pkgs), is that a thing?
<danielrf[m]> I have vague memories about that existing, but I've never tried it
<danielrf[m]> matthewcroughan: although robotnix does support ccache which helps a little bit
<matthewcroughan> Magic..
<matthewcroughan> danielrf[m]: so how would I get started producing an output for my oneplus one?
<matthewcroughan> --arg configuration '{ device="crosshatch"; flavor="vanilla"; }' \
<matthewcroughan> in this, where is device= handed off to a function?
<danielrf[m]> Try: nix-build --arg configuration '{ device="bacon"; flavor="lineageos"; }' -A ota
<danielrf[m]> In the lineageos flavor, it uses the `config.device` option to determine which lineageos device directories to include in the build
<matthewcroughan> Right, and without official support, this obviously won't work, so how do you add support without adding it to lineageos itself?
<matthewcroughan> I've got a bunch of devices that aren't supported upstream
<danielrf[m]> See this example:
<danielrf[m]> I need to document it, but here he has added a number of device-specific directories to the build
<danielrf[m]> via the `source.dirs` option
<matthewcroughan> so mako is not a device that is supported by lineage?
<danielrf[m]> Not for lineageos 17.1, I don't believe
<matthewcroughan> So what about devices that are absolutely unsupported by lineage in the first place and have been ported by some random dudes?
<danielrf[m]> Still should be like the example earlier. If the unofficial port targets lineageos then you'd use the lineageos flavor in conjunction with setting some `source.dirs` manually
<matthewcroughan> sweet
<matthewcroughan> so all robotnix does is easily compile sources in AOSP *or* LineageOS format?
<matthewcroughan> If the sources are in another format, things will be much harder?
<danielrf[m]> That link you sent uses lineageos 12.1, which is very old though, so that would probably require some generic work on robotnix to make it work
<matthewcroughan> danielrf[m]: What stops me from using a later version of LineageOS?
<matthewcroughan> What is required to take this device from 12.1 -> Latest?
<danielrf[m]> Unless someone has already done the android work, probably a lot of development :)
<danielrf[m]> But robotnix could also be extended to work with older android versions
<matthewcroughan> I've never understood why you can't just take a device's old sources and take it up
<danielrf[m]> The core android project does make backwards-incompatible changes that the device dirs need to be updated for
<matthewcroughan> And in particular, why can't you just take the kernel sources and merge the latest in?
<matthewcroughan> well what kind of changes are needed? Is this purely a merge conflict that needs to be resolved? Or does code actually need to be reformatted and everything?
<danielrf[m]> I've personally never done this forward-porting of android devices, so it's a bit out of my expertise
<danielrf[m]> But I expect they are nontrivial changes
<matthewcroughan> danielrf[m]: what's the `-A ota` bit for?
<danielrf[m]> That chooses the "over-the-air" zip output
<danielrf[m]> since lineageos expects you to sideload via recovery instead of using `fastboot update`
<matthewcroughan> danielrf[m]: ah crap
<danielrf[m]> ugg. I have experienced this in the past
<matthewcroughan> `Unable to checkout c780a2e17197f70b4cde3aaac1021a78f48a516f from https://github.com/LineageOS/android_external_chromium-webview.`
<matthewcroughan> they force pushed, I'm guessing?
<danielrf[m]> LineageOS rebases / pushes new commits and then the old commits are garbage collected
<danielrf[m]> I assume so
<danielrf[m]> Makes it very difficult to reproduce things in the past :(
<matthewcroughan> so much for reproducibility
<danielrf[m]> If that's the only repo with that problem, maybe try setting `source.dirs."external/chromium-webview".enable = false;`
<matthewcroughan> this happens in any build system that relies on external repos
<matthewcroughan> danielrf[m]: how do I set that?
<danielrf[m]> And then enable the robotnix-built chromium by setting `apps.chromium.enable = true;` and `webview.chromium.enable = true;`
<danielrf[m]> So, at this point the config is getting large enough that I'd create a `.nix` file containing this
<matthewcroughan> so you'd make bacon.nix?
<danielrf[m]> and then when running `nix-build` pass something like `--arg configuration ./bacon.nix`.
<danielrf[m]> Sure
<matthewcroughan> so the contents contain only one line which is that?
<matthewcroughan> danielrf[m]: ssh ro-Z8fRtV6EV24BCEGMJhVQDM2Jr@lon1.tmate.io
<matthewcroughan> you can see my ssh session here if you so please :p
<danielrf[m]> Cool
<danielrf[m]> Yeah, also add `device="bacon";` and `flavor="lineageos";`
<danielrf[m]> Also set: `apps.chromium.enable = true;` and `webview.chromium.enable = true;`
<danielrf[m]> Because I don't believe that your build would work without any webview enabled
<danielrf[m]> You have a trailing ` on the second line
<matthewcroughan> good spot ;D
<matthewcroughan> that is unbelieveable
<danielrf[m]> This will also build chromium from source--which will also take a long time as well
<matthewcroughan> the fact that you can just make a tool like robotnix and pass it a path with --arg like that
<matthewcroughan> cross compiling for aarch64 too?
<danielrf[m]> yep :)
<matthewcroughan> danielrf[m]: using pkgsCross?
<danielrf[m]> No, not with pkgscross. It uses chromium's build system support for cross compilation with clang
<matthewcroughan> oh, well what about the rest of this build that's happening with robotnix?
<matthewcroughan> isn't everything else using it?
<danielrf[m]> using pkgsCross, you mean?
<matthewcroughan> yeah
<danielrf[m]> Still no, the android build also uses their own prebuilt clang
<danielrf[m]> I'd love to have the option to use the toolchain from nixpkgs, but we'd have to rip out a bunch of stuff from the chromium and android build systems to do that
<matthewcroughan> I'm still amazed that it's going to do that and work fine though, so that's great
<matthewcroughan> I don't see a lot of activities on my cores, is it still just fetching git repos? I think it might be hah
<danielrf[m]> Yup, lots of source to fetch :)
<matthewcroughan> is it using repo sync? Or is it just using git?
<danielrf[m]> just git
<danielrf[m]> I have a `mk-repo-file.py` script to produce files like this: https://github.com/danielfullmer/robotnix/blob/master/flavors/vanilla/repo-android-11.0.0_r29.json
<danielrf[m]> so all the git repos are pinned to specific tags/revisions
<matthewcroughan> danielrf[m]: Do you use any Google Services on your phone?
<danielrf[m]> I personally use microg
<matthewcroughan> yeah me too, a pain in the ass isn't it? ;D
<danielrf[m]> Mostly for push notifications
<danielrf[m]> yup :)
<matthewcroughan> yeah man, it's so screwed up how all push notifs go to google
<matthewcroughan> some architecture in tech today is beyond the strangest scifi
<matthewcroughan> danielrf[m]: can we do resurrection remix?
<danielrf[m]> maybe? I haven't looked into it thus far. I've tried to make the robotnix internals generic enough so it's possible to add new flavors
<danielrf[m]> Additional flavors in robotnix would require someone motivaed to maintain it as well
<matthewcroughan> yeah because it looks like those guys are maintaining for my device
<matthewcroughan> danielrf[m]: I believe I would be
<matthewcroughan> it's my daily driver and I don't buy new devices, and I like looking at these internals
<danielrf[m]> Ah cool! Yeah, adding flavors isn't documented yet, but it should be possible to look at the vanilla/grapheneos/lineageos flavors and see how it could work
<matthewcroughan> brilliant
<matthewcroughan> will the build I did really take up 240GB of space by the time I wake up tomorrow?
<matthewcroughan> I don't actually have that space in the system, so I'll need to wait if so.
<danielrf[m]> haha, yes, it might!
<ajs124> if you have /tmp on a tmpfs and <128GB (or what was the number?) RAM, you'll wake up and the build will have failed ^^
<matthewcroughan> lmao
<danielrf[m]> yeah approx 100GB is needed for just the intermediate android build products
<matthewcroughan> so I can't do this without more storage? Damnit :D
<danielrf[m]> and IIRC, maybe another 50GB for source?
<matthewcroughan> I only have 191GB
<danielrf[m]> So, maybe 150GB total?
<danielrf[m]> it might fit, but it would be tight
<matthewcroughan> I'll wake up tomorrow and find out hah
<matthewcroughan> and it's probably gonna kill my SSDs right?
<danielrf[m]> Hopefully not :)
<ajs124> Is it a Kingston A2000? If so, it just might.
<matthewcroughan> LOL, that's the absolute bastard that killed my server
<matthewcroughan> that exact ssd
<matthewcroughan> omg
<danielrf[m]> I'm going to modify the lineageos flavor to build webview from robotnix, instead of using the prebuilt webview apk
<danielrf[m]> This brings it in line with vanilla and grapheneos
<danielrf[m]> So the workaround should no longer be necessary if using the latest version of robotnix
<danielrf[m]> No need to stop your current build though