claudiii_ has quit [Read error: Connection reset by peer]
betawaffle has joined #nixos-on-your-router
claudiii_ has joined #nixos-on-your-router
<q3k>
so, anyone ever tried running NixOS on an ONIE-compatible network switch?
<q3k>
google says andi- asked about this a few months ago here :P
<q3k>
i have an ONIE-running Celestica SmallstoneXP/D4040 on my desk, and i might mess around with NixOS on it
<hexa->
the thing is … you will run into trouble getting the switch drivers usually
<hexa->
especially if they are broadcom ones
<q3k>
yeah this is a trident2
<q3k>
i'm ready for a world of pain
<hexa->
awesome :)
<q3k>
but a) there's some SDKs that live on this ICOS installation that came with it
<q3k>
b) at celestica has open sourced some of its stuff, i see kernel patches for a CPLD module, etc
<q3k>
c) i have a disassembler handy
<q3k>
also this is more researchy than me trying to actually get this thing to ever be back in a DC again
<hexa->
i have no hands-on experience with ONIE, due to a lack of hardware .)
<hexa->
:)
<q3k>
yeah i only have this switch because i found it cheap on ebay
<q3k>
sniped it within minutes of the save search appearing
<hexa->
which one is it?
<q3k>
Celestica SmallstoneXP/D4040
<q3k>
32x40G
<hexa->
fancy
<hexa->
> • 4GB NAND flash, optional removable media
<{^_^}>
error: syntax error, unexpected $undefined, at (string):317:1
<q3k>
too fancy, i now need to get QSFP+ NICs to test this thing :P
<q3k>
i haven't looked into the NAND yet, i have an SSD (mSATA) dump though
<hexa->
dac cables should be ~100 €
<q3k>
yeah i ordered a QSFP+ splitter
<q3k>
but i also want to test a QSFP+ full speed iface at some point
<hexa->
good luck
<q3k>
yeah, this is going to be a long-term/never-done project, i feel
<andi->
q3k: what does cheap on ebay mean?
<andi->
How cheap?
<andi->
I think NixOS on that machine would be feasible if we have the user space blobs and kernel interface to talk to the ASIC. Given the current state of network hardware that is probably as best as it gets without buying mellanox
<andi->
ok, they start as low as 159$ interesting
<q3k>
andi-: 350eur
<q3k>
andi-: (EU, shipped)
<q3k>
so i'm not 100% on the software architecture
<q3k>
but it does seem like the kernel stuff is at leasy somewhat open, and that it's mostly shims?
<q3k>
my switch came with an instance of the OpEN Open Ethernet Networking API (sdk+docs)
<q3k>
not sure if that's what ICOS uses internally to actually program the switch, too
<q3k>
and regardless that API isn't available publically
<q3k>
however, OpenNSL is, and it's supposed to support trident2
<q3k>
OpenNSL is still not open-open, parts of it are under a weird source-available broadcom license
<q3k>
but it's an official release
<q3k>
and then again i'm not sure into how much you can build on top of OpenNSL
<q3k>
it'd also be nice to know how much of the SONiC NOS could be reused
<q3k>
or OpenNetworkLinux
<andi->
q3k: let me know how that progresses.. I do not have a use case myself but I'd be up for investing that amount of money and to invest some time.
<andi->
I am still not sure how much the trident2 platfroms still differentiate between each other
<q3k>
they seem to be pretty much the same regarding the blobby broadcom stuff
<q3k>
with some support software from vendors
<andi->
yeah
<andi->
probably i2c to configure interfaces etc.. is still different?
<q3k>
yeah
<q3k>
the d4040 has a sprinkle of MachXO CPLDs
<q3k>
and has a kernel shim for that
<q3k>
(for which i found some source in ONIE patches)
<q3k>
not sure what talks to it, though
<q3k>
the OpEN SDK tarball i have seems to be smallstonexp branded, so there might be some platform-specific stuff in there
<q3k>
hard to say
<q3k>
the onie build system is... certainly a system.
<q3k>
can't wait to use nix instead
<flokli>
q3k: heh
<flokli>
spent some time today on obsoleting some petalinux fpga build system
<flokli>
so somewhat related
<flokli>
(when it comes to scary compiler toolchains, and really ugly closed source software :-) )
<q3k>
so i think i made something that's a valid ONIE installer
<q3k>
this should boot the nixos installer (without actually touching the host system, apart from making an empty /nix directory for a temporary bindmount)
<q3k>
this should actually run everywhere, and is effectively a self-contained overtake-via-kexec-and-boot-a-nixos-installer thing
<hexa->
> text/x-shellscript (622 MB)
<{^_^}>
attempt to call something which is not a function but a path, at (string):317:1