KeiraT has quit [Remote host closed the connection]
KeiraT has joined #nixos-chat
<andi->
joepie91: that isn't ruma, right?
<joepie91>
it's one you've never heard of :D
<joepie91>
andi-: nop
<joepie91>
New Home is the name so far
<andi->
joepie91: Curious how that will evolve.. I share the feelings about synapse.
<joepie91>
most of the work so far has been on the database side, implementing a performant graph schema
<joepie91>
results so far are promising
<joepie91>
(storing and querying graph data is... non-trivial :P)
<ashkitten>
joepie91: what's the goal of your implementation?
<joepie91>
ashkitten: being a performant (though not necessarily *optimally* performant) implementation with low technical debt / high code quality, that's easy to adapt for experimental features
<joepie91>
(well, and viable for prod usage, though that's kind of a given for me :P)
malSet has quit [Ping timeout: 248 seconds]
<ashkitten>
joepie91: that sounds good.. i've heard from others that the matrix spec is a bit messy so i wish you luck
<joepie91>
a bit messy, sure, though it's not too bad (especially as there are people to clarify it)
<joepie91>
the spec isn't the biggest challenge :D
<ashkitten>
i'm looking to run my own matrix server at some point and synapse is not something i want to run...
<ashkitten>
i've tried before and it's just awful
<joepie91>
graph data is a bit of a nightmare to deal with, and Matrix is basically made of graph data
<ashkitten>
heh
<ashkitten>
what language?
<joepie91>
but yeah, I don't want to run Synapse either
<joepie91>
ashkitten: JS, but unlike how typical JS looks like
<joepie91>
to just get that out of the way right away :P
<ashkitten>
lol
<ashkitten>
better than python at least, performance wise
<ashkitten>
and in my opinion nicer to work with than python
<joepie91>
well, sure, but I don't think the language matters terribly much on the perf front tbh
<joepie91>
the real bottleneck is on the data handling side
<ashkitten>
ehh python doesn't scale well due to the GIL
<joepie91>
ie. the bit that postgres is gonna be dealing with
<joepie91>
sure, but there just isn't much computation that your server actually needs to do
<joepie91>
if you're writing performant queries
<ashkitten>
that's fair
<joepie91>
it's a typical 'data shoveling with a bit of access control' type project, for the most part :P
<joepie91>
so I think it'd be possible to write a performant-enough Python implementation as well
<joepie91>
but yeah, I much prefer JS in terms of how good of a codebase it lets me build, and the extra performance is a nice bonus
<ashkitten>
i know when i ran synapse it was very sluggish and people said it was probably due to the huge chunks of json it's got to decode all the time and since it doesn't really do threading properly it can't handle other tasks at the same time
<joepie91>
ashkitten: the main challenge I ran into was the state resolution bit
<ashkitten>
on something like that
<joepie91>
needed a way to do state resolution without a million roundtrips to the DB or loading half the room history into memory
<joepie91>
I've solved that problem
<joepie91>
so now I can do stateres on tiny subsets of the room history
<ashkitten>
oh cool
<joepie91>
that was the biggest performance hazard I think
<ashkitten>
actually, i wonder if it'd be possible to write a python implementation without a GIL and benchmark it against standard python
<ashkitten>
though that'd be a massive undertaking for a language i don't actually like so nvm lol
<joepie91>
ashkitten: room history is basically a DAG; the TL;DR of how my solution works is that each event has a 'branch value'; root event has a value of 1, if it splits into 2 then each of those has 1/2 (ie. 0.5, but fractional), if one of those splits into 3 then each of *those* has 1/6, and so on; reconverging nodes get their fractions added together (so eg. a 1/2 event + a 1/6 event would become 4/6 or 2/3 in the converging event)
<ashkitten>
oh huh
<joepie91>
and since stateres only has to happen on events where a parallel branch exists, I can just do a recursive selection of parent and then child events
<joepie91>
and treat 1/1 nodes as the boundary
<ashkitten>
that is a wild history model
<joepie91>
the segment inbetween the closest 1/1 nodes on either side of the target event, is the full segment that the target event exists in which may have branches
<joepie91>
and that is all the data that is needed for stateres
<joepie91>
so in a single (recursive) query I can select the full set of events that is needed to do stateres and determine the validity and relation to other events of any given one event
<ashkitten>
cool!
<joepie91>
and no more than that
<joepie91>
also: first time I've had a genuine use for fractions in programming
<joepie91>
lol
<joepie91>
(this is not accurately expressible with floats/decimals)
<ashkitten>
heh
<joepie91>
anyhow, the reason for the DAG is to handle the distributed aspect
<joepie91>
as rooms don't exist on one server, but on all servers which have at least one user participating in the room
<joepie91>
so it's the equivalent of netsplit resolution
<joepie91>
but for persistent room history and room state, and with fully untrusted servers
<joepie91>
I have some... doubts about how robust the stateres algorithm is, but the approach is interesting
<joepie91>
and the DAG allows representing a diverging history in a way that can be sorted out later, eventually consistent
<ashkitten>
huh
<ashkitten>
so what causes the state to split?
<joepie91>
ashkitten: legitimate splits occur upon race conditions; each server knows which event was 'last', and this is the parent ID of the event that *they* produce when one of their users does something... except if the 'last' message has changed before that new event was created and the server just didn't know yet due to network latency, you now have two events with the same parent ID
<joepie91>
and that is a split
<ashkitten>
ath
<ashkitten>
ahh
<joepie91>
malicious splits could of course be crafted by just producing events with wrong parent IDs
<joepie91>
the stateres algorithm is meant to deal with both
<ashkitten>
very cool
<joepie91>
(Matrix is secretly just a bespoke multi-master distributed database, not a messaging protocol :P)
qyliss has quit [Quit: bye]
qyliss has joined #nixos-chat
<joepie91>
anyhow, for anyone interested in New Home who's also on Matrix, #new-home:pixie.town (cc andi- ashkitten)
<joepie91>
that's where updates are very occasionally posted :P
<ashkitten>
oh cool
<ashkitten>
i'll have to join when i'm on my computer because riotx doesn't seem to work with room or user lookups
<joepie91>
unrelated: whee, just rearchitected an entire library and it's working again :D
<joepie91>
yeah, RiotX seems to generally be missing a lot of functionality..
<joepie91>
I hear people complain about it a lot anyway :P
<ashkitten>
yeah
<ashkitten>
i use it because idk lol
<ashkitten>
alternately you could invite me to the room, @ashkitten:cybre.space
<joepie91>
immutable collection type that allows refining down a cursor and modifying subsets of that collection, batching operations together internally where possible
<joepie91>
to avoid unnecessary array allocations
<joepie91>
(the idea being that you can create your own immutable collection types with custom methods like `.selectVisibleDocuments()` or whatever)
malSet has quit [Quit: Quit.]
malSet has joined #nixos-chat
* joepie91
is off to bed now
<ashkitten>
i'm confused what immutable means if you can in fact modify it
<gchristensen>
it is making a new thing
<ashkitten>
oh, a copy of the collection
waleee-cl has quit [Quit: Connection closed for inactivity]
<infinisil>
qyliss: Yeah about same sleeping times for me
<infinisil>
In my current situation the night is just the best time to be awake
<infinisil>
As there are no other people awake in my household
<infinisil>
So it's nice and quite and non-distracting
<qyliss>
I've recently rediscovered the University Library here
<qyliss>
Which it turns out I have access to as a former student, and is open 24/7
<qyliss>
It's almost empty at night
<qyliss>
My productivity has been through the roof
<infinisil>
Going to a library at night? :o
cole-h has joined #nixos-chat
Taneb has quit [Quit: I seem to have stopped.]
<ashkitten>
ooh libraries seem like nice places to hang out at night
<qyliss>
most libraries aren't opened at night
<qyliss>
but lots of university ones are
drakonis has quit [Quit: WeeChat 2.7]
endformationage has quit [Ping timeout: 268 seconds]
__monty__ has joined #nixos-chat
cole-h has quit [Ping timeout: 272 seconds]
<colemickens>
is copumpkin still around? I remember them from when I started using nixos
Irenes[m] has joined #nixos-chat
<Irenes[m]>
I've seen them on Twitter a bit, I assume they're somewhere.
<Irenes[m]>
Yep, still on Twitter. Also still using he/him pronouns.
<Irenes[m]>
I never like to assume when it's been a while, heh