gchristensen changed the topic of #nixos-chat to: NixOS but much less topical || https://logs.nix.samueldr.com/nixos-chat
KeiraT has quit [Remote host closed the connection]
KeiraT has joined #nixos-chat
<andi-> joepie91: that isn't ruma, right?
<joepie91> it's one you've never heard of :D
<joepie91> andi-: nop
<joepie91> New Home is the name so far
<andi-> joepie91: Curious how that will evolve.. I share the feelings about synapse.
<joepie91> most of the work so far has been on the database side, implementing a performant graph schema
<joepie91> results so far are promising
<joepie91> (storing and querying graph data is... non-trivial :P)
<ashkitten> joepie91: what's the goal of your implementation?
<joepie91> ashkitten: being a performant (though not necessarily *optimally* performant) implementation with low technical debt / high code quality, that's easy to adapt for experimental features
<joepie91> (well, and viable for prod usage, though that's kind of a given for me :P)
malSet has quit [Ping timeout: 248 seconds]
<ashkitten> joepie91: that sounds good.. i've heard from others that the matrix spec is a bit messy so i wish you luck
<joepie91> a bit messy, sure, though it's not too bad (especially as there are people to clarify it)
<joepie91> the spec isn't the biggest challenge :D
<ashkitten> i'm looking to run my own matrix server at some point and synapse is not something i want to run...
<ashkitten> i've tried before and it's just awful
<joepie91> graph data is a bit of a nightmare to deal with, and Matrix is basically made of graph data
<ashkitten> heh
<ashkitten> what language?
<joepie91> but yeah, I don't want to run Synapse either
<joepie91> ashkitten: JS, but unlike how typical JS looks like
<joepie91> to just get that out of the way right away :P
<ashkitten> lol
<ashkitten> better than python at least, performance wise
<ashkitten> and in my opinion nicer to work with than python
<joepie91> well, sure, but I don't think the language matters terribly much on the perf front tbh
<joepie91> the real bottleneck is on the data handling side
<ashkitten> ehh python doesn't scale well due to the GIL
<joepie91> ie. the bit that postgres is gonna be dealing with
<joepie91> sure, but there just isn't much computation that your server actually needs to do
<joepie91> if you're writing performant queries
<ashkitten> that's fair
<joepie91> it's a typical 'data shoveling with a bit of access control' type project, for the most part :P
<joepie91> so I think it'd be possible to write a performant-enough Python implementation as well
<joepie91> but yeah, I much prefer JS in terms of how good of a codebase it lets me build, and the extra performance is a nice bonus
<ashkitten> i know when i ran synapse it was very sluggish and people said it was probably due to the huge chunks of json it's got to decode all the time and since it doesn't really do threading properly it can't handle other tasks at the same time
<joepie91> ashkitten: the main challenge I ran into was the state resolution bit
<ashkitten> on something like that
<joepie91> needed a way to do state resolution without a million roundtrips to the DB or loading half the room history into memory
<joepie91> I've solved that problem
<joepie91> so now I can do stateres on tiny subsets of the room history
<ashkitten> oh cool
<joepie91> that was the biggest performance hazard I think
<ashkitten> actually, i wonder if it'd be possible to write a python implementation without a GIL and benchmark it against standard python
<ashkitten> though that'd be a massive undertaking for a language i don't actually like so nvm lol
<joepie91> ashkitten: room history is basically a DAG; the TL;DR of how my solution works is that each event has a 'branch value'; root event has a value of 1, if it splits into 2 then each of those has 1/2 (ie. 0.5, but fractional), if one of those splits into 3 then each of *those* has 1/6, and so on; reconverging nodes get their fractions added together (so eg. a 1/2 event + a 1/6 event would become 4/6 or 2/3 in the converging event)
<ashkitten> oh huh
<joepie91> and since stateres only has to happen on events where a parallel branch exists, I can just do a recursive selection of parent and then child events
<joepie91> and treat 1/1 nodes as the boundary
<ashkitten> that is a wild history model
<joepie91> the segment inbetween the closest 1/1 nodes on either side of the target event, is the full segment that the target event exists in which may have branches
<joepie91> and that is all the data that is needed for stateres
<joepie91> so in a single (recursive) query I can select the full set of events that is needed to do stateres and determine the validity and relation to other events of any given one event
<ashkitten> cool!
<joepie91> and no more than that
<joepie91> also: first time I've had a genuine use for fractions in programming
<joepie91> lol
<joepie91> (this is not accurately expressible with floats/decimals)
<ashkitten> heh
<joepie91> anyhow, the reason for the DAG is to handle the distributed aspect
<joepie91> as rooms don't exist on one server, but on all servers which have at least one user participating in the room
<joepie91> so it's the equivalent of netsplit resolution
<joepie91> but for persistent room history and room state, and with fully untrusted servers
<joepie91> I have some... doubts about how robust the stateres algorithm is, but the approach is interesting
<joepie91> and the DAG allows representing a diverging history in a way that can be sorted out later, eventually consistent
<ashkitten> huh
<ashkitten> so what causes the state to split?
<joepie91> ashkitten: legitimate splits occur upon race conditions; each server knows which event was 'last', and this is the parent ID of the event that *they* produce when one of their users does something... except if the 'last' message has changed before that new event was created and the server just didn't know yet due to network latency, you now have two events with the same parent ID
<joepie91> and that is a split
<ashkitten> ath
<ashkitten> ahh
<joepie91> malicious splits could of course be crafted by just producing events with wrong parent IDs
<joepie91> the stateres algorithm is meant to deal with both
<ashkitten> very cool
<joepie91> (Matrix is secretly just a bespoke multi-master distributed database, not a messaging protocol :P)
qyliss has quit [Quit: bye]
qyliss has joined #nixos-chat
<joepie91> anyhow, for anyone interested in New Home who's also on Matrix, #new-home:pixie.town (cc andi- ashkitten)
<joepie91> that's where updates are very occasionally posted :P
<ashkitten> oh cool
<ashkitten> i'll have to join when i'm on my computer because riotx doesn't seem to work with room or user lookups
<joepie91> unrelated: whee, just rearchitected an entire library and it's working again :D
<joepie91> yeah, RiotX seems to generally be missing a lot of functionality..
<joepie91> I hear people complain about it a lot anyway :P
<ashkitten> yeah
<ashkitten> i use it because idk lol
<ashkitten> alternately you could invite me to the room, @ashkitten:cybre.space
<joepie91> ashkitten: done :P
drakonis has joined #nixos-chat
malSet has joined #nixos-chat
KeiraT has quit [Remote host closed the connection]
KeiraT has joined #nixos-chat
<joepie91> ashkitten: you may find this interesting, btw: https://gist.github.com/joepie91/107b793d46e4ac32584cc026d1e15e91?ts=4 and https://gist.github.com/joepie91/67acc5ab3af5f86697819d4db2627b7e?ts=4 (only the last part of the code is different, initialization / test data is the same)
<joepie91> immutable collection type that allows refining down a cursor and modifying subsets of that collection, batching operations together internally where possible
<joepie91> to avoid unnecessary array allocations
<joepie91> (the idea being that you can create your own immutable collection types with custom methods like `.selectVisibleDocuments()` or whatever)
malSet has quit [Quit: Quit.]
malSet has joined #nixos-chat
* joepie91 is off to bed now
<ashkitten> i'm confused what immutable means if you can in fact modify it
<gchristensen> it is making a new thing
<ashkitten> oh, a copy of the collection
waleee-cl has quit [Quit: Connection closed for inactivity]
<infinisil> qyliss: Yeah about same sleeping times for me
<infinisil> In my current situation the night is just the best time to be awake
<infinisil> As there are no other people awake in my household
<infinisil> So it's nice and quite and non-distracting
<qyliss> I've recently rediscovered the University Library here
<qyliss> Which it turns out I have access to as a former student, and is open 24/7
<qyliss> It's almost empty at night
<qyliss> My productivity has been through the roof
<infinisil> Going to a library at night? :o
cole-h has joined #nixos-chat
Taneb has quit [Quit: I seem to have stopped.]
<ashkitten> ooh libraries seem like nice places to hang out at night
<qyliss> most libraries aren't opened at night
<qyliss> but lots of university ones are
drakonis has quit [Quit: WeeChat 2.7]
endformationage has quit [Ping timeout: 268 seconds]
__monty__ has joined #nixos-chat
cole-h has quit [Ping timeout: 272 seconds]
<colemickens> is copumpkin still around? I remember them from when I started using nixos
Irenes[m] has joined #nixos-chat
<Irenes[m]> I've seen them on Twitter a bit, I assume they're somewhere.
<Irenes[m]> Yep, still on Twitter. Also still using he/him pronouns.
<Irenes[m]> I never like to assume when it's been a while, heh
Taneb has joined #nixos-chat
waleee-cl has joined #nixos-chat
<pie_[bnc]> yay, my r issue is finally getting some attention! https://github.com/rstudio/rstudio/issues/4965#issuecomment-583557949
vesper has joined #nixos-chat
hl has quit [Quit: Quit]
waleee-cl has quit [Quit: Connection closed for inactivity]
hl has joined #nixos-chat
waleee-cl has joined #nixos-chat
endformationage has joined #nixos-chat
tilpner_ has joined #nixos-chat
cole-h has joined #nixos-chat
drakonis_ has joined #nixos-chat
KeiraT has quit [Ping timeout: 240 seconds]
<Church-> gchristensen: Huh, are you an op over in ##aws? Just realized you set the title. :P
<gchristensen> yup
<Church-> Heheheh
<Church-> I'm doing battle with AWS and hating it
<Church-> Since they technically caused my problem to begin with
<gchristensen> I still have eyes and ears in that channel, but I'm not active in the channel or an active AWS user
<Church-> :D
<Church-> Nod, nod
drakonis_ has quit [Remote host closed the connection]
drakonis_ has joined #nixos-chat
wildtrees has joined #nixos-chat
<ar> Church-: i hate error msgs aws apis/webui gives me :/
<Church-> ar: Heh try Amazon hugging one of your instances to death with traffic one of their subsidaries send you.
<Church-> What fun
KeiraT has joined #nixos-chat
waleee-cl has quit [Quit: Connection closed for inactivity]
lovesegfault has joined #nixos-chat
waleee-cl has joined #nixos-chat
drakonis_ has quit [Read error: Connection reset by peer]
claudiii has quit [Quit: Connection closed for inactivity]
lovesegfault has quit [Quit: WeeChat 2.7]
<__monty__> Pretty interesting read for those of us who like math: http://math.andrej.com/2006/03/27/sometimes-all-functions-are-continuous/
<__monty__> Woops, that was meant for another channel >.<
<__monty__> Gonna sign off in shame...
<samueldr> nah, it's fine, just... too thickly mathsy for my even thicker head
<__monty__> Just skim, pretend you understand and go around telling everyone who'll listen that discontinuous functions are fake news.
<__monty__> That's my plan at least ; )
<pie_[bnc]> for a second i thought i was in one of my type theorist channels
<__monty__> Ooo, TT channels? Do tell. I could afford to be in more of those.
__monty__ has quit [Quit: leaving]