jasongrossman has quit [Ping timeout: 252 seconds]
emily has quit [Ping timeout: 250 seconds]
jasongrossman has joined #nixos-chat
jasongrossman has quit [Quit: ERC (IRC client for Emacs 26.1)]
drakonis_ has quit [Read error: Connection reset by peer]
<infinisil>
Random question: is there some machine learning method for finding points in a k-dimensional space by just the distances between them?
<infinisil>
So e.g. I'd say that A and B are 0.5 apart, B and C are 0.1 apart, so it knows A and C are also about 0.5 apart, and does some ML stuff to assign each point a number between 0 to 1 (maybe such that the avg is around 0.5)
<infinisil>
I'm interested in using this to classify my music. Distance would be similarity between them, and the k-dim space would be the genre. Seems like a promising idea
jackdk has quit [Ping timeout: 268 seconds]
<ekleog>
infinisil: hmm… isn't what you want a constraint solver? just set A to be (0, ..., 0), then add constraints for each distance
<infinisil>
There will be thousands of points, maybe 10 dimensions, and the distances between points are only approximate, not exact
<ekleog>
and then once you have your space run k-means over it to split it into genres
<ekleog>
infinisil: I'd say it's an exercise of trying to optimize by minimizing the errors to distances, then, so minimizing the outcome of f() over a space of… a few ten thousands dimensions? sounds like something gradient descent can do
siers has joined #nixos-chat
* ekleog
doesn't know much about machine learning, just basics (disclaimer)
<ekleog>
where the inputs of f would be all the coordinates of all points
<infinisil>
Hmm.. I did take a machine learning course, if gradient descent works that would be nice
<infinisil>
Simulate a k-dimensional space where distances are attractive forces
<infinisil>
Run until the error is low enough
<infinisil>
I guess that's not simulated annealing though, rather just a simulation
<infinisil>
Could get rather expensive though, because with 1000 points attracting each other that's 1000000 forces, each of them being 10 dimensional, but I think that's handleable
<ekleog>
hmm, you could look at how things like d3js do it to draw stuff in 2D, if you're going the simulation route
Synthetica has joined #nixos-chat
<sphalerite>
infinisil: yeah the limiting factor there would be the number of points more than the number of dimensions I think
<sphalerite>
complexity is linear on the number of dimensions but quadratic on the number of points
Peetz0r has quit [Quit: vanwege redenen enzo]
__Sander__ has joined #nixos-chat
Peetz0r has joined #nixos-chat
Peetz0r has quit [Quit: brb]
Peetz0r has joined #nixos-chat
<infinisil>
Yeah
__monty__ has joined #nixos-chat
<ekleog>
sphalerite: (about port 25 filtering) IMO the fact outgoing 25 is blocked by default is a very good thing. Just imagine how much worse the spam would be with all virus-infected computers having port25 access…
<ekleog>
also, it allows you to run your mail server from your ISP without being automatically in all blacklists because there's too much spam coming from this IP space
<sphalerite>
ekleog: it's *not* port 25, it's port 587 that it blocks!
<ekleog>
win/win, except when you're trying to debug stuff
<ekleog>
oh. read “smtp” ._.
<sphalerite>
I agree that 25 should be blocked
<sphalerite>
probably better to reject them than blackhole them, as many seem to do…
* ekleog
didn't notice the “(submission)” just after
<sphalerite>
but yeah blocking submission seems insane to me
<ekleog>
indeed
<ekleog>
unless some mailserver admins allow inter-server smtp on submission and add clients to blacklists based on that judgement…
* ekleog
thinks that unfortunately also exist, but this time it's mailserver admins who should be banned…
__Sander__ has quit [Remote host closed the connection]
<gchristensen>
I didn't know Linux had proper support for thunderbolt device security levels
siers has quit [Ping timeout: 268 seconds]
siers has joined #nixos-chat
drakonis has quit [Ping timeout: 272 seconds]
__Sander__ has quit [Quit: Konversation terminated!]
drakonis has joined #nixos-chat
lopsided98 has quit [Quit: Disconnected]
lopsided98 has joined #nixos-chat
drakonis has quit [Ping timeout: 260 seconds]
drakonis has joined #nixos-chat
lopsided98 has quit [Quit: Disconnected]
lopsided98 has joined #nixos-chat
<elvishjerricco>
I'm guessing that if I make a 500M Nix file, then importing that is likely to require Nix to consume 500M of memory minimum, right?
<gchristensen>
omg elvishjerricco what are you doing
<joepie91>
hahaha, was about to say the same thing
<joepie91>
this sounds like a Bad Idea brewing
<elvishjerricco>
gchristensen: Lol cramming all of Hackage into a Nix file :P
<siers>
hasn't that been done already though
<siers>
?
<elvishjerricco>
siers: nixpkgs has a nix file with a single version of every package. I'm doing ALL versions and revisions
<siers>
ah
<gchristensen>
O.o
<elvishjerricco>
Currently I've got it making one file per package version / revision.
<gchristensen>
o.O
<elvishjerricco>
But that prevents me from compressing all of it because I can't import a nix file directly from a tar file and allow it to reference other files in the tarball
<elvishjerricco>
So I'm considering one gigantic file :P
<elvishjerricco>
But I'm guessing that's a bad idea
<elvishjerricco>
Compression seems critical for this though. 1G uncompressed, 44M compressed
<gchristensen>
O.O
<elvishjerricco>
Looots of redundancy :P
<__monty__>
Can't you refactor out some of that redundancy?
<elvishjerricco>
Well each file is basically the output of cabal2nix on some cabal file
<elvishjerricco>
So I dunno how i'd do that
<__monty__>
I dunno either but in my head it sounds like obvious because we're working in a nice functional language : )
<elvishjerricco>
I think removing the redundancy would amount to writing a generic Nix expression compressor :P
<__monty__>
As long as it doesn't turn into obfuscated nix...
<elvishjerricco>
Ideally Nix would just support treating tarballs like directories as far as importing goes
<elvishjerricco>
But there's lots of hairy things to work out for that
<gchristensen>
scary
<elvishjerricco>
Maybe like a mode for `builtins.path`. Something like `import (builtins.path { path = ./tarball.tar.gz; type = "tar"; } + /foo.nix)`
<gchristensen>
or recursive nix, buildFromTarball ...
<elvishjerricco>
gchristensen: Well we can do something like that with IFD already
<gchristensen>
right
<elvishjerricco>
Something like that would require unpacking the whole tarball though, defeating the point