<tazjin>
infinisil: still awake? I'm building a spooky struct/record "type checker"
<infinisil>
I am, oh?
<tazjin>
the actual record checker needs a few more minutes, but I've written some Common Lisp style type-checker primitives, let me make a demo screenshot
<eyJhb>
etu: it was Rick & Morty regarding the Titanic :p
pie_ has quit [Ping timeout: 252 seconds]
<eyJhb>
I have throw out every stand for every monitor I have, except I have three stands for flatscreen TVs, to which I don't own a TV for one of them...
<eyJhb>
I hate past me
<etu>
eyjhb: Ah, that makes sense as well yes :D
<eyJhb>
Yeah, just saw the episode :D After 15+ hours of Friends, I needed some change :p
pie_ has joined #nixos-chat
<eyJhb>
pie_: you survived ;)
<pie_>
eyjhb: its a catastrophe
<pie_>
not me i mean kernel partition info
pie_ has quit [Remote host closed the connection]
<elvishjerricco>
Took another look at ZFS encryption. I think it's even worse than I thought. Not only does it leak data, it seems incredibly easy to tamper with. They have done a fine job making sure you can't write arbitrary custom data without the key, but it seems trivial to rearrange encrypted blocks however you like.
<elvishjerricco>
Basically, if ZFS is just a big hierarchy of block pointers, only the absolute leaf nodes of file data are encrypted (including directory listings, which secures posix permissions and whatnot)
<elvishjerricco>
That makes it really easy to see where files are and how big they are. With their sizes, you can likely find files whose contents you already know. Dnode numbers can maybe tell you which files are which in relation to one another.
<elvishjerricco>
And from there you can just start copying, deleting, and moving encrypted blocks all over the place.
<eyJhb>
elvishjerricco: I would love seeing that as a PoC
<elvishjerricco>
eyjhb: The easiest proof of concept would be finding a file like sshd by its probably fairly unique size and literally replacing it with any other program that you can identify. ZFS won't notice a thing, and you'll spawn an executable of your choice as root.
<elvishjerricco>
Of course the program has to already be on their disk though, so I dunno what dangerous software you could replace sshd with
<elvishjerricco>
The scariest thing would be replacing / deleting blocks of kernel code.
<eyJhb>
elvishjerricco: sooo many conf files you could replace. But if you replace a encrypted file with a non-encrypted file, won't that screw with things?
<eyJhb>
Plus... Could you do that live?
<elvishjerricco>
eyjhb: You could only replace an encrypted file with a file already on the same file system
<elvishjerricco>
Definitely can't do it live
pie_ has joined #nixos-chat
<elvishjerricco>
I guess the worst case attack vector here is when you thought you could trust some server with your backup because you used `zfs send -w` to send an encrypted stream. It turns out they can mess with all your files, so restoring from such a backup is quite dangerous
<etu>
That's why one probably should do gpg signatures on your backup files...
<etu>
So one can verify the integrity
<etu>
and this is true for any backup system
<elvishjerricco>
etu: How would you do something like that with e.g. znapzend?
<etu>
elvishjerricco: For znapzend I don't know, but thinging zfs send | gpg :)
<elvishjerricco>
But then you don't get to have another zpool as the receiver, and can't delete old snapshots if you're using incremental sends.
<etu>
that's true
<sphalerite>
yeah, for all the zfs fanciness to work, you'd need a remote block device
<sphalerite>
elvishjerricco: also, wouldn't swapping pointers around break the metadata hashes?
<elvishjerricco>
sphalerite: Yes but you can just rewrite them. The only ones that require the secret key are the ones that you're moving to new places and not modifying
pie_ has quit [Ping timeout: 252 seconds]
<elvishjerricco>
i.e. the level 0 block pointers
<elvishjerricco>
I guess that's not quite accurate. The level 0, file / directory data block pointers are the only ones that need the key to rewrite, and you're not modifying the data they point to.
<elvishjerricco>
Whoops, disregard everything I've said :P I misinterpreted their bit about the key salt.
<elvishjerricco>
They essentially rotate the key frequently by running the master key and a new salt through a HKDF
<elvishjerricco>
So the vulnerability I describe is only relevant to those blocks that share a salt. I have no idea how frequently they rotate this though
<sphalerite>
qyliss: are you familiar with iqubic yet?
<qyliss>
no?
<sphalerite>
qyliss: just a heads up, I have had them on /ignore for a long time and very few regrets about it :p
<elvishjerricco>
Oh, nonono, the salt is stored in the block pointer, so I was right originally
<qyliss>
thanks for the warning :)
<elvishjerricco>
Though with NixOS, much of the concern is alleviated by `nix verify` :P
<gchristensen>
surely zfs' encryption is tamper resistent......... :|
<gchristensen>
elvishjerricco: maybe you should try doing it
<elvishjerricco>
gchristensen: Doesn't really seem like it, to an upsetting degree. As far as I can tell, though you can't create custom encrypted blocks, you can essentially move / copy them between files freely.
<elvishjerricco>
Though you have to be able to identify the files somehow, which can be done via their size in many cases
<gchristensen>
wouldn't that cause ZFS to detect corruption?
<elvishjerricco>
gchristensen: You can just rewrite all the parent checksums trivially
<adisbladis>
elvishjerricco: I'm starting to lean in that direction too... It feels like talking to a wall.
<elvishjerricco>
The only checksum that requires the encryption key is the one you don't need to change
<elvishjerricco>
And that's just a MAC used for authenticated encryption
<elvishjerricco>
But otherwise, only the leaf-most nodes in the file data trees are encrypted. Everything else is an open, editable book
<gchristensen>
do you think it is feasibly something you might be able to do?
<elvishjerricco>
gchristensen: If I spent a ton of time learning the ZFS code base, yea, assuming this video's description of the encryption system is accurate: https://youtu.be/frnLiXclAMo
<gchristensen>
ah yeah that one
<gchristensen>
well dang
<gchristensen>
forgive me but I hope you're wrong
<elvishjerricco>
Yea. My only use case for ZFS encryption (remote backups) would largely not care about this issue if there were an effective way to sign a snapshot.
<elvishjerricco>
Me too :P
<elvishjerricco>
At least the attack vector is somewhat slim. You have to find files you already know exist on their system, and figure out how to copy / move / delete chunks of them in whatever order you like to get a new file that does what you want
<gchristensen>
right
<etu>
And if you do it slightly wrong, a scrub would scream at you.
<gchristensen>
not just that, reading will fail and scream at you. (a scrub is nothing special, just literally reading every file on disk)
<elvishjerricco>
Yea. Any portion you screw up the checksums for will completely fail *and* alert them of the attack.
<sphalerite>
adisbladis: was that highlight meant for me?
<adisbladis>
Yes it was
<etu>
gchristensen: ah, right
* etu
just started using ZFS so he's reading along with great interest
<sphalerite>
adisbladis: can recommend, it improved my quality of life ;)
<aanderse>
etu: thanks for the pic! that is awesome!
<sphalerite>
infinisil: for youtube, I use newpipe on my phone and mpv to play stuff on my laptop, it's not really going to stop them from knowing what I watch and stuff, but at least the tracking isn't as aggressive as what it can do in the browser
<sphalerite>
oh, and I search it using `dillo youtube.com` :p
<gchristensen>
ah man, a few __functors slipped in to nixpkgs :') (okay)
<gchristensen>
elvishjerricco: woot
<elvishjerricco>
Not fully comprehending that block of comment yet so I may be wrong... Like maybe the SHA512 they create can be forged as well. I see no reason it couldn't be
<infinisil>
Yeah that's actually the same behavior as lib.isFunction too
<elvishjerricco>
Sounds like maybe the calculate MACs for the whole tree just as they do with checksums?? That doesn't sound right; where are they stored?
<{^_^}>
import-from-derivation (IFD) is when you evaluate nix from a derivation result, for example `import (pkgs.writeText "n" "1 + 1")` will evaluate to 2. This is sometimes problematic because it requires evaluating some, building some, and then evaluating the build result.
<joepie91>
well yeah I know what it is
<joepie91>
but why is it a problem?
<manveru>
because it's slow
<joepie91>
but... why :P
<manveru>
god knows, or eelco :P
<manveru>
i really don't know, i basically did `(import (writeFile "bar.nix" "args: ```${readFile "bar.html"}```")) {baz = 1;}`
<manveru>
irccloud really doesn't like that line
<manveru>
anyway, my idea was to have normal HTML files where you just use `${}` to access whatever stuff you want, or include other templates, or build CSS/JS...
<colemickens>
well, in my quest to find more undoc'd functionality in Azure, I had a very precise Gogole query with only a few results, and for some reason google had indexed a github issue through niniyou instead of GH. I'm not sure a VPN would've done anything here.
<ivan>
I reported to Google
<colemickens>
Unless being in Belgium made it prefer to show me that link over a GH one? I'm more inclined to chalk it up to Google weirdness.
<gchristensen>
aye
<ivan>
yeah Google indexes a lot of reverse proxy / spam things like that
<colemickens>
ivan: weird, so is GitHub down with that proxying? I'd like ot think their login page would include some JS to disable logins on non-https?
<ivan>
no, GitHub would not find that acceptable
<colemickens>
s/down/complicit or okay/g (bad colloquialism there)
<gchristensen>
it is very easily not a direct proxy
<ivan>
what Google search found that issue?
<ivan>
you might want to check if your results are being replaced
<eyJhb>
I love this `If you want to mount a mirror to the wall permanently`, that is strong..
<joepie91>
eyjhb: that looks like normal tape
<eyJhb>
I use it for when I need to drill holes too, as I can temp place it in place, drill and put in the screws
<joepie91>
proper carpet tape has much higher adhesive strength
<joepie91>
more like duct tape
<eyJhb>
Well, cannot say, but this I have right now (which looks like that and is same brand) has been holding much better than duct tape ever has for me :/
<joepie91>
hm, maybe their normal tape is similar to carpet tape then :P
<eyJhb>
I think so :p The only place it sucks is on some cheap wood, in my "server closet"
<eyJhb>
It cannot for the life of it function in-there.. But have been using it to tape up advertisments outside, which.. I think are still stuck some places 2-3 years later
<eyJhb>
joepie91: currently using it to hold my monitor in place :p
<joepie91>
living on the edge :)
<joepie91>
(that'll probably cost you a monitor at some point, if it's wall-mounted)
<eyJhb>
No no, not at all. I just threw out ALL monitor stands, but I have three TV stands.. So I have just placed a strip of tape on the bottom of the stand, so that .. 2 sec,
<cransom>
i built an ergodox and an iris and they were fun to make, maybe more so than use. i did some layout switching while i tried out the ortholinear stuff and it just broke my brain.
<eyJhb>
tazjin: you have been holding out on us
<eyJhb>
cransom: ever tried a prebuild one?
<cransom>
of either type? no.
<cransom>
i also got paralyzed a bit by the amount of customization possible. i thought about how to change it to be awesome rather than just using it and adopting.
<eyJhb>
Wonder if one could use layers more effectively with nixery.dev
<eyJhb>
True... Done the same. So now I am just used to my layout
<eyJhb>
Even if it isn't the best. E.g. I have no right shift
<aleph->
Heh who needs layers
<aleph->
Single image layers!
<eyJhb>
Because.. æøå
<aleph->
Go small or go home!
<eyJhb>
Wouldn't that be big aleph- ? :p
<cransom>
i'd be interested in seeing how one of the model 01s feel as it looks pretty and i like the thumb cluster. but the likelyhood of seeing one in the wild is small.
<eyJhb>
joepie91: I can hear my monitor slowly coming down.
<eyJhb>
I think I would die, as it would force my ways in what I find a somwehat weird position :/
<eyJhb>
I need something to block this damn monitor from falling this slowly...
<eyJhb>
Maybe more tape
<tazjin>
eyjhb: pretty sure Nixery has been brought up here before 😉
psyanticy has quit [Quit: Connection closed for inactivity]
<tazjin>
eyjhb: what do you mean by using layers more effectively? I've wanted to collect some stats from the public instance to see how many cache busts it actually gets that could be avoided (https://github.com/google/nixery/issues/15)
<eyJhb>
tazjin: add a new layer for each tool used, to it can build upon itself, e.g. it seems like it will always download a 130 mb layer when you change anything. But it is quite tricky to do anything, as the layers need to come in the correct order
<tazjin>
no the order is irrelevant
<eyJhb>
You sure? Because that does not seem right
<tazjin>
I should have a keyboard shortcut for linking to this post 🙂
<etu>
gchristensen: I sent that to my colleges in attempt to use nix to build docker images. Now they are even more sure that they want *ubuntu* as base since it can be shared anyways :/
<gchristensen>
that is absolutely the wrong takeaway
<etu>
Indeed.
<etu>
I don't think I can win though :p
<tazjin>
I should probably increase the max layers in Nixery
<tazjin>
The default is very conservative
<gchristensen>
120 is a good number I've found
<tazjin>
Isn't that the max?
<gchristensen>
124
<tazjin>
I think it should allow ~5-10 steps of extendability, just in case
<tazjin>
Ah
<gchristensen>
iirc :)
<tazjin>
How different is 120 from, say, 110 in practice?
<gchristensen>
probably not very
<eyJhb>
tazjin: what is the current limit?
<tazjin>
24 or something, I'd have to check
<eyJhb>
gchristensen: btw. good write :D
<eyJhb>
tazjin: increasing it from what I read would difinitely be nice!
<gchristensen>
thanks!
<gchristensen>
I think I set it so low because the original overlay fs was VERY limited.
<eyJhb>
OH! A phone number! Can I now prank call you?
<gchristensen>
sure
<gchristensen>
I'd be annoyed though
<eyJhb>
I feel like not many would call you from a new number each day, so I am somewhat outed
<eyJhb>
Did you set it to 24 or what gchristensen ?
<gchristensen>
I did
<eyJhb>
Is there no toc of the entries on your blog ?
<gchristensen>
go to the end
<eyJhb>
But is that _all_ of them, as it doesn't seem clear to me
<gchristensen>
that is all of them
<eyJhb>
Great, thanks :p I am just used to it only being a subset
<gchristensen>
I write infrequently :)
<eyJhb>
Oh. You should not checkout my blog then.
<eyJhb>
And waaay less technical, and... Proffread
<gchristensen>
yeah, well, I deleted all the posts that weren't very technical or well written :)
<eyJhb>
Yeah okay, makes sense
<eyJhb>
I would love to write more.. And I am currently trying to scout some english students at my uni
<gchristensen>
I love the immutable web stuff, but at the end of the day I too want to "craft" my public appearance :P
<tazjin>
I'm gonna set it to 96 and make it configurable
<tazjin>
96 is a number I pulled out of a hat that was under my bed
<gchristensen>
tazjin: want to send a PR to nixpkgs bumping the 24 to 96?
<tazjin>
gchristensen: let me experiment with what that yields on the public instance for a bit
<eyJhb>
gchristensen: but isn't that already somewhat well established?
<eyJhb>
tazjin: say if you need some.. Help, I am curious as well
<gchristensen>
yeah maybe
<gchristensen>
but for users who extend it, they'll get an error -- not a deploy failure. and users who don't, they'll get a better experience
<manveru>
i also tried the layered image thing with hakyll... was 120 layers of a few mb each, and a huge one with about 1.5GB :)
<manveru>
usually it's pretty good, but breaks down when you have thousands of packages i guess
<manveru>
(numbers pulled out of my ass, because it's been a while)
<tazjin>
manveru: haskell packages are a PITA in that way because of the dynamic linking to trillions of tiny libraries
<manveru>
node is similar, just less linking
<manveru>
and doesn't need years to compile
<cransom>
just takes years to run
<manveru>
true that :)
<manveru>
i'm kinda curious why the language ecosystems evolved that way
<joepie91>
manveru: "that way" = ?
<manveru>
i guess both haskell and js don't come with batteries
<joepie91>
oh, the large amount of tiny libraries?
<manveru>
when users feel less able to solve common problems themselves?
<joepie91>
it's by and large a conscious decision
<joepie91>
not just a product of circumstance
<joepie91>
eg. Node core intentionally avoids adding stuff to core
<joepie91>
(I can only speak for the JS ecosystem here)
<manveru>
yeah
<joepie91>
basically, anything that's managed separately instead of core (regardless of whether it's still published by core devs) can be separately versioned, forked, replaced, and so on
<joepie91>
so you don't run into the problem of "anything in core must forever stay compatible"
<joepie91>
nor the problem of "I need this new feature but my runtime is too old"
<manveru>
i like these problems :D
<joepie91>
means old crusty standard libraries (hi Python!) are not an issue either
<joepie91>
because when something becomes obsolete, it can just be put into maintenance mode and effectively replaced by a newer, better alternative
<manveru>
that's why ruby is still my go-to for quick scripts, because i learned the stdlib like 15 years ago and can still use most of it
<joepie91>
since it's all just independently versioned packages
<joepie91>
manveru: that's no different in JS, though. just the bits you've learned are separately versionable :)
<tazjin>
we have a rule at work that there can only be one version of any given external dependency across the whole company
<joepie91>
"being in core" doesn't actually bring any advantages, in and of itself
<joepie91>
from an end user perspective
<tazjin>
they all get vendored into the monorepo, so it's possible to keep track (ish)
<joepie91>
some things /need/ to be in core for binding-to-the-runtime reasons, but that's not a concern you deal with as an end user
<tazjin>
and I find that rule really interesting because it highlights how much of an accounting issue these deps actually are, which devs often like to ignore
<manveru>
joepie91: the advantage is that you don't need package managers to parse some yaml
<joepie91>
manveru: so here's the thing; that's a benefit that only exists on paper
<joepie91>
core is never going to contain everything you need, which means that either a) you end up reinventing wheels or b) you need a package manager anyway, and there goes your 'benefit'
<joepie91>
I hopefully don't need to explain why A is undesirable :p
<joepie91>
in practice, "needing a package manager for simple things" just isn't really an issue
<manveru>
i'm getting old i guess :)
<manveru>
our definition of simple things is probably different
<joepie91>
tazjin: it's an accounting issue mostly because of bad tooling
<joepie91>
saw a thing recently that could significantly improve upon this
<joepie91>
sec
<joepie91>
well this is not helpful, not finding it in my bookmarks...
<joepie91>
tazjin: nah, a tool for distributed dependency review
<joepie91>
which is a tool that has long been missing
<gchristensen>
+1 Nix was a big hit with the Tumblr security team. they really really liked the transparency.
<joepie91>
what's the damn thing called
<gchristensen>
RFC2822? ;)
<manveru>
isn't that the date standard?
<srhb>
That's 8221..
<gchristensen>
internet message format (email)
<srhb>
Er, no, 8601. Dammit..
<manveru>
heh
<joepie91>
well, I give up for tonight
<joepie91>
anyway, there exists a tool for reviewing dependencies and submitting them to a public repository
<gchristensen>
too much heckling? (sorry)
<joepie91>
the reviews, that is
<joepie91>
nah, I give up on finding the tool :P
<manveru>
:D
<joepie91>
anyway, much of the problem with dependency management is that there's no good tooling for tracking 'approved' dependencies and potential issues
<joepie91>
reviewing all dependencies of a typical JS project, for example, would /in principle/ be possible, /if/ you had the tooling for bookkeeping
<joepie91>
that tool seemed to basically be trying to be that
<manveru>
i wish more people were interested in that...
<tazjin>
we have that implicitly because all vendored third-party code must be reviewed, and if it's in the vendored folder and its build targets are public then you can use it
<tazjin>
the one version rule tackles some other things in addition though
<joepie91>
tazjin: yeah but that's at the cost of proper dependency management :P
<joepie91>
I'm talking about a mechanism that you can drop into how package managers work today
<joepie91>
without arbitrary restrictions
<joepie91>
on eg. versions
<manveru>
my stance is still that i try to read and then own the code of every dependency i use... but i've yet to work at a company where anybody gives a damn about that
<tazjin>
all third-party code is reviewed and has at least 2 human owners here, I would assume other large companies have similar processes in place
<joepie91>
anyway, for context: I'm sitting in the weird spot on the spectrum where I care about both a) safe and reliable dependency usage, and b) the ability to arbitrarily pick and add dependencies at no cost, and I don't believe that they need to be at odds
<tazjin>
joepie91: it sounds like a solution to a different problem
<tazjin>
I don't really want adding more dependencies to be /easier/
<joepie91>
manveru: see, I agree with that :P
<joepie91>
it's mostly a tiresome discussion for me to have because SO MANY people immediately assume "oh that means you must restrict dependencies", being absolutely /convinced/ that an inherent tradeoff exists there
<joepie91>
and I don't believe that it does
<joepie91>
tazjin: I do.
<manveru>
also, if i have to start writing a package.json or Gemfile or whatever to write a quick script, it starts feeling like a project and i lose momentum :)
<manveru>
so i'm very much in the camp of "let's have a <lang>.withPackages for all the things"
* joepie91
has been idly thinking about an 'inline package.json' thing
<joepie91>
(where the package metadata would be embedded into your single-file quick script)
<manveru>
man, nixfmt would be almost perfect if it didn't mess with argument attrsets so much
<tazjin>
what does it do to them? I haven't looked at any of the formatters yet
<manveru>
well, it's not horrible, just a bit annoying :)
<manveru>
also from some reason it always turns urls to strings
<manveru>
i assume there's a reason for it, just can't think of any
<tazjin>
style decision?
<tazjin>
I guess the point of a tool like this is to be opinionated
<manveru>
well, i get that, but why ignore a language feature so much?
<manveru>
anyway, i still use it on all my nix code, it keeps getting better, so i'm not gonna complain much :)
<tazjin>
is there a feature in plain URLs other than omitting the quotes?
<tazjin>
in fact typing a plain URL into the REPL returns a quoted string
<manveru>
they are strings, afaik
<manveru>
it's just a special case of path i think
<manveru>
it works for everything that matches .+:/ i think
<joepie91>
there's discussion of deprecating URL syntax entirely
<joepie91>
as far as I can tell there's no point in having it
<joepie91>
and all it does is adding more stuff for people to learn and get confused by
<tazjin>
+1
<manveru>
fair enough :)
<manveru>
also just saw that they fixed the argument attrsets somewhat, before the latest version it would split them at any space, now at least it tries to not split the default arguments in separate lines
<manveru>
anw, i'll take this minor annoyance over having to constantly reformat everything myself :)
<tazjin>
nixery.dev now serves a maximum of 96 layers
<tazjin>
lets see how this goes
<manveru>
nice :)
Jackneill has joined #nixos-chat
Jackneill has quit [Remote host closed the connection]