<arianvp>
andi-: one problem I face when updating NixOS modules to be more sandboxed is that if you enable DynamicUser=True
<arianvp>
that the runtime directory is /var/lib/private/blah instead of /var/lib/<service-name>
<arianvp>
and there is no good story on how to migrate existing services that don't run under a DynamicUser to a one that do
<arianvp>
The nice thing about DynamicUser being that we don't need to pre-allocate uids and gids for security-critical stuff
<arianvp>
but how do you migrate a specific-uid owned state directory to the DynamicUser model gracefully? Do we want to support migration of this kind of stuff?
<arianvp>
or do we just depend on `stateVersion` and keep the old behaviour as well? and when `stateVersion = 19.03` we use DynamicUser?
<arianvp>
I guess stateVersion _is_ the standard way to handle this right
<andi->
Isn't it counter productive to store state with random uids?
<andi->
I think we might just allocate a uid for them and be done?
<ekleog>
andi-: the uid list is getting filled too quickly for my taste
<arianvp>
the whole point of DynamicUser is that you can store state _and_ have random allocated uids
<ekleog>
like, the rss2email module has been pending for only ~2-3 weeks and I've already had to rebase it twice to bump it from like uid=308 to 312 or something like that
<arianvp>
When DynamicUser is enabled, the behaviour of StateDirectory changes such that systemd does the right thing
<arianvp>
when the service isn't running, the state directory is owned by `nobody`
<andi->
Does it assign the same uid on next boot? Does it uid-map the state?
<arianvp>
and when it is running, it's bind-mounted with the right uid through user namespaces
<ekleog>
(and yeah it can't be DynamicUser because rss2email looks for its own username, or something like that -- anyway it traceback'd when run as DynamicUser)
<arianvp>
andi-: no it uses user namespaces to accomplish this
<andi->
OK, so maybe we need another flag for the state dir?
<andi->
Would probably be accepted upstream if it us indeed the right meaning of the feature
<arianvp>
yeh it's exactly developed for the problem of "we need to preallocate uids"
<arianvp>
afaik
<andi->
I mean the storage part :)
<andi->
Cab try to have a look in the evening if some time frees up
<arianvp>
`man systemd.exec` and then the `DynamicUser=` section and the `StateDirectory=` section
periklis has quit [Read error: Connection reset by peer]
periklis has joined #nixos-security
<andi->
Oh
<andi->
So fixed? :)
<arianvp>
yeh systemd will create the statedir
periklis has quit [Remote host closed the connection]
<andi->
Does it create a structure there that different from running without those two parameters? How does a migration look like.
<andi->
?
ekleog has quit [Quit: WeeChat 2.2]
erictapen has joined #nixos-security
ekleog has joined #nixos-security
__Sander__ has joined #nixos-security
erictapen has quit [Ping timeout: 252 seconds]
ckauhaus has joined #nixos-security
<arianvp>
andi-: Yes instead of running in /var/lib/<state-directory-name> it creates a directory in /var/lib/private/<state-directory-name>
<arianvp>
and then /var/lib/private/blah is mounted as /var/lib/blah in the service
<arianvp>
and user-namespace mapping is done such that blah is owned by the dynamically allocated uid
<arianvp>
this means, for migration, we need to copy /var/lib/blah to /var/lib/private/blah
<arianvp>
and make it owned by the nobody group and user iirc
<arianvp>
instead of by the pre-allocated uid and gid that we had before
<andi->
I thought that iwas what StateDirectory= could fix? E..g whenever we start adding DyanmicUser=true we also put a StateDirectory=/var/lib/<service-name> ?
<arianvp>
no
<arianvp>
StateDirectory=servicename
<arianvp>
StateDirectory doesn't take an absolute path, but a relative path
<andi->
ahh, well then that would still need a nice patch to fixup that..
<arianvp>
and it either will create a directory in /var/lib/private or /var/lib based on whether DynamicUser is true or false
<andi->
StateDirectoryRoot=/var/lib ? :D
<arianvp>
nooo it _has_ to do this
<arianvp>
because otherwise you're vulnerable to uid reuse
<arianvp>
the whole idea is that state is not owned by any UID, and only mapped to the right UID when the service is started up, using a bind-mount
<ekleog>
arianvp: there are tricks about the owning user & group: I still have a directory with uid/gid 61580 from my tests, in /var/lib/private
<andi->
yes but the directory should survive between two reboots shouldn't it?
<arianvp>
no
<andi->
I don't see how that extra dir would preven tsuch issues
<andi->
ahh
<arianvp>
/var/lib/service-name does _not exist_ on disk
<arianvp>
it's only a bind-mount when the service is started up
<arianvp>
and /var/lib/private/service-name is persisted
<arianvp>
and is not associated to any UID
<ekleog>
(said tests being to run systemd-spawn or something similar to check whether it actually wasn't vulnerable to uid reuse, because what's claimed in the manpage isn't sufficient to ensure security -- but it looks like something does prevent it anyway)
<ekleog>
also, it's not a bind-mount but a `ln -s`
<arianvp>
ah yeh, symlink then
<ekleog>
(or at least there's a `ln -s` outside of the thing, maybe there's a bind-mount over it in a mountns, idk :p)
<ekleog>
but yeah, andi- , the description thy give in the manpage really makes no sense, esp. when considering it *leaves* the symlink (iirc without reowning it to root) when the service goes down…
<ekleog>
but somehow it appears to not have collided even once when running systemd-run in a loop for a night, so…
<arianvp>
I do rememnber Lennart saying 2 years ago that it's still a bit finnicky
<arianvp>
oh it keeps the owner or <random-uid> around after stop?
<ekleog>
yeah
<arianvp>
that sounds... like a bug
<ekleog>
that's why I was thinking it was a security issue
<ekleog>
but I can't get it to reallocate, and after a reboot it appears to reset to root, so it may be storing the association in ram
<arianvp>
It does say in the manpage that the system only works under the assumption that nothing else allocates in the uid pool it reserves
<arianvp>
so maybe it just makes sure when there is already a uid it picks uid+1 ?
<arianvp>
maybe it's still a bit too early to start converting all our units to DynamicUser ...
<arianvp>
no other distro is doing it so far.
<arianvp>
but the question is: are they doing it because they don't trust it yet; or because it's a PITA to migrate?
<ekleog>
reproducer: sudo systemd-run --pty --property=StateDirectory=hello --property=DynamicUser=yes sh
<ekleog>
ctrl-d and look in /var/lib/{hello,private/hello}
<arianvp>
yeh so /var/lib/private/hello is owned by 61917 after exit indeed
<arianvp>
I see
<ekleog>
my testbed was while systemd-run --pty --property=DynamicUser=yes cat /var/lib/ekl-test/test |& grep 'Permission denied' > /dev/null; do echo -n '.'; done
<arianvp>
but shouldn't each systemd-run allocate a new uid?
<ekleog>
a night and it appears to not have stopped
<arianvp>
now it keeps recycling the old one indeed.
<ekleog>
but it somehow stopped working
<ekleog>
so yeah, I think there's a trick like it's scanning /var/lib/private for used UIDs and never reusing the UIDs without rebooting
<arianvp>
but... for me it's actually reusing the UID
<ekleog>
(the /var/lib/ekl-test/test had been created with a --property=DynamicUser=yes --property=StateDirectory=ekl-test)
<ekleog>
when you still pass the StateDirectory, or not?
<arianvp>
when I still pass StateDirectory
<arianvp>
oooh that makes sense of course
<arianvp>
:P
<ekleog>
heh ^^
<ekleog>
now I wonder how it would turn out if you *change* the set of StateDirectory / LogDirectory / the like with overlaps
<arianvp>
Yeh if both a.service and b.service use StateDirectory=foo
<arianvp>
Lets try :P just add --unit=a.service to systemd-run
<ekleog>
but it isn't relevant to nixpkgs' security nor likely most systems' security and is less trivial to test than a bash loop so I didn't (a)
<arianvp>
so yeh that does raise the question
<arianvp>
if we want to migrate an existing nixos servide to this modle
<arianvp>
Do we just sudo mv /var/lib/blah to /var/lib/private/blah
<arianvp>
and that's it?
<arianvp>
or do we need to set uids and gids to something in the 60000 range
<{^_^}>
#50112 (by samueldr, 2 days ago, open): Renames stateVersion to stateEpoch
<arianvp>
makes sense
<arianvp>
First time I used nixos I thought you had to update by changing the stateVersion variable :P
<ekleog>
same here ^^'
<ekleog>
and a friend just removed it thinking it was useless, just to see postgresql breaking on him during an update
<arianvp>
Perhaps we can write an RFC for changing to DynamicUser? as it is a change that would touch almost everything, and might need to happen i nstages
<gchristensen>
yea!
<ekleog>
methinks this can be started without an RFC and should be turned into a RFC once the cooking RFC-process RFC gets done
<gchristensen>
hard +1!
<gchristensen>
it is about done fwiw -- globin and I are going to submit it after I can type again
<ekleog>
:DD
<arianvp>
One issue is that many modules either abuse users.homeDir, systemd.tmpfiles or activationScripts to create their state directory
<arianvp>
instead of using StateDirectory
<arianvp>
so that's step one probably, and that can happen already without anything DynamicUser
erictapen has joined #nixos-security
<arianvp>
ekleog: what happens if you put DynamicUser=false afterwards? does it migrate back to /var/lib/<service-name> ?
<ekleog>
so we do need this stateVersion/Epoch bump :/
<arianvp>
understood
<arianvp>
Seems a bit odd though. Might not hurt filing that as an issue with systemd
<arianvp>
because that behaviour is not documented and might even be undesirable
<arianvp>
nah wait I think it's the expected behaviour
<ekleog>
well, even if it's changed we can't rely on it, as older versions would likely have older versions of systemd
<arianvp>
DynamicUser only works if the User= and Group= are names that _are not_ associated to a uid yet
<arianvp>
if no such user/group exists, it's unclear how to "roll back"
<ekleog>
and I don't really know whether it should be changed, actually
<ekleog>
… wait, what?
mmercier has joined #nixos-security
<arianvp>
yes... DynamicUser doesn't work if there is already a uid attached to the user
<arianvp>
it says it in the docs at least :P
<arianvp>
so we'll have to remove "users.users.<service-name>" as well
<ekleog>
oh, yes
<ekleog>
but it's useless to use User with DynamicUser, AFAIU
<ekleog>
~ sudo systemd-run --pty --property=DynamicUser=yes --property=User=iamatest --property=StateDirectory=ekl-test-2 whoami <-- /run/current-system/sw/bin/whoami: cannot find name for user ID 63039
<arianvp>
no it's not. some services expect a certain username for example
<arianvp>
wut
<ekleog>
well, yeah, that's exactly the reason why rss2email isn't a DynamicUser in its current proposal :D
<arianvp>
When used in conjunction with DynamicUser= the user/group name specified
<arianvp>
is dynamically allocated at the time the service is started, and released
<arianvp>
at the time the service is stopped — unless it is already allocated
<arianvp>
statically (see below).
<arianvp>
this behaviour doesn't match what is documented :P
<ekleog>
so that's definitely a systemd bug, I guess
<ekleog>
I'll let you report, it's time to sleep here :p
<arianvp>
yeh will do. sleep well!
<ekleog>
feel free to poke me in the issue :)
<ekleog>
(@Ekleog)
<ekleog>
'night!
<{^_^}>
Night!
<gchristensen>
where are you that it is night time?
<ekleog>
japan, currently :)
<gchristensen>
ah!
<ekleog>
(still ~2 months left :D)
erictapen has quit [Ping timeout: 268 seconds]
mmercier has quit [Quit: mmercier]
mmercier has joined #nixos-security
<arianvp>
Okay apparently we misconfigured systemd according to lennart