ChanServ changed the topic of #nixos-systemd to: NixOS <3 systemd | https://jitsi.nixcon.net/systemd | Next meeting 08.12.2020 14:00 UTC (every two weeks)
andi- has quit [Remote host closed the connection]
andi- has joined #nixos-systemd
<damjan>
andi-: if systemctl is invoked, it needs to be invoked with --no-ask-password
<andi->
damjan: then we must be doing that wrong since basically forever
<damjan>
afaik, systemctl always starts the ask password agent, in case a service *does* need a password
<andi->
how does that even work? Is there some service that requires passwords?
<damjan>
which process exactly are you seeing?
<andi->
systemd-tty-ask-password-agent --watch
<damjan>
right, the agent
<damjan>
so, if a service does require a password, it'll send a request to all the agents
<andi->
Ok, so maybe my issue wasn't that the agent was running then.
<andi->
Or very likely it wasn't the issue.
<damjan>
probably not
<andi->
We need a better story around restarting a list of services and honoring their dependencies. I fear it might just be another case of that.. Some service expecting something (as it is scheduled After=… or Requires=…) but systemctl restart -- <all the services> doesn't really honor dependencies.
<damjan>
maybe use `isolate` to basic.target then back?
<andi->
We tried that but that would stop X11 sessions etc..
<andi->
like someone experimented with using a target for each system generation
<andi->
IRRC when you switch between them it tears down and restarts everything (even those that weren't changed)
<damjan>
andi-: so you (not personally) want to restart all except the X11 session services? hmm. might be a bit hard to express that one. why not just require a reboot by the user?
<andi->
a reboot on all config changes?
<andi->
So change a whitespace in the nginx configuration -> reboot.. I would probably stop using NixOS or would never have started to use it.
<damjan>
:)
<damjan>
so does nix have the information about those service dependencies?
<damjan>
maybe it can sprinkle drop-ins with PartOf= directives so that restarting a service restarts all that depend on it
<andi->
Well we already declare the service dependencies via the WantedBy, After, … fields
<andi->
so IMHO systemctl restart should be fixed to also honor dependencies between those services that you have given on the CLI and not just process them in order.
<andi->
We have a common issue where we request letsencrypt certificates but those units should only be started after the nss-lookup.target has been reached. If you have your dns resolver in the nss-lookup.target but restart both the resolver and the letsencrypt service the startup order is not correct as the nss-lookup.target is already reached and systemd happily start the LE unit before the
<andi->
resolver is back up.
<andi->
We could special case this but that would mean more downstream code..
<damjan>
alas After/Before are not dependencies. just ordering
<damjan>
now WantedBy= is one type of dependency, but PartOf=/BindsTo= another, and they are separate because there are seprate needs
<andi->
I know but in reality it makes for a good enough approximation most of the time.
<andi->
So, if I would declare everything that usually has After=network.target to (additionally?) BindTo=network.target that would be more correct?
<andi->
If that is the case I assume most/all of the units out there in the wild (not just on NixOS) are wrong.
<damjan>
most services won't need BindsTo=network.target
<damjan>
because they would not fail without it
<andi->
if I tear down networking the webserver doesn't have the required interfaces anymore.
<andi->
so if I stop networking the webserver *must* go down, no?
<damjan>
andi-: well not neccesseary
<damjan>
in the end, it's up to the system admin to decide
<andi->
ok
<andi->
Lets go back to the situation with systemctl restart <all the things>. If I have a service that is PartOf=nss-lookup.target and another service that is After=nss-lookup.target the 2nd service is allow to start before the first one is back up, right?
<andi->
Because the target has already been reached and a service in there was "just" restarted which doesn't matter.
<damjan>
right
<damjan>
if you need that level of control, you'd need to stop the target
<andi->
So, what I'd like to have is a way that doesn't stop the 2nd service when the first one restarts but required the target (nss-lookup) to be fully reached (as in no failed/stopped services) before it goes.
<andi->
*up
<andi->
Right now it feels like we have to script that out in your profile activation script.
<andi->
Possibly reimplementing a lot of the systemd internal code.
<damjan>
seems like overengineering to me
<damjan>
I mean, nss-lookup.target doesn't guarantee that the DNS server is still operational, so in that sense it's never a guarantee
<damjan>
and services need to be prepared to work with that
<andi->
If there is no guarantees why do we even bother with stuff like Type=notify etc..? if we could just tell everyone to retry until it works.
<andi->
It seems like we have the assurance of best effort & maybe working which doesn't make that much of a difference
<andi->
Also I am not saying systemd does it wrong right now. I am trying to find a solution that doesn't just work for us but is a sane way to deal with this without rewriting all the software.
<andi->
I actually tried to fix the letsencrypt issue upstream but apparently they would rather not do it that way as "DNS just works" or whatever
elvishjerricco has quit [Ping timeout: 260 seconds]
globin_ has quit [Quit: o/]
globin has joined #nixos-systemd
ckauhaus has joined #nixos-systemd
globin has joined #nixos-systemd
globin has quit [Changing host]
elvishjerricco has joined #nixos-systemd
Emantor has quit [Ping timeout: 260 seconds]
Emantor_ has joined #nixos-systemd
<flokli>
andi-: the whole discussion here is basically again about "how to fix our activation script"
<flokli>
something I'd like to have a discussion with aanderse too ;-)