samueldr changed the topic of #nixops to: NixOps related talk | logs: https://logs.nix.samueldr.com/nixops/
{^_^} has quit [Remote host closed the connection]
{^_^} has joined #nixops
myskran has quit [Ping timeout: 256 seconds]
psyanticy has joined #nixops
myskran has joined #nixops
abathur has joined #nixops
myskran has quit [Ping timeout: 265 seconds]
myskran has joined #nixops
abathur has quit [Ping timeout: 265 seconds]
dhess has joined #nixops
<dhess> Anyone around? I'm running into a really strange NixOps issue related to S3 binary caches
<gchristensen> ohL
<gchristensen> ?
<dhess> hi gchristensen :)
<dhess> got a few min to try to help me figure something out?
<gchristensen> I can try..!
<dhess> thank you kind sir
<dhess> so we have a private S3 Nix binary cache. It works great. Our hydra pushes products to it, and our Macs use it as a substituter
<dhess> We can also use it as a substituter on our NixOS machines
<dhess> Until we try to nixops deploy to them from the Macs, that is
<dhess> as soon as I enable the S3 Nix binary cache on the NixOS machines, the next nixops deploy gets erros like these:
<dhess> error (ignored): AWS error fetching '4x1ibz5ha6q4vsyggs5chsiqyfmr7m65.narinfo': Access Denied
<dhess> On the Mac that's trying to deploy, I'm seeing processes like this: nix-copy-closure --to root@foo.example.com /nix/store/i0sprws679zg1qpavfjymc1fgfamqdpi-nixos-system-foo-a-20.09pre-git --use-substitutes
<dhess> (the hashes don't match there... these are examples from 2 different attempts so don't mind that)
<dhess> now the AWS creds to read the S3 nix binary cache on the Mac are owned by root because it's the one running nix-daemon. (These are multi-user Nix installs on the Macs BTW)
<dhess> and my user account that's running the nix-copy-closure process doesn't have any AWS creds
<dhess> so I guess that makes sense, my user account is trying to read from the S3 binary cache I guess?
<dhess> but then I add creds to enable that and it still doesn't work
<dhess> because some of the narinfo files it's trying to read don't exist :(
<dhess> it's really strange
<dhess> as I said this *only* happens once I enable the S3 binary cache on the target NixOS host. If it's not enabled on the NixOS target host, everything works fine.
<dhess> any thoughts?
<dhess> as a workaround, I'd be happy to just disable whatever it is that NixOps is doing to try to be clever and copy the closure from the S3 binary cache
myskran has quit [Quit: myskran]
<gchristensen> hrm
<dhess> it seems to me that NixOps should probably tell the target host to grab the derivations from the S3 binary cache itself, rather than copying them from the deployment host. Maybe this has to do with hasFastConnection or whatever that setting is?
abathur has joined #nixops
abathur has quit [Quit: abathur]
abathur has joined #nixops
psyanticy has quit [Quit: Connection closed for inactivity]
mikky has joined #nixops