hexa- has quit [Quit: WeeChat 2.9]
hexa- has joined #nixos-kubernetes
devhell has joined #nixos-kubernetes
<devhell> heya, can anyone enlighten me please as to how to get a Ceph RGW working properly. I've deployed my cluster, all systems are fine, MON, MGR, MDS, all good. I've tried deploying RGW to one of the nodes, and used https://access.redhat.com/documentation/en-us/red_hat_ceph_storage/3/html/installation_guide_for_red_hat_enterprise_linux/manually-installing-ceph-object-gateway to bootstrap the RGW
<devhell> service. Specifically, the 'ceph auth ...' part, which worked fine. When I restart the RGW systemd service it errors out saying: "... -1 monclient(hunting): handle_auth_bad_method server allowed_methods [2] but i only support [2]" and "failed to fetch mon config (--no-mon-config to skip)
<devhell> they keyring is in the right place
<devhell> both, ceph.client.admin.keyring and the one I had it fetch and place in /var/lib/ceph/radosgw/ceph-foobar-node1/keyring
<devhell> checked the perms which seem to be fine
<devhell> is there something I should be adding to the RGW nixos configuration?
<devhell> right now it just enables RGW on my nodes
<devhell> alright, there is also a different issue, when rebooting the cluster, /var/lib/ceph/osd/ceph-$i/ is empty
<devhell> and the systemd service for each ceph osd fails because keyring for that osd is missing in that directory
<devhell> so /var/lib/ceph/osd/ceph-0/ is now empty after reboot, and ceph-osd-0.service is failing because keyring is missing
<devhell> I'm sure I've forgotten something, but I can't put my finger on what.
<devhell> ah, looks like the OSDs need to be manually activated or via a custom systemd unit...
<devhell> okay, the OSD stuff is fine now
<devhell> however RGW still won't work :/