gchristensen changed the topic of #nixos-borg to: https://www.patreon.com/ofborg https://monitoring.nix.ci/dashboard/db/ofborg?refresh=10s&orgId=1&from=now-1h&to=now "I get to skip reviewing the PHP code and just wait until it is rewritten in something sane, like POSIX shell. || https://logs.nix.samueldr.com/nixos-borg
<MichaelRaskin> Hm. You do have the rules to accept the traffic of replies?
<ekleog> (the `-A nixos-fw -m conntrack --ctstate RELATED,ESTABLISHED -j nixos-fw-accept`)
<ekleog> also, did you run that by hand? nixos adds a `-A nixos-fw -j nixos-fw-log-refuse` if I can trust my `iptables-save`, so if you ran that after nixos added this rule it won't be taken into account
<gchristensen> I didn't run it by hand
<gchristensen> but I think looking in dmesg has a hint for me ...
<gchristensen> [88829.150585] rejected connection: IN=bond0 OUT= MAC=0c:c4:7a:e5:48:00:f4:cc:55:b4:d8:00:08:00 SRC=147.75.199.209 DST=10.99.147.129 LEN=60 TOS=0x10 PREC=0x00 TTL=61 ID=39240 DF PROTO=TCP SPT=57438 DPT=4369 WINDOW=29200 RES=0x00 SYN URGP=0
<gchristensen> I'm whitelisting the private IP and it is talking from the public IP
<MichaelRaskin> I guess a bit of ip route on the other side is needed
<gchristensen> the routes are working I think?
<gchristensen> but the primary IP on the interface is the public IP
<MichaelRaskin> ip route entries can include src
<gchristensen> ohh
<MichaelRaskin> «Ohh» is what you are supposed to say when you discover ip rule !
<gchristensen> scaling out this rabbitmq cluster is going to end me
<ekleog> nah, `ip rule` is « Oooh » (with horror in the voice)
<MichaelRaskin> ip rule is nice. As long as you have less than twenty custom rules…
<ekleog> well, let's agree on the fact it solves issues almost no other tools allow to take care of? :) (we've had issues with only 3 tables, that said I never really understood how the three tables actually did approximately what we wanted, as it was mixed with layers of iptables-level NAT and of level7 redirections)
<ekleog> s/level7/layer7/ actually
<MichaelRaskin> Well, I just used it to manage routing on a server connected via two ISPs at once.
<MichaelRaskin> Then the network admins got an AS of our own, and there was much rejoicing once the routing was actually up.
<ekleog> indeed that sounds better :) (we used ip rules on a front-end server, vps, that had access to the private network via a vpn, and depending on which direction packets flowed we either proxied at nginx level, or did PAT, but the issue was mostly handling SNAT-or-not and giving free access to the internet from some servers on the private network -- esp. the matrix server)
<ekleog> (oh, and to make things easier, some connections were proxied over the VPN, and some connections were proxied over one of the few open ports we had been allowed to use)
* ekleog still doesn't understand how that mess ever worked
<MichaelRaskin> ekleog: well, ip rule multihoming is something I could write in 10 lines of scripts, routing for AS took… more effort
<MichaelRaskin> (not my effort, although at some point I was used for rubber-ducking some of the plans)
<ekleog> oh, I wasn't thinking about the “technical” better, mostly about the “end-user” better (like, actually having a single IP) :)
<MichaelRaskin> Well, it is technically better once it is running
<MichaelRaskin> Failover is actualy automatic…
<ekleog> (hopefully you didn't run a full-route ASes and just considered both upstreams had 0.0.0.0 available?)
<ekleog> oh, I thought you had a ping-ing loop that switched on failover
<ekleog> s/ASes/AS/
<gchristensen> so
<gchristensen> how can I make nixos setup them routing rules?
<ekleog> looks like `networking.interfaces.<name?>.ipv4.routes` is missing a `.src`, so I guess that'll be in something like networking.extraCommands… that doesn't exist either, so networking.firewall.extraCommands?
<gchristensen> yeah :/ I thought so
<ekleog> well, best way is maybe to just add the `.src` to `networking.interfaces.<name?>.ipv[46].routes`, but ISTR from my time mingling with it that it had two backends, and thus was a bit of a PITA to change
<MichaelRaskin> Frankly, I would put ip route rules into the long list of things you need to understand in advance, so putting such stuff in extraCommands sounds like the only sane solution
<ekleog> issue is, one of the backends is systemd, so I'd guess it doesn't accept just adding stuff to extraCommands
<ekleog> (and pushing it to the firewall sounds wrong, like, likely to break at random times)
<ekleog> ooooh nvm
<ekleog> networking.localCommands
<ekleog> just wasn't called extraCom
<ekleog> just wasn't called extraCommands
<gchristensen> well, one day, one day I'll get a second core node up
<ekleog> just to sum up, networking.localCommands = "ip route add 10.99.147.129 [the same dev and/or via as in `ip route`'s output] src 10.99.147.131"; should do it for the routing issue :) (and good luck and good night!)
<samueldr> 'evening
<samueldr> I pushed the fix I briefly talked about this morning https://github.com/samueldr/github-to-irc/commit/8ebc1fa35466ac5fdf5ca3141d89fcac68c322a1
<samueldr> nothing like the fresh smell of deleted code
<samueldr> looking at this morning's log, it would have reduced {^_^} spam from 24 to 9 lines
<infinisil> samueldr: Very nice!
jtojnar has quit [Remote host closed the connection]
jtojnar has joined #nixos-borg
jtojnar has quit [Quit: jtojnar]
jtojnar has joined #nixos-borg
orivej has quit [Ping timeout: 255 seconds]
<infinisil> samueldr: Um, it's back like it was before now
FRidh has joined #nixos-borg
MichaelRaskin has quit [Quit: MichaelRaskin]
FRidh has quit [Quit: Konversation terminated!]
orivej has joined #nixos-borg
orivej has quit [Ping timeout: 260 seconds]
orivej has joined #nixos-borg
<samueldr> infinisil: I don't host the service, it will need to be deployed by gchristensen at his convenience; I only updated the code and announced I did
<infinisil> Oh really? I remember the bot not outputting 4 messages on each merge yesterday
<samueldr> if it did, it wasn't me :)
<samueldr> looking at the #nixos log
<samueldr> push events with one commit won't output additional lines (it happened)
<infinisil> Ah I see
FRidh has joined #nixos-borg
orivej has quit [Ping timeout: 260 seconds]
FRidh has quit [Quit: Konversation terminated!]
MichaelRaskin has joined #nixos-borg
orivej has joined #nixos-borg
orivej has quit [Ping timeout: 240 seconds]
infinisil has quit [Quit: Configuring ZNC, sorry for the join/quits!]
infinisil has joined #nixos-borg
infinisil has quit [Client Quit]
infinisil has joined #nixos-borg