<gchristensen> andi-: no luck , borrowing your config
<gchristensen> starting up wireshark on a pcap ...
<Shados> I'll need to start playing around with IPv6 setup soon too... my isp apparently assigns a single address by dhcp, along with a /56 prefix delegation
<clever> 2019-04-30 19:36:18 < pterp> I'm trying to get the initrd to load on multiboot2, but it complains with a kernel not found error, even though boot works. The module command is "module /boot/initrd initrd" and it is after the command to load the kernel. What's happening?
<clever> 2019-04-30 19:39:28 < zid> have you tried putting it... before?
<clever> Shados: might this be related to some issues you had?
<clever> 2019-04-30 19:40:24 < pterp> before? no. I'll try.
<clever> 2019-04-30 19:41:30 < pterp> Ah. I just realized I should use module2 when using the multiboot2 spec.
<gchristensen> Shados:what is your ISP?
<Shados> clever: No, I was using module2, as per the stuff I linked. I haven't tried putting the module2 commands prior to the multiboot2 commands in the menu entry, but I think that would either make no difference or not work... I'm pretty sure Xen is actually getting the modules, it complains if it doesn't.
<Shados> gchristensen: Aussie Broadband
<clever> Shados: ah
<gchristensen> ah
<clever> something interesting, is that my ISP doesnt have v6 at all
<clever> but strangely, my dads ISP has native v6, and the v4 dhcp server keeps crapping out, leaving him with no dns and half the devices dead :P
<Shados> Actually had IPv6 working with my previous ISP, but they got eaten by a foreign company, then their residential fixed-line division was sold off to another company. Everything basically went to shit, lost my static IP, IPv6 support was dropped, etc. Chose a different ISP when I moved to a new apartment -_-.
<gchristensen> how novel, "choice"
<Shados> Yeah, we have choice again, for now.
<clever> we also get choice up here, 2 or 3 people to pick between
<Shados> Until our government once again sells of their telecom monopoly and we go back to getting reamed...
<gchristensen> I get the "choice" between Spectrum Cable or ...
<clever> but ive heard how aussie land ISP's force eachother to cap their clients
<gchristensen> Dialup
<Shados> No?
<andi-> 3G? 4G? :-)
<Shados> The situation here is very complicated. Could give you guys a deep dive on it if you want. Fuck all to do at work currently, kinda worryingly quiet the last few months :o.
<gchristensen> andi-:lol.
<Shados> gchristensen: Satellite?
Guanin_ has quit [Quit: Leaving]
<Shados> Fixed wireless?
<Shados> USB drive by carrier pidgeon?
<gchristensen> fixed wireless would take 5 hops to get to a decent connection
<gchristensen> satellite is terrible
<andi-> Everyones favorite topic.. Internet connections in different areas... I used to say "build your own". Did that and quit networking entirely.. It is an expensive undertaking here and it doesn't pay of to do all of that juts for a few residential customers.
<gchristensen> I tried to actually
<gchristensen> that is why I know it is 5 hops :)
<andi-> Satellite really depends on your use case and the ISP... Did that for a few years and it can be decent for standard "browsing" but SSH is unuseable. Even with mosh it isn't fun.
<Shados> andi-: For now, anyone and their dog can start an ISP here. NBN has made things pretty easy in that regard by being a universal middleman. Although if you wanna do things well and/or have any real control over service, you still end up needing to run your own fibre between POIs...
<andi-> Shados: being an ISP means running my own cables/last mile connections/last mile wireless/… to me
<andi-> all the other stuff is just moving numbers around
<Shados> Yep. NBN now does the last mile for everyone, almost, kinda.
<andi-> Had a 3y law-suite with my city regarding accessing their infrastructure... We've a law over here (Germany) that allows public telecommunication companies to share infrastructure to lay cables throughout the city... They are really trying to make it easy by passing proper laws but then the actual decision makers are not complying.
<clever> in this region of canada, there are 2 main internet providers, aliant and rogers
<clever> aliant originally did DSL (and was also the phone company), but now they are switching to fiber, and 1 fiber does tv+phone+internet
<clever> rogers was originally only tv service, but have since branched out into internet via cable modems
<clever> so both companies had pre-existing last-mile infra, and just had to upgrade the hardware to allow more over the existing wires
<andi-> typical DOCSIS model.. they paid for the cables once and can now add some more "value" to thei investments
<clever> until alient switched to fiber, and redid the entire network
<gchristensen> and that is how you get internet over MPEG...
<andi-> MPEG-TS ;-)
<clever> but funnily enough, alient is now reusing coax for the "last 20 feet"
<andi-> 166 bytes at a time
<gchristensen> aye
<clever> aliant*
<clever> the problem, is that they now have iptv boxes, that need ethernet
<clever> but most houses lack ethernet infra
<Shados> ...Fibre to the curb, seriously? You guys too?
<clever> Shados: fiber to the furnace room :P
<clever> so, they use HPNA, which is ethernet (100mbit?) over coax
<andi-> fiber to the basement.. and through the entire house.. Took me 5 days to lay it all down and splice..
<Shados> If it is at least terminating in the basement, that's not too bad
<clever> so, the "router" can take the signal from the modem, and then inject ethernet into your house's existing coax infra
<clever> and then the STB's can connect to the coax, and get mpeg-ts/rtp packets
<clever> Shados: i believe its called a passive optical network, 6 modems feed into a single passive optical splitter/merger, and then 1 fiber goes to a headend port on a nearby switch
<Shados> Our government has done progressively more insane things basically for the sake of having a different plan than the opposition, which has resulted in "fibre to the curb", wherein they roll individual fibres up to the street outside your house, then connect a terminating device to the fibre and to existing phone line to your house. You then have to power this terminating device... over the phone line... which you also use to talk to it... over VDSL.
<clever> so, within that 1 fiber, you are timesharing the lightwaves
<andi-> GPON... yeah
<Shados> Why time-division multiplexing?!
<andi-> one malicious fiber and a whole street is offline..
<clever> andi-: i think legally, the modem and fiber belongs to the telco, and i can get in major trouble for that :P
<gchristensen> cransom: do you have a dhcpcd config whic does ipv6 to share?
<clever> Shados: downstream traffic is encrypted per-house, except for multicast traffic, upstream traffic is based on timeslots, so the upstream bandwidth is staticly allocated, but downstream is dynamic
<andi-> clever: well, YOU would never do that. ;-)
<andi-> clever: I can tell from *stories* that it is fun to see the technician search through 600 appartments to see which one is bad.
<clever> heh
<clever> but its not 600 way here, just 6 way
<clever> the splitter is just outside, at the top of a nearby pole
<Shados> ._., surely wavelength-division multiplexing would be saner
<Shados> And also not induce unnecessary extra latency
<andi-> Shados: well neither of the multiplexing make much sense. It is a <5$ cost difference in hardware + a few cents per meter per customer
<clever> 64 bytes from 142.166.182.21: icmp_seq=3 ttl=63 time=3.92 ms
<clever> the ping to a machine 1 hop beyond my router
<andi-> 64 bytes from 1.1.1.1: icmp_seq=1 ttl=58 time=1.43 ms from bed via wifi through a wall..
<clever> 64 bytes from 1.1.1.1: icmp_seq=8 ttl=57 time=27.3 ms
<clever> passing thru the rest of the telco infra and stuff, is where the cost comes
<andi-> cache.nixos.org (fastly) in about 1.7ms :-)
<clever> 64 bytes from 151.101.126.217: icmp_seq=2 ttl=52 time=39.7 ms
<clever> cache.nixos.org ^^
<cransom> the only hitch that ive found, that i ahve no tracked down, is that something, somewhere likes to remove the link local address from my wan interface, which confuses dhcpcd thoroughly and breaks ipv6. add one back in and it figures it out.
<clever> the other weird thing with my fiber link, is the vlans
<clever> the fiber goes into a device i'm calling a "modem"
<clever> its technically a router, but the ISP disabled all routing features, and i have no way to control it :P
<cransom> is it gpon?
<clever> its a box with 1 GPON fiber input, 2 phone jacks, and 4 ethernet jacks
<clever> but, only 1 phone and 1 ethernet work, and there is no admin IP available
<gchristensen> this is driving me batty :(
<clever> that "modem" also has a dedicated battery backup box (for 911 reasons)
<Shados> haha
<clever> and if the battery starts to run low, it disables the ethernet port, to keep the telephone port alive
<Shados> we have a similar termination device/modem for fibre to the premise deployments
<clever> coming out of the ethernet port, is 802.1q tagged vlans
<Shados> although you can actually make use of the extra ethernet ports by hooking up separate connections to them -- which is pretty useful if you want to try out another ISP, or add some redundancy
<clever> vlan 34 is iptv service, dhcp gives you a 10.x.y.z IP
<clever> subscribe to the right multicast group, and the ISP practically DoS's you with an rtp stream containing mpegts :P
<clever> if your on wifi, it can bring your wifi down
<Shados> yeah don't do multicast over wifi
<clever> its more to do with the bandwidth then the protocol
<clever> i only have 54mbit wifi
<clever> second main vlan, is vlan 35
<clever> dhcp gives you a single public ip, and then binds to your mac
<clever> dhcp will then ignore any other mac until you either release the lease, or wait a day or 2
<clever> NAT away and your done
<clever> the router the ISP provides, will create 2 uplinks, over vlan34 and vlan35, then choose which uplink to use, based on a routing table
<cransom> i've wanted a provider with mcast video streams for so long. now i probably don't care, but it would be nice.
<gchristensen> May 01 01:09:41 lord-nibbler dhcpcd[2783]: enp1s0: delegated prefix 2604:6000:e6c1:d400::/56
<gchristensen> !
<gchristensen> andi-: ^
<andi-> \o/
<andi-> what did it?
<gchristensen> copy-pasta'ing cransom's config
<andi-> the indention?
<gchristensen> literally no idea
<clever> the scarry part, is that there is a 3rd vlan i discovered
<clever> which the isp router, will bridge into the internal lan!
<andi-> I was hesitant to ask the earlier because the docuemtnation had that and you didn't but that would be stupid?!
<cransom> i don't think the indentation matters.
<cransom> but, i've never had it unindented. and i'm totally not changing it now because it will then be another 3 hours as i bring it back to life because it failed for other weird reasons
<gchristensen> I hate that this works now
<andi-> can't you use AFL to "bisect" all the valid configurations? ;-)
<gchristensen> lol
<clever> due to stability issues with my isp provided router, i have replaced it with nixos
<clever> but ive not gotten multicast to pass thru nixos
<clever> so i need to run 2 routers on the same modem
<andi-> clever: pass through or routed multicast?
<gchristensen> thank you for your help cransom :x
<gchristensen> I'm glad it works, I just hate that I have no idea why. time to diff.
<clever> andi-: a dumb gigabit switch, that just links both routers to 1 modem
<clever> andi-: and then i intentionally misconfigure the isp router to pppoe mode
<clever> and it then tries to do pppoe over vlan 34, lol
<clever> so the isp router fails to get a public ip, and only the tv vlan works
<clever> and the nixos router does dhcp on vlan34, and gets internet
<clever> and for extra fun, i then run a pppoe server on vlan 34, facing the modem and isp router
<clever> so it can get an IP, and double-nat
<clever> but tv service isnt double-nat'ing
<andi-> how do you even have time for TV? :D
<clever> i dont even turn it on :P
<clever> and every time my dad tries to cancel the tv service, they claim the bill will go up!
<gchristensen> curl 'http://[2604:6000:e6c1:d402:8e89:a5ff:fe10:53f0]:3000' does this work for anyone?
<cransom> nadda
<gchristensen> same, externally
<gchristensen> seems I have more work to do.
<gchristensen> (not just this. for ipv6 in general. computer -> router, router -> interent works, but computer -> internet is broken. but for now, bed.
<cransom> ipv6 forwarding enabled?
<gchristensen> yeah
<cransom> and iptables6 forward rules?
<gchristensen> I believe so
<gchristensen> anyway. tomorrow.
<cransom> it does seem like v6 kernel forwarding didn't get flipped on yet.
lopsided98_ is now known as lopsided98
makefu has quit [Ping timeout: 252 seconds]
<gchristensen> I Have net.ipv6.conf.all.forwarding = 1 and net.ipv6.conf.default.forwarding = 1
makefu has joined #nixos-on-your-router
<gchristensen> so I stumped andi earlier today on why my router doesn't forward ipv6. anyone else want to try? :)
<andi-> do you alos have forwarding on the respective interfaces set to 1?
<andi-> I forgot what the output was from this morning.
<gchristensen> yeah
<gchristensen> I can paste a bunch of info in about 10min
<gchristensen> new and interesting output andi-
<gchristensen> PING ipv6.icanhazip.com(packet-sjc.icanhazip.com (2604:1380:1000:af00::1)) 56 data bytes
<gchristensen> From 2604:6000:e6c1:d403::1 (2604:6000:e6c1:d403::1): icmp_seq=1 Destination unreachable: No route
<cransom> do you clients have a v6 default route?
mmlb39491 has quit [Ping timeout: 276 seconds]
<gchristensen> so this is different from the problem this morning, here comes some info.
<clever> i still need to fully fix my ipv6 config
<clever> its partially manual, and broke recently
<gchristensen> laptop + router information: https://gist.github.com/grahamc/b7addc57b24bca9a0241a71882423faa ( cransom )
<gchristensen> actually, forget that gist. something is "more" wrong than before. let me roll back a few revisions and recreate that.
<cransom> and router's routing table has a default route too? !N is interesting.
<gchristensen> ok I have something similar to this morning going on now
<gchristensen> sudo tcpdump -i enp1s0 -n -vvv ' icmp6' on the router shows pings being forwarded
<clever> gchristensen: it can help to run that on both sides of the NAT
<clever> or in this case, just gateway, no translation with v6!
<gchristensen> enp1s0 is the public interface
<clever> ah
<clever> and is a reply coming back?
<gchristensen> sudo tcpdump -i nougatwifi -n -vvv ' icmp6' also showed the pings :)
<clever> if i'm reading the IP's right, no reply from the far end
<clever> one min
<clever> gchristensen: try to ping 2001:470:1c:19a::2
<gchristensen> no replies
<clever> gchristensen: id say your v6 uplink is bork
<clever> my v6 uplink works, and i can ping6 irc.freenode.net
<clever> but i'm not hearing your pings to me at all
<clever> 21:59:40.727681 IP6 2001:470:1c:19a::2 > 2001:948:7:7::140: ICMP6, echo request, seq 1, length 64
<clever> 21:59:40.918449 IP6 2001:948:7:7::140 > 2001:470:1c:19a::2: ICMP6, echo reply, seq 1, length 64
<clever> thats what i get when i ping6 freenode
<gchristensen> ping me clever ? 2604:6000:ffc0:58:15a3:aaac:9049:776d
<clever> 22:01:00.770702 IP6 2001:470:1c:19a::2 > 2604:6000:ffc0:58:15a3:aaac:9049:776d: ICMP6, echo request, seq 1, length 64
<clever> 22:01:00.866651 IP6 2604:6000:ffc0:58:15a3:aaac:9049:776d > 2001:470:1c:19a::2: ICMP6, echo reply, seq 1, length 64
<clever> works
<gchristensen> lol
<clever> do you see those on your tcpdump?
<gchristensen> yeah22:01:03.824940 IP6 (flowlabel 0x37366, hlim 54, next-header ICMPv6 (58) payload length: 64) 2001:470:1c:19a::2 > 2604:6000:ffc0:58:15a3:aaac:9049:776d: [icmp6 sum ok] ICMP6, echo request, seq 4
<gchristensen> 22:01:03.825047 IP6 (flowlabel 0x99b71, hlim 64, next-header ICMPv6 (58) payload length: 64) 2604:6000:ffc0:58:15a3:aaac:9049:776d > 2001:470:1c:19a::2: [icmp6 sum ok] ICMP6, echo reply, seq 4
<clever> 22:01:17.637559 IP6 2606:a000:4505:1a02:e811:8f49:4c78:4666 > 2001:470:1c:19a::2: ICMP6, echo request, seq 1, length 64
<clever> 22:01:17.637628 IP6 2001:470:1c:19a::2 > 2606:a000:4505:1a02:e811:8f49:4c78:4666: ICMP6, echo reply, seq 1, length 64
<clever> who that? lol
<clever> gchristensen: try to ping me from the router, so no gateways are at play
<gchristensen> I did try from the router ;X
<gchristensen> oh
<gchristensen> no I didn't :|
<gchristensen> see those now? :)
<clever> yep
<clever> and i'm replying
<gchristensen> yep you are
<clever> so it seems to be a problem in the forwarding area, same as me
<clever> my router just stopped forwarding, and i couldnt figure out why, so i turned off radvd
<clever> havent looked into why that deeply yet
<gchristensen> well damn
<gchristensen> paging cransom and andi- :P
<cransom> that was me, getting all excited at pinging things over v6
<clever> cransom: ah
<gchristensen> haha
<andi-> So it somewhat works now?
<clever> andi-: i believe for both of us, it works from the gateway directly, but not any LAN machines
<andi-> Wut
<andi-> So nothing changed
<gchristensen> the one thing I seen ow is a ping seems to be forwarded out my public nic but no reply is received
<gchristensen> 22:14:10.421324 IP6 (flowlabel 0x4be51, hlim 63, next-header ICMPv6 (58) payload length: 64) 2604:6000:e6c1:d403:8ced:1667:6ca4:cc51 > 2001:470:1c:19a::2: [icmp6 sum ok] ICMP6, echo request, seq 4
<gchristensen> ^ Morbo pinging clever
<gchristensen> 22:15:07.186049 IP6 (flowlabel 0x0821f, hlim 64, next-header ICMPv6 (58) payload length: 64) 2604:6000:ffc0:58:15a3:aaac:9049:776d > 2001:470:1c:19a::2: [icmp6 sum ok] ICMP6, echo request, seq 2
<gchristensen> ^ router pinging clever (this one works)
<clever> 22:15:07.230419 IP6 2604:6000:ffc0:58:15a3:aaac:9049:776d > 2001:470:1c:19a::2: ICMP6, echo request, seq 2, length 64
<clever> 22:15:07.230478 IP6 2001:470:1c:19a::2 > 2604:6000:ffc0:58:15a3:aaac:9049:776d: ICMP6, echo reply, seq 2, length 64
<clever> and i saw that one
<gchristensen> note the source IPs are quite different
<clever> gchristensen: sounds like morbo has the wrong ip set on it
<clever> radvd misconfigured?
<clever> dhcpv6?
<gchristensen> well
<gchristensen> one sec
<clever> check `ip addr` on every machine, and confirm they are all part of the subnet your ISP gave you
<gchristensen> right they gave me two!
<gchristensen> May 01 21:48:11 lord-nibbler dhcpcd[1017]: enp1s0: adding address 2604:6000:ffc0:58:15a3:aaac:9049:776d/128
<gchristensen> May 01 21:48:11 lord-nibbler dhcpcd[1017]: enp1s0: delegated prefix 2604:6000:e6c1:d400::/56
<gchristensen> May 01 21:48:11 lord-nibbler dhcpcd[1017]: enp2s0: adding address 2604:6000:e6c1:d402::1/64
<gchristensen> May 01 21:48:11 lord-nibbler dhcpcd[1017]: nougatwifi: adding address 2604:6000:e6c1:d403::1/64
<gchristensen> so the IPs match up, just not sure what to do about this
<clever> radvd doesnt need any special client, the kernel just reacts to its packets automatically
<clever> though i havent configured mine like yours...
<gchristensen> yeah
<gchristensen> radvd is sending out IPs as is appropriate according to the config dhcpcd has setup, afaict
<gchristensen> back in 10min
<andi-> Yeah ::/64 works like one wants it to.. Great simplification :)
<gchristensen> should my public IP be part of that /56 ?
<gchristensen> (of the router)
<clever> probably
<gchristensen> will asigning my public interface a /64 do ...something weird?
<gchristensen> and should it also get the other public IP/
<clever> possibly
<clever> you may be picking IP's you dont "own" and the ISP is filtering the outbound
<gchristensen> I don't think so, because the /128 is assigned and works
<gchristensen> and the /64 is delegated to me by the ISP
<cransom> it probably wouldn't hurt anything, but it would be... 'weird'. if you wanted a /64 for the box, you'd put it on a loopback
<gchristensen> I don't really want a /64 on this box :P
<andi-> You get two addresses.. One /128 for the ISP facing interface. The subent (/56) will be routed towards that address (usually) and you add the subnet to a local interface (as a sink). Add /64's as required to other interfaces.
<gchristensen> yeah, so that is what I have now, right andi-?
<andi-> Yes
<gchristensen> so my packets are getting out but nothing is coming back for my /56
<gchristensen> 22:42:59.002782 IP6 (flowlabel 0xaf4c5, hlim 63, next-header TCP (6) payload length: 40) 2604:6000:e6c1:d403:3936:fec5:982a:b293.42990 > 2604:1380:1000:af00::1.80: Flags [S], cksum 0x7485 (correct), seq 1804100304, win 64800, options [mss 1350,sackOK,TS val 580551052 ecr 0,nop,wscale 7], length 0
<andi-> Did you ever get a reply from your ISP about yesterday's question?
<gchristensen> maximum delegatable prefix?
<andi-> yeah
<gchristensen> they said /52 but I tested it and couldn't get one
<gchristensen> they're worse than useless
<andi-> ok
<gchristensen> they are delegating me a 56 though...
<andi-> fun fact: When I ping 2604:6000:e6c1:d401::1 I get a no route to host back from the IA_NA address to you got leased
<andi-> to that /64 definitely ends up on your router
<andi-> the d402::1 doesn't reply with an unreachable, the d402::2 does again
<gchristensen> d402::1 is my router
<gchristensen> on the enp2s0
<andi-> I know
<andi-> so for whatever reason it doesn't reply back