Recent searches
Search options
I got my delivery of my new #kubernetes nodes (#minisforum #ms01). Now I'm waiting for ram and cables. Rather than decide between using the 10Gbit SFP+ ports and the USB-4 ports for the ring network (for #Ceph), I'm just going to set up both. That should give me 30Gbit between each node without using a switch.
Now I need to decide which OS I'll use. Stick with #RHEL 9, or change to #RockyLinux or #Fedora?
This is what the new cluster looks like cabled up. Sexy, eh?
For the OS, I decided to go with RHEL 9.5. There are reasons. Red Hat really made some super dumb moves around CentOS Stream, but I don't think they are evil.
Setting up the routing for the ring topology network using #FRR seems to work. I'm getting about 26.6Gbit/sec across the Thunderbolt 4 interfaces. That is using ipv6. ipv4 is not quite working on those for me. And that is my blocker for #rookceph.
For #rookceph, I first tried to use #multus for the cluster networking. While I got it to work on my test pods, I could not get it to work through the cluster.yaml config for Rook. The issue being that it needs a single interface to tie into and my FRR ring topology has two NICs per node. I've given up on that and switched to provider.host while specifying the internal cluster network. My hope is that will work after I get my ring net working with ipv4. Uncharted territory.
@makuharigaijin 26Gbps. Nice. Measured with iperf?
@benoit Yeah, iperf3