Since the single switch was the cause of 3 hours of headaches and could have resulted in data-loss plus lots of additional hours rebuilding everything, I am working on a plan for upgrading my network to be both faster and more redundant within the cluster.
Materials Needed:
- 2 x MikroTik CRS510-8XS-2XQ-IN (2x 100Gbit 8x 25Gbit ports)
- 5 x Intel E810-XXVAM2 25Gbe Network Cards
- 6 x 1 Meter 25Gbit Twinax Cables
- 4 x 2 Meter 25 Gbit Twinax Cables
- 2 x ASRock B550M Pro4 Motherboards
This will upgrade the aggregate network speed to each host to 50Gbit/s in a bonded pair with one connection to each of the two switches. It also adds redundancy if one switch fails the link is simply reduced to half that speed rather than VMs crashing and data corrupting.
I will have to upgrade the motherboards in the NAS and Proxmox Backup Server. The intel E810 NIC requires a PCI-E 4.0 connection with 8 lanes minimum. The boards in those to machines are B450 boards which don’t support PCI-E 4.0.
The total cost as of this writing for the entire upgrade is roughly $4,200. I will have to budget and save for that but keeping my production cluster healthy is important since some of my Income depends on keeping it up and running.
Leave a Reply