The notion that you need more bandwidth between GPU clusters is a tax on stupidity, nothing more. Almost no one sees this clearly, because most engineers have never designed anything from scratch in their lives. Their brains are trained on iteration and refining someone else’s idea. When that’s all you’ve ever done, the big picture is invisible to you by construction. These people were never on the frontier so I wouldn’t take their advice on where that frontier is going.
