This article presents the vision of multi-band communication networks (MBN) in 6G, where optical and terahertz (THz) transmissions will coexist with the conventional radio frequency (RF) spectrum. Wel first pin-point the fundamental challenges in MBN architectures at the PHYsical (PHY) and Medium Access (MAC) layer, such as unique channel propagation and estimation issues, user offloading and resource allocation, multi-band transceiver design and antenna systems, mobility and handoff management, backhauling, and so on. We then perform a quantitative performance assessment of the two fundamental MBN architectures, that is, stand-alone MBN and integrated MBN, considering critical factors like achievable rate and capital/operational deployment cost. Our results show that stand-alone deployment is prone to higher capital and operational expenses for a predefined data rate requirement. Stand-alone deployment, however, offers flexibility and enables controlling the number of access points in different transmission bands. In addition, we propose a molecular absorption-aware user offloading metric for MBNs and demonstrate its performance gains over conventional user offloading schemes. Finally, open research directions are presented.