Rethinking Metro Networks

Marc De Leenheer, Open Networking Lab; and Tom Tofigh, AT&T

 

Abstract: To achieve network-wide programmability and observability, open source and vendor-agnostic interfaces are critical. We discuss the re-design of the metro network, demonstrate practical and scalable operational models, and introduce a disaggregated ROADM architecture.

1. Introduction

Metro networks form a key component of service providers’ complex infrastructure. A metro network connects the Central Offices in a metro area and serves as an aggregation network between the access networks and the wide area network. Innovating in the metro is challenging due to the tight interworking between vendor equipment and the control functions. Every vendor offers their own implementation of popular metro services, and requires the service provider to acquire and operate a vertically integrated set of network elements that span the whole metro. This work tries to enable service providers to build their infrastructure using white boxes (both packet and optical), open source software defined networking (SDN) platforms, and a set of network control and management applications. Our approach avoids vendor lock-in and enables service providers, partners and vendors to rapidly innovate on this infrastructure.

This work is in the context of CORD: Central Office Re-architected as a Datacenter [1]. The goal of CORD is to bring the economies of a datacenter and the agility of the cloud to telco service providers. The metro network design builds on this approach in three significant ways: (1) it uses a single SDN control plane built using ONOS to create a converged packet-optical network for the metro area; (2) it demonstrates several innovative network services on this converged network; and (3) it shows how one can build even Reconfigurable Optical Add Drop Multiplexers (ROADM) as white boxes using a set of merchant optical building blocks.

2. SDN Control Plane with Open Source Network Operating System (ONOS)

ONOS [2] is an open-source SDN OS architected to meet the stringent availability, scalability and performance demands of service provider networks. It was built from the ground up with a distributed architecture and introduces innovative state management techniques. ONOS championed the intent abstraction, which frees application developers from the need to specify and control low-level parameters, leaving them for the OS to optimize as network conditions change. Additionally, ONOS offers a converged topology abstraction, in which multiple network layers and technologies are presented as a single logical graph. Finally, ONOS comes with a built-in multi-layer path computation element (PCE), as well as a modular system design allowing external PCEs to be plugged in dynamically.

Figure 1 – (a) Central Office Re-architected as a Datacenter consists of a leaf-spine IP fabric, servers for computational and storage resources, I/O blocks for virtualized access technologies G.fast and GPON, and disaggregated ROADM.
(b) Metro network consisting of Central Offices (CO) each with a local ONOS controller. The global controller maintains the converged packet/optical model and runs advanced applications in support of enterprise connectivity and video distribution.

ONOS’ southbound interface forms a layer of abstraction to handle the forwarding, monitoring and configuration of network elements such as devices, links, and hosts, ensuring no protocol-specific or vendor-proprietary logic leaks into the distributed core. Today ONOS supports a wide range of protocols such as OpenFlow, Network Configuration Protocol (NETCONF), PCE Protocol, Simple Network Management Protocol (SNMP), and several Transaction Language 1 (TL1) implementations from multiple vendors.

3. Metro Services

ONOS has native support for bandwidth on-demand, calendaring and automatic restoration in the optical layer. These features form the foundation to support two of the most popular metro services, which both require tight coupling of packet and optical layer control.

1. Enterprise connectivity has been standardized by the Metro Ethernet Forum, which defined services such as E-Line, E-LAN and E-Tree [3]. Most stakeholders have eagerly adopted these Layer-2 VPN solutions.
2. Video distribution is supported in two ways. First, live TV is delivered to residential users over multicast. We have added support for Protocol Independent Multicast (PIM) and Internet Group Management Protocol (IGMP) to allow white box packet switches to function as multicast routers. Second, video on-demand services rely on optimized mapping onto available optical capacity.

4. Disaggregated ROADM

Vertical integration of networking hardware has been acknowledged time and again as a way to achieve vendor lock-in. Our goal is to open the network to those that own and operate it, and to achieve programmability and observability, ultimately leading to zero-touch configuration and autonomic networks that can detect and repair themselves without human intervention. Realizing these goals require decomposing the optical devices into functional blocks [4], and defining generic interfaces that work across vendor and technology-specific implementations. Disaggregation of a chassis-based design into commodity components also allows service providers to rightsize their infrastructure and grow as needed. Figure 1a shows the disaggregated ROADM model as envisioned inside a CORD deployment.

1. The switching fabric forms the heart of the ROADM; it is based on wavelength selective switches, along with necessary amplifiers and monitoring elements.
2. The trans/mux/flexponder becomes a separate component in our model. In the early stages, identical (or at least from the same vendor) transponders will be required on both ends of a connection. Multi-vendor data plane compatibility will eventually become possible using this model, based on shared open standards for common DSP functions such as error correction, modulation, clock recovery, etc.
3. Disaggregating traditional ROADM designs implies the backplane is lost. It can be recreated by manual interconnection of the different transponders and switch fabric. The loss of flexibility can be overcome using a fiber switch; these micro-electro-mechanical or MEMS-based network elements are available in huge switch matrices (320x320), have acceptable loss figures (~3.5dB), and switching speeds in the order of milliseconds.
4. The L0 controller exists for several reasons. First, without it, disaggregated components would need to implement a full TCP/IP stack to offer protocol support such as NETCONF. Second, the L0 controller can easily be deployed on x86 hardware and scale to many hundreds of such components. Third, alarm correlation is critical to off-load work away from the local central office controller.

All of these components are rapidly becoming commercially available, and are available in convenient 1U through 4U form factors. Finally, note that the metro network is the ideal candidate for disaggregating packet/optical infrastructure. Indeed, the distances involved are reasonable (tens of km), bandwidth levels are typically just below the state of the art (i.e., some commoditization has already taken place on the hardware), and optical impairments are not as challenging as compared to, e.g., the wide area network.

5. Conclusion

We presented a re-design of the metro network based on SDN control of converged packet optical network (realized using open source ONOS), a set of innovative metro services, and disaggregated ROADM architecture. We are building a proof of concept of our approach.

6. References

[1] A. Al-Shabibi and L. Peterson, “CORD: Central Office Re-architected as a Datacenter,” OpenStack Summit 2015.

[2] P. Berde, et al., “ONOS: Towards an Open, Distributed SDN OS,” HotSDN 2014.

[3] R. Santitoro, “Metro Ethernet Services – A Technical Overview”, Metro Ethernet Forum, 2003.

[4] M. Birk, et al., “AT&T’s direction towards a Whitebox ROADM,” Open Networking Summit 2015.

 

---

 

Marc De Leenheer is a Member of Technical Staff at ON.Lab, a non-profit organization that is building the key open source tools for software defined networking. Marc leads the IP/optical optical activities, is the maintainer of the OVX network virtualization platform, and oversees its integration into the OpenStack cloud platform. He received the PhD degree in computer science from Ghent University, Belgium in 2008, followed by post-doctoral stays at the University of California Davis and Stanford University.

 

Tom Tofigh is currently a Principal Member of Technical Staff in the AT&T’s Domain 2.0 architecture and planning Labs at AT&T. Tom has been responsible for planning and implementation of large-scale telecommunications equipment. He has extensive experience that spans across many disciplines including emerging mobility networks, switches, enhanced protocols, cross layer design, smart devices, and chip designs. Products introduced include innovative solutions for the mobile devices, switches, platforms, and QOS mechanisms. Tom is currently active member of ON.Lab representing AT&T for implementation of open source solutions for SDN/NFV.

 

Editor:

Chris Hrivnak is a senior member of the Institute of Electrical and Electronic Engineers (IEEE) and IEEE Photonics Society. He is also a member of the IEEE Cloud Computing Community, IEEE Life Sciences Community, IEEE Smart Grid Community, the IEEE Software Defined Networks (SDN) Community, IEEE Internet of Things (IoT) Technical Community and the IEEE Internet Technology Policy Community.