Architecting Multi-Layer Orchestration for Telco Networks

Raghavendra Rao Tadepalli , Wipro; and Vinay Devadatta , Practice Head, Wipro

IEEE Softwarization, November 2018

Just a few years back the word “orchestration” was hardly used by Telecom operators (telcos) and was mostly used in the world of music. Today any telco management discussion is incomplete without discussions on “orchestration”. There are different views on what “orchestration” means; it is often used interchangeably with management, controller etc... When ETSI^[1] introduced NFV (Network Function Virtualization) to the communication industry, the NFV Orchestrator was a new functional block that caught everybody’s attention. Since then, a number of orchestration solutions have been specified and implemented by different forums, specification bodies, communication service providers and product vendors. Overtime, it was further realized that having just one orchestrator would not be sufficient and/or efficient and having multiple layers (at least two) of orchestration is required. For example, Open Source MANO (OSM)^[2] defines 2 layers of orchestration i.e. Service Orchestrator (SO) and Resource Orchestrator (RO). Similarly, Open Network Automation Platform (ONAP)^[3], defines at least 2 layers of orchestration i.e. Service Orchestrator (SO) and Virtual Function Controller (VF-C), where the latter is aligned with the ETSI NFV-Orchestrator function.

Defining Orchestration

The authors propose that the function of orchestration consists of coordination and management. While the coordination part is a must, managerial functionality is optional. The coordinator does not have absolute authority on components or entities that it needs to work with, whereas a manager has complete authority on the involved entities. The manager can decide on every aspect of the life cycle of subject components on its own. The manager itself might be controlled by a higher-level management entity, but downwards it enjoys exclusive absolute authority on the components it is managing. The coordinator, on the other hand, must work with entities which may or may not execute actions issued by it. Till the command is issued, the coordinator is not aware whether the command will be executed successfully. Hence, a coordinator in the true sense makes requests, rather than sending commands, and based on the response, realigns its action to achieve the intended goal. The coordinator must work within these constraints to achieve its goals. Another term used, specifically in SDN context, is controller. This basically controls a certain behavior or aspect of the subject entity. While it does not have absolute authority on the life cycle of the component, it exercises control on specific behaviors, and sometimes, shares this authority with other controllers. The consequence of coordination may result in the creation of a transient element (e.g. a Network Service), and the coordinator may manage the life cycle of this new element.

Orchestration in ETSI NFV MANO

In the case of the NFV MANO architecture, the NFV Orchestrator (NFVO) coordinates with VNFM (Virtual Network Function Manager) and VIM (Virtual Infrastructure Manager). While the latter two manage the Virtual Network Function (VNF) and Virtual Infrastructure (VI) respectively, the NFVO coordinates with these two managers to create a new transient entity: the Network Service (NS) . The NFVO, in its role as a manager, manages the life cycle of the NS using requests to VNFM(s) and VIMs(s). The orchestrator in its main role as a coordinator can only guarantee best effort results. In most architectures, we see NFVO working with a service orchestrator.

Multi-Layered Orchestration

In a hierarchical orchestration solution, an orchestrator at a given layer will need to work with a higher layer orchestrator rather than a manager. This is because a higher layer of manager would expect a command-acknowledgment behaviour rather than best-effort request-response behaviour. The lower layer orchestrator, being a coordinator, cannot guarantee requested action to the higher-level entity. At a generic level, in a multi-layered orchestration architecture, an orchestrator at the top of the hierarchy orchestrates multiple next level orchestrators and/or other management entities (e.g. VIM or VNFM).

There are multiple factors that influence how the hierarchical orchestration solution is structured for a specific context. Some of these factors are:

Geographical spread and placement of entities that need to be orchestrated (e.g. distributed NFVI PoPs).
Scalability constraints of the orchestrator.
Business and operational requirements of the service provider deploying the orchestration solution.
Different network domains and the abstractions that each of the entities in these domains (e.g. VNFs, SDN Controllers, VNFMs, etc.) offer over their interfaces towards the orchestration layer.
The different layers of abstraction (e.g. resource, service, customer and product)

The following figure illustrates a hierarchical layer of orchestrators; each layer of orchestration deals with different levels of abstraction and some of the layers are also structured based on the geographical region and network domain. There are obviously other multiple combinations in which such hierarchical layers of orchestrators can be structured and below is an illustration of such a possible scenario.

Figure 1: Illustration of hierarchical orchestration

Coordination Interfaces between Hierarchical Orchestrators

The interface constructs between the multiple layers of orchestrators are quite different from the traditional management systems. Typically, the interfaces between management systems are command/operation oriented i.e. a higher level management system issues a command or triggers an operation on the lower level management system and the command/operation is either successful or an error is reported back. The multi-orchestrator environment requires a different approach to interface design between the different layers of orchestrators.

The authors propose some of the key generic characteristics of the interface between multiple orchestrators (apart from interfaces reflecting functionality of the orchestrators ):

Intent-based interfaces instead of command/operation based:

A higher-level orchestrator expresses its intent to the lower level orchestrator and the lower level orchestrator, in turn, decides how to achieve that intent by orchestrating various entities in its scope.

Conversational style of interfaces instead of request/response paradigm

The orchestrators exchange conversations instead of request/response, for example, a service orchestrator might request a WAN orchestrator to create of a network service in a particular geographical region(s) with specific QoS, and the WAN orchestrator analyzes the request based on information it gathers from management systems, analytics feeds and VIMs and gets back to the service orchestrator on the alternatives that are feasible (say the original QoS requirements cannot be completely satisfied etc.). The service orchestrator can then either choose one of the alternatives or even decide to change the original request, considering the alternatives suggested. Such a conversational style enables the orchestrators acting at different levels of abstraction the scope to mitigate potential deadlocks, ensure proper reservations of network resources to efficiently achieve the primary intent.

Best-effort instead of guaranteed action semantics

Unlike the conventional management systems, the orchestrators, by definition, are oriented towards best-effort based semantics. For example, if a service orchestrator expresses its intent to instantiate a set of VNFs to a NFV orchestrator, the NFV orchestrator may choose a slightly different combination or configuration of VNFs to instantiate based on the real-time information it has on the NFVI resource availability etc., to fulfil the intent. Such intent based best-effort semantics imply that the approach adopted to achieve the end to end system’s reliability has to be different from the conventional management system. One of the possible approaches could be to leverage coordination and management functions of orchestrator.

Multi-Layered Orchestration – Key Benefits

The orchestrators, unlike the management systems, are primarily entities responsible for sharing control, handle conflicts, and ensure that the primary intent (which includes various QoS characteristics required for operational efficiency and effectiveness) is achieved on a best-effort basis. The real benefit of embracing NFV and SDN in a telecommunication network can be achieved only if a collaborative style of multi-layered orchestration capability is employed.

Conclusion

Telco networks are complex and need to be managed from multiple perspectives. Orchestration architecture enables loose coupling of monitoring and control from different perspectives while ensuring consistent behavior across the entire system. A coordinative approach rather than a master -slave approach ensures that lower level as well as higher-level objectives are dynamically balanced. In a multilayer orchestration architecture, care should be taken that the coordinative approach is preserved across layers and highest intents flow dynamically through the management system.

References

ETSI GS NFV-MAN 001 V1.1.1 (2014-12) : Network Functions Virtualisation (NFV); Management and Orchestration

https://www.etsi.org/deliver/etsi_gs/NFV-MAN/001_099/001/01.01.01_60/gs_NFV-MAN001v010101p.pdf

Open Source MANO : OSM Release FOUR Technical Overview May 2018

https://www.kernel.org/doc/Documentation/networking/vrf.txt

Introducing the ONAP Architecture (Amsterdam Release)

https://onap.readthedocs.io/en/amsterdam/guides/onap-developer/architecture/onap-architecture.html

Raghavendra Rao Tadepalli

Raghavendra has extensive experience in the telecoms industry, working with equipment vendors and standard bodies like TM Forum, ETSI NFV and ONF. He is responsible for the Cloud Native applications portfolio as part of Wipro’s Telecom Network Equipment Provider business. His primary focus is R&D of OSS applications, Orchestration and automation, Network Function Virtualization and Software Defined Networks, and works with the Network Equipment Vendor business within Wipro. He holds a B.Tech in Computer Science.

Vinay Devadatta

Vinay has extensive experience in the communications industry, working with equipment vendors, communication service providers, OSS product vendors, OSS solution providers and telco management standards bodies like TM Forum, ETSI NFV, NGMN NGCOR. He is responsible for leveraging external and internal innovation and creating new service lines at Wipro Technologies. His expertise is in the areas OSS, NFV, Orchestration, automation, digital customer experience, automation and 5G. He holds a B.Tech in Electronics & Communication Engineering and an M.Tech in Computer Science & Technology from IIT Roorkee.

Editor:

Eliezer Dekel

Eliezer Dekel a Chief Architect for Huawei Technologies Corporate Reliability Department. He is researching RAS for SDN and NFV. He retired from IBM Research - Haifa, as a Senior Technical Staff Member and Chief Architect for Distributed Systems. In his this role he focused on developing infrastructure technologies for very large scale distributed systems.

Eliezer Dekel is the editor in chief of EAI Endorsed Transaction on Cloud Systems. He is also an Associate Editor for ACM Computing Surveys and a member of the editorial board for IEEE SDN Newsletter. Eliezer served on numerous conference program committees and organized, or served as chair in some of them. He has been involved in research in the areas of distributed and fault-tolerant computing, service-oriented technology, and software engineering. He was recently working on technologies for providing Quality of Service, with a focus on dependability, in very large scale multi-tier environments. For this area he initiated together with colleagues the very successful International Workshop on Large Scale Distributed Systems and Middleware (LADIS). This workshop, sponsored by ACM. It was one of the first workshops to focus on the foundations of "cloud computing." He was an organizer of CloudSlam'09 the first cloud computing virtual conference. Eliezer was also involved in several EU FP7 ICT funded projects.

Eliezer has a Ph.D. and M.Sc. in computer science from the University of Minnesota, and a B.Sc. in mathematics from Ben Gurion University, Israel. Prior to joining IBM Research - Haifa, Eliezer served on the faculty of the University of Texas at Dallas computer science department for more than ten years.