[Docs] [txt|pdf] [Tracker] [Email] [Diff1] [Diff2] [Nits]

Versions: 00 01 02

DetNet                                                         H. Wang
Internet Draft                                                 P. Wang
Intended status: Standards Track                              C. Zhang
Expires: June 11, 2018                                         Y. Yang
                                               Chongqing University of
                                          Posts and Telecommunications
                                                      December 8, 2017

         Joint Scheduling Architecture for Deterministic Industrial
                          Field/Backhaul Networks


   The joint scheduling between industrial field network and backhaul
   network is important to satisfy the requirements of deterministic
   delay for data flows in factories. This document describes a joint
   scheduling architecture for deterministic industrial field/backhaul
   networks. Taking WIA-PA, an international standard about industrial
   wireless field network, and IPv6-based backhaul network as an
   example, this document depicts how the joint scheduling architecture
   works in detail.

Status of this Memo

   This Internet-Draft is submitted in full conformance with the
   provisions of BCP 78 and BCP 79.

   Internet-Drafts are working documents of the Internet Engineering
   Task Force (IETF), its areas, and its working groups. Note that
   other groups may also distribute working documents as Internet-

   Internet-Drafts are draft documents valid for a maximum of six
   months and may be updated, replaced, or obsoleted by other documents
   at any time. It is inappropriate to use Internet-Drafts as reference
   material or to cite them other than as "work in progress."

   The list of current Internet-Drafts can be accessed at

   The list of Internet-Draft Shadow Directories can be accessed at

   This Internet-Draft will expire on June 11, 2018.

Wang, et al.            Expires June 11, 2018                 [Page 1]

 Internet-Draft        DetNet Backhaul Architecture       December 2017

Copyright Notice

   Copyright (c) 2017 IETF Trust and the persons identified as the
   document authors. All rights reserved.

   This document is subject to BCP 78 and the IETF Trust's Legal
   Provisions Relating to IETF Documents
   (http://trustee.ietf.org/license-info) in effect on the date of
   publication of this document. Please review these documents
   carefully, as they describe your rights and restrictions with
   respect to this document. Code Components extracted from this
   document must include Simplified BSD License text as described in
   Section 4.e of the Trust Legal Provisions and are provided without
   warranty as described in the Simplified BSD License.

Table of Contents

   1. Introduction ................................................ 2
   2. Joint Scheduling Architecture ............................... 3
      2.1. Distributed Architecture ............................... 4
      2.2. Centralized Architecture ............................... 5
      2.3. Joint Scheduling Architecture .......................... 6
   3. Joint Scheduling Scheme ..................................... 9
      3.1. WIA-PA Network Joint Scheduling ....................... 10
      3.2. Protocol Conversion ................................... 10
      3.3. Industrial Backhaul Network Scheduling ................ 12
      3.4. Bandwidth guarantee method ............................ 14
   4. Security Considerations .................................... 14
   5. IANA Considerations ........................................ 14
   6. References ................................................. 14
      6.1. Normative References .................................. 14
      6.2. Informative References ................................ 14
   Authors' Addresses ............................................ 16

1. Introduction

   Deterministic network is one of essential elements of industrial
   network. With the help of deterministic network, industrial field
   network can greatly enhance the network performance in terms of
   reducing transmission delay. Thus, applying deterministic network
   into the whole industrial network has attracted a lot of attention
   recently. Deterministic network is mainly focused on the industrial
   field networks, such as ISA100.11a [IEC62734], WirelessHART
   [IEC62591] and WIA-PA [IEC62601]. In order to solve the problem of
   data transmission in different industrial field networks and the
   issue of data flows between industrial field networks and wide area

Wang, et al.            Expires June 11, 2018                 [Page 2]

 Internet-Draft        DetNet Backhaul Architecture       December 2017

   networks, industrial backhaul network is deployed in factory.
   However, there are little considerations about joint scheduling
   scheme that can be applied to industrial networks.

   The emerging Software Defined Networks (SDN) technology on the
   Internet brings a new choice to solve joint scheduling problem. SDN
   has been proposed as a new network architecture in recent years. The
   network architecture separates the network control plane from the
   forwarding plane, which brings a revolution for the network
   architecture. By separating control plane from forwarding plane ,and
   the open communication protocol, SDN breaks the closure of
   traditional network device provider. Besides, open interfaces and
   free programmability also make network management more efficient and

   In document [I-D.bas-usecase-detnet] and [I-D.finn-detnet-
   architecture] submitted by the IETF DetNet working group,
   deterministic network based on Ethernet has been researched already.
   They propose a network architecture based on SDN technology that can
   accurately control the transmission of data streams. However, the
   characteristics of the industrial backhaul network and the actual
   condition of industrial field deterministic networks are not
   considered. Firstly, the data that transmits in industrial backhaul
   network is highly sensitive to transmission delay. Secondly, the
   existing deterministic networks have been widely deployed in
   industrial field environment, thus the direct replacement for
   original networks will consume many workers and material resources.

   Based on existing research in document [I-D.finn-detnet-
   architecture], this document proposes a joint scheduling
   architecture for deterministic industrial networks. It will firstly
   replace the industrial backhaul networks and other non-deterministic
   networks located in industrial networks with deterministic Ethernet
   network. Then this document proposes a joint scheduler based on SDN
   technology. By deploying the deterministic network in complete
   industrial network, it can realize the end-to-end deterministic
   scheduling between different industrial field networks.

2. Joint Scheduling Architecture

   There are many types of network controllers in industrial networks,
   which constitute the control plane of the whole industrial network
   together. The control plane is very important in the entire network,
   especially when it refers to cross-domain transmission of time-
   sensitive data. The control plane architecture affects the
   performance of the network greatly. It is becoming a hot research on

Wang, et al.            Expires June 11, 2018                 [Page 3]

 Internet-Draft        DetNet Backhaul Architecture       December 2017

   how to give full play to the performance of their respective
   networks when multiple controllers are in the joint work.

   However, there is no unified standard for the joint architecture of
   multiple controllers in the industry presently. The mainstream of
   architecture includes distributed architecture and centralized

2.1. Distributed Architecture

   Distributed architecture is known as East-West architecture. In the
   architecture, the status of all network controllers are equal, these
   controllers connect with each other to form an unstructured network,
   and implement cross-domain transmission task by exchanging
   information, as shown in Figure 1.

   In distributed architecture, controller can exchange different
   network topologies and the accessibility of information by east-west
   interface, and each controller can establish a global network
   topology. From a global network perspective, each controller is
   equal, thus it can serve as a server role as well as the ability to
   start deterministic cross-network transmission task.

Wang, et al.            Expires June 11, 2018                 [Page 4]

 Internet-Draft        DetNet Backhaul Architecture       December 2017

             |                                                       |
 Application |       +--------+     +--------+      +-------+        |
    Plane    |       |   APP  |     |   APP  |      |  APP  |        |
             |       +----+---+     +----+---+      +---+---+        |
             |                           |                           |
                      |                  |                 |
             |        |                  |                 |         |
    Control  |   +----+-----+       +----+-----+       +----+-----+  |
    Plane    |   |Controller|------>|Controller|------>|Controller|  |
             |   |          |<------|          |<------|          |  |
             |   +----------+       +----------+       +----------+  |
             |        |                  |                  |        |
                      |                  |                  |
                      |                  |                  |
                 +----+-----+       +----+-----+       +----+-----+
    Forwarding   |  WIA-PA  |------>| backhaul |------>|  WIA-PA  |
    Plane        |  network |<------|  network |<------|  network |
                 +----------+       +----------+       +----------+
                   Figure 1. Distributed Architecture

2.2. Centralized Architecture

   Centralized architecture is also called a vertical multi-level
   architecture. In this architecture, the control plane is divided
   into two parts, one part is the basic control plane, which is
   composed of a variety of network controllers, and another is a main
   network controller, which is responsible for controlling the basic
   control plane. The detailed description of centralized architecture
   is shown in Figure 2.

Wang, et al.            Expires June 11, 2018                 [Page 5]

 Internet-Draft        DetNet Backhaul Architecture       December 2017

                 |                                                  |
    Application  | +--------+        +--------+         +-------+   |
       Plane     | |   APP  |        |   APP  |         |  APP  |   |
                 | +--------+        +----+---+         +-------+   |
                 |                        |                         |
                 |                        |                         |
                 |                 +------+-----+                   |
                 |      +----------|    Main    |-----------+       |
                 |      |          | Controller |           |       |
       Control   |      |          +------+-----+           |       |
       Plane     |      |                 |                 |       |
                 | +----+-----+      +----+-----+      +----+-----+ |
                 | |Controller|      |Controller|      |Controller| |
                 | +----+-----+      +----+-----+      +----+-----+ |
                 |      |                 |                 |       |
                        |                 |                 |
                        |                 |                 |
                   +----+-----+      +----+-----+      +----+-----+
        Forwarding |  WIA-PA  |----->| backhaul |----->|  WIA-PA  |
        Plane      |  network |<-----|  network |<-----|  network |
                   +----------+      +----------+      +----------+

                   Figure 2. Centralized Architecture

   The centralized architecture does not need to expand east-west
   interface. It only needs to establish a connection with the basic
   controllers using southbound interface. After the connection is
   established, the main controller obtains the every topology of
   network domain by the API interface provided by the basic
   controllers, and storages global network topology. Main controller
   can also assign tasks to basic controllers by API interface.

2.3. Joint Scheduling Architecture

   In practical application, distributed architecture not only needs to
   extend the east-west interface, but also maintains a global network

Wang, et al.            Expires June 11, 2018                 [Page 6]

 Internet-Draft        DetNet Backhaul Architecture       December 2017

   topology in each controller. Only each controller maintains such a
   global network topology, can it ensure the deterministic control of
   whole network.

   For deterministic industrial network, the scale of network is not
   very large. Besides, in industrial backhaul network, a single SDN
   controller is sufficient to meet the demands of control. If
   centralized architecture is directly applied to an industrial
   network, it will not only be unable to make full use of advantages
   of the multi-controller architecture, but also cause unnecessary
   information interaction between controllers wasting network resource.

   Considering the problems existing in above two architectures, this
   document proposes a joint scheduling architecture based on the
   architecture document [I-D.finn-detnet-architecture]. The
   architecture is optimized according to the characteristics of
   deterministic industrial network. A single SDN controller can unite
   the WIA-PA network system manager to manage the entire industrial
   network, and provide support for the deterministic scheduling of
   cross-network data transmission through industrial backhaul network
   located in different WIA-PA networks.

Wang, et al.            Expires June 11, 2018                 [Page 7]

 Internet-Draft        DetNet Backhaul Architecture       December 2017

               |                                                     |
    Application|     +--------+       +--------+       +-------+     |
    Plane      |     |   APP  |       |   APP  |       |  APP  |     |
               |     +--------+       +----+---+       +-------+     |
               |                           |                         |
             |                             |                           |
    Control  |  +--------------+      +----+-----+     +--------------+|
    Plane    |  |    WIA-PA    |------|   SDN    |-----|   WIA-PA     ||
             |  |System Manager|      |Controller|     |System Manager||
             |  +------+-------+      +----+-----+     +-------+------+|
             |         |                   |                   |       |
                       |                   |                   |
                       |                   |                   |
                  +----+-----+        +----+-----+       +-----+----+
    Forwarding    |  WIA-PA  |------->| backhaul |------>|  WIA-PA  |
    Plane         |  network |<-------| network  |<------|  network |
                  +----------+        +----------+       +----------+

                  Figure 3. Joint scheduling architecture

   Figure 3 depicts the joint scheduling architecture. The architecture
   can be mainly classified into three planes:

   o Forwarding plane: this plane contains various types of network
      equipment belonged to different networks. It is the physical
      entities for network transmission. To obtain the desired network
      functions from network manager, devices should abstract their own
      resources to provide to its network manager respectively.

   o Control plane: this plane is composed by WIA-PA system manager
      and SDN controller. Joint scheduler is integrated into the SDN
      controller by a way of plugin, and other WIA-PA system managers
      communicate with joint scheduler by establishing a connection
      with the SDN controller. Meanwhile, joint scheduler implement the
      management of industrial backhaul network by directly invoking
      the corresponding module of SDN controller.

Wang, et al.            Expires June 11, 2018                 [Page 8]

 Internet-Draft        DetNet Backhaul Architecture       December 2017

   o Application plane: this plane provides users with a unified
      interface about many variety of resources for the whole network.
      At the same time, it also provides an intuitive and user-friendly
      interface, which shields the complex network information of the

   When application plane triggers a joint scheduling task, SDN
   controller calculates path and allocates resource according to the
   task request from the application plane. Upon finishing calculation,
   SDN controller sends them using the unified joint scheduling
   interface to corresponding network manager, and then the network
   manager sends them to the industrial field network.

   Based on joint scheduling architecture, joint scheduler can realize
   control and scheduling for the entire industrial network, thus it
   can provide a real-time guarantee for each data stream.

3.  Joint Scheduling Scheme

   Taking WIA-PA network and IPv6-based backhaul network as an example,
   this section describes how the joint scheduling architecture works.
   Existing WIA-PA scheduling scheme only applies to WIA-PA field
   network. Scheduling scheme will fail once data transfers to backhaul
   networks. Joint scheduling scheme is an innovation and expansion
   compared to WIA-PA scheduling scheme.

   Firstly, original scheduling scheme based on SDN in industry
   backhaul network is added to the proposed scheduling scheme, thus,
   data can flow in the industrial backhaul network.

   Secondly, by conducting an optimization for original WIA-PA
   scheduling scheme, original scheduling scheme can work with joint
   scheduler, and simultaneously be applied to cross-domain network.

   Thirdly, due to the specificity of cross-border transmission
   services, the joint scheduling scheme for WIA-PA network VCR_ID and
   route ID is reclassified.

   Finally, due to system manager allocates short address to field
   device based on WIA-PA network address information independently.
   Thus the short address of field device in entire industrial network
   is uncertain. In order to identify the field device belonged to
   different network domains, the network identifier (PAN_ID) is
   applied to the joint scheduling scheme to identify different WIA-PA

Wang, et al.            Expires June 11, 2018                 [Page 9]

 Internet-Draft        DetNet Backhaul Architecture       December 2017

   After the SDN controller initiates joint scheduling module, WIA-PA
   system manager will actively establish a connection with the united
   scheduler. After the scheduler receives a cross-border transmission
   request, joint scheduler will send a request for obtaining topology
   information and node information to WIA-PA System Manager. Then, the
   scheduler will assign paths and network resources according to this
   information by pre-defined scheduling algorithm. After the path and
   network resources have been calculated, joint scheduler will
   configure and deploy networks by the corresponding network

3.1. WIA-PA Network Joint Scheduling

   In joint scheduling process, path deployment and resource allocation
   for WIA-PA network are performed by employing the WIA-PA system
   manager API interface. System manager will query the corresponding
   information of the field device in the network upon receiving the
   command about joint operation for the network information, and then
   return the received information to the joint scheduler. The system
   manager will configure communication resources for the corresponding
   gateway device, routing equipment and field equipment when it
   receives configuration commands from joint scheduler.

3.2. Protocol Conversion

   For cross-domain transmission, industrial backhaul network is
   different from WIA-PA network which is not an IP-based Ethernet.
   Protocol conversion for WIA-PA packet in gateway is needed when data
   generated from WIA-PA network needs to transmit to another field
   network through industrial backhaul. Meanwhile, according to the
   joint scheduling scheme, SDN controller is able to recognize the
   WIA-PA data stream and allocate resources according to data stream
   type. Therefore, in the protocol conversion process, scheduling and
   control of WIA-PA data flow can be realized by SDN controller by
   combining the VCR of WIA-PA data stream and the priority filled in
   IPv6 header.

Wang, et al.            Expires June 11, 2018                [Page 10]

 Internet-Draft        DetNet Backhaul Architecture       December 2017

          | Start |
        |  Receiving  |
        |data packets |
        /-------------\           +--------------+
        /Whether is the \          | Forwarded to |
        |  management   |-- Yes -->|  the system  |
        \     data      /          |    manager   |
        \-------------/           +--------------+
        | Resolution|
        |   Packet  |
        /------------\             /------------\         +------------+
       /   Find the   \           /   Find the   \        | Encapsulate|
       |corresponding |-- Yes -->| corresponding |- Yes-->|  and sent  |
       \     VCR      /           \ IPv6 address /        | IPv6 packet|
        \------------/             \------------/         +------------+
             |                          |                        |
             No                         No                       |
             |                          |                        |
         +--------+                     |                        |
         |  End   |<--------------------+------------------------+

          Figure 4. The process of protocol conversion in gateway

   As shown in Figure 4, gateway will receive the address mapping
   configuration command from joint scheduler. Then VCR tables and IPv6
   address-mapping tables will be formed according to this information.
   When gateway receives WIA-PA packets, it will firstly parse Route ID,
   Object ID and Instance ID, and find corresponding VCR from VCR
   tables. Meanwhile, the gateway finds the corresponding IPv6 address
   according to Route ID in IPv6 address mapping table. Then, gateway
   begins to encapsulate WIA-PA packets based on IPv6 format, fill
   VCR_ID in IPv6 header flow label field and the priority of WIA-PA
   packet in IPv6 header fields.

Wang, et al.            Expires June 11, 2018                [Page 11]

 Internet-Draft        DetNet Backhaul Architecture       December 2017

   When receiving IPv6 packets from industrial backhaul networks,
   gateway will recognize VCR_ID from IPv6 packet header, and obtain
   packet VCR according to the VCR ID in VCR table, then replace it
   with the information of original packet.

3.3. Industrial Backhaul Network Scheduling

   In deterministic network based on SDN, joint scheduler can recognize
   WIA-PA data stream by matching IPv6 flow label field. According to
   priority in IPv6 header field and VCR_ID type, joint scheduling can
   allocate the necessary resources to communication and ensure that
   important data flow is not affected when adding new data flows in
   existing network. It can also monitor the real-time data flow. To
   guarantee the real-time performance of critical data flows,
   redundant paths are also considered when necessary. The scheduling
   process of industrial backhaul network is shown in Figure 5.

Wang, et al.            Expires June 11, 2018                [Page 12]

 Internet-Draft        DetNet Backhaul Architecture       December 2017

                   | Start |
                |Obtain network|
                |   topology   |
              |    Calculate the  |
              | path and allocates|<---------------+
              |    resources      |                |
              +-------------------+                |
                      |                            |
                 +----------+                      |
                 |Query path|                      |
                 +----------+                      |
                      |                            |
                /---------------\         +----------------+
               /whether the path \        |  Calculate the |
               |meets the resource|- No ->|   weight and   |
               \  requirements   /        | adjustment path|
                \---------------/         +----------------+
                | Deployment |
                | flow table |
                   | End |

      Figure 5. The scheduling process of industrial backhaul network

   When receiving the request for service, the joint scheduler will
   calculate route information and resource allocation. Once the path
   and resource allocation are determined, joint scheduling will
   confirm whether the resource and path are capable of meeting
   business requirements by the inside module of SDN controller. If it
   meets, then the flow table is deployed by SDN controller. Otherwise,
   the path information and resource allocation are recalculated to
   choose the correct paths to transmit data flow.

Wang, et al.            Expires June 11, 2018                [Page 13]

 Internet-Draft        DetNet Backhaul Architecture       December 2017

3.4. Bandwidth guarantee method

   Bandwidth guarantee method is implemented on the basis of joint
   scheduling mechanism, in order to solve the problem that industrial
   backhaul networks can not identify fine-grained and cross-network
   data transmission. By filling the priority information and the
   RouteID defined in WIA-PA network into the IPv6 header, the SDN
   controller can not only identify cross-network transmission of the
   WIA-PA data stream, but also obtain priority information about the
   WIA-PA data stream. In industrial backhaul network, the SDN switch
   employs the mechanism of priority queue to allocate network
   bandwidth. Thus SND switch can distribute WIA-PA data streams into
   corresponding queues of ports according to the received flow table
   from SDN controller. Therefore, the bandwidth of data stream is

   By using the above methods, joint scheduling mechanism can
   distinguish data streams in a fine-grained way and guarantee
   bandwidth when data transmits in industrial backhaul network. For
   example, real-time data in WIA-PA network is sensitive to delay,
   thus it should be allocated more bandwidth to reduce transmission
   delay. For not urgent data, it can be assigned less bandwidth to
   reserve bandwidth for real-time data. Meanwhile, SDN controller can
   flexibly adjust bandwidth allocation strategy to relieve network

4. Security Considerations

5. IANA Considerations

   This memo includes no request to IANA.

6. References

6.1. Normative References

6.2. Informative References

           ISA/IEC, "ISA100.11a, Wireless Systems for Automation,
           also IEC 62734", 2011, <http://www.isa100wci.org/enUS/

Wang, et al.            Expires June 11, 2018                [Page 14]

 Internet-Draft        DetNet Backhaul Architecture       December 2017

           IEC, "Industrial Communication Networks -
           Wireless Communication Network and Communication Profiles
           - WirelessHART - IEC 62591", 2010,

           IEC, "Industrial networks - Wireless communication network
           and communication profiles - WIA-PA - IEC 62601", 2015, <

           Finn, N. and P. Thubert, "Deterministic Networking Problem
           Statement", draft-finn-detnet-problem-statement-05 (work in
           progress), March 2016.

           Finn, N., Thubert, P., and M. Teener, "Deterministic
           Networking Architecture", draft-finn-detnet-architecture-08
           (work in progress), August 2016.

           Kaneko, Y., Toshiba and Das, S, "Building Automation Use
           Cases and Requirements for Deterministic Networking", draft-
           bas-usecase-detnet-00 (work in progress), October 2015.

Wang, et al.            Expires June 11, 2018                [Page 15]

 Internet-Draft        DetNet Backhaul Architecture       December 2017

Authors' Addresses

   Heng Wang
   Chongqing University of Posts and Telecommunications
   2 Chongwen Road
   Chongqing, 400065

   Phone: (86)-23-6248-7845
   Email: wangheng@cqupt.edu.cn

   Ping Wang
   Chongqing University of Posts and Telecommunications
   2 Chongwen Road
   Chongqing, 400065

   Phone: (86)-23-6246-1061
   Email: wangping@cqupt.edu.cn

   Chang Zhang
   Chongqing University of Posts and Telecommunications
   2 Chongwen Road
   Chongqing, 400065

   Phone: (86)-23-6246-1061
   Email: zc910522@126.com

   Yi Yang
   Chongqing University of Posts and Telecommunications
   2 Chongwen Road
   Chongqing, 400065

   Phone: (86)-23-6246-1061
   Email: 15023705316@163.com

Wang, et al.            Expires June 11, 2018                [Page 16]

Html markup produced by rfcmarkup 1.126, available from https://tools.ietf.org/tools/rfcmarkup/