Transporting PTP messages (1588) over MPLS Networks

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC2119 . When used in lower case, these words convey their typical use in common language, and are not to be interpreted as described in RFC2119 .

The objective of Precision Time Protocol (PTP) is to synchronize independent clocks running on separate nodes of a distributed system. defines PTP messages for clock and time synchronization. The PTP messages include PTP PDUs over UDP/IP (Annex D & E of ) and PTP PDUs over Ethernet (Annex F of ). This document defines mapping and transport of the PTP messages defined in over MPLS networks. PTP defines intermediate clock functions (called transparent clocks) between the source of time (Master) and the Slave clocks. Boundary Clocks (BC) form Master-Slave hierarchy with the Master clock as root. The messages related to synchronization, establishing the Master-Slave hierarchy, and signaling, terminate in the protocol engine of a boundary clock and are not forwarded. Management messages however, are forwarded to other ports on the boundary clock. Transparent clocks modify a "correction field" (CF) within the synchronization messages to compensate for residence and propagation delays. Transparent clocks do not terminate synchronization, Master-Slave hierarchy control messages or signaling messages. There is a need to transport PTP messages over MPLS networks. The MPLS network could be a transit network between 1588 Masters and Slaves. The accuracy of the recovered clock improves and the Slave logic simplifies when intermediate nodes (e.g. LSRs) properly handle PTP messages (e.g. perform TC), otherwise the jitter at the 1588 Slave may be excessive and therefore the Slave may not be able to properly recover the clock and time of day. This document requires that MPLS nodes (LSRs) SHOULD be able to support the Transparent Clock (TC) function, meaning that they should be able to modify the CF of the proper PTP messages, via a 1-step or 2-step process. Such an LSR is referred to as a "1588-aware LSR" in this document. TC requires a 1588-aware LSR in the middle of an LSP to identify the PTP messages and perform proper update of the CF. More generally this document requires that an LSR SHOULD be able to properly handle the PTP messages. For instance for those cases when the TC function is not viable (e.g. due to layer violation) as an alternative it should be possible to instead control the delay for these messages on both directions across the node. In the above cases it is beneficial that PTP packets can be easily identified when carried over MPLS. This document provides two methods for transporting PTP messages over MPLS. The main objectives are for LSRs to be able to deterministically detect and identify the PTP messages.

1588: The timing and synchronization as defined by IEEE 1588 PTP: The timing and synchronization protocol used by 1588 Master: The Source of 1588 Timing and clock Slave: The Destination of 1588 Timing and clock that tries to follow the Master clock OC: Ordinary Clock TC: Transparent Clocking, a time stamping method applied by intermediate nodes between Master and Slave BC: Border Clock, is a node that recovers the Master clock via a Slave function and uses that clock as the Master for other Slaves PTP LSP: An LSP dedicated to carry PTP messages PTP PW: A PW within a PTP LSP that MAY correspond to a Master/Slave flow. CW: Pseudo wire Control Word CW: Pseudo wire Control Word LAG: Link Aggregation ECMP: Equal Cost Multipath CF: Correction Field, a field inside certain PTP messages (message type 0-3)that holds the accumulative transit time inside intermediate switches

When PTP messages are transported over MPLS networks, there is a need for intermediate LSRs to detect such messages and perform proper processing (e.g. Transparent Clock (TC)). Note the TC processing could be in the form of 1-Step or 2-Step time stamping. PTP messages over Ethernet or IP can always be tunneled over MPLS. However the 1588 over MPLS mapping defined in this document is applicable whenever MPLS LSRs are 1588-aware and the intention is for those LSRs to perform proper processing on these packets. When 1588-awareness is needed, PTP messages should not be transported over LSPs or PWs that are carrying customer traffic because LSRs perform Label switching based on the top label in the stack. To detect PTP messages inside such LSPs require special Hardware (HW) to do deep packet inspection at line rate. Even if one assumes a deep packet inspection HW at line rate exists, the payload can't be deterministically identified by LSRs because the payload type is a context of the PW label and the PW label and its context are only known to the Edge routers (PEs) and LSRs don't know what is a PW's payload (Ethernet, ATM, FR, CES, etc). Even if one assumes only Ethernet PWs are permitted in an LSP, the LSRs don't have the knowledge of whether PW Control Word (CW) is present or not and therefore can't deterministically identify the payload. Therefore a generic method is defined in this document that does not require deep packet inspection at line rate, and can deterministically identify PTP messages. The defined method is applicable to both MPLS and MPLS-TP networks.

Many methods were considered for identifying the 1588 messages when they are encapsulated in MPLS such as by using GAL/ACH or a new reserved label. These methods were not attractive since they either required deep packet inspection and snooping at line rate or they required use of scarce new reserved label. Also one of the goals was to reuse existing OAM and protection mechanisms. The method defined in this document can be used by LSRs to identify PTP messages in MPLS tunnels by using dedicated LSPs to carry PTP messages. Compliant implementations MUST use dedicated LSPs to carry PTP messages over MPLS. Let's call these LSPs as the "PTP LSPs" and the labels associated with these LSPs as "PTP labels". These LSPs could be P2P or P2MP LSPs. The PTP LSP between Master and Slaves MAY be P2MP or P2P LSP while the PTP LSP between each Slave and Master SHOULD be P2P LSP. The PTP LSP between a Master and a Slave and the PTP LSP between the same Slave and Master MUST be co-routed. Alternatively, a single bidirectional co-routed LSP can be used. The PTP LSP MAY be MPLS LSP or MPLS-TP LSP. The PTP LSPs could be configured or signaled via RSVP-TE/GMPLS. New RSVP-TE/GMPLS TLVs and objects are defined in this document to indicate that these LSPs are PTP LSPs. Note that the PTP LSPs MUST only carry PTP messages and MAY carry MPLS/MPLS-TP control and management messages such as BFD and LSP-Ping.

This document defines two methods for carrying PTP messages over MPLS. The first method is carrying IP encapsulated PTP messages over PTP LSPs and the second method is to carry PTP messages over dedicated Ethernet PWs (called PTP PWs) inside PTP LSPs.

The simplest method of transporting PTP messages over MPLS is to encapsulate PTP PDUs in UDP/IP and then encapsulate them in PTP LSP. The 1588 over LSP format is shown in Figure 1.

+----------------------+ | PTP Tunnel Label | +----------------------+ | IPv4/6 | +----------------------+ | UDP | +----------------------+ | PTP PDU | +----------------------+ Figure 1 - 1588 over LSP Encapsulation This encapsulation is very simple and is useful when the networks between 1588 Master and Slave are IP/MPLS networks. In order for an LSR to process PTP messages, the PTP Label MUST be the top label of the label stack. The UDP/IP encapsulation of PTP MUST follow Annex D and E of .

Another method of transporting 1588 over MPLS networks is by encapsulating PTP PDUs in Ethernet and then transporting them over Ethernet PW (PTP PW) as defined in , which in turn is transported over PTP LSPs. Alternatively PTP PDUs MAY be encapsulated in UDP/IP/Ethernet and then transported over Ethernet PW. Both Raw and Tagged modes for Ethernet PW are permitted. The 1588 over PW format is shown in Figure 2.

+----------------+ |PTP Tunnel Label| +----------------+ | PW Label | +----------------+ | Entropy Label | | (optional) | +----------------+ | Control Word | +----------------+ | Ethernet | | Header | +----------------+ | VLANs | | (optional) | +----------------+ | IPV4/V6 | | (optional) | +----------------+ | UDP | | (optional) | +----------------+ | PTP PDU | +----------------+ Figure 2 - 1588 over PW Encapsulation The Control Word (CW) as specified in SHOULD be used to ensure a more robust detection of PTP messages inside the MPLS packet. If CW is used, the use of Sequence number is optional. The use of VLAN and UDP/IP are optional. Note that 1 or 2 VLANs MAY exist in the PW payload. In order for an LSR to process PTP messages, the top label of the label stack (the Tunnel Label) MUST be from PTP label range. However in some applications the PW label may be the top label in the stack, such as cases where there is only one-hop between PEs or in case of PHP. In such cases, the PW label SHOULD be chosen from the PTP Label range. An Entropy label MAY be present at the bottom of stack. The Ethernet encapsulation of PTP MUST follow Annex F of and the UDP/IP encapsulation of PTP MUST follow Annex D and E of .

1588 protocol comprises of the following message types: Announce SYNC FOLLOW UP DELAY REQ (Delay Request) DELAY RESP (Delay Response) PDELAY REQ (Peer Delay Request) PDELAY RESP (Peer Delay Response) PDELAY RESP FOLLOW UP (Peer Delay Response Follow up) Management Signaling A subset of PTP message types that require TC processing are called Event messages: SYNC DELAY REQ (Delay Request) PDELAY REQ (Peer Delay Request) PDELAY RESP (Peer Delay Response) SYNC and DELAY_REQ are exchanged between Master and Slave and MUST be transported over PTP LSPs. PDELAY_REQ and PDELAY_RESP are exchanged between adjacent routers and MAY be transported over single hop PTP LSPs. If Two Step Transparent clocks are present, then the FOLLOW_UP and DELAY_RESP messages must also be transported over the PTP LSPs. For a given instance of 1588 protocol, SYNC and DELAY_REQ MUST be transported over two PTP LSPs that are in opposite directions. These PTP LSPs, which are in opposite directions MUST be congruent and co-routed. Alternatively, a single bidirectional co-routed LSP can be used. Except as indicated above for the two-step Transparent clocks, Non-Event PTP message types don't need to be processed by intermediate routers. These message types MAY be carried in PTP Tunnel LSPs.

In order to ensure continuous uninterrupted operation of 1588 Slaves, usually as a general practice, Redundant Masters are tracked by each Slave. It is the responsibility of the network operator to ensure that physically disjoint PTP tunnels that don't share any link are used between the redundant Masters and a Slave. When redundant Masters are tracked by a Slave, any PTP LSP or PTP PW failure will trigger the slave to switch to the Redundant Master. However LSP/PW protection such as Linear Protection Switching (1:1,1+1), Ring protection switching or MPLS Fast Reroute (FRR) SHOULD still be used to ensure the LSP/PW is ready for a future failure. Note that any protection or reroute mechanism that adds additional label to the label stack, such as Facility Backup Fast Reroute, MUST ensure that the pushed label is a PTP Label to ensure proper processing of PTP messages by LSRs in the backup path.

To ensure the proper operation of 1588 Slaves, the physical path for PTP messages from Master to Slave and vice versa MUST be the same for all PTP messages listed in section 7 and MUST not change even in the presence of ECMP in the MPLS network. To ensure the forward and reverse paths are the same PTP LSPs and PWs MUST NOT be subject to ECMP.

In order to manage PTP LSPs and PTP PWs, they MAY carry OAM, Control and Management messages. These control and management messages can be differentiated from PTP messages via already defined IETF methods. In particular BFD , and LSP-Ping MAY run over PTP LSPs via UDP/IP encapsulation or via GAL/G-ACH. These Management protocols are easily identified by the UDP Destination Port number or by GAL/ACH respectively. Also BFD, LSP-Ping and other Management messages MAY run over PTP PW via one of the defined VCCVs (Type 1, 2 or 3) . In this case G-ACH, Router Alert Label (RAL), or PW label (TTL=1) are used to identify such management messages.

The PTP messages are time critical and must be treated with the highest priority. Therefore 1588 over MPLS messages must be treated with the highest priority in the routers. This can be achieved by proper setup of PTP tunnels. The PTP LSPs MUST be setup and marked properly to indicate EF-PHB for the CoS and Green for drop eligibility

Ethernet FCS MUST be recalculated at every LSR that performs the TC processing and FCS retention described in MUST not be used.

MPLS-TE routing relies on extensions to OSPF in order to advertise Traffic Engineering (TE) link information used for constraint-based routing. Indeed, it is useful to advertise data plane TE node capabilities, such as the capability for a router to be 1588-aware. This capability MUST then be taken into account during path computation to prefer nodes that advertise themselves as 1588-aware, so that the PTP LSPs can be properly handled. For this purpose, the following sections specify OSPF extensions in order to advertise 1588 aware capabilities of a node.

This extension makes use of the Router Information (RI) Opaque LSA defined in for both OSPFv2 and OSPFv3, by defining a new OSPF Router Information (RI) TLV - The 1588-aware Capability TLV. The 1588-aware Capability TLV is OPTIONAL and is defined as follows:

0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Type | Length | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Flags | +-+-+-+-+-+-+-+-+ Figure 3: 1588-aware Capability TLV Where: Type, 16 bits: 1588-aware Capability TLV where the value is TBD Length, 16 bits: Gives the length of the flags field in octets, and is currently set to 1 Flags, 8 bits: The bits are defined least-significant-bit (LSB) first, so bit 7 is the least significant bit of the flags octet.

+-+-+-+-+-+-+-+-+ | Reserved |C| +-+-+-+-+-+-+-+-+ Figure 4: Flags Format Correction (C) field Update field, 1 bit: Setting the C bit to 1 indicates that the node is capable of recognizing the PTP event packets and can compensate for residence time by updating the PTP packet Correction Field. When this is set to 0, it means that this node cannot perform the residence time correction but is capable of performing MPLS frame forwarding of the frames with PTP labels using a method that support the end to end delivery of accurate timing. The exact method is not defined herein. Reserved, 7 bits: Reserved for future use. The reserved bits must be ignored by the receiver. The 1588-aware Capability TLV is applicable to both OSPFv2 and OSPFv3. The 1588-aware Capability TLV MAY be advertised within an area-local or autonomous system (AS) scope Router Information (RI) LSA. But the 1588-aware Capability TLV SHOULD NOT be advertised into an area in more than one RI LSA irrespective of the scope of the LSA. The flooding scope is controlled by the Opaque LSA type in OSPFv2 and by the S1 and S2 bits in OSPFv3. For area scope, the 1588-aware Capability TLV MUST be carried within an OSPFv2 Type 10 RI LSA or an OSPFv3 RI LSA with the S1 bit set and S2 bit clear. If the flooding scope is the entire routing domain (AS scope), the 1588-aware Capability TLV MUST be carried within an OSPFv2 Type 11 RI LSA or OSPFv3 RI LSA with the S1 bit clear and the S2 bit set.

RSVP-TE signaling MAY be used to setup the PTP LSPs. A new RSVP object is defined to signal that this is a PTP LSP. The OFFSET to the start of the PTP message header MAY also be signaled. Implementations can trivially locate the correctionField (CF) location given this information. The OFFSET points to the start of the PTP header as a node may want to check the PTP messageType before it touches the correctionField (CF). The LSRs that receive and process the RSVP-TE/GMPLS messages MAY use the OFFSET to locate the start of the PTP message header. Note that the new object/TLV Must be ignored by LSRs that are not compliant to this specification. The new RSVP 1588_PTP_LSP object should be included in signaling PTP LSPs and is defined as follows:

0 1 2 3 +-------------+-------------+-------------+-------------+ | Length (bytes) | Class-Num | C-Type | +-------------+-------------+-------------+-------------+ | Offset to locate the start of the PTP message header | +-------------+-------------+-------------+-------------+ Figure 5: RSVP 1588_PTP_LSP object The ingress LSR MUST include this object in the RSVP PATH Message. It is just a normal RSVP path that is exclusively set up for PTP messages

A 1588-aware LER advertises it's 1588-awareness via the OSPF procedure explained in earlier section of this specification. The 1588-aware LER then signals PTP LSPs by including the 1588_PTP_LSP object in the RSVP-TE signaling. When a 1588 message is received from a non-MPLS interface, the LER MUST redirect them to a previously established PTP LSP. When a 1588 over MPLS message is received from an MPLS interface, the processing is similar to 1588-aware LSR processing.

1588-aware LSRs are LSRs that understand the 1588_PTP_LSP RSVP object and can perform 1588 processing (e.g. TC processing). A 1588-aware LSR advertises it's 1588-awareness via the OSPF procedure explained in earlier section of this specification. When a 1588-aware LSR distributes a label for PTP LSP, it maintains this information. When the 1588-aware LSR receives an MPLS packet, it performs a label lookup and if the label lookup indicates it is a PTP label then further parsing must be done to positively identify that the payload is 1588 and not OAM, BFD or control and management. Ruling out non-1588 messages can easily be done when parsing indicates the presence of GAL, ACH or VCCV (Type 1, 2, 3) or when the UDP port number does not match one of the 1588 UDP port numbers. After a 1588 message is positively identified in a PTP LSP, the PTP message type indicates what type of processing (TC) if any is required. After 1588 processing the packet is forwarded as a normal MPLS packet to downstream node.

It is most beneficial that all LSRs in the path of a PTP LSP be 1588-aware LSRs. This would ensure the highest quality time and clock synchronization by 1588 Slaves. However, this specification does not mandate that all LSRs in path of a PTP LSP be 1588-aware. Non-1588-aware LSRs are LSRs that either don't have the capability to process 1588 packets (e.g. TC processing) or don't understand the 1588_PTP_LSP RSVP object. Non-155-aware LSRs ignore the RSVP 1588_PTP_LSP object and just switch the MPLS packets carrying 1588 messages as data packets and don't perform any TC processing. However as explained in QoS section the 1588 over MPLS packets MUST be still be treated with the highest priority.

The use of Explicit Null (Label= 0 or 2) is acceptable as long as either the Explicit Null label is the bottom of stack label (applicable only to UDP/IP encapsulation) or the label below the Explicit Null label is a PTP label. The use of Penultimate Hop Pop (PHP) is acceptable as long as either the PHP label is the bottom of stack label (applicable only to UDP/IP encapsulation) or the label below the PHP label is a PTP label.

MPLS PW security considerations in general are discussed in and ,and those considerations also apply to this document. An experimental security protocol is defined in [1]. The PTP security extension and protocol provide group source authentication, message integrity, and replay attack protection for PTP messages.

A new 1588-aware Capability TLV is required to signal 1588-awreness via OSPF. IANA needs to assign a new Type for such TLV. Also a 1588_PTP_LSP RSVP object is required to signal PTP LSP. IANA needs to assign a new CLASS-Num and C-Type for 1588_PTP_LSP object.