IPMX Required Features Overview

Presenters : Andreas Hildebrand - ALC NetworX

Year : 2020

dlp_document_download : Overview on essential Ingredients Referenc System Architecture Guide | April 2017 Integrated Systems Europe (ISE), RAI Amsterdam, Room G110 February, 11th 2020 Andreas Hildebrand RAVENNA Technology Evangelist ALC NetworX, Munich Andreas Hildebrand , RAVENNA Technology Evangelist • more than 25 years in the professional audio / broadcasting industry • graduate diploma in computer science • R&D, project & product management experience • member of AES67 TG and ST2110 DG ALC NetworX GmbH , Munich / Germany • established 2008 • R&D center • developing & promoting RAVENNA • Partnerships with > 40 manufacturers RAVENNA • IP media networking technology • designed to meet requirements of professional audio / broadcasting applications • open technology approach , license -free • fully AES67/ST2110 -compliant ( built -in ) What is IPMX? Internet Protocol Media ExperienceInternet Protocol Media Experience – Proposed Roadmap ( Draft ) SMPTE ST 2110 -22 CBR Compression in ST 2110 ISO/IEC 21122 JPEG XS Codec NMOS IS -04 & IS -05 Discovery, registration & connection management EDID / DisplayID / HPD Support* HDCP* Copy protection General Purpose I/O* IR Remotes, GPIO, USB, RS232 over IP NMOS IS -08* Audio channel mapping IPv6* Network addressing Security* Authentication, Encryption Audio over IP Standardized Transport of Audio, Video, & ANC Elements ProAV Standards & Specifications AES67 Audio Over IP SMPTE ST 2110 -10 Timing & Definitions SMPTE ST 2110 -20 Uncompressed Video SMPTE ST 2110 -21 Packet Pacing SMPTE ST 2110 -30 AES67 Audio SMPTE ST 2110 -31 AES3 Audio Transport SMPTE ST 2110 -40 Ancillary Data AES67 SMPTE ST 2110 (Required) (Optional)  AES67 − defines high -performance AoIP interoperability  SMPTE ST 2110 − defines elementary essence data transport on managed IP networks  SMPTE ST 2059 − defines PTP profile for SMPTE ST 2110  AMWA NMOS − defines upper layer operational & management functionalities (device discovery & connection management) Relevant Standards AES67 What was the original goal? • "Provide a method to connect disparate Audio -over -IP systems to achieve workaround -free networked audio interoperability" What is AES67? • Interoperability Standard for high performance Audio -over -IP networks • Based on existing protocols and trusted IT standards • This ensures compatibility with existing network infrastructure • Allows coexistence with other IT data • High adoption rate by all major solution providers AES67 AES67 Problem Statement • Audio -over -IP (aka Networked Audio) provides simpler and better connection between audio equipment • Coupled with many advantages, one clear challenge presented itself: Compatibility • While each Audio -over -IP solution offered in - system connectivity, there was no standard to provide inter -system connectivity AES67 IP RAVENNA Livewire Q -LAN WheatNet Dante X X X AES67 IP RAVENNA Livewire Q -LAN WheatNet Dante X X X X AES67 AES67 AES67 Synchronisation Media Clock Transport QoS Encoding Session Description Connection Management Discovery Not specified IEEE 1588 -2008 (PTPv2) 48 kHz RTP / UDP / IP, unicast & multicast Differentiated Services ( DiffServ w/ 3 CoS ) L16/L24, 1..8 ch , 48 samples SDP (RFC4566, RFC7273) SIP ( unicast ), IGMP ( multicast ) AES67 technology components AES67 - Status 2013 – initial publication • 3 years of work • 100+ participants (manufacturers, consultants, system integrators, end users) 2015 – first revision • Corrigendum & clarifications • Backward compatibility (no new requirements) 2018 – second revision • Further clarifications • Backward compatibility (no new requirements) • PICS added Current status • Task Group is working on further topics (multicast addressing, SDP, homologation w/ ST 2110 etc.) AES67 Further Work in AES Task Groups AES70 – Open Control Architecture • Initially published in 2017 • AES70 -2018 – improvements + additions (i.e. CM3, web sockets / UDP support, Task mechanism etc.) • Current work: further improvements, AES70 for AES67 Connection Management, NMOS convergence X238 – Requirements for Media Network Directories and Directory Services • setting forth technical recommendations for media network directories and directory -related services • Covering: Registration, query, and administration protocols; security mechanisms; directory data model; query language and related semantics; scalability strategies X242 – Streaming audio metadata over IP • Defining a transport mechanism for real -time audio meta data associated with an AES67 audio stream • Synchronization, RTP payload format (but agnostic to actual payload data), association & alignment w/ AES67 streams • Seeking alignment / coordination w/ SMPTE ST 2110 -41 ("Transport of Extensible Fast Metadata") SMPTE ST 2110 – Professional Media over Managed IP Networks The SMPTE ST 2110 standards suite specifies • the transport, synchronization and description of • separate elementary essence streams (video, audio, ancillary data) • over managed IP networks (at any speed, from 1GbE to 100 GbE and beyond) • for real -time production, playout and other professional media applications. Two Fundamental Approaches to IP Transport • Bundled (Audio, Video, Metadata together) • Audio/Video/Metadata/Sync travel coherently • Requires extra work to "unpack" separate essences • Essence -based (Audio, Video, Metadata separate) • Ideal for Studio/Production workflows • Individual essence kept in sync using PTP timing The Bundled Approach: SMPTE ST 2022 - 6 Published beginning in 2012 The Essence -based Approach: SMPTE ST 2110 Published in 2017 ST 2022 -6 ST 2110 Bundled vs. Essence - based Approach ST2110 Sender ST2110 Receiver 1 ST2110 Receiver 2 SDI The Essence Based Approach: SMPTE ST 2110 Document structure (published ): • 2110 -10: System Timing & Definitions − defines transport layer and synchronization (SMPTE2059, clocks, RTP, SDP etc.) • 2110 -20: Uncompressed Active Video − defines payload format for raw video (RFC4175, RTP, SDP, constraints) • 2110 -21: Traffic Shaping and Delivery Timing for Uncompressed Active Video − defines timing model for senders and receivers (traffic shaping requirements) The SMPTE ST 2110 Suite of Standards Document structure (published ): • 2110 -30: PCM Digital Audio − defines payload format for linear audio (AES67, constraints) • 2110 -31: AES3 Transparent Transport − defines payload format for non -linear audio (RAVENNA AM824) • 2110 -40: Transport of SMPTE Ancillary Data − defines RTP payload format for SDI ancillary data (new IETF draft) The SMPTE ST 2110 Suite of Standards P recision T ime P rotocol (IEEE 1588 -2008) • A method for distributing precise, GPS -referenced time over an IP network • Proven technology used in multiple industries • Used for synchronization and alignment of devices and media signals Synchronization and Alignment in ST 2110 Both AES67 and SMPTE ST 2110 use PTP R eal -time T ransport P rotocol (RFC 3550) • A format -agnostic transport protocol for real -time media data • Includes time information for precise media alignment Media Transport in ST 2110 Application Presentation Session Transport Network Data Link Physical OSI Layer RTP UDP IP Ethernet Copper / Fiber (up tp 1440 bytes ) S ession D escription P rotocol (RFC 4566 ) • Required to describe stream formatting, synchronization and connection information • Provided by a sender (or management instance) for each stream • Human -readable text: Signalization in ST 2110 v=0 o=1 0 IN IP4 192.168.1.100 s=RAVENNA demo stream t=0 0 a= ts -refclk:ptp =IEEE1588 -2008:00 -60 -6e -ff-fe -7c -23 -0f:0 a= mediaclk:direct =0 m=audio 5004 RTP/AVP 98 a=rtpmap:98 L24/48000/2 c=IN IP4 239.3.14.142 a= recvonly a=ptime:1 Specifies the payload format for uncompressed active video essence • Raster size independent 4 up to 32K x 32K pixels • Agnostic to: 4 Colour sampling: 4:1:1 to 4:4:4+ 4 Bit depth: 8 to 16 -Bit+ 4 Frame -rate: 23.98 to 120 fps+ • Support for HDR 4 PQ & HLG • Significant bandwidth efficiency, i.e. 1080p50: 4 ST 2022 -6 = 3,074Gbps 4 ST 2110 -20 = 2,143Gbps SMPTE ST 2110 - 20 Video - Uncompressed  30% bandwidth saving ! SMPTE ST 2110 -21 Traffic Shaping and Delivery Timing for Video Problem Overview • Video flows require fairly high data rates • Multiples flows are concurrently traversing the network • Network switches (and receivers) have limited buffer capacity • Constant data flow will not overload the buffers as long as total used bandwidth stays below maximum bandwidth Inflow Data out Buffer Drain Data in Problem Overview • Random and unregulated traffic patterns may temporarily overflow buffers, even if average bandwidth is not exceeded Bursty inflow Larger buffer required Constant drain Constant data out Bursty data in Bucket may overflow ! Packets may get dropped ! SMPTE ST 2110 -21 Traffic Shaping and Delivery Timing for Video Solution • Define sender drain behaviour (packet egress pacing and spacing)* • 3 models for sender traffic shaping: 4 Narrow -linear (NL) – packet are drained evenly distributed across frame period 4 Narrow (N) – packet drain closely follows SDI signal timing (no packets during VBI and VANC) 4 Wide (W) – allows increased burstiness (accommodates SW -based senders) SMPTE ST 2110 -21 Traffic Shaping and Delivery Timing for Video *and (receiver) buffer requirements Solution • Define sender drain behaviour (packet egress pacing and spacing) • 3 models: 4 Narrow -linear 4 Narrow 4 Wide SMPTE ST 2110 -21 Traffic Shaping and Delivery Timing for Video Solution • Define sender drain behaviour (packet egress pacing and spacing) • 3 models for sender traffic shaping : 4 Narrow (N) – packet drain closely follows SDI signal timing (no packets during VBI and VANC) 4 Narrow -linear (NL) – packet are drained evenly distributed across frame period 4 Wide (W) – allows increased burstiness (accommodates SW -based senders) • Sender behaviour is signalled in the SDP in the a= fmtp :line: 4 TP=2110TPN 4 TP=2110TPNL 4 TP=2110TPW • Has impact on buffer requirements of network switches and receiver devices and sender / receiver compatibility SMPTE ST 2110 -21 Traffic Shaping and Delivery Timing for Video Specifies the payload format for PCM digital audio streams • Uncompressed Linear PCM Audio only • Based on AES67 • Relatively flexible: 4 48kHz and 96kHz sampling 4 16 and 24 -bit depth 4 Variable packet timing - 125us to 1ms 4 Channel -count based on packet timing - 8 channels @ 1ms up to 64 channels @ 125us (conformance levels A / B / C) • Low -bandwidth consumption, i.e. 4 8 channels x 24 bits x 48,000 samples = 9.9Mbits/sec (incl. packet overhead) SMPTE ST 2110 - 30 Linear PCM Audio AES67 SMPTE ST 2110 -30 and AES67 Compatibility AES67 Specifies the payload format for transparent transport of AES3 audio data • Can transport any format which can be encapsulated in AES3: 4 L24 PCM w/ AES3 subframe meta data (PCUV bits) 4 non -PCM audio and data formats as defined by SMPTE ST 337 / 338 (i.e. Dolby®E etc.) • Builds on RAVENNA’s AM824 (IEC 61883 -6) payload definition: 4 retains AES67 definitions for synchronization and RTP usage 4 uses 3 bytes for PCM24 + 1 byte for AES3 meta data 4 RTP payload format signaled in SDP: a= rtpmap :< pt > AM824 /48000/< nchan > 4 retains all other SDP parms SMPTE ST 2110 - 31 AES3 Audio Data 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |0|0| B|F| PCUV | DATA24 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ SMPTE ST 2110 -40 T ransport of Ancillary Data Specifies the method of transporting (SDI) ancillary data via RTP • Covers a ncillary data as specified in SMPTE ST 291 -1, i.e.: 4 Timecode 4 Closed captions 4 Subtitles 4 Active format descriptions • Not intended for the carriage of audio data ( → SMPTE ST 2110 -31) or EDH (error, detection and handling) Document structure (in development) : • 2110 -22: Constant Bit -rate Compressed Video − defines payload format for CBR compressed video and a SMPTE registry for various payload formats (codecs) • 2110 -23: Single Video Essence Transport over Multiple 2110 -20 Streams − defines how to split high -bandwidth signals into several lower -bandwidth 2110 -20 tributary streams (constraints, grouping, addressing, RTP timestamps, SDP …) • 2110 -41: Extensible Fast Metadata Transport − defines how to transport extensible, dynamic meta data in ST2110 context (including synchronization) The SMPTE ST 2110 Suite of Standards Further Work in SMPTE DGs ST 2110 • 1-year review of: -10 / -20 / -21 / -40 • New work: -24 (SDO Definitions) • PICS (analog to AES67 -2018) ST 2059 • 1-year review of: -1 & -2 • ST 2059 Security • PTP Monitoring Related : VSF • ST 2110 -over -WAN, ST 2110 format enumeration IEEE1588 -2020 (PTPv2.1) • Improvements and extensions, fully backwards -compatible What else is required for a working system ?  Establishing connections!* − Not covered by SMPTE 2110  − AMWA: Advanced Media Workflow Association − NMOS: "A growing family of specifications […] which are complementary to and co -exist with industry specifications like ST2110 and AES67" (* and control…) NMOS specifications : • IS -04: Discovery & Registration − enumeration and registration of available system resources • IS -05: Connection Management − connecting receivers to available streams • IS -06: Network Control − controlling network resources (configuring routing tables) NMOS specifications : • IS -07: Event & Tally Specification − communicate current states and state changes • IS -08: Audio Channel Mapping − how to match flow channels with inputs / outputs • BCP -002 -01: Grouping of NMOS Resources − identifying which flows are related to each other • … more in the works… Key elements Node Device Receiver Sender 256E5638 -0EB2 -4E70 - B45B -3B24BEB6A478 83 C42DF8 -284E -4351 - 8349 -E50DA22AC419 6C46C3F0 -97F0 -4852 - 9D85 -8D4B0325A488 A38548F4 -99EE -40CF - 9582 -94A51B1BCAC1 Source Flow 60752 B84 -F055 -4E32 - 9D7F -9619DA31406A 23F85482 -7AE1 -4366 - 9DD3 -BD8483957A91 Identity IS - 04 Discovery & Registration Ensure that parts of a networked media system can find each other Node Device Source Flow Receiver Sender Registry Registration Query 256E5638 -0EB2 -4E70 - B45B -3B24BEB6A478 83 C42DF8 -284E -4351 - 8349 -E50DA22AC419 6C46C3F0 -97F0 -4852 - 9D85 -8D4B0325A488 A38548F4 -99EE -40CF - 9582 -94A51B1BCAC1 60752 B84 -F055 -4E32 - 9D7F -9619DA31406A 23F85482 -7AE1 -4366 - 9DD3 -BD8483957A91 IS - 05 Connection Management Make it simple for applications to (dis)connect flows Node Device Sender Node Device Receiver Application Logic IS -04 Registry Registration any format / protocol Create Connection stream Challenge: 6 channel surround audio Challenge: Challenge: R L LFE RS C R L LFE LS RS 6 unicast streams with individual channels Option 1:  not very efficient LS C 6 Challenge: 6 6 6 6 6 6 1 multicast stream w/ all 6 channels Option 2: LS C R L LFE RS Challenge: 1 multicast stream w/ all 6 channels IS - 08 Audio Channel Mapping Map flow channels (tracks) to device I/O channels NMOS IS -08 AMWA NMOS IS -08 - Audio Channel Mapping Sources Input1 2ch Output1 Sinks Device Map 2ch • Interaction with NMOS IS -05 – connecting incoming stream channels to device output channels AMWA NMOS IS -08 - Audio Channel Mapping AMWA NMOS IS -08 - Audio Channel Mapping IP -SDI Gateway More information on NMOS wiki on Github : https://github.com/AMWA - TV/nmos/wiki Standards: AES/ IEEE / SMPTE Standard Description Status AES67 -2018 High -performance streaming audio -over -IP interoperability Approved IEEE1588 -2008 Precision Time Protocol Approved SMPTE ST 2022 -6 High Bit -rate Media Transport (SDI over IP) Approved SMPTE ST 2022 -7 Seamless Protection Switching (RTP streams) Approved SMPTE ST 2022 -8 Timing of ST 2022 -6 streams in ST 2110 -10 systems In Progress SMPTE ST 2059 -1 Generation of PTP Signals (including definition of Epoch) Approved SMPTE ST 2059 -2 SMPTE Profile for IEEE1588 -2008 (operating parameters) Approved Standards: SMPTE ST 2110 Suite Standard Description Status SMPTE ST 2110 -10 System -PTP, RTP and SDP Approved SMPTE ST 2110 -20 Video -Uncompressed Approved SMPTE ST 2110 -21 Video -Traffic Shaping (packet pacing, bursts and gaps) Approved SMPTE ST 2110 -22 Video -Compressed (CBR) Approved SMPTE ST 2110 -23 Video – Multistream (high -bw stream as separate low -bw streams) In Progress SMPTE ST 2110 -30 Audio -Uncompressed (PCM) Approved SMPTE ST 2110 -31 Audio -Compressed (AES3, non -PCM) Approved SMPTE ST 2110 -40 Data -Ancillary (SDI) Approved SMPTE ST 2110 -41 Data -Fast Meta Data In Progress Standards ? Technical Recommendations Standard Description AMWA NMOS IS -04 Device Discovery and Registration AMWA NMOS IS -05 Connection Management AMWA NMOS IS -06 Network Control AMWA NMOS IS -07 Event & Tally AMWA NMOS IS -08 Audio Channel Bundling AMWA NMOS IS -xy Flow Grouping, ID & Timing, Scalability, Security & more SMPTE ST RDD 34 Sony LLVC compression SMPTE ST RDD 35 IntoPIX TICO compression JT -NM TR -1001 -1 System Environment and Device Behavior Questions ? www.ravenna -network.com/resources ST2110 / AES67 Resources: www.aimsalliance.org ( resources ) www.smpte.org/smpte -st -2110 -faq Answers ! Contact information : www.ravenna -network.com Andreas Hildebrand ALC NetworX GmbH ravenna@alcnetworx.de RAVENNA booth @ 7 -V220