Managing Audio in IP Production

Presenters : Andy Rayner - Nevion

Year : 2022

dlp_document_download : Audio in IP production Andy Rayner, Chief Technologist, Nevion arayner@nevion.com +44 7711 196609 Come and catch up on the Sony stand in Hall 13 Queen Elizabeth II 1926-2022 The Queen and I Don’t forget the audio- I am an sound guy! Event location or regional facilities OB truck or local production Central facility Distributed production Management Public Cloud 5G Home Private Cloud IP transformation - Places, Processing & People Audio requirements drive scale in facilities AUDIO -30 The Audio folks did facility IP first! ‘Most of the complexity of a production environment is the audio’ Dante 2006 Ravenna 2010 Audio facility interconnects Remote commentary location Event/ alternate location Production Center Re-sync Re-sync History of sampling - PCM is 100 years old! The First Disclosure of PCM: Paul M. Rainey, " Facimile Telegraph System," U.S. Patent 1,608,527, Filed July 20, 1921, Determining sampling rate Williard M. Miner, "Multiplex Telephony," U.S. Patent 745,734, Filed February 26, 1903 Then Nyquist in 1924 9 A-D conversion! 1954 "DATRAC" 11-bit, 50-kSPS Vacuum Tube ADC Designed by Bernard M. Gordon at EPSCO 500watts, 150lbs, $8500 AD572 12-Bit, 25-μs Mil Hybrid ADC, 1977 Audio acquisition Sample frequency Bit depth Time of capture Standardized Audio interfaces Standardized IP formats Proprietary IP formats Lots of different audio interfaces and formats! Analogue AES3 SDI embedded IP 2022-6 embedded IP - 3326 IP - AES67 AES10 MADI Livewire DANTE RAVENNA EBU 3326 AES67 ST2110-30/31 AES50 AoE WheatNet Q-LAN Audio production interoperability Audio data flows – almost identical Audio control plane – very different Audio parameters……………lots of choices AES10 - MADI: 28, 56, 64 channels Sample rate: 32kHz - 96kHz Bit depth: 16bit, 20bit, 24bit AES3: 2 channels Sample rate: 16kHz, 22kHz,32kHz,44.1kHz,48kHz,88.1kHz,96kHz,192kHz Bit depth: 16bit, 20bit, 24bit AES3 frame period: 1ms – 12ms AES67: 1 - 120 channels ( 1-8 ) Sample rate: 44.1kHz, 48kHz , 96kHz Bit depth: 16bit, 24bit IP Packet period: 125us, 250us, 333⅓us, 1ms , 4ms ST2110-30: 1-8 channels (1-64 channels Level C) Sample rate: 48kHz (level A) , 96kHz (level –x) Bit depth: 16bit, 24bit IP Packet period: 1ms (level A) 125us (level B), SYSTEM -10 AUDIO -30 VIDEO -20 COMPRESSED VIDEO -22 AES3-32 bit AUDIO -31 TIMING -21 2022-8 COMPOSITE ANCILLARY DATA -40 MULTI-PART VIDEO -23 FAST METADATA -41 FMX -42 SD VIDEO -24 IS-04 Discovery and Registration IS-05 Connection Management IS-08 Audio Channel Mapping IS-07 Event and Tally IS-09 System BCP-003-0x Security suite The ST2110 suite & NMOS 15 Handling ‘other’ audio: the ST2110-31 bits AES3-32 bit AUDIO -31 AUDIO -30 Timing and referencing GNSS GPS TaaS ST2059 – applying PTP IEEE1588 to media Media payload RTP UDP IP VLAN MAC/ PoS / GFP/MPLS Origination time stamping in RTP Audio per-sample timing TS=n TS=n+1 TS=n+2 Time e.g. 20ms Video – per-frame sampling timing ST2110-20 freezes RTP timestamp! What about the standardized control plane? Audio data stream Audio data stream Controller IS-04, IS-05, IS-08 The SDP - RFC 4566 – used in ST2110 Sender description Video and/or audio essence Raster size (in pixels) Frame-rate (video) Channel count (audio) Sampling structure (audio/video) Bit depth (audio/video) Colorimetry Source IP address and port RTP payload ID (audio/video) PTP grandmaster source and domain FAST METADATA -41 FMX -42 Coming soon Audio manipulation required in IP domain GAIN DELAY DOWNMIX SHUFFLE IP IP How to shuffle in the modern world Mono AES67 Mono AES67 Mono AES67 Mono AES67 Mono AES67 Mono AES67 Mono AES67 Mono AES67 IP audio proc AES67 1-64ch AES67 1-64ch AES67 1-64ch AES67 1-64ch AES67 1-64ch AES67 1-64ch AES67 1-64ch AES67 1-64ch End device selects AES67 1-64ch Audio templates – a few practical options One channel - mono Two channels - stereo Six channels – 5.1 Eight channels – 4 x stereo Video-associated audio format interfacing ST 2022-6 16 Mono ST 2022-6 16 Mono ST 2022-6 16 Mono ST 2022-6 16 Mono ST2110-30 8 Mono ST2110-30 8 Mono ST2110-30 8 Mono ST2110-30 8 Mono SDI 16 Mono SDI 16 Mono SDI 16 Mono SDI 16 Mono ST 2022-6 16 Mono ST 2022-6 16 Mono ST 2022-6 16 Mono ST 2022-6 16 Mono ST2110-30 8 Mono ST2110-30 8 Mono ST2110-30 8 Mono ST2110-30 8 Mono SDI 16 Mono SDI 16 Mono SDI 16 Mono SDI 16 Mono Audio-only format interfacing 2110-31 16 Mono 2110-31 16 Mono 2110-31 16 Mono 2110-31 16 Mono 2100-30 8 Mono 2110-30 8 Mono 2110-30 8 Mono 2110-30 8 Mono Async in 16 Mono Async in 16 Mono Async in 16 Mono Async in 16 Mono AES3 16 stereo Analogue 16 Mono AES3 16 stereo Analogue 16 Mono AES67 64 Mono AES67 64 Mono AES67 64 Mono AES67 64 Mono MADI 64 Mono MADI 64 Mono MADI 64 Mono MADI 64 Mono Moving outside the campus – distributed production WAN connectivity involved Longer latencies (Potentially) Asynchronous sources Layer 2 too limiting Layer 3 (routed) needed for larger and multi-campus networks 28 PTP GM PTP GM Interconnecting proprietary audio Islands 29 Plug and play ‘toolkit’ needed to transition to standardised formats Gateway via ‘tie-lines’ SDN interconnect AES67 – defines link offset link time Hybrid timing reconciliation Processing time Processing time Processing time Origination time Use time link time link time link time link time link time Some chain timing preservation Control system Provides time coord PTP holdover is capable of being very long – let’s make it so! Protection – on and off campus – video & audio n n+1 n+2 n+4 n+5 n+6 n+8 n+9 n+10 n+3 n+7 n+11 n+12 n+13 n+14 n+16 n+17 n+18 n+20 n+21 n+22 n+15 n+19 n+23 n+24 n+25 n+26 n+27 n+28 n+29 n+30 n+31 n+32 n+33 n+34 n+35 n+36 n+37 n+38 n+39 n+40 n+41 n+42 n+43 (1,1) (L,1) (1,2) (1,D) (L,D) (2,1) (2,2) 1 L 2 flow A flow B SOURCE DESTINATION flow A flow B DESTINATION SOURCE delay Spatial Spatial + temporal FEC International use case - Audio signal flows Orchestration Secure networks need managed networks SDN control Going off-campus – the IP facility media edge PTP TIMING DEVICE DISCOVERY & CTL MEDIA FLOW IP ADDRESSING UNCOMPRESSED ESSENCE FLOWS PROTECTION TERMINATION ALTERNATIVE TIMING DOMAINS RESTRICTED/PROXY DISCOVERY & CTL DIFFERENT IP ADDRESSING (NAT) ESSENCE/COMPOSITE UN/COMPRESSED FLOWS PROTECTION TERMINATION ‘EXOTIC’ PROTOCOLS ST2110-WAN AG - two layers of focus – data and control Media essences data flows Registration, discovery and control TR-09 Gateway G1 TR-09 Gateway G2 Facility 1 Facility 2 WAN Encoding Encryption Error Protection Encapsulation CONTROL PLANE DATA PLANE Decoding Decryption Error Correction Decapsulation NMOS Resource Proxying NMOS Resource Proxying ST2110-WAN TR-09 Gateway G1 TR-09 Gateway G2 Facility 1 Facility 2 WAN CONTROL PLANE NMOS Resource Proxying NMOS Resource Proxying Proxy Receiver Proxy Receiver Receiver Sender Proxy Sender Proxy Sender IS-04 Query API IS-05 Conn API IS-10 Auth API Calls IS-04 Query API Calls IS-05 Conn API Calls IS-10 Auth API 2110-WAN control plane The broadcast end game Time-aware media processing chain Acquisition Consumption Production = real time & time aligned Deterministic data transfer Linear and non-linear TCP Infiniband SRD RTP/UDP FEC 2022-7 ARQ 41 In conclusion Remote commentary location Event/ alternate location Production Center Re-sync Re-sync Thank you! Andy Rayner Chief Technologist arayner@nevion.com +44 7711 196609 Come and catch up on the Sony stand in Hall 13 Check out our other presentations on-line Any Questions?