Linear and Non-Linear Timing Workflows in Live IP Production

Presenters : Andy Rayner - Nevion

Year : 2022

dlp_document_download : Timing planes in IP production linear and non-linear Andy Rayner, Chief Technologist, Nevion arayner@nevion.com +44 7711 196609 Come and catch up on the Sony stand in Hall 13 Queen Elizabeth II 1926-2022 The Queen and I The fundamental challenges Linear connection-based signal flow Non-linear compute-based data flow Capturing time Tracking time Optimising time The relevant linear standards journeys RFC 3550 RTP RFC 4175 Uncompressed video mapping AES67 IP audio VSF TR-03 ES media over IP SMPTE ST 2210 RFC 768 UDP SMPTE ST 292M HD-SDI SMPTE ST 424M 3G-SDI IEEE 1588 PTP SMPTE ST 2059 use of PTP time SYSTEM -10 AUDIO -30 VIDEO -20 COMPRESSED VIDEO -22 AES3-32 bit AUDIO -31 TIMING -21 2022-8 COMPOSITE ANCILLARY DATA -40 MULTI-PART VIDEO -23 FAST METADATA -41 FMX -42 SD VIDEO -24 IS-04 Discovery and Registration IS-05 Connection Management IS-08 Audio Channel Mapping IS-07 Event and Tally IS-09 System BCP-003-0x Security suite The linear ST2110 suite (& NMOS ctl ) 5 The broadcast end game Time-aware media processing chain Acquisition Consumption Production = real time & time aligned Why is ST2110-21 currently so critical? IP Packet integrity UDP TCP SRD RoCE Media payload RTP UDP IP VLAN MAC/ PoS /GFP/MPLS IETF RFC3550 Data transfer between Virtual Processing Functions Deterministic data transfer Media mapping Media mapping 8 Compute does not do linear data transfer easily! Deterministic data transfer TCP Infiniband SRD RTP/UDP FEC 2022-7 ARQ 9 AWS CDI SRD 10 Reconciling media essence timings Processing time Processing time Processing time Origination time Use time link time link time link time link time link time SYSTEM -10 AUDIO -30 VIDEO -20 COMPRESSED VIDEO -22 AES3-32 bit AUDIO -31 TIMING -21 2022-8 COMPOSITE ANCILLARY DATA -40 MULTI-PART VIDEO -23 FAST METADATA -41 FMX -42 SD VIDEO -24 IS-04 Discovery and Registration IS-05 Connection Management IS-08 Audio Channel Mapping IS-07 Event and Tally IS-09 System BCP-003-0x Security suite The ST2110 suite & NMOS - revisions 12 Timing propagation through system – ST 2110 revisions RTP mode: NEW T0 RTP mode: PRESERVE Egress time point = t0+ta RTP mode: PRESERVE Egress time point = t0+ta+tb RTP mode: NEW Egress time point = t0+ta+tb+tc Time plane is T1 Ta = time from egress of previous device to time of egressing this device T1-T0 = T NEWRTP - T RTP Tb tc Vision Mixer Audio Mixer Host Production Replay ISO To File Cam Vision Mixer Package Graphics (match clock) Program Pres (unilateral) Vision Mixer Cam Studio Channel Presentation Vision And Audio Mixer Mic Audio Mixer Audio Mixer Mic Mic SSM Camera Playout Commentary File Package File Data Stats Camera Example drawing by Prin Boon, Phabrix Example of production timing planes Sources that cross timing planes t0 Event Production switcher Presentation switcher ISO feeds also used directly Multiple production timing planes t0 Final Channel playout Remote control surfaces Remote commentary Event Production switcher Presentation switcher Remote/ Radio Cameras ! Profanity delay AR/VR set time Specific real-world use-case in 2021 VR system 200ms Cloud <-> Ground Cloud <-> Cloud VSF GCCG The raster scan Linear broadcast technology Software procession execution time t max Compute doesn’t easily do linear data handoff Compute latency and variability Processing Step A Processing Step B Processing Step X t t LA MIN LA 99% t LW 99% t LX MIN LX 99% +LW 99% LX 99% Diagram courtesy of VSF GCCG AG 23 VPF 1 VPF 3 VPF 4 VPF 5 VPF 2 Concatenated virtual processing functions, each with defined (max) execution time t1 t2 t3 t4 t5 Time orchestration Same principles apply in virtualized world Processing time Processing time Processing time Origination time Use time link time link time link time link time link time t max t max t max Media Source timings t0 Local PTP-locked sources Remote PTP-locked sources Transit delay – phase shift t0 Asynchronous ‘rogue’ remote sources Proxy remote production timing reconciliation Compressed lower resolution proxy images + audio (t=x) Edit, mix, tally control data (t=y) delay = transit time plus any compression etc t+0 t+x Audio-video Iip sync ~5ms Coherent audio sync ~20us Absolute time delivery ~20ms Timing requirement examples Hybrid timing reconciliation Processing time Processing time Processing time Origination time Use time link time link time link time link time link time Some chain timing preservation Control system Provides time coord What is needed to make this work Control system provides time coordination All media processing devices throughout chain provide necessary information and accept required configurations In-flight SMPTE RP/EG: ST2110-11 Draft an Recommended Practice document that defines a workable ecosystem to provide ‘automatic’ reconciliation of media essence timing at any point along a production chain. Still time to get involved! Optimising latency Frame n Frame n+1 Frame n+2 Frame n+3 The fundamental challenges Linear connection-based signal flow Non-linear compute-based data flow Capturing time Tracking time Optimising time IP networking tutorial Back 13:00 Sunday & 10:30 Monday 11:00 SUNDAY Audio in live IP production 10:30 MONDAY Thank you! Andy Rayner Chief Technologist arayner@nevion.com +44 7711 196609 Come and catch up on the Sony stand in Hall 13 Any Questions?