Audio Meta Data Transport

Presenters : Kent Terry - Dolby

Year : 2019

dlp_document_download : Audio Metadata Tra n s p o r tAES New York 2019AVoIPPavilion -Session AIP22Kent TerrySr. Mgr., Sound Technology, Office of the CTO, Dolby LaboratoriesTask Group Leader, AES SC-02-12-R (Task group on Streaming audio metadata over IP) © 2016-17 DOLBY LABORATORIES, INC.Audio Metadata for Real Time Interfaces and Live WorkflowsAudio Metadata•All information about an audio program other than the PCM waveforms•Many coded signals include at least some metadata in the bitstreamStatic Metadata (per program/stream)•Number of channels, sample rate, program title, channel labels, …•Tra n s m i s s i o n t i m i n g va r i a b l e ; s o m e p e r st re a m s e t u p ( e . g . s a m p l e ra t e )Dynamic (time-varying) Metadata•Loudness, spatial location, dialog levels, …•Timing required, potentially "sample accurate"Optional vs. Required•In current channel based practice metadata can be "optional" (e.g. fixed # channels, channel assignments, …)•Object based methods and other Next Generation Audio applications "require" metadata–i.e. not possible to accurately reproduce intended audio2 © 2016-17 DOLBY LABORATORIES, INC."Microphone to Speaker" metadata3Audio Metadata in Live IP Streamsemission codermixing console © 2016-17 DOLBY LABORATORIES, INC.Audio Metadata in Legacy/"Non-IP" Live ProductionLive production currently channel based in practice•2.0 or 5.1 most commonLimited support for audio metadata, especially dynamic (time varying) metadata•But generally "sufficient" for current production practicesLack of standards defining audio metadata and transport of metadata for live production•Very limited support for dynamic audio metadataSMPTE ST 337 –"Non-PCM data in AES3" in common use•Includes formats for compressed audio with included metadata (e.g. Dolby E)Ver y limited support for > 5.1 and new immersive formats (e. g. object based audio)•Dolby ED2 for production/distribution4 © 2016-17 DOLBY LABORATORIES, INC.SMPTE ST 337 Data Types for Audio Metadata over AES3SMPTE ST 2109 –"A u d i o M e t a d a t a o v e r A E S 3 "•Defines general framework for transport of audio metadata in AES3 via ST337•Payload based model•Defines transport only -open to any metadata format•SMPTE RDD 49 (PMD) defines a payload based audio metadata format optimized for ST 2109SMPTE ST 2116 –"Serial ADM over AES3"•Tra n s p o r t o f I T U-R BS.2125 audio metadata in AES3 via ST 337•Audio Definition Model (ADM) BS.2076 data formatted for serial interchange•Currently SMPTE Draft PublicationBoth support sample accurate transport of live dynamic audio metadata5 © 2016-17 DOLBY LABORATORIES, INC.Audio Metadata in Legacy/"Non-IP" Interfaces6SDIHD-SDIMPEG-2 TSAES3Serial Transport InterfacesPCM Audio with dynamic metadataPMD or other"Non-PCM" DataPCM AudioSMPTEST 272SMPTEST 299SMPTEST 302AES3SMPTEST 337SMPTEST 2109Serial ADMSMPTEST 337SMPTEST 2116Audio Metadata © 2016-17 DOLBY LABORATORIES, INC.Audio over IP (AoIP)7DanteLivewireRAVENNAWheatnetQ-LANAES67AES67 -streaming audio-over-IP interoperabilityA single standard for interchangeAES67 is "PCM Audio" only with limited signaling metadataMetadata interchange? © 2016-17 DOLBY LABORATORIES, INC.Audio/Video over IP (AVoIP)SMPTE ST 2022-6 -Tra n s p o r t o f H i g h B i t R a te M e d i a S i g n a l s o v e r I P N e t w o r ks ( H B R M T )•Includes SDI audio (embedded in ancillary space)SMPTE ST 2110 Series -Professional Media Over Managed IP NetworksSMPTE ST 2110-30 -PCM Digital Audio•Based on AES67•As with AES67 transports PCM Audio only; no support for "non-PCM" dataSMPTE ST 2110-31 -AES3 Transparent Transport•Based on RAVENNA AM824•Carries full AES3 signal with channel status/user data•Supports "non-PCM" data; carriage of coded audio a primary application8 © 2016-17 DOLBY LABORATORIES, INC.Audio Metadata over IP (current)9SDI over IPMPEG-2 TS over IPAES3 over IPIP/RTP TransportPCM Audio over IPSMPTEST 2022-6SMPTEST 2022-2SMPTEST 2110-31SMPTE ST 299 SMPTEST 302 AES67/ST 2110-30PMD or other"Non-PCM" DataPCM AudioAES3SMPTEST 337SMPTEST 2109Serial ADMSMPTEST 337SMPTEST 2116Audio Metadata © 2016-17 DOLBY LABORATORIES, INC.IP Audio with Native Metadata (as proposed)10SMPTEST 2109Audio MetadataAES67/ST 2110-30PCM AudioIETFRFC 6597PCM Audio over IPAudio Metadataover IPIn developmentST 2110-xx (?)•Map ST 2109 containers directlyto RTP•Free of legacy constraints with AES3/ST337 tunneling•Full metadata support for NGA and advanced audio applicationsAES-X242Container(no ST 337) © 2016-17 DOLBY LABORATORIES, INC.Project AES-X242, Streaming audio metadata over IP networkshttp://www.aes.org/standards/meetings/init-projects/aes-x242-init.cfm•To define a standardized method for transporting metadata associated with audio in an AES67 stream. The audio metadata shall be transported in a separate stream that is sent in parallel to AES67 streams rather than part of the AES67 stream. The standard shall define synchronization between the audio metadata transport and the associated AES67 transport. The transmission method shall be low latency and have a level of network performance equivalent to AES67. Within the scope is formatting of the streaming audio metadata for transport. Suggested is an open standards based framework that supports both static and dynamic, time synchronous metadata that is optimized for live workflow applications. The standard shall consider all use cases for metadata associated with AES67,support existing AES audio metadata standards, and be extensible for future metadata requirements. The standard will consider binding between the audio metadata transport and the associated AES67 transport.Audio metadata for AES67•Separate RTP stream for metadata associated with PCM essence in AES67•Intended to support broader AES67 user base, not just broadcast applications•Format agnostic, open to any kind of audio metadataWork started in AES SC-02-12-R in 2017•Currently in draft state•Publication targeted for 202011 © 2016-17 DOLBY LABORATORIES, INC.SMPTE ST 2110-40 –Ancillary Data over IP12 Ancillary Data Flag (ADF) 10 bits Data ID (DID) Secondary Data ID (SDID) Data Count (DC) Checksum 8 bits ~Parity Parity User Data Words (UDW) 255 max SMPTE ST 291 -1 ANC Data Packet References IETF RFC 8331 RTP Payload for SMPTE ST 291-1 Ancillary DataSpecific for transport of ST 291-1 ANC data packetsOpen to arbitrary data but subject to restrictions of ANC packet structure © 2016-17 DOLBY LABORATORIES, INC.32NF-60 Studio Video over IP (SVIP): ST 2110-41 "Fast Metadata"https://kws.smpte.org/higherlogic/ws/public/projects/558/details•Problem to be solved:: In the ST 2110 suite, there is provision for carriage of legacy ST 291 ANC metadata. This legacy SDI-centric method is described in ST 2110-40 and provides an excellent bridge to IP for SDI services. In the IP-centric future, there will undoubtedly be new metadata requirements which will have no relevance or application in the SDI domain and additionally may not be appropriate for efficiently wrapping in legacy ST 291 SDI ANC packets for IP transport with 2110-40. This project will create a new transport method for arbitrary metadata using RTP packetsin the same manner as the other 2110 standards. This method will not be strictly tied to any type or form of metadata. Use of the 2110 synchronization mechanism will permit transmission in association with a 2110 essence stream…•Similar application in legacy vs. IP metadata transportTra n s m i s s i o n o f a r b i t ra r y m e t a d a t a w i t h l i m i t a t i o n s o f A N C p a c ke t s t r u c t u r eShares many common requirements with AES-X242 projectAES-X242 limited to audio metadataST 2110-41 open to "any" metadata13 © 2016-17 DOLBY LABORATORIES, INC.IP Audio with Native Metadata (revised)14Audio MetadataAES67/ST 2110-30PCM AudioSMPTE ST 2110-41PCM Audio over IPAudio Metadataover IPIn developmentST 2110-xx (proposed)AES-X242•AES/SMPTE liaison to investigate "harmonization" of AES-X242 and ST 2110-41•i.e. share common RTP streaming format•Still free of legacy constraints with AES3/ST337 tunneling•Both standards under active development –participation encouraged! © 2016-17 DOLBY LABORATORIES, INC.AES-X242 Use Cases and RequirementsLive Broadcast•RDD 49 (PMD) and Serial ADMMusic and Film Production•MIDI, OSC-like protocolsLive SoundRadioAudio StreamingGeneral Transport…15 © 2016-17 DOLBY LABORATORIES, INC.AES SC-02-12-R –Task group on Streaming audio metadata over IPTa s k g r o u p t o d e v e l o p A E S-X242 standardSC-02-12-R seeking user input on AES-X242 use cases and requirementsAES-X242 development and interop testing neededInput and participation encouraged!See RAVENNA/Dolby demo of AES-X242 in LawoBooth16 © 2016-17 DOLBY LABORATORIES, INC.Standards relevant to Audio Metadata over IPSMPTE•ST 2109 –Audio Metadata in AES3•RDD 49 –Professional Metadata (PMD)•ST 2116 –Serial ADM in AES3 (Draft Publication)•ST 2110-30 –PCM Audio over IP•ST 2110-31 –AES3 over IP•ST 2110-41 –Fast Metadata (in development)AES•AES67 –Audio over IP Interoperability•AES-X242 –Audio Metadata over IP (in development)ITU•ITU-R BS.2125-0 –Serial ADM (Audio Definition Model)17 Kent Terrykbt@dolby.comThank you