IP Workflow with ST 2110 and VC2 Compression

Presenters : Tae-Han Kim - IML

Year : 2019

dlp_document_download : AIMS IP Showcase IBC 2019 September 2019 Curated by Video Services Forum vsf.tv 1 I P S H O W C A S E T H E AT R E AT I B C 2 0 1 9 : 1 3 –1 7 S E P T 2 0 1 9 C U R A T E D B Y IP Workflow Development with ST2110 and VC2 Compression Tae -Han Kim CEO @ IML Co. Ltd. I P S H O W C A S E T H E AT R E AT I B C 2 0 1 9 : 1 3 –1 7 S E P T 2 0 1 9 IP Workflow Testbed Design with ST2110 & VC2 1. Introduction to the Testbed 2. SDI -ST2110 Converter 3. VC2 Codec for IP Workflow 4. Audio over IP ( treated as Ancillary data) 5. Discussion 2 AIMS IP Showcase IBC 2019 September 2019 Curated by Video Services Forum vsf.tv 2 I P S H O W C A S E T H E AT R E AT I B C 2 0 1 9 : 1 3 –1 7 S E P T 2 0 1 9 •Where is the Testbed studio ? ‒KBS Laboratory Building, IP Workflow Testbed (Panoramic view) ‒Miniature Studio 3 ST2110 & VC2 : Introduction I P S H O W C A S E T H E AT R E AT I B C 2 0 1 9 : 1 3 –1 7 S E P T 2 0 1 9 ST2110 & VC2 : Introduction •Who built the workflow ? (Consortium Organization) ‒KBS (Korean Broadcasting System) •Architecture Developer, Testbed holder •ST2110 -20/30 supporting media server developer ‒Lumantek •ST2110 -20 supporting video switcher developer •Web -based workflow manager developer 4 AIMS IP Showcase IBC 2019 September 2019 Curated by Video Services Forum vsf.tv 3 I P S H O W C A S E T H E AT R E AT I B C 2 0 1 9 : 1 3 –1 7 S E P T 2 0 1 9 •Who built the workflow ? (Consortium Organization) ‒Atto -research •SDN OpenFlow switch & network structure developer ‒IML •ST2110 -22 VC -2(ST2042/2047) compression supporting SDI -IP converter developer •Stand -alone workflow manager developer 5 ST2110 & VC2 : Introduction I P S H O W C A S E T H E AT R E AT I B C 2 0 1 9 : 1 3 –1 7 S E P T 2 0 1 9 •Why IP Workflow ? ‒Study New Technology •KBS is already operating an IP Studio with Grass Valley -Cisco solution •Test ST2110 interoperability with legacy UHD equipment ‒IP-enabled Video Switcher Development •Lumantek extends product line with IP capability •ST2110 -20 supporting modular video switcher 6 ST2110 & VC2 : Introduction AIMS IP Showcase IBC 2019 September 2019 Curated by Video Services Forum vsf.tv 4 I P S H O W C A S E T H E AT R E AT I B C 2 0 1 9 : 1 3 –1 7 S E P T 2 0 1 9 •Why IP Workflow ? ‒UHD Video Compression for 10G Network •ST2042/2047 VC -2, the only open & released low latency codec •Long -distance source with limited communication bandwidth ‒Network Infrastructure •Broadcasting Company -wide IP network connectivity •All hardware & media assets are managed with IP connection 7 ST2110 & VC2 : Introduction I P S H O W C A S E T H E AT R E AT I B C 2 0 1 9 : 1 3 –1 7 S E P T 2 0 1 9 •How to build ? ‒Use new IP standard •SMPTE ST2110 for media transport •AMWA NMOS for workflow management ‒Target Media •Video : UHD (3,840x2,160) @59.94p •Audio : only embedded 16ch audio is supported, yet to be extended to ST2110 -30 8 ST2110 & VC2 : Introduction AIMS IP Showcase IBC 2019 September 2019 Curated by Video Services Forum vsf.tv 5 I P S H O W C A S E T H E AT R E AT I B C 2 0 1 9 : 1 3 –1 7 S E P T 2 0 1 9 •How to build ? ‒Network •1G/ 10G /25G/40G/100G ethernet switches mixed in spine -leaf design •Non -proprietary SDN(OpenFlow) with legacy IGMP support ‒Video Coding •ST2110 -20 uncompressed and ST2110 -22 compressed with VC2 co -existing design •draft -ietf -payload -rtp -vc2hq -03 for VC2 transport 9 ST2110 & VC2 : Introduction I P S H O W C A S E T H E AT R E AT I B C 2 0 1 9 : 1 3 –1 7 S E P T 2 0 1 9 •How to build ? (Components) 10 Camera Video Server CG Server PTP Generator Video Switcher Multi -viewer Monitoring Control System Spine Leaf Network Control Miniature Studio ST2110 & VC2 : Introduction AIMS IP Showcase IBC 2019 September 2019 Curated by Video Services Forum vsf.tv 6 I P S H O W C A S E T H E AT R E AT I B C 2 0 1 9 : 1 3 –1 7 S E P T 2 0 1 9 •How to build ? (Block design; Simplified) 11 1G/10G/ 40G/100G Spine -Leaf Switches Workflow Control & Management SDN Controller PTP Grand Master Display ST2110 -20 Converter Video Server w/ ST2110 Card UHD Video Switcher ST2110 -20 &SDI Camera ST2110 -20 Converter Display ST2110 -22 Converter Video Recorder ST2110 -22 Converter SDI SDI 25GE PC TPG w/ 25GE NIC 25GE 10GE 40GE 40GE 40GE 10GE 40GE SDI SDI 1GE 1GE 1GE SDI ST2110 & VC2 : Introduction I P S H O W C A S E T H E AT R E AT I B C 2 0 1 9 : 1 3 –1 7 S E P T 2 0 1 9 ST2110 & VC2 : SDI -IP Converter •What is developed ? ‒VC2 Encoder 12 10GE MAC SFP+ PHY 10G Ethernet Ethernet Framer Softcore NMOS 2110 -40 Packetizer 2110 -22 VC2 CODEC Video Streamer Audio Streamer SDI Interface UHD Video w/ Audio Signal Demuxer Packet Muxer AIMS IP Showcase IBC 2019 September 2019 Curated by Video Services Forum vsf.tv 7 I P S H O W C A S E T H E AT R E AT I B C 2 0 1 9 : 1 3 –1 7 S E P T 2 0 1 9 •What is developed ? ‒VC2 Decoder 13 10GE MAC SFP+ PHY 10G Ethernet Ethernet Filter Softcore NMOS 2110 -30 Packetizer 2110 -22 VC2 DEC Video Streamer Audio Streamer SDI Interface SDI Generator Packet De- Mux UHD Video w/ Audio Timing Synchronizer ST2110 & VC2 : SDI -IP Converter I P S H O W C A S E T H E AT R E AT I B C 2 0 1 9 : 1 3 –1 7 S E P T 2 0 1 9 •What is developed ? ‒NMOS Manager •IS-04, IS -05 support •Built -in registration server •Connection control user interface 14 Icon View Matrix View Device View ST2110 & VC2 : SDI -IP Converter AIMS IP Showcase IBC 2019 September 2019 Curated by Video Services Forum vsf.tv 8 I P S H O W C A S E T H E AT R E AT I B C 2 0 1 9 : 1 3 –1 7 S E P T 2 0 1 9 •VC2 Video Compression ‒Input SDI •12G -SDI 3,840x2,160 UHD 60p,59.94p, 50p ‒Wavelet Slice & Filter •SMPTE ST2042 -1-2017 slice 64x2 Luma Wavelet Slice •Legal53 filter ( Haar0, Haar1 ) ‒Compression Ratio •4:1~10:1 Run -time Programmable, •slice to slice exact CBR for even video quality over a frame ‒Latency •200us for encoding & decoding & Buffering @( Legal53, 64x2 slice, slice CBR) ‒Implementation •IML’s own FPGA IP, https://www.xilinx.com/products/intellectual -property/1 -puxmo7.html 15 ST2110 & VC2 : VC2 Codec I P S H O W C A S E T H E AT R E AT I B C 2 0 1 9 : 1 3 –1 7 S E P T 2 0 1 9 •VC2 Codec Structure ‒Encoder ‒Decoder 16 DWT Slice Indexing Quantizer Pipe Quantizer Pipe Quantizer Pipe Quantizer PIPE Entropy coding Quantizer Pipe Quantizer Pipe Quantizer Pipe RTP Packetizer Quantizer PIPE Entropy coding Slice Formatter Indexing Formatter Slice Parser De- Packetizer Decoder De - quantizer Decoder De - quantizer IDWT Framer ST2110 & VC2 : VC2 Codec AIMS IP Showcase IBC 2019 September 2019 Curated by Video Services Forum vsf.tv 9 I P S H O W C A S E T H E AT R E AT I B C 2 0 1 9 : 1 3 –1 7 S E P T 2 0 1 9 •Transmission of VC2 -compressed Video ‒RTP packet structure •IETF draft -ietf -payload -rtp -vc2hq -03 is followed •Fixed number of compressed slices are packed in the payload  RTP packet size varies •VC2 frame header packet is generated at each start of a frame  Compression ratio, filter type can be dynamically changed 17 ST2110 & VC2 : VC2 Codec I P S H O W C A S E T H E AT R E AT I B C 2 0 1 9 : 1 3 –1 7 S E P T 2 0 1 9 •Transmission of VC2 -compressed Video ‒RTP packet sequence 18 1st Packet Parse Information 2nd Packet Sequence Header 3rd ~ Packet Encoded Active Picture Last Packet EOF RTP MARK Single Frame ST2110 & VC2 : VC2 Codec AIMS IP Showcase IBC 2019 September 2019 Curated by Video Services Forum vsf.tv 10 I P S H O W C A S E T H E AT R E AT I B C 2 0 1 9 : 1 3 –1 7 S E P T 2 0 1 9 •Transmission of VC2 -compressed Video ‒Time Synchronization •No PTP timing information is used in encoder’s RTP frame generation •Minimal latency from pixel input to RTP packet  No frame timing sync for RTP stream •Decoder generates video timing with input RTP packet data speed ‒Identification of Stream Type •SDP file "a=rtpmap:99 VC2/90000" for NMOS connecting managements •Detailed parameters in the first packet of a frame 19 ST2110 & VC2 : VC2 Codec I P S H O W C A S E T H E AT R E AT I B C 2 0 1 9 : 1 3 –1 7 S E P T 2 0 1 9 •Transmission of VC2 -compressed Video ‒Phase Synchronized Transmission  Multi -frame delay ‒Minimum Latency Transmission 20 ST2110 & VC2 : VC2 Codec Converter buffer IP Node buffer IP Node buffer Converter buffer N+1 th Frame Tick Nth Frame Tick N+2 th Frame Tick N+3 th Frame Tick N+4 th Frame Tick Converter buffer IP Node buffer IP Node buffer Converter buffer Nth Frame Tick Nth Frame Tick + 60 line delay AIMS IP Showcase IBC 2019 September 2019 Curated by Video Services Forum vsf.tv 11 I P S H O W C A S E T H E AT R E AT I B C 2 0 1 9 : 1 3 –1 7 S E P T 2 0 1 9 •Audio Over IP ‒Input SDI •12G -SDI 16ch PCM audio ‒Transport •SMPTE ST2110 -40 (NOT 30) •SMPTE ST272M, ST2082 -10 Audio packet (16ch, 48kHz) treated as ANC data ‒Time Sync •The audio packet in SDI is regenerated in the original position in the SDI frame 21 ST2110 & VC2 : Audio as ANC data I P S H O W C A S E T H E AT R E AT I B C 2 0 1 9 : 1 3 –1 7 S E P T 2 0 1 9 •Audio Over IP ‒Stream Management •Automatic port mapping with the video NMOS connection •Independent audio stream can also be generated ‒Usage •SDI audio embedded/de -embedded can be used 22 ST2110 & VC2 : Audio as ANC data AIMS IP Showcase IBC 2019 September 2019 Curated by Video Services Forum vsf.tv 12 I P S H O W C A S E T H E AT R E AT I B C 2 0 1 9 : 1 3 –1 7 S E P T 2 0 1 9 •Audio Over IP ‒Audio packets are regenerated in the original position •Possible from VC2 Encoding/Decoding delay 23 Active Video VC2 Encoder Audio Packet ST2110 -40 Packetizer Source SDI frame Active Video ST2110 -40 De-packetizer VC2 Decoder Audio Packet Positioning Reconstructed SDI frame Ethernet Ethernet ST2110 & VC2 : Audio as ANC data I P S H O W C A S E T H E AT R E AT I B C 2 0 1 9 : 1 3 –1 7 S E P T 2 0 1 9 •What we got from the testbed ‒Interoperability of legacy(12G -SDI) equipment & ST2110 deviceV •Working fine •Unified management is required •IP-converter should "NMOS" for legacy equipment •Ethernet based configuration & management of legacy equipment may be used for generating NMOS data 24 ST2110 & VC2 : Discussion AIMS IP Showcase IBC 2019 September 2019 Curated by Video Services Forum vsf.tv 13 I P S H O W C A S E T H E AT R E AT I B C 2 0 1 9 : 1 3 –1 7 S E P T 2 0 1 9 •What we got from the testbed ‒Timing Synchronization •All devices may not require PTP connection •Time synchronization is really done in contribution/distribution encoder •Frequency synchronization is required in video/audio creator •Video mixers should have frame buffer for mixed ST2110 WIDE, Gapped/Linear devices  Phase synchronization •The other devices can follow the source timing with minimum latency 25 ST2110 & VC2 : Discussion I P S H O W C A S E T H E AT R E AT I B C 2 0 1 9 : 1 3 –1 7 S E P T 2 0 1 9 •Future Works ‒Application of the Latest & New AMWA Standards •IS-04/05 update •IS-06/07/08/09/10 … •Full -stack by VSF ‒VC2 Codec + •Add envelop coding in wavelet coefficients  better PSNR •Better quantization factor guess algorithm for reduced logic size ‒Workflow Manager •Open source project for easily integrated IP workflow 26 ST2110 & VC2 : Discussion AIMS IP Showcase IBC 2019 September 2019 Curated by Video Services Forum vsf.tv 14 I P S H O W C A S E T H E AT R E AT I B C 2 0 1 9 : 1 3 –1 7 S E P T 2 0 1 9 Thank you Thank you to our Media Partners Tae -Han Kim, IML Co. Ltd. ceo@iml.co.kr +82 -2-830 -5173 27 For Further Discussion, Visit IML@2.A31