SMPTE ST 2110 Using JPEG-XS

Presenters : Jean-Baptiste Lorent - intoPix

Year : 2019

dlp_document_download : AIMS IP Showcase IBC 2019 September 2019 Curated by Video Services Forum vsf.tv 1 I P S H O W C A S E T H E AT R E AT I B C 2 0 1 9 : 1 3 –1 7 S E P T 2 0 1 9 C U R A T E D B Y JPEG -XS and ST 2110 Jean -Baptiste Lorent intoPIX Copyright intoPIX 2019: This presentation contains information prepared for IP Showcase Theater and may be reused if credit is given to the IP Showcase and intoPIX SA I P S H O W C A S E T H E AT R E AT I B C 2 0 1 9 : 1 3 –1 7 S E P T 2 0 1 9 ST 2110 IS TAKING OFF It is designed to become the infrastructure of choice 2 IP S H O W C A S E T H E AT R E AT I B C 2 0 1 9 : 1 3 –1 7 S E P T 2 0 1 9 AIMS IP Showcase IBC 2019 September 2019 Curated by Video Services Forum vsf.tv 2 I P S H O W C A S E T H E AT R E AT I B C 2 0 1 9 : 1 3 –1 7 S E P T 2 0 1 9 GOING IP BRINGS MANY BENEFITS  re-routing  easy configuration  less space  simplified workflows  smaller buildings & OB REDUCING COMPLEXITY INCREASING AGILITY LOWERING COSTS  less cables  bi-directional  less cables  bi-directional  COTS equipment I P S H O W C A S E T H E AT R E AT I B C 2 0 1 9 : 1 3 –1 7 S E P T 2 0 1 9 GOING IP BRINGS MANY BENEFITS  re-routing  easy configuration  less space  simplified workflows  smaller buildings & OB REDUCING COMPLEXITY INCREASING AGILITY  less cables  bi-directional  less cables  bi-directional  COTS equipment LOWERING COSTS AIMS IP Showcase IBC 2019 September 2019 Curated by Video Services Forum vsf.tv 3 I P S H O W C A S E T H E AT R E AT I B C 2 0 1 9 : 1 3 –1 7 S E P T 2 0 1 9 5 "Can we put more cars on the road without creating traffic jams & delaying arrival times of each passenger?" IP S H O W C A S E T H E AT R E AT I B C 2 0 1 9 : 1 3 –1 7 S E P T 2 0 1 9 … but the roads are jammed already! WE HAVE MORE PIXELS TO MANAGE, STORE AND TRANSPORT I P S H O W C A S E T H E AT R E AT I B C 2 0 1 9 : 1 3 –1 7 S E P T 2 0 1 9 $ HD NEEDS 10G B E INFRASTRUCTURES 6 IP CAMERAS IP REPLAY & STORAGE IP PRODUCTION SWITCHER IP MONITORING 10GbE COTS IP SWITCH 10GbE INFRASTRUCTURE HD Uncompressed = 2.4 Gbps for HD60 Intra -frame compression is needed for the storage to ease read & write access to the disks & to reduce storage cost of all incoming streams Many solutions exist for downscaling great amounts of streams in low resolution COTS in this case means at least 10GbE ports for all devices & switches (At least Cat 6 cables ) AIMS IP Showcase IBC 2019 September 2019 Curated by Video Services Forum vsf.tv 4 I P S H O W C A S E T H E AT R E AT I B C 2 0 1 9 : 1 3 –1 7 S E P T 2 0 1 9 $ 4K NEEDS 25G B E INFRASTRUCTURES 7 IP CAMERAS IP REPLAY & STORAGE IP PRODUCTION SWITCHER IP MONITORING 25 GbE COTS IP SWITCH 4K 25GbE INFRASTRUCTURE Uncompressed = 9.6 Gbps for 4K60 Intra -frame compression is needed for the storage to ease read & write access to the disks & to reduce storage cost of all incoming streams New scaling capabilities needed for monitoring COTS in this case means at least 25GbE ports for all devices & switches I P S H O W C A S E T H E AT R E AT I B C 2 0 1 9 : 1 3 –1 7 S E P T 2 0 1 9 $ 8K NEEDS 100G B E INFRASTRUCTURES 8 IP CAMERAS IP REPLAY & STORAGE IP PRODUCTION SWITCHER IP MONITORING 100/400GbE COTS IP SWITCH 8K 100GbE INFRASTRUCTURE Uncompressed: 38.4 Gbps for 8K60, 76.8 Gbps for 8K120 Intra -frame compression is needed for the storage to ease read & write access to the disks & to reduce storage cost of all incoming streams New scaling capabilities needed for monitoring (even more steps down) COTS in this case means at least 100/400GbE ports for all devices & switches AIMS IP Showcase IBC 2019 September 2019 Curated by Video Services Forum vsf.tv 5 I P S H O W C A S E T H E AT R E AT I B C 2 0 1 9 : 1 3 –1 7 S E P T 2 0 1 9 9 … easily managing more pixels over a limited bandwidth, safeguarding low latency and a pixel perfect quality? IP S H O W C A S E T H E AT R E AT I B C 2 0 1 9 : 1 3 –1 7 S E P T 2 0 1 9 WHAT IF A TECHNOLOGY COULD HELP? I P S H O W C A S E T H E AT R E AT I B C 2 0 1 9 : 1 3 –1 7 S E P T 2 0 1 9 CALL FOR A NEW STANDARD ISO -standardization of JPEG XS 10 Save cost & power Preserve quality with no latency Simplify ST2110 connectivity Manage more pixels AIMS IP Showcase IBC 2019 September 2019 Curated by Video Services Forum vsf.tv 6 I P S H O W C A S E T H E AT R E AT I B C 2 0 1 9 : 1 3 –1 7 S E P T 2 0 1 9 11 2016 2017 2018 2019 Call for proposal A new low -latency ligthweight image coding system Liaison with AIMS, SMPTE and VSF TICO selected as baseline amongst 6 international proposals. Collaborative work . The standard moves to voting and publication phases JPEG -XS goes Life ! First products adopted /deplyed at IBC 2019 CALL FOR A NEW STANDARD ISO -standardization of JPEG XS XS= Xtra Small Xtra Speed I P S H O W C A S E T H E AT R E AT I B C 2 0 1 9 : 1 3 –1 7 S E P T 2 0 1 9 WHERE can JPEG XS be implemented? 12 AIMS IP Showcase IBC 2019 September 2019 Curated by Video Services Forum vsf.tv 7 I P S H O W C A S E T H E AT R E AT I B C 2 0 1 9 : 1 3 –1 7 S E P T 2 0 1 9 EVERYWHERE uncompressed is currently used 13 I P S H O W C A S E T H E AT R E AT I B C 2 0 1 9 : 1 3 –1 7 S E P T 2 0 1 9 14 Where can JPEG XS be implemented? In any application for which pixel perfect quality, minimal latency, low complexity, and efficient video bandwidth are crucial! AIMS IP Showcase IBC 2019 September 2019 Curated by Video Services Forum vsf.tv 8 I P S H O W C A S E T H E AT R E AT I B C 2 0 1 9 : 1 3 –1 7 S E P T 2 0 1 9 A DIFFERENT APPROACH TO CREATING A CODEC Combining the best speed, complexity and quality in one codec 15 Low Latency High Compression Ratio Low Complexity All Platforms (SW/HW) High Quality JPEG XS Intra-frame codec Distribution Codec I P S H O W C A S E T H E AT R E AT I B C 2 0 1 9 : 1 3 –1 7 S E P T 2 0 1 9 JPEG XS’ BENEFITS The most advanced lightweight codec 16 VISUALLY LOSSLESS OPEN SPECIFICATIONS & INTEROPERABILITY NO LATENCY CONSTANT QUALITY SCALABILITY MULTI - PLATFORM LOW COMPLEXITY AIMS IP Showcase IBC 2019 September 2019 Curated by Video Services Forum vsf.tv 9 I P S H O W C A S E T H E AT R E AT I B C 2 0 1 9 : 1 3 –1 7 S E P T 2 0 1 9 SUBJECTIVE AND OBJECTIVE QUALITY EVALUATION Using CGI, desktop and natural content 17 I P S H O W C A S E T H E AT R E AT I B C 2 0 1 9 : 1 3 –1 7 S E P T 2 0 1 9 RIGOROUS ISO QUALITY ASSESSMENTS 18 New ISO/IEC 29170 -2 method for near -lossless quality assessment on both natural & synthetic images) Full transparency to uncompressed down to 3bpp (10:1) Visually lossless down to 1.5bpp (20:1) on film/TV content Smooth degradation down to 0.5bpp (ringing artefacts/ no blocking artefacts!) "FLICKER TEST" Uncompressed Interleaving original and compressed every 1/8 sec (after 7 th encode/decode) Test on 360 scores (= persons) in total (from 4 universities/research centers) AIMS IP Showcase IBC 2019 September 2019 Curated by Video Services Forum vsf.tv 10 I P S H O W C A S E T H E AT R E AT I B C 2 0 1 9 : 1 3 –1 7 S E P T 2 0 1 9 HIGH QUALITY IN MICROSECONDS 19 Natural image with equally distributed details vertically on JPEG -XS I P S H O W C A S E T H E AT R E AT I B C 2 0 1 9 : 1 3 –1 7 S E P T 2 0 1 9 HIGH QUALITY IN MICROSECONDS 20 Down/Upsampling to 422 with minor effect In this image Above 40dB typically visual lossless PSNR (dB) bpp compression ratio 35 40 45 50 55 60 0 1 2 3 4 5 6 7 8 Content: Alexa Drums 444 / 8 Bit JPEG XS (WD)VC2ProResJPEG2000 6:1 8:1 12:1 24:1 ProRes 4444 ProRes 422 HQ ProRes 422 ProRes 422 LT ProRes 422 Proxy 4:1 Natural image with equally distributed details vertically on JPEG -XS AIMS IP Showcase IBC 2019 September 2019 Curated by Video Services Forum vsf.tv 11 I P S H O W C A S E T H E AT R E AT I B C 2 0 1 9 : 1 3 –1 7 S E P T 2 0 1 9 HIGH QUALITY IN MICROSECONDS 21 Mixed desktop content with natural images, CGI content, and text on JPEG -XS I P S H O W C A S E T H E AT R E AT I B C 2 0 1 9 : 1 3 –1 7 S E P T 2 0 1 9 HIGH QUALITY IN MICROSECONDS 22 25 30 35 40 45 50 55 60 65 0 1 2 3 4 5 6 7 8 Richter ScreenContent 444 / 8Bit JPEG XS (WD) VC2 LD ProRes JPEG2000 4:1 6:1 8:1 12:1 ProRes 422 LT ProRes 422 ProRes 422 HQ ProRes 4444 422 Downsampling reduces quality 444 recommended for text and mixed content PSNR (dB) bpp compression ratio Mixed desktop content with natural images, CGI content, and text on JPEG -XS AIMS IP Showcase IBC 2019 September 2019 Curated by Video Services Forum vsf.tv 12 I P S H O W C A S E T H E AT R E AT I B C 2 0 1 9 : 1 3 –1 7 S E P T 2 0 1 9 BEST QUALITY IN SINGLE - AND MULTI -GENERATION 23 SINGLE GENERATION EXPERIMENT ENCODER DECODER ORIGINAL FRAME DECODED FRAME PSNR PSNR 1 PSNR 2 ENCODER DECODER ORIGINAL FRAME DECODED FRAME 1 ENCODER DECODER DECODED FRAME 2 MULTI GENERATION EXPERIMENT Performed for 10 generations I P S H O W C A S E T H E AT R E AT I B C 2 0 1 9 : 1 3 –1 7 S E P T 2 0 1 9 BEST QUALITY IN MULTI -GENERATION 24 Notes:  JPEG is 8bit only  J2K is 3 to 5x more complex and needs external memory AIMS IP Showcase IBC 2019 September 2019 Curated by Video Services Forum vsf.tv 13 I P S H O W C A S E T H E AT R E AT I B C 2 0 1 9 : 1 3 –1 7 S E P T 2 0 1 9 MINIMAL COMPLEXITY ON ALL PLATFORMS •Multiple degrees of parallelism for optimed CPU, GPU, FPGA and ASIC implementations. •Multiple profiles for low power, Low logic ‒no external memory in hardware (FPGA, ASIC) ‒The smallest codec for FPGA at this efficiency •Optimal syntax for software and speed optimizations (CPU, GPU) ‒5x faster or more than JPEG2000 ISO standard in CPU, GPU •Best trade -off between cost, power consumption and quality 25 …leading to maximum efficiency I P S H O W C A S E T H E AT R E AT I B C 2 0 1 9 : 1 3 –1 7 S E P T 2 0 1 9 MINIMAL LATENCY •Down to a few microseconds (down to 1/10 of a millisecond) = only a few video lines. •Maximum responsiveness (few µs) - perfect for any latency critical applications •CBR (constant bitrate) for reliable video over IP transport. 26 Humans are able to detect a latency only above 13 milliseconds. Massachusetts Institute of Technology (MIT) " " AIMS IP Showcase IBC 2019 September 2019 Curated by Video Services Forum vsf.tv 14 I P S H O W C A S E T H E AT R E AT I B C 2 0 1 9 : 1 3 –1 7 S E P T 2 0 1 9 MAXIMUM FLEXIBILITY 27 Future proof thanks to support of extensive number of video formats Multiple resolutions Multiple chroma formats Multiple color formats Multiple bit depths HDR support ‒ HD, 4K, 8K… up to 16Kx16K ‒ 4:4:4, 4:2:2, 4:2:0, grayscale ‒ RGB, YUV, … ‒ From 8, 10, 12, 14 to 16bit I P S H O W C A S E T H E AT R E AT I B C 2 0 1 9 : 1 3 –1 7 S E P T 2 0 1 9 MAXIMUM FLEXIBILITY 28 •HD/4K/8K downscaler within workflows (i.e. for monitoring purpose) •Lower CPU/GPU decoding requirements (less consumption to decode HD than 4K & 8K ) •Partial extraction for faster analytics and detection Built -in 1 - to 2 -level downscaler and partial extraction HD 4K 8K 8K codestream HD 4K Original 8K Partial HD Partial 4K Partial 8K HD 4K 8K AIMS IP Showcase IBC 2019 September 2019 Curated by Video Services Forum vsf.tv 15 I P S H O W C A S E T H E AT R E AT I B C 2 0 1 9 : 1 3 –1 7 S E P T 2 0 1 9 JPEG XS IS AN ISO STANDARD ITEM DESCRIPTION STATUS TARGET DATE ISO/IEC 21122 -1 Part 1: Core coding system Published n.a . ISO/IEC 21122 -2 Part 2: Profiles and buffer models Published n.a . ISO/IEC 21122 -3 Part 3: Transport and container formats Published n.a ISO/IEC 21122 -4 Part 4: Conformance testing Under last ballot - DIS Q3 2019 ISO/IEC 21122 -5 Part 5: Reference software First ballot - CD Q4 2019 ISO/IEC 21122 -1 AMD1 Amd 1 : extended capabilities Working Draft Q2 2020 ISO/IEC 13818 -1 AMD1 Carriage of JPEG XS over MPEG -2 TS Under last ballot ( Draft AMD - DAM) Q4 2019 IETF RFC JPEG XS RTP JPEG XS RTP payload (https://datatracker.ietf.org/doc/draft-ietf -payload -rtp-jpegxs/ ) Current version formally adopted by IETF payload WG Q4 2019 SMPTE ST 2110 -22 Compressed essence in ST2110 Published n.a . 29 Open specifications ensuring interoperability I P S H O W C A S E T H E AT R E AT I B C 2 0 1 9 : 1 3 –1 7 S E P T 2 0 1 9 JPEG -XS, WITH ST 2110 Part -22 ( Compressed video essence) 30 AIMS IP Showcase IBC 2019 September 2019 Curated by Video Services Forum vsf.tv 16 I P S H O W C A S E T H E AT R E AT I B C 2 0 1 9 : 1 3 –1 7 S E P T 2 0 1 9 JPEG XS  ST 2110 BANDWIDTH -EFFICIENT WORKFLOW FORMATS JPEG XS BITRATES IP NETWORKS & SDI MAPPING INFRASTRUCTURE HD 720p60 HD 1080i60 70 Mbps - 200 Mbps 1 to x streams over 1GbE CAT5e HD 1080p60 150 Mbps - 400 Mbps 1 to x streams over 1GbE CAT5e 4K 2160p60 500 Mbps - 1,6 Gbps 1 stream over 1GbE 1 to x streams over 10GbE Down to a single SDI cable CAT5e CAT6 HD/3G -SDI 8K 4320p60 2 Gbps - 6,4 Gbps 1 stream over 2.5GbE up to 4 streams over 10GbE Down to a single SDI cable CAT5e CAT6 3G/6G/12G -SDI 8K 4320p120 4 Gbps - 12,8 Gbps 2 streams over 10GbE Down to a single SDI cable CAT6 6G/12G -SDI 31 I P S H O W C A S E T H E AT R E AT I B C 2 0 1 9 : 1 3 –1 7 S E P T 2 0 1 9 JPEG XS  COST -EFFECTIVE ST2110 INFRASTRUCTURES 32 IP CAMERAS IP REPLAY & STORAGE IP PRODUCTION SWITCHER IP MONITORING 10GbE COTS IP SWITCH 10 GbE INFRASTRUCTURE 8K up to Full access to 8K proxies directly from the coded streams No scaling required, even from a simple laptop COTS in this case means max. 10GbE ports for all devices & switches AIMS IP Showcase IBC 2019 September 2019 Curated by Video Services Forum vsf.tv 17 I P S H O W C A S E T H E AT R E AT I B C 2 0 1 9 : 1 3 –1 7 S E P T 2 0 1 9 = 33  XS meets all the ST2110 quality requirements  CBR, latency, quality, complexity, …  All existing advantages of moving to IP are kept  flexibility, scalability, unlimited accessibility  XS bandwidth -reduction enables to achieve more with ST2110  Higher pixel rates, more streams, cheaper cables (CAT5e, 3G -SDI) and interfaces (<1Gpbs, <10Gbps), reduced costs, reduced storage, reduced IP packets, …  Upgrade capability  Ease the remote production and cloud migration  Fully standardized + cost -effective, bandwidth -efficient, and high quality IP production workflows I P S H O W C A S E T H E AT R E AT I B C 2 0 1 9 : 1 3 –1 7 S E P T 2 0 1 9 C U R A T E D B Y Thank You Live CPU, GPU & FPGA demos at intoPIX booth (Hall 10 D31) in HD, 4K & 8K jb.lorent@intopix.com Thank you to our Media Partners AIMS IP Showcase IBC 2019 September 2019 Curated by Video Services Forum vsf.tv 18 I P S H O W C A S E T H E AT R E AT I B C 2 0 1 9 : 1 3 –1 7 S E P T 2 0 1 9 About us, intoPIX •Founded in 2006, HQ in Belgium. •Technology provider of innovative compression technologies empowering visual communications. •Member of AIMS, VSF, SMPTE and JPEG committees. •Deliver unique FPGA/ASIC IP cores and fast CPU/GPU SDKs to manage more pixels, preserve quality with no latency, save cost & power and simplify connectivity. •Track record in terms of success stories, innovation achievements and effectiveness in enhancing Broadcast applications. ‒EMMY for technology & engineering on JPEG2000 VSF TR01 , Invention & Standardization of TICO at SMPTE RDD35 and at JPEG as JPEG -XS, IABM Game Changer, IABM Peter Wayne Award for Innovation, EY Belgian Most promising growing company finalist, Delloite Fast50,…Serving 100+ customers worldwide •More info on : www.intoPIX.com