JPEG-XS and ST 2110

Presenters : Jean-Baptiste Lorent - intoPix

Year : 2019

dlp_document_download : C U R A T E D B Y I P S H O W C A S E T H E AT E R AT N A B – A P R I L 8 -11 , 2 01 9 JPEG - XS and ST 2110 Jean -Baptiste Lorent intoPIX Copyright intoPIX 2019: This presentation contains information prepared for IP Showcase Theate rand may be reused if credit is given to the IP Showcase and intoPIX SA ST 2110 is taking off It is designed to become the infrastructure of choice IP What is the « real » economics of going IP ? • Reducing complexity ‒ less cables , bi -directional • Becoming more agile ‒ re -routing , easy configuration, less space , smaller building & OB , simplified workflows • Reducing Cost ? W e have more pixels to manage, store and transport 4 … but the roads are jammed already « Can we put more cars on a road without creating traffic jam & delaying the arrival time of each passengers ? » Needs 10GbE infrastructures Uncompressed: 2.4 Gbps for HD 60fps 5 IP INFRASTRUCTURE 10GbE Intra -frame compression is needed for the storage to ease read & write access to the disks & to reduce storage cost of all incoming streams Many solutions exist for downscaling great amounts of streams in low resolution COTS in this case means at least 10GbE ports for all devices & switches (At least Cat 6 cables ) IP CAMERAS IP REPLAY & STORAGE IP PRODUCTION SWITCHER IP MONITORING HD: COTS IP SWITCH Needs at least 25GbE infrastructures Uncompressed : 9.6Gbps for 4K 60fps 6 IP INFRASTRUCTURE 25GbE ? Intra -frame compression is needed for the storage to ease read & write access to the disks & to reduce storage cost of all incoming streams New scaling capabilities needed for monitoring COTS in this case means at least 25GbE ports for all devices & switches 4K: IP CAMERAS IP REPLAY & STORAGE IP PRODUCTION SWITCHER IP MONITORING COTS IP SWITCH Needs at least 100/400GbE infrastructures Uncompressed: 38,4Gbps for 8K 60fps and 76,8Gbps for 8K 120fps 7 IP INFRASTRUCTURE 100/400GbE? Intra -frame compression is needed for the storage to ease read & write access to the disks & to reduce storage cost of all incoming streams New scaling capabilities needed for monitoring (even more steps down) COTS in this case means at least 100/400GbE ports for all devices & switches 8K: IP CAMERAS IP REPLAY & STORAGE IP PRODUCTION SWITCHER IP MONITORING COTS IP SWITCH What if a technology could help 8 …managing easily more pixels over a limited bandwidth, safeguarding low latency and a pixel perfect quality? Call for a new standard 9 Manage more pixels! Save cost & power! Preserve quality with no latency! Simplify ST 2110 connectivity ! Call for a new standard 10 2016 2017 2018 2019 Call for proposal A ne w low -latency ligthweight image coding system Liaison with AIMS, SMPTE and VSF TICO selected as baseline amongst 6 international proposals. Collaborative work . The standard moves to voting and publication phases JPEG -XS goes Life ! First implementations shown at NAB 2019 XS= X tra S mall X tra S peed Where can JPEG XS be implemented? Where can JPEG XS be implemented? In any applications for which pixel perfect quality, minimal latency , low complexity and efficient video bandwidth are crucial! JPEG - XS, Coming to ST2110 • The new Part -22 - Compressed video essence 13 JPEG - XS, Coming to ST 2110 ongoing standardisation ITEM Description Target Date ISO/IEC 21122 -1 Part 1 : Core coding system Q2 2019: PROOF – to be soon published + AMD for extended cap. Q2 2020 ISO/IEC 21122 -2 Part 2 : Profiles and buffer models Q2 2019: submitted to ISO for publication + AMD for extended cap Q2 2020 ISO/IEC 21122 -3 Part 3 : Transport and container formats Q3 2019: under last ballot – Final DIS ISO/IEC 21122 -4 Part 4: Conformance testing Q3 2019: under last ballot -DIS ISO/IEC 21122 -5 Part 5 Reference software Q4 2019: first ballot -CD IETF RFC JPEG -XS RTP JPEG -XS RTP payload Draft formally adopted by IETF payload WG https://datatracker.ietf.org/doc/draft -ietf - payload -rtp -jpegxs/ SMPTE 2110 -22 Compressed essence in ST 2110 Final Stage. 14 JPEG - XS, Benefits to ST 2110 • transport of compressed essence instead of uncompressed . ‒ better in bandwidth to manage multiple streams in HD, 4K and 8K • keep all existing advantages of moving to IP ‒ flexibility , scalability, unlimited accessibility • better impact on operating and infrastructure costs ‒ upgrade capability, lower investments, lighter infrastructures & systems smaller interfaces, ease the remote production and cloud migration. 15 JPEG - XS , Replacing Uncompressed 16 Combining the best speed, complexity and quality in one codec JPEG XS, Rigorous ISO Quality Assessments Tests with objective and subjective methods New ISO /IEC 29170 -2 method for near -lossless quality assessment on both natural & synthetic images ) ✓ Full transparency to uncompressed down to 3bpp (10:1) ✓ Visually lossless down to 1.5bpp (20:1) on film/TV content ✓ Smooth degradation down to 0.5bpp (ringing artefacts/ no blocking artefacts!) 17 "FLICKER TEST" Uncompressed Interleaving original and compressed every 1/8 sec (after 7 thencode/decode) Test on 360 scores (= persons) in total (from 4 universities/research centers) JPEG XS, Example content (CGI, desktop, natural) 18 JPEG XS, Best quality in single - and multi - generation 19 SINGLE GENERATION EXPERIMENT ENCODER DECODER ORIGINAL FRAME DECODED FRAME PSNR PSNR 1 PSNR 2 ENCODER DECODER ORIGINAL FRAME DECODED FRAME 1 ENCODER DECODER DECODED FRAME 2 MULTI GENERATION EXPERIMENT Performed for 10 generations JPEG XS, Best quality in multi - generation 20 Notes: ▪ JPEG is 8bit only ▪ J2K is 3 to 5x more complex and needs external memory JPEG - XS, High Quality in microseconds 21 Test: Natural image with equally distributed details vertically on JPEG -XS JPEG - XS, High Quality in microseconds 22 Down/Upsampling to 422 with minor effect In this image Above 40dB typically visual lossless PSNR (dB) bpp compression ratio 35 40 45 50 55 60 0 1 2 3 4 5 6 7 8 Content: Alexa Drums 444 / 8 Bit JPEG XS (WD) VC2 ProRes JPEG2000 6:1 8:1 12:1 24:1 ProRes 4444 ProRes 422 HQ ProRes 422 ProRes 422 LT ProRes 422 Proxy 4:1 Test: Natural image with equally distributed details vertically on JPEG -XS JPEG - XS, High Quality in microseconds 23 Test: Mixed desktop content with natural images, CGI content, and text on JPEG -XS JPEG - XS, High Quality in microseconds 24 25 30 35 40 45 50 55 60 65 0 1 2 3 4 5 6 7 8 Richter ScreenContent 444 / 8Bit JPEG XS (WD) VC2 LD ProRes JPEG2000 4:1 6:1 8:1 12:1 ProRes 422 LT ProRes 422 ProRes 422 HQ ProRes 4444 422 Downsampling reduces quality 444 recommended for text and mixed content PSNR (dB) bpp compression ratio Test: Mixed desktop content with natural images, CGI content, and text on JPEG -XS JPEG - XS, M inimal latency • Down to a few microseconds (down to 1/10 of a millisecond): : only a few video lines . • Maximum responsiveness (few µs) – lines - perfect for any latency critical applications • CBR (constant bitrate) for reliable video over IP transport. 25 Humans are able to detect a latency only above 13 milliseconds. Massachusetts Institute of Technology (MIT) " " JPEG XS, All platforms Minimal complexity…leading to maximum efficiency • Multiple profiles for Low power, L ow logic ‒ no external memory in hardware (FPGA, ASIC) ‒ The smallest codec for FPGA at this efficiency • Optimal syntax for software and speed optimizations (CPU, GPU) ‒ up to 5x faster or more than JPEG2000 ISO standard in CPU , GPU • Best ratio cost/infrastructure trade -off & Best ratio power trade -off 26 JPEG XS, Maximum Flexibility • Multiple resolutions : HD, 4K, 8K… up to at least 16Kx16K • Multiple chroma formats : 4:4:4, 4:2:2, 4:2:0, grayscale • Multiple color formats : RGB, YUV, … • Multiple bit depths: From 8, 10, 12, 14 to 16bit • HDR support : HDR support 27 JPEG XS, Maximum Flexibility 28 HD 4K 8K 8K codestream HD 4K Original 8K Partial HD HD Partial 4K 4K Partia l8K 8K Built -in 1-to 2 -level downscaler • HD/4K /8K downscaler within workflows (i.e. for monitoring purpose) • Lower CPU/GPU decoding requirements (less consumption to decode HD than 4K & 8K ) • Partial extraction for faster analytics and detection JPEG XS, ST 2110 Bandwidth - efficient workflow FORMATS JPEG -XS IP NETWORKS & SDI MAPPING HD 720p60 /1080i60 200 Mbps -70 Mbps 1 to x streams over 1GbE (CAT 5e) HD 1080p60 400 Mbps -150 Mbps 1 to x streams over 1GbE (CAT 5e) 4K 2160p60 1,6 Gbps -500 Mbps 1 stream over 1GbE (CAT 5e) 1 to x streams over 10GbE (CAT 6) Down to a single SDI cable (HD/3G -SDI) 8K 4320p60 6,4 Gbps -2 Gbps 1 to 4 streams over 10 GbE (CAT 6) Down to a single SDI cable (3G/6G/12G -SDI) 8K 4320p120 12,8 Gbps -4 Gbps 1 to 2 streams over 10 GbE (CAT 6) Down to a single SDI cable (6G/12G -SDI) 29 JPEG XS, ST 2110 Bandwidth - efficient workflow 30 IP INFRASTRUCTURE max. 10GbE Full access to 8K proxies directly from the coded streams No scaling required, even from a simple laptop COTS in this case means max. 10GbE ports for all devices & switches IP CAMERAS IP REPLAY & STORAGE IP PRODUCTION SWITCHER IP MONITORING COTS IP SWITCH Conclusion • JPEG -XS meets all the ST2110 quality requirements ‒ CBR, latency, quality, complexity, … • JPEG -XS bandwidth -reduction enables to achieve more with ST2110 ‒ higher pixel rates, more streams, cheaper cables (CAT5e, 3G -SDI) and interfaces (<1Gpbs, <10Gbps), reduced costs, reduced storage, reduced IP packets, … ST2110 -22 & JPEG -XS are enabling to create cost -effective, bandwidth -efficient and high quality IP production workflows C U R A T E D B Y I P S H O W C A S E T H E AT E R AT N A B – A P R I L 8 -11 , 2 01 9 Thank You Jean -Baptiste Lorent, intoPIX (Central Hall C8626) jb. lorent@intopix.com +3249654175 5 About us, intoPIX • Founded in 2006, HQ in Belgium. • Technology provider of innovative compression technologies empowering visual communications. • Member of AIMS, VSF, SMPTE and JPEG committees. • Deliver unique FPGA/ASIC IP cores and fast SDKs to manage more pixels, preserve quality with no latency, save cost & power and simplify connectivity. • Track record in terms of success stories, innovation achievements and effectiveness in enhancing Broadcast applications. ‒ EMMY for technology & engineering on JPEG2000 VSF TR01 , Invention & Standardization of TICO at SMPTE RDD35 and at JPEG as JPEG -XS, IABM Game Changer, IABM Peter Wayne Award for Innovation, EY Belgian Most promising growing company finalist, Delloite Fast50,…Serving 100+ customers worldwide • More info on : www.intoPIX.com