Undertaking an Interoperability Lab at NFL Network

Presenters : Alan Wollenstein / Charley Haggarty - NFL

Year : 2019

dlp_document_download : AIMS IP Showcase IBC 2019 September 2019 Curated by Video Services Forum vsf.tv 1 I P S H O W C A S E T H E AT R E AT I B C 2 0 1 9 : 1 3 –1 7 S E P T 2 0 1 9 C U R A T E D B Y Undertaking an Interoperability Lab at the NFL Network We will break your gear Alan Wollenstein, Sr Manager Charley Haggarty, IT and Broadcast Engineer NFL Network 1 I P S H O W C A S E T H E AT R E AT I B C 2 0 1 9 : 1 3 –1 7 S E P T 2 0 1 9 Undertaking an Interoperability Lab Goals Preparation Lessons 2 AIMS IP Showcase IBC 2019 September 2019 Curated by Video Services Forum vsf.tv 2 I P S H O W C A S E T H E AT R E AT I B C 2 0 1 9 : 1 3 –1 7 S E P T 2 0 1 9 C U R A T E D B Y Goals Why should we build one? We BREAK things. 3 I P S H O W C A S E T H E AT R E AT I B C 2 0 1 9 : 1 3 –1 7 S E P T 2 0 1 9 Goals •What is ST 2110? ‒How is it disruptive? What's new? What has changed? What are the standards? ‒What we need to generate. •What is a ST 2110 broadcast facility. ‒How do today’s workflows look in tomorrow’s 2110 facility? ‒How do you troubleshoot? ‒What needs to be monitored, and what are the thresholds? •Best of Breed ‒What are the products, how do we compare them? ‒How do vendors interoperate? 4 AIMS IP Showcase IBC 2019 September 2019 Curated by Video Services Forum vsf.tv 3 I P S H O W C A S E T H E AT R E AT I B C 2 0 1 9 : 1 3 –1 7 S E P T 2 0 1 9 C U R A T E D B Y Preparation What it took to build a 2110 lab & how we broke it. 5 I P S H O W C A S E T H E AT R E AT I B C 2 0 1 9 : 1 3 –1 7 S E P T 2 0 1 9 Preparation Education Selection Documentation 6 AIMS IP Showcase IBC 2019 September 2019 Curated by Video Services Forum vsf.tv 4 I P S H O W C A S E T H E AT R E AT I B C 2 0 1 9 : 1 3 –1 7 S E P T 2 0 1 9 Education •Topics ‒The Standard – ST 2110 ‒Protocols ‒Tools •How to generate content? •How to introduce impairments? •How to measure results? •Research ‒Documents ‒Conferences ‒Videos ‒Subject Experts •Hands on 7 I P S H O W C A S E T H E AT R E AT I B C 2 0 1 9 : 1 3 –1 7 S E P T 2 0 1 9 Selection Tools Configurations Objectives Infrastructure Inventory 8 AIMS IP Showcase IBC 2019 September 2019 Curated by Video Services Forum vsf.tv 5 I P S H O W C A S E T H E AT R E AT I B C 2 0 1 9 : 1 3 –1 7 S E P T 2 0 1 9 Configuration – Part 1 •Formats ‒1080i, 1080p or 4k ‒SDR or HDR ‒Stream counts ‒Audio channels per stream 1080i and 2 streams of 8 channels for audio. •Protocols ‒NMOS, Ember+, APIs & Drivers ‒Static Multicast Routing (SDN) vs Dynamic Multicast Routing Control: In band vs Out of Band 9 I P S H O W C A S E T H E AT R E AT I B C 2 0 1 9 : 1 3 –1 7 S E P T 2 0 1 9 Configuration – Part 2 •Networks ‒Single switch (Monolithic), Spine -Leaf or Start Topology ‒Media, Management, Control, Audio, Intercom and PTP distribution 3 Networks in Spine -Leaf oAmber and Blue for Media using ST 2022 -7 and using 2 manufactures. oThird network for management, etc. •Cables ‒Single Mode or Multi Mode ‒Active Optical Cable, Active Copper or Passive Copper We mixed and matched to test for differences. 10 AIMS IP Showcase IBC 2019 September 2019 Curated by Video Services Forum vsf.tv 6 I P S H O W C A S E T H E AT R E AT I B C 2 0 1 9 : 1 3 –1 7 S E P T 2 0 1 9 Objectives •Data Gathering ‒The Tangible •Outcome when introducing known issues, like the zebra pattern, reaction to PTP Grand Master failover, etc. •Capture measurements, like processing delay, jitter, etc. •Performance of each vendor. ‒Tracking Software/Firmware versions, configuration settings parameters. ‒The Intangible •How is setup and configurations done? Can we do it ourselves? •Compare approach and execution of each vendor. 11 I P S H O W C A S E T H E AT R E AT I B C 2 0 1 9 : 1 3 –1 7 S E P T 2 0 1 9 Test Subjects •What systems do we need? •Processing Nodes •Production Switchers •Network Switchers •Cameras and CCUs •Intercoms •Audio Systems •Phone System •Multiviewers •Broadcast Controllers •SDNs •Orchestrators •Monitoring •PTP Grand Master Generator 12 AIMS IP Showcase IBC 2019 September 2019 Curated by Video Services Forum vsf.tv 7 I P S H O W C A S E T H E AT R E AT I B C 2 0 1 9 : 1 3 –1 7 S E P T 2 0 1 9 Infrastructure •Facility ‒Rack & Physical Space ‒Workstations ‒Panels and Consoles •Isolation ‒Physical ‒Logical •Cables ‒Single Mode vs Multi Mode ‒Patches 13 I P S H O W C A S E T H E AT R E AT I B C 2 0 1 9 : 1 3 –1 7 S E P T 2 0 1 9 Inventory – Hardware – Part 1 •Measure ‒Scopes, Waveform ‒A.V. Timing ‒Network packets •IP Gateways ‒Encapsulator ‒Decapsulator •Servers ‒Virtual Machines ‒Bare Metal •Additional ‒Panels ‒Distribution Amplifiers 14 AIMS IP Showcase IBC 2019 September 2019 Curated by Video Services Forum vsf.tv 8 I P S H O W C A S E T H E AT R E AT I B C 2 0 1 9 : 1 3 –1 7 S E P T 2 0 1 9 Inventory – Hardware – Part 2 •Connectivity ‒Switches •Media and Audio •Management •PoE ‒Adapters and Cables •Generators ‒PTP Grand Master ‒Video and Audio Signals (bars, tones, and AV sync) ‒Network emulator and impairment ‒Playback and Record devices 15 Test Subjects!!! Also knows as "Device Under Testing" I P S H O W C A S E T H E AT R E AT I B C 2 0 1 9 : 1 3 –1 7 S E P T 2 0 1 9 Inventory – Software •Software ‒Broadcast Controller ‒Scopes & Waveform monitors ‒Multiviewer & Layout Design ‒Network Orchestration •Monitoring •Configure •Control (SDN) ‒End Node Orchestration ‒Virtual Machine Cluster Management ‒Container hosts ‒DNS Server & NMOS registry 16 AIMS IP Showcase IBC 2019 September 2019 Curated by Video Services Forum vsf.tv 9 I P S H O W C A S E T H E AT R E AT I B C 2 0 1 9 : 1 3 –1 7 S E P T 2 0 1 9 Documentation •Procedures ‒Measure Delay ‒Capture failover behavior ‒Audio/Video Timing ‒Audio Shuffling •Drawings ‒Topology ‒Elevations ‒Flow Diagrams •Spreadsheets ‒IP Addresses ‒Multicast Groups ‒Results Common repository to share information and findings. 17 I P S H O W C A S E T H E AT R E AT I B C 2 0 1 9 : 1 3 –1 7 S E P T 2 0 1 9 C U R A T E D B Y Lessons What the Lab taught us & What we taught ourselves breaking it 18 AIMS IP Showcase IBC 2019 September 2019 Curated by Video Services Forum vsf.tv 10 I P S H O W C A S E T H E AT R E AT I B C 2 0 1 9 : 1 3 –1 7 S E P T 2 0 1 9 Lessons – Part 1 •Learning the basics. ‒Networking – Primer on routing and multicast. ‒Streaming media – Understanding of Payload ID and formats in the SDP file. ‒PTP – Profiles and accuracy dependencies. You can free run but we strongly recommend getting GPS . •Having a schedule. ‒A way to arrange and confirm delivery of gear. ‒Configuration changes to the lab for different test. ‒Consider setup, training, and duration of loan. ‒You can plan for overlap to test actual interactions. ‒Be flexible , the first time testing is not indicative of the consecutive ones. 19 I P S H O W C A S E T H E AT R E AT I B C 2 0 1 9 : 1 3 –1 7 S E P T 2 0 1 9 Lessons – Part 2 •Owning and understanding the environment ‒Testing procedures are only as good as your understanding, some test where not ascertaining what we expected. ‒Do quick changes as needed. ‒Account for how the setup affects the results. ‒Having strong support from manufacturers . •Enforcing the documentation ‒Easier to make and track changes to the lab and/or procedures. ‒Quickly provide any information to all parties involved with the assessment. 20 AIMS IP Showcase IBC 2019 September 2019 Curated by Video Services Forum vsf.tv 11 I P S H O W C A S E T H E AT R E AT I B C 2 0 1 9 : 1 3 –1 7 S E P T 2 0 1 9 Lessons – Part 3 •Space allocation ‒Quite space for: •Workstations to access systems. •Audio and intercom testing. ‒Space for production switchers and audio consoles. ‒Panels and displays. ‒Patches for fiber, copper and baseband. ‒There will be not only racks, but rolling cases. ‒Large or multiple groups in at the same time. •Reality of Scale ‒The desire scale was an expensive endeavor. ‒The sample size we did attain did not expand to cover concerns we had like actual CPU load on switches. ‒Just have to accept there will be limitations . 21 I P S H O W C A S E T H E AT R E AT I B C 2 0 1 9 : 1 3 –1 7 S E P T 2 0 1 9 Lessons – Part 4 •Last but not least, actually the most: BUDGET!!! ‒We started thinking we only needed switches, a waveform monitor, IP gateways and software. ‒Ended up adding more •Software applications. •Other gateways to compare •Signal and impairment generators and additional sources . ‒On that note, some of the gear we started with did not work out. •Lacking features, not using the standards or compatible. •Changes either on operations or the product line itself. •Manufactures approach was not aligning with ours. 22 AIMS IP Showcase IBC 2019 September 2019 Curated by Video Services Forum vsf.tv 12 I P S H O W C A S E T H E AT R E AT I B C 2 0 1 9 : 1 3 –1 7 S E P T 2 0 1 9 Lessons – Part 5 ‒Yet, we are seeing positive returns. •The lab is good for training staff because we have a controlled environment that can be broken and quickly repaired. •Continuing testing of new gear or updates to existing. •Configuration can be different from workflows based on availability and market trends. With the current progress of adoption, things are evolving. As they change we will be able to consider them. 23 I P S H O W C A S E T H E AT R E AT I B C 2 0 1 9 : 1 3 –1 7 S E P T 2 0 1 9 About Us •NFL Network ‒California based, provides live and pre -recorded sports programming for the NFL Media Worldwide. •Alan Wollenstein – Senior Manager ‒Over 20 years experience in both IT and Broadcast Media, as a vendor and customer. •Charley Haggarty – Broadcast and IT Engineer ‒About 10 years experience in both IT and Broadcast Media, primarily Sports Media. •Bruce Goldfeder – VP Broadcast Engineering ‒The financial backer. 24 AIMS IP Showcase IBC 2019 September 2019 Curated by Video Services Forum vsf.tv 13 I P S H O W C A S E T H E AT R E AT I B C 2 0 1 9 : 1 3 –1 7 S E P T 2 0 1 9 Thank you Thank you to our Media Partners Alan Wollenstein & Charley Haggarty, NFL Network alan.wollenstein@nfl.com 424 -345 -3107 charley.haggarty@nfl.com 424 -258 -7851 25