Scalability and Performance of IS-04 and IS-05

Presenters : Rob Porter - Sony

Year : 2019

dlp_document_download : C U R A T E D B Y I P S H O W C A S E T H E AT E R AT N A B – A P R I L 8 -11 , 2 01 9 Scalability and Performance of IS - 04 and IS - 05 and How TR - 1001 - 1 Helps Rob Porter Sony Europe B.V. 2 AMWA NMOS IS -04 and IS -05 APIs The AMWA NMOS Scalability Study Scalability Study Results Best Practice Recommendations and How JT -NM TR -1001 -1 Helps ... ... Reg Node Node ... Node Node Node ... Node Node Node 3 AMWA NMOS IS -04 and IS -05 APIs The AMWA NMOS Scalability Study Scalability Study Results Best Practice Recommendations and How JT -NM TR -1001 -1 Helps ... ... Reg Node Node ... Node Node Node ... Node Node Node 4 Professional Media Network 5 Network Infrastructure • One or more network switches • Any architecture • Single monolithic switch • Multi -layer spine -leaf network • Node is a logical host connected to the network • Can host one or more Devices each with any number of associated Senders , Receivers , Sources and Flows • These are known as resources and are all defined by the JT - NM content model Media Nodes Node Device Sender Receiver Receiver Sender Node Device Receiver Receiver Receiver Device Receiver Receiver Receiver Device Receiver Receiver Receiver Device Receiver Receiver Receiver Device Receiver Receiver Receiver Device Receiver Receiver Receiver Device Receiver Receiver Receiver Device Receiver Receiver Receiver Device Receiver Receiver Receiver 7 • RDS comprises one or more Registry instances • Each includes a database storing all registered resources • RDS exposes two APIs: • Registration API • Query API IS -04 Registration and Discovery System (RDS) 8 • Control system, e.g. • Broadcast Controller • Routing panel • Allows connections between Media Nodes to be set up NMOS Client 9 IS -04 Registration API IS -04 Registration API • On connecting a Node to the network: • Node discovers Registration APIs advertised over DNS -SD • Node selects a Registration API (by highest priority) • Node registers its Node resource with selected Registration API • Node registers each of its sub - resources (Devices, Senders, Receivers, Sources, Flows) and begins to post regular heartbeats 10 Node Registry Registration API Request ( Node→RDS ): POST /x -nmos/registration/v1.2/resource HTTP/1.1 Content -Type: application/ json { "type": "node", "data": { "version": "1441973902:879053935", "hostname": "host1", "label": "host1", "description": "host1", "tags": {}, "href ": "http://172.29.80.65:12345/", "api ": { "versions": ["v1.1", "v1.2"], "endpoints": [ { "host": "172.29.80.65", "port": 12345, "protocol": "http" }, { "host": "172.29.80.65", "port": 443, "protocol": "https" } ] }, 11 Node Registry Registration API Request ( Node→RDS ): POST /x -nmos/registration/v1.2/resource HTTP/1.1 Content -Type: application/ json { "type": "node", "data": { "version": "1441973902:879053935", "hostname": "host1", "label": "host1", "description": "host1", "tags": {}, "href ": "http://172.29.80.65:12345/", "api ": { "versions": ["v1.1", "v1.2"], "endpoints": [ { "host": "172.29.80.65", "port": 12345, "protocol": "http" }, { "host": "172.29.80.65", "port": 443, "protocol": "https" } ] }, Registration API Response ( RDS→Node ): HTTP/1.1 201 Created /x -nmos/registration/v1.2/resource/nodes/3b8be755 - 08ff -452b -b217 -c9151eb21193/ 12 Node Registry Heartbeat Request ( Node→RDS ): POST /x -nmos/registration/v1.2/health/nodes/3b8be755 - 08ff -452b -b217 -c9151eb21193 13 Node Registry Heartbeat Request ( Node→RDS ): POST /x -nmos/registration/v1.2/health/nodes/3b8be755 - 08ff -452b -b217 -c9151eb21193 Heartbeat Response ( RDS→Node ): HTTP/1.1 200 OK { "health": "1441974485" } 14 IS -04 Query API • Client is able to get list of registered resources from Registry using IS -04 Query API • It can also subscribe to WebSocket notifications of changes in the RDS IS -04 Query API 15 IS -05 Connection API • Client can use a Node’s IS -05 Connection API to make a connection • Get transport file from Sender • Patch transport file to Receiver • Join multicast group IS -05 Connection API (Patch transport file to Receiver) IGMP Join IS -05 Connection API (Get transport file from Sender) x 16 Client Node (Receiver) Transport file (SDP file): v=0 o= - 3755583281 3755583281 IN IP4 192.168.9.142 s=Camera 1 Video t=0 0 a= group:DUP PRIMARY SECONDARY m=video 50020 RTP/AVP 96 c=IN IP4 239.22.142.1/32 a= ts -refclk:ptp =IEEE1588 -2008:traceable a= mediaclk:direct =0 a=source -filter: incl IN IP4 239.22.142.1 192.168.9.142 a=rtpmap:96 raw/90000 a=fmtp:96 width=1920; height=1080; exactframerate =30000/1001; interlace; sampling=YCbCr -4:2:2; depth=10; colorimetry=BT709; TCS=SDR; PM=2110GPM; SSN=ST2110 -20:2017; TP=2110TPN; a=mid:PRIMARY m=video 50120 RTP/AVP 96 c=IN IP4 239.122.142.1/32 a= ts -refclk:ptp =IEEE1588 -2008:traceable a= mediaclk:direct =0 a=source -filter: incl IN IP4 239.122.142.1 192.168.109.142 a=rtpmap:96 raw/90000 a=fmtp:96 width=1920; height=1080; exactframerate =30000/1001; interlace; sampling=YCbCr -4:2:2; depth=10; colorimetry=BT709; TCS=SDR; PM=2110GPM; SSN=ST2110 -20:2017; TP=2110TPN; a=mid:SECONDARY 17 AMWA NMOS IS -04 and IS -05 APIs The AMWA NMOS Scalability Study Scalability Study Results Best Practice Recommendations and How JT -NM TR -1001 -1 Helps ... ... Reg Node Node ... Node Node Node ... Node Node Node 18 The AMWA NMOS Scalability Study • A key requirement of the AMWA IS -04 and IS -05 APIs is that they can be used reliably at scale • i.e. for very large networks comprising thousands of NMOS Nodes such as might be found in a typical broadcast installation. • The aim of the AMWA NMOS Scalability Study was to help address this • Study took place within the AMWA community and was led by Sony • The study used a virtualised network to test and make timing measurements of various IS -04 and IS -05 operations at scale 19 Scalability Study Methodology • Use Mininet virtualised network to simulate large number of network endpoints • Mininet extended for NMOS to allow NMOS processes to be run on each Mininet host Physical h ost comput er (Wind ows , Lin ux or Mac ) macOS Physical h ost comput er (Wind ows , Lin ux or Mac ) Mininet Virtual Machin e (Lin ux ) macOS Physical h ost comput er (Wind ows , Lin ux or Mac ) Mininet Virtual Machin e (Lin ux ) macOS Net work Controller (optional ) Physical h ost comput er (Wind ows , Lin ux or Mac ) Mininet Virtual Machin e (Lin ux ) Hosts macOS Net work Controller (optional ) ... ... ... Physical h ost comput er (Wind ows , Lin ux or Mac ) Mininet Virtual Machin e (Lin ux ) Switches Hosts macOS ... Net work Controller (optional ) ... ... ... Physical h ost comput er (Wind ows , Lin ux or Mac ) Mininet Virtual Machin e (Lin ux ) Switches Net worklinks Hosts macOS ... Net work Controller (optional ) ... ... ... 20 Scalability Study Methodology • Use Mininet virtualised network to simulate large number of network endpoints • Mininet extended for NMOS to allow NMOS processes to be run on each Mininet host • Run nmos -cpp -registry* on one Mininet host * https://github.com/sony/nmos -cpp Physical h ost comput er (Wind ows , Lin ux or Mac ) Mininet Virtual Machin e (Lin ux ) Switches Net worklinks Hosts NMOSprocesses macOS ... Net work Controller (optional ) ... Reg ... ... 21 Scalability Study Methodology • Use Mininet virtualised network to simulate large number of network endpoints • Mininet extended for NMOS to allow NMOS processes to be run on each Mininet host • Run nmos -cpp -registry* on one Mininet host • Run multiple instances of nmos -cpp -node* on multiple other Mininet hosts * https://github.com/sony/nmos -cpp Physical h ost comput er (Wind ows , Lin ux or Mac ) Mininet Virtual Machin e (Lin ux ) Switches Net worklinks Hosts NMOSprocesses macOS ... Net work Controller (optional ) ... Reg Node Node ... Node Node Node ... Node Node Node 22 Kibana Dashboard Logst as h w ith Custom Plugi n Physical h ost comput er (Wind ows , Lin ux or Mac ) Mininet Virtual Machin e (Lin ux ) Switches Net worklinks Hosts NMOSprocesses macOS ... Net work Controller (optional ) ... Reg Node Node ... Node Node Node ... Node Node Node Elasti csear ch Database 23 AMWA NMOS IS -04 and IS -05 APIs The AMWA NMOS Scalability Study Scalability Study Results Best Practice Recommendations and How JT -NM TR -1001 -1 Helps ... ... Reg Node Node ... Node Node Node ... Node Node Node 24 Initial registration Test : Registration of 100 - 2,500 Nodes with 6 resources per Node - one of each resource type: Node, Device, Sender, Receiver, Source, Flow Result : Total registration time scales linearly with number of resources 25 Initial registration Test : Registration of 2,500 Nodes with 6 resources per Node Result : Total registration time > 3.5 minutes Rate of registration varies over time – slow start and long tail > 3.5 minutes 26 Initial registration Test : Registration of 750 Nodes with 6 resources per Node Result : Total registration time > 50 seconds Slow start and long tail due to long DNS -SD and HTTP timeout and retry intervals > 50 seconds Slow start Long tail 27 Initial registration Test : Registration of 750 Nodes with 6 resources per Node. With optimisations to: • DNS -SD retry interval • HTTP timeout Result : Total registration time < 30 seconds No slow start or long tail < 30 seconds 28 Recovery after a failure Test: Register 750 Nodes with 6 resources per Node. Break network link to registry for a short period Result 1 : Break for 5s -> Full recovery within 30s 5s 30s Result 2 : Break for 30s -> Full recovery within 90s 29 Multiple registries Test : Registration of 750 Nodes with 6 resources per Node with two -way replication in a federated RDS Result : Total registration time reduced from 30 seconds to <15 seconds < 15 seconds Connection management at scale Test : Populate Client’s crosspoint matrix (2,500 x 2,500) using IS -04 Query API Result : Total time to populate matrix < 1.0s Connection management at scale Test : Update 750 crosspoints using IS -05 Connection API Result : Total time to update crosspoints < 3.0s Optimised API usage is important - make good use of paging, push notifications and WebSocket API. 32 AMWA NMOS IS -04 and IS -05 APIs The AMWA NMOS Scalability Study Scalability Study Results Best Practice Recommendations and How JT -NM TR -1001 -1 Helps ... ... Reg Node Node ... Node Node Node ... Node Node Node 33 Best Practice Recommendation • Use IS -04 and IS -05 across all Media Nodes • Ensures interoperability! How JT -NM TR -1001 -1 Helps • Mandates use of IS -04 and IS -05 Media Nodes shall expose an AMWA NMOS IS -05 1.0.2 STABLE or higher device connection management API Media Nodes shall expose an AMWA NMOS IS -04 1.2 or higher node API, and shall register using the IS -04 registration API 34 Best Practice Recommendation • Use clustered / federated registries to improve performance • Faster registration • Better fault tolerance How JT -NM TR -1001 -1 Helps • Multiple registries supported The Network Environment shall contain one or more registries implementing AMWA NMOS IS -04 Registration and Query APIs 35 Best Practice Recommendation • Choose heartbeat and registry expiry intervals carefully • Use common heartbeat intervals across all Media Nodes • Can be used to guide choice of DNS -SD and HTTP timeout and retry intervals • Recommended values from IS -04 spec worked well in our experiments How JT -NM TR -1001 -1 Helps • Heartbeat interval is stored in System Resource and must be used by all Media Nodes Media Nodes shall use the registry heartbeat_interval value specified in the System Resource … when maintaining their registration 36 Best Practice Recommendation • Use unicast DNS -SD • Improved registration performance • Better for scalability • Essential for layer 3 networks • Multicast DNS is being deprecated from IS -04 How JT -NM TR -1001 -1 Helps • Unicast DNS -SD is mandated Media Nodes shall use unicast DNS Service Discovery (DNS -SD) to locate the registration APIs as described in IS -04 37 Resources • AMWA NMOS Scalability Study test environment • https://github.com/AMWA -TV/nmos -scalability • Sony nmos -cpp open source software for IS -04 Registry and IS -04/ -05 Node • https://github.com/sony/nmos -cpp • SMPTE Annual Technical Conference 2018 paper • "Scalability and Performance of the AMWA IS -04 and IS -05 NMOS Specifications for Networked Media " – Robert Porter and Gareth Sylvester -Bradley • https://ieeexplore.ieee.org/document/8610041 C U R A T E D B Y I P S H O W C A S E T H E AT E R AT N A B – A P R I L 8 -11 , 2 01 9 Thank You Rob Porter, Sony Europe B.V. Rob.Porter@sony.com