DOCUMENT #: GSC15-PLEN-51 FOR: Presentation SOURCE: ITU AGENDA ITEM: Plenary 6.10 CONTACT(S): [email protected] Conformance and Interoperability Testing Reinhard Scholl ITU Global Standards Collaboration (GSC) GSC-15
Jan 02, 2016
DOCUMENT #: GSC15-PLEN-51
FOR: Presentation
SOURCE: ITU
AGENDA ITEM: Plenary 6.10
CONTACT(S): [email protected]
Conformance and Interoperability Testing
Reinhard SchollITU
Global Standards Collaboration (GSC) GSC-15
Highlight of Current Activities (1/3)
ITU-T SG 11: Lead study group on test specifications
ITU-T SG 15: Has extensive testing ecosystem ITU-T SG 16: its IPTV standards (H-series) was
put to test in ITU-T’s first interop event (20-23 July)• 3 IPTV testing specs approved; others further developed
ITU-T SG 17: Lead study group on languages and description techniques• Also: Q14/17 on Testing methodology, specification language and
Generic test suites for CIT
2
Highlight of Current Activities (2/3)
ITU-T JCA-CIT: Joint Coordination Activity on Conformance and Interoperability Testing• Works with all Study Groups to:
address the topics relevant to conformance and interoperability testing
enhance the quality of ITU-T Recommendations• Eliminate errors, ambiguities unverified behaviours
• Include sufficient detail for implementation and testing
• Include interfaces needed for interconnection and testing
develop methodologies and frameworks for testing
3
Highlight of Current Activities (3/3)
Regional ITU consultations on conformance assessment and interoperability• Quito, 8 July 2010
• Nairobi, 30-31 July 2010
• Sydney, 16-17 Sep. 2010
Pilot conformity database developed ITU Interop started
• IPTV (July in Geneva; Sep. in Singapore; Dec. in India)
• Home networking
4
Strategic Direction WTSA-08 Res. 76 on conformance & interop … supported & complemented by WTDC-10
Res. 47 (ITU-D) ITU Council-09 mandated
• a pilot conformity assessment database • interoperability events • human resources capacity building • assistance in the establishment of test facilities in
developing countries • a business plan for long-term implementation
5
Challenges
Regional ITU consultations on conformance assessment and interoperability are a useful discussion platform, see e.g.• Presentations at Kenya workshop, in particular
TSB’s overview presentation on status of implementation of WTSA-08 Res. 76
6
Next Steps/Actions
Continue implementing steps outlined under “Strategic directions”
Study groups to continue working on test specs Keep momentum of ITU’s IPTV Interop event
• Expand to other ITU standards, in collaboration with partners as appropriate
Populate conformance database
7
Supplementary Slides
8
ITU-T SG 11’s Questions on Testing
ITU-T SG 11: Signalling, protocols and test specs• Q 8/11 Protocol Test Specifications for NGN • Q 9/11 Monitoring parameters for NGN
protocols • Q 10/11 Service test specification for NGN • Q 11/11 QoS tests specification for NGN • Q 12/11 NID and USN test specification
ITU-T SG 11 Approved Standards Q.3900 Methods of testing and model network architecture for NGN
technical means testing as applied to public telecommunication networks
Q.3901 Distribution of tests and services for NGN technical means testing in the model and operator networks
Q.3902 Parameters to be monitored in the process of operation when implementing NGN technical means in public telecommunication networks
Q.3903 Formalized presentation of testing results Q.3904 The scenarios, list and types of tests for TM local and NUT
testing for IMS on the model networks Q.3910 Monitoring parameters set for NGN protocols Q.3911 Monitoring parameters set for voice services in NGN
10
11
ITU-T SG 15: “Optical transport networks and access network infrastructures”
Tight collaboration:• Access (DSL, PON):
Broadband Forum
• Optical transport, framing, cross connecting Optical Internetworking Forum (OIF) Metro Ethernet Forum (MEF)
• Home networking HomeGrid Forum Broadband Forum
Nairobi, Kenya, 30-31 July 2010
ITU-T SG 15’s Extensive Testing Ecosystem
Develop conformance testing methodology and framework
Develop interoperability testing methodology and framework
Assist standards developers for applying conformance and interoperability testing methodology to their specific needs
Progress work in the area of Testing and Test Control Notation, version 3 (TTCN-3)
ITU-T SG 17’s Testing languages, methodologies and framework
ITU-T SG 17’s approved standards Conformance Testing Methodology and Framework
X.290 - General Concepts X.291 - Abstract Test Suite Specification X.292 - Tree and Tabular Combined
Notation X.293 - Test Realization X.294 - Requirements on Test
Laboratories and Clients X.295 - Protocol Profile Test Specification X.296 - Implementation Conformance
Statements X Suppl. 4 - Supplement on generic
approach to interoperability testing X Suppl. 5 - Supplement on
interoperability testing framework and methodology
Testing and Test Control Notation version 3 (TTCN-3) Z.161: TTCN-3 core language Z.162: TTCN-3 tabular presentation
format (TFT) Z.163: TTCN-3 graphical presentation
format (GFT) Z.164: TTCN-3 operational semantics Z.165: TTCN-3 runtime interface (TRI) Z.166: TTCN-3 control interface (TCI) Z.167: TTCN-3 mapping from ASN.1 Z.168: TTCN-3 mapping from CORBA
IDL Z.169: TTCN-3 mapping from XML data
definition Z.170: TTCN-3 documentation comment
specification
14
Imp
lem
en
tatio
n o
f the IT
U
Con
form
ity
Pro
gra
mm
e
ITU-T Recs & test suites
Supplier’s conformity
routedecision
Test lab (certification body
responsibility)
1st 2nd or 3rd party accredited lab
(ISO/IEC 17025)
ITU Conformity Database
Test results(ITU-T X.290)
1st partyEvaluation
Supplier’s ConformityDeclaration
(ISO/IEC 17050)
Test results(ITU-T X.290)
Accredited(ISO/IEC Guide 65) Certification body
Evaluation
ConformityCertificate issued by
CertificationBody
Con
form
ity A
ssessm
en
t / C
ertifi
catio
nIT
U C
IP s
erv
ices
Supplier’s Request to ITU
Route 1
Route 2
15
Supplier’s Declaration of Conformity – SDoC 1/2
16
Supplier’s Declaration of Conformity – SDoC- 2/2
Putting to the test: ITU-T’s IPTV suite of standards: H.721 et al.
Geneva, 20-23 July: testing, followed by showcasing Participating companies highly satisfied: 4.5 rating (out of 5) Event highlight on YouTube, ITU website:
http://youtube.com/watch?v=CpJmI_5aQrA
ITU press release: http://itu.int/net/pressoffice/press_releases/2010/31.aspx
Photos on Flickr: http://flickr.com/photos/itupictures
ITU’s 1st Interop Event (1/2)
Test report will be fed into the standards development process
H.721 Conformity Check List
H.762 Conformity Check List
H.770 Conformity Check List
H.750 Conformity Check List
ITU’s 1st Interop Event (2/2)
Testing specs:• HSTP.CONF-H721• HSTP.CONF-H762• HSTP.CONF-H770