Home

GFI user manual - Data Communications Company

1. Figure 26 Open Filter Dialog button GI LOG Filter Level up to Area MNGE TDRIV RPORT EQUIP ACB UNK TEST SS KRP CRYPT Limit visible entrys to Figure 27 Log Filter This will facilitate manual checks on exchanged messages The filter applies to the log level and the area but further filtering is possible by text keywords If a log is very extensive it is possible to limit the number of log entries in the log filter by limiting the visible entries Page 24 of 29 Copyright 2015 Smart DCC Ltd All Rights reserved DCC Public Export li is possible to export the files of a text execution for archiving purposes This action may be invoked from the Run context menu Dialogs will be displayed to select the output path for the produced files Explorer et p Vi Properties ESME MPAN xm X Scenaria ESME MPAN xm G i ESME MPAN E Properties ESME MPAN xml ae file home gfiuser runtime EclipseApplication Camp BEER E amp Campaign ESME MPAN TEST INFORMATION gt IE Run 2015 06 19 15 18 27 ESMEMPAN e Run_2015 06 19 15 18 04 Setting the MPAN on an ESME Run 2015 06 19 14 4A N Execution Date 2015 06 19 14 46 01 WEST A execution loc New Test verall Result PASS Run kj test report ht ie EST CASES RESULTS e f Campaign ESME esse ast Case Result gt Test Delete Delete G TC ECS39a 1 wv PASS e Run 2015 06 1 Refresh
2. Table 1 Input and Output artefacts The following Sections contain a description for each one of these artefacts and the role they play in the system Page 5 of 29 Copyright 2015 Smart DCC Ltd All Rights reserved DCC Public 2 3 1 Configuration File To setup a working environment for a Test all the configurations should be defined in the scenario file lt scenario file gt xml In this file the settings for the produced outputs test scheduler emulators equipment and codecs used may be tweaked to cope with environment needs For instance a Test can be executed in a scenario with real ESME GSME devices or in a scenario with emulated devices This configuration is defined in this file 2 3 2 Test Execution File The Test Cases that constitute the Test should be defined in the Test Execution file lt test execution gt xml After the Test specific configurations at the top of the file information regarding each Test Case directly mapped to a SMETS Use Case should be added namely input parameters and expected outputs 2 3 3 Test Report Files Upon the Test execution a report file is generated a formatted HTML with all the sets checks action prints and exchanged messages with a human readable appearance This file along with the Execution Log may be considered the final output and contains the overall PASS FAIL information as well as the detailed PASS FAIL for each expected result 2 3 4 Console Output All the r
3. G TC ECS40 1 wf PASS gt eeii m ATG TC ECS39a 2 w PASS ATG TC ECS40 2 w PASS L test report html e fi Campaign_ESME MPAN_002 TEST CASES EXECUTION me Tae ATG TC ECS39a 1 S ip Test Case Information ee ATG TCECS39q Figure 28 Export Test Run The selected output path should have write permissions for the user Export Run 2015 06 19 14 46 02 from Campaign ESME MPAN to 4 fi Create Folder A NF gfiuser Desktop Cre older Places Name Modified 5 O Search ype name of new folde G3 Recently Used i ofiuser bel Desktop E File System fj Documents Music jm Pictures B Videos Figure 29 Select output path Page 25 of 29 Copyright 2015 Smart DCC Ltd All Rights reserved DCC Public Generated files should then be available in the selected location En T dn E home gfiuser Desktop Run_2015 06 19 14 46 02 DEVICES Run 2015 06 19 14 46 02 File System PLACES execution test report html test report xml A anisat log csv ES Desktop Ei Trash Documents ww Music Ei Pictures Re Videos Fs Downloads 3 items 1 0 MB Free space 26 1 GB Figure 30 Exported files Along with the execution log and the test report HTML an XML report file is also generated should any report integration tool require an unformatted report Page 26 of 29 Copyright 2015 Smart DCC Ltd All Rights reserved DCC Public 5 Glossary Table 2 presents the list of definitions us
4. valu db alue Recon TEST CASES EXECUTION lt resourcepc gt ATG TC ECS39a 1 resourcepool hee ee name MeterPointAdministrationNumber combinedMPANS value gt gt 1234 lt valu Test Case information g Rm Sp e gt Test Case Id ATG TC ECS39a lt resourcepool gt The pu of this s to validate Use lt resourcepool class CResourcePoolString name MeterPointAdministrationNumber combinedMPANS result gt Wee ESCH a Set GE by Eesen lt V CSN alu Test Case Purpose r settin gt ES in ape D PAN gt 0 lt valu n on the ME and i Zira 50 epool gt Si stc Test Case Assumptions And stcase thes CTestCaseECS40 name ATG TC ECS46 2 Constraints eport gt Author lt item name TEST CASE ID gt ATG TC ECS46 lt item gt z Version Iteration 1 Design Source Inputs gt Console 9 a Lo o pr e e p Running Test Project Campaign_ ESME MPAN 00 00 00 399 MNGE 2 on ATG TC ECS46 2 Co er ate 00 00 00 399 MNGE 2 stcase ATG TC ECS46 2 Overall Result PASS 00 00 00 400 MNGE 6 Com wie te Ru n loi 60 06 00 466 MNGE 8 Comple Test Execution I 1 item selected Figure 19 IDE layout Properties and Report However the purpose as described in the beginning of the Step by step example was to assign different values to the MPAN parameter This may be done by editing the Properties file in the Test changing the input parameter to whichever value is required for the test purpose In this example the second update
5. Figure 9 but after the first Test is created on yet another New Test the fields are populated with data from the last configuration Using the button Restore Defaults will revert to that same last configuration from a successfully added Test Search and Select Test Cases In this section test cases are selected to build the sequence for the new test Clicking in the Add Test Cases button on the upper left corner of the dialog will display a list of the test cases in the test library the SMETS Use Cases already coded Search and Select Test Cases ai e Search and Select Test cases to add to the Test A wg Add Test Cases Cancel Figure 10 Search and Select Test Cases The required test cases should then be selected and added to the sequence A filter may be applied to refine the test case list as shown in Figure 11 Upon selection the information of a test case is displayed in the pane on the right Selected test case mio Also multiple selection is allowed for addition Page 13 of 29 Copyright 2015 Smart DCC Ltd All Rights reserved DCC Public Filter Available Test Cases MPAN Select Test Cases to open Selected Test Case Info ATG TC ECS26e Read ESME Configuration Data Device Information device ii Purpose Information ATG TC ECS39a Set MPAN Value on the ESME ATG TC ECS39b Set Export MPAN Value on the ESME ATG TC ECS40 Read MPAN Value on the ESME The purpose of this test case is
6. TC ECS39a 1 Test Case Information Test Case Id ATG TC ECS39a The purpose of this test case is to validate Use Case ECS39a Set MPAN MPRN Value by Test Case Purpose validating the construction of the command used by the Supplier for setting the Import MPAN Value configuration on the ESME and its respective response Tanne man Aes ee pm Siamese Aas od kel GFI i ole Log X Run Run_2015 06 19 14 46 02 Start 2015 06 19 14 46 01 WEST Stop 2015 06 19 14 46 01 WEST Area Level Message 00 00 00 223 wuer Created Equipment SimulationManager 00 00 00 223 MNGE Created Equipment ACBEmulator 00 00 00 225 MNGE Created Equipment RPEmulator 00 00 00 225 TDRIV Setting TestDrive context where Sets and Sends will be performed 4 00 00 00 225 TDRIV Setting TestDrive context where Gets and Checks will be performed 3 00 00 00 232 MNGE Created Equipment ESMEEmulator Lan nn nn 222 itor cotting Gett context where Sate and Sands will he nerforned e A A 1 item selected Figure 17 Execution finished This concludes the Test execution As mentioned before should any further executions of this Campaign be performed new Run items will be added to the Campaign Old runs may be deleted but are kept by default for later analysis Note If a Test is executed a new Campaign is generated If a Campaign is executed a new Run in this Campaign is generated Campaig
7. TEST PURPOSE gt Setting the MPAN on an ESME device and checking the parameter on the meter The procedu lt report gt lt testcase class CTestCaseECS39a name ATG TC ECS39a 1 gt lt report gt lt item name TEST CASE ID gt ATG TC ECS39a lt item gt lt item name TEST CASE PURPOSE gt The purpose of this test case is to validate Use Case ECS39a Set MPAN MPRN Va lt item name TEST CASE ASSUMPTIONS AND CONSTRAINTS gt N A lt item gt lt item name AUTHOR gt CSW lt item gt lt item name VERSION gt 01 lt item gt lt report gt lt resource combiner class CLinearCombinations name linearCombinator gt lt resourcepool class CResourcePoolString name BusinessOriginatorName gt lt value gt SuppLlierl lt value gt lt value gt SuppLlierl lt value gt lt resourcepool gt lt resourcepool class CResourcePoolString name BusinessTargetName gt lt vaLlue gt ESME1 lt value gt egen luess E EMEZ e aal an e Design Source GFI Console D Log xs Y D Run Start Stop Seq Time Area Level Message Figure 13 New Test added Page 15 of 29 Copyright 2015 Smart DCC Ltd All Rights reserved DCC Public The data in each test case is set by default and has been previously tested The values are mainly gathered from the SMETS documentation where they are described as the default values for the SMETS object in the meter Each input or output in a test case is defined as a resource as
8. exportMPAN results B Scenario ESME MPAN xm lt value gt 0 lt value gt ATG TESTING TOOL TEST REPORT a value g i lt value gt amp Campaign_ESME MPAN we sourcepool gt TEST INFORMATION gt lt testcase Test Name ESME Set and Check MPAN gt Run 2015 06 19 15 18 27 lt testcase gue CTestCaseECS39a name ATG TC ECS39a 2 gt Test Id ESME MPAN 8 lt report gt Setting the MPAN lt ESME I Run_2015 06 19 15 18 04 lt item name TEST CASE ID gt ATG TC ECS39a lt item gt Seet Papin device and checking the A execution log csv lt item name TEST CASE PURPOSE gt The purpose of this test case is to validate Use Case ECS39a Set MEA parameter on the meter The G test report htm lt item name TEST CASE ASSUMPTIONS AND CONSTRAINTS gt N A lt item gt pare ae Sees lt item name AUTHOR gt CSW lt item gt Execution Date 2015 06 19 14 46 01 WEST TF Run_2015 06 19 14 46 02 lt item name VERSION gt 61 lt item gt Overall Result PASS A execution log csv lt report gt f ES n 8 ON test report htmI lt resource combiner class CLinearCombinations name linearCombinator gt TEST CASES RESULTS i rcep sat pr paren eal name BusinessOriginatorName gt Test Case Result lt y gt St elie ri lt valt ATG TC ECS393 w PASS value gt Supplierl lt value gt ATG TC ECS40 1 w PASS r esourcepool gt ATG TC ECS39a 2 wv PASS epool class bommuebegbereegtaien A name BusinessTargetName gt ATG TC ECS4042 w PASS gt ESME1 lt lt
9. it could be used in any possible way and included in a lt resourcepool gt element for that parameter Each entry in a resource pool will be used in each of the test case s iteration n entries in the resource pool will become n iterations in the test case All resource pools should have the same number of data entries as on the scope of GFI only linear combinations will be used a resource pool with a lower number of entries will loop through the values in the extra iterations On Section 4 3 3 a practical example is presented 4 3 2 Test Execution Using the example presented previously this section will detail the test execution procedure A Test previously created may be executed through the Run button in the toolbar or the option Run in the Test s context menu File Edit Search Design Window 8 2 CA Explorer BS ee x Properties ESME MPAN xml 2 en ESME MPAN l lt xml version 1 6 encoding UTF 8 gt e New lest EK D Scenario_ES estSpec name FSi ASA Delete Delete IT NAME gt ESME Set T ID gt ESME MPAN lt Refresh T PURPOSE gt Settin Figure 14 Run Test from context menu After clicking OK in the confirmation box the Test is executed wW Are you sure you want to run Test ESME MPAN Figure 15 Test execution confirmation The console activity may be observed in the console pane It is the actual real time output of the GFI execution as if it would have been run
10. lt testcase class CTestCaseECS40 name ATG TC ECS40 2 gt lt report gt lt item name TEST CASE ID gt ATG TC ECS40 lt item gt lt item name TEST CASE PURPOSE gt The purpose of this test case is to validate Use Case ECS40 Read MPAN lt item name TEST CASE ASSUMPTIONS AND CONSTRAINTS gt GIST Issue 3145 ECS40 Notes for attribute Met lt item name AUTHOR gt CSW lt item gt lt item name VERSION gt 01 lt item gt lt report gt lt resource combiner class CLinearCombinations name lLinearCombinator gt lt resourcepool class CResourcePoolString name BusinessOriginatorName gt lt value gt ACB lt value gt lt value gt ACB lt value gt lt resourcepool gt lt resourcepool class CResourcePoolString name BusinessTargetName gt lt value gt ESME1 lt value gt lt valLue gt ESME1 lt value gt lt resourcepool gt lt resourcepool class CResourcePoolString name SupplementaryRemotePartyName gt lt value gt SuppLlierl lt value gt lt value gt lt resourcepool gt lt resourcepool class CResourcePoolString name MeterAdministrationPointNumber combinedMPANS value gt lt value gt value gt lt value gt 5678 lt vaLlue gt lt resourcepool gt lt resourcepool class CResourcePoolString name MeterAdministrationPointNumber exportMPAN value gt lt vaLue gt 1312345678333 lt value gt lt valLue gt 1312345678333 lt value gt lt resourcepool gt lt resourcepool
11. of the tool lays a message oriented infrastructure where the message field is the most elementary entity In a simplified overview the tool executes elementary operations over message fields and messages namely sets checks gets sends and waits among a few others Therefore a fundamental concept that should always be kept in mind is the concept of message and respective fields Messages and respective fields are defined in a database automatically generated from message specifications This database creates an abstraction layer between the engine of the testing tool and the tests themselves and the protocols and interfaces through which messages are sent and received leveraging a high level of decoupling between Tests communication protocols and transmission medium To execute and produce the respective reports modify and or create Use Cases a number of steps need to be taken The following Sections provide detailed technical information for standard and advanced User operation Also a top level architecture overview is provided in the next Section describing each one of the main modules that form the testing tool Figure 1 provides a high level overview of the system architecture Besides the GFI Testing Tool Graphical User Interface there are six main modules shown in purple GFl Testing Tool application interface VSIS Core VSIS Comms Simulation manager Test Library and Messages Database GFI TESTING TOOL GUI Report lt HTM
12. test case AT G TC ECS39a 2 will be setting a different value Page 19 of 29 Copyright 2015 Smart DCC Ltd All Rights reserved DCC Public This test case runs two iterations as there are two data values per resource pool If the purpose of this test case in the Test is to set a specific value to the parameter rather than testing parameter boundaries only one iteration is required to attain this objective There are two ways go about it Remove the redundant iterations Leave just one value per resource pool and set the value of the resource pool that relates to the parameter requiring change Change the last iteration The easier way as no significant editing is required Just set the value of the resource pool that relates to the parameter requiring change in the last iteration Values set in previous iterations will be overwritten in the last one Going with the second option only the value highlighted in Figure 20 needs to be changed x Properties ESME MPAN xml 3 x Scenario ESME MPAN lt testcase class C TestCaseECS39a name ATG TC ECS39a 2 gt lt report gt lt item name TEST CASE ID gt ATG TC ECS39a lt item gt lt item name TEST CASE PURPOSE gt The purpose of this test case is to validate Use Case ECS39a Set MPAN lt item name TEST CASE ASSUMPTIONS AND CONSTRAINTS N A lt item gt lt item name AUTHOR CSW lt item gt lt item name VERSION gt 01 lt item gt lt report gt lt re
13. user has the required profile for the GFI testing tool which logs in automatically in the UI desktop and has the following credentials Username gfi Password gfi 3 3 Run Eclipse with GFI plugin The Testing Tool GUI is built as an Eclipse CDT C C Development Tooling plugin Therefore it is advised for Users without any background in Eclipse to read the online documentation namely the Workbench User Guide which is available at http help eclipse org luna index jsp The Test environment displays the newly created Tests and shows only the indispensable menus and options which allows for a simple use when the purpose is to create new test procedures Page 7 of 29 Copyright 2015 Smart DCC Ltd All Rights reserved DCC Public 4 GFI Testing Tool GUI Operation The Eclipse test environment includes a navigation pane Explorer where tests and campaigns are listed two output panes for the console buffer and the generated log with filtering capabilities and working area for test case editing and report analysis File Edit New Test Edit View Pane Execution Log Pane Stop Test Execution Explorer Pane GJ GFI Console Running Test Project dfsd GFI Console Pane Figure 3 GFI Testing Tool GUI The following sections will detail each of these components 4 1 Running Eclipse To start the framework GUI use the shortcut for the application on the desktop On start up the tests and campaigns created pr
14. CC Public 1 Introduction 1 1 Objective This document is a Graphical User Interface Manual for the GFI testing tool intended to provide detailed technical information for standard and advanced user operation and configuration 1 2 Document structure Section 1 Introduction presents a general description of this document s contents Section 2 Overview presents a general description of the framework Section 3 Environment Start up presents a description of the environment start up Section 4 GFI Testing Tool GUI Operation presents a detailed description of the Test creation execution and analysis process using the graphical user interface Section 5 Glossary presents the definitions and acronyms used throughout the document Page 2 of 29 Copyright 2015 Smart DCC Ltd All Rights reserved DCC Public 2 Overview GIT for Industry GFI is a software tool provided by Smart DCC for anybody that wishes to check whether their interpretation of the Great Britain Specification Companion for smart meters GBCS is consistent with Smart DCC s At the time of writing GFI supports all Use Cases for GBCS v0 8 1 over a ZigBee HAN In addition to the library of Use Cases GFI allows end users to create new or extend existing GBCS Use Cases This manual provides detailed technical information for advanced operation of the tool 2 1 High Level Architecture GFI is a testing tool and systems validation competence centre In the core
15. Data a Communications Company Graphical User Interface Manual GIT for Industry Smart DCC Ltd v1 0 Draft 2015 06 29 The contents of this document are under copyright of Smart DCC Ltd released on condition that it shall not be copied in whole in part or otherwise reproduced whether by photographic or any other method and therefore shall not be divulged to any person other than the addressee save to other authorized offices of his organization having need to know such contents for the purpose for which disclosure is made without prior written consent of submitting company Table of Contents 4 1 1 1 2 2 2 1 2 2 2 3 3 3 1 3 2 3 3 4 4 1 4 2 4 3 5 OCON aE E E E A ee te ee ere ee 2 ele EE 2 DOCUM ONT MUCU a E E E E 2 TEE 3 High Level Architecture E 3 Ria e FO ee an E E E E 4 mpu and Ke re 5 Environment Start up cccccceecccscecceeeecencecsseeecececeaeeeseuseseeeeseueeceaeeeseucesesesseeessusessueeeseneesseseeseas 7 A0066 HAN NEtWOrk e ele te 7 EE 7 Run Eclipse with GFI plugin cccccccccceececeeeeeeeeceeeeee seca eeseeeseeeseeeesaeeseueeseeesseeeseueeseeesaneesaess 7 GP esting TOOG UFOS AON E 8 RUNNING Se let 8 Work Environment description ccccccceecccseeeceeeceuceceeeeceeceueecuecsueessueeseeessueesseeeseeesueeseeeseas 8 Se e Ee EI E aa EE EEEE 10 NOS EE 27 Page 1 of 29 Copyright 2015 Smart DCC Ltd All Rights reserved D
16. INT_NUMBER_ equal to i EXPORT MAN ACCESS RESPONSE TY a e a 13123456781111 00 00 00 385 CHECK or S equal to EE COMBINED MPANS E ECS40_RESPONSE PAYLOAD ACCESS R ESPONSE_BODY ACCESS_RESPONSE LI 00 00 00 385 CHECK ST_OF_D TA REQUEST_RESPONSES ME 1312345678333 equal to 1312345678333 TER_ADMINISTRATION_POINT_NUMBER EXPORT MPAN ECS40 RESPONSE PAYLOAD ACCESS H ESPONSE_ BODY ACCESS RESPONSE Ss 00 00 00 385 CHECK PECIFICATION REQUEST_RESULTS MET equal to ER_ADMINISTRATION_POINT_NUMBER_ COMBINED MPANS RESULT ECS40 RESPONSE PAYLOAD ACCESS H ESPONSE_ BODY ACCESS RESPONSE F 00 00 00 385 CHECK PECIFICATION REQUEST_RESULTS MET equal to ER_ADMINISTRATION_POINT_NUMBER_ EXPORT MPAN RESULT Overall Iteration Expected Results Matched 96 3 26 27 Result Figure 23 Test FAIL detail Changes need to be made so the check verifies the last MPAN value set in the equipment it should be the value set in the last iteration of the preceding update test case Again if the purpose is to check the value only one iteration is needed Either redundant iterations should be removed or all iterations should account for the parameter value and expected values in all iterations should be changed Figure 24 shows how the XML Properties file should look like after resource pool is changed to hold the correct expected values regarding the last test case ATG TC ECS40 2 Page 22 of 29 Copyright 2015 Smart DCC Ltd All Rights reserved DCC Public
17. L gt Console lt TXT gt Test Messages Scenario lt XML gt Specifications Test lt XML gt LOG lt CSV gt Specifications lt XML gt lt XLS gt Test Library GFI TESTING TOOL lt code gt Messages Database lt code gt ESME Emulator GSME Emulator SIMULATION CHF Emulator VSIS CORE CHF ZigBee Adapter VSIS COMMS Figure 1 GFI high level architecture Page 3 of 29 Copyright 2015 Smart DCC Ltd All Rights reserved DCC Public 2 1 1 VSIS Core This is the central element of the framework provided as a library This is where all the logic and functionalities of the testing tool are concentrated Generically this module is responsible for The creation of the test environment and all the necessary resources for the test execution The execution of the test Generating all the log information Generating all the execution reports 2 1 2 VSIS Comms This is a library responsible for handling the communications that are external to the application namely the routing of GBCS messages from the KRP to the devices and vice versa using the CHF ZigBee Adapter when the test is executed in a real scenario with non emulated Devices 2 1 3 Simulation The Simulation Manager is responsible for handling the routing of GBCS messages from the KRP to the Devices and vice versa when the test is executed in an emulated scenario with emulated Devices 2 1 4 GFI Testing Tool This is the highes
18. NGE Executing Run 1 of 1 MNGE 2 Testcase ATG TC ECS39a 1 Starting 00 00 00 277 Figure 16 Console activity When the execution is finished the execution log is presented in the Log pane and the report shown in the working area Also a new Campaign item regarding the Tests execution is added to the Explorer pane along with all the respective configuration and execution files Page 17 of 29 Copyright 2015 Smart DCC Ltd All Rights reserved DCC Public FF Resource B GFi Explorer d Properties ESME MPAN d Scenario ESME MPAN ATG TESTING TOOL TEST REPORT 5 P EH g bad file home gfiuser runtime EclipseApplication Campaign_ESME MPAN Run_2015 06 19 14 46 02 test re v gt v G i ESME MPAN L Properties ESME MPAN xn Bi Scenario ESME MPAN xmi TEST INFORMATION Tec ESME MPAN Test Name ESME Set and Check MPAN ampaign_ Test Id ESME MPAN TO Test Setting the MPAN on an ESME device and checking the parameter on the meter B Properties Campaign ES eschen The procedure is repeated two times Execution Date 2015 06 19 14 46 01 WEST Overall Result PASS ATG TESTING TOOL TEST REPORT D Scenario Campaign_ESh v Run_2015 06 19 14 46 02 A execution log csv TEST CASES RESULTS Test Case Result ATG TC ECS39a 1 PASS ATG TC ECS40 1 PASS ATG TC ECS39a 2 PASS ATG TC ECS40 2 PASS TEST CASES EXECUTION ATG
19. class CResourcePoolString name MeterAdministrationPointNumber combinedMPANS result gt lt value gt lt value gt lt value gt lt value gt Figure 24 Change expected value Again the file should be saved and the Test executed This time the result is PASS as the expected value matches the response message field read Meaning that the parameter was updated successfully ie i f Resource H GA Explorer BS se gp v amp ESME MPAN x Properties ESME MPAN xm x Scenario ESME MPAN vm ATG TESTING TOOL TEST REPORT 33 ATG TESTING TOOL TEST REPORT X a G Properties ESME MPAN xml 8 5 file mome gfuser runtime EclipseApplication Campaign_ESME MPAN_002 Run_2015 gt BH 8 file homeigfiuserjruntime EclipseApplication Campaign_ESME MPAN_001 Run_2015 gt H G Scenario_ESME MPAN xml v Campaign_ESME MPAN ATG TESTING TOOL TEST REPORT TEST INFORMATION TEST INFORMATION ATG TESTING TOOL TEST REPORT gt amp Test ESME Set and Check MPAN Test Name ESME Set and Check MPAN gt gt Run_2015 06 19 15 18 27 Test id Ee Test Id ESME MPAN z z Setting the MPAN on an ESME device and checking the 7 a Ee Test Purpose parameter on the esak Saena eene en ES Test Purpose parameter on the meter The procedure is aaa two of v t Campaign ESME MPAN_001 Saari ane E 66 10 160000 REST Execution Date 2015 06 19 16 22 57 WEST
20. configurations need to be properly set Page 11 of 29 Copyright 2015 Smart DCC Ltd All Rights reserved DCC Public Scenario selection Define Test Scenario properties Scenario Physical Meters All fields required Active Meter ESME Electric Smart Meter O GSME Gas Smart Meter Id Installation Credentials Digital Signature Certificate Key Agreement Certificate Pan Id Extend Pan Id Network Key Port Join TimeOut Browse Browse dev ttyUSBO 60 seconds Restore Defaults Figure 9 Scenario selection physical meters ID Entity Identifier of the Device in the Whitelist of the CHF Installation Credentials Installation credentials installation code of the Device in the Whitelist of the CHF Digital Signature Certificate Path to the der file containing the device certificate Key Agreement Certificate Path to the der file containing the device certificate MAC Entity ID matching ZigBee EUI64 address as in GBCS documentation Pan ID The ZigBee network Pan ID Extended Pan ID The ZigBee network extended Pan ID Page 12 of 29 Copyright 2015 Smart DCC Ltd All Rights reserved DCC Public Network Key The ZigBee network key Port The serial port where the ZigBee Adapter is connected Join Timeout The join timeout specified in seconds On the first use this dialog s fields are shown blank as in
21. e EclipseApplication Campaign_ESME MPAN_001 Run_2015 06 19 16 10 04 test report htm l ATG TC ECS39a 1 Figure 25 Test reports side by side Page 23 of 29 Copyright 2015 Smart DCC Ltd All Rights reserved DCC Public As shown in Figure 25 a new Campaign is added for every Test run The Test had to be run not the Campaign as the test cases in the sequence were subject changes not allowed in the Campaign Also the reports may be viewed side by side for result analysis in a comparative fashion This is allowed for every input or output artefact console output or execution logs Logging Further analysis may be performed in the test Log using filters in some columns or just by text The icon on the upper right corner of the Log pane will display a Filter dialog Log GO GI 06 19 16 22 58 Start 2015 06 19 16 22 57 WEST Stop 2015 06 19 16 22 58 WEST Area Level Message 1 00 208 MNGE 6 Created Equipment SimulationManager Op 2081 MNGE 6 Created Equipment ACBEmulator 00 216 MNGE 6 Created Equipment RPEmulator 00 216 TDRIV 6 i Setting TestDrive context where Sets and Sends will be performed 4 1 00 216 TDRIV 6 Setting TestDrive context where Gets and Checks will be performed 3 100 223 MNGE 6 Created Equipment ESMEEmulator 00 223 TDRIV 6 Setting TestDrive context where Sets and Sends will be performed 4 1 00 223 TDRIV 6 Setting TestDrive context where Gets and Checks will be performed 3
22. e class CTestCaseECS40 name ATG TC ECS40 2 gt a lt report gt f lt item name TEST CASE ID gt ATG TC ECS40 lt item gt B al Design Source Figure 21 Edit Test Properties after The file should then be saved and the Test executed After the new execution the Test will fail because the subsequent read test case AT G TC ECS40 2 is still expecting to find the default values Page 21 of 29 Copyright 2015 Smart DCC Ltd All Rights reserved DCC Public ATG TESTING TOOL TEST REPORT TEST INFORMATION ee Test Testid Setting the MPAN on an ESME device and ee device EE parameter ov the weter The the parameter on the meter The Se procedure is repeated two times Execution Date 2015 06 19 16 10 04 WEST Overall Result FAIL TEST CASES RESULTS Result esoe eme ATG TC ECS40 1 w PASS ATG TC ECS39a 2 w PASS ATG TC ECS40 2 TEST CASES EXECUTION ATG TC ECS39a 1 Test Case Id ATG TC ECS39a The purpose of this test case is to validate Use Case ECS39a Set MPAN MPRN Value by validating the Test Case Purpose construction of the command used by the Supplier for setting the Import MPAN Value configuration on the ESME and its respective response Test Case Assumptions And N A Constraints CSW Figure 22 Test FAIL 11re ECS40_RESPONSE PAYLOAD ACCESS R ESPONSE BODY ACCESS RESPONSE 5 SCH PECIFICATION REQUEST_RESULTS MET 00 00 00 385 CHECK ER_ADMINISTRATION_P
23. ed throughout this document Applicable Document Reference Document Test Test Case VSIS A document is considered applicable if tt complements this document All its content is directly applied as if it was stated as an annex of this document A document is considered a reference if it is referred but not applicable to this document Reference documents are mainly used to provide further reading A set of Test Cases ranging from one to N that are grouped together with the purpose of being executed in one run A Use Case as defined by SMETS A CSW Critical Systems Validation Platform Table 2 Definitions Table 3 presents the list of acronyms used throughout this document ACB AD ATG CBKE CHF CSW DUT ESME GBCS Access Control Broker Applicable Document Automated Test of GBCS Certificate Based Key Exchange Communications Hub Function Critical Software S A Device Under Test Electricity Smart Metering Equipment Great Britain Companion Specification Page 27 of 29 Copyright 2015 Smart DCC Ltd All Rights reserved DCC Public GFI GIT For Industry GSME Gas Smart Metering Equipment NA Not Applicable RD Reference Document TBD To be defined ZCL ZigBee Cluster Library Table 3 Acronyms Page 28 of 29 Copyright 2015 Smart DCC Ltd All Rights reserved DCC Public
24. ed will be shown in the console pane In the same way the execution log will be available for analysis in the Log pane See Section 4 3 3 GJ GFI Console choo mv P P Running Test Project dfsd Figure 5 Console 4 2 3 Edit View Pane On the working area it is possible to edit the properties of a test and of a scenario through the XML files generated by the test wizard Also the execution output reports and the log files may be viewed in this area for detailed analysis Page 9 of 29 Copyright 2015 Smart DCC Ltd All Rights reserved DCC Public 4 3 Step by step Example In this section a Step by step example will be presented to illustrate the testing tool operation This is a sequence to be run in an ESME device in which the Meter Point Administration Number is altered between different values and checked in the meter after every change 4 3 1 Test Creation Wizard Starting from a clean state the first thing to do is to create a New Test This may be done by invoking the New Test wizard from the context menu on the Explorer from the File menu or from the respective button on the toolbar then filling in the required options and configurations Zi Explorer amp Refresh Gl GFI Console Running Test Project dfsd Figure 6 New Test Properties This dialog gathers the information required for the test Properties the sequence of use cases needed to verify a given functionality From this inputs the
25. elevant actions executed in the Test are outputted to the console The detail level of this execution output is maximum by default 2 3 5 Execution Log As for the console output the tool also produces an execution log with a configurable level of detail defined in the Scenario Configuration file lt scenario file gt xml The higher the value chosen for the detail level the more information will be recorded in the log It should always be the highest possible value default for the sake of record keeping Filters may be applied in the Log view pane for detailed analysis Page 6 of 29 Copyright 2015 Smart DCC Ltd All Rights reserved DCC Public 3 Environment Start up The contents of this user manual are based on the following assumptions The user has access to a machine with the GFI testing tool installed and working The user has access to the system console and permissions to fully operate in the test environment workspace 3 1 Zigbee HAN network connection To allow the communications between the testing tool and the metering devices a HAN network needs be setup For this purpose a ZigBee USB stick should be connected to a USB 2 0 port The purpose of this network is described in Section 2 2 The configurations of this network connection should be defined in the Scenario configuration as described in Section 4 3 1 3 2 System Start up The operating system setup is a Linux Ubuntu distribution with a default user This
26. eviously will be shown 4 2 Work Environment description 4 2 1 Explorer The Explorer shows the available Tests and Campaigns In the context of the GFI GUI a Test is a procedure enclosing a sequence of test cases and a Campaign is a set of executions of a specified Test A Campaign is created every time a Test is run Test A Test is composed of two files the Properties file which contains the information of all test cases in the sequence configurations and input output parameters as well as items that will be featured in the test report Such as the Test Id Test Name and Test Purpose and the Scenario file in which the Page 8 of 29 Copyright 2015 Smart DCC Ltd All Rights reserved DCC Public various configurations needed to run the tool are set up like the output console and log levels schedulers remote parties and metering devices emulators SMETS object storage information and message codecs Campaign A Campaign contains a test section and a set of Runs The test section contains a copy of the test files at the date and time of the first Run i e when the Campaign was created These files are a snapshot of the original test and are hence not editable Future runs of this Campaign will use these same test files Each Run contains the execution Report and Log files for analysis y Explorer G amp v P Figure 4 Explorer 4 2 2 Console Log Output Upon a given test execution the console fe
27. file referred to as lt execution file gt xml will be generated 4 3 1 1 1 ID This is the identifier that will be used for the execution file name and shown on the Explorer pane 4 3 1 1 2 Name A user defined name typically a short but meaningful designation 4 3 1 1 3 Description The detail used to include in the test purpose section of the test report see example in Figure 7 Page 10 of 29 Copyright 2015 Smart DCC Ltd All Rights reserved DCC Public New Test properties G e Create a new Test and define Test properties A All fields required identifier ESME MPAN Test Name ESME Set and Check MPAN Description Setting the MPAN on an ESME device and checking the parameter on the meter The procedure is repeated two times Figure 7 New Test Properties Scenario This dialog gathers the information required for the test scenario all the configurations for the networking outputs scheduling equipment etc From these inputs the file referred to as lt scenario file gt xml will be generated lf the test uses meter emulation only the type of meter is available for selection Scenario selection G Define Test Scenario properties A Scenario Emulators only e Active Meter ESME Electric Smart Meter O GSME Gas Smart Meter Cancel Figure 8 Scenario selection emulators lf an actual metering device is used the meter
28. gt amp Test Overall Result FAIL E Overall Result PASS Y Run_2015 06 19 16 10 04 teas eee TEST CASES RESULTS A execution log csv Test Case Result Test Case Result D test report htm ATG TC ECS39a 1 wv PASS ee ng v Campaign ESME MPAN_002 ATGTCEGA0 1 wv PASS SE ams ATG TC ECS39a 2 w PASS ATG TCECS39a 2 w PASS ee ATG TC ECS40 2 FAL EE WEE gt Run_2015 06 19 16 22 58 TEST CASES EXECUTION ATG TC ECS39a 1 TEST CASES EXECUTION ATG TC ECS39a 1 Test Case Id See oF his t st lidat Case ECS39a Set MPAN The purpose of this test case is to validate Use Case ECS39a Set MPAN e purpose of this test case 5 to validate Use Case e I P ae MPRN Value by validating the construction of the command used by the MPRN Value by validating the construction of the command used by the Test Case Purpose A A e e Test Case Purpose Supplier for setting the import MPAN Value configuration on the ESME and Supplier for setting the Import MPAN Value configuration on the ESME and its respective response 2 its respective response z Test Case Assumptions And Test Case Assumptions And N A Cii N A Constraints Author CSW sel csw Version 01 Version 01 GFI Console amp Log amp Console 2 RE r e BR e e Running Test Project ESME MPAN Complete VSIS SVF Execution CG All clean up exiting file home gfiuser runtim
29. licated test cases will be added to the end of the list if needed they should then be moved to their correct order in the sequence Page 14 of 29 Copyright 2015 Smart DCC Ltd All Rights reserved DCC Public Search and Select Test Cases G E Search and Select Test cases to add to the Test A je x A Sequence Test Case ID Purpose ATG TC ECS39a Set MPAN Value on the ESME _ ATG TC ECS40 Read MPAN Value on the ESME ATG TC ECS39a Set MPAN Value on the ESME ATG TC ECS40 Read MPAN Value on the ESME Figure 12 New Test populated sequence Clicking Finish in this dialog will create the new test which will then be listed in the GUI main window s Explorer Properties and Scenario XML files may be viewed and edited in the working area should any changes be required for a specific sequence e g expected values on the output parameters An example of this is given in Section 4 3 3 File Edit Search Run Design Window BR A E Resource H GFI Explorer B G amp G e D X Properties ESME MPAN xml X x P v ei ESME MPAN lt xml version 1 0 encoding UTF 8 gt E E gt lt executionconfiguration gt W Properties_ESME MPAN xml lt testmanager master _node svfmaster gt BD Scenario _ESME MPAN xml lt testspec class CTestSpec name ESME Set and Check MPAN gt R lt report gt lt item name TEST NAME gt ESME Set and Check MPAN lt item gt lt item name TEST ID gt ESME MPAN lt item gt lt item name
30. ng name MeterAdministrationPointNumber exportMPAN result gt lt value gt 6 lt value gt lt value gt lt value gt lt resourcepool gt lt testcase gt lt testcase class CTestCaseFCS39a name ATG TC ECS39a 2 gt lt report gt lt item name TEST CASE ID gt ATG TC ECS39a lt item gt lt item name TEST CASE PURPOSE gt The purpose of this test case is to validate Use Case ECS39a Set MPA lt item name TEST CASE ASSUMPTIONS AND CONSTRAINTS N A lt item gt lt item name AUTHOR gt C5W lt item gt lt item name VERSION gt 01 lt item gt lt report gt lt resource combiner class CLinearCombinations name linearCombinator gt lt resourcepool class CResourcePoolString name BusinessOriginatorName gt lt value gt SuppLlierl lt value gt lt value gt SuppLlierl lt value gt lt resourcepool gt lt resourcepool class CResourcePoolString name BusinessTargetName gt lt Va Lue gt ESME1 lt vaLlue gt lt Va Lue gt ESME1 lt vaLlue gt lt resourcepool gt lt resourcepool class CResourcePoolString name MeterPointAdministrationNumber combinedMPANS value gt lt Value gt 1234 lt value gt lt va Lue gt iigp lt value gt lt resourcepool gt lt resourcepool class CResourcePoolString name MeterPointAdministrationNumber combinedMPANS result gt lt value gt lt value gt lt va Lue gt lt value gt lt resourcepool gt lt testcase gt lt testcas
31. ns are ALWAYS executed based on the same test conditions Properties and Scenario Campaign Tests are not editable only standalone Tests may be changed Figure 18 depicts multiple executions of a Campaign Page 18 of 29 Copyright 2015 Smart DCC Ltd All Rights reserved DCC Public Explorer B 7 m Properties _ ESME X Scen 7 GI ESME MPAN L Properties ESME MPAN xml Bi Scenario ESME MPAN xm Les Campaign ESME MPAN TEST INFORMATION Test Name Test Id amp file home gfiuss k Test b Run_2015 06 19 15 18 27 Run_2015 06 19 15 18 04 A execution log csv L test report html Run _2015 06 19 14 46 02 A execution log csv L test report html Test Purpose Execution Date Overall Result TEST CASES RESULTS Test Case ATG TC ECS39a 1 ATG TC ECS40 1 ATG TC ECS39a 2 Figure 18 Multiple Runs 4 3 3 Results Analysis In the previous section the Test was executed without any changes to the original test cases in the library so the result was PASS for all of them the default values were previously tested and known to yield pass results Ki Va B B Co Resource H GA Explorer Se p Exp 3 X Properties ESME MPAM vm 2 ATG TESTING TOOL TEST REPORT amp v ESME MPAN e lt resourcepool gt ei file home gfiuser runtime EclipseApplication Campaign ES e Pa gt lt resourcepool class CResourcePoolString name MeterAdministrationPointNumber
32. source combiner class CLinearCombinations name lLinearCombinator gt lt resourcepool class CResourcePoolString name BusinessOriginatorName gt lt value gt SuppLlierl lt value gt lt value gt SuppLlierl lt value gt lt resourcepool gt lt resourcepool class CResourcePoolString name BusinessTargetName gt lt Va Lue gt ESME1 lt vaLlue gt lt Va Lue gt ESME1 lt vaLlue gt lt resourcepool gt lt resourcepool class CResourcePoolString name MeterPointAdministrationNumber combinedMPANS value gt lt Va Lue gt 1312345678333 lt value gt lt va Lue gt EFFELE eORESPEL ELIE yeey lt value gt lt resourcepool gt lt resourcepool class CResourcePoolString name MeterPointAdministrationNumber combinedMPANS result gt lt VaLue gt 0 lt value gt lt VaLue gt 0 lt value gt lt resourcepool gt lt testcase gt lt testcase class CTestCaseECS40 name ATG TC ECS40 2 gt lt report gt lt item name TEST CASE ID gt ATG TC ECS40 lt item gt lt item name TEST CASE PURPOSE gt The purpose of this test case is to validate Use Case EC540 Read MPAN Figure 20 Edit Test Properties before After assigning different values to that resource pool the XML should look like Figure 21 Page 20 of 29 Copyright 2015 Smart DCC Ltd All Rights reserved DCC Public x Properties ESME MPAN xml x Scenario ESME MPAN xm lt resourcepool gt lt resourcepool class CResourcePoolStri
33. sting tool only performs conformance tests against the GBCS v0 8 1 protocol not functional tests Although some minimal functional tests can be implemented and are in fact supported that is not the purpose of the tool Page 4 of 29 Copyright 2015 Smart DCC Ltd All Rights reserved DCC Public Testbed e e HAN Figure 2 GFI Testbed diagram 2 3 Inputs and Outputs Table 1 presents the inputs and outputs produced by the testing tool A short description for each artefact is also presented lt scenario file gt xml lt execution file gt xml lt test report gt html lt execution log gt csv Input Configuration file Input Test execution file Output Test report file Output Execution log file The scenario configuration file required by the tool Specifies which equipment to use the device configurations keys and certificates among other configurations The execution file required by the tool Specifies which Test Cases should be executed and the inputs for each Test Case The final execution report produced by the tool in HTML format Highlights all the relevant actions executed in the test as well as the results of each Test Case Test Case iteration and Test Case step The full execution output in comma separated values format Contains the full detail available from the tool s execution Every single action set check send wait performed by the tool is recorded in this file
34. t layer of the test framework responsible for establishing the connections between Core Comms Simulation Manager and the Test Library This tool may be invoked either by the Graphical User Interface which generates the command to be executed or directly by the command line 2 1 5 Test Library This is the library containing all the Use Cases specified by GBCS and available for the Tester to use in the construction of Tests 2 1 6 Messages Database This is a database containing all the messages that provide support for the execution of the Use Cases For instance this database contains the General Ciphering Message and General Signing Message as well as all the other messages specified by GBCS 2 1 7 GFI Testing Tool GUI Provides a Graphical User Interface for the Testing Tool that allows for all the editing execution and analysis functionality in the user implemented Test procedures 2 2 Reference Testbed The GFI framework operates as a home area network HAN and emulates Remote Parties ACB and CHF It creates a ZigBee network and allows Devices to join Through this network it communicates with Devices in order to execute Tests In these communications messages are exchanged using the GBCS protocol commands are sent and the relevant Devices responses and alerts are gathered to produce the Test Reports and to verify the Devices conformance with the protocol The reference Testbed is presented in Figure 2 Note The GFI te
35. through the command line Page 16 of 29 Copyright 2015 Smart DCC Ltd All Rights reserved DCC Public LG GFI Console M Log Console Running Test Project ESME MPAN Found 12 argument Searching for file home gfiuser runtime EclipseApplication MPAN Scenario MPAN xm1 Found 22 argument Searching for file home gfiuser runtime EclipseApplication MPAN Properties MPAN xm1 Initializing the Software Database SDB Starting VSIS SVF Execution TIME AREA LV MESSAGE 00 00 00 243 MNGE 0 Preparing Test Execution 00 00 00 244 RPORT 4 Report location home gfiuser gui tmp test report html 00 00 00 258 ACB 4 ACBEmulator Start processing thread for messages received from the KRP 00 00 00 258 ACB 4 ACBEmulator Waiting for new messages from KRP 00 00 00 258 ACB 4 ACBEmulator Start processing thread for messages received from xSME 00 00 00 258 ACB 4 ACBEmuLlator Waiting for new messages from xSME 00 00 00 259 EQUIP 4 CHFEmulator Start processing thread 00 00 00 259 EQUIP 4 CHFEmulator Waiting for new messages 00 00 00 275 EQUIP 4 GSMEEmulator Start processing thread 00 00 00 275 EQUIP 4 GSMEEmuLlator Waiting for new messages 00 00 00 276 EQUIP 4 ESMEEmulator Start processing thread 00 00 00 276 EQUIP 4 ESMEEmulator Waiting for new messages 00 00 00 277 MNGE 0 Starting Test Execution 00 00 00 277 MNGE 0 Starting Run 1 of 1 00 00 00 277 M
36. to validate Use Case ECS39a Set MPAN MPRN Value by validating the construction of the command used by the Supplier for setting the Import MPAN Value configuration on the ESME and its respective response Assumptions and Constraints N A Parameters Name BusinessOriginatorName BusinessTargetName MeterPointAdministrationNumber combinedMPANS value MeterPointAdministrationNumber combinedMPANS result Use Case ID Name Description Message Code Protocol Remote Party Roles Message Type ECS39a Set MPAN Value on the ESME This Use Case is for setting 0x004C DLMS Supplier Command and Response Message Type Category SME C NC Read Update Update Target Devices ESME Capable of Future Evocation No gt ane bn han zm Amaiak Pominss Bn allan alle Ae e Figure 11 Add Test Case After all the required test cases are added the sequence in the Search and Select Test Cases dialog is populated At this point tools are available to tune the Test procedure On the upper right corner the arrow icon buttons Move Up and Move Down allow a selected test case to be moved up or down the sequence On the upper left corner next to the Add Test Cases button lay the Delete and the Duplicate buttons As their names imply they allow the possibility to remove or duplicate one or more test cases as multiple selection is permitted for these functions Dup

GFI user manual - Data Communications Company

Contents

Download Pdf Manuals

Related Search

Related Contents