New Eagle Automated Testing User Manual
Contents
1. Figure 14 Log Dialog NEAT Log If there is a MATLAB build error then consult the MATLAB Log to see the build error i lt lt Session Log NEAT Log MATLAB Log Real Time Workshop beginning target language compiler Checking for I O Conflicts Invoking Target Language Compiler on DataStorage 7000 Using System Target File C DEV_AREA Raptor Source build rap Loading TLC function libraries Initial pass through model to cache user defined code THHHHHHPHHEBHHHHHHEHBHHEHHHRHHHRHHHRHHHEHHEE Error During Raptor Build THHHHHHHHHHHHHHHHHBHHHHHHHHHHEHEH E E S Error Simulink Coder Error Block lt S7 gt Analog I0 raptor_digita Figure 15 Log Dialog MATLAB log The log from each test is viewable by clicking on the log link next to the test 5 Command Line Interface When NEAT is invoked from the command line there are various command line arguments to control its behavior This mode is useful for automating test execution NEAT is found in one of two places depending on the host OS For 32 bit Windows C Program FilesNNewEagleNNEATNNEAT exe For 64 bit Windows C Program Files x86 NNewEagleNNEATNNEAT New Eagle Consulting 3588 Plymouth Road Box 274 Ann Arbor MI 48105 2603 877 234 1410 FAX 928 395 2114 New Eagle Confidential New Eagle Option ea ce ee Ero m mmee 000000 Fr meme oo2. Test Id Test 001 Desc Verify Empty DM1 Response lt 1 Setup Phase gt lt InitialConditionMacro Name LampsAreClear gt Macro Name DisablePeriodicMessages gt lt Macro Name ClearFaults gt lt 1 Trigger Phase gt New Eagle Consulting 3588 Plymouth Road Box 274 Ann Arbor MI 48105 2603 PH 877 234 1410 FAX 928 395 2114 New Eagle Confidential N d 5 Ca Kt New Eagle lt CANTX Id 0x18EAFDF9 Ext true Data CA FE 00 lt 1 Verify Phase gt lt VerifyCANRx Id x18FECAFD Ext true Data 00 FF 00 00 00 00 FF FF VerifyTime 1250 gt lt 1 Clean Up Phase gt lt CleanUp gt Macro Name ClearFaults gt lt Macro Name EnablePeriodicMessages gt lt CleanUp gt lt Test gt 9 2 Test Writing Best Practices These are guidelines for making tests robust reliable and useful Each test should be as narrowly written as possible Test just one piece of functionality It is common to have several tests that share common setup and clean up code and have just a couple of differences in the test actions Each test should be independent Do not rely on the behavior of other tests as NEAT does not guarantee execution order or that other tests will be executed at all Use InitialCondition actions to verify that the module is in an expected state before running the test This will help prevent false results as a side effect of other tests Every value that is
3. 6 1 3 XCP Slave Configuration The XcpConf ig element sets the XCP request id for the module under test All requests to read or write values from the module are sent to this id It is important that each ECU on the CAN bus have a different request id but the same broadcast id For example New Eagle Consulting 3588 Plymouth Road Box 274 Ann Arbor MI 48105 2603 PH 877 234 1410 FAX 928 395 2114 New Eagle Confidential m New Eagle Request Id Broadcast Id Module Under Test 0x100 0 200 Coordinator Module 0x101 0x200 Figure 17 Sample XCP Configuration The XCP broadcast id is an application setting which can be configured with the NEAT Configuration tool 6 1 4 ERI Configuration The EriConfig element sets the source tool city id destination ECU city id and access level to use for communicating with the ECU It is advisable to use a source id other than 0x02 so as to not interfere with MotoTune If the test contains both a module under test and coordinator the destination city ids should be different but the source city ids should be the same ECU Source City Id Destination City Id Figure 18 Sample ERI Configuration 6 1 5 CAN Configuration The CanConfig element configures the CAN connection Set the CAN channel and baud rate for use with this ECU NEAT currently only supports one configuration for both ECUs so any CAN configuration set for the Test Target takes precedence over a
4. Short PDF report ignored if p is omitted 1 N Logging level defaults to 0 0 Normal logging 1 Trace logging 2 Trace and Communications logging At least one of r or p must be specified Figure 16 Command Line Options Examples Generating an XML report NEAT exe t C tests test xml r C results test_results xml Generating a PDF report NEAT exe t C tests test xml p C results test_ results pdf Running with full logging NEAT exe t C tests test xml r C results test_results xml 12 6 Test Scripts NEAT test scripts are XML files that conform to the NEATTestScript or NEATTestScriptV2 schema The XSD files for these schemas can be found installed with NEAT A test script consists of several sections in order Test Target configuration build flash and connection settings optional Test Coordinator configuration build flash and connection settings Included script files Test run reference values Macro definitions Test definitions 6 1 Test Target and Coordinator Settings Each of the ECUs can be configured separately including building the model flashing the ECU and connection information XCP or ERI The coordinator ECU is optional For both New Eagle Consulting 3588 Plymouth Road Box 274 Ann Arbor MI 48105 2603 PH 877 234 1410 FAX 928 395 2114 New Eagle Confidential C AN w ECUS the and flash settings are optional The two ECUs do not have to
5. AG WO NS rac eaaa 20 x ME o cc 20 Ta VERY e 21 CU pc 21 TA MBRIEYCANIUX 22 7 9 22 76 CANSTARTRX 22 747 INTIAL CONDITION 23 79 23 ESSE uu 23 LE o vU HH 9 23 ALT LOGMPSSAGE Yn RO RERO VOD COD 23 MOU O 24 7 43 PAUSE v 24 e 004 RESULTS cc 24 TEST STATUS 24 sie dU EN Oro uec 25 ES POr REPOR Dc S X 26 Eo dul c 26 9 1 ANATOMY OF A DEST no
6. CAN configuration set for the Coordinator Future releases may allow different CAN configurations for the two ECUS e g connect to the Test Target over CAN1 and the Coordinator over CAN2 If no CAN configuration is included then the default values of CAN1 at 250 000 baud will be used 6 2 Reference Values There can be as a many ReferenceValue elements as is desired Each is a value that will be read from the module under test at the start of the test run and reported in the XML and PDF reports 6 3 Included Script Files Use the Include element to import the contents of other script files Each file is included only once even if it is referenced multiple times and can include additional scripts through the use of the Include element Everything in an included script is imported except the New Eagle Consulting 3588 Plymouth Road Box 274 Ann Arbor MI 48105 2603 877 234 1410 FAX 928 395 2114 New Eagle Confidential N PF 5 Ca Kt New Eagle Model and Flash elements which are ignored in all scripts except the original script Macros test groups etc all must have unique names or ids across all scripts included 6 4 Macro Definitions There are two types of macros normal macros and initial condition macros Both are named sets of actions that are run in sequence with the other actions in a test Each type of macro can contain references to other macros of the same type These actions are executed i
7. Dialog Box If you have lots of macros you can enter part of a macro s name in the search box to narrow down the listed macros InitCond Macro Assert Steady State I AxleRatio HighGear lt 10 AxleRatio LowGear lt 6 4 v Setup Test Figure 10 Macro Actions Both types of macro actions have an arrow that you can click to see the actions in the macro You cannot modify the actions in a macro directly in a test In order to edit a macro select it from the Test Navigator changes made a macro will be reflected wherever it is called New Eagle Consulting 3588 Plymouth Road Box 274 Ann Arbor MI 48105 2603 PH 877 234 1410 FAX 928 395 2114 New Eagle Confidential am 9mm I T m ms C New Eagle 4 4 4 Other Toolbox Actions There are several other actions that are useful for composing tests though they do not interact with the ECU directly This test is not done yet Log Message This appears in the test log when run Figure 11 Utility Actions The To Do action is always placed at the start of the test and will always fail It is intended to flag tests in progress and prevent one or more of the ECUs being put into an unknown state which could cause other tests to fail unexpectedly The Log Message action always passes It simply inserts a message into the test run log This is useful for explaining complicated behavior or non obvious expectations The Pause
8. Executes a set of actions a specified number of times This action contains other actions The entire sequence is executed the specified number of times If any of the actions fail during any iteration then the Repeat action fails Example lt Repeat Iterations 3 gt lt Macro Name CompleteDriveCycle gt lt Repeat gt 7 11 LogMessage Enters a message into the log This is useful for debugging complicated tests The LogMessage action does not support any variables or parameters it just writes a fixed text New Eagle Consulting 3588 Plymouth Road Box 4274 Arbor MI 48105 2603 877 234 1410 FAX 928 395 2114 New Eagle Confidential ES a d gt Ca Kt New Eagle message into the log The log message always shows up in the log regardless of what log detail is specified Example lt LogMessage gt Faults should be clear by now lt LogMessage gt 7 12 ToDo Enters a message in the log and sets the test status to NotRun This must be the first action in a test A test with this action is not run but is not failed This is useful for test driven development where tests have been written for functionality that is not yet implemented The test run will not have a Pass status if everything else passes but it will not fail Example Test Id Test 001 Desc Verify Empty DM1 Response gt lt ToDo Message Waiting on implementation of lamp status gt lt Test gt 7 13 Pause Pauses test
9. action always passes pausing test execution for the specified number of milliseconds Note NEAT does not guarantee specific execution time so the actual delay be 50 ms or more depending on the host system If you need to measure precise timing a better method is to implement that logic in the coordinator module and read the results from NEAT 4 5 Test Runner To run multiple tests switch from the editor view to the Test Runner by clicking on either the Run One gt 11 buttons Test Navigator toolbar If the test script contains model building or flashing setting then the Build Model and Flash Target ECU sections in the sidebar will contain the relevant information Unchecking either box will cause NEAT to skip that step when running the tests Note that if the Build Model section is enabled the Flash Target ECU section cannot be disabled If a model is built then the output of the build is scanned to determine which file to use to flash the ECU The file that is explicitly listed in the flash section is ignored though the rest of the flash settings are used Test groups and individual tests can be toggled on or off for an individual test run by checking or unchecking the box next to the test or group To run the selected tests click the Run Test button in the sidebar New Eagle Consulting 3588 Plymouth Road Box 274 Ann Arbor MI 48105 2603 PH 877 234 1410 FAX 928 395 2114 New
10. compares it to an expected value In Raptor this is done the XCP protocol and the value must be set up as an Adjustment or Measurement in the model In MotoHawk this is done via the ERI protocol and the value must be set up as a calibration or probe in the model with an appropriate access level Options e Target Indicates which module the value should be read from the module under test or the coordinator module If no target is specified then the value will be read from the module under test e VerifyTime Indicates how long in milliseconds NEAT should wait for the value in the ECU to match the expected value NEAT will poll the value approximately every 50 ms If at the end of this time the value in the ECU does not match the expected value then the test will fail If this option is not specified NEAT will use the default value specified on the root NEATScript element e Delta Indicates how close a floating point value can be to the expected value and be considered equal This is only applied to floating point values integer and string values must be an exact match If this option is not specified NEAT will use the default value specified on the root NEATScript element Examples lt Verify Measurement 0ilPressure Value 12 gt lt Verify Measurement 0ilPressure Value 12 VerifyTime 200 gt Verify Measurement 0ilPressure Value 12 4 Delta 0 01 Verify Measurement Target DigOutput Value 1 Target C
11. there is no coordinator module these labels are not displayed as all signals are on the test target New Eagle Consulting 3588 Plymouth Road Box 274 Ann Arbor MI 48105 2603 PH 877 234 1410 FAX 928 395 2114 New Eagle Confidential N N ms C New Eagle 4 4 2 Initial Condition actions have an operation selector where you can choose lt gt and not equal and an expected value Set actions have a Verify check box if checked NEAT will perform a read after the write to verify that the value was written properly Verify actions have an operation selector where you can choose lt gt and not equal and an expected value Verify actions have a 9 check box which optionally shows the Verify Time for this action It is set to the default value specified when the script was created to start but can be changed as needed for a test When checked the verify time is displayed as a text box below the expected value with the label within Verify actions can have a check box when appropriate which optionally shows the Delta for the operation It is set to the default value specified when the script was created to start but can be changed as needed for a test When checked the delta can be seen below the expected value The signal referenced in Set Verify actions can be changed by double clicking the signal name CAN Actions NEAT supports sending and receiving raw CAN frames These frames
12. use same framework MotoHawk or Raptor 6 1 1 Model Building Configuration The RaptorModel and MotoHawkModel elements indicate which version of MATLAB and Raptor or MotoHawk to use to build the model file It also indicates which model file to build NOTE The Raptor MotoHawk version option is not currently supported The model will be built with whichever version of Raptor or MotoHawk is configured to run with the indicated version of MATLAB This section is optional If no model should be built for the tests simply omit the Model element entirely 6 1 2 Module Flash Configuration The VanscoF lash and MotProg elements indicate which VSF or SR file to use when flashing the module under test For Raptor projects use VanscoF lash for MotoHawk projects use MotProg Note that when a model is built by NEAT the MATLAB output is parsed to find the file to flash onto the ECU and any file specified in those elements is ignored For VanscoF Lash the target source address is set as an attribute This allows multiple modules to be on the same CAN bus as the module under test For MotProg only the source file is specified The connection settings will be taken from the EriConfig element This section is optional If no file should be flashed for the tests simply omit the element entirely However if a MotoHawkModel or RaptorModel element is supplied then the test script must also contain a corresponding MotProg or VanscoFlash element
13. 05 Desc Verify DM5 SPN1222 Id 006 Desc Verify DM5 SPN1223 lt Id Test_001 Desc Verify DM6 Empty Response X Test Run Complete Passed Status Passed Status Passed Status Passed Status Passed Status Passed Status Passed Status Passed Status Passed Status Passed log Status Passed log Status Passed log Status Passed log Status Passed lt lt log Status Passed log Status Passed log Status Passed log Status Passed log Status Passed log Status Passed log Status Passed log Status Passed log Status Passed Figure 13 NEAT Console after test run The full results can be exported in XML or PDF format with the Save XML and Save PDF buttons respectively New Eagle Consulting 3588 Plymouth Road 274 Ann Arbor MI 48105 2603 New Eagle Confidential PH 877 234 1410 FAX 928 395 2114 7 re P d New Eagle The Show Log button will display the session log This log does not contain any information about individual tests but will be useful for diagnosing things like CAN connection issues and MATLAB build errors NEATLog Session Log NEAT Log MATLAB Log i Info Starting process C Program Files x86 MATLAB R2012a bi i Info Process matlab exe Exit Code Q Error Raptor Build Failed check build log for details
14. 1 nennen nnn nnn nnn nnn nnn nnn 16 D TESI SCRIPTS 030i seo Te c Hao ERES Rr EROR USE ERN UUN 17 6 1 TEST TARGET AND COORDINATOR SETTINGS 17 6 1 1 MODEL BUILDING CONPIGURATION uciectvcksxeeirixuEr dE aH 18 61 2 MODULEJRBLASH CONEIGUEPATIONGaiacssiontvtle tuners I 18 D L3 XCPSEAVECONEIGUEATION usqne tesis 18 6 1 4 ERI CONFIGURATION enne nnnm nnnm errasse nasse thats aar assa sra sas asa sna sra snas snas nsa 19 E 19 6 2 REFERENCE 19 6 3 INCLUDED SCRIPT FILES 19 New Eagle Consulting 3588 Plymouth Road 274 48105 2603 877 234 1410 FAX 928 395 2114 New Eagle Confidential Sef C w 6 4 MACRO DEFINITIONS 20 6 5 TEST DEFINITIONS 20 7 TEE
15. AG 20 New Eagle CONTROLSYSTEM SOLUTIONS New Eagle Automated Testing User Manual Revision 007 Build 2 0 261 56 02 17 2014 New Eagle Consulting 3588 Plymouth Road Box 274 Arbor MI 48105 2603 877 234 1410 FAX 928 395 2114 New Eagle Confidential p a 5 Ca NM New Eagle Contents 1 INTRODUCTION enana aa Ea AEEA 4 2 AUTOMATED TESTING SYSTEM 4 3 CONPIGURA TION eea EEE EEEE EA 5 3 1 CONFIGURATION 58 5 4 AUTOMATED TESTING CONSOLE 1 11 rrr nn nnn nnn nnn nnn 6 4 1 CREATING AN AUTOMATED TESTING SCRIPT eere uuu uu RR RR RR RR RR R RR RR RR RR RR RRRRRRRRRRRRRRRRRRR RR RR Rus 6 A2 SC RPTEDITOR RR E EE 7 4 3 TEST NAVIGATOR 9 n TET OR cece de eee cee ene de ecco ec enced 9 E SENRRT 10 A42 CAN ACTIONS 11 AVS WRC ROA TIO ING 12 444 OTHERTOOLPOXACTION TECH ER DUE IN CIUV TI EL CETUR 14 45S TEST RUNNER RC E 14 5 COMMAND LINE INTERFACE
16. AN message is only transmitted when a tracked value exceeds a threshold In that case NEAT may miss the message and the test may incorrectly fail To prevent that scenario place a Start CAN Rx action before the action that triggers the CAN message to start transmitting CAN Rx verifies that a particular CAN message is received within the specified time frame Both the CAN id and the data payload must match in order for this action to pass CAN Tx transmits CAN message on the bus This message is only sent once no repeating messages CAN No Rx verifies that a CAN message is not received within the specified time frame Just the CAN id is checked for this action the data payload is ignored 4 4 3 Macro Actions There are two kinds of macros supported by NEAT Macros and Initial Condition Macros Macros can contain any action other than an Initial Condition Action Initial Condition Macro Action this means that macros can execute other macros Initial Condition Macros can only contain Initial Condition Actions or Initial Condition Macro Actions To call a macro drag Macro or Initial Condition Macro from the Toolbox A dialog will be displayed showing defined macros of the appropriate type New Eagle Consulting 3588 Plymouth Road Box 274 Ann Arbor MI 48105 2603 PH 877 234 1410 FAX 928 395 2114 New Eagle Confidential C New Eagle Select Macro Search Setup Test Cleanup Test Figure 9 Select Macro
17. Eagle Confidential d oe New Eagle During a test run the list of tests will be updated as tests are run and pass or fail The progress bar at the top of the window indicates how many tests remain to run If any reference values were included in the scripts the read values will be displayed in the sidebar once they are read at the start of the test run Script File J1939 10000 Tests neat re nse Test group DM5 Diagnostic Readiness 1 Build Model All Tests Model File J1939 10000 mdl Matlab Version R2012a Raptor Version AREAMRapton Source Id Test 001 Desc Verify Empty DM1 Response Id Test 002 Desc Verify DM1 Response Single Fault Point to Point 4 Id Test 003 Desc Verify DM1 Response Single Fault Broadcast Id Test 004 Desc Verify DM1 Response Double Fault Point to Point Id Test 005 Desc Verify DM1 Response Double Fault Broadcast Id Test 006 Desc Verify Lamp Reporting No Flash Id Test 007 Desc Verify Lamp Reporting Slow Flash Id Test 008 Desc Verify Lamp Reporting Fast Flash Id Test 009 Desc Verify DM1 Single Fault Inc Occ Cnt Flash Target ECU VSF File 11939 10000 O vsf Target Address OxFD Log Level Normal Y Reference Values XCP Test Target Request Id 0x200 S XCP Test Target Device Id 11939 10000 0 RaptorVersion 2013a 0 0 3 MatlabVersion R2012a 7 14 ModelName J1939 10000 BuildNumber 0 Id Test 001 Desc Verify DM2 Emp
18. U dS 26 9 2 TEST WRITING BEST PRACTICES ban nv dono Faro oo DADO E HD Rn OR 27 10 EIROUBLEESHOO 28 10 1 TEST COORDINATOR TARGET MODEL HAS NOT BEEN BUILT YET eere nnne 28 New Eagle Consulting 3588 Plymouth Road Box 274 Ann Arbor MI 48105 2603 PH 877 234 1410 FAX 928 395 2114 New Eagle Confidential 7 Lo WESS New Eagle 1 Introduction The New Eagle Automated Test NEAT system is a PC based tool that allows developers to write scripted processor in the loop tests for their Raptor and MotoHawk projects These tests can be used for regression testing and test driven development NEAT can be run either interactively or via the command line so it is well suited for use in automated build systems such as Cruise Control NEAT includes the ability to build models in MATLAB and flash an ECU 2 Automated Testing System Architecture Module Under Test Coordinator Module Test Harness Set Input Port Read Output Port Automated Testing S mE Test Reports Figure 1 Automated Testing System Data Flow The hardware components of a NEAT system are Windows based desktop computer for running NEAT A cable for connecting the Windows machine to the CAN bus Kvaser etc The module to test optional A coordinator module for testing I O optional A wiring harness which con
19. When a test has this status the results will contain details about what caused the failure Error An error occurred during test execution that rendered the results invalid This could be a missing value in the ECU communications failure etc Example Status Status Priority Accumulated Status Figure 2 Sample Test Group Status Here Test 3 sstatus of Failed has the lowest priority of any test in the group so the entire group has that status 8 2 XML Report The generated XML report adheres to the NEATScriptResults schema The XSD file defining this schema can be found installed next to the NEAT executable Each test group and test in the script has a corresponding test group or test entry in the results file Within each test result entry is a copy of the log for what happened during the test The amount of information in the log depends on what log level was set at the command line with the 1 option or what log level was set in the sidebar in the NEAT Console New Eagle Consulting 3588 Plymouth Road Box 274 Ann Arbor MI 48105 2603 PH 877 234 1410 FAX 928 395 2114 New Eagle Confidential amu t C w 8 3 PDF Report The PDF contains the status of all tests run It is built on a PDF template that is installed with NEAT An alternative PDF template can be set with the NEAT Configuration tool The PDF template should be a single page PDF US Letter size with header and footers approximately the s
20. age will be properly received New Eagle Consulting 3588 Plymouth Road Box 274 Ann Arbor MI 48105 2603 PH 877 234 1410 FAX 928 395 2114 New Eagle Confidential amu t C w 7 7 InitialCondition This action can only used at start of a test It reads a value from ECU and compares it against an expected value using an equality operation specified in the action If the read value does not pass the test based on the equality operation and the expected value the action fails Possible operations are IsEqual IsNotEqual IsGreaterThan IsLessThan Examples lt InitialCondition Name SPN1213 Operation IsEqual Value 0 gt lt InitialCondition Name SPN623 Operation IsGreaterThan Value 0 gt 7 8 InitialConditionMacro Invokes an initial condition macro by name All InitialCondition actions in the macro must pass in order for this action to pass The macro may be defined in this script file or in another file that is included Example lt InitialConditionMacro Name LampsAreClear gt 7 9 Macro Invokes a normal macro by name All actions in the macro must pass in order for this action to pass The macro may be defined in this script file or in another file that is included Macros do not support parameterization or return values they are simply an easy way to execute common sets of commands without having to copy and paste Example lt Macro Name ClearFaults gt 7 10 Repeat
21. ame size as the default template 9 Test Writing 9 1 Anatomy of a Test As a general rule a test should have four distinct phases 1 Setup 2 Trigger 3 Verify 4 Clean Up The Setup phase is exactly that do whatever needs to be done to get the ECU in the state needed to test the target functionality This can include setting overrides on ECU inputs disabling certain periodic messages setting calibration values etc It is good practice to use a macro for setup actions that will be common across multiple tests The Trigger phase is where the target functionality is triggered This can be done by setting values in the ECU or by sending CAN messages In some tests such as when testing steady state values there will not be an explicit action needed to trigger the target functionality The Verify phase is where the target functionality is tested This is usually done by reading one or more values from the ECU or by listening for a specific CAN Message The Clean Up phase is where the ECU is returned to a known state This phase is the only one explicitly supported by NEAT tests because this phase needs to occur regardless of what happens in the other phases It is good practice to use a macro for clean up actions that will be common across multiple tests Example This test is from a set of tests for Raptor s J1939 block library The model should respond to receiving OX18EAFDF9 CAN message with the 0x18FECAFD CAN message
22. are sent or received from the CAN channel set when you created the script CAN Rx Ox 123 x cana win CAN Tx od FF1277 x 0 Figure 8 CAN Actions CAN frame ids are specified as hexadecimal with no prefix e g Ox or To use an extended CAN id check the X checkbox CAN data is specified as a series of bytes in hexadecimal again with no prefix Each byte is represented by two digits so zero pad small values e g for the value 11 decimal enter Ob New Eagle Consulting 3588 Plymouth Road Box 274 Ann Arbor MI 48105 2603 PH 877 234 1410 FAX 928 395 2114 New Eagle Confidential N 7 dd Kt New Eagle Each byte is separated by a single space If NEAT detects an error in the data format it will show the error notification displayed above on CAN Tx action in this case the 4 byte is missing a digit The message length either transmitted or expected is set to the number of bytes specified so the length for the CAN Rx action above would be 4 There are four CAN actions Start CAN Rxtells NEAT to start recording CAN frames NEAT does not record CAN frames unless it has reached a CAN action Normally this works fine and reduces clutter in the test execution log especially if the CAN bus is busy However in some circumstances the trigger for transmitting a CAN message in the target model occurs before the execution of a CAN action in NEAT e g a C
23. d 001 Desc Verify Empty DM1 Response Id Test 002 Desc Verify DM1 Response Single Fault Point to Point Id Test 003 Desc Verify DM1 Response Single Fault Broadcast Id Test 004 Desc Verify DM1 Response Double Fault Point to Point Id Test 005 Desc Verify DM1 Response Double Fault Broadcast Id Test 006 Desc Verify Lamp Reporting No Flash Id Test 007 Desc Verify Lamp Reporting Slow Flash Id Test 008 Desc Verify Lamp Reporting Fast Flash Id Test 009 Desc Verify DM1 Single Fault Inc Occ Cnt Flash Target ECU VSF File 11939 10000 0 51 Target Address OxFD Reference Values lt lt Log Level Id Test 001 Desc Verify DM2 Empty Response Id Test 002 Desc Verify DM2 Empty Response Flash Lamps Fast Flash lt lt XCP Test Target Request Id 0 200 S XCP Test Target Device Id J1939 10000 0 RaptorVersion 2013a 0 0 3 MatlabVersion R2012a 7 14 ModelName J1939 10000 BuildNumber 0 Id Test_001 Desc Verify DM3 Receive Trigger Global Id Test_002 Desc Verify DM3 Receive Trigger Point to Point Id 001 Desc Verify DM4 Empty Response Id Test 002 Desc Verify DM4 Single DTC Response Point to Point K Id Test_001 Desc Verify DM5 Global Empty Id Test_002 Desc Verify DM5 Point to Point Empty Id Test 003 Desc Verify DM5 SPN1220 Id Test_004 Desc Verify DM5 SPN1221 Id 0
24. execution for a fixed amount of time The Time attribute specifies the amount of time to wait in milliseconds This action cannot fail Example Pause Time 2500 gt 8 Test Results The results of a test run are reported in two different forms based on the command line options used to invoke NEAT or based on what button is used in the NEAT Console An XML report using the r option e A PDF report using the p option Each of these contains the same data just in different formats 8 1 Test Status A test can have one of four statuses depending on what happened during the execution of the test see Table 1 Test Statuses The statuses of tests and subgroups in a test group accumulate to determine the status of the test group The status of the group is set to the lowest priority status of any test or subgroup in the group The results of an entire script are compiled in the same manner from the statuses of all test groups New Eagle Consulting 3588 Plymouth Road Box 274 Ann Arbor MI 48105 2603 PH 877 234 1410 FAX 928 395 2114 New Eagle Confidential 5 fi New Eagle Table 1 Test Statuses Status Description Priority Passed All checks in the test passed and all actions completed without 4 error NotRun The test was not run This can happen if the test has a ToDo 3 action or if the user is running NEAT interactively and chose not to run certain tests Failed One of the checks in the test failed
25. fyCANNoRx Waits for a CAN message with the specified id to be transmitted on the bus If the message is transmitted this action fails If no such message is received then execution continues No data is specified for this action it simply checks for receipt of any CAN message with the specified id The amount of time to wait is specified with the VerifyTime attribute Examples VerifyCANNoRx Id x170 Ext 0 VerifyTime 250 gt VerifyCANNoRx Idz 0x3200 Ext 1 VerifyTime 1000 gt 7 6 CANStartRx This action forces NEAT to begin processing CAN messages Normally NEAT does not begin processing CAN messages until a CAN action CANTx VerifyCANRx VerifyCANNoRx is executed This is done for performance and to keep noise in the log down in situations where there is a lot of CAN traffic on the bus However in some circumstances a test needs to verify receipt of a CAN message that may have been sent before NEAT gets to a CAN action In this case use the CANStartRx action Example lt CANStartRx gt lt Set Adjustment StatusMessage Enable Value 1 gt lt VerifyCANRx Id x230 Ext 0 Data 12 34 56 78 90 ab cd ef Here CAN processing would not normally start until the action is executed but the model may have sent the message immediately upon having StatusMessage Enable set to 1 so NEAT would miss the message The CANStartRx action forces NEAT to begin listening to the CAN bus so the mess
26. ialog Select the framework for the Test Target module Raptor or MotoHawk and the Coordinator module if any For both Raptor and MotoHawk select the SimuLink model file MDL or SLX for the software that will be running on the module and select the Matlab version that should be used for automated building The Raptor Version and MotoHawk Version fields are optional and are intended for future use For Raptor models Enter the flash address used for flashing the module for targets supporting the Vansco Flash Loader This is a hexadecimal number ranging from 0x00 to OxFF 255 Note that as of Raptor 2013b there is no way to change this value in the model from the default of OxFD 253 e Enter the XCP address on which the model will receive XCP messages This can be found and modified in the XCP Protocol Definition block in the Raptor model NEAT uses XCP to communicate with Raptor models so the model must contain an XCP Protocol Definition block to be used with NEAT For MotoHawk models e Enter the city id for the ECU This is the value set in the MotoHawk CAN Definition block in the MotoHawk model e Enter the city id for the tool NEAT The default value of this is 0x03 so as to not collide with MotoTune which uses 0x02 e Select the access level at which to connect to the module If the model has custom security settings an appropriately authorized MotoTune dongle must be present when running scripts with NEAT Additiona
27. l Settings New Eagle Consulting 3588 Plymouth Road Box 274 Ann Arbor MI 48105 2603 PH 877 234 1410 FAX 928 395 2114 New Eagle Confidential N rad umm P 7 New Eagle e Verify Time This is the default amount of time NEAT will wait before determining that a value does not match the required value in a Verify action see Verify 7 27 2 below e Default Delta This is the default value for floating point equality comparisons e CAN Channels This is where the user may add CAN channels to use on the Kvaser device that connects to the test harness e Baud Rate This is the baud rate used on the designated CAN channel e Name Name of the CAN channel as defined in the model file Build Model MatlabVersion R2012a Raptor Version 2013b Flash Model ECU Flash Address ox FD ECU XCP Address 0x 288 Channel 1 Build Model Figure 3 Create Test File Dialog Note These test script settings can be edited later by clicking the button above the navigator pane on the left side of the NEAT console 4 2 Script Editor Once the script is loaded you will be in the test editor which looks like this New Eagle Consulting 3588 Plymouth Road Box 274 Ann Arbor MI 48105 2603 PH 877 234 1410 FAX 928 395 2114 New Eagle Confidential eA New Eagle 3 amp gt Tes 001 Macro Y Setup Test Initial Condition Macros ACCmpOp
28. le Confidential
29. modified during a test should be reset to a neutral value during the Clean Up phase This will help prevent side effects that could influence other tests When dealing with CAN messages it is useful to add hooks to the model that allow periodic messages to be selectively disabled This will cut down on CAN traffic and makes debugging issues in unrelated functionality easier Tests should be grouped logically For instance if the script is testing J1939 functionality a logical grouping would be one test group for each message type DM1 DM2 etc Each test should have a unique concise description that indicates what is being tested This will help speed debugging when a test fails New Eagle Consulting 3588 Plymouth Road Box 274 Ann Arbor MI 48105 2603 PH 877 234 1410 FAX 928 395 2114 New Eagle Confidential nmm P j A C Tm le 10 Troubleshooting 10 1 Test Coordinator Target Model has not been built yet Model Not Built X Test Target Model A AREA ip i am mg Sega gene o D md has not been built yet This dialog will be shown when creating a new NEAT test suite for a Raptor model if the model has not yet been built with Simulink If the model has been built and this dialog is still Showing make sure verify the NEAT configuration section 3 New Eagle Consulting 3588 Plymouth Road Box 274 Ann Arbor MI 48105 2603 PH 877 234 1410 FAX 928 395 2114 New Eag
30. n order when the macro is referenced in a test There is no support for parameters or return values 6 5 Test Definitions Tests are grouped into a hierarchy under TestGroup elements test consists of e optional Initial condition verification e Asequence of actions that perform the test optional clean up actions During a test run if any action fails or encounters an error the test fails and execution is aborted If there is a CleanUp section defined those actions will be executed even if the test fails 7 Test Actions 7 1 Set The Set action sets a value in a module In Raptor this is done via the XCP protocol and the value must be set up as an Adjustment in the model In MotoHawk this is done via the ERI protocol and the value must be set up as a calibration in the model with an appropriate access level Options e Target Indicates which module the value should be set in the module under test or the coordinator module If no target is specified then the value will be set in the module under test Examples Set Adjustment CompleteDriveCycle Value 1 gt Set Adjustment CompleteDriveCycle Value 1 Target TestTarget gt Set Adjustment Target DigInput1 Value 1 Target Coordinator New Eagle Consulting 3588 Plymouth Road Box 274 Ann Arbor MI 48105 2603 PH 877 234 1410 FAX 928 395 2114 New Eagle Confidential C AN w 7 2 Verify Reads a value a module and
31. nects the coordinator module to the module under test SIE ew p De pa The software components of a NEAT system are 1 The NEAT test executor software on Windows based PC 2 The Raptor or MotoHawk model flashed onto the module under test 3 optional The coordinator model flashed onto the coordinator module 1 http en wikipedia org wiki Test driven development New Eagle Consulting 3588 Plymouth Road Box 274 Ann Arbor MI 48105 2603 PH 877 234 1410 FAX 928 395 2114 New Eagle Confidential New Eagle 3 Configuration A NEAT configuration executable is provided alongside NEAT to allow users to edit settings NEAT Configuration A2L Base Path E 2 Ez ECU Flash Program C Utils VanscoAlashLoader_Kvaser FlashLoader exe C Flash Device Id 177 Leaf Light Leaf Light USB Broadcast Address 100 z Extended Hex Flash Timeout sec 150 PDF Template C Program Files 86 NewEagle NEAT Template pdf zl MotoHawk TDBDLL Path Jc ECUFiles TDBDLL MotoHawk Dump DLL Utility C Program Files x86 NewEagle NEAT motohawk_dumpdll exe e zx Build Config Metadata Save Reload Close Figure 2 NEAT Configuration Editor Note you must press Save before exiting the NEAT Configuration editor or the new configuration settings will not be saved 3 1 Configuration Settings A2L Base Path Directory where 21 files will be stored
32. onfigured then there will be a drop down at the top of the Signals List that allows you to switch between the two modules Search for a signal by typing part of its name in the Search bar To set or verify a signal value drag the signal name onto the Test Editor Action Toolbox This contains test actions that can be put into your test that do not deal directly with an ECU signal See Section 7 Test Actions below page 20 for a description of the operation of each action New Eagle Consulting 3588 Plymouth Road Box 274 Ann Arbor MI 48105 2603 PH 877 234 1410 FAX 928 395 2114 New Eagle Confidential p P d T 72 m pa New Eagle 4 3 Test Navigator The Test Navigator allows you to organize your tests and create new tests groups and macros The toolbar at the top of the navigator contains several buttons Icon Name Function Create new group under the current group If the Tests node is p selected creates a new root group NewTest Creates a new test under the current group New Macro Creates a new Initial Condition Macro Runs the selected test in the Test Runner see section 4 5 Test Run One Runner page 14 Ae Run All Runs all of the tests in the Test Runner x Delete Deletes the currently selected group test or macro If a group is deleted all tests and groups under that group are also deleted To view a test or macro simply select it in the tree view and the test s
33. oordinator gt 7 3 CANTx Sends a raw CAN message onto the bus Multi packet is not supported so if a long message needs to be sent include multiple CANTx elements Standard and Extended CAN ids are supported CAN ids are always specified in hexadecimal even if the Ox prefix is omitted Message data is specified as a series of two digit hexadecimal values one for each byte of data The length of the message is determined from the number of bytes specified The data will show up in Kvaser CANKing exactly as it is written in the CANTx element Examples lt Id 0x131 Ext 0 Data 01 ba lt Id 0x3000 Ext 1 Data 12 34 56 78 90 ab ef gt New Eagle Consulting 3588 Plymouth Road Box 274 Ann Arbor MI 48105 2603 PH 877 234 1410 FAX 928 395 2114 New Eagle Confidential Sef C w 7 4 VerifyCANRx Waits for a CAN message be transmitted bus message id expected message data are specified The first message that matches the id is assumed to be the intended message and that message s data is checked against the expected data If the data does not match this action fails If a message with the expected id is not received within the amount of time specified in VerifyTime then this action fails Examples VerifyCANRx Id x150 Ext 0 Data 12 34 56 78 VerifyTime 200 gt VerifyCANRx Id 0x3000 Ext 1 Data VerifyTime 200 gt 7 5 Veri
34. or macro s actions will be displayed in the Test Editor 4 4 Test Editor This is where you can view and edit what occurs during a test Actions can be reordered by dragging them up or down To delete an action simply select itin the Test Editor and press the Delete key Start CAN Rx X 98 12 33 45 250 CAN Tx Ox FF1277 FF FF 12 CAN No 125 Ex Figure 5 Moving an Action New Eagle Consulting 3588 Plymouth Road Box 274 Ann Arbor MI 48105 2603 PH 877 234 1410 FAX 928 395 2114 New Eagle Confidential A P F adi C eA Ne Eagle 4 4 1 Signal Actions To set a value in the ECU find the signal in the Signals List Drop the signal where you want the action to occur in the action sequence A small pop up will appear where you can indicate if you want to set the value or verify the value AxleRatio Figure 6 Signal Actions Set or Verify actions can occur anywhere in the test but Initial Condition actions must occur at the start of the test NEAT will move Initial Condition actions to the Start of the test to prevent an invalid sequence VehOdmtr Figure 7 Signal Actions The signal action blocks contain a lot of information in a small amount of space e TheI SandV labels the left stand for Initial Condition Set and Verify which is the action type e The CandT labels on the right stand for Coordinator and Test Target They indicate which ECU has the signal If
35. tn Macros 4 ApplicationCompatibilityArr 4 TODO Add actions from the signals list and actions toolbox ApplicationinitStatus Test Group 1 ApplicationinterruptTaskCPU ApplicationinterruptTaskStackMargin ApplicationinterruptTaskStackMinBytesFree ApplicationMonitorEnable ApplicationPausePoint ApplicationStatus Te st ApplicationStopReason d ApplicationstopReasonAppLink N avigator ArrayTest TC ArrayTest WR Signals AxleRatio d BGNDTaskCPU List BGNDTaskStackMargin d BGNDTaskStackMinBytesFree CAN No Rx CAN Start Rx Initial Condition Macro Log Message Figure 4 NEAT Console Test Editor There are several important regions on this screen labeled in blue above Test Navigator this is a tree that shows all of your tests groups and macros Tests are organized in a nested hierarchy Search for a test or group by typing part of its name in the Search bar Test Editor this is where a test s actions are listed To add actions to a test drag the action in from the Action Toolbox ora signal in from the Signals List Clean Up Actions Pane this area can be expanded by clicking the arrow next to Clean Up Actions label It contains all of the actions that are executed in the clean up phase of the test These actions are always executed even if the test failed or was aborted due to an error Signals List This lists all of the signals available in the ECU If both a Test Target and Coordinator module are c
36. ty Response Id Test 002 Desc Verify DM2 Empty Response Flash Lamps Fast Flash Id Test 001 Desc Verify Receive Trigger Global Id Test 002 Desc Verify DM3 Receive Trigger Point to Point Id Test 001 Desc Verify DM4 Empty Response Id Test 002 Desc Verify DM4 Single DTC Response Point to Point Id Test 001 Desc Verify DM5 Global Empty Id Test 002 Desc Verify DM5 Point to Point Empty Id Test 003 Desc Verify DM5 SPN1220 Id Test 004 Desc Verify DM5 SPN1221 Id Test 005 Desc Verify DM5 SPN1222 Id Test 006 Desc Verify DM5 SPN1223 Group Id DM6 Name DM6 Emission Related Pending DTCs Id Test 001 Desc Verify DM6 Empty Response Status Passed Status Passed Status Passed Status Passed Status Passed Status Passed Status Passed Status Passed Status Passed EEEE o FEBE Status Passed Status Passed BE log Status Passed log Status Passed log Status Passed log Status Passed log Status Passed log Status Passed Status Running Status NotRun Status NotRun Status NotRun Status NotRun Status NotRun Figure 12 NEAT Console during test run Once the test is complete there are several options for examining the results of the run fa NEAT Script File J1939_10000_Tests neat Build Model Model File J1939 10000 mdl Matlab Version R2012a Raptor Version CADEV AREAMRapton Source lt gt 3 a lt I
37. when Raptor models built e ECU Flash Program Path to the executable used to flash test and coordinator modules Flash DLL dll file used by ECU flash utility Flash Device Id ID of device used to flash 5 XCP Broadcast Address XCP Address used to communicate with modules Flash Timeout sec Number of seconds that NEAT will wait while waiting to flash an ECU if applicable e PDF Template Path to file that NEAT will use as a template when generating pdf reports MotoHawk TDBDLL Path Directory where MotoHawk dll files will be stored when MotoHawk models are built e MotoHawk Dump DLL Utility Path to MotoHawk Dump DLL Utility New Eagle Consulting 3588 Plymouth Road Box 274 Ann Arbor MI 48105 2603 PH 877 234 1410 FAX 928 395 2114 New Eagle Confidential 2 5 Ca NM New Eagle 4 Automated Testing Console The NEAT Console is where you can author edit and run tests interactively Launch the NEAT Console from the Windows Start Menu or by double clicking on a NEAT test script in Windows Explorer If you start NEAT via the Windows Start Menu you will have the option of manually opening an existing NEAT file creating a new file or opening a recently used file 4 1 Creating an Automated Testing Script To create anew NEAT script select the New option from the File menu NEAT will show you a dialog box that lets you configure the new test script See Figure 3 Create Test File D
Download Pdf Manuals
Related Search