Home

Sun Microsystems Telephone Accessories 1.2 User's Manual

1. Keywords Keywords are tokens associated with specific tests Keywords have the following functions m Convey information to the harness about how to execute the tests m Serve as a basis for including and excluding tests during test runs Users specify keyword expressions in the harness Configuration Editor to filter tests during test runs Keywords are specified by the test developer in the keywords field of the test description Test suites can provide additional custom keywords TABLE C 2 identifies the Framework keywords and describes the function of their values in the test description file TABLE C 2 Framework Keywords Keyword Description interactive Identifies tests that require human interaction negative The component under test must terminate with and detect an error An operation performed by a negative test on the component under test must not succeed OTA Identifies OTA tests positive The component under test must terminate normally An operation performed by the test on the component under test must succeed 94 Java ME TCK Framework Developer s Guide July 2007 TABLE C 2 Framework Keywords Continued Keyword Description runtime Identifies tests used with runtime products trusted Identifies tests which must be run in a trusted operator manufacturer and trusted third party security domain untrusted Identifies tests which must be run in an untrusted unidentified third party security
2. 1 getNextApp 2 Application Bundle agent tests getNextTest O Test Name O sendTestResult Execution Server Agent Diagram of a standard automated test configuration Distributed Tests Distributed tests are a special type of automated tests Not only do they have a device side test component which is executed under the control of a test agent as with any regular automated tests but they also have one or more remote components on other devices or the Java SE platform side The distributed test components have names and communicate with each other by sending messages to each other by way of a messaging service The remote components of a distributed test typically run on a harness host by using a passive agent in the same virtual machine as the harness and provide some additional functionality needed by the test For example a test verifies that an HTTPS connection can be made to the remote host The remote component that runs on the Java SE platform would be an HTTPS server The test on the device performs the following sequence of actions 1 Sends a message requesting that the server start 2 Connects to the server and verify that the connection is OK 3 Sends a message to stop the server Java ME TCK Framework Developer s Guide July 2007 Distributed tests are typically slower due to extra communication between remote components and more complex than simple automated tests and should be used
3. Items 3 4 and 5 repeat until all tests in the application bundle are run When all tests in the application bundle are run the AMS requests the next application bundle items 1 and 2 and the sequence is repeated until all tests and all test bundles in the test suite are run Testing Communications or Networking With Distributed Tests Distributed tests are a special type of automated tests Not only do they have a device side test component which is executed under the control of a test agent as with any regular automated tests but they also have one or more remote components on other devices or the Java SE platform side The distributed test components have names and communicate with each other by sending messages to each other by way of a messaging service The remote components of a distributed test typically run on a harness host by using a passive agent in the same virtual machine as the harness and provide some additional functionality needed by the test For example a test verifies that an HTTPS connection can be made to the remote host The remote component that runs on the Java SE platform would be an HTTPS server To develop distributed tests the test writer must write classes for both the client and remote test components as well as create an appropriate test description file The SimpleTestSuite directory contains examples of distributed tests Client Test Component In the MIDP case the client test component of the d
4. The following example is a complete configuration interview SampleInterview java that contains the added question code public class SampleInterview extends Interview param parent throws Fault aA public SampleInterview MidpTckBaseInterview parent throws Fault super parent sample init 74 Java ME TCK Framework Developer s Guide July 2007 private void init throws Fault setResourceBundle il8n setHelpSet help sampleInterview first new StringQuestion this hello first setExporter Exporters getStringValueExporter sample string value first linkTo qFeatureSupport setFirstQuestion first This is the added question Does the implementation support the feature private YesNoQuestion gqFeatureSupport new YesNoQuestion this featureSupport public void clear setValue YesNoQuestion NO protected void export Map data boolean b value YES data put isFeatureSupported String valueOf b protected Question getNext return end hi private StringQuestion first private final FinalQuestion end new FinalQuestion this end 3 Add interview question smry and text entries to the properties file For each added interview question the developer must create corresponding smry and text entries in the resource file properties The following is an example of the smry and text entries added fo
5. Java ME TCK Framework Developer s Guide July 2007 3 Use the Configuration Editor to provide configuration information required to run the updated test suite 4 After you complete the interview choose Run Test gt Start on the Test Manager main menu to start the test suite 5 Use the following command to start the WTK c WIK_InstallDir bin emulator Xautotest http JAVATEST_HOSTS3 3JAVATEST_PORT test getNextApp jad If you answered the interview questions correctly all of the tests run successfully Chapter 2 Writing a Simple Automated Test 21 22 Java ME TCK Framework Developer s Guide July 2007 CHAPTER 3 Test Suite Construction This chapter describes the organization and construction of a Java ME technology test suite that uses Framework resources Additional information about constructing test suites for use with the JavaTest harness can be located in Chapters 4 and 8 of the JavaTest Architect s Guide This chapter contains the following sections m Test Suite Structure m Creating a Test Suite Configuration Interview m Building a Test Suite Test Suite Structure Test suites are the main unit of test development and deployment A test suite is a self contained collection of tests designed to test a major feature or a major subset of an API or a profile When architects and developers define the contents and structure of a test suite they should group tests that use the same test setup
6. log println Running NAME jar n String cmt new String Some comments String qn Some Questions s handler run jadURL NAME cmt qn if s null s isPassed log println Could not run midlet suite return s return otaServer getStatus testCaselID finally if handler removeAll null isPassed ref println Could not remove midlet suites Chapter 4 Writing Tests 59 CODE EXAMPLE 4 12 Server Test Component Example Continued Testing Security Constrained Functionality With Security Tests This section addresses writing different types of security tests using keywords and attributes to mark different types of security tests and using attributes in the test description to grant or deny the security permissions for the tests Types of Security Tests Security related tests can be grouped into the following types m Untrusted m Trusted a Double duty Untrusted Tests Untrusted tests verify API implementation behavior for unsigned MIDlet Suites Untrusted tests verify that the assertions related to untrusted MIDlet suites are properly implemented All untrusted MIDlet suites can run in a restricted environment where access to protected APIs or functions is either not allowed or allowed only with explicit user permission Untrusted tests must not be run in trusted security mode See Marking Untrusted Tests on page 61 for a description of how to us
7. This class provides an implementation of the harness distribution mechanism on top of HTTP a com sun tck midp javatest ExportFilter Exports tests for offline standalone execution a com sun tck midp javatest TestRegistry Static class that keeps track of the global properties of individual tests Tests are keyed by their unique IDs a com sun tck midp javatest MidBundle MIDP specific extension that builds on top of the generic CLDC mechanism a com sun tck midp javatest MidMessageClient Support for distributed tests on MIDP This class provides an implementation of the harness distribution mechanism on top of HTTP doc Directory The doc directory contains the Framework Release Notes and the Java ME TCK Framework Developer s Guide Appendix B Framework Bundle 89 90 Java ME TCK Framework Developer s Guide July 2007 APPENDIX C Test Description Fields and Keywords This appendix describes the Framework supported fields and keywords for test description files The JavaTest harness requires that each test is accompanied by machine readable descriptive data in the form of test suite specific name value pairs contained in a test description file The JavaTest harness uses the contents of the test description file to configure and run tests See the Java Technology Test Suite Development Guide for detailed information about creating test description files Test Description Fields The test descr
8. compatibility rules 99 compatibility testing 99 configuration 99 Configuration Editor 99 configuration interview 99 configuration templates 99 configuration value 99 D distributed tests 100 E equivalence class partitioning 100 exclude list 100 F first level appeals process 100 G glass box testing 104 Graphical User Interface 100 GUI 100 H HTML test description 100 107 l instantiation 100 interactive tests 100 ISO 7816 4 98 J JAM 101 JAR 101 Java Application Manager 101 Java Archive 101 Java Community Process Program 101 Java Compatibility Test Tools 101 Java CTT 101 Java Platform Libraries 101 Java platform Standard Edition 101 Java SE 101 Java specification 98 101 Java Specification Request 101 Java technology 101 Java Technology Compatibility Kit 101 JavaTest agent 101 JavaTest harness 102 JSR 101 jtx files 100 K keywords 102 M Maintenance Lead 102 method 102 ML 102 MultiTest 102 N namespace 102 O object oriented 102 objects 102 P packages 102 passive agent 102 108 Java ME TCK Framework Developer s Guide July 2007 PMO 103 prior status 102 Profile Specification 102 Program Management Office 103 protected API 103 protection domain 103 R Reference Implementation 103 RI 103 S second level appeals process 103 security domain 103 security policy 103 signature file 103 si
9. interface is implemented by all Java ME technology tests Each test must define the run method as follows CODE EXAMPLE A 1 run Method public Status run String args PrintStream log PrintStream ref 77 A test must also define main as follows CODE EXAMPLE A 2 Definition of main public static void main String args Test t new lt test class name gt Status s t run args System err System out s exit Defining main in this manner enables the test to also be run standalone independent of the harness Status Class name com sun tck cldc lib Status This is a Java ME technology version of the standard harness Status class It embodies the result of a test a status code and a related message MultiTest Class name com sun tck cldc lib MultiTest This is a Java ME technology version of the standard harness MultiTest class It serves as a base class for tests with multiple sub test cases This base class implements the standard com sun tck cldc lib Test features so that you can provide the additional test cases with only a little concern about the boilerplate needed to execute individual test case methods namely update the runTestCases method to add check and invocation statement MultiTest is designed as a base class used during development of new test classes You must add individual test case methods to your derived test class to create a useful test class Each test case m
10. lt TD gt lt B gt source lt B gt lt TD gt lt TD gt lt A HREF midlet Test_MIDlet java gt midlet Test_MIDlet java lt A gt lt TD gt lt TR gt lt TR gt lt TD gt lt B gt executeClass lt B gt lt TD gt lt TD gt sample pkg OTA Test_MIDlet lt TD gt lt TR gt lt TR gt lt TD gt lt B gt keywords lt B gt lt TD gt lt TD gt runtime positive trusted ota lt TD gt lt TR gt lt TD gt lt B gt remote lt B gt lt TD gt lt TD gt networkAgent sample pkg OTA OTATest1 httpServerPort ShttpServerPort testDir testDir OTAHandlerClass SOTAHandlerClass OTAHandlerArgs SOTAHandlerArgs signer jks signer signerArgs S jks signer args lt TD gt lt TR gt lt TD gt lt B gt remoteSource lt B gt lt TD gt lt TD gt lt A HREF OTATest1 java gt OTATest1 java lt TD gt lt TABLE gt lt BODY gt lt HTML gt OTA Test Execution In FIGURE 4 4 arrows indicate the direction of dataflow between the device and the workstation The numbered items indicate the sequence and the content of the dataflow Chapter 4 Writing Tests 57 FIGURE 4 4 OTA Test Execution PC or Workstation Device JavaTest Harness Request to Download Test Application i O Test Application Server Side _ Test OTA Server O Send Test Result Test Passive Application Agent Request an Action q 1 1 1 1
11. Framework and harness documentation in PDF and HTML format The following documentation provides detailed information about the Java programming language and the harness included with this release Application Title JavaTest Harness JavaTest Harness User s Guide Graphical User Interface JavaTest Harness User s Guide Command Line Interface JavaTest Architect s Guide Programming Reference The Java Programming Language Programming Reference The Java Language Specification Java ME TCK Framework Developer s Guide July 2007 Accessing Sun Documentation Online The Java Developer Connection program web site enables you to access Java platform technical documentation at http java sun com Sun Welcomes Your Comments We are interested in improving our documentation and welcome your comments and suggestions Provide feedback to Sun at http java sun com docs forms sendusmail html Preface xix xx Java ME TCK Framework Developer s Guide July 2007 CHAPTER 1 Introduction This chapter introduces using the Framework resources to develop test suites for the Java ME platform including descriptions of the kinds of tests that a test suite can include This chapter contains the following sections m Getting Started m Use of the Framework m Framework Bundle Test Types Getting Started For test developers who do not already have an understanding of the TCK and test suite development proc
12. TD gt lt TD gt lt A HREF Test java gt Test java lt A gt lt TD gt lt TR gt lt TR gt lt TD SCOPE row gt lt B gt executeClass lt B gt lt TD gt lt TD gt sample pkg Test lt TD gt lt TR gt lt TR gt lt TD SCOPE row gt lt B gt grant lt B gt lt TD gt lt TD gt javax microedition io Connector file read Chapter 4 Writing Tests 65 66 CODE EXAMPLE 4 19 Test Description That Grants Permissions for a Security Test javax microedition io Connector file write lt TD gt lt TR gt lt TABLE gt lt BODY gt lt HTML gt Denying Security Permissions When writing security tests a test writer can specify the permissions that the security policy must not grant in the protection domain for this test application to execute and pass Test writers can specify the denied security permissions by including a deny attribute in the test description If the security policy grants the specified permissions the test must be filtered out of the test run For example suppose a test application is written to expect that a security exception is thrown because the javax microedition io Connector file read permission is not granted If the security policy grants the javax microedition io Connector file read permission to the test application the test must be filtered out and not run The following is an example of the deny attribute and permission added to a test descripti
13. They are supplied to the converter when the CAP file is generated Software used to download store and execute Java applications Another name for AMS is Java Application Manager JAM An API defines calling conventions by which an application program accesses the operating system and other services A script that is sent to the test applet as defined by ISO 7816 4 A statement contained in a structured Java technology API specification to specify some necessary aspect of the API Assertions are statements of required behavior either positive or negative that are made within the Java specification Compatibility testing based on testing assertions in a specification Test that run without any intervention by a user Automated tests can be queued and run by the test harness and their results recorded without anyone being present A set of test development methodologies that are based on the description behavior or requirements of the system under test not the structure of that system This is commonly known as black box testing Java ME TCK Framework Developer s Guide July 2007 boundary value analysis class classes compatibility rules compatibility testing configuration Configuration Editor configuration interview configuration templates configuration value A test case development technique that entails developing additional test cases based on the boundaries defined by previously categori
14. and whether it is a positive or negative test Keywords are often used to select or deselect tests from a test run The most common keywords are runtime and positive See Chapter 4 for a description of the specific keywords required for the different types of tests that can compose a test suite See Appendix C for a summary list of the Framework test description fields and keywords classes Directory The classes directory contains all of the compiled test classes and library classes required to run the tests Chapter 3 Test Suite Construction 27 The classes directory generally contains following sub directories a classes preverified Contains the preverified test classes a classes shared testClasses 1st The test class dependency file testClasses 1st provides information to the Framework test bundling infrastructure regarding which classes should be bundled for each test This information is used in CLDC and MIDP but ignored in CDC doc Directory The doc directory contains test suite specific documentation such as User Guides that describe how to run the test suite Creating a Test Suite Configuration Interview All nontrivial test suites require additional information about the tests in order for the test harness to execute them This additional information referred to as the test configuration is obtained from the user through a test suite specific configuration interview written by the test developer Additio
15. Distributed test communication client interface m j2me_communication jar Service that provides communication between the harness and the remote device CommClients The following are the CommClients jar files provided in the 1ib directory m midp_commClient jar Distributed test communication client interface implementation that uses HTTP for communication 84 Java ME TCK Framework Developer s Guide July 2007 cldc_commClient jar Implementation of the Client interface for the CLDC device This client uses HTTP for communication datagram_commClient jar Distributed test communication client interface implementation that uses datagrams for communication CommServers The following are the CommServers jar files provided in the 1ib directory datagram_comServer jar Datagram based communication server for use with distributed tests j2me_httpCommServer jar HTTP based communication server for use with distributed tests Messaging j2me_messaging jar provides the Java ME technology messaging service classes These classes are the base classes for distributed tests Test Export Support Libraries The following JAR files are automatically copied into the export directory when exporting tests and are used from the exported Ant build script build xml exportSigner jar Command line tool for signing JAD files when rebuilding exported tests provisioning_server jar Simple HTTP server used for OTA provisi
16. EXAMPLE 4 18 grant Attribute Entry and Security Permissions lt TR gt lt TD SCOPE row gt lt B gt grant lt B gt lt TD gt lt TD gt javax microedition io Connector file read javax microedition io Connector file write lt TD gt lt TR gt The value of the grant attribute is a list of space separated permissions in the example javax microedition io Connector file read and javax microedition io Connector file write that must be granted by the security policy for this test application to execute and pass If these permissions are not granted for the test application the test application must be filtered out The following is an example of a test description file that includes the grant attribute and permissions CODE EXAMPLE 4 19 Test Description That Grants Permissions for a Security Test lt TITLE gt Test Specifications and Descriptions for Test lt TITLE gt lt HEAD gt lt BODY gt lt H1 gt Test Specifications and Descriptions for Test lt H1 gt lt HR gt lt a name Test gt lt a gt lt TABLE BORDER 1 SUMMARY Javatest Test Description CLASS TestDescription gt lt THEAD gt lt TR gt lt TH SCOPE col gt Item lt TH gt lt TH SCOPE col gt Value lt TH gt lt TR gt lt THEAD gt lt TR gt lt TD SCOPE row gt lt B gt title lt B gt lt TD gt lt TD gt checking grant permission lt TD gt lt TR gt lt TR gt lt TD SCOPE row gt lt B gt source lt B gt lt
17. Framework project After becoming familiar with TCK and test suite components and development processes test developers can begin using this Developer s Guide to write tests and create Java ME technology test suites that use Framework resources To help familiarize test developers new to the Framework with the process of writing tests and building test suites the chapters in this Developer s Guide are presented in the following sequence that begins with examples of simple tasks such as writing a simple test and updating a test suite and progressively introduces the more complex tasks of creating custom test suites tests and test configurations m Chapter 2 describes how to write a simple test that can be added to an existing test suite provided by the Framework and run by the harness m Chapter 3 describes how to use Framework resources when constructing a custom test suite and test configuration m Chapter 4 describes how to use Framework resources when writing different types of tests In addition to the preceding chapters the following appendices of this guide provide additional information useful to test developers when creating custom tests and test suites for Java ME technology implementations m Appendix A describes the test APIs for different types of tests m Appendix B describes the contents of the Framework bundle the Framework resources and the Framework components m Appendix C provides a summary of Framewor
18. TD gt lt TR gt executeClass Attribute The executeClass attribute specifies the client test class while the remote attribute specifies the execution command for the server test component The following is an example of the executeClass attribute entry that must be added to the test description file CODE EXAMPLE 4 7 executeClass Attribute Entry R gt D gt lt B gt executeClass lt B gt lt TD gt D gt sample pkg OTA Test_MIDlet lt TD gt lt TR gt Java ME TCK Framework Developer s Guide July 2007 remote Attribute The remote attribute specifies the execution command for the server test component The following is an example of the remote attribute entry that must be added to the test description file CODE EXAMPLE 4 8 remote Attribute Entry lt TR gt lt TD gt lt B gt remote lt B gt lt TD gt lt TD gt networkAgent sample pkg OTA OTATest1 httpServerPort ShttpServerPort testDir testDir OTAHandlerClass SOTAHandlerClass OTAHandlerArgs SOTAHandlerArgs lt TD gt lt TR gt In the remote attribute sample pkg OTA OTATest1 is the test class of the server component and all other arguments httpServerPort httpServerPort testDir testDir OTAHandlerClass S OTAHandlerClass OTAHandlerArgs OTAHandlerArgs are required for OTA tests If the test is written for a trusted MIDlet the argument signer jks signer signerArgs S jks signer args must also be included in t
19. are created in the following form interview class name question name The interview properties file contains the following types of elements m The title of the full interview m A title for each question Question titles take the following form question key smry title text m The text for each question Question text takes the following form question key text question text m Additional entries for choice items that are localized For every interview question that you create you must add corresponding smry and text entries in the interview properties file You can either update the existing file or create a new one In the example the interview class is named SampleInterview and the question name is hello For the example the following entries must be in the interview properties file SampleInterview hello smry Sample Config Property SampleInterview hello text Enter Hello here and the test will verify the value To view the complete contents of the file see the i18n properties file in the following Simple Test Suite source location SimpleTestSuite src sample suite 5 Set up the More Info system The JavaHelp system libraries required to display More Info in the Configuration Editor are included in the javatest jar However you must configure the More Info system to include corresponding More Info topics for the questions in the interview The following steps describe how to set up the More Info syst
20. domain Appendix C Test Description Fields and Keywords 95 96 Java ME TCK Framework Developer s Guide July 2007 Glossary The definitions in this glossary are intended for Java Compatibility Test Tools Java CTT and Java Technology Compatibility Kits TCK Some of these terms might have different definitions or connotations in other contexts This is a generic glossary covering all of Sun s CTTs and TCKs and therefore it might contain some terms that are not relevant to the specific product described in this manual active agent active applet instance agent monitor agents all values API member API member tests A type of test agent that initiates a connection to the JavaTest harness Active test agents enable you to run tests in parallel using many agents at once and to specify the test machines at the time you run the tests Use the agent monitor to view the list of registered active agents and synchronize active agents with the JavaTest harness before running tests See also test agent passive agent and JavaTest harness agent An applet instance that is selected on at least one of the logical channels The JavaTest window that is used to synchronize s and to monitor agent activity The Agent Monitor window displays the agent pool and the agents currently in use See test agent active agent passive agent and ross All of the configuration values required for a test suite All values include the test en
21. gt lt TD gt lt TD gt lt A HREF Test java gt Test java lt A gt lt TD gt lt TR gt lt TR gt lt TD SCOPE row gt lt B gt executeClass lt B gt lt TD gt lt TD gt sample pkg Test lt TD gt lt TR gt lt TR gt lt TD SCOPE row gt lt B gt DoubleDutySecurity lt B gt lt TD gt lt TD gt yes lt TD gt lt TR gt lt TABLE gt lt BODY gt lt HTML Granting or Denying Security Permissions A security test might require that certain permissions be granted or denied for the test to pass The test is run or not run according to the permissions that a test writer grants or denies Granting Security Permissions When writing security tests a test writer can specify the permissions that the security policy must grant in the protection domain for the test application to execute and pass A test writer can specify the required security permissions by including a grant attribute in the test description If the security policy doesn t grant the specified permissions the test must be filtered out of the test run For example a test application is written based on the assumption that following permissions are granted m javax microedition io Connector file read m javax microedition io Connector file write 64 Java ME TCK Framework Developer s Guide July 2007 The following is an example of a grant attribute and permissions added to a test description file CODE
22. guide is available from the Java ME Technology APIs and Docs web site at http java sun com javame reference apis jsp m JavaTest Harness User s Guide Command Line Interface This guide provides a description of using the harness command line interface This guide is available from the Java ME Technology APIs and Docs web site at http java sun com javame reference apis jsp Intended Audience This guide is intended for Java ME technology test suite developers and test writers who are using the Framework resources to create test suites xvi How This Book Is Organized Chapter 1 introduces using the Framework resources to develop test suites for the Java ME platform Chapter 2 describes the process of creating a simple automated test that uses Framework resources Chapter 3 describes the process required to build test suites that use Framework resources Chapter 4 describes the types of Java ME technology tests that can be written Appendix A describes the test Application Programming Interfaces APIs for different types of test suites Appendix B describes the contents of the Framework bundle Appendix C contains a summary of the Framework test description fields and keywords Glossary contains the definitions of words and phrases used in this book Java ME TCK Framework Developer s Guide July 2007 Platform Commands This document does not contain information about basic platform commands and procedure
23. is used to demonstrate the following aspects of creating a test class The format of an automated test class How a test case method is implemented How a test case is selected in the runTestCases method How each test case returns a Status object How to use ref to output reference information for debugging purpose A text editor or an integrated development environment IDE of your choice is the only tool required to develop a test 1 Enter the following test code into a text editor or IDE of your choice package sample pkg3 import com sun tck cldc lib MultiTest import com sun tck cldc lib Status Sample test with simple test cases Java ME TCK Framework Developer s Guide July 2007 Af public class Test3 extends MultiTest protected void runTestCases if isSelected helloWorld addStatus helloWorld JESE public Status helloWorld String messagel new String Hello World String message2 new String Hello World ref println messagel messagel ref println message2 message2 if message2 equals messagel return Status failed Failed see ref for details return Status passed OK 2 Save this file in the Simple Test Suite source as SimpleTestSuite tests pkg3 Test3 java The pkg3 folder does not exist in SimpleTestSuite tests and can be created by the test developer when saving Test3 java Sources for the tests should be placed ins
24. m Done The user is required to perform certain actions and confirm the completion but cannot otherwise affect the outcome of the test Information only A visual component is presented to the user but no user action is required Appendix A TestAPI 79 OTATest Class name com sun tck midp lib OTATest This is the base class for the Java SE technology component of an OTA test It provides various convenience methods which trigger installation execution or removal of an application 80 Java ME TCK Framework Developer s Guide July 2007 APPENDIX B Framework Bundle This appendix describes the contents of the redistributables directory provided by the Framework bundle The contents of the redistributables directory are organized in the following structure m lib Directory m src Directory m doc Directory lib Directory The 1ib directory contains the Framework JAR files and keystore files The files provided in the 1ib directory are not part of the Framework functional groupings The following figure illustrates the dependencies between the JAR files in this directory 81 CLDC and MIDP Execution DTF CLDC and MIDP Agents and Clients CommClients midp_xprteint jar midp_agentjar midp_commClient jar midp_httpclnt jar l midp_agent10 jar cldc_commClientjar cldc_xprtcint jar cldc_agent jar datagram_commClient jar CommService client jar Plugins j2me_commClients jar 4
25. the description behavior or requirements of that system This is commonly known as white box or glass box testing Compatibility testing does not make use of structure based test techniques Refers to the combination of operating system platform Java programming language and JavaTest harness tools and settings A test description that is embedded in the Java programming language source file of each test The suite of tests tools and documentation that enable an implementor of a Java specification to determine if the implementation is compliant with the specification A file used by the Java CTT Spec Trac tool to track the test coverage of a test suite during test development It binds test cases to their related assertion in the specification The bindings make it possible to generate statistical reports on test coverage Any binary representation of the form and function defined by a Java specification An application that receives tests from the test harness runs them on the implementation being tested and reports the results to the test harness Test agents are normally only used when the TCK and implementation being tested are running on different platforms See also active agent passive agent and JavaTest harness agent The source code and any accompanying information that exercise a particular feature or part of a feature of a technology implementation to make sure that the feature complies with the Java specification
26. 1 1 1 1 Action 1 Request an Action The server side test requests an action from the test operator 2 Action The test operator action is sent to the AMS 3 Request to Download Test Application The AMS sends the OTA server a request for a download of a test application 4 Test Application The OTA server sends the AMS a test application The AMS downloads and executes the test application on the device 5 Send Test Result The test application sends the test results to the OTA server Java ME TCK Framework Developer s Guide July 2007 Example of OTA Test The following is an example of an OTA server component In the example the source of the MIDlet suite is named Test_MIDlet java and the JAD file is Test_MIDlet jad The JAD file contains the manifest file and the Test_MIDlet class CODE EXAMPLE 4 12 Server Test Component Example public class OTATest1 extends OTATest public static final String NAME Test_MIDlet public static final String testCaseID OTA_Sample public Status OTA_Sample Attributes attr new Attributes attr putValue reportURL address TestResult attr putValue testCaseID testCaselID String jadURL address NAME jad log println Installing NAME jad n Status s installWithNotification attr NAME null if s null s isPassed log println Could not install the application return s try
27. B gt lt TD gt lt TD gt sample pkg OTA Test_MIDlet lt TD gt lt TR gt lt TR gt lt TD gt lt B gt keywords lt B gt lt TD gt lt TD gt runtime positive ota lt TD gt lt TR gt lt TD gt lt B gt remote lt B gt lt TD gt lt TD gt networkAgent sample pkg OTA OTATest1l httpServerPort ShttpServerPort testDir testDir OTAHandlerClass SOTAHandlerClass OTAHandlerArgs S OTAHandlerArgs lt TD gt lt TR gt lt TD gt lt B gt remoteSource lt B gt lt TD gt lt TD gt lt A HREF OTATest1 java gt OTATest1 java lt TD gt lt TABLE gt lt BODY gt lt HTML gt The following is an example of a test description for a trusted OTA test See remote Attribute on page 55 for a description of the values that must be set in the remote attribute for a trusted OTA test CODE EXAMPLE 4 11 Trusted OTA Test Description File lt TITLE gt Test Specifications and Descriptions for Test lt TITLE gt lt HEAD gt lt BODY gt lt H1 gt Test Specifications and Descriptions for Test lt H1 gt lt HR gt lt a name Test gt lt a gt 56 Java ME TCK Framework Developer s Guide July 2007 CODE EXAMPLE 4 11 Trusted OTA Test Description File Continued lt TABLE BORDER 1 CLASS TestDescription gt lt TR gt lt TD gt lt B gt title lt B gt lt TD gt lt TD gt Over the Air Test for trusted midlet lt TD gt lt TR gt lt TR gt
28. CHAPTER 4 Writing Tests This chapter describes how to write different types of Java ME technology tests that use Framework resources and to set special properties required by the tests General information about the process of creating tests can be located in Chapters 5 and 6 of the Java Technology Test Suite Development Guide Additional information about writing tests for use with the JavaTest harness can be located in Chapters 3 of the JavaTest Architect s Guide This chapter contains the following sections m Testing Devices With Automated Tests m Testing Communications or Networking With Distributed Tests m Testing User Interfaces With Interactive Tests m Testing Application Provisioning With OTA Tests m Testing Security Constrained Functionality With Security Tests m Adding Resource Files in Tests m Enabling Test Selection Testing Devices With Automated Tests Automated tests are the most common and the most simple of the tests that are run on the device In this configuration the harness running on a PC or a workstation sends an application bundle containing the tests and an agent to the device where they are unpacked by the Application Management Software AMS built into the device and run In this configuration user interaction is not required to run each test See Chapter 2 for a detailed description procedure and example of writing automated tests In addition SimplestTestSuite SimpleTestSuite and Advanc
29. S KYUN microsystems Java ME TCK Framework Developers Guide Version 1 2 Sun Microsystems Inc www sun com July 2007 Copyright 2007 Sun Microsystems Inc 4150 Network Circle Santa Clara California 95054 U S A All rights reserved Test suite developer s guide for the Java ME TCK Framework THIS PRODUCT CONTAINS CONFIDENTIAL INFORMATION AND TRADE SECRETS OF SUN MICROSYSTEMS INC USE DISCLOSURE OR REPRODUCTION IS PROHIBITED WITHOUT THE PRIOR EXPRESS WRITTEN PERMISSION OF SUN MICROSYSTEMS INC U S Government Rights Commercial software Government users are subject to the Sun Microsystems Inc standard license agreement and applicable provisions of the FAR and its supplements Sun Sun Microsystems the Sun logo Java Solaris JavaTest JavaHelp Java Community Process JCP Javadoc JDK and Java Developer Connection are trademarks or registered trademarks of Sun Microsystems Inc in the U S and other countries The Display PostScript logo is a trademark or registered trademark of Adobe Systems Incorporated Products covered by and information contained in this service manual are controlled by U S Export Control laws and may be subject to the export or import laws in other countries Nuclear missile chemical biological weapons or nuclear maritime end uses or end users whether direct or indirect are strictly prohibited Export or reexport to countries subject to U S embargo or to entities identified on U S
30. Service Requested Action Request an Action if interactive Diagram of the interactive test configuration OTA Tests OTA tests are MIDP specific tests that verify the over the air OTA application provisioning implementation This includes obtaining installing and removing applications MIDlet suites and enforcing security requirements Each OTA test has an associated MIDlet suite that you download from the provisioning server and install and launch on the test device Multiple instances of each can run in parallel sharing one OTA server Chapter 1 Introduction 13 OTA tests validate API functionality while the device is connected to a remote host the PC or workstation where the harness runs In this configuration one part of the OTA test runs on the remote host the PC or workstation where the harness runs using a passive agent and the other part of the test runs on the device OTA tests require user interaction as a part of each test FIGURE 1 4 illustrates the Framework configuration for running OTA tests See Testing Application Provisioning With OTA Tests on page 53 in Chapter 4 for information about writing OTA tests and OTA test execution FIGURE 1 4 Framework Configuration for OTA Tests PC or Workstation Device JavaTest Harness Request to Download Test Application OTA i O Test Application Server l O Send Test Result Server Side _ 4 Test Test Applicati
31. TITLE gt Test Specifications and Descriptions lt TITLE gt lt HEAD gt lt BODY gt lt H1 gt Test Specifications and Descriptions for Test lt H1 gt lt HR gt lt a name Test gt lt a gt lt TABLE BORDER 1 SUMMARY JavaTest Test Description CLASS TestDescription gt lt THEAD gt lt TR gt lt TH SCOPE col gt Item lt TH gt lt TH SCOPE col gt Value lt TH gt lt TR gt lt THEAD gt lt TR gt lt TD SCOPE row gt lt B gt title lt B gt lt TD gt lt TD gt Checking constructors lt TD gt lt TR gt lt TR gt lt TD SCOPE row gt lt B gt source lt B gt lt TD gt lt TD gt lt A HREF Test java gt Test java lt A gt lt TD gt lt TR gt lt TR gt lt TD SCOPE row gt lt B gt executeClass lt B gt lt TD gt lt TD gt sample pkg Test lt TD gt lt TR gt lt TR gt lt TD SCOPE row gt lt B gt keywords lt B gt lt TD gt lt TD gt runtime positive lt TD gt lt TR gt Java ME TCK Framework Developer s Guide July 2007 lt TR gt lt TD SCOPE row gt lt B gt selectIf lt B gt lt TD gt lt TD gt isFeatureSupported lt TD gt lt TR gt lt TABI LE gt lt BODY gt lt HTMI L gt Chapter 4 Writing Tests 73 2 Write an interview question that obtains the value of the environment variable from the user To obtain the value of environment variables from users test d
32. TestSuite build directory containing the test suite After creating the updated test suite you must test it see Testing an Updated Simple Test Suite on page 20 before making the test suite available to users 20 Testing an Updated Simple Test Suite The following steps demonstrate how to test an updated test suite General information about testing can be located in Chapter 7 of the Java Technology Test Suite Development Guide To Test an Updated Test Suite After completing the procedures contained in Building an Updated Simple Test Suite on page 19 you must test the updated test suite by executing it with the JavaTest harness Before executing the test suite your development environment must meet the minimum requirements described in Development Environment on page 3 You must also have the following tools installed to test the updated Simple Test Suite m Java ME TCK Framework 1 2 m JavaTest harness 3 2 2 m Wireless Toolkit WTK version 2 5 Note The following commands are shown using Microsoft Windows system prompts and syntax 1 Make the Simple Test Suite root directory your current directory 2 Start the JavaTest harness by using the following command java jar lib javatest jar newDesktop The JavaTest harness displays either the Welcome to JavaTest dialog box or the Quick Start wizard See the JavaTest harness documentation for detailed information about using the JavaTest harness
33. The following is an example of the distributed keyword entry that must be added to the test description file CODE EXAMPLE 4 1 Required Distributed Test Keyword lt TR gt lt TD SCOPE row gt lt B gt keywords lt B gt lt TD gt lt TD gt distributed lt TD gt lt TR gt remote Attribute The remote attribute contains the execution command for the remote test components of the distributed test group The values for these attributes are exported from the configuration interview and defined in the test environment file 44 Java ME TCK Framework Developer s Guide July 2007 The following is an example of the remote attribute entry that must be added to the test description file CODE EXAMPLE 4 2 remote Attribute lt TR gt lt TD gt lt B gt remote lt B gt lt TD gt lt TD gt networkAgent sample pkg SampleDistributedTest msgSwitch StestMsgSwitch lt TD gt lt TR gt In the example the remote attribute execution command is composed of the following values m sample pkg SampleDistributedtTest is the fully qualified name of the remote test class m testMsgSwitch is an environment variable used by the Framework to start the remote test component remoteSource Attribute The remoteSource attribute contains the name of the remote test class of the distributed test group If more than one remote test source file is specified in the test description the names must be separated by white sp
34. Updated Test Suite After completing the procedures contained in Writing an Automated Test on page 15 you must run the build to assemble the updated test suite Before building the updated test suite your development environment must meet the minimum requirements described in Development Environment on page 3 You must also have the following tools installed to build the updated Simple Test Suite m Ant 1 6 5 m Java ME TCK Framework version 1 2 m JavaTest harness version 3 2 2 m WTK Wireless Toolkit version 2 5 or 2 5 1 1 Uncomment and modify the four required properties in SimpleTestSuite build build properties file in accordance with your environment The following is an example of changes for a Windows environment ME_FRAMEWORK_LIB_DTIR D javame_tck_framework_12 lib WIRELESS_TOOLKIT D WTK25 JTHARNESS_JAR D javatest 3 2 2 lib javatest jar JAVAHELP_JAR D javatest 3 2 2 lib javatest jar Note When the commercial version of the JavaTest harness is used JAVAHELP_JAR should point to the commercial version of the javatest jar file There is no need to download the JavaHelp binaries since they are present inside the javatest Jar file 2 Use a terminal window or console to make the build directory your current directory Chapter 2 Writing a Simple Automated Test 19 3 Enter the ant command to invoke the ant build script The build creates a Simple
35. a perform some operation or judge whether or not the implementation passed or failed the test Java ME TCK Framework Developer s Guide July 2007 Java Platform Standard Edition Java SE platform Java Application Manager JAM Java Archive JAR file Java Compatibility Test Tools Java CTT Java Community Process JCP program Java platform libraries Java specification Java Specification Request JSR Java technology Java Technology Compatibility Kit JavaTest harness agent A set of specifications that defines the desktop runtime environment required for the deployment of Java technology applications Java SE platform implementations are available for a variety of platforms but most notably the Solaris and Windows operating systems A native application used to download store and execute applications Another name for JAM is Application Management Software AMS A platform independent file format that combines many files into one Tools documents templates and samples that can be used to design and build TCKs Using the Java CTT simplifies compatibility test development and makes developing and running tests more efficient An open organization of international Java community software developers and licensees whose charter is to develop and revise Java specifications and their associated Reference Implementation RI and Technology Compatibility Kit TCK The class libraries that are defi
36. ace CODE EXAMPLE 4 5 Required Interactive Test Keywords lt TR gt lt TD SCOPE row gt lt B gt keywords lt B gt lt TD gt lt TD gt distributed interactive lt TD gt lt TR gt Interactive Test Execution FIGURE 4 3 and the associated text are for MIDP execution mode In the diagram arrows indicate the direction of dataflow between the device the workstation and the tester The numbered items indicate the sequence and content of the dataflow 50 Java ME TCK Framework Developer s Guide July 2007 FIGURE 4 3 Interactive Test Execution PC or Workstation Device JavaTest Harness 1 getNextApp 27 Application Bundle agent tests Execution Server Passive Agent Server Side Test Check Send Message Messaging Service Requested Action Request an Action if interactive 1 getNextApp The AMS issues a getNextApp command to the execution server on the workstation The AMS implementation is device specific and must be provided by the licensee 2 Application Bundle The execution server sends an application bundle to the AMS The AMS downloads and executes the application bundle on the device Test bundles are created by the Framework 3 getNextTest The agent issues a getNext Test request to the execution server Chapter 4 Writing Tests 51 52 4 Test Name The execution server returns the name of the next test in the application
37. ace The following is an example of the remoteSource attribute entry in which the remote test class is SampleDistributedTest java CODE EXAMPLE 4 3 remoteSource Attribute lt TR gt lt TD gt lt B gt remoteSource lt B gt lt TD gt lt TD gt SampleDistributedTest java lt TD gt lt TR gt executeArgs Attribute The executeArgs attribute contains the environment variables that are passed to the test classes or client test component being executed The values for these attributes are exported from the configuration interview and defined in the test environment When more than one environment variable is specified in the test description the names and values must be separated by white space Chapter 4 Writing Tests 45 The following is an example of the executeArgs attribute entry in which the testMsgSwitch and timeOut environment variables are specified CODE EXAMPLE 4 4 executeArgs Attribute With Multiple Environment Variables lt TR gt lt TD gt lt B gt executeArgs lt B gt lt TD gt msgSwitch S testMsgSwitch timeout timeOut lt TR gt Distributed Test Execution FIGURE 4 2 and the associated text are for MIDP execution mode In the diagram arrows indicate the direction of dataflow between the device and the workstation The numbered items indicate the sequence and the content of the dataflow 46 Java ME TCK Framework Developer s Guide July 2007 FIGURE 4 2 Distributed Test E
38. ass is javasoft sgqe javatest lib MultiTest m Developers must add individual test case methods to the derived test class to create a useful test class 15 16 Each test case method must take no arguments and must return a Status object that represents the outcome of the test case The Status can be either Status passed String or Status failed String Developers must update the runTestCases method to add the test cases that are executed This is a CLDC MIDP specific method the abstract method is defined in CLDC specific MultiTest For CDC specific Multitest this method is not required because reflection is available on CDC stacks Each test in a test suite has a corresponding test description that is typically contained in an HTML file The test description file contains all information required to run the test such as the source file to use source the class to execute executeClass and how the test is executed keyword v To Create a Simple Automated Test The following steps demonstrate how to create an automated test class suitable for CLDC MIDP and CDC Test3 java to create its test description file index html and to update the test class dependencies file testClasses 1st The test class Test 3 java is a simple automated test class This sample uses the SimpleTestSuite located at samples sources SimpleTestSuite not the AdvancedTestSuite This test class does not test a particular API Instead it
39. basic configuration interview that defines parameters common for all Java ME technology test suites If your test suite requires additional parameters you must create a sub interview and link it to the Framework interview In the SampleInterview example the test requires an additional parameter The following steps describe how to create the sub interview that adds the additional parameter to the configuration 30 Java ME TCK Framework Developer s Guide July 2007 a Create a class SampleInterview that is a subtype of the Interview class The following code creates the SampleInterview class public class SampleInterview extends Interview b Identify the new interview SampleInterview as a sub interview of the parent interview In the new sub interview class the constructor must take a reference to the parent interview as an argument and pass this reference to the superclass constructor This identifies the interview as a sub interview of the parent interview The following code identifies SampleInterview as a sub interview of MidpTckBaselInterview public class SampleInterview extends Interview public SampleInterview MidpTckBaseInterview parent throws Fault super parent sample c Optional Identify the resource file and helpset file used by the sub interview By default a sub interview shares a resource file and More Info help files with its parent interview However you
40. bundle that the AMS should run While the tests from the application bundle are loaded onto the device by the AMS the execution server establishes the sequence in which the tests in the bundle are run 5 Check or Send Message The test sends either a message or a check message request to the Framework messaging service on the harness 6 Get Message The Framework messaging service on the harness sends the test a get message command 7 Check or Send Message The server side test sends either a message or a check message request to the Framework messaging service on the harness 8 Get Message The Framework messaging service on the harness sends the server side test a get message command 9 Request an Action The Framework messaging service sends a message request to the user that requires an action on a test 10 Action The user performs the action on the test requested by the messaging service 11 sendTestResult The agent returns the test results to the harness The agent repeats items 3 4 and 5 until all tests in the application bundle are run When all tests in the application bundle are run the AMS requests the next application bundle items 1 and 2 and the sequence is repeated until all tests and all test bundles in the test suite are run Example of an Interactive Test Examples of interactive tests can be viewed and executed in the AdvancedTest Suite directory of the Framework bundle Java ME TCK Framew
41. can choose to use a different resource file and helpset file The following code in the SampleInterview example specifies a different resource file and helpset file public class SampleInterview extends Interview public SampleInterview MidpTckBaseInterview parent throws Fault super parent sample setResourceBundle il8n setHelpSet help sampleInterview In the example i18n is the properties file updated in Step 4 and sampleInterview is the More Info helpset file updated in Step 5 Chapter 3 Test Suite Construction 31 d Use the setFirstQuestion method in the constructor to specify the first question in the sub interview The following setFirstQuestion code in the SampleInterview example specifies the first question public class SampleInterview extends Interview public SampleInterview MidpTckBaseInterview parent throws Fault super parent sample setResourceBundle il8n setHelpSet help sampleInterview setFirstQuestion first e Specify the question type The Question class is a base class that provides the different types of questions required to build the sub interview In the example the question gets string information Therefore the example must use the StringQuestion type The following code in the SampleInterview example specifies the StringQuestion type StringQuestion first new StringQuestion this hello In the example hello is a uni
42. ce This is the primary communication interface for the pluggable communication protocol used by CLDC based agents a midp_agent jar Standard harness agent for MIDP 2 x This agent works in conjunction with an HTTP based communication client midp_httpclnt Jjar Application model MIDlet Communication channel HTTP a midp_agent10 jar Standard harness agent for MIDP 1 0 This agent works in conjunction with an HTTP based communication client midp_httpclnt jar This agent does not support the MIDP 2 x security model Application model MIDlet Communication channel HTTP Appendix B Framework Bundle 83 m cldc_agent jar Standard harness agent for CLDC Application model Main Communication channel pluggable Plug ins The following are the plug in jar files provided in the 1ib directory m cldc_httpsrvr jar Implementation of the server interface for the harness plug in This server works with the HTTP based Client for CLDC a midp_httpsrvr jar Implementation of the server interface for the harness plug in This server works with the HTTP based Client for MIDP m jkssigner jar X 509 signer for MIDP application JAR files DTF The Framework provides the following four functional groupings of jar files for distributed testing CommService CommClients CommServers m Messaging CommService The following are the CommService jar files provided in the 1ib directory m j2me_commClients jJjar
43. compatibility rules A single test can contain multiple test cases Accompanying information can include test documentation auxiliary data files or other resources used by the source code Tests correspond to assertions of the specification A small test that is run as part of a set of similar tests Test cases are implemented using the JavaTest harness MultiTest library class A test case tests a specification assertion or a particular feature or part of a feature of an assertion A class that knows how to execute test classes in different environments Test commands are used by the test script to execute tests Java ME TCK Framework Developer s Guide July 2007 test command template test description test environment test execution model test finder test framework test harness test script test specification test suite A generalized specification of a test command in a test environment The test command is specified in the test environment using variables so that it can execute any test in the test suite regardless of its arguments Machine readable information that describes a test to the test harness so that it can correctly process and run the related test The actual form and type of test description depends on the attributes of the test suite A test description exists for every test in the test suite and is read by the test finder When using the JavaTest harness the test description is a set of test su
44. d tests and reports the results back to the harness m Tests Used in CLDC MIDP and CDC Automated Distributed and OTA test configurations The source code and any accompanying information that exercise a particular feature or part of a feature of a technology implementation to make sure that the feature complies with the Java specification A single test can contain multiple test cases Accompanying information can include test documentation auxiliary data files or other resources required by the source code Tests correspond to assertions of the specification 8 Test Types The developer uses the Framework resources for and organizes the tests based on the test type or testing function for example automated tests must be grouped separately from interactive tests because they use different Framework resources TABLE 1 1 presents a simple matrix of the different test configurations and the types of tests that the Framework resources support TABLE 1 1 Configurations and Supported Test Types Configuration Automated Tests Distributed Tests Interactive Tests OTA Tests CLDC Supported Unsupported Unsupported Unsupported without MIDP MIDP Supported Supported Supported Supported CDC Supported Supported Supported Unsupported Java ME TCK Framework Developer s Guide July 2007 Automated Tests Automated tests for CLDC MIDP and CDC configurations execute on the test device without requiring user interaction Automated
45. e 59 for an example of a server component Client Test Component of an OTA Test The client test component called the MIDlet suite contains the actual test case The MIDlet suite must be in the format of a JAR file or a Java Application Descriptor JAD file so that it can be downloaded from the provisioning server and installed and launched on the test device The creation of the JAR file or JAD file is usually Chapter 4 Writing Tests 53 54 done at the build time If the test writer creates a JAR file or a JAD file with special information about the MIDlet suite as part of the bundle the test writer must also create a manifest file mf Test Description for OTA Tests The test description file for OTA tests must contain the ota keyword the executeClass attribute and the remote attribute The test description might also include other attributes such as the remoteSource and executeArgs attributes See OTA Test Description Examples on page 55 for examples of complete test description files Required OTA Test Keyword The ota keyword identifies the type of test to the harness and enables test selection by the user If more than one keyword is specified the names must be separated by white space The following is an example of the ota keyword entry that must be added to the test description file CODE EXAMPLE 4 6 Required OTA Test Keyword lt TR gt lt TD gt lt B gt keywords lt B gt lt TD gt lt TD gt ota lt
46. e keywords to mark untrusted tests 60 Java ME TCK Framework Developer s Guide July 2007 Trusted Tests Trusted tests verify API implementation behavior for signed MIDlet suites In most cases these tests verify that specification assertions related to signed MIDlet suites are properly implemented Trusted MIDlet suites can be permitted to access APIs that are considered sensitive or to which access is restricted without any user action The required permissions are granted without any explicit user action Trusted tests must not be run in untrusted security mode See Marking Trusted Tests on page 62 for a description of how to use keywords to mark trusted tests Double duty Tests Double duty tests verify API implementation behavior that depends on security factors For example tests for a security sensitive API that require specific permissions to be granted or denied Double duty tests must be run in both the trusted and untrusted security mode See Using an Attribute to Mark Double Duty Tests on page 63 for a description of how to use the DoubleDutySecurity attribute to mark double duty tests Using Keywords to Mark Security Tests When developing security tests tests writer should use an appropriate keyword in the test description to mark the type of test The keyword enables users to select or exclude tests from a test run based on the security mode Marking Untrusted Tests When developing untrusted tests th
47. e test writers should include the untrusted keyword in the test description With the untrusted keyword included in the test description the untrusted test is selected and executed during a test run in untrusted security mode Tests marked with the trusted keyword are not selected and executed in the untrusted security mode The following is an example of an untrusted keyword entry added to a test description file CODE EXAMPLE 4 13 untrusted Keyword Entry in the Test Description lt TR gt lt TD SCOPE row gt lt B gt keywords lt B gt lt TD gt lt TD gt untrusted lt TD gt lt TR gt Chapter 4 Writing Tests 61 62 If other keywords such as positive are used they would be included in the same line as the untrusted keyword The following is an example of a test description file that uses runtime positive and untrusted keywords CODE EXAMPLE 4 14 Test Description for an untrusted Test lt TITLE gt Test Specifications and Descriptions for Test lt TITLE gt lt HEAD gt lt BODY gt lt H1 gt Test Specifications and Descriptions for Test lt H1 gt lt HR gt lt a name Test gt lt a gt lt TABLE BORDER 1 SUMMARY JavaTest Test Description CLASS TestDescription gt lt THEAD gt lt TR gt lt TH SCOPE col gt Item lt TH gt lt TH SCOPE col gt Value lt TH gt lt TR gt lt THEAD gt lt TR gt lt TD SCOPE row gt lt B gt title lt B gt lt TD gt lt TD gt checking untrusted t
48. e user change something with a device which triggers event generation or require the user to verify that a device plays sound or vibrates But most typically interactive tests are used to validate user interface on the device Like distributed tests interactive tests validate API functionality while the device is connected to a remote host the PC or workstation where the harness runs In this configuration one part of the distributed test runs on the device and the other part of the test runs on a remote host the PC or workstation where the harness runs using a passive agent running on in the same VM as the harness Interactive tests are not supported in pure CLDC without MIDP FIGURE 1 3 illustrates the Framework configuration for MIDP interactive tests For CDC the agent is started downloads the tests via the communication channel and executes them without being restarted a single agent runs from the beginning to the end of the test run See Testing User Interfaces With Interactive Tests on page 48 in Chapter 4 for information about writing interactive tests and interactive test execution Java ME TCK Framework Developer s Guide July 2007 FIGURE 1 3 Framework Configuration for Interactive Tests PC or Workstation Device JavaTest Harness 1 getNextApp 27 Application Bundle agent tests Execution Server Passive Agent Server Side Test Check Send Message Messaging
49. ecifications and Descriptions for Test2 lt TITLE gt lt HEAD gt lt BODY gt lt H1l gt Test Specifications and Descriptions for Test2 lt H1 gt lt HR gt lt a name Test2 gt lt a gt lt TABLE BORDER 1 SUMMARY Javatest Test Description CLASS TestDescription gt lt THEAD gt lt TR gt lt TH SCOPE col gt Item lt TH gt lt TH SCOPE col gt Value lt TH gt lt TR gt lt THEAD gt lt TR gt lt TD SCOPE row gt lt B gt title lt B gt lt TD gt lt TD gt Checking image creation lt TD gt lt TR gt lt TR gt lt TD SCOPE row gt lt B gt source lt B gt lt TD gt lt TD gt lt A HREF Test2 java gt Test2 java lt A gt lt TD gt lt TR gt lt TR gt lt TD SCOPE row gt lt B gt executeClass lt B gt lt TD gt lt TD gt sample pkg3 Test2 lt TD gt lt TR gt lt TR gt lt TD SCOPE row gt lt B gt keywords lt B gt lt TD gt lt TD gt runtime positive lt TD gt lt TR gt lt TD SCOPE row gt lt B gt resources lt B gt lt TD gt lt TD gt shared sample pkg3 Duke png lt TD gt lt TR gt lt TABLE gt lt BODY gt lt HTML gt Chapter 4 Writing Tests 69 70 Enabling Test Selection Test selection in a test run is of value to a user when a test suite includes tests for optional features that were not implemented Because the assumption of tests is that the target implementation must support th
50. ecurity attribute with the value set to yes in the test description to mark these tests Tests that have a DoubleDutySecurity attribute with a value of yes in their test description file are selected and executed in both the trusted and the untrusted security modes The following is an example of a DoubleDutySecurity attribute with a value of yes added to a test description file CODE EXAMPLE 4 16 DoubleDutySecurity Attribute lt TR gt lt TD SCOPE row gt lt B gt DoubleDutySecurity lt B gt lt TD gt lt TD gt yes lt TD gt lt TR gt The following is an example of a test description file for a double duty test CODE EXAMPLE 4 17 Test Description for a Double Duty Test lt TITLE gt Test Specifications and Descriptions for Test lt TITLE gt lt HEAD gt lt BODY gt lt H1 gt Test Specifications and Descriptions for Test lt H1 gt lt HR gt lt a name Test gt lt a gt lt TABLE BORDER 1 SUMMARY JavaTest Test Description CLASS TestDescription gt Chapter 4 Writing Tests 63 CODE EXAMPLE 4 17 Test Description for a Double Duty Test Continued lt THEAD gt lt TR gt lt TH SCOPE col gt Item lt TH gt lt TH SCOPE col gt Value lt TH gt lt TR gt lt THEAD gt lt TR gt lt TD SCOPE row gt lt B gt title lt B gt lt TD gt lt TD gt checking double duty test lt TD gt lt TR gt lt TR gt lt TD SCOPE row gt lt B gt source lt B
51. edTest Suite directories contain examples of automated tests 41 Automated Test Execution FIGURE 4 1 and the associated text are for CLDC and MIDP execution mode In the diagram arrows indicate the direction of dataflow between the device and the workstation The numbered items indicate the sequence and the content of the dataflow FIGURE 4 1 Automated Test Execution PC or Workstation Device JavaTest Harness 1 getNextApp 2 Application Bundle agent tests getNextTest Execution Server O Test Name O sendTestResult 1 getNextApp The AMS issues a getNextApp command to the execution server on the workstation The AMS implementation is device specific and must be provided by the licensee 2 Application Bundle The execution server sends an application bundle to the AMS The AMS downloads and executes the application bundle on the device Test bundles are created by the Framework 3 getNextTest The agent issues a getNextTest request to the execution server 4 Test Name The execution server returns the name of the next test in the application bundle that the agent should run 42 Java ME TCK Framework Developer s Guide July 2007 While the tests from the application bundle are loaded onto the device by the AMS the execution server establishes the sequence in which the tests in the bundle are run 5 sendTestResult The agent returns the test results to the execution server
52. em 36 Java ME TCK Framework Developer s Guide July 2007 a Create a help directory under the directory where the interview classes are located For this example use the following location SimpleTestSuite src sample suite help Create a helpset file sampleInterview hs under the help directory The helpset file specifies the location of the map file for the More Info system The following example shows the contents of the sampleInterview hs helpset file used in the Simple Test Suite lt DOCTYPE helpset PUBLIC Sun Microsystems Inc DTD JavaHelp HelpSet Version 1 0 EN lt helpset version 1 0 gt lt title gt lt title gt Simple Test Suite Configuration Interview Help lt title gt lt maps gt lt maps gt lt mapref location default sampleInterview jhm gt lt maps gt lt helpset gt http java sun com products javahelp helpset_1_0 dtd gt In the preceding helpset example sampleInterview jhm is the map file specified for the More Info system You can also view the sampleInterview hs file in the following location SimpleTestSuite src sample suite help Create a default directory under the help directory For the SampleTestSuite example use the following location SimpleTestSuite src sample suite help default Chapter 3 Test Suite Construction 37 d Create a map file sampleInterview jhm in the default directory The JavaHelp sys
53. environment variable to run the test Because the value for sampleValue cannot be known ahead of time it must be provided in the configuration interview by the user The decodeArg String int method decodes the argument and passes the value to the sampleValue environment variable The harness uses the stringValue argument in accordance with the executeArgs argument entry in the test description file 2 Specify the environment value using the test description entry The test description file index html can use either the executeArgs or context to identify the environment variables required by the test In our example the environment variable is called sample string value The following is an example of using the executeArgs argument also see SimpleTestSuite tests sample pkg2 index htm1 lt tr gt lt td scope row gt lt b gt executeArgs lt b gt lt td gt lt td gt stringValue S sample string value lt td gt lt tr gt As an alternative you can use the context field If you use the context field in the test description the argument in the decodeArgs String int method must be changed accordingly with sample string value used as the argument The following is an example of using the context field lt tr gt lt td scope row gt lt b gt context lt b gt lt td gt lt td gt sample string value lt td gt lt tr gt 3 Write the configuration interview class The Framework provides a
54. es not have question text More Info or a getNext method The following code in the SampleInterview example identifies the final question in the sub interview F Question qXXX Question getNext return qgEnd Question qEnd new FinalQuestion this The following SampleInterview java class is used for the example in the Simple Test Suite It puts everything together that was described in Step a through Step j public class SampleInterview extends Interview public SampleInterview MidpTckBaseInterview parent throws Fault super parent sample setResourceBundle il8n setHelpSet help sampleInterview first new StringQuestion this hello first setl Exporter Exporters getStringValueExporter sample string value first linkTo end setFirstQuestion firstQuestion private StringQuestion first private final FinalQuestion end new FinalQuestion this end You can also view the SampleInterview java file in the following location SimpleTestSuite src sample suite Chapter 3 Test Suite Construction 35 4 Update the interview properties resource file All question text is located in the interview properties file associated with the interview class files and is identified by a unique question key The question key is based on a name assigned by the test developer and must uniquely identify the question with the interview Question keys
55. ese features the implementation fails tests that are not be applicable to it This section describes how a developer can enable the filtering of the tests test selection in a test suite Factors and Mechanisms for Test Selection The following factors affect how tests are selected for a test run and describe several mechanisms that exist for users to enable test selection m Keywords A standard test selection mechanism provided by the harness that enables a user to filter tests based on specific descriptive keywords assigned by a developer to a test This mechanism is convenient for logical test grouping The following are examples of test filtering using keywords in the test description file a Type distributed interactive or OTA See Required Distributed Test Keyword on page 44 Required Interactive Test Keywords on page 50 and Required OTA Test Keyword on page 54 a Security mode trusted or untrusted See Using Keywords to Mark Security Tests on page 61 a Custom keywords specified by the user See Appendix C for a list of the Framework keywords m Prior test status A built in mechanism in the harness that enables a user to filter tests based on the previous runs The test developer is not required to perform any action to enable this filtering mechanism Filtering is normally set by the user through the Prior Status question in the standard configuration interview m Exclude list Excludes cer
56. esn t support Connectivity Requirements Each Java technology has a unique set of connectivity requirements When using the Framework resources to develop test suites developers should consider the following connectivity requirements a CLDC without MIDP No connectivity required by specification however the Framework requires bi directional communication a MIDP HTTP is required a CDC Datagram connection is optional in the specification The Framework requires bi directional communication Datagram TCP IP HTTP or custom communications are supported a FP on CDC Full TCP IP is optional in the specification The Framework requires bi directional communication Datagram TCP IP HTTP or custom communications are supported Java ME TCK Framework Developer s Guide July 2007 Resource Limitations Hardware resources on test devices are often limited Resource constrained devices can quit operation when excess native resources are requested The Framework can run tests on resource constrained target devices with limited available memory and persistent storage When developing test suites for the Java ME technology device the developer must be aware of device limitations such as the following and use the appropriate Framework resources m Memory constraints of the device Minimum requirements listed in the appropriate specification for profiles a Maximum number of connections HTTP SSL or Socket TCP IP on the
57. ess it can take considerable time and effort to put the pieces of the TCK puzzle in place Consequently test developers should read the TCK Project Planning Guide and the Java Technology Test Suite Development Guide to understand the principles involved in constructing TCKs and test suites before using this guide These documents provide the following fundamental information about TCK and test suite development m For test developers who are creating a TCK the TCK Project Planning Guide provides general descriptions of the components required for a TCK explanations of the process of creating a TCK descriptions of the resources required to create a TCK and a description of the planning process for a TCK development project m For test developers who are creating a test suite the Java Technology Test Suite Development Guide provides general descriptions of the methods used to create a test suite with its tests and include an overview of test development techniques The chapters in this document are presented in a sequence that developers who are new to the process of creating test suites can follow when creating their own test suite 2 Note The TCK Project Planning Guide and the Java Technology Test Suite Development Guide documents were originally developed as part of a product used by the Java Community Process JCP Specification Leads For that reason these documents refer to JCP procedures that might not apply to a test developer s
58. est lt TD gt lt TR gt lt TR gt lt TD SCOPE row gt lt B gt source lt B gt lt TD gt lt TD gt lt A HREF Test java gt Test java lt A gt lt TD gt lt TR gt lt TR gt lt TD SCOPE row gt lt B gt executeClass lt B gt lt TD gt lt TD gt sample pkg Test lt TD gt lt TR gt lt TR gt lt TD SCOPE row gt lt B gt keywords lt B gt lt TD gt lt TD gt runtime positive untrusted lt TD gt lt TR gt lt TABLE gt lt BODY gt lt HTML Marking Trusted Tests When developing trusted tests the test writers should use the trusted keyword in the test description to mark these tests With the trusted keyword all the trusted tests are selected and executed during the trusted security mode Tests that have the untrusted keyword are not selected and executed in the trusted security mode Java ME TCK Framework Developer s Guide July 2007 The following is an example of adding a trusted keyword entry to a test description file CODE EXAMPLE 4 15 trusted Keyword Entry lt TR gt lt TD SCOPE row gt lt B gt keywords lt B gt lt TD gt lt TD gt trusted lt TD gt lt TR gt When other keywords such as positive are used they are included in the same line as the trusted keyword and separated by white space Using an Attribute to Mark Double Duty Tests When developing double duty tests test writers should use the DoubleDutyS
59. est3 java test class Save this file in the Simple Test Suite source as SimpleTestSuite tests pkg3 index html The index html must be contained in the same directory as its test class in this case Test3 java Update the test class dependency file testClasses 1st in the build directory SimpleTestSuite build by adding the following line that identifies the new test and its class sample pkg3 index html Test3 sample pkg3 Test3 class The test class dependency file testClasses 1st provides information to the Framework test bundling infrastructure regarding which classes should be bundled for each test This information is used in CLDC and MIDP but ignored in CDC Note In Framework version 1 2 the test bundling mechanism was improved so that in simple cases such as this the complete test is in a single file without using other test classes updating the testClasses 1st file is not required After creating the new test class creating the test description file and updating the test class dependency file you must build see Building an Updated Simple Test Suite on page 19 and test the updated test suite see Testing an Updated Simple Test Suite on page 20 before making the test suite available to users 18 Java ME TCK Framework Developer s Guide July 2007 Building an Updated Simple Test Suite The following steps demonstrate how to build an updated test suite after adding a test To Build an
60. estCasel if isSelected testCase2 addStatus testCase2 public Status testCasel if 1 1 2 return Status passed OK else return Status failed 1 1 did not make 2 public Status testCase2 if 2 2 4 Java ME TCK Framework Developer s Guide July 2007 CODE EXAMPLE 3 2 Simple Test Class Continued return Status passed OK else return Status failed 2 2 did not make 4 Additional examples Testl java and Test2 java can be found in the following Simple Test Suite directories m SimpleTestSuite tests sample pkg1 m SimpleTestSuite tests sample pkg2 Test Description file Each subdirectory that contains one or more test classes must also contain a corresponding test description The test description is contained in HTML format The test description file generally contains the following fields m title A descriptive string that identifies what the test does The title appears in reports and in the harness status window m source List of source files of the test When the test sources are listed in this field they can be accessed and viewed from the harness GUI m executeClass Specifies the name of the test s executable class file It is assumed to be located in the classes directory m keywords String tokens that can be associated with a given test They describe attributes or characteristics of the test for example how to execute the test
61. ethod must take no arguments If you need to pass an argument into a test method design a wrapper test case to calculate the argument values and then call the test method with the correct arguments The test case methods must follow the following convention CODE EXAMPLE A 3 Test Case Method public Status methodName 78 Java ME TCK Framework Developer s Guide July 2007 J2MEDistributedTest Class name com sun tck j2me services messagingService J2MEDistributedTest This is the base class for distributed tests for the Java ME technology component of the test Each distributed test has a unique name used for identification during message exchange The send and handleMessage methods can be used to send and receive messages to or from the other named components of the test J2SEDistributedTest Class name com sun tck j2me services messagingService J2SEDistributedTest This is the base class for distributed tests for the Java SE technology component of the test This is a Java SE technology counterpart for J2MEDistributedTest DistribInteractiveTest Class name com sun tck midp lib DistribInteractiveTest This is the base class for the Java SE technology component of the interactive test It is a subclass of J2SEDistributedTest that requires user action before test result is determined Three different user interface types are supported m Yes No The user is required to determine the outcome of the test
62. evelopers must write an interview class and create the required interview questions If an interview class already exists in the test source the developer can either add new questions to the existing interview or create a new sub interview class containing the required questions and link it to the existing interview class For this example we will assume that we have an existing interview class SampleInterview java in the Simple Test Suite source and that we are adding a new question about whether the target implementation supports a specific feature For an example of linking a new interview class to an existing interview see Chapter 3 Creating a Test Suite Configuration Interview on page 28 The following is an example of question code that can be added to an interview The added question Does the implementation support the feature private YesNoQuestion gqFeatureSupport new YesNoQuestion this featureSupport public void clear setValue YesNoQuestion NO protected void export Map data boolean b value YES data put isFeatureSupported String valueOf b In the question code the isFeatureSupported environment variable name must be consistent with the name used in Step 1 for selectIf in the test description file Also in the question code featureSupport is the unique question name that becomes part of the question key created in Step 3 for use in both the map and the properties files
63. export exclusion lists including but not limited to the denied persons and specially designated nationals lists is strictly prohibited DOCUMENTATION IS PROVIDED AS IS AND ALL EXPRESS OR IMPLIED CONDITIONS REPRESENTATIONS AND WARRANTIES INCLUDING ANY IMPLIED WARRANTY OF MERCHANTABILITY FITNESS FOR A PARTICULAR PURPOSE OR NON INFRINGEMENT ARE DISCLAIMED EXCEPT TO THE EXTENT THAT SUCH DISCLAIMERS ARE HELD TO BE LEGALLY INVALID Copyright 2007 Sun Microsystems Inc 4150 Network Circle Santa Clara Californie 95054 Etats Unis Tous droits r serv s CE PRODUIT CONTIENT DES INFORMATIONS CONFIDENTIELLES ET DES SECRETS COMMERCIAUX DE SUN MICROSYSTEMS INC SON UTILISATION SA DIVULGATION ET SA REPRODUCTION SONT INTERDITES SANS L AUTORISATION EXPRESSE ECRITE ET PREALABLE DE SUN MICROSYSTEMS INC Sun Sun Microsystems le logo Sun Java Solaris JavaTest JavaHelp Java Community Process JCP Javadoc JDK et Java Developer Connection sont des marques de fabrique ou des marques diposies de Sun Microsystems Inc aux Etats Unis et dans d autres pays Le logo Display PostScript est une marque de fabrique ou une marque d pos e de Adobe Systems Incorporated Les produits qui font l objet de ce manuel d entretien et les informations qu il contient sont regis par la legislation americaine en matiere de controle des exportations et peuvent etre soumis au droit d autres pays dans le domaine des exportations et importations Les utilisations fi
64. for OTA Tests 54 Required OTA Test Keyword 54 executeClass Attribute 54 remote Attribute 55 OTA Test Description Examples 55 OTA Test Execution 57 Example of OTA Test 59 Testing Security Constrained Functionality With Security Tests 60 Types of Security Tests 60 Untrusted Tests 60 Trusted Tests 61 Double duty Tests 61 Using Keywords to Mark Security Tests 61 Marking Untrusted Tests 61 Contents v Marking Trusted Tests 62 Using an Attribute to Mark Double Duty Tests 63 Granting or Denying Security Permissions 64 Granting Security Permissions 64 Denying Security Permissions 66 Adding Resource Files in Tests 67 Enabling Test Selection 70 Factors and Mechanisms for Test Selection 70 selectIf Test Selection 71 v To Enable Test Selection with the select IF Expression 71 A Test API 77 Test 77 Status 78 MultiTest 78 J2MEDistributedTest 79 J2SEDistributedTest 79 DistribInteractiveTest 79 OTATest 80 B Framework Bundle 81 lib Directory 81 Core 83 CLDC and MIDP Execution 83 CLDC and MIDP Agents and Clients 83 Plug ins 84 DTF 84 CommService 84 CommClients 84 vi Java ME TCK Framework Developer s Guide July 2007 CommServers 85 Messaging 85 Test Export Support Libraries 85 CDC Agents 85 src Directory 86 Framework Server Classes and Interfaces 86 Agent Classes 87 Digital Signer 87 Preverification Script 87 Java ME Technology Version of Harness Classes 87 Basic Interview Classes Contai
65. gnature test 103 standard values 103 structure based testing 104 system configuration 104 T tag test description 104 TCK 104 TCK coverage file 104 Technology Compatibility Kit 104 technology implementation 104 test 104 test agent 104 test cases 104 test command 104 test command template 105 test description 105 test environment 105 test execution model 105 test finder 105 test framework 105 test harness 105 test script 105 test specification 105 test suite 105 WwW white box testing 104 work directory 106 Index 109 110 Java ME TCK Framework Developer s Guide July 2007
66. gt lt LINK REL stylesheet TYPE text css HREF wizard css TITLE Style gt lt head gt lt body gt The test suite architect can provide More Info here explaining the question in more detail lt body gt lt html gt Additional examples of More Info files can be found in the following directory SimpleTestSuite src sample suite help default SampleInterview 6 Create a JAR file containing the configuration interview After creating the interview you must package it into a JAR file for inclusion with the test suite If you include other custom components with your test suite they can be packaged with the interview This is usually done in the build If you successfully ran the build as described in Chapter 2 Building an Updated Simple Test Suite on page 19 you have an example of this JAR file under the lib directory In the example the JAR file is named SimpleTestSuite build lib sample_jt jar 7 Add the JAR file to the classpath entry of the testsuite jtt file The test suite containing the configuration interview and associated tests can now be built See Building a Test Suite on page 40 Chapter 3 Test Suite Construction 39 Building a Test Suite Refer to Chapter 2 Building an Updated Simple Test Suite on page 19 for the procedure to build a Java ME technology test suite by using Framework resources 40 Java ME TCK Framework Developer s Guide July 2007
67. gt source lt B gt lt TD gt lt TD gt lt A HREF Test java gt Test java lt A gt lt TD gt lt TR gt lt TR gt lt TD SCOPE row gt lt B gt executeClass lt B gt lt TD gt lt TD gt sample pkg Test lt TD gt lt TR gt lt TR gt lt TD SCOPE row gt lt B gt deny lt B gt lt TD gt lt TD gt javax microedition io Connector file read lt TD gt lt TR gt lt TABLE gt lt BODY gt lt HTML gt Adding Resource Files in Tests Test writers must sometimes develop tests that require extra resource files such as image data or class files for the test execution When writing tests for CLDC and MIDP based implementations test writers can use the resources entry in the test description file to specify the location of the resource files During test execution the test execution framework bundles the resource files specified in the test description and the test class files listed in the testClasses 1st into a test JAR file Chapter 4 Writing Tests 67 68 For example the following test requires a Duke png to create the Image object for successful execution CODE EXAMPLE 4 22 Public class Test2 public Status testImage Image imgI null String imageDir try imgI Image createImage catch IOException e if imgI null shared sample pkg3 return Status failed Failed no image is created return Status
68. he remote attribute This argument is used by the Framework to sign the MIDlet The following is an example of the remote attribute entry added to the test description file for an OTA test that is written for a trusted MIDlet CODE EXAMPLE 4 9 remote Attribute Entry for Trusted MIDlet lt TR gt lt TD gt lt B gt remote lt B gt lt TD gt lt TD gt networkAgent sample pkg OTA OTATest1 httpServerPort ShttpServerPort testDir testDir OTAHandlerClass SOTAHandlerClass OTAHandlerArgs SOTAHandlerArgs signer jks signer signerArgs Sjks signer args lt TD gt lt TR gt OTA Test Description Examples The following is an example of a basic OTA test description file CODE EXAMPLE 4 10 OTA Test Description File lt TITLE gt Test Specifications and Descriptions for Test lt TITLE gt lt HEAD gt lt BODY gt Chapter 4 Writing Tests 55 CODE EXAMPLE 4 10 OTA Test Description File Continued lt H1 gt Test Specifications and Descriptions for Test lt H1 gt lt HR gt lt a name Test gt lt a gt lt TABLE BORDER 1 CLASS TestDescription gt lt TR gt lt TD gt lt B gt title lt B gt lt TD gt lt TD gt Over the Air Test lt TD gt lt TR gt lt TR gt lt TD gt lt B gt source lt B gt lt TD gt lt TD gt lt A HREF midlet Test_MIDlet java gt midlet Test_MIDlet java lt A gt lt TD gt lt TR gt lt TR gt lt TD gt lt B gt executeClass lt
69. ib Test Basic Interview Classes Containing General Questions The following classes are used as an example of an interview It is possible to build on these classes if the test suite architect or developer is creating a simple test suite and only adding a few questions More complex test suites require additional changes not reflected in the following interview classes Appendix B Framework Bundle 87 n t n t n t n t n t n t n t n t n t com su com su com su com su com su com su com su com su com su k midp in k midp in k midp in k midp in k midp in k midp in k midp in k midp in k midp in terview VmAdvancediInterview terview Vminterview terview MidpTckInterview terview MidpTckBaseInterview terview ConnectionInterview terview Trustedinterview terview MidpCldcTckInterview terview OTAInterview terview SigtestInterview Communication Channel The following classes execute on the server side These classes are closely related to the communication channel but their purpose is primarily in defining the way in which files are bundled in the JAR file com su com su n cldc communication Server Interface n cldc communication TestProvider Generic interface for the test bundler mechanism com sun tck cldc javatest TestBuilder Class that packs test classes and resources into jar files com sun tck cldc javatest TestBundler Bundler that knows how to package severa
70. ide the tests directory of the test suite and organized by package 3 Enter the following HTML code into a text editor or IDE of your choice lt DOCTYPE HTML PUBLIC W3C DTD HTML 4 0 Transitional EN gt lt HTML gt lt HEAD gt lt TITLE gt Test Specifications and Descriptions for Test3 lt TITLE gt lt HEAD gt lt BODY gt lt H1 gt Test Specifications and Descriptions for Test3 lt H1 gt lt HR gt lt a name Test3 gt lt a gt lt TABLE BORDER 1 SUMMARY JavaTest Test Description CLASS TestDescription gt lt THEAD gt lt TR gt lt TH SCOPE col gt Item lt TH gt lt TH SCOPE col gt Value lt TH gt lt TR gt lt THEAD gt lt TR gt lt TD SCOPE row gt lt B gt title lt B gt lt TD gt lt TD gt Hello World test lt TD gt lt TR gt lt TR gt lt TD SCOPE row gt lt B gt source lt B gt lt TD gt lt TD gt lt A HREF Test3 java gt Test3 java lt A gt lt TD gt lt TR gt Chapter 2 Writing a Simple Automated Test 17 lt TR gt lt TD SCOPE row gt lt B gt executeClass lt B gt lt TD gt lt TD gt sample pkg3 Test3 lt TD gt lt TR gt lt TR gt lt TD SCOPE row gt lt B gt keywords lt B gt lt TD gt lt TD gt runtime positive lt TD gt lt TR gt lt TABI LE gt lt BODY gt lt HTM L gt Note The contents of this test description file correspond to the T
71. ided to users as a zip bundle When the bundle is unzipped the following directories and files are placed in the Framework root directory m doc Contains the Java ME TCK Framework Developer s Guide PDF and HTML formats a redistributables Contains the following directories a javame_tck_framework_12 src Contains the source files a javame_tck_framework_12 1ib Contains compiled Framework classes prepackaged into Java Archive JAR files a javame_tck_framework_12 doc javame_tck_framework api Contains the Framework API documentation a samples Contains the following directories a samples binaries Contains prebuilt samples ready for use samples sources Contains sources for the samples and the build files Note Test developers use the sample code contained in samples sources when following the examples in this guide m ReleaseNotes me_framework html Contains additional information about the Framework release m COPYRIGHT me_framework html Contains the copyright notice for the Framework m index html Contains descriptions of the Framework bundles as well as hyperlinks to user documentation provided by the Framework m document css Style sheet used by HTML files See Appendix B for detailed information about the contents of the Framework bundle Framework Components The Framework provides a set of existing components that developers can include in a test suite to create a bridge between a w
72. ify compatibility The JavaTest harness uses exclude list files j tx to filter out of a test run those tests that do not have to be passed The exclude list provides a level playing field for all implementors by ensuring that when a test is determined to be invalid no implementation is required to pass it Exclude lists are maintained by the Maintenance Lead ML and are made available to all technology developers The ML might add tests to the exclude list for the test suite as needed at any time An updated exclude list replaces any previous exclude lists for that test suite The process by which a technology implementor can appeal or challenge a TCK test First level appeals are resolved by the Expert Group responsible for the technology specification and TCK See also appeals process and second level appeals process See test framework Provides application control through the use of graphic images A test description that is embodied in an HTML table in a file separate from the test source file See technology implementation In object oriented programming means to produce a particular object from its class template This involves allocation of a data structure with the types specified by the template and initialization of instance variables with either default values or those provided by the class s constructor function Tests that require some intervention by the user For example the user might have to provide some dat
73. igurations Messaging service Used in CLDC MIDP and CDC Distributed test configuration Interview classes and support classes Used in CLDC MIDP and CDC to create interviews Framework Components on the Device Side The device used to run tests might be a resource constrained device in which available memory and persistent storage are limited The Framework includes the following components for running CLDC MIDP and CDC tests on a device in the Automated Distributed and OTA test configurations AMS Used in CLDC and MIDP Application management code required on the target device to receive the bundle with the test execution agent and the tests from the harness is called the Application Management Software AMS In some contexts AMS is referred to as Java Application Manager JAM Agent Used in CLDC MIDP and CDC Automated and Distributed test configurations Chapter 1 Introduction 7 An agent is a separate program that works in conjunction with the harness to run tests on the device In CDC the agent is started once and when started in a loop it uses the communication channel to the harness to download the test classes and to execute them on the device and then reports back the results In CLDC and MIDP configurations the execution server supplies the test bundle it includes the test agent and the tests and the device s AMS fetches the bundle and then executes the test agent which in turn executes the bundle
74. ion from the user about the computing environment in which the TCK is being run This information is used to produce a test environment that the JavaTest harness uses to execute tests Files used by the JavaTest harness to configure individual test runs The JavaTest harness uses the file name extension jti to store test harness configuration templates Information about your computing environment required to execute a TCK test or tests The JavaTest harness uses a configuration interview to collect configuration values Glossary 99 100 distributed tests domain equivalence class partitioning exclude list first level appeals process framework Graphical User Interface GUI HTML test description implementation instantiation interactive tests Tests consisting of multiple components that are running on both the device and the JavaTest harness host Dividing test components between the device and JavaTest harness is often used for tests of communication APIs tests that are heavily dependent on external resources tests designed to run on devices with constrained resources such as a small display and data transfer tests See security domain A test case development technique that entails breaking a large number of test cases into smaller subsets with each subset representing an equivalent category of test cases A list of TCK tests that a technology implementation is not required to pass in order to cert
75. ion names Specifies MIDP permissions that must be denied for this test application If tests are run as trusted MIDlets these permissions are not included in the MIDlet Permissions attribute for the test MIDlet If tests are run as untrusted MIDlets the test can be filtered out if the security policy has all of these permissions granted in the untrusted domain Specifies a condition that must be satisfied for the test to be executed This field is constructed using environment values Java programming language literals and the full set of Boolean operators lt gt lt gt amp Example integerValue gt 4 amp display my_computer 0 If the Boolean expression evaluates to false the test is not run If the expression evaluates to true the test is run If any of the values are not defined in the test environment the harness considers the test to be in error Appendix C Test Description Fields and Keywords 93 TABLEC 1 Framework Test Description Fields Continued Field Description remote Contains information required to run distributed network tests resources Contains the location of the resource files that are needed by the test class The resources files can be image data or class files Separate multiple resources with white space remoteSource Contains the source name of the remote test class of this distributed test group Separate multiple remote source files with white space
76. ion such as Connected Limited Device Configuration CLDC or Connected Device Chapter 1 Introduction 3 4 Configuration CDC and a profile such as Mobile Information Device Profile MIDP Foundation Profile FP Personal Basis Profile PBP or Personal Profile PP Configuration CLDC and CDC are configurations that provides a basic set of libraries and virtual machine features that must be present in an implementation of a Java ME environment When coupled with one or more profiles the configuration provides developers with a stable Java platform for creating applications for consumer and embedded devices Each configuration supports optional packages that enable product designers to balance the functionality needs of a design against its resource constraints m Profile A set of standard APIs that support a category of devices for a specific configuration A specific profile is combined with a corresponding configuration to provide a complete Java application environment for the target device class Optional packages A set of technology specific APIs that extends the functionality of a Java application environment Optional packages provide specific areas of functionality The ability to specify bundles enables test developers to match software and hardware capabilities They can use APIs that provide easy access to the components that a device has without the overhead of APIs designed for capabilities the device do
77. iption file provides the harness with critical information required to run the specified test Test description files contain fields that supply the following information to the harness m Source files that belong to the test m Class or executable to run m Information to determine how to run the test 91 TABLE C 1 lists the test description fields supported by the Framework and describes how their values are used by the harness when the tests are run TABLEC 1 Framework Test Description Fields Field Description title A descriptive string that identifies what the test does The title appears in reports and in the harness status window source For compiler tests contains the names of the files that are compiled during the test run For runtime tests contains the names of the files previously compiled to create the test s class files Precompiled class files are included with the test suite Source files are included for reference only Source files are often java files but can also be jasmor jcod files e jasm is a low level bytecode assembler that assembles class files containing sets of bytecodes that are unusual or invalid for use in runtime tests e jcod is a class level assembler that builds classes with unusual or invalid structure for use in runtime tests These tools are used to generate class files that cannot be reliably generated by a Java programming language compiler For most XML parser tests in the test s
78. irst level appeals process A set of permissions that define what an application is allowed to do in relationship to restricted APIs and secure communications The set of permissions that a technology implementation or Application Programming Interface API requires an application to have for the application to access the implementation or API A text representation of the set of public features provided by an API that is part of a finished TCK It is used as a signature reference during the TCK signature test for comparison to the technology implementation under test Checks that all the necessary API members are present and that there are no extra members that illegally extend the API It compares the API being tested with a reference API and confirms if the API being tested and the reference API are mutually binary compatible See Java specification A configuration value used by the JavaTest harness to determine which tests in the test suite to run and how to run them The user can change standard values using either the all values or standard values view in the Configuration Editor Glossary 103 104 structure based testing system configuration tag test description Technology Compatibility Kit TCK TCK coverage file technology implementation test agent test test cases test command A set of test development methodologies that are based on the internal structure or logic of the system under test not
79. istributed test must extend the com sun tck j2me services messagingService J2MEDistributedTest class In the CDC case the client test component of the distributed test must extend the com sun tck j2me services messagingService CDCDistributedTest class These are the base classes for distributed tests for the Java ME technology component of the test Each component of the distributed test has a unique name used for identification during message exchange Chapter 4 Writing Tests 43 The client test component must use the send and handleMessage methods in the J2MEDistributedTest class to send and receive messages to or from the other named components of the distributed test Remote Test Component The remote test component of the distributed test must extend the com sun tck j2me services messagingService J2SEDistributedTest class and to send messages to the client component of the distributed test invoke the send method in the DistributedTest Test Description for Distributed Tests The test description file for the distributed test must contain the distributed keyword and the remote attribute The test description might also include additional attributes such as the remoteSource and executeArgs attributes Required Distributed Test Keyword The distributed keyword identifies the type of test to the harness and enables test selection by the user If more than one keyword is specified the names must be separated by white space
80. ite specific name value pairs in either HTML tables or Javadoc tool style tags One or more test command templates that the test script uses to execute tests and a set of name value pairs that define test description entries or other values required to run the tests The steps involved in executing the tests in a test suite The test execution model is implemented by the test script When using the JavaTest harness a nominated class or set of classes that read verify and process the files that contain test descriptions in a test suite All test descriptions that are located are handed off to the JavaTest harness for further processing Software designed and implemented to customize a test harness for a particular test environment In many cases test framework components must be provided by the TCK user In addition to the test harness a test framework might or might not include items such as a configuration interview Java Application Manager JAM test agent test finder test script and so forth A test framework might also include other user supplied software components plug ins to provide support for implementation specific protocols The applications and tools that are used for test execution and test suite management The JavaTest harness is an example of a test harness A Java technology software class whose job it is to interpret the test description values run the tests and report the results back to the JavaTest harne
81. j2me_communication jar cldc_httpsrvr jar midp_httpsrvr jar Messaging jkssigner jar j2me_messaging jar CommServers tckbuild jar a j2mefw_jt jar g M datagram_comServer jar j2me_httpCommServer ar interviewlib jar CDC Agents main_agent jar xlet_agent jar applet_agent jar a tckbuild jar Utility classes used by TCK build m j2me tck framework_12 txt Framework version marker this file has no content m midptck ks Standard keystore containing default certificates used for MIDP application signing 82 Java ME TCK Framework Developer s Guide July 2007 Core The following are core Framework files provided in the 1ib directory m j2mefw_jt jar Harness plug in code Most of the Java SE platform code to support ME TCKs is in this file m interviewlib jar Helper library for interview creation CLDC and MIDP Execution The Framework provides the following two functional groupings of CLDC and MIDP execution files a Agent and client jar files a Plug in jar files CLDC and MIDP Agents and Clients The following are CLDC and MIDP agent and client jar files provided in the lib directory a midp_xprtclnt jar Client for use in Test Export mode This client does not implement a communication channel with a remote machine m midp_httpclnt jar HTTP based communication client for MIDP m cldc_httpclnt jar HTTP based communication client for CLDC m client jar Client interfa
82. k j2me interview lib package API documentation The following code in the SampleInterview example implements error checking StringQuestion first new StringQuestion this hello public boolean isValueValid return value null amp amp value public void export Map map map put sample string value String valueOf value Chapter 3 Test Suite Construction 33 h Use the getNext method to determine the next question in the sub interview Every question except the Final question must provide a getNext method that determines the next successor question or null Alternatively if the Framework s interview extension library is used it s possible to link questions without subclassing via Question setPathResolver or Question 1linkTo methods For detailed information see the com sun tck j2me interview 1lib package API documentation The following code in the SampleInterview example specifies the next configuration question Question qXXX Question getNext return qNextQuestion 1 i Repeat Step e through Step h until all configuration questions are added to the sub interview 34 Java ME TCK Framework Developer s Guide July 2007 j Use the FinalQuestion marker to identify the last question in the sub interview At the end of the sub interview have the last question return an instance of FinalQuestion FinalQuestion is only a marker and do
83. k keywords and test description fields Test developers can also refer to the JavaTest Architect s Guide for more in depth explanations about creating test suites that run on the JavaTest harness TheJavaTest Architect s Guide is divided into two parts Part I The Basics is useful for aspiring test suite architects and includes basic topics such as a tutorial that introduces the JavaTest GUI as well as descriptions of test suite components and basic processes of creating test suites tests and configuration interviews that work with the JavaTest harness Part II Advanced Topics includes more in depth information about working with the JavaTest harness such as developing test scripts that run tests and using JavaTest harness standard commands that configure an environment for running test suites on specific test platforms Java ME TCK Framework Developer s Guide July 2007 Development Environment The following tools and resources are the minimum software requirements for the Framework development environment m JDK version 5 0 or later m JavaTest harness version 3 2 2 or later For TCK development download the latest Java Compatibility Test Tools Java CTT from the Java Community Process JCP program web site Use of the Framework The Framework is a bundled set of resources used by test suite developers and test writers to create tests and test suites for Java ME technology implementations as well as to provide a bridge bet
84. l tests and the agent ina jar file This class keeps track of all resources conflicts and jar file size limits n tc class n com su to fetch com sun tc and resourc k cldc javatest util ClassPathReader Utility class used e files from the classpath k cldc javatest util JarBuilder JAR file builder This is a utility class that creates jar files com sun tc Implementation o com sun tc of the Communication Channel com sun tc Implementation o com sun tc Communicati com sun on MIDP CC k cldc co k cldc co k cldc co mmunication TestResultListener f the Communication Channel mmunication GenericTestBundle Implementation mnunication GenericTestProvider f the Communication Channel k cldc co mmunication TestBundle Implementation of the on Channel k midp javatest MessageClient Support for distributed tests 88 Java ME TCK Framework Developer s Guide July 2007 This class provides an implementation of the harness distribution mechanism on top of HTTP a com sun tck midp javatest MidBundler MIDP specific extension that builds on top of the generic CLDC mechanism a com sun tck midp javatest MessageSwitch Support for distributed tests on MIDP This class provides an implementation of the harness distribution mechanism on top of HTTP a com sun tck midp javatest RemoteManager Support for distributed tests on MIDP
85. n the harness sends the server side test a get message command 9 sendTestResult The agent returns the test result to the execution server The agent repeats items 3 4 and 9 until all tests in the application bundle are run When all tests in the application bundle are run the AMS requests the next application bundle items 1 and 2 and the sequence is repeated until all tests and all test bundles in the test suite are run 48 Testing User Interfaces With Interactive Tests Interactive tests are the tests that require some form of user interaction and cannot be executed without such interaction From a design point of view interactive tests are a subtype of distributed test As a subtype of distribute test interactive tests generally execute on the test device under the control of a component called an agent However unlike distributed tests interactive tests also require some form of user interaction as a part of the test Interactive tests might require that the user change something with a device which triggers event generation or require the user to verify that a device plays sound or vibrates But most typically interactive tests are used to validate user interface on the device The Framework supports the following three types of interactive modes m Yes No The user must determine the outcome of the test a Done The user must perform specific actions and confirm the completion but cannot affect the outcome of the
86. nal information about creating configuration interviews for use with the JavaTest harness can be located in Chapter 6 of the JavaTest Architect s Guide The configuration interview consists of a series of questions about the test configuration or specific target implementation which the user must answer before running tests The Configuration Editor displays the configuration interview in the harness and exports the user s answers in a test environment object from which the harness and the Framework can access and obtain the data For example if a test must get the hostName and the portNumber during the test run the following configuration conditions must be satisfied for the test to run m The test description file must include an appropriate executeArgs argument The following is an example of how executeArgs might be specified in the test description file lt tr gt lt td scope row gt lt b gt executeArgs lt b gt lt td gt lt td gt host S testHost port testPort lt td gt lt tr gt 28 Java ME TCK Framework Developer s Guide July 2007 In the test description file StestHost and testPort are test environment variables and are replaced by actual values obtained from the test environment The test suite configuration interview must include a corresponding and appropriate question asking the user to specify the values of the host and port number The configuration interview creates entries in the test e
87. nales ou utilisateurs finaux pour des armes nucleaires des missiles des armes biologiques et chimiques ou du nucleaire maritime directement ou indirectement sont strictement interdites Les exportations ou reexportations vers des pays sous embargo des Etats Unis ou vers des entites figurant sur les listes d exclusion d exportation americaines y compris mais de maniere non exclusive la liste de personnes qui font objet d un ordre de ne pas participer d une facon directe ou indirecte aux exportations des produits ou des services qui sont regi par la legislation americaine en matiere de controle des exportations et la liste de ressortissants specifiquement designes sont rigoureusement interdites LA DOCUMENTATION EST FOURNIE EN L ETAT ET TOUTES AUTRES CONDITIONS DECLARATIONS ET GARANTIES EXPRESSES OU TACITES SONT FORMELLEMENT EXCLUES DANS LA MESURE AUTORISEE PAR LA LOI APPLICABLE Y COMPRIS NOTAMMENT TOUTE GARANTIE IMPLICITE RELATIVE A LA QUALITE MARCHANDE A L APTITUDE A UNE UTILISATION PARTICULIERE OU A L ABSENCE DE CONTREFACON eo ie 4 Adobe PostScript 2 Contents Preface xv Introduction 1 Getting Started 1 Development Environment 3 Use of the Framework 3 Target Environment 3 Connectivity Requirements 4 Resource Limitations 5 Framework Bundle 6 Framework Components 6 Framework Components on the Harness Side 7 Framework Components on the Device Side 7 Test Types 8 Automated Tests 9 Distributed Tests 10 Interactive Te
88. ned for each particular version of a Java technology in its Java specification A written specification for some aspect of Java technology The actual descriptions of proposed and final technology specifications for the Java platform A Java specification and its Reference Implementation RI Examples of Java technologies are the J ava SE platform the Connected Limited Device Configuration CLDC and the Mobile Information Device Profile MIDP See Technology Compatibility Kit TCK A test agent supplied with the JavaTest harness to run TCK tests on a Java technology implementation where it is not possible or desirable to run the main JavaTest harness See also test agent active agent and passive agent Glossary 101 JavaTest harness keywords Maintenance Lead ML methods MultiTest namespace object oriented objects packages passive agent prior status Profile specification 102 A test harness developed by Sun to manage test execution and result reporting for a Technology Compatibility Kit TCK The harness configures sequences and runs test suites The JavaTest harness provides flexible and customizable test execution It includes everything a test architect needs to design and implement tests for implementations of a Java specification Used to direct the JavaTest harness to include or exclude tests from a test run Keywords are defined for a test by the test suite architect The person respo
89. networking subsystem that can be open at any one time m Graphics and image subsystem limits CLDC Target Device The CLDC configuration provides a Java platform for network connected devices that have limited processing power memory and graphical capability such as cellular phones pagers low end personal organizers and machine to machine equipment CLDC can also be deployed in home appliances TV set top boxes and point of sale terminals CLDC target devices typically have the following capabilities a A 16 bit or 32 bit processor with a minimum clock speed of 16 megahertz m At least 160 kilobytes of non volatile memory allocated for the CLDC libraries and virtual machine m At least 192 kilobytes of total memory available for the Java platform m Low power consumption often operating on battery power m Connectivity to a network often through an intermittent wireless connection with limited bandwidth CDC Target Device The CDC configuration provides a Java platform for network connected consumer and embedded devices including smart communicators pagers high end PDAs and set top boxes CDC target devices typically have the following capabilities m A 32 bit microprocessor or controller m 2 megabytes of RAM available to the Java application environment m 2 5 megabytes of ROM available to the Java application environment Chapter 1 Introduction 5 6 Framework Bundle The Framework resources are packaged and prov
90. ning General Questions 87 Communication Channel 88 doc Directory 89 Test Description Fields and Keywords 91 Test Description Fields 91 Keywords 94 Glossary 97 Index 107 Contents vii viii Java ME TCK Framework Developer s Guide July 2007 GURE 1 1 GURE 1 2 GURE 1 3 GURE 1 4 GURE 4 1 GURE 4 2 GURE 4 3 GURE 4 4 Figures Framework Configuration for Standard Automated Tests 10 Framework Configuration for Distributed Tests 11 Framework Configuration for Interactive Tests 13 Framework Configuration for OTA Tests 14 Automated Test Execution 42 Distributed Test Execution 47 Interactive Test Execution 51 OTA Test Execution 58 x Java ME TCK Framework Developer s Guide July 2007 TABLE 1 1 TABLE C 1 TABLE C 2 Tables Configurations and Supported Test Types 8 Framework Test Description Fields 92 Framework Keywords 94 xi xii Java ME TCK Framework Developer s Guide July 2007 CODE EXAMPLE 3 1 CODE EXAMPLE 3 2 CODE EXAMPLE 4 1 CODE EXAMPLE 4 2 CODE EXAMPLE 4 3 CODE EXAMPLE 4 4 CODE EXAMPLE 4 5 CODE EXAMPLE 4 6 CODE EXAMPLE 4 7 CODE EXAMPLE 4 8 CODE EXAMPLE 4 9 CODE EXAMPLE 4 10 CODE EXAMPLE 4 11 CODE EXAMPLE 4 12 CODE EXAMPLE 4 13 CODE EXAMPLE 4 14 CODE EXAMPLE 4 15 CODE EXAMPLE 4 16 CODE EXAMPLE 4 17 CODE EXAMPLE 4 18 Code Examples Simple Test Suite testsuite jtt File 24 Simple Test Class 26 Required Distributed Test Keyw
91. nsible for maintaining an existing Java specification related Reference Implementation RI and Technology Compatibility Kit TCK The ML manages the TCK appeals process exclude list and any revisions needed to the specification TCK or RI Procedures or routines associated with one or more classes in object oriented languages A JavaTest harness library class that enables tests to include multiple test cases Each test case can be addressed individually in a test suite exclude list A set of names in which all names are unique A category of programming languages and techniques based on the concept of objects which are data structures encapsulated with a set of routines called methods that operate on the data In object oriented programming objects are unique instances of a data structure defined according to the template provided by its class Each object has its own values for the variables belonging to its class and can respond to the messages methods defined by its class A namespace within the Java programming language It can have classes and interfaces A package is the smallest unit within the Java programming language A type of test agent that must wait for a request from the JavaTest harness before they can run tests The JavaTest harness initiates connections to passive agents as needed See also test agent active agent and JavaTest harness agent A JavaTest harness filter used to restrict the set of tests in a
92. nvironment from user answers as name value pairs The value of testHost and testPort are defined in the configuration from those user answers Users can display the test environment from within the harness by choosing Configure gt Show Test Environment from the Test Manager menu bar The following is an example of name value pairs that the configuration interview might create in a configuration file from user answers testHost 129 42 1 50 testPort 8080 v To Create a Configuration Interview Through the Interview Class The following procedure uses the Test2 java test class as an example to demonstrate how to accomplish the following tasks Create a configuration interview through the interview class Export an environment variable Specify the environment value by using context or executeArgs Decode the argument into the test class SimpleTestSuite tests sample pkg2 Test2 java is a simple test class created to demonstrate the basic principle of writing a test that uses a configuration interview In addition to the procedures provided in this section refer to Chapter 6 of the JavaTest Architect s Guide for additional information regarding creating configuration interviews Chapter 3 Test Suite Construction 29 1 Define the environment variable required by the test In the example Test2 java the test case SampleStringValue checks if the sampleValue that is passed in equals Hello The harness requires the sampleValue
93. on Passive Agent D Request an Action We ee ee ee He HHH 1 1 1 1 1 1 1 1 2 Action Diagram of the OTA test configuration 14 Java ME TCK Framework Developer s Guide July 2007 CHAPTER 2 Writing a Simple Automated Test Automated test execution is the most common and simple of the test configurations that are run on the test device User interaction is not required to run automated tests The following instructions describe how to create a simple automated test and add it to an existing Framework supplied test suite This chapter contains the following sections presented in the sequence that a test developer follows when writing a test that is added to an existing test suite a Writing an Automated Test m Building an Updated Simple Test Suite m Testing an Updated Simple Test Suite Writing an Automated Test This chapter provides the source code and instructions for creating a simple automated test General information about the process of creating tests can be located in Chapters 5 and 6 of the Java Technology Test Suite Development Guide When creating an automated test test developers must consider the following factors m Automated tests must extend com sun tck cldc 1ib MultiTest a base class for tests with multiple sub test cases This example is valid for CLDC MIDP and CDC automated tests For tests that are to be executed only on a CDC stack and never executed on CLDC MIDP the base cl
94. on file CODE EXAMPLE 4 20 deny Attribute in the Test Description lt TR gt lt TD SCOPE row gt lt B gt deny lt B gt lt TD gt lt TD gt javax microedition io Connector file read lt TD gt lt TR gt The value of the deny attribute is a list of space separated permissions that must be denied by the security policy for this test application to execute and pass If the permissions are granted in the protection domain for the test application the test application must be filtered out and not run The following is an example of a test description file that includes the deny attribute and permissions CODE EXAMPLE 4 21 Test Description That Denies Permissions for a Security Test lt TITLE gt Test Specifications and Descriptions for Test lt TITLE gt lt HEAD gt lt BODY gt lt H1 gt Test Specifications and Descriptions for Test lt H1 gt Java ME TCK Framework Developer s Guide July 2007 CODE EXAMPLE 4 21 Test Description That Denies Permissions for a Security Test lt HR gt lt a name Test gt lt a gt lt TABLE BORDER 1 SUMMARY Javatest Test Description CLASS TestDescription gt lt THEAD gt lt TR gt lt TH SCOPE col gt Item lt TH gt lt TH SCOPE col gt Value lt TH gt lt TR gt lt THEAD gt lt TR gt lt TD SCOPE row gt lt B gt title lt B gt lt TD gt lt TD gt checking deny permission lt TD gt lt TR gt lt TR gt lt TD SCOPE row gt lt B
95. oning of exported tests CDC Agents The following are CDC agent files provided in the 1ib directory main_agent jar JavaTest agent for Foundation Profile Personal Basis Profile and Personal Profile Application model Main Pluggable communication channel xlet_agent jar JavaTest agent for Personal Basis Profile and Personal Profile Appendix B Framework Bundle 85 Application model Xlet Pluggable communication channel m applet_agent jar JavaTest agent for Personal Profile Application model Applet Pluggable communication channel src Directory The src directory contains the Framework test sources test descriptions precompiled Framework classes and scripts for tests precompilation Framework classes consist of server agent interview and communication channel source files Framework Server Classes and Interfaces The following are Framework server classes and interfaces provided in the src directory com su string c com su com su the JAD file com su n cldc co onstants n cldc co n cldc co interfac n cldc co mmu mmu mmu e mmu nication nication nication nication midp HttpConstants HTTP code and midp SuiteSigner Used to sign JAR files midp ContentHandler Used to generate midp DefaultContentHandler Content handler generating default JAD file A custom content handler can be substituted if device specific or technolog
96. only when it s not possible to write simple automated tests FIGURE 1 2 illustrates the Framework configuration for running CLDC and MIDP distributed tests Distributed tests are currently not supported in CLDC without MIDP For CDC the agent is started downloads the tests via the communication channel and executes them without being restarted a single agent runs from the beginning to the end of the test run See Testing Communications or Networking With Distributed Tests on page 43 in Chapter 4 for information about writing distributed tests and distributed test execution FIGURE 1 2 Framework Configuration for Distributed Tests PC or Workstation Device JavaTest Harness 1 getNextApp Execution Server Passive Agent Server Side Test Check Send a Message Tests QO Get Message Messaging Service Diagram of the distributed test configuration Chapter 1 Introduction 11 12 Interactive Tests Interactive tests are the tests that require some form of user interaction and cannot be executed without such interaction From a design point of view interactive tests are a subtype of distributed test As a subtype of distribute test interactive tests generally execute on the test device under the control of a component called an agent However unlike distributed tests interactive tests also require some form of user interaction as a part of the test Interactive tests might require that th
97. ord 44 remote Attribute 45 remoteSource Attribute 45 executeArgs Attribute With Multiple Environment Variables 46 Required Interactive Test Keywords 50 Required OTA Test Keyword 54 executeClass Attribute Entry 54 remote Attribute Entry 55 remote Attribute Entry for Trusted MIDlet 55 OTA Test Description File 55 Trusted OTA Test Description File 56 Server Test Component Example 59 untrusted Keyword Entry in the Test Description 61 Test Description for an untrusted Test 62 trusted Keyword Entry 63 DoubleDutySecurity Attribute 63 Test Description for a Double Duty Test 63 grant Attribute Entry and Security Permissions 65 xiii CODE EXAMPLE 4 19 CODE EXAMPLE 4 20 CODE EXAMPLE 4 21 CODE EXAMPLE 4 22 CODE EXAMPLE 4 23 CODE EXAMPLE 4 24 CODE EXAMPLE A 1 CODE EXAMPLE A 2 CODE EXAMPLE A 3 Test Description That Grants Permissions for a Security Test 65 deny Attribute in the Test Description 66 Test Description That Denies Permissions for a Security Test 66 Test That Requires an Image Resource 68 resources Attribute in the Test Description 68 Test Description That Includes Resources 69 run Method 77 Definition of main 78 Test Case Method 78 xiv Java ME TCK Framework Developer s Guide July 2007 Preface This guide describes how to use resources from the Java Platform Micro Edition Technology Configuration Kit Framework Framework to develop and configure test suites and tests for Java Pla
98. ork Developer s Guide July 2007 Testing Application Provisioning With OTA Tests Over the air OTA tests are MIDP specific tests that verify an OTA application provisioning implementation This includes obtaining installing and removing applications MIDlet suites and enforcing security requirements Each OTA test has an associated application a MIDlet suite that is downloaded from the provisioning server and is installed and launched on the test device When developing an OTA test a test writer must write classes for both the server and client test components as well as create an appropriate test description file Server Component of an OTA Test The server component of the OTA tests must extend the com sun tck midp lib OTATest class This is the base class for OTA tests and provides convenience methods that install execute and remove applications called MIDlet suites There are different kinds of OTA tests with each kind of OTA test having one or more of the following functional requirements m Requires the successful installation of the MIDlet suite m Requires that the MIDlet suite installation fail m Executes the MIDlet suite m Does not execute the MIDlet suite After the execution of the OTA test the installed MIDlet suite must be removed The server component of the typical OTA test invokes the install run and remove methods to install execute and remove the MIDlet suite See Example of OTA Test on pag
99. orkstation or PC running the harness and a device containing the application under test As a bridge Framework components plug into the harness and the device Java ME TCK Framework Developer s Guide July 2007 Framework Components on the Harness Side The following components are used for running CLDC MIDP and CDC tests in the Distributed OTA and Automated test configurations with the harness Execution server Used in CLDC and MIDP Distributed and Automated test configurations The execution server contains no internal test related logic Its only function is to forward data It is as lightweight as possible Test provider In addition to the execution server a test provider acts as a server to the execution server The execution server knows its test provider and calls its methods to pass the data from the client to the test provider and vice versa OTA provisioning server Used in the MIDP OTA test configuration The OTA provisioning server supplies applications over the air to wireless devices Passive agent Used in CLDC MIDP and CDC Distributed and OTA test configurations An agent is a Java SE side component that works in conjunction with the harness to run server side parts of the tests on Java SE on the same or different system that is running the harness Passive agents wait for a request from the harness before running tests Server side test Used in CLDC MIDP and CDC Distributed and OTA test conf
100. passed OKAY Test That Requires an Image Resource imageDir Duke png For this test example the following is a resources entry that might be added to the test description file CODE EXAMPLE 4 23 resources Attribute in the Test Description lt TR gt lt TD SCOPE row gt lt B gt resources lt B gt lt TD gt lt TD gt shared sample pkg3 Duke png lt TD gt lt TR gt In the resources entry the value of the resou rce shared sample pkg3 Duke png is the qualified name of the resource file If multiple resources are added to the test description file separate them with white space If the resource file is a class file the file name must include the class extension During build time the Duke png image file is copied into the specified destination directory shared sample pkg3 Duke png During test execution the execution framework checks the testClasses 1st file and the test URL During the test execution the framework bundles the resource files specified in the resource entry shared sample pkg3 in testClasses 1st into a test JAR file Java ME TCK Framework Developer s Guide July 2007 Duke png and the class files listed The following is an example of a test description file containing a resources entry directory and file name CODE EXAMPLE 4 24 Test Description That Includes Resources lt HTML gt lt HEAD gt lt TITLE gt Test Sp
101. que id that identifies the specific question name This name is used in the properties and map files 32 Java ME TCK Framework Developer s Guide July 2007 f Implement exporting the configuration value to the test environment One of the goals of the interview is to export the configuration value to the test environment Each question has an export method that is used for this purpose The following code in the SampleInterview example exports the value to the test environment StringQuestion first new StringQuestion this hello public void export Map map map put sample string value String valueOf value An alternative is to use the setExporter method to export the value The following code is an example of using the setExporter method first new StringQuestion this hello first setExporter Exporters getStringValueExporter sample string value g Implement error checking for the question answer If the user provides an invalid answer to a question the interview cannot proceed For most questions error conditions are handled by returning null which causes the Configuration Editor to display an invalid response message in red at the bottom of the question pane Alternatively if the Framework s interview extension library is used it s possible to implement validation without subclassing the question via Question setValidator For detailed information see the com sun tc
102. r the new question SampleInterview featureSupport smry Feature Support SampleInterview featureSupport text Does your system support the feature The smry and text entries contain elements used to perform the following functions a SampleInterview featureSupport is a unique question key that the Configuration Editor uses to identify the required question and its corresponding More Info topic Chapter 4 Writing Tests 75 Question keys are created in the following form interview class name question name m The smry entry specifies the question title in the example Feature Support that the Configuration Editor displays to the user m The text entry specifies the question text in the example Does your system support the feature that that the Configuration Editor displays to the user 4 Create a More Info topic file for the question The More Info system of the Configuration Editor displays topic files for each question Each file provides detailed information that the user needs when answering its associated interview question Refer to Chapter 6 in the JavaTest Architect s Guide for additional information about creating More Info topic files Examples of More Info topic files can be found in the following SampleTestSuite location AdvancedTestSuite sampletck src sample suite help default SampleInterview 5 Update the map file After the topic file is created the map file must be
103. rs The JavaTest Architect s Guide describes creating custom filters selectIf Test Selection The following procedures describe how developers can use the selectIf expression to enable filtering tests that are not applicable for an implementation To Enable Test Selection with the select IF Expression The selectIf entry in the test description file contains a Boolean expression which is evaluated by a test filter If the Boolean expression evaluates to false the test is not run If the expression evaluates to true the test is run Chapter 4 Writing Tests 71 72 1 Add a selectIf entry to the test description file for a test class that users might be required to filter out of a test run The following is an example of a selectIF entry added to a test description file lt TR gt lt TD SCOPE row gt lt B gt selectIf f lt B gt lt TD gt lt TD gt isFeatureSupported lt TD gt lt TR gt In the example isFeatureSupported is an environment variable Most environment variables can be used as Boolean expressions to filter out tests from a test run Usually the value of an environment variable is target implementation specific and must be provided by the user through the configuration interview Step 2 describes the procedure for writing an interview question that collects this information from the user The following example is a complete test description file that includes a selectIf entry lt
104. s such as shutting down the system booting the system and configuring devices Refer to the following for this information m Software documentation that you received with your system m Solaris Operating System documentation at http docs sun com Examples Examples in this guide might contain the following shell prompts Shell Prompt C shell machine name C shell superuser machine name Bourne shell and Korn shell Bourne shell and Korn shell superuser Examples in this guide might also contain the character at the end of a line to break a long line of code into two or more lines Users must type these lines as a single line of code Preface xvii Typographic Conventions This guide uses the following typographic conventions Typeface Meaning Examples AaBbCc123 The names of commands files Edit your login file and directories on screen Use 1s a to list all files computer output You have mail AaBbCc123 What you type when su contrasted with on screen Password computer output AaBbCc123 Book titles new words or terms Read Chapter 6 in the User s Guide words to be emphasized These are called class options Replace command line variables with real names or values To delete a file type rm filename The settings on your browser might differ from these settings xviii Related Documentation When installed the Framework includes a doc directory that contains both
105. ss The test script must understand how to interpret the test description information returned to it by the test finder A human readable description in logical terms of what a test does and the expected results Test descriptions are written for test users who need to know in specific detail what a test does The common practice is to write the test specification in HTML format and store it in the test suite s test directory tree A collection of tests used with the test harness to verify compliance of the technology implementation to a Java specification Every Technology Compatibility Kit TCK contains one or more test suites Glossary 105 work directory A directory associated with a specific test suite and used by the JavaTest harness to store files containing information about the test suite and its tests 106 Java ME TCK Framework Developer s Guide July 2007 Index Symbols j tx files 100 A active agent 97 active applet instance 97 agent 97 agent monitor 97 AID 98 all values 97 AMS 98 APDU 98 API 98 API member 97 API member test 97 appeals process 98 first level 100 second level 103 Application IDentifier 98 Application Management Software 98 Application Programming Interface 98 Application Protocol Data Unit 98 assertion testing 98 assertions 98 automatic tests 98 B behavior based testing 98 black box testing 98 boundary value analysis 99 Cc class 99 classes 99
106. st class is the base class for the Java Platform Standard Edition Java SE platform technology component of the interactive test It is a subclass of the J2SEDistributedTest class that requires user action before a test result is determined The remote test component must invoke the send method to exchange messages from the client test components A remote test component usually contains the information that is displayed by the harness such as the test instruction expected image or the pass fail messages for the test cases Test Description for Interactive Tests Like distributed tests the test description file for the interactive test must contain the distributed keyword and the remote attribute In addition interactive test description files must also contain the interactive keyword See remote Attribute on page 44 for a description and example of the remote attribute Chapter 4 Writing Tests 49 The test description might also include the remoteSource and executeArgs attributes See remoteSource Attribute on page 45 and executeArgs Attribute on page 45 for a description and example of these attribute entries Required Interactive Test Keywords The following is an example of the keywords that must be added to the test description file for an interactive test Both the distributed keyword and the interactive keyword are required When additional keywords are specified the names must be separated by white sp
107. st descriptions grouped by the test developer according to a specific design principle Tests might be grouped in a test directory in the following cases ma All tests that examine the same component or functionality m All tests that have a configuration in common Chapter 3 Test Suite Construction 25 26 Organize the tests hierarchically the way you want them displayed in the test tree The Test Manager in the harness displays the test hierarchy enabling users to select and run specific groups of tests from the GUI Test Class A test class or test source is a Java technology class that either implements the test interface or extends the test class A test class can rely on inner sub or independent classes and contains one or more test cases Users must add individual test case methods to the derived test class to create a useful test class See Chapter 4 for information required to write different types of tests for the Framework Test Case A test case represents a single test and is the smallest test entity If a test class defines only one test case the test class is equivalent to the test case Each test case must return a Status object that represents the outcome of the test case The following example shows a very simple test class which contains several test cases CODE EXAMPLE 3 2 Simple Test Class public class MyTest extends MultiTest protected void runTestCases if isSelected testCasel addStatus t
108. sts 12 OTA Tests 13 Writing a Simple Automated Test 15 Writing an Automated Test 15 v To Create a Simple Automated Test 16 Building an Updated Simple Test Suite 19 v To Build an Updated Test Suite 19 Testing an Updated Simple Test Suite 20 v To Test an Updated Test Suite 20 Test Suite Construction 23 Test Suite Structure 23 testsuite jtt File 24 lib Directory 25 tests Directory 25 Test Class 26 Test Case 26 Test Description file 27 classes Directory 27 doc Directory 28 Creating a Test Suite Configuration Interview 28 v To Create a Configuration Interview Through the Interview Class 29 Building a Test Suite 40 Writing Tests 41 Testing Devices With Automated Tests 41 Automated Test Execution 42 Testing Communications or Networking With Distributed Tests 43 Client Test Component 43 Remote Test Component 44 Test Description for Distributed Tests 44 Required Distributed Test Keyword 44 Java ME TCK Framework Developer s Guide July 2007 remote Attribute 44 remoteSource Attribute 45 executeArgs Attribute 45 Distributed Test Execution 46 Testing User Interfaces With Interactive Tests 48 Client Test Component 49 Remote Test Component 49 Test Description for Interactive Tests 49 Required Interactive Test Keywords 50 Interactive Test Execution 50 Example of an Interactive Test 52 Testing Application Provisioning With OTA Tests 53 Server Component of an OTA Test 53 Client Test Component of an OTA Test 53 Test Description
109. t of substituting values can be seen in the resulting jtr files These arguments form the basis for the set of arguments that are passed into the tests defined in the executeClass field The default value of any variable not defined in the test environment is an empty string A value specified in seconds used to override the default ten minute timeout used with all test suite tests Specifies configuration values required by the test When a test requires information about the technology under test context to determine the expected results this information is identified in the context field of the test description table The harness checks to be sure that all values specified in the context field are defined in the test environment before it attempts to execute the test If any of the values are not set the test is not executed and the test is considered to be in error See the Java Technology Test Suite Development Guide for detailed information about setting context sensitive properties for a test Space separated list of MIDP permission names Specifies MIDP permissions that must be granted for this test application If tests are run as trusted MIDlets these permissions are included in the MIDlet Permissions attribute for the test MIDlet If tests are run as untrusted MIDlets the tests are filtered out if the security policy does not have all of these permissions granted in the untrusted domain Space separated list of MIDP permiss
110. tain test cases from the certification test run The exclude list testsuite jtx file identifies the tests in a test suite that should not be run The exclude list is located in the test suite 1ib directory Test developers use the following format to add tests to the exclude list Test URL Test Cases BugID Keyword See Chapter 3 for additional information about the exclude list Java ME TCK Framework Developer s Guide July 2007 m selectIf expression in test description This mechanism provides a flexible way for developers to make individual tests selectable com sun tck j2me javatest ExprFilter by including a selectIf expression in test description The test developer can include a selectIf expression in the test description file for a test Each selectIf field contains a Boolean expression that is evaluated by the filter The test is selected by the harness for a test run if the value is true See selectIf Test Selection on page 71 m Grant or deny mechanism This mechanism enables users to select tests based on the security requirements com sun tck midp policy PermissionFilter specified by the test developer Tests are selected if all the permissions listed by the developer in the grant or deny test description field are granted or denied See Granting or Denying Security Permissions on page 64 Custom test suite specific filters Developers can extend the test suite with custom test suite specific filte
111. tem uses IDs from a map file to identify both the location and the HTML files that it loads for the More Info system Each More Info file must have a corresponding entry in the map file of the form lt mapID target name url location filename htm1 gt The following example shows the contents of the map file used in the Simple Test Suite lt DOCTYPE map PUBLIC Sun Microsystems Inc DTD JavaHelp Map Version 1 0 EN http java sun com products javahelp map_1_0 dtd gt lt map version 1 0 gt lt SampleInterview gt lt mapID target SampleInterview hello url SampleInterview sampleStringValue html gt lt map gt The target attribute defines a unique ID for the topic file The ur1 attribute defines the path to and name of the HTML topic file relative to the map file The html files in the map file example are the More Info files You can also view the sampleInterview jhm map file in the following location SimpleTestSuite src sample suite help default 38 Java ME TCK Framework Developer s Guide July 2007 e Create the More Info topic files The More Info topic files are provided in HTML format and displayed in the Configuration Editor s More Info pane Refer to the JavaTest Architect s Guide for the procedure for creating More Info topic files The following is an example of a More Info file lt html gt lt head gt lt title gt SampleInterview hello lt title
112. test m Information only The visual component is presented to the users but no user action is required this is the default mode To develop interactive tests the test writer must write classes for both the client and the remote test components of the interactive test as well as create an appropriate test description file The AdvancedTestSuite directory contains examples of interactive tests Java ME TCK Framework Developer s Guide July 2007 Client Test Component The client test component of the interactive test must extend the MidDistribInteractivetTest class This is the base class for distributed interactive tests and extends the com sun tck j2me services messagingService J2MEDistributedTest class The MidDistribInteractiveTest class provides all of the common interfaces required by distributed interactive tests such as setFormTitle waitForMessage messageReceived addStatus and cleanUp The client test component must call the send and handleMessage methods in the MidDistribInteractiveTest class to send and receive messages to or from the other components of the interactive test group The following directory contains an example of a client test component MidDistribInteractiveTest java AdvancedTestSuite src share classes com sun tck midp lib Remote Test Component The remote test component of the interactive test must extend the com sun tck midp lib DistribInteractiveTest class The DistribInteractiveTe
113. test jar file that provides all of the classes required to execute the harness all of the JAR files under the 1ib directory of the Java ME TCK Framework and the library classes The test suite developer can use the library classes to simplify the creation of tests With javatest jar in the lib directory the harness automatically locates the test suite and does not prompt the user for the path to test suite directory This directory also contains additional resource files required by the test suite These files might include the following m testsuite Jar If a custom interview or if customized harness plug in classes are used package the classes and interview files in a customtestsuite jar file and place it in the 1ib directory In the Simple Test Suite example this file is named sample_jt jar and located under the SimpleTestSuite 1lib directory m testsuite jtx The exclude list testsuite jtx file identifies the tests in a test suite that are not required to be run Tests are not excluded in the Simple Test Suite so it does not contain a testsuite jtx file The exclude list file usually has the following format Test URL Test Cases BugID Keyword The following is an example of two lines from an exclude list file api java_awt EventQueue index html Invoke EventQueue2006 6406330 test api java_awt Adjustable index html SetGetValue 4682598 spec_jdk tests Directory The tests directory contains test sources and te
114. test run based on the last test result information stored in the test result files jtr A specification that references one of the platform edition specifications and zero or more other Java specifications that are not already a part of a platform edition specification APIs from the referenced platform edition must be Java ME TCK Framework Developer s Guide July 2007 Program Management Office PMO protected API protection domain Reference Implementation RI second level appeals process security domain security policy signature file signature test specification standard values included according to the referencing rules set out in that platform edition specification Other referenced specifications must be referenced in their entirety The administrative structure that implements the Java Community Process JCP program APIs that require that an applet have permission to access them An attempt to use a protected API without the necessary permissions cause a security exception error A set of permissions that control which protected APIs an applet can use The prototype or proof of concept implementation of a Java specification All new or revised specifications must include an RI A specification RI must pass all of the TCK tests for that specification Allows technology implementors who are not satisfied with a first level appeal decision to appeal the decision See also appeals process and f
115. tests can be queued up and run by the test harness and their results recorded without the user being present The configuration for standard automated test execution is the most common and the most simple of the tests that are run on the device Automated tests are also the most convenient and the fastest tests for a user to run since they are fully automated The majority of tests in a test suite should be automated tests with other types of tests used only when it s impossible to write automated tests In CLDC and MIDP configurations the harness running on a PC or a workstation sends an application bundle containing the tests and an agent to the device where they are unpacked by the application management software AMS built into the device and run In this configuration user interaction is not required to run each test FIGURE 1 1 illustrates the Framework configuration for running CLDC and MIDP standard automated tests For CDC the agent is started downloads the tests via the communication channel and executes them without being restarted a single agent runs from the beginning to the end of the test run See Chapter 2 for an example of writing an automated test and Testing Devices With Automated Tests on page 41 in Chapter 4 for information about automated test execution Chapter 1 Introduction 9 10 FIGURE 1 1 Framework Configuration for Standard Automated Tests PC or Workstation Device JavaTest Harness
116. tform Micro Edition Java ME platform technologies Before You Read This Book To fully use the information in this document you must read and understand the topics discussed in the following books TCK Project Planning Guide A high level planning guide that describes a process for developing a Technology Configuration Kit TCK A TCK is a suite of tests tools and documentation that enable an implementor of a Java technology specification to determine if the implementation is compliant with the specification This guide is available from the Java ME Technology APIs and Docs web site at http java sun com javame reference apis jsp Java Technology Test Suite Development Guide Describes how to design and write tests for any TCK It also provides how to instructions that describe how to build a TCK and write the tests that become the TCK s test suite This guide is available from the Java ME Technology APIs and Docs web site at http java sun com javame reference apis jsp JavaTest Architect s Guide This guide provides a description of the process of creating test suites and configuration interviews that JavaTest harness harness can run This guide is available from the Java ME Technology APIs and Docs web site at http java sun com javame reference apis jsp XV m JavaTest Harness User s Guide Graphical User Interface This guide provides a description of using the harness Graphical User Interface GUI This
117. to interact with the test device Grouping tests in this way enables users to run all tests in the test suite without changing the test setup For example tests for a profile might be divided into two types One type of test exercises the technology s API implemented on the test device The other type of test exercises the technology implementation s interaction with an Over the Air OTA server Because the test setup for these two kinds of tests is substantially different the architect and developer might group these tests into two independently run test subsets to make them easier for the user to configure and run The top level test suite directory generally contains the following files and directories m testsuite jtt file 23 lib directory tests directory classes directory doc directory testsuite jtt File The testsuite jtt file is located in the root directory of the test suite It provides a registry of information about the test suite and defines test suite properties The harness uses these properties to instantiate a Test Suite object that acts as a portal to all information about the test suite Whenever the harness requires information about the test suite it queries the Test Suite object The testsuite jtt file generally contains the following entries that tell the JavaTest harness how to start the TestSuite class It might also contain other entries See the JavaTest Architect s Guide for detailed information abo
118. uite runtime the source field contains the names of the files that are processed during the test run These files are XML schema sources and XML documents usually having file name extensions of xsd and xml respectively Such tests share a single precompiled class TestRun that invokes the XML parser under test through the Java technology API and passes the source file names to the parser for processing The test model is similar to compiler testing because the sources used in the tests contain valid and invalid use of various constructs of corresponding languages keywords String tokens that can be associated with a given test They describe attributes or characteristics of the test for example how to execute the test and whether it is a positive or negative test Keywords are often used to select or deselect tests from a test run See Keywords on page 94 executeClass The main test class that the harness loads and runs This class might in turn load other classes when the test is run 92 Java ME TCK Framework Developer s Guide July 2007 TABLEC 1 Framework Test Description Fields Continued Field Description executeArgs timeout context grant deny selectIf An array of strings that are passed to the test classes being executed The arguments might be fixed but often involve symbolic values that are substituted from the test environment variables defined elsewhere in the test environment The resul
119. updated If the topic file for the new question is named featureSupport html1 the test developer must add the following line to the sampleInterview jhm file lt mapID target SampleInterview featureSupported url SampleInterview featureSupport html gt Note The sampleInterview jhm map file is located in the AdvancedTestSuite sampletck src sample suite help default directory 6 Create the JAR file After creating the interview you must package it into a JAR file for inclusion with the test suite during the build time If you successfully ran the build as described in Chapter 2 Building an Updated Simple Test Suite on page 19 you have an example of this JAR file under the lib directory In the example the JAR file is named AdvancedTestSuite lib sample_jt jar 7 Add the JAR file to the classpath entry of the testsuite jtt file The test suite containing the configuration interview and associated tests can now be built See Building a Test Suite on page 40 in Chapter 3 76 Java ME TCK Framework Developer s Guide July 2007 APPENDIX A Test API The following is the Java ME technology specific API that every test developer must know Test Status MultiTest J2MEDistributedTest J2SEDistributedTest DistribInteractiveTest OTATest Test Interface name com sun tck cldc lib Test This is a Java ME technology version of the standard harness Test interface This
120. ut the standard properties used by the Test Suite that can be specified in the testsuite jtt file name The name of the test suite id A unique identifier composed of letters digits underscore minus and hyphen used to identify a specific version of a test suite tests optional By default the JavaTest harness looks for test source files and test descriptions in the tests directory in the test suite root directory classpath Entry that specifies the class path on which the main TestSuite class can be found typically a JAR file that contains test suite specific components This entry is required if the Test Suite class or any other classes the TestSuite refers to are not located within javatest Jjar testsuite Entry that specifies the name of the TestSuite class and any arguments the class requires The following is the testsuite jtt file used by the Simple Test Suite that comes with the Framework CODE EXAMPLE 3 1 Simple Test Suite testsuite jtt File name Simple Test Suite id Sample_TestSuite_1 tests tests classpath lib j2mefw_jt jar lib sample_jt jar lib interviewlib jar testsuite sample suite SampleTestSuite 24 Java ME TCK Framework Developer s Guide July 2007 The testsuite jtt file is located under the root directory of the Simple Test Suite You can also find it under the build directory of the Simple Test Suite source lib Directory The lib directory usually contains the java
121. vironment values specific to that test suite and the JavaTest harness standard values Fields methods and constructors for all public classes that are defined in the specification Tests sometimes referred to as class and methods tests that verify the semantics of API members 97 appeals process Application IDentifier AID Application Management Software AMS Application Programming Interface 98 API Application Protocol Data Unit APDU assertion assertion testing automated tests behavior based testing A process for challenging the fairness validity accuracy or relevance of one or more TCK tests Tests that are successfully challenged are either corrected or added to the TCK s exclude list See also first level appeals process second level appeals process and exclude list An identifier that is unique in the TCK namespace As defined by ISO 7816 5 it is a string used to uniquely identify card applications and certain types of files in card file systems An AID consists of two distinct pieces a 5 byte RID resource identifier and a 0 to 11 byte PIX proprietary identifier extension The RID is a resource identifier assigned to companies by ISO The PIX identifiers are assigned by companies There is a unique AID for each package and a unique AID for each applet in the package The package AID and the default AID for each applet defined in the package are specified in the CAP file
122. ween the device and the harness when users run tests Java ME technology tests and test suites are run on a device by the harness Because of Java ME technology connectivity requirements see Connectivity Requirements on page 4 and limited hardware resources test devices see Resource Limitations on page 5 test suite and test developers are presented with a number of challenges To help simplify test development the Framework provides a set of components harness plug ins and support classes for developers When running tests the Framework and its components act as a bridge between the harness see Framework Components on the Harness Side on page 7 and a test device see Framework Components on the Device Side on page 7 The Framework resources for both the harness host and the test device enable developers to reduce the complexity of test suite development while optimizing test performance Because Framework classes and resources are shared by multiple test suites they are fully developed extensively tested and stable In addition to reducing the complexity of Java ME test suite development Framework classes and resources can improve the reliability of the test suite Target Environment The Framework resources enable developers to package and deliver tests developed for a device in an appropriate form for a particular platform implemented on a device The Java ME application environment includes both a configurat
123. xecution PC or Workstation Device JavaTest Harness 1 getNextApp Execution Server Passive Agent Server Side Test Check Send Message E a O Get Message Messaging Service 1 getNextApp The AMS issues a getNextApp command to the execution server on the workstation The AMS implementation is device specific and must be provided by the licensee 2 Application Bundle The execution server sends an application bundle to the AMS The AMS downloads and executes the application bundle on the device Test bundles are created by the Framework 3 getNextTest The agent issues a getNextTest request to the execution server 4 Test Name The execution server returns the name of the next test in the application bundle that the AMS should run While the tests from the application bundle are loaded onto the device by the AMS the execution server establishes the sequence in which the tests in the bundle are run Chapter 4 Writing Tests 47 5 Check or Send Message The test sends either a message or a check message request to the Framework messaging service on the harness 6 Get Message The Framework messaging service on the harness sends the test a get message command 7 Check or Send Message The server side test sends either a message or a check message request to the Framework messaging service on the harness 8 Get Message The Framework messaging service o
124. y specific JAD file attributes are needed com sun c l ldc communication midp BaseServer Abstract class com sun cldc communication midp BaseHttpServer Used as a basis for HTTP connection te com sun cl sts Ldc communication midp HttpServer Interface com sun cldc communication midp MIDHttpExecutionServer Actual server used for application delivery 86 Java ME TCK Framework Developer s Guide July 2007 Agent Classes The following are the agent classes and interfaces provided in the src directory m com sun cldc communication Client Communication interface m com sun cldc communication MultiClient Communication interface a com sun tck midp javatest agent MIDletAgent Standard agent a com sun tck cldc javatest agent CldcAgent Standard agent Digital Signer com sun tck midp signer JKSSigner implements SuiteSigner interface Developers can provide their own version if they are using their own non standard version of the SuiteSigner interface Preverification Script com sun tck cldc javatest PreverificationScript is used to verify class files when creating a test suite All class files must be preverified The preverification script is used during development Java ME Technology Version of Harness Classes The following are Java ME technology versions of the harness classes provided in the src directory m com sun tck cldc lib Status m com sun tck cldc lib MultiTest m com sun tck cldc l
125. zed equivalence classes The prototype for an object in an object oriented language A class might also be considered a set of objects which share a common structure and behavior The structure of a class is determined by the class variables that represent the state of an object of that class and the behavior is given by a set of methods associated with the class See also classes Classes are related in a class hierarchy One class might be a specialization a subclass of another one of its superclasses may be composed of other classes or might use other classes in a client server relationship See also class Define the criteria a Java technology implementation must meet to be certified as compatible with the technology specification See also compatibility testing The process of testing an implementation to make sure it is compatible with the corresponding Java specification A suite of tests contained in a Technology Compatibility Kit TCK is typically used to test that the implementation meets and passes all of the compatibility rules of that specification Information about your computing environment required to execute a Technology Compatibility Kit TCK test suite The JavaTest harness uses a configuration interview to collect and store configuration information The dialog box used by the JavaTest harness to present the configuration interview A series of questions displayed by the JavaTest harness to gather informat

Sun Microsystems Telephone Accessories 1.2 User's Manual

Contents

Download Pdf Manuals

Related Search

Related Contents