Test Report - Ascom Partner Extranet
Contents
1. A client application typically used in a Call Center environment which allows graphical call control of the agent s phone A place where calls are answered and or calls are made A typical call center will have lots of people called agents answering phones Outbound calls are typically made using a machine automated process and inbound calls are typically answered by an IVR system before being placed in an on hold queue to wait for a live agent A TAPI or JTAPI based program that allows call control through a Windows interface but does not actually handle termination for a CTI port device like a SoftPhone does Java Telephony API is a set of APIs for Java based telephony control JTAPI applications like Java are platform independent and depend on another API TAPI in the case of Cisco to control the actual telephony hardware Like speech quality except that the call is between a phone and an automated system such as IVR The name applied to a Cisco proprietary end point simulator A SimClient machine can simulate the traffic of up to 1000 simultaneous calls Used by Cisco and Cisco partners to stress test applications Controls and handles media termination for a CTI port device Typically refers to a software program programmed in TAPI or JTAPI which acts like a PBX phone An example of a PBX phone is the Cisco 7960 When grading the quality of sound as passed between two phones we call it speech quality Printed version2. Click Trace gt Configuration and select the server service you would like to enable tracing for Verify that Tracing On is enabled and that the Debug Trace Level is set to Error Verify that for each service where tracing is being enabled the Enable File Trace Log option has been enabled and configured Repeat for each service and each CUCM machine For all testing the following traces will be set on CDR ON CCM ON SDL ON Trace Utility for TAPI JTAPI 45 Printed versions of this document are uncontrolled Look on line for the controlled version 45 January 31 2011 Cisco Systems Inc company confidential Cisco Confidential Turning on tracing for the TAPI Service Provider Go to Start Settings Control Panel and select Phone and Modem Options Go to Advanced tab Select the CiscoTSPOXX and click Configure button Go to Trace tab Select Trace On check box and select the 1 TSP Trace to trace the TSP internal messages Select Error to just log errors in the TSP Select Detailed to log internal messages for debugging purposes 2 CTI Trace to trace the messages sent between CTI and TSP 3 TSPI Trace to trace the requests and events sent between TSP and TAPI Directory is the path for the trace log For example c Temp No of Files Set this to a value greater than or equal to 1 enables rolling log files For example a value of 10 will cause up to 10 log files to be used in a cyclic fa
3. Cisco Systems Inc company confidential 40 Cisco Confidential 11 Appendix B Test Approach The goal of the IVT is to test in a customer like environment to ensure that it behaves in a reliable manner under conditions that reflect typical customer usage and product deployment This includes the exercise of provisioning generating call traffic of supported types maintenance activities troubleshooting and fault insertion The IVT will focus on the following seven main areas Phase 1 Installation amp Configuration Phase 2 Personal Assistant Reliability Verification Phase 3 Scalability Stress Performance and Load Testing Phase 4 Informational tests Phase 5 Network Management and Operations Administration and Maintenance Phase 6 Negative Tests Phase 7 Voice Quality IVT Entrance Criteria General Partner Application needs to comply with the Cisco Developer Network CDN Partner program checklist Documentation Product documentation needs to be available drafts Customer oriented documentation Provisioning documentation The following documents need to be in Approved state IVT Questionnaire Test Plan OoOooooo Software D Component software needs to undergo Bug Scrubs to have a clear idea of the active problems in the code No severity 1 or 2 defects that affect the Application Acceptance of severity 3 defects depends on the impact in the test effort 100 component code development comple
4. 6 2 1 Application is available for a valid device Test Case Details Title Application is available for a valid device Description The objective of these sets of tests is to verify that the application is available and functioning appropriately Test Setup Properly associate device with User Procedure 1 Send Text message from application 2 After text is invoked visually inspect IP Phone display to ensure that time date etc are still displayed Expected 1 Verify that all buttons and displays return to normal state after text is deleted Results Observations Passed v Passed v Passed with observation limited functionality or unsupported phone model 7906v 7925 7940 7942v 7960 7962V 7970 6941v 79117 7985v 7941v 7945v 7961 7965 7971V 7975 6 2 2 Application Interaction is correctly handled for IM page Test Case Details Title Application Interaction is Correctly Handled for IM page Description The objective of these sets of tests is to verify the IP phone default soft keys are sill functional after receiving the specified text message from the application server Test Setup Properly associate device with User Procedure 1 Attempt to complete call when IM page is sent 2 Use the soft keys implemented by the 3ra party applications Expected Each phone is able to press the Call Button and complete the call to the Results c
5. E A A A A nagtire hated A A redial 39 10 Intermal ROVIOW tecc cite asses e a e bits e a tensa ceed owen e AEA 39 Appendix A 3rd Party Serviceability Information ccccecceecseeeeeeeeeeeeee cee eeeaaeeeeaeeseeeeecaaeeeeaaeedeaeeseeeeesaeeeeeeeeeeeess 40 3 Party Support Contact IMO ALON ica Secmac a suonaccce caste fosssecusoneitutossloncrnnin easuaadchumdedtnaosbinnts aiiese wiettnceuetens 40 3 Party Product Isolation PROCEC UNOS cass sutesde A aubus cae taste cosssacasy tats taubacysuurkecahnaste kt nasr ttan ts hetcesoecnduaitan inet bs 40 TOUDIOSMOOTING ieten en cnet cae sects fed aaiae te ea aaa ea aaa aandaa Taaa ado a anata adoa iadaaa eais 40 Installation and Configuration cccceceecceecceeeeeeeeeeeeeceaeeeeeaeeeeaeeseeeeeceaeeeeaaesdeeeeseaaeeesaaesgeneeseeeesaeeseaaeeeeeeesaas 40 Upgrades and patch application sareriereirierari aae eise eee riina a ay iape en rii Er a aaa ideii 40 Debugging and DiagGnOStieS t scczccide cPhecs cceceaa sees Enon EA EEANN E nE A EE sch aageds Ore AE EEAO ER 40 Error MGSSAQCS Hex sects ececed e a Ea E a a a ateamnetbaebanane 40 TOGS AIOT E E E E TE E T reece rere rerrere cere reer 40 EIST OL COMMANGS ores ceibeceS25 cos ecuhesaycsusacesoeta quae ausuecsstt A swish N aE a A EE aE 40 Redundancy and sReCOvery ecrane eraen eE EARR deen gesdnesducaansadeeasscbeades acaneeisatanceasitunecaiegudeesiade 40 TI Appendix B Test Approacn 2 scccsce dccezsccen cadena gectnceneceas tadeed I cena desvacnnues E
6. EEE aE EES E ORE eE 41 IW Entrance Crie a a a E E a aa A E aE eee a aE E ES 41 IMT EXIteG Sled ite ea ar a Actas eras a E E E a A dara tae e a E AS 42 Error report handling software and hardware cccccceceeeceeececeeeee eee eeeeaeeeeaeeseeeeecaeeeeaaeedeaeeseeeeessaeeeeeeseeeees 42 Defect Tracking Information Problem Reporting and Severity Levels s cceecceceeeeeeeeeeeeseeseeeeeseaeeesaeeeeneeees 42 12 Appendix C IVT Troubleshooting Tools and Default configurations cccceeeeeeeeeeeeeeeeeeeeeeeseeeeetaeeeeneeees 45 Teace Utility tor TAPIITAR sesa a a eaaa aaa a a suas e aE A E a En 45 Even LOG eE E A EEA E A S E EOE A 46 What to collect when preparing for and reporting CM related problemS sssssesesssesrresrrssrrssrssrrssrissrrrssrrssens 46 4 Printed versions of this document are uncontrolled Look on line for the controlled version 4 January 31 2011 Cisco Systems Inc company confidential Cisco Confidential 1 Introduction This document is the detailed IVT test Report for 1 21 of Unite Messaging Suite It defines the scope approach resources schedule risks mitigations and entry exit criteria for IVT testing 1 1 Objective s The principle objective of the Interoperability Verification Testing IVT is to provide Cisco CDN program partners with an approved consistent and comprehensive verification methodology for acceptance of their application product into the Cisco CDN Partner Program
7. Testing is focused on functionality and compliance with the open standards followed by the Cisco Call Manager and design within context of a Cisco internetworking infrastructure 1 2 Scope The IVT test activities between third party devices and the Cisco IVT test bed determines whether the implementation of third party devices provides basic functionality without compromising the Cisco network This document provides the test strategy and test case procedures to validate the mandatory standard requirements and a variety of optional requirements for the device The device under test must satisfy all the mandatory requirements and selected optional requirements in order to pass the test This test determines whether the product performs as required and inter operates on a pass fail basis The test suite therefore emphasizes error detection not error diagnosis or correction however if there is a failure data will be captured for further analysis and if possible will be traced to the specification of the standard to determine the cause of the problem Any problems found will be reported to Cisco and to the vendor for correction Please note additional tests may be included in the testing session that is not listed in this test plan The purpose of such additional testing is for investigative purposes to ensure the solution is reliable The results of the additional tests will be included in any subsequent review It is expected that the partner will
8. VETAR EAA SANRA ORTE VEE AEAEE AESA ASAA ERr 11 die Problem Repoti crierai aa aE iR AA IAEE TAA U AETAT ANAD TAA A e 12 aia TESES CIUDS h anna aa A a N a rep oer 13 4 1 alEO NITE AENEA A E T 14 4 2 SAMa RV Ea a a a a a a aa a Sabin Aaa aa 15 4 3 TStEGUPMEN enara n AE A A E EA A E E ONEAN 15 4 4 Fest Fools SoftWare eaaa A r N AA 15 4 5 Supported Signaling Protocols and Interfaces sesseseeseesiesirssirssiissrisstisstisstisstnnstnnntnnntnnntnnntrnn te 15 5 aK Wn ELE A ETE E N A N E reer EET 16 5 1 Test Engineers and Technology Contacts cccccccesseceeneeceeeeeceaeeeeeeeeeeeeecaeeesaaeseeaeeseeeeesaeeesaeeseneeees 16 5 2 Issues from previous IVT teSting cecccceceeeeeeeeeceeeeeeaeeeeaaeeeeeeecaeeeeaaeeseaeeseeeeesaaesseaaeseeeeeseaeeeseaeseneeess 17 6 Detailed Test Cases icine ai k eitea hea See vacua iabcacesu ah sacsarsbenceuesssaetacabsacend souancacees sectadss snes cevabenteastaneasealenedsesvenes 18 6 1 Installation and Configuration TeSts c cccccceccsceceeceecceeeeeaeeeeeeeceaeeeeeaeeseaeeseeeeseaesesaaeseeeeesiaeeseaeseeeeess 18 6 1 1 3 party Server installation and registration ox sstb essence csshisseasiascatecieessancesces caine mind cawteabaulencgstants 18 6 1 2 Client application installation c ccc cee eeese cece sees eeeeeeceeeee seas eeeeaeeseeeeseeeeesaaaeseeeeeseeessaesseeeseneeess 19 6 1 3 3rd party Personal Assistant installation and registration ccececeeceeceeeeeeeeeeet
9. document are uncontrolled Look on line for the controlled version 46 January 31 2011 Cisco Systems Inc company confidential Cisco Confidential save it off increase the SdiTraceTotalNumFiles appropriately SdiTraceTypeFlags set to Ox8000EB15 Important When changing the parameters Don t forget to click the update button after making any changes 2 Make sure the SDL SDLxxx_200_xxxxxx files trace settings are set appropriately Check set the following CTI Manager service parameters from the CM administration Under Service gt Service Parameters select the IP address of the server Then select the Cisco CTIManager service from the list of services SdiTraceDataFlags set to 0x000001 10 SdilTraceFlag set to T SdilTraceMaxLines set to between 10000 and 20000 SdilTraceTotalNumFiles set it to at least 100 If you are running high traffic this number should be increased significantly The best thing to do is after running for a while see how much time is being captured by looking at the date time of the SDL files If the files are being overwritten before you would be able to save it off increase the SdiTraceTotalNumFiles appropriately SdiTraceTypeFlags set to Ox0000CB15 Important When changing the parameters Don t forget to click the update button after making any changes 3 Make sure the CUCM serviceability Cccmxxxxxxxx files trace settings are setup for the CUCM service In the CM administr
10. have already carried out their own testing above and beyond the requirements of the test plan to ensure a quality solution 1 3 Code Used During IVT The IVT operates a closed code program Any modification of the Application will be deemed an IVT failure If your application fails IVT on three occasions you will be barred from the program as such a failure indicates poor product quality Cisco will perform due diligence to ensure the failure reason is not due to a Cisco Call Manager issues You will be presented with the appropriate Call Manager Logs for the time of the failure It is then your responsibility to isolate the failing reason and present this information to Developer Support DS for verification via a DS case The program team will then decide if the failure reason can be wavered 1 4 Pass Fail Criteria On completion of the verification tests if there are any issues deemed as severity 1 any issues deemed as severity 2 or 3 issues deemed as severity 3 the product will be considered to have failed IVT Note The approval process can increase decrease the severity after the test cycle if considered necessary 5 Printed versions of this document are uncontrolled Look on line for the controlled version 5 January 31 2011 Cisco Systems Inc company confidential Cisco Confidential 6 Printed versions of this document are uncontrolled Look on line for the controlled version January 31 2011 Cisco Systems Inc company con
11. programs and manual configurations needed Expected 1 Verify that via the partner Personal Assistant that the Server has Results installed correctly 2 Record any changes to the CUCM devices required by the 3 party installation Most likely this will consist of making CTI ports and route points on the CUCM for the 3 party product Observations Passed This is a turnkey installation No error reported 6 1 4 3rd party Personal Assistant configuration with reference to 3rd party Server Test Case Details Title 3rd party client Personal Assistant with reference to 3rd party Server Description Verify that that the client Personal Assistant registers properly with the server Personal Assistant Test Setup Client Personal Assistant installed on the required hardware and physically connected to the network Server Personal Assistant installed with the required configuration parameters to register and connect the client Personal Assistant Procedure 1 Connect the Call Centre Personal Assistant platform to the CUCM network through the TAPI JTAPI interface This may have been done as part of installation in the previous section 2 Spot check the ability of the Personal Assistant to control the configured CTI Devices IP Phones configured on the publisher CUCM Use the instructions provided in the 3 party installation and configuration guide to confirm that the product is correctly installed and communicati
12. to customer problems For example customer affecting Defects will be fixed before Defect of equal severity that does not affect a customer Priority Definition Level Showstopper Testing cannot continue The problem requires immediate resolution Current development may be interrupted There is a high impact to a customer or shipping products There are regular status updates sent to the project management Response time 24 hours High Testing may continue but must be modified Resolution is required in the current product release There is significant urgency Full time resources are assigned to resolving the problem There are regular status updates sent to the project management Response time 48 hours Medium Testing continues in other areas A resolution is required before the next 3 scheduled product release An out of cycle phase in is required There are regular status updates sent to the project management Response time 5 days Low No customer testing or development impact Problem is obviously an error but 4 does not cause confusion problem is intermittent A fix can be phased into a scheduled release Very Low Problem is so rare or minor it is likely to be missed by most users and has no real impact Although there is a close relationship between severity and priority the levels do not have to match Customer affecting Defect with a low severity can have a high priority to indicate t
13. 25 79407 7942 v 7960 7962V 7970 69417 7911v 7985v 7941 Vv 79457 79617 7965 79717 7975 6 3 2 Test Case Overnight CM performance monitoring Test Case Details Title Overnight CM performance monitoring Processing run Description Overnight processing run to verify 3 Party application s stability over long duration and high load Test Setup TURN OFF RTP on the SimClients Auto registration configured on the Communications Manager Make sure CDRs are being collected on the Communications Counter log configured in Call Manger s Win 2K Perfmon Traces configuration SDL ON info ON by default CCM ON info OFF by default CDR ON info OFF by default Procedure 1 Before commencing the overnight run it is advisable to run the SimClients for Y2 to 1 hour and verify that calls are succeeding Once this has been done reset the SimClients On the CUCMs launch Win2K Perfmon System Monitoring Start the SimClients for an 8 hour test The following day complete the corresponding tables in the performance monitoring observations section AON Expected Results 1 New ip phones are created on the CUCM Phones register and obtain a DN 2 Perfmon monitoring started 3 SimClient call generation continues for 8 hours 4 Record the results Paste a screen shot of the Perfmon graph with the PROCESSOR PROC
14. ESSOR TIME highlighted Save CSV file to the IVT Doc Centre Observations Passed Generate a 20000 BHCA call load on the CUCM cluster sing Camelot CUCATT clients 200 callers amp 200 receivers While under the Camelot load test send a text message to 18 IP phones one text message every 5 second The text messages are automatically deleted at the same time 27 Printed versions of this document are uncontrolled Look on line for the controlled version 27 January 31 2011 Cisco Systems Inc company confidential Cisco Confidential As result a total of 3500 messages were successfully received and deleted on each Phone or a total of 18 X 3500 63000 messages during 10 hours period The average CPU utilization on the CUCM cluster stayed below 60 28 Printed versions of this document are uncontrolled Look on line for the controlled version 28 January 31 2011 Cisco Systems Inc company confidential Cisco Confidential 6 4 Informational Tests These tests are executed to verify specific information about the third party product to Cisco This is in relation to the IVT Questionnaire supplied by the vendor 6 4 1 Calculate Time to receive text message from application Test Case Details Title Calculate Time to receive text message from application Description The objective of these sets of tests is to verify the amount of time it takes to invoke receive text messa
15. G Cisco to implement this function which is illustrated in figure 2 The ECG Cisco keeps a small message database for each phone Each database consists of the 20 latest received messages and the messages are distributed first hand in priority order and secondly for messages with the same priority in time order 7 Printed versions of this document are uncontrolled Look on line for the controlled version January 31 2011 Cisco Systems Inc company confidential Cisco Confidential ECG Cisco Cisco IP phones j gt a w5 Figure 2 The ECG Cisco sends a stored message to a Cisco JP phone In this section briefly describe what the product is and what it does 2 1 3 Party Application Components Services List all the 3 party application services run on the 3 party server 3 Party Services running on the Server Version Unite Messaging Server 1 21 2 2 CallManager Device Table CallManager Device Summary Template CallManager Description Configuration Min Max Suggested Device Interface Type Comments Configurable Provisioning 2 3 What Will Be Tested 8 Printed versions of this document are uncontrolled Look on line for the controlled version January 31 2011 Cisco Systems Inc company confidential Cisco Confidential Unite Messaging Suite Version 1 21 will be tested for functionality and performance as per the specifications det
16. TNT cisco Document Number EDCS 893513 ascom Detailed IVT Test Report for ASCOM Unite Messaging Suite Test Result PASS Test Date 10 Jan 2011 Product Name Unite Messaging Suite Product Version 1 21 must be generally available CUCM Version X X x 8 0 2 40000 1 Cisco Phones Tested 7900 series Product Type IP Phone Application API Protocol s Used XML HTTP Developer Services Contract 3206825 Partner IVT Contact Name Staffan Ornbratt Partner IVT Contact Phone 46 31 559300 Partner IVT Contact Email Staffan ornbratt ascom se IVT Lab Location Galway EMEA or US Partner Main Support Number 46 31 55 94 50 Partner Main Support Email support ascom se Cisco Confidential Revision History 1 0 Gerry Pearson 8 28 2006 Initial draft for Cisco review 1 1 Gerry Pearson 10 04 2006 Updated comments after review 1 2 Gerry Pearson 05 07 2007 Updated based on the IVTQ Response 1 3 David Meredith 02 16 2010 Updated for CUCM 8 0 1 4 Tony Nally 01 07 2011 Updated to CDN format 1 5 Oumar Dakissia 01 26 2011 Updated Supported Test Cases 2 Printed versions of this document are uncontrolled Look on line for the controlled version 2 January 31 2011 Cisco Systems Inc company confidential Cisco Confidential Table of Contents 1 Majico eroin oIa EEE EE ETE EAA TE A dy bahigeo adel AEA s
17. ailed by ASCOM in the IVT Questionnaire The main focus of the IVT is to verify that the application operates with no or little impact on the Call Manager environment Standard features of Unite Messaging Suite Version will be tested in conjunction with Communications Manager as well as extended feature testing and failure scenarios Performance testing will be undertaken to verify the performance of Unite Messaging Suite Version 1 21 application in relation to the Communications Manager cluster Cisco Unified Communications Manager 8 x Feature specific test cases Application specific test cases 2 4 What Will Not Be Tested Test Cases that are not applicable or unsupported Test Cases as per IVT questionnaire 2 5 Administration and Debugging tools List 3 party administration and debugging tools Version None 9 Printed versions of this document are uncontrolled Look on line for the controlled version 9 January 31 2011 Cisco Systems Inc company confidential Cisco Confidential 3 Executive Summary ASCOM Unite Messaging Suite Version 1 21 PASSED the IVT with CUCM version 8 0 2 40000 1 ASCOM Unite Messaging Suite Version 1 21 software performed very well throughout IVT testing No issues were found during this IVT However some observations were noted regarding different behavior of certain Cisco IP phone type particularly the low end IP phone series such as the 7906 and 7911 with limited functiona
18. ails Title IP phone loses connection while message is displayed reconnects after 5 seconds Description Check the behavior of the IP phone when it loses connection with message being displayed reconnects after 5 seconds Test Setup Properly associate device with User Procedure 1 Send a text message to the IP Phone 2 While the message is being displayed disconnect the IP Phone from the network 3 After 5 seconds reconnect IP Phone to network Expected Once IP Phone is reconnected to network the message will be resent from Results application Depending on time to live TTL timers and resend timer Observations Passed 32 Printed versions of this document are uncontrolled Look on line for the controlled version 32 January 31 2011 Cisco Systems Inc company confidential Cisco Confidential v Passed v Passed with observation limited functionality or unsupported phone model After the phones are reconnected to the network the message repapears and can be deleted by pressing the DELETE button on the phone 7906v 7925V 7940V 7942V 7960 7962V 7970 69417 731v 7985v 7941 Vv 7945v 79617 7965 79717 7975 33 33 Printed versions of this document are uncontrolled Look on line for the controlled version January 31 2011 Cisco Systems Inc company confidential Cisco Confidential 6 6 Phase 6 Additional Application Spe
19. ation under Personal Assistant gt CUCM Serviceability select Trace gt Configuration Select the IP address of the server from the list of servers Select CUCM from the list of Configured Services The Trace On check box should be checked Under Trace Filter Settings The Debug Trace Level should be set to Detailed Under CUCM Trace Fields make sure the check boxes for the trace related to the problem are checked For example if the problem has to do with a call made through a DT 24 gateway check the check box labeled Enable DT 24 DE 30 Trace Under Trace Output Settings The Enable File Trace Log check box should be checked The File Name edit box should contain a valid path and file name The default for Maximum No of Files is 250 The default for Maximum No of Lines per File is 10000 The default for Maximum No of Minutes per File is 1440 As with the SDL traces after the test has been running a while check to see if the CUCM trace files are being overwritten too soon before they can be copied If they are increase the Maximum No of Files parameter appropriately 4 Make sure the CTIManager serviceability ccmxxxxxxxx files trace settings are setup for the CTIManager service In the CM administration under Personal Assistant gt Cisco CUCM Serviceability select Trace gt Configuration Select the IP address of the server from the list of servers 47 Printed versions of this document are uncontrolled Look on line
20. bscriber support services The IVT test architecture attempts to mirror the CCM design guides functional requirements where possible Figure 4 1 Example Test Setup System Level View Device Level View PSTNGW CUCM Sub2 Dox CUCME Office 01753444XXx 13 Printed versions of this document are uncontrolled Look on line for the controlled version 13 January 31 2011 Cisco Systems Inc company confidential Cisco Confidential 4 1 Hardware a a ol Ce ee WS C3750 24P C3750 IPBASE M Vers 12 2 25 SEB4 aa Ce eae ee bce ies eee SCCP Standard phone 7960 P00308010200 7911 SCCP11 9 0 2SR1S Unite Messaging Server N A 14 Printed versions of this document are uncontrolled Look on line for the controlled version 14 January 31 2011 Cisco Systems Inc company confidential SIP42 9 0 2SR1S 7962G SIP42 9 0 2SR1S 7971G GE SIP70 9 0 2SR1S SIP45 9 0 2SR1S 1 1 1 1 Cisco Confidential 4 2 Software Firmware 4 3 Test Equipment Windows 2003 Camelot Call Generation 4 4 Test Tools software Windows 2003 Camelot Call Generation 4 5 Supported Signaling Protocols and Interfaces 15 Printed versions of this document are uncontrolled Look on line for the controlled version 15 January 31 2011 Cisco Systems Inc company confidential Cisco Confidential 5 Test Resources In this section please state all logistics in relation to performing and carrying out the IVT test
21. cific Test cases These tests are executed to verify specific functionality of the third party product 6 6 1 Read New Messages 6 6 1 Read new Messages Messages in correct priority sequence Message queue updated properly 6 6 2 Message Notification Sound played when message received Sound plays as each sounds message is recieved 6 6 3 Softkey Translation Softkey labels display correctly Soft phone keys on different phone displayed properly typess 6 6 4 Backup amp Restore System back up and restore operates correctly N T 6 6 5 Receive Messages No impact to call processing transfer No error reported during call Conference release 6 6 6 Message updates No impact to call processing transfer No error reported during call Conference release 6 6 7 Exceed 20 Messages Latest 20 messgaes stored First in First out basis to Phone 6 6 8 Extension Mobility Active messages available N T 6 6 9 Remove IP phone Active messages available No error reported from CUCM with actve messages 34 Printed versions of this document are uncontrolled Look on line for the controlled version January 31 2011 Cisco Systems Inc company confidential 34 Cisco Confidential 7 Reference Note Any CPU pegs over 80 sustaining for 5 seconds or more and any sustained memory increases should be noted 8 Glossary The following list describes specific acronyms and definitions for terms u
22. cord any changes to the CUCM devices required by the 3 party installation Most likely this will consist of making CTI ports and route points on the CUCM for the 3 party product Observations Passed 18 Printed versions of this document are uncontrolled Look on line for the controlled version 18 January 31 2011 Cisco Systems Inc company confidential Cisco Confidential This is a turnkey installation ASCOM Unite Messaging Suite Version 1 21 software is installed by ASCOM engineer refer to installation and User Manual for detail No error reported 6 1 2 Client application installation Test Case Details Title Client application installation Description Verify that the server Personal Assistant registers properly with the CUCM Test Setup Server Personal Assistant installed on the required hardware and physically connected to the network CUCM installed with the required configuration parameters to register and connect the server Procedure 1 Connect the Call Center Personal Assistant platform to the CTDP network through the TAPI JTAPI interface This may have been done as part of installation in the previous section 2 Spot check the ability of the Personal Assistant to control the configured CTI Devices IP Phones configured on the publisher CUCM Use the instructions provided in the 3 party installation and configuration guide to confirm that the product is correctly installe
23. d and communicating with the CTI Devices IP Phones configured on the publisher CUCM Expected 1 Verify that the Call Centre Personal Assistant recognizes all CTI Results Port CTI Route Point IP Phone directory numbers as assigned in CUCM Observations Not Applicable 6 1 3 3rd party Personal Assistant installation and registration Test Case Details Title 3rd party Client Personal Assistant installation and registration Description Installation of the client Personal Assistant Verification of the completeness and accuracy of the client Personal Assistant installation guide Documentation of any required software installation or configuration change on the CUCM Test Setup Required hardware and operating system installed on the client machine Server Personal Assistant must be installed and configured to register and connect the client 19 Printed versions of this document are uncontrolled Look on line for the controlled version 19 January 31 2011 Cisco Systems Inc company confidential Cisco Confidential Procedure 1 Read the 3 party installation instructions 2 If 3 party product is not turnkey On a designated client machine not one of the CUCM machines install the 3 party product Record length of time required for install 3 Perform any configuration the 3 party product according to the installation instructions Document the various separate installation
24. e Maldl eas 5 1 1 Slo E A E E E EEE T AEE AEA E A AT E 5 1 2 STel 01 AEEA E EEE T E T AEE E A T dete uacebete ters eevblaales 5 1 3 God Wsed During IV Te sa scceisecisadsehsciccsassadas eee ecuauhephagphsancasusts dad sets cas EE 5 1 4 Pass Fail rite ra osita e eerie eE chgkorse iuskaxd vadbbe Ue cluskixevaskucasta bate cdusctrevagtedes ELAES REESE rE ESRAR 5 2 PrOCUCE QVEIVICW i moetie e eae a E EA vaGhbcdesatendcevaseisaecdunkins AEE RE E cacklateckunkins casbontesuteadee ESAERA KRENE KERRAT 7 2 1 3 Party Application CompoOnenits SErvices ais sessiachscsicistsansassssacsstoss in ilctcatsiseecvinudincsonstecsscvehlncdssatabaces 8 2 2 CallManager Device Table ccecccececcceeseeeeenceceeeeeeeaeeeeaaeeeeeeeceaeeeeaaeseaeesaeeeseaaesgeaeesseeeesaeeseaaesseneessaees 8 2 3 What Wile Tested mierea santas cates aa seetuy adeno tae aAa a aa A a a aa Eaa a eaaa E NT aaa EE TAA DATE aTa Rn tae 8 2 4 What Will Not Be Tested Aesan aaraa aA aaea aa aaa aE aaa Aaria Eaa Ai La aaa Aeaaeai Ade 9 2 5 Administration and Debugging tools isrrasiisranrienrsaninkirann eiiie arie nanki y ana ENE aa i AORE A TAE EATER aY 9 3 EXO CUTIVE SUMIMALY erasora ye ceanetetacdetenh the oasacceheaecvausnedadavbashovavenbesdacdesuni iesaeseoucivided daedsbedvagesteadbngeatties sole 10 3 1 S mmary oftest rests araisa aitean taaan a Aaaa aad anea A aa naaa Ad aea Raai AE EUa MAATE NA aa A aae 11 SEE Product Test RESUS ss cvccgacecics asst sacdacesk ot gastos TES AARNE E OE
25. ective of these sets of tests is to verify the timeout feature for a Text message in queue Test Setup Properly associate device with User Procedure 1 From the application generate the Text message 2 Set the application timeout parameter for the text message to 100 seconds Expected Confirm that the message is removed from the phones at the end of the time out Results period 23 Printed versions of this document are uncontrolled Look on line for the controlled version 23 January 31 2011 Cisco Systems Inc company confidential Cisco Confidential Observations Passed v Passed v Passed with observation limited functionality or unsupported phone model 7906v 7925 7940 7942v 7960 7962 7970 69417 79117 7985v 7941 Vv 7945v 7961 v 7965V 7971V 7975 6 2 5 Incoming phone call not interrupted by Application sending Text message Test Case Details Title Incoming Phone call not interrupted by Application sending Text Message Description The objective of this test is to verify that the application will not interrupt incoming call Test Setup Properly associate device with User Procedure 1 Place call to IP Phone 2 While phone is ringing have application send text page 3 Complete call and hang up Expected After call is completed text message appears Normal phone operation resumes Results Observations Passed v Passed v Pass
26. ed with observation limited functionality or unsupported phone model 7906v 7925v 7940 7942v 7960 7962V 7970 6941v 7911v 7985v 7941 Vv 7945v 7961 v 7965V 7971 7975 6 2 6 Application places text message in queue Test Case Details Title Application places text message in queue Description The objective of these sets of tests is to verify that the application continues to work when more than one message is in queue Test Setup Properly associate device with User Procedure 1 Send text to phone via application 2 Send 2natext to phone via application 3 Delete first text Then delete second text 24 Printed versions of this document are uncontrolled Look on line for the controlled version 24 January 31 2011 Cisco Systems Inc company confidential Cisco Confidential Expected Results After first message is deleted the second should appear After both are deleted the phones display should return to normal Observations Passed v Passed v Passed with observation limited functionality or unsupported phone model 7906v 7925V 7940V 7942V 7960 7962V 7970 69417 79117 79857 79417 79457 79617 79657 79717 7975 6 2 7 Application able to use Non Alpha Numeric Characters Test Case Details Title Application able to use Non Alpha Numeric Characters Descripti
27. eeeaaeseceeesaeeesaeeseneeeeaees 26 6 3 2 Test Case Overnight CM performance monitoring 0 c cccceecceceeeeeeeeeeeeeeteeeeseaeeeeaaeeeenees 27 6 4 Informational LeSts hss tahu ect aene oe aa atari dete Metis atin ees 29 6 4 1 Calculate Time to receive text message from application ccccceeeeeeeceeeeeseeeeeeeeeeeaees 29 6 5 Phase S Negalive Tests its ncas Vai tet padre ache eal it a al NU A oar 31 6 5 1 Primary server loses connection reconnects after the IP Phone has received text message 31 6 5 2 Primary server looses power after text message has been sent to phone l 31 6 5 3 IP Phone loses connection while message is displayed reconnects after 5 seconds 32 6 6 Phase 6 Additional Application Specific Test cases cccccceeeeceeseeeeeeeceeeeeeaeeeeaaeseeeeeseaeeesaeeeeneeee 34 6 6 1 Read New Messages 20 dan ce aa de da ce td ee dee ce ae 34 6 6 2 Read New Messages ccccccceceeseeeeeeeeneeeeeeaeeeeeenaeeeeeenaeeeenes Error Bookmark not defined 7 pelore oaeee e cet a a a a a awed a dieawssa ll ceuehametatainteaestnetesialgaesiait beens cneeesiee 35 3 Printed versions of this document are uncontrolled Look on line for the controlled version 3 January 31 2011 Cisco Systems Inc company confidential Cisco Confidential 8 ETOT I AAEE E coat aot tari ecco Sap a coe needs end Anca eed aye Svat ois Sb cere tle gene totes 35 S 1 GATAGMIMONUS sess ccecs lates
28. eeeeeeeeeseesenaneeeeeees 19 6 1 4 3rd party Personal Assistant configuration with reference to 3rd party Server cccceceeseeeeee 20 6 1 5 3rd party application Shutdown eeeecceeeeeeeeeee eee aeeeeaeeeeeeee cae eeceaeedeeeeseeeeeceaeseseeeeseaeeseaeeeeeeeeeeeess 21 6 2 Phase 2 Application Reliability Verification ceccccecceeeeeeceeeeeceneeeeeaeeeeeeeseeeeeseaeeeeaaeseeeeeseaeeeseeeseeeeees 22 6 2 1 Application is available for a valid MEVICE ee cece ee eeeeee ee eeee ee ee eeeeeeeeaeaeeeteeaeeeeeeeaaees 22 6 2 2 Application Interaction is correctly handled for IM page eect eeeeeeeeeeeeeeeeeeeeeaees 22 6 2 3 Application able to use Non Alpha Numeric Character s cc cccccesseeesssseeeessssseeesseesaees 23 6 2 4 Timeout Support for Specified Text messages ccccececeee cee ee cee eeeeaeeeeeeeseaeeesaeeeeaaeeeenees 23 6 2 5 Incoming phone call not interrupted by Application sending Text message 00 24 6 2 6 Application places text Message in queue 0 ee cece ee eee eee eeeeeeeeeteeeeeeeeeeaeeeeeteaeeeeeneates 24 6 2 7 Application able to use Non Alpha Numeric Characters cc cccccceseeeesseeteeesseseeeesssseaees 25 6 3 Phase 3 Scalability Stress Performance and Load Testing cccccesceceeeeeeeeeeeeeeeeeeeeeseeeeseaeeneneeees 26 6 3 1 CUCM Baseline Measurement cccccccccceeeeeeceeeeeceeeeceaeeeeaae cease ceae
29. fidential Cisco Confidential 2 Product Overview ECG Cisco is a gateway between the Unite messaging system and Cisco IP phones and sends messages from Unite to Cisco It contains a simple configuration Graphical User Interface GUI and interacts with the Cisco Unified Communications Manager CUCM and the Cisco IP phones The Cisco Unified Communications Manager is the central node in a Cisco phone network with information about all phones and calls It also keeps track of which phones that exist and which phones that are connected z CiscolP phones gt t Configuration GUI Client I AN LAN TT a gt l S mema FEA CCM ECG Cisco ra ii o j r ile nite v _ c Figure 1 An example with the ECG Gsco gateway and par of a Unite system The ECG makes inquiries every minute about added or modified devices and the status of every device i e if they are logged in using SNMP To make sure nothing is inconsistent the ECG also extracts a complete list of all registered phones in the CUCM at a regular configurable interval default once a day When a message is received from Unite the status of the intended receiver is checked and ECG Cisco accepts the message if the phone is currently online A common way to handle messaging is to let the phone store a number of the latest messages for viewing but the Cisco phones are not equipped with any local message storage It is therefore up to the EC
30. for the controlled version 47 January 31 2011 Cisco Systems Inc company confidential Cisco Confidential Select Cisco CTIManager from the list of Configured Services The Trace On check box should be checked Under Trace Filter Settings The Debug Trace Level should be set to Detailed The Cisco CTIManager Trace Fields check box should be checked The Enable All Trace check box should be checked Under Trace Output Settings The Enable File Trace Log check box should be checked The File Name edit box should contain a valid path and file name The default for Maximum No of Files is 250 The default for Maximum No of Lines per File is 10000 The default for Maximum No of Minutes per File is 1440 As with the SDL traces after the test has been running a while check to see if the CUCM trace files are being overwritten too soon before they can be copied If they are increase the Maximum No of Files parameter appropriately END 48 Printed versions of this document are uncontrolled Look on line for the controlled version 48 January 31 2011 Cisco Systems Inc company confidential
31. ge from application Test Setup Properly associate device with User Procedure 1 From the application send text to all phone models listed below 2 Calculate the amount of time it takes to the text message Expected Phones should receive text message in less than 2 seconds Results Observations Passed v Passed v Passed with observation limited functionality or unsupported phone model 7906v 7925V 7940V 7942V 7960 7962V 7970 69417 79117 7985v 7941 Vv 79457 79617 7965 7971x 7975 29 Printed versions of this document are uncontrolled Look on line for the controlled version 29 January 31 2011 Cisco Systems Inc company confidential Cisco Confidential 30 Printed versions of this document are uncontrolled Look on line for the controlled version January 31 2011 Cisco Systems Inc company confidential 30 Cisco Confidential 6 5 Phase 5 Negative Tests These tests are executed to determine the ability of the impact on calls the Call Manager and the 3rd party application when combinations of the aforementioned fail by power failure or network connectivity problems Testing robustness of the application through hardware and software fault insertion i e Failover falloack These tests are executed to verify specific information about the third party product to Cisco This is in relation to the IVT Questionnaire supplied by the vendor 6 5 1 Primary server l
32. he urgency of needing to have the Defect fixed Problem resolution times depend on the priority various attributes of software hardware and the development team hence it is not predictable 44 Printed versions of this document are uncontrolled Look on line for the controlled version 44 January 31 2011 Cisco Systems Inc company confidential Cisco Confidential 12 Appendix C IVT Troubleshooting Tools and Default configurations Trace Utility for CUCM This trace provides the greatest level of detailed information of a call This can be enabled with follow available trace levels Error Traces alarm conditions and events Special Traces all error conditions plus process and device initialization messages State Transition Traces all Special conditions plus subsystem state transitions that occur during normal operation Significant Traces All State Transition conditions plus media events that occur during normal operation Entry Exit This trace level is not currently used Arbitrary Traces all significant conditions plus low level debugging information Detailed Traces all Arbitrary conditions plus detailed debugging information Note Phases 3 Performance and 6 Failover should have trace settings configured for the CTI Manager and CUCM services for each CUCM machine Use the following instructions to configure this tracing Open CUCM Administration gt Personal Assistant gt CUCM Serviceability
33. ing 5 1 Test Engineers and Technology Contacts The engineers responsible for the IVT test efforts are listed in this section with their roles and responsibilities A second table includes the internal external email aliases used during testing to relay status information or for technical consultation In addition in this table lists of internal external web pages are identified as IVT resources Table 5 1 IVT Test Engineers Oumar Dalissia Test Lead Table 5 2 Vendor Contacts Staffan Ornbratt 46 31 559300 Staffan ornbratt Product Manager ascom se 16 Printed versions of this document are uncontrolled Look on line for the controlled version 16 January 31 2011 Cisco Systems Inc company confidential Cisco Confidential Table 5 3 IVT related email aliases and web pages 5 2 Issues from previous IVT testing None 17 Printed versions of this document are uncontrolled Look on line for the controlled version January 31 2011 Cisco Systems Inc company confidential 17 Cisco Confidential 6 Detailed Test Cases This section details the tests that will be performed during the testing period Unless otherwise noted in the individual test case all tests shall be executed with a background load provided by a load generation utility such as Camelot or SimClient resulting sustained in Call Manager CPU utilization of 65 Refer to Table 1 Te
34. lity and or the device is not supported by the partner such as the 7985 video IP phone A number of test cases were either not supported N S or not applicable N A Observation Test Case 6 2 4 Timeout Support for Specified Text messages All IP phones respond normally to the third party application as expected Except the 7906 7911 amp 7985 models which have limited functionality and cannot delete the last message from the screen by pressing the DELETE button A workaound solution is to simply either lift the handset or press services button and the display will return to normal or refresh Note also that during the installation and configuration process the Web access parameter must be enabled on all IP phones through the CUCM Administration Page for the application to function normally The Partner should alson update the User Installation Guide to include this key information to facilitate Intallation of the software ASCOM Unite Messaging Suite Version 1 21 is a very useful tool easy to deploy 10 Printed versions of this document are uncontrolled Look on line for the controlled version 10 January 31 2011 Cisco Systems Inc company confidential Cisco Confidential 3 1 Summary of test results This is an example of summary test results from IVT SDL testing The results of the summary are taken from the Test Cases section The purpose of this is for AVVID Partner Program Team to review without the need
35. ment are uncontrolled Look on line for the controlled version 37 January 31 2011 Cisco Systems Inc company confidential Cisco Confidential Indicator MWI user s voice mail and set to OFF whenever a new message has been listened to Mute Mute allows you to listen but the other party cannot hear you Shared Line A shared line extension is an extension shared by two or more phones Extension similar to a POTS party line However unlike a party line if one of the shared line IP phones makes or receives a call the other phones cannot hear the conversation Speakerphone You can use speakerphone to place and answer calls without using the handset Use speakerphone with any other feature Transfer Transfer allows you to send an existing call to another extension You can cancel the transfer by pressing HOLD 38 Printed versions of this document are uncontrolled Look on line for the controlled version 38 January 31 2011 Cisco Systems Inc company confidential Cisco Confidential 9 Attachments None This section documents the partner issues arising from this IVT and the action we expect them to take to rectify any issues The second table is used to identify minor changes to documentation that we require 1 Partner to resolve advice Description Test Test Table Case Please review Executive Summary 2 Partner to reflect the following outcome in their documentation Descri
36. ng measurement against it Test Setup Reboot of the CUCM machines Counter log configured in CM1 CM2 and CM3 s Win 2K Perfmon No J TAPI connection between the Test Product and the CUCM cluster Traces configuration CDR ON default CCM ON ON by default SDL ON ON by default CTI_Manager ON Procedure 1 Load SimClient configuration and register phones clients to be used 2 Five minutes after SimClient registration start Perfmon logging on CUCM machines being used Wait five minutes to gather idle baseline usage information Start test calls Wait ten minutes Stop phone calls Wait five minutes to gather post test idle baseline usage information 7 Gather SimClient performance results i e Calls attempted Calls completed Off hook to dial tone Hang up to on hook etc Stop Performance Monitor logging on amp Expected 1 Verify that CPU usage of CUCM never went above 80 and that average CPU Results usage never went above 60 2 Record the results Paste a screen shot of the Perfmon graph with the PROCESSOR PROCESSOR TIME highlighted Save CSV file to the IVT Doc Centre Observations Passed v Passed v Passed with observation limited functionality or unsupported phone model 26 Printed versions of this document are uncontrolled Look on line for the controlled version 26 January 31 2011 Cisco Systems Inc company confidential Cisco Confidential 7906v 79
37. ng with the CTI Devices IP Phones configured on the publisher CUCM Expected 1 Verify that the Call Centre Personal Assistant recognizes all CTI Results Port CTI Route Point IP Phone directory numbers as assigned in CUCM Observations Passed This is a turnkey installation No error reported 20 Printed versions of this document are uncontrolled Look on line for the controlled version 20 January 31 2011 Cisco Systems Inc company confidential Cisco Confidential 6 1 5 3rd party application shutdown Title 3rd party application shutdown Description Verify that that the client Personal Assistant shuts down properly Test Setup None Procedure Follow installation guide s 1 Document how the product can be shut down 2 Validate that this works 3 Record the result Expected As per stated in the procedure Results Passed No error reported 21 Printed versions of this document are uncontrolled Look on line for the controlled version January 31 2011 Cisco Systems Inc company confidential 21 Cisco Confidential 6 2 Phase 2 Application Reliability Verification These tests test the various features of the 3 party application and its various components This involves the testing of the application against the Application note and IVT questionnaire requirements to ensure that it functions reliably and consistently in a manner that meets the requirements
38. of going into the full detail of the report v pass x fail v See Defect Tracking Information Problem Reporting and Severity Levels section N A If any section is not applicable to your product please state N A N S Not supported N T Not tested 3 1 1 Product Test Results Phase 1 Installation amp Configuration Phase 2 Application Reliability Verification Phase 3 Scalability Stress Performance and Load Testing Phase 4 Informational tests Phase 5 Negative Tests Phase 6 Live Monitoring of VoIP Recording Phase 7 Recording on Demand Phase 8 Voice Quality oO oO O O O O O O O OI oO O O O O O O O OI O O O O O O O O OI O O O O O O O O OI O O O O O O O O OI O oO O O O O O O Final Problem Counts 11 Printed versions of this document are uncontrolled Look on line for the controlled version 11 January 31 2011 Cisco Systems Inc company confidential Cisco Confidential 3 1 2 Problem Report The following table highlights all problems discovered and their date resolved if any 12 Printed versions of this document are uncontrolled Look on line for the controlled version 12 January 31 2011 Cisco Systems Inc company confidential Cisco Confidential 4 Test Setups The IVT test beds represented below include the following diagrams to support transit voice calls and su
39. of this document are uncontrolled Look on line for the controlled version 31 January 31 2011 Cisco Systems Inc company confidential Cisco Confidential Description Check the behavior of the module when the primary server is shutdown Test Setup Properly associate device with User Procedure 1 Send text message to the phone 2 While the message is displayed on the IP Phone remove power from application server 3 Make extension to extension call Expected 1 Message is displayed on the IP Phone Results 2 After the extension to extension call phone display returns to normal 3 Record the way the IP Phone reacts to the server s failure Observations Passed v Passed v Passed with observation limited functionality or unsupported phone model The Server must be connected to the network for the CALL or DELETE command pressed on the phone to work All phones meet the expected results If the IP user presses the delete button the text message cannot be deleted The phone will try to request the service and will time out A workaround solution is to simply lift handset or press services button and the display will return to normal 7906v 7925V 7940V 7942V 79607 7962V 7970 69417 7911v 79857 7941 v 79457 79617 79657 7971x 7975 6 5 3 IP Phone loses connection while message is displayed reconnects after 5 seconds Test Case Det
40. on The objective of these sets of tests is to verify that the IP Phone can and will support non Alphanumeric characters Test Setup Properly associate device with User Procedure 1 Where appropriate attempt to use the IP Phone service with characters containing non alpha numeric characters e g amp amp etc Expected The phones will be able to display non alph numeric characters Results Observations Passed v Passed v Passed with observation limited functionality or unsupported phone model 7906v 7925 7940V 7942V 7960 7962V 7970 69417 79117 7985v 7941 Vv 7945v 79617 79657 79717 7975 25 Printed versions of this document are uncontrolled Look on line for the controlled version 25 January 31 2011 Cisco Systems Inc company confidential Cisco Confidential 6 3 Phase 3 Scalability Stress Performance and Load Testing These tests are executed to determine the impact of the 3 party software on the Cisco Communications s ability to process calls Testing will also determine the outer limits of the applications ability to properly function under stress and perform characterizing measurements on the Communications Manager 6 3 1 CUCM Baseline Measurement Test Case Details Title CUCM Baseline Measurement Description CUCM Baseline idle state no CTI link to the Test Product Have a performance baseline to compare further performance monitori
41. onfigured phone Observations Passed 22 Printed versions of this document are uncontrolled Look on line for the controlled version 22 January 31 2011 Cisco Systems Inc company confidential Cisco Confidential v Passed v Passed with observation limited functionality or unsupported phone model 7906v 7925V 7940 7942v 7960 7962V 7970 6941 7911v 7985v 7941v 7945v 7961 7965 7971 7975 6 2 3 Application able to use Non Alpha Numeric Characters Test Case Details Title Application able to use Non Alpha Numeric Characters Description The objective of these sets of tests is to verify that the IP Phone can and will support non Alphanumeric characters Test Setup Properly associate device with User Procedure 1 Where appropriate attempt to use the IP Phone service with characters containing non alpha numeric characters e g amp amp etc Expected The phones will be able to display non alph numeric characters Results Observations Passed v Passed v Passed with observation limited functionality or unsupported phone model 7906v 7925 7940 7942v 7960 7962V 7970 6941v 7911v 7985v 7941v 7945v 7961 7965 7971 7975 6 2 4 Timeout Support for Specified Text messages Test Case Details Title Timeout Support for Specified Text Messages Description The obj
42. oses connection reconnects after the IP Phone has received text message Test Case Details Title Primary server loses connection reconnects after the IP Phone has received text message Description Check the behavior of the 3ra party server when it loses connection reconnects after the IP Phone has received text message Test Setup Properly associate device with User Procedure From application send text messages to all phones listed below Confirm all phones received text message Disconnect the 3ra party s primary server from the network Send second text message Expected Results There should be no interaction required for server to connect with CUCM p 2 3 4 Reconnect 3ra party s primary server to the network 5 1 2 All phones should see both messages and be able to delete both messages Observations Passed v Passed Passed with observation limited functionality or unsupported phone model The Server must be connected to the network for the CALL or DELETE command pressed on the phone to work 7906v 7925 79407 7942 79607 7962V 7970 69417 79117 7985v 7941 Vv 7945v 7961V 7965 7971x 7975 6 5 2 Primary server looses power after text message has been sent to phone Test Case Details Title Primary server looses power after text message has been sent to phone 31 Printed versions
43. ption Test Test Table Case None 10 Internal Review This document was reviewed internally by Cisco comments can be found at 39 Printed versions of this document are uncontrolled Look on line for the controlled version 39 January 31 2011 Cisco Systems Inc company confidential Cisco Confidential Appendix A 3rd Party Serviceability Information 3 Party Support Contact Information Please list in this section all support contact details as supplied in the IVT Questionnaire Team Main Location Additional Location Additional Location Phone Fax E mail Hours The IP Telephony category requires 24x7 support contact List all applicable after hours numbers 3 Party Product Isolation Procedures Please describe in this section how to isolate your product from the CUCM to help in determining the cause of any issue Troubleshooting In this section please provide the links or names of the appropriate PDF files to help in trouble shooting the application that was tested You will have to also provide the PDF documents with this report in order to have coordinated support Installation and Configuration Upgrades and patch application Debugging and Diagnostics Error Messages Tools Aids List of Commands Redundancy and Recovery 40 Printed versions of this document are uncontrolled Look on line for the controlled version January 31 2011
44. rface workaround This is the highest level for documentation bugs Things fail under very unusual circumstances Very unusual and recover pretty much by themselves Users circumstances cause don t need to install any work around and failure Automatic performance impact is tolerable Example The recovery without first segment for each TCP connection is workaround Any always retransmitted performance impact is tolerable Cosmetic No real detrimental effect on system No detrimental effect functionality Example connection spelled on system connection in a display functionality Enhance requests Enhance requests should normally get cosmetic It is expected that any problems witnessed by IVT testing teams will result in the opening of a Defect The following information is required to successfully create the Defect Severity use the guidelines listed above to determine the severity In addition to a severity level Defects are also given a priority status Priorities are used to indicate the urgency of fixing a problem This is especially important for issues facing products that are in the deployment phase of their lifecycle The DE Priority field helps to set monitor and communicate 43 Printed versions of this document are uncontrolled Look on line for the controlled version 43 January 31 2011 Cisco Systems Inc company confidential Cisco Confidential business priorities that allow appropriate response
45. s of this document are uncontrolled Look on line for the controlled version 36 January 31 2011 Cisco Systems Inc company confidential Cisco Confidential TAPI Auto Registration Call Forward Busy Call Forward No Answer Call Park Call Waiting Conference Configuration file Group Pick Up Failover Failback Forward All Hold Manual Registration Message Waiting 37 AKA Microsoft Windows Telephony API TAPI is a standard group of Win32 APIs that allow communications applications to control telephony functions Auto Registration is a feature of CUCM where a phone can be added to the network The phone will get an IP address and TFIP address through DHCP and find the CUCM through the CNF provided information Configurable feature that re routes incoming calls to an alternate line when the first line is in use Configurable feature that re routes incoming calls from one phone to another phone when the first phone is not answered after a certain number of rings Call Park allows you to place a call on hold at an extension specified by your system administrator so that anyone on the IP Telephony network can retrieve it For example you could park a call at extension 3000 Anyone on the system can dial 3000 to retrieve the call Call Waiting lets you receive a second incoming call on the same line without disconnecting the first call When the second call arrives you hear a brief call waiting indicator
46. sed throughout this document ACD Automatic Call Distributor A device that distributes calls to agents based on administratively settable rules BHCA Busy Hour Calls Attempted BHCC Busy Hour Calls Completed DID Direct Inward Dialing DNIS Dial Number Identification Service the telephone number being dialed same as called party number DSP Digital Signal Processor E1 32 64kpbs timeslots on a 2 048 Mbps serial interface ICS Integrated Communication System IP Internet Protocol ISDN Integrated Services Digital Network IVR Interactive Voice Response IVT Interoperability Verification Testing 35 Printed versions of this document are uncontrolled Look on line for the controlled version 35 January 31 2011 Cisco Systems Inc company confidential Cisco Confidential LAN MCS MCU POTS PRI PSTN RAS RTCP RTP T1 UM Agent Phone Call Center Client App JTAPI Prompt quality SimClient SoftPhone Speech quality 36 Local Area Network Media Convergence Server Multi point Control Unit Plain Old Telephone Service Primary Rate Interface ISDN interface to 64kbps D channel plus 23 T1 or 30 E1 B channels for voice or data Public Switched Telephone Network Registration Admission Status Real Time Transport Control Protocol Real Time Transport Protocol 24 64kpbs timeslots on a 1 544 Mbps serial interface Unified Messaging capabilities A voice mail system that includes fax and email
47. shion Max lines file specifies the maximum number of trace statements that will be written to each log file For example a value of 1000 will cause up to 1000 trace statements to be written to each log file Event Log CUCM events are logged in the Windows 2000 Event Log User can use the Event Viewer to see the system security and Personal Assistant events for the Windows 2000 Server and CUCM services The CUCM errors are logged under Personal Assistant logs What to collect when preparing for and reporting CM related problems When you are collecting information to report or troubleshoot a CCM problem please use the following procedure to ensure you are gathering the correct information 1 Make sure the SDL SDLxxx_100_xxxxxx files trace settings are set appropriately Check set the following CUCM service parameters from the CM administration Under Service gt Service Parameters select the IP address of the server Then select the CUCM service from the list of services SdiTraceDataFlags set to 0Ox000001 10 SdilTraceFlag set to T SdilTraceMaxLines set to between 10000 and 20000 SdilTraceTotalNumFiles set it to at least 100 If you are running high traffic this number should be increased significantly The best thing to do is after running for a while see how much time looking at the date time of the SDL files is capturing If the files are being overwritten before you would be able to 46 Printed versions of this
48. st Results Legend for the definitions of the annotations in the Pass Fail column 6 1 Installation and Configuration Tests This phase includes making sure the 3 Party application installed correctly and that it registered correctly with the Call Manager These tests will also focus on clean installation configuration and removal of any software components on the Call Manager server s 6 1 1 3 party Server installation and registration Test Case Details Title 3 party server installation and registration Description Installation of the server Personal Assistant Verification of the completeness and accuracy of the server Personal Assistant installation guide Documentation of any required software installation or configuration change on the CUCM Test Setup Very important configure CTI ports to a CTI Manager that is not on the primary CUCM This is typically the database publisher machine Procedure 1 Read the 3 party installation instructions 2 If 3 party product is not turnkey On a designated client machine not one of the CUCM machines install the 3 party product Record length of time required for install 3 Perform any configuration the 3 party product according to the installation instructions Document the various separate installation programs and manual configurations needed Expected 1 Verify that via the partner Personal Assistant that the Server has Results installed correctly 2 Re
49. te 100 component product specific feature test complete Component test coverage and results documented and reviewed Ooooo 41 Printed versions of this document are uncontrolled Look on line for the controlled version 41 January 31 2011 Cisco Systems Inc company confidential Cisco Confidential a Integration test complete Results reviewed and approved by the CTDP Partner Team a Components successfully integrate into IVT lab Hardware a ONLY General Availability hardware will be used for IVT prototypes are accepted by approval from Cisco a Components successfully integrate into IVT lab IVT Exit Criteria All test cases executed and passed Release notes written and reviewed for all defects filed during the test Test results documented reviewed and analyzed in an IVT Test Report IVT Test Report Template ENG 249889 Error report handling software and hardware AON Code corrections to defects found during the IVT should be tested before being introduced in the test bed s Workarounds that deviate from the test end goals are not to be accepted IVT Test engineers should understand and agree on the bug cause effect and solution Test cases that were affected by the bug should be executed again These extra test case executions should be documented Proper documentation should indicate the new version of code and or hardware used Defect Tracking Information Problem Reporting and Severit
50. tone Conference allows you to connect three or more people into one phone conversation An unformatted ASCII file that stores initialization information for an application For CUCM files in the cnf format define the parameters for Cisco IP Phone connection A feature that allows users to pick up incoming calls within their own group or within other call pickup groups by dialing the group call pickup number for that group Failover describes the action a 3 party phone must take to re register to the Secondary CUCM if the Primary CUCM is no longer available Failoack describes what happens when the Primary CUCM becomes available again Forward All lets you set up a Cisco IP Phone so that all calls destined for that Cisco IP Phone automatically ring another phone You can forward all calls to Cisco IP Phones or non Cisco IP Phones Forward All remains in effect until you cancel it Hold lets you store a call on the line for later retrieval While a call is on hold you can place another call or use any of the other features of the phone While on hold the caller hears an intermittent tone Manual registration describes the process by which a user configures an IP telephone manually on the target CUCM Manager Manual registration also requires the following to be manually entered into the phone IP submask DNS TFTP and Default Gateway MWI should be set to ON whenever a new message has been left in a Printed versions of this docu
51. y Levels The following table lists the defined alarm categories and description that are being used by Cisco s DEFECT Distributed Defect Tracking System 42 Printed versions of this document are uncontrolled Look on line for the controlled version 42 January 31 2011 Cisco Systems Inc company confidential Cisco Confidential 1 Catastrophic Reasonably common circumstances cause the Common entire system to fail or a major subsystem to circumstance causes stop working or other devices on the network the entire system or to be disrupted and there s no workaround a major subsystem to Example turning on DECNET causing the stop working affects router to crash after sending gratuitous IP ARP other areas devices replies claiming the addresses of all the no workaround neighboring routers Severe Important functions are unusable and there s Important functions no workaround but the router s other functions are unusable does and the rest of the network work normally not affect other Example IP helper addresses being ignored areas devices no workaround Moderate Things fail under very unusual circumstances Very unusual or minor feature doesn t work at all or things circumstances cause fail but there s a low impact workaround This failure minor feature is the highest level for documentation bugs doesn t work at all Example LAT not working unless you have an there s a low impact IP address on the inte
Download Pdf Manuals
Related Search