Test Automation Tool for POLQA® and PESQ® Speech Quality
Contents
1. The mobile output calibration has to be run for each individual mobile if codec settings are changed the user volume control is changed or if any other settings in the mobile affecting the downlink signal path are changed Rohde amp Schwarz 1GA62 18 Using the Application Program 2 4 5 In Out Calibration For each transmission direction an In Out Calibration routine is offered for the case that the network interface is not accessible loopback on the network side or does not provide the Encoder Cal and Decoder Cal routines In the case of uplink or loopback on the network side the input of the transmission path in the electric input of the mobile under test connected to UPV generator output 1 In case of downlink and loopback on the mobile side the input of the transmission path is the speech coder input on the network side CMU connected to UPV generator output 2 In case of uplink or loopback on the mobile side the output of the transmission path is the speech decoder output on the network side CMU connected to UPV analyzer input 2 In case of downlink or loopback on the network side the output of the transmission path is the electric output of the mobile under test connected to UPV analyzer input 1 The In Out Cal Button starts the calibration routine A pulsed 1 KHz tone is swept in amplitude starting from a high voltage which is gradually decreased In an ideal case the curve of outp2. Wideband extension to Recommendation P 862 for the assessment of wideband telephone networks and speech codecs http www itu int e ITU T P 862 3 2005 Application guide for objective quality measurement based on Recommendations P 862 P 862 1 and P 862 2 http www itu int e ITU T P 863 2011 Methods for objective and subjective assessment of speech quality Perceptual objective listening quality assessment http www itu int e User Manual POLQA OEM Library Opticom GmbH e Application Note Voice Quality with ITU T P 863 POLQA Swissqual http www swissqual com index php pages downloader Application Note_ _Voice_Quality_with_ITU_P 863 POLQA pdf e ITU T Application Guide for P 863 unpublished draft Rohde amp Schwarz 1GA62 32 Ordering Information 4 Ordering Information Ordering Information Type of instrument Instrument type Ordering number Audio Analyzer R amp S UPV or 1146 2003 02 R amp S UPV66 1146 2003 66 XLR BNC Adapter Set R amp S UP Z1MF 1411 3306 02 Headset Cable Set for UPV R amp S UP Z9 1411 3106 02 Universal Sequence Controller R amp S UPV K1 1401 7009 02 SW for PESQ Measurements R amp S UPV K61 1401 7309 02 Listening quality analysis R amp S UPV K63 1402 1156 02 POLQA to ITU T rec P 863 Universal Radio Communication R amp S CMU200 1100 0008 02 Tester Wideband Radio Communication R amp S CMW500 1201 0002K50 Tester For R amp S CMU and R amp S
3. the calibration routine For encoder calibration the application program adjusts the voltage at output 2 of the UPV generator to the encoder circuit on the network side CMU such that the full scale decoder output voltage is measured at UPV analyzer input 2 For a circuit switched connection the CMU must be set to Encoder Cal in GSM Traffic Mode on the Network tab in WCDMA Voice Settings gt Voice Source in the BS Signal tab in CDMA2000 Primary Service Class gt Speech Service gt Service Option 3 gt Voice Coder in the Service Cfg Tab This operation mode requires an active call to the mobile under test In other setups the analog output on the network side must be configured to loop back the signal from the input to the output with a gain of 1 Fora VoLTE setup with external media server auxiliary program CsWavRec must be started while the media server software is idle or closed When the Encoder Cal button in the application program is clicked the measurement routine starts and tries to adjust the generator output voltage such that the full scale decoder output voltage obtained in 2 4 1 is achieved If the full scale decoder output voltage cannot be adjusted a warning message is output and the routine is aborted without storing the result If a stable result is obtained the routine additionally measures the full scale output voltage from the electric interface of the mobile Fil
4. 1 uplink CSS_32k 4 0 12 50 57 12 51 55 247 646 246 000 246 917 2 620 E F Downlink CSS_32k 4 0 12 52 03 12 53 27 205 488 205 792 205 208 2 537 Figure 35 Controls of the delay results tab Rohde amp Schwarz 1GA62 31 2e Literature Literature e R amp S UPV Operating Manual http www2 rohde schwarz com en service_and_support downloads download_search e Application Note 1GA4 7 Calibrating Measuring Microphones and Sound Sources for Acoustic Measurements with Audio Analyzer R amp S UPV http www2 rohde schwarz com en service_and_support Downloads Application_Notes e Application Note 1GA49 Psychoacoustic Audio Quality Measurements Using R amp S UPV Audio Analyzer http www2 rohde schwarz com en service_and_support Downloads Application_Notes e Application Note 1GA50 Calibration Tool for PESQ Speech Quality Tests http www2 rohde schwarz com en service_and_support Downloads Application_Notes e Application Note 1MA202 Next Generation 3G 4G Voice Quality Testing with POLQA http www2 rohde schwarz com file 1MA202_1e pdf e ITU T P 862 2001 Perceptual evaluation of speech quality PESQ An objective method for end to end speech quality assessment of narrow band telephone networks and speech codecs http www itu int e ITU T P 862 1 2003 Mapping function for transforming P 862 raw result scores to MOS LQO http www itu int e ITU T P 862 2 2005
5. and delay measurements set the speech coder to Handset Low For the calibration functions follow the instructions on the screen WE AN Switch i LAN paiia server USB upv Figure 6 VoLTE downlink signal path For voice over LTE connect e UPV generator output 2 to inputs 1 and 2 of the USB sound card e mobile output headset or handsfree carkit connector to UPV analyzer input 1 For the PESQ POLQA and delay measurements establish a VoLTE call between media server and mobile For the calibration functions follow the instructions on the screen Rohde amp Schwarz 1GA62 12 Using the Application Program 2 3 3 Loopback on the Network Side External Audio Headset or Handsfree Car Kit RF Connection Loopback Figure 7 Signal path for circuit switched loopback on the network side For circuit switched speech connect e UPV generator output 1 to mobile input headset or handsfree car kit connector e mobile output headset or handsfree car kit connector to UPV analyzer input 1 Establish call from the CMU to the mobile For the PESQ POLQA and delay measurements set the speech coder to Loopback For the calibration functions follow the instructions on the screen Figure 8 Signal path for loopback on the network side in VoLTE For voice over LTE connect e UPV generator output 1 to mobile input headset or handsfree carkit connector e mobile output headset
6. cannot be adjusted a warning message is output and the routine is aborted without storing the result If a stable result is obtained a file selector allows to choose an appropriate location and file name for the result value The working directory D UPV 1GA62 should be the preferred location for these files Mobile input calibration with sine burst signal is only applicable if the end to end delay is between 50 ms and 260 ms As a default mobile input speech calibration should be used see 2 4 3 2 The mobile input calibration value is alternatively obtained with the In Out Calibration routines of downlink or loopback on the mobile side M I Speech Cal For codecs which do not transmit a sinewave signal with sufficient fidelity and for end to end delays below 50 ms and above 260 ms this button provides a calibration method using natural speech 2 4 4 Mobile Output Calibration This calibration value is required for downlink measurements and for measurements with loopback on the network side A valid encoder calibration value is prerequisite for the calibration routine Another prerequisite for the mobile output calibration is an active end to end speech connection When the Mobile Output Cal button in the application program is clicked the UPV generator sends a speech signal and measures the peak output voltage of the mobile This value can later be used to set the fix analyzer range for the measurements
7. min bl Delay mz 6 000 6 000 6 000 8 000 6 000 6 000 5 000 6 000 8 000 6 000 6 000 6 000 8 000 6 000 6 000 5 000 null 3 61 eRtis 3 65 ge 3 49 3 56 3 46 3 54 4 01 3 99 a 91 3 06 4 06 4 94 4 06 3 07 CERF 3 90 a 75 a4 3 65 4 61 3 65 3 66 3 62 4 04 4 15 3 99 4 00 4 15 4 07 4 15 4 01 4 16 Figure 27 Overview over averaged results in the results data grid 3 76 2 61 3 39 3 65 344 2 97 a323 aoe 3 93 42 3 65 ae a a4 4 76 4 00 a2 2 61 142 147 144 144 147 147 147 147 146 147 147 146 153 147 147 149 147 mas D min 146 143 L 2 149 147 147 147 147 147 146 147 147 175 145 145 169 175 110 145 144 147 147 146 147 146 146 147 ag 114 147 105 147 147 oo E Clicking on one of the small signs in the row headers opens a new row with a link to another table showing the detailed results from which the respective row was averaged M POLOA SWB Uplink StatisticsResult ef sO01 Swe Figure 28 Link to detailed results in the results data grid Clicking on the link displays the detailed data Results 8 000 3 81 4 90 3 76 142 p o 4 Statistics No 1 Measurement POLOA BW SWB Direction Uplink Signal ef 2_s001_3 P oo Nol Measure B i Direction Signal FOLQA FOLQA FOLQA FOLQA FOLQA FOLQA POLQA SWE SWE SWE SWE SWE SWE SWE Uplink Uplink Uplink Uplink Uplin
8. point of the input output curve For Voice over LTE VoLTE tests with a VoIP client installed on a PC or on the UPV itself the Decoder Cal and Encoder Cal functionality can be realized by auxiliary programs or by appropriate settings in an audio editor The application program allows to specify a test signal and the test signal level relative to full scale of the input of the transmission path Furthermore it allows to load a prepared setup and enter the generator reference value and analyzer range according to the determined full scale values To achieve a correct spectrum of the speech signal in the sound pressure at the so called Mouth Reference Point MRP an equalization of the electric input signal is required A coarse equalization can be done with an appropriate filter loaded in the Generator function panel of the UPV The described application program provides means for loading such a filter automatically A filter file for equalizing the typical average frequency response of the Bruel amp Kjaer Head and Torso simulator 4128 is included with the files provided The microphone in the artificial ear delivers an electric signal which is proportional to the sound pressure at its membrane Eardrum Reference Point DRP To obtain a flat frequency response related to a virtual Ear Reference Point ERP or related to the sound pressure in a diffuse sound field or free field respectively appropriate correction
9. tests A mapping function may be applied for better matching between the objective and subjective MOS scores PESQ and POLQA are so called full reference test methods They analyze the original Reference speech signal and the degraded signal and compare attributes extracted from both for obtaining an assessment of the degradation 1 2 Bandwidth and Listening Situation In telephony different bandwidths are defined e Narrowband 300 Hz 3400 Hz e Wideband 100 Hz 7 kHz e Superwideband 50 Hz 14 kHz e Fullband 20 Hz 20 kHz PESQ provides a narrowband and a wideband mode In both cases listening tests for the verification of the algorithm were performed monaural using a so called IRS type receiver The same applies to the POLQA narrowband mode In POLQA superwideband mode the subjective tests are performed with high quality stereo headphones POLQA does not provide a wideband mode Rohde amp Schwarz 1GA62 4 2e ITU T P 863 Signal bandwidth mode Super Superwideband wideband wideband narrowband Narrowband Narrowband Listening situation Flat headphone diffuse field equalized diotic presentation IRS receive handset monotic presentation POLQA and PESQ Measurements MOS ACR superwideband scale Narrowband scale comparable to b ITU T P 862 in MOS LQOsw MOS LQOn conjunction with b ITU T P 862 1 Figure 1 Modes of POLQA and their applicat
10. 6 0 dBmo In Out Cal In Out Speech Cal signal Test Signal wav File fc UPV Config ref PolqaRefa8000 wav Browse Clear parameters Play Set Fs of wav file to 4 0 H dBFs pk of mobile input Scale Pk to FS Single Cont load setup _ Load Setup D UPV IGA62 POLOA_Wide_Uplink_Autogain set Browse and Load Setup enter signal calibration l Use Generator Filter Browse values and filters for analyzer and generator M Minimize A i Change to Test Control Minimize After Loading Setup Atar Loading Setup Clear All Values Close fe Start Measurement After Loading Setup fv Change to Results for Test Figure 2 User interface of the application program The main window of the application program provides nine tabs The first four tabs are labeled according to the four transmission types uplink downlink Loopback on Network Side and Loopback on Mobile Side Each tab provides the appropriate calibration functions for the respective transmission type In addition there is one tab each for monitor test control and results delay measurement and delay results 3 tt Rohde amp Schwarz 1GA62 9 2e Using the Application Program Two help buttons on each of the first four tabs give instructions in schematic and in text how to set up the connections Connection types can be circuit switched with CMU200 and Voice over LTE with CMW500 and external VoLTE client Media Server T
11. 8 500021 3 851822 3923364 3 762919 483 9594 507 6291 462 425 01 14 2013 09 44 49 A GA QAI Mohiln line nd cl RAT SPAY 4 RIA 4 RAS OS FAJI AAR Se OA PEA MIA ASNT NA AR AA Figure 33 Test results imported to a spreadsheet excerpt Rohde amp Schwarz 1GA62 29 2e Using the Application Program 2 9 Delay Measurements The Delay Measurements tab provides controls for measuring the end to end delay of a speech connection ioj Choose one of the AAL au Taa T E __ fr The speed factor i i l two available test Ap ON MARATE SAAR AOOO PODNO SKA NCE TIMECODE PAn speeds up the cross signals according to Signal ee correlation by the maximum expected end to end delay Resolution 0 083 ms undersampling 55 with 32k PH size maximum delay 0 6 5 k Speed Fact imtesi f 2 93 dBFs peak of encodes input Scale Pk to FS Start single delay oet the level of the measurement test sign al Subtract Delay Single Measurement Specify test Uplink J ms Uplink _ Downlink _ Network Loop Mobile Loop Delay result equipment delay to Jm Del ay ms Figure of merit for the be subtracted from 0 0 ms cross correlation If the end to end delay Loop on Mobile Side ms S N Factor to Next Lower Crosscorr Peak the S N factor is below 2 the result is Set repetition count and ii deemed to be invalid i Humber of repettions for es lowest values parameters
12. CMW options see the brochures and the R amp S web site Rohde amp Schwarz 1GA62 33 About Rohde amp Schwarz Rohde amp Schwarz is an independent group of companies specializing in electronics It is a leading supplier of solutions in the fields of test and measurement broadcasting radiomonitoring and radiolocation as well as secure communications Established 75 years ago Rohde amp Schwarz has a global presence and a dedicated service network in over 70 countries Company headquarters are in Munich Germany Regional contact Europe Africa Middle East 49 1805 12 42 42 or 49 89 4129 137 74 customersupport rohde schwarz com North America 1 888 TEST RSA 1 888 837 8772 customer support rsa rohde schwarz com Latin America 1 410 910 7988 customersupport la rohde schwarz com Asia Pacific 65 65 13 04 88 customersupport asia rohde schwarz com This application note and the supplied programs may only be used subject to the conditions of use set forth in the download area of the Rohde amp Schwarz website
13. E A A 12 2 3 3 Loopback on the Network Side ccsscceeeeeeeeeeeceeeeeeeeeeneseeneeeessneseeees 13 2 3 4 Loopback on the Mobile Side ccccceeeeceeeeeneeeeeeeeeeseeecneseaneeeesseneneeees 14 2 3 5 Connection Cable between UPV and CMU Speech Connector 15 2 3 6 Connection Cable between UPV and UE Headset Connector 15 2 4 Calibration Functions aie set tase cance seer ce seneoemnctmenscevecnwetensneyaceedeneeeerevnewee 16 2 4 1 DOCOGEE CANNON sasasi ERN EAEE SENEE EEA 16 2 4 2 Encoder C AMPA OR sssr Eaa aaah 17 2 4 3 Mobile Input Calibration cccccccceeeeeeeeeeeeeeneeeneeeeseeneesensenessaneeneseenesonees 17 243 1 Moile NIUE Cakna aaaeei 18 24 3 2 Wig Vs DDECEIN Cal esinsin aa a ai 18 2 4 4 Mobile Output Calibration nannnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnn nnne 18 2 4 5 mM OUE G AU ACEO erra R EERE EAE ESE ES 19 2 4 6 Loading Calibration Values from File cccccesseesseeeeeeeeseeeeeneseeneeeeees 19 2 5 Loading a Setup and Starting a Measurement in Manual Mode 20 2 5 1 Configuring the Test Signal c cccceeeeeesseceeeeeeeeeeeneeseneseesessenseeneeseenes 20 2 5 2 Choosing an equalizer filter ccccccceeeeeeeeceeeeeeeeeeneeseneseeseseenseeneeseneees 21 2 5 3 Options to be Carried out after Loading the Setup cssseeeeeees 21 2 5 4 Eer oiia e hi CD OE sce en eats eee tone E E A 21 2 6 Using the Monitor fU
14. OLQA according to ITU T P 863 is provided with option UPV K63 A description of all parameters and results is given in the current manual of the UPV and in the on screen context help Detailed application guides are available in application notes 1GA49 and 1MA202 1 5 POLQA vs PESQ POLQA was developed to enhance the range of test factors coding technologies and applications compared to PESQ Comparison PESQ POLQA Test Factors Test Factor PESQ Speech input levels to a codec X Transmission channel errors Packet loss and packet loss concealment Bit rates if a codec has more than one bit rate mode Transcodings Acoustic noise in sending environment Effect of varying delay in listening only tests Short term time warping of audio signal Long term time warping of audio signal Listening levels between 53 and 78 dB A SPL in superwideband mode Packet loss and packet loss concealment with PCM type codecs rene i E e e Le O Temporal and amplitude clipping of speech Rohde amp Schwarz 1GA62 6 POLQA and PESQ Measurements Comparison PESQ POLQA Test Factors Test Factor PESQ POLQA Linear distortions including bandwidth limitations and spectral shaping non flat frequency responses X Frequency response X Effect of delay in conversational tests a Talker echo o Sidetone o Acoustic noise in receiving environment The last four test factors are addressed
15. Test Automation Tool for POLQA and PESQ Speech Quality Tests Application Note Products R amp S UPV R amp S UPV K63 R amp S UPV66 R amp S UPL Z1 R amp S UPV K1 R amp S CMU200 R amp S UPV K61 R amp S CMW500 This application note provides an application program for level alignment test automation and reporting with PESQ POLQA and delay measurements on the Audio Analyzer R amp S UPV or R amp S UPVEE An introductory section explains theoretical background gives application hints and defines requirements for test signals Application Note 06 2013 2e Thomas Lechner 1GPP Table of Contents Table of Contents 1 POLQA and PESQ Measurements ccscssssesesesseseeseeees 4 1 1 Objective Perceptual Speech Test Methods ccccccccssseeeeeeeeeeeeeeenees 4 1 2 Bandwidth and Listening Situation cccccsseeeseeeeeeeeeeeeeeeeeeeeseeneeeneeees 4 1 3 Reference SINAIS ates rec ett cee este sateen ee ene emanate 5 1 4 POLQA and PESQ on the R amp S Audio Analyzer UPV csseecce 6 1 5 POLOGA vs PESO mermon E 6 1 6 Ce VS EE cen ete E E E A A E E E 7 2 Using the Application Program cccecesseesseeeseeeeeeeeeseeenees 9 2 1 HVS CAN AU O a P E T 9 2 2 Userin er CC a A 9 2 3 Choosing a signal Path a sansnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnn nnmnnn nnne 10 2 3 1 BaT T a A ape ne E A E E A E E E 11 2 3 2 OWN ai La A A E A E E E
16. andwidth limited to 50 Hz 3 8 kHz and sampled at a rate of 8 KHz For PESQ narrowband tests according to P 862 and P 862 1 the file must have a sample rate of 8 or 16 kHz For PESQ wideband according to P 826 2 the file must have a sample rate of 16 kHz The specified wave file should contain speech signal with approximately 8 10 seconds length The reference signals should have a leading and a trailing silence at least as long as the expected end to end delay in the transmission path under test This allows Rohde amp Schwarz 1GA62 5 POLQA and PESQ Measurements temporal alignment of the degraded signal with the reference signal without loosing portions of the speech Codecs and other signal processing algorithms as well as the POLQA and PESQ algorithms may react differently to different voices Therefore it is recommended to conduct tests with two female and two male voices and average the results Speech samples English female 1 and English male 2 of ITU T recommendation B ITU T P 501 fulfill the requirements for POLQA reference files They should additionally be filtered to the SWB or NB frequency range respectively These filtered files are included with the installer of this application note 1 4 POLQA and PESQ on the R amp S Audio Analyzer UPV The PESQ algorithm according to ITU T P 862 is available on the R amp S Audio analyzer UPV called herein below UPV as keycode option UPV K61 P
17. art of the degraded signal To avoid this effect switch the Play mode to Single Rohde amp Schwarz 1GA62 20 Using the Application Program 2 5 2 Choosing an equalizer filter For uplink and for loopback on the network side a filter can be loaded to the generator for approximate equalization of a speaker response e g in case an artificial mouth is used to send the signal to the microphone of the mobile under test Note that the mobile input Speech calibration has to be re run when the downlink setting has been changed For downlink and for loopback on the network side another filter can be loaded to the analyzer for DRP ERP correction of an artificial ear or for diffuse field or free field equalization of a head and torso simulator HATS Click the associated Browse button to specify the desired filter file which will automatically be loaded after a setup has been loaded In folder D UPV 1GA62 some equalization filters are provided for the HATS 4128C npz provides an equalization for the average mouth frequency response of HATS type 4128C 33 DIFF npz and 33 ERP npz provide the standard diffuse field equalization and DRP to ERP correction for the type 3 3 artificial ear Note that the mobile output calibration has to be re run when the downlink setting has been changed 2 5 3 Options to be Carried out after Loading the Setup The application program can be configured to minimize the windo
18. button 2 8 2 Entering Auxiliary Data In the Report group box data to be included into the printed report can be administered Report Details W Commt Operator Edit W DUT Info Edit W Preview Rep Sel Rep All Figure 31 Report group box in the results tab The Details checkbox decides whether tables with the detailed results are included into the printed report In this case the statistic results will be separated row by row each with the associated detailed results table below If the Commt Checkbox is activated a comment will be included into the report An entry window will pop up when the report is started The Operator checkbox specifies whether an operator s name is to be included into the printed report For entering a name click the Edit button next to the checkbox The DUT checkbox allows to add information about the device under test For entering name type manufacturer and serial number click the Edit button next to it Rohde amp Schwarz 1GA62 28 Using the Application Program 2 8 3 Previewing a Report For previewing a report in the Windows print preview activate the Preview checkbox before clicking Rep Sel Or Rep All The latter button starts preview of all results in the data grid The Rep Sel Button starts exclusive preview of results checked in the column of the table Finally the Windows print preview opens with a p
19. button Rohde amp Schwarz 1GA62 21 Using the Application Program 2 6 Using the Monitor function The Monitor tab can be used to listen to the speech signal in the current connection using the built in loudspeaker of the UPV or a headset connected to the headset jack on the front panel Before the monitor is switched on an appropriate setup with appropriate signal file has to be loaded using one of the first four tabs select the signal source Source Choose output s Generator C Analyzer Switch monitor PUDUR on and off V Use Speaker l Use Headphones M Use ANLG AUX OUT Stat The ronto goes C On fe Off Keep Monitor on when Leaving this Tab automatically off when another tab Volume is chosen In the yertize headphones the volume will be initially set to 20 dB tab control Use this checkbox to keep the monitor 120 110 100 90 80 FO 60 50 40 30 20 0 on after changing the tab Change the volume Figure 13 Controls in the Monitor tab When the headphone output is enabled and the volume setting is higher than 20 dB the volume is automatically reduced to 20 dB to protect the listener s ears The volume can subsequently be re adjusted also to higher values When another tab in the tab control is selected the monitor output is disabled unless the Keep Monitor on when Leaving this Tab checkbox is checked In the latter case the speaker remains active even if a new se
20. by neither of both methods Comparison PESQ POLQA Applications Application PESQ Codec evaluation Terminal testing influence of the acoustical path and the transducer in sending and receiving direction NOTE Acoustical path in receiving direction only for SWB mode Bandwidth extensions Live network testing using digital or analogue connection to the network X POLQA gt lt Testing of emulated and prototype networks X UMTS CDMA GSM TETRA WB DECT VoIP POTS PSTN Video Telephony Bluetooth X Voice Activity Detection VAD Automatic Gain Control AGC Voice Enhancement Devices VED Noise Reduction NR Discontinuous Transmission DTX Comfort Noise Insertion Non intrusive measurements Two way communications performance The last two applications are supported by neither of both methods 1 6 Signal Levels Due to an inherent level alignment the PESQ algorithm itself is rather insensitive to absolute levels Despite this fact the signal level on the transmission path under test may heavily influence the PESQ result Overdriving the transmissions path causes degradation by clipping A low level on the transmission path may cause degradation by noise In particular in cases where speech codecs noise suppression and other non linear signal processing is involved a reproducible signal level on the transmission path may be prerequisite for reproducible PESQ results Rohde amp Schwar
21. e selectors allow to choose appropriate location and file names for both result values The working directory D UPV 1GA62 should be the preferred location for these files The encoder calibration value is alternatively obtained with the In Out Calibration routines of downlink or loopback on the mobile side The In Out calibration requires an end to end connection of the type chosen with the respective tab In a circuit switched connection a new encoder calibration is only necessary if the CMU or the link handler in the CMU is changed In a VoLTE connection with external media server the encoder calibration has to be re run if the external soundcard is exchanged or if any hardware or software control settings for the sound car are changed 2 4 3 Mobile Input Calibration This calibration value is required for uplink measurements and for measurements with loopback on the network side A valid decoder calibration value is prerequisite for the calibration routine Prerequisite for the mobile input calibration is an active end to end speech connection For mobile input calibration the application program adjusts the voltage at UPV generator output 1 to the mobile under test such that a certain decoder output voltage is measured at UPV analyzer input 2 The CMU must be set to Handset Low in GSM Traffic Mode on the Network tab to Speechcodec Low in WCDMA Voice settings gt Voice Source in the BS Signa
22. file and the relative level can be set Row number in the list sionat a I U wav file name itn Length of the wav file anme et02_s003_SWeB Signal level peak FS O i of wav file relative to Relative pieri 1 000 peak FS of path input Folder path D NITU T_new_speech SWB Folder path of signal file J Browse signal file Browse Fie Cancel m h Figure 17 Entry window for editing test signals or adding test signals to the list To add a signal to the list click the Add button in the Test Control tab click the Browse File button in the signal window and browse to the desired speech signal file The relative signal level is preset with the value from the first four tabs of the main window It can be adjusted individually for each signal Add Edit Remy Up Down Figure 18 Buttons for modifying the signal list With the buttons Edit Remv Up and Down entries in the signal list and their order in the list can be modified A subset of the signal files in the list can be selected for use with the checkboxes in the Sel Column Test signal lists can be saved loaded and appended using the respective entries in the context menu of the test signal list 2 7 3 Test and Averaging Options Test Repetitians Number of repetitions for each signal F Ignore it highest and it lowest scores Figure 19 Test repetitions group box in the test con
23. for Start automated delay automated delay Test Automation tests testing DAUPYACurves E Store Curv PUP Urs i Browse Run Delay Tests f i po AELA Choose signal path s Set folder for storing ann Senn a a eee ate a ne se for delay tests the cross correlation curves Figure 34 Controls of the delay measurement tab Delay Measurements are performed on a pseudo random multisine signal by cross correlation The test signal further comprises a leading voiced part to activate the speech coder The length of the pseudorandom part determines the maximum measureable end to end delay With a 32k samples pseudorandom noise a maximum delay of 0 6 seconds can be measured With the 46k samples pseudorandom noise the maximum delay which can be measured is 1 2 seconds The delay measurement works with an end to end delay gt 10 ms To speed up the cross correlation calculation the number of samples can be reduced by the speed factor by undersampling the test signal before cross correlation This of course is a tradeoff against time resolution and noise immunity For the two available test signals separate speed factors are stored by the program In the Subtract Delay groupbox it is possible to specify for each of the four signal paths delay values of the system simulator to be subtracted from the end to end delay Note that any processing delay occurring in the UPV itself is inherently subtracted from the results before they are d
24. gure 3 Circuit switched uplink signal path For circuit switched speech connect e UPV generator output 1 to mobile input headset or handsfree car kit connector e CMU speech output to UPV analyzer input 2 using link handler cable 1117 4082 00 Use adapter 1117 4124 00 for link handler 2 if required Establish call from the CMU to the mobile For the PESQ POLQA and delay measurements set the CMU speech codec to Handset Low For the calibration functions follow the instructions on the screen LAN peria server USB Figure 4 VoLTE uplink signal path For voice over LTE connect e UPV generator output 1 to mobile input headset or handsfree carkit connector e Output 1 of the USB sound card to UPV analyzer input 2 For the PESQ POLQA and delay measurements establish a VoLTE call between media server and mobile For the calibration functions follow the instructions on the screen Rohde amp Schwarz 1GA62 11 2e Using the Application Program Downlink External Audio Out Headset or Handsfree Car Kit RF Connection Figure 5 Circuit switched downlink signal path For circuit switched speech connect e UPV generator output 2 to CMU speech input using link handler cable 1117 4082 00 Use adapter 1117 4124 00 for link handler 2 if required e mobile output headset or handsfree car kit connector to UPV analyzer input 1 Establish call from the CMU to the mobile For the PESQ POLQA
25. gured to output a full scale 1 kHz sinewave signal Ina VoLTE setup with external media server auxiliary program CsSinWav exe must be started while the media server software is idle or closed When the Decoder Cal button in the application program is clicked the measurement routine starts In case of an instable measurement result a warning message is output and the routine is aborted without storing the result If a stable result is obtained a file selector allows to choose an appropriate location and file name The working directory D UPV 1GA62 should be the preferred location for these files The decoder calibration value is alternatively obtained with the In Out Calibration routines of uplink or loopback on the mobile side The In Out calibration requires an end to end connection of the type chosen with the respective tab In a circuit switched connection a new decoder calibration is only necessary if the CMU or the link handler in the CMU is changed In a VoLTE connection with external media server the decoder calibration has to be re run if the external soundcard is exchanged or if any hardware or software control settings for the sound car are changed Rohde amp Schwarz 1GA62 16 Using the Application Program 2 4 2 Encoder Calibration This calibration value is required for downlink measurements and for measurements with loopback on the mobile side A valid decoder calibration value is prerequisite for
26. he next section of the user interface provides a row for each required calibration value The value if available is shown on the respective right end The buttons on the left end start the appropriate calibration routines In the center a textbox shows the name of the file which stores the current calibration value Stored values can be loaded using the Browse buttons Another section follows which allows to specify a wave file to be used as test signal file and the full scale value of the wave file relative to the determined full scale value of the signal path input The last section in each tab handles the setup to be used for the test The text box in the center shows the file name With the Load Setup button the setup is loaded to the UPV the generator reference value signal file name and voltage value are entered and the analyzer range is set appropriately To load a setup from a file different from the one shown in the text box use the Browse and Load Setup button For the uplink direction it is possible to load a filter file for soeaker equalization In the downlink tab a filter file for ear correction can be loaded The checkboxes below the tabs allow to automatically minimize the window of the application program after the setup has been loaded and or to start a single measurement immediately afterwards Two more checkboxes allow to configure to change to the test control tab after a setup has been loaded and to cha
27. hwarz 1GA62 3 POLQA and PESQ Measurements 1 POLQA and PESQ Measurements 1 1 Objective Perceptual Speech Test Methods POLQA and PESQ are objective perceptual speech test methods A speech test method rates the quality of a speech signal as opposed to a conversational rating where impairments like round trip delay are judged for their impact on a conversation A perceptual test method delivers results which assess degradations the same way as human beings would do Non perceptual test methods usually determine physical entities like voltage level A possible implementation of a perceptual test is a subjective listening test e g according to ITU T P 800 In a subjective test speech signals are presented to subjects under standardized listening conditions level playback equipment with a well defined judgement task or question Such a task could be to categorize the quality of a presented speech signal according to the question how good does the voice sample sound into one of five steps between 5 Excellent and 1 Bad The answers from many subjects are averaged to obtain a so called MOS Mean Opinion Score An objective perceptual speech test method implements a complex algorithm which predicts results expected from subjective tests by applying psychoacoustic and perceptive models to the test signals The algorithm is verified by comparing the results to MOS scores obtained in subjective listening
28. ion from ITU T P 863 POLQA superwideband mode can be applied to narrowband wideband and Superwideband signals The MOS scores will also reflect the degradation caused by the respective bandwidth limitation POLQA narrowband mode is provided for compatibility with assessment of narrowband telephony and backward compatibility with P 862 1 PESQ with mapping of PESQ scores to a MOS scale In all other application cases the superwideband mode of POLQA should be used 1 3 Reference Signals Clean speech reference signals should preferably have the following properties at least three seconds of active speech at least one second of silence between active speech periods no more than 6 seconds of active speech total length of test sample including silence should be no more than 12 seconds active speech level of 26 dBov 16 bit linear PCM encoded noise floor lt 80dBov A speech pauses should not be digital zero but contain noise signal lt 80dBov A The level in the speech reference files is specified in dBov dB overload which is the level referenced to a full scale square wave signal A full scale sinewave signal has arms level of 3 dBov Usually one reference signal file contains a pair of sentences separated by a short silence Reference signals for POLQA superwideband should be sampled at a rate of 48 kHz and bandwidth limited to 50 Hz 14 kHz Reference signals for POLQA narrowband should be b
29. isplayed The buttons in the Single Measurement groupbox start delay measurements for each of the four signal paths and display the result in the textbox below The S N factor displayed is the amplitude ratio between the highest cross correlation peak and the second highest cross correlation peak If this ratio is less than 2 the result is deemed to be invalid In this case you can reduce the speed factor to improve the noise immunity of the delay measurement In the Test Automation groupbox it is possible to start multiple delay measurements for each of the selected signal paths and do a statistical evaluation as specified in the Test Repetitions groupbox It is possible to store the cross correlation result as curve in an ASCII file to a specified folder Rohde amp Schwarz 1GA62 30 2e Using the Application Program 2 10 Delay Results Reporting and handling of delay results is similar to that of the speech quality test results Please refer to sections 2 8 1 to 2 8 5 Open link to detailed results Context menu of delay results table Clean up results Export all results to ASCII file Start report generation Report settings 4 f gt Pol QA and Delay Test Tool i Loop on Network Side Loop on Mobile Side Monitor Test Control Results Delay Meas Delay Results alel Results Nol S Direction Signal Sp Subtr Sy First Test Last Test Avg Dela Max Del Min Dela Max S
30. k Uplink Uplink ef02_s001_SWB ef02_s001_SWB eff s001_SWB eff s001_ SWB eff s001_ SWB eff s001_SWB eff s001_SWB Nal MOS Gain Length Ret L Deg L Att d Delay S N A M 3 84 3 80 3 82 3 77 3 90 3 76 3 81 Figure 29 Detailed results table in the results data grid 10 04 10 04 10 04 10 04 10 04 10 04 10 04 8 000 8 000 8 000 8 000 8 000 8 000 8 000 30 1 30 1 30 1 30 1 30 1 30 1 30 1 26 0 25 5 25 9 25 8 26 0 25 9 26 0 4 586 4 56 4 56 4 55 4 55 4 59 4 56 147 110 147 146 14 123 146 39 26 39 26 39 28 39 28 39 28 39 28 39 28 The white back arrow in the caption is the link for closing the detailed table and returning to the statistic overview table In the details view the entry Play Degraded Signal of Marked Result is enabled in the context menu of the results data grid if the degraded signal is available in a wav file When this functionality is chosen a window for controlling the playback of the degraded signal opens Rohde amp Schwarz 1GA62 27 2e Using the Application Program Speaker Headset Yolume 100 50 0 W Speaker Enable I Phone Enable Play again Close ae a Figure 30 Control window for playback of degraded signals The window allows to choose the output and set the volume The file is played once Then the playback can be restarted using the Play again
31. l tab and to 8k Enhanced Low in CDMA2000 Primary Service Class gt Speech Service gt Service Option 3 gt Voice Coder in the Service Cfg Tab respectively This operation mode requires an active call to the mobile under test In other setups the analog output on the network side must be configured to provide a signal connection from the mobile under test to the network output Rohde amp Schwarz 1GA62 17 2e 2 4 3 1 2 4 3 2 Using the Application Program The RMS level sent from UPV generator output 1 to the mobile input is adjusted such that the CMU decoder output connected to UPV analyzer input 2 corresponds to the dBm0 level selected in the Calibration Level field The standard speech level is 16 dBm0 However some mobiles with low sending sensitivity or with dynamic compression algorithms do not allow to reach this level In this case the calibration level of 26 dBm0 can be used This level is expected to be within the linear operating range of the uplink path The mobile input calibration has to be done for each individual mobile if codec settings are changed or if settings in a mobile affecting the uplink signal path are changed Mobile Input Cal When the Mobile Input Cal button in the application program is clicked the measurement routine for mobile input level calibration with a sine burst signal starts If the decoder output voltage corresponding to the selected calibration level
32. n Rohde amp Schwarz 1GA62 14 2e Using the Application Program 2 3 5 Connection Cable between UPV and CMU Speech Connector A cable with male analyzer and female generator XLR connector is required for connection of the UPV to the Speech connector of the Digital Radio Communication Tester CMU The schematic is as shown below UPL Analyzer Generator Figure 11 Cable for CMU200 speech connector Instead of XLR connectors BNC connectors can be used in connection with UPL Z1 XLR to BNC adapters The cable as shown is configured for connection to link handler 1 in the CMU Depending on the CMU hardware link handler 2 in the CMU can also be used for GSM in this case pin 5 of the DB9 connector has to be connected to the UPV generator instead of pin 2 and pin 6 of the dB connector has to be connected to the UPV analyzer instead of pin 3 The described cable part no 1117 4082 00 and an adapter for link handler 2 part no 1117 4124 00 is enclosed in option UPV K9 Base SW for mobile phone tests 2 3 6 Connection Cable between UPV and UE Headset Connector Cables for connection between UPV and the headset connector of the UE under test are available as option UP Z9 part no 1411 3106 02 The accessory set comprises two cable assemblies one with ground on the sleeve of the 4 pole 3 5 mm jack plug and one with ground on the second ring of the 4 pole 3 5 mm jack plug Rohde amp Schwarz 1GA62 15 Using the Ap
33. n Program 2 5 Loading a Setup and Starting a Measurement in Manual Mode 2 5 1 Configuring the Test Signal The application program allows to specify an appropriate wave wav file as test signal for the PESQ or POLQA test For requirements on the test signals see section 1 3 The test signal file can be selected after clicking the browse button in the respective line The name of the selected file is entered in the text box in the center of the line If no file name is specified empty string in the text box no wave file is loaded to the UPV setup which means that the wave file specified in the loaded setup will be used The textbox can be cleared using the Clear button on the right end of the line The full scale level of the test signal relative to the full scale level at the input of the transmission path under test is specified in the numeric up down control in the next line of the user interface If the full scale value is not reached in the wave file the specified full scale value will not be reached at the UPV generator output The signal from the wave file can be maximized in the UPV generator after loading by activating the checkbox Scale Pk to FS As different files usually have different crest factor Scale Pk to FS does not mean that the rms level or active speech level at the generator output is normalized The level in the speech reference files is specified in dBov dB overload which i
34. nCtion ccccceseceeeeeeeeeeeeseeeeeeeeneeseessaeeeeeeesseneesaees 22 Rohde amp Schwarz 1GA62 2 Table of Contents 2 7 Running POLQA or PESQ Tests Automatically cccscsesecssseseseeee 22 2 7 1 User Interface of the Test Automation Tab cccssssseesseeeeeeeeesseeneneees 22 2 1 2 Compiling a List of Test Signals ccc cccseceeeeeeeeeeeeeeseseseeeeeeenseeneeseeees 23 2 7 3 Test and Averaging Options cccccceeecseeeeeeeeeseneeeeeeeeeneeeneeseneeeneeseaneeees 24 2 1 4 Starting a Test Sequence weniesseecateneeseccaetnesaventunwantententesesnezaccnersecenneevsacusuene 25 2 8 POLQA and PESQ Test Results s sssssssssssssssesssssssesssseesssessesseseeseeaee 26 2 8 1 Viewing Results in the Data Grid ec ccecesseeeeeeeeeeeeeeeeeeeeeeeneeeeneeeeees 27 2 8 2 Entering Auxiliary Dale viccicscecectecrrveddetsvcancatsawtdateeddesdbeteeesinelevedavemaccsiviecinn 28 2 8 3 PIEVICWING a REDOT reaa aaa a aa Hiudvad a aa aaa 29 2 8 4 Printing a Repor stinsccvesaseacescutusenseosvaorecssiconseesersawuniconctaerierisenecateicsinteneds 29 2 8 5 Export of Result Data to an ASCII file 00nannannnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnne 29 2 9 Delay Measurements nusnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnn nnmnnn nnn 30 2 10 Delay RESUS iii a a aaa 31 3 Literatu e aaa a a 32 4 OFGEFING INTOFMAtION sewseewscsisiveseusiinnstwnavenenweeeredinniunedtuneennase 33 Rohde amp Sc
35. nge to the results tab as soon as a test sequence starts The user interface of the last six tabs is described with monitor test automation and reporting functionality further below The program memorizes all settings and calibration values in a special file from where they are loaded when the program starts The button Clear all Values can be used to clear the settings and calibration values in all tabs 2 3 Choosing a signal Path Besides the traditional transmission directions uplink also called sending seen from the mobile phone s point of view and downlink also called receiving the application program also handles setups with loopback of the signal The loopback may be established on the network side CMU or on the mobile side In the following section electrical connections and settings are described for use of the CMU as network interface Each transmission direction requires an RF connection between CMU and mobile phone In other situations other equipment may be connected in a similar matter Particularly for POLQA in superwideband mode the electric connections may be replaced by a head and torso simulator in case the acoustic performance of the UE should be part of the path to be evaluated or if no electric connection is possible Rohde amp Schwarz 1GA62 10 2e Using the Application Program 2 3 1 Uplink External Audio In Headset or Handsfree Car Kit RF Connection Fi
36. nk efo2_s002_SWB 8 000 3 35 3 75 2 61 147 148 145 detailed va JE V POLQA SWB Uplink ef02_s003_SwB 8 000 3 65 3 74 3 39 148 172 148 results Pi Bia V POLQA SWB Uplink ef02_s004_5WB 6 000 3 72 3 85 3 68 148 i43 147 Fi Es F POLQA SWB Uplink ef03_s001_SWB 8 000 3 49 3 61 344 147 147 147 Link to detailed Bis V POLQA SWB Uplink ef03_s002 Swe 8 000 3 58 3 65 2 97 147 147 146 results E7 V POLQA SWB Uplink ef03_s003_5WB 6 000 3 46 3 66 3 23 147 i47 147 Bis M porga swe uplink ef03_s004_SWB 6 000 3 54 3 62 3 52 147 i47 146 Clean up Ho POLQA SWB Uplink em01_s001_SWB 8 000 4 01 4 04 3 93 146 147 146 results K G POLOA SWB Uplink em0l su07 SWB 8 000 3 99 4 415 342 147 1468 14 7 Bjiil POLQA SWB Uplink emO1_s003_SWB 8 000 3 91 3 99 3 85 147 i47 89 Export 12 V POLQA SWB Uplink emO1_s004_SWB 8 000 3 88 4 00 3 77 146 147 114 all results to THis POLQA SWB Uplink em02_s001_SWB 8 000 4 06 4 18 3 34 153 175 147 FOLQA SWE Uplink emuz su0 SWB 8 000 3 94 4 07 3 76 147 148 105 ASCII file B15 M po QA SWB Uplink em02_s003_SWB 6 000 4 06 4 15 4 00 147 148 147 Biicl POLOA Swe Uolink emO02 s004 SWB 6 000 3 87 4 01 3 72 149 169 147 5 Report Edit OUTInfo edit F Preview Rep all s i n n Start report B Minimize After Loading rel gah Neral Close generation i alat Maxniramant After Loading Sti ea a rt Measurement After Loading Setup F Change to Results for Test Figure 25 Results tab C Select All C Deselect All C Export Resul
37. on Mobile Side Figure 24 Direction group box in the test control tab Before starting a test sequence the desired test direction s should be selected in the group box below the start buttons Thus it is possible to run e g uplink and downlink tests in one sequence Rohde amp Schwarz 1GA62 25 2e Using the Application Program If no direction is selected before the start of a test sequence it is assumed that all tests are to be run with a setup that has been loaded to the UPV firmware either manually or using the functionality of the first four tabs in this program A confirmation for this is prompted before the sequence starts For each selected direction the setup specified in the respective tab of the user interface is automatically loaded However test signals and levels are set according to the test signal list in the Test Control tab 2 8 POLQA and PESQ Test Results The last tab in the user interface is dedicated to viewing reporting exporting and managing results gt POLQA Test Tool Overview over Uplink Downlink Loopback on Network Side Loopback on Mobile Side Test Control Results results 7 Results averaged per No S Measure BW Direction Signal Length MOS max min M Delay ms max D min amp signal GAF PoLoA SWB Uplink ef02_s001_5WB 6 000 3 81 3 90 3 76 142 148 110 l Open link to T2 F POLQA swe Upli
38. or handsfree carkit connector to UPV analyzer input 1 For the PESQ POLQA and delay measurements establish a VoLTE call in loopback mode Make sure that the sidetone on the mobile is disabled For the calibration functions follow the instructions on the screen Rohde amp Schwarz 1GA62 13 2e Using the Application Program 2 3 4 Loopback on the Mobile Side RF Connection Figure 9 Signal path for loopback on the mobile side in a circuit switched connection For circuit switched speech connect e UPV generator output 2 to CMU speech input using link handler cable 1117 4082 00 e CMU speech output to UPV analyzer input 2 using link handler cable 1117 4082 00 Use adapter 1117 4124 00 for link handler 2 if required Establish call from the CMU to the mobile For the PESQ POLQA and delay measurements set the speech coder to Codec Cal For the calibration functions follow the instructions on the screen WE AN Switch LAN USB ma dia Saret Figure 10 Signal path for loopback on the mobile side in VoLTE For voice over LTE connect e UPV generator output 2 to input 1 and 2 of the USB sound card e Output of the USB sound card to UPV analyzer input 2 For the PESQ POLQA and delay measurements establish a VoLTE call between media server and mobile and establish a loopback in the mobile For the calibration functions follow the instructions on the scree
39. plication Program 2 4 Calibration Functions start calibration file name load calibration calibration calibration value from file value DAURA LGA AA CMU Dec cel rose 10 58 mi oN Fae Decoder Cal DUP WIGd ee Mob_In ecl Calibration Level nfout Speech Cal InfOut Cal mobile input calibration overall calibration function target speech level to be using speech signal based on in out curve adjusted on network side Figure 12 Calibration functions for uplink signal path For each signal path the appropriate calibration functions are displayed 2 4 1 Decoder Calibration This calibration value is required for uplink measurements for the encoder calibration and mobile input calibration routines and for measurements with loopback on the mobile side For decoder calibration the application program measures the peak full scale output of the speech decoder circuit on the network side CMU which is connected to UPV analyzer input 2 For a circuit switched connection the CMU must be set to Decoder Cal in GSM Traffic Mode on the Network tab in WCDMA Voice Settings gt Voice Source in the BS Signal tab in CDMA2000 Primary Service Class gt Speech Service gt Service Option 3 gt Voice Coder in the Service Cfg Tab This operation mode requires an active call to the mobile under test In other setups the analog output on the network side must be confi
40. review of the report as configured E Print preview Start printout on lll ial installed printer Setzoomof preview Ae Choose A PA multi page display Tra 1a mir Igra La oe la ie A Fa Pe Fa ia ia im ap ap y Close preview Select Pal displayed page r MIHINI ii i i HHY Report preview mH Figure 32 Report preview 2 8 4 Printing a Report A report can be printed on any printer installed in the Windows operating system including XPS document writer or pdf distiller The printing can either be started from the preview window by clicking the printer icon top left or by clicking one of the Re Sel or Rep All buttons without the Preview checkbox activated The printer can only be selected if the latter way of starting the printout is chosen 2 8 5 Export of Result Data to an ASCII file The Export results button saves all available results to a tab separated txt file or a comma separated csv file which can easily be imported to a spreadsheet A B D E F G H J K L M N 1 Statistics 2 No Measurem Bandwidth Direction Signal Length A MOS Max MOS Min MOS Avg delay Max dela Min delay Identifier eB POLOA SWE Mobile Loc e02 s001 6500206 3 906971 3 953615 3 657636 445 5949 464 6147 424 2725 0144 2013 09 40 45 d 2POLQ4 SWB Mobile Loc e2 _ s002 5 5 3 945699 40004 3 661595 429 6406 450 0769 410 1642 0144 2015 09 43 56 5 3 POLO4A SWE Mobile Locefl2_ s003
41. s the level referenced to a full scale square wave signal A full scale sinewave signal has a rms level of 3 dBov The level at the speech encoder is either specified in dBFS referenced to the rms level of a full scale sinewave or in digital dBm0O with 0 dBm0 3 14 dBFS It is recommended to use speech files which have an internal active speech level of 26dBov 23 dBFS To reach the nominal level of 16 dBm0 19 14 dBFS in the encoder the full scale level of these test signals relative to full scale of the encoder input or mobile input respectively should be set to 4 dB with Scale Peak to FS deactivated If the test signal has a high crest factor a full scale generator output voltage higher than the full scale input voltage of the transmission path under test i e positive level difference in the numeric up down control may be required for optimal PESQ or POLQA result In this case the positive effect of the higher average level has more influence than the potentially detrimental effect of temporal clipping at the signal peaks In the Play group box the play mode for the test signal can be switched between Single and Continuous Continuous playback of the test signal keeps the speech path in an activated state When a test is started the UPV generator is reset and starts with the beginning of the file When a significant speech delay is present the end of the truncated speech can appear at the st
42. s have to be applied which can be automatically loaded as an analyzer filter by the application program Filters for DRP ERP correction free field correction and diffuse field correction of the type 3 3 artificial ear of the head and torso simulator are included with the files provided Rohde amp Schwarz 1GA62 8 Using the Application Program 2 Using the Application Program 2 1 Installation Copy the installer 1GA62 msi to the UPV hard disk e g to D R amp S Software and start the installer Follow the instructions on the screen The installer creates the program directory C Program Files Rohde amp Schwarz 1 GA62 with all required files Furthermore the installation creates shortcuts on the desktop and in subfolder R amp S UPV Applications in the program menu Upon the first start of the program a subdirectory 1GA62 will be created in D UPV which contains all related user data like UPV setups program settings and files with calibration data 2 2 User Interface ores and Delay Test Tool ah signal Downlink Loop on Network Side Loop on Mobile Side Monitor Test Control Results Delay Me 4 help on Show Schematic Show Instructions 1 f cs CMU VoLTE w ext MS M connections Decoder Cal DNUPVIGASZ CMU_Desod Browse 1034 58 mV calibration Mobile Input Cal D UPV 1GA62 Mob_In cel Browse 90 38 mw functions Nobio trout cal T e Calibration Level M I Speech Cal 1
43. trol tab Each signal can be applied multiple times as specified in the Test Repetitions group box Erroneously extreme results can be excluded from averaging However all results will appear in the detailed results list and taken into account for the minimum and maximum result values in the overview Normalization POLO4 only PESO is always normalized Normalize degraded signal once per test signal for each test run f use Auto Gain Figure 20 POLQA normalization group box in the test control tab Rohde amp Schwarz 1GA62 24 2e Using the Application Program As POLQA as standardized in ITU T P 863 does not apply automatic level normalization of the degraded signal additional normalization should be applied unless a meaningful Gain value calculated from all involved external gains is available and can be entered in the Analyzer function panel Normalization can be done once per test signal keeping the Gain value determined in the first test run for the subsequent tests with the same signal Another option is to repeat the normalization for each test run which may increase test time From UPV firmware 3 3 1 analyzer function POLQA offers an Auto Gain feature which does the level normalization for each test run in the analyzer This way a repetition of the test with changed gain value is not necessary Test Signal MOS Weighting f Equal Acc to length Figure 21 Weighting group box in the test con
44. trol tab The results from all test runs on the same signal are averaged with the possible exclusions specified in the Test Repetitions field For each sequence run a total average is calculated from the averaged per test signal results This averaging over all applied test signals can be done with equal weight or weighted according to the respective length of the test signal files These options can be chosen in the Test Signal MOS Weighting group box Store Degraded Signals Store to DUP Degraded Browse Figure 22 Folder for degraded signals in the test control tab Degraded test signals can be collected in a specified folder for subsequent analysis of degradation The folder is specified in the Store Degraded Signals group box The wav file names are automatically generated from the test file name and a number extension Each degraded file name will appear in the detailed results list to unambiguously assign the degraded signal file to each result Minimum degraded level W Abort if deg level EE 2 dB Figure 23 Minimum degraded level groupbox in the test control tab The Minimum Degraded Level group box allows to set a threshold for the degraded level If the associated checkbox is activated a running test sequence is aborted when this limit is violated This allows to automatically react to a lost speech connection 2 7 4 Starting a Test Sequence W Uplink I Downlink Loopback on Network Side Loopback
45. ts Cc Report Selected Results co Report All Results co Clear Marked Result a Clear Selected Results Clear All Results i Save Results Load Results g Play Degraded Signal of Marked Result rai ral rat rat rai mr m4 mr rm rm rm r Figure 26 Context menu of the results datagrid The context menu can be used for reporting and deleting results and for saving and loading result lists Rohde amp Schwarz 1GA62 26 2e 2 8 1 Viewing Results in the Data Grid Using the Application Program Initially the data grid shows the averages for each test signal per sequence run Results ae Nol S Measure By Direction Signal aor ior ioe ar as as or as ice lt lt 1 lt 1 lt 1 lt 1 lt 1 lt lt 1 lt 1 POLQA POLQA FOLQA POLQA POLQA POLQA POLQA POLQA POLQA mjin Poga Bia M potas Biz I polos His Poga Blia J POQA His PoLQA Bie I potas 17 V POLOA 4 SWE SWE SWE SWE SWB SWE SWE SWE SWE SWE SWE SWE SWE SWB SWB SWE SWE Uplink Uplink Uplink Uplink Uplink Uplink Uplink Uplink Uplink Uplink Uplink Uplink Uplink Uplink Uplink Uplink Uplink ef02_s001_S5WB ef02_s002_5WB eff soos SWB eff s00d_ SWB ef03_sO01_ SWB ef03_ soz SWB eff3_s003 SWB eff 3_s004_ SWB emO1_s001 SWB emO1 soo SWB emO1_s003 SWB emO1_s004 SWB emo s001_ SWB emo soo SwB emo s003 SWB emo s004 swB 16 signals average Length MOS mat
46. tup Change to Test Control After Loading Setup Close Start Measurement After Loading Setup V Change to Results for Test Figure 14 User interface of the Test Control tab 2 7 2 Compiling a List of Test Signals The data grid in the center of the tab shows the compilation of test signals with which the tests are to be run Test Signals a Sel Signal Name Length s Pk re FS d Folder Path eft s001_ SWB 6 000 0 00 DeITU T_new_ speech eft s007_ SWB 6 000 0 00 DeITU T_new_ speech efo so03_SwBe 6 000 0 00 DeITU T_new _ speech ef0 _s004 Swe 6 000 0 00 DeITU T_new_ speech ef03_s001 Swe 6 000 0 00 Ds ITU T_new_speech ef0S_ s007_ Swe 6 000 0 00 a Figure 15 List of test signals The context menu of the test signal list provides the following entries Co al ma E a I Select All Deselect All 4dd Signal Edit Marked Signal Move Marked Signal Up Move Marked Signal Down Run Test with Marked Signal Run Tests with Selected Signals Run Tests with All Signals Remove Marked Signal Remove Selected Signals Remove All Signals Save Signal List i Load Signal List Append Signal List Rohde amp Schwarz 1GA62 23 Using the Application Program Figure 16 Context menu of test signal list The list can be edited using the buttons below the left half of the grid or with the respective entries in the context menu The Add and Edit buttons open an entry window where the source
47. tup is loaded or automatic tests are started The controls in the Monitor tab are disabled while automated tests are running 2 7 Running POLQA or PESQ Tests Automatically 2 7 1 User Interface of the Test Automation Tab The Test Control tab provides all controls necessary to set up and run POLQA and PESQ test sequences Rohde amp Schwarz 1GA62 22 Using the Application Program Options for test gt POLQA and Delay Test Tool z iaj xX iteration and plink _ Downlink Loop on Network Side Loop on Mobile Side Monitor Test Control Results Delay Mez4 averaging results Play Test Repetitions I from one signal e Single Cont Repet per signal E Ignore jo highest and fo lowest annres Normalization Normalization POLQA only PESQ is always normalized Minimum degraded level ear pade e once per test signal for each test run use Auto Gain P Abort if deg level lt 93 dB with POLQA wea Signals s e Watch for dropped o AUP Degraded Browse connection a i Test Signals Weighting with oe Dur s L dB Pk to FS Folder Path averaging of 8 4 sD UPV LGaA62 different signals ie i lem2_swe DAUPY AGA62 Collection of degraded signals in wav files Test signals Edit signal list Start test sequence Specify test direction s I Uplink Minimize Afte ding Se
48. ut level logarithmic scale of the output voltage over input level exhibits an almost horizontal branch at high levels due to the saturation of the transmission path an a diagonally rising branch at medium levels Full scale input and output voltage can be determined from the intersection point of the asymptotes of both branches Caution Make sure that the input of the transmission path e g mobile headset input or loudspeaker with power amplifier is not damaged by the maximum UPV generator output voltage of 10dBV rms with sine burst and 10 V peak with speech signal Do not use this function if you are not sure that the device is robust enough If the calibration routine is successful file selectors allow to choose appropriate location and file names for both result values The working directory D UPV 1GA62 should be the preferred location for these files Also the In Out calibration is provided in two versions with pulsed sinewave and with natural speech as test signal 2 4 6 Loading Calibration Values from File The Browse button in each calibration line allows to load previously stored calibration values for each calibration type The file name of the loaded value is entered in the text box in the center of the line If the application program is ended all calibration values and file names are stored in a file and reloaded at the next start of the application program Rohde amp Schwarz 1GA62 19 Using the Applicatio
49. w of its user interface after loading a setup Minimize After Loading Setup This allows to control and watch the UPV window with the result displays If preferred a single measurement can be started immediately after loading a setup Start Measurement After Loading Setup Alternatively the tab control in the user interface can change to the Test Control tab after loading a setup 2 5 4 Loading a Setup The button Browse and Load Setup in the tab area opens a file selector which allows to specify a UPV setup for the respective signal path For each of the four types various example setups are present in subfolder D UPV 1GA62 After selecting an appropriate file the name is entered in the text box in the center of the line and the setup is loaded to the UPV Thereafter the relevant calibration value of the respective input of the transmission path is entered as generator reference voltage the specified wave file is loaded to the Play function of the generator and the specified relative peak voltage is set in dBr units The analyzer range is set to accommodate the maximum output voltage of the transmission path under test as specified in the relevant calibration value Attention After changes in the calibration values in the specification of the test signal and or in the selection of a filter to be applied the UPV setup specified in the text box has to be re loaded using the Load Setup
50. z 1GA62 7 POLQA and PESQ Measurements The POLQA algorithm in superwideband mode as defined in ITU T P 863 rates the absolute listening level For obtaining a correct rating a gain factor has to be specified calculated from several gain factors like artificial ear sensitivity and a factor depending on the analyzer input range If there is not sufficient information for calculating this gain factor or for the narrowband mode the level of the degraded signal should be normalized for the POLQA algorithm From UPV firmware 3 3 1 the analyzer function POLQA offers an Auto Gain parameter which can be activated for this purpose This application note provides an application program which runs on the Audio Analyzer R amp S UPV or R amp S UPV66 and can be used to calibrate signal levels for the PESQ and POLQA measurements For the R amp S Universal Radio Communication Tester CMU200 called herein below CMU calibration routines are provided which make use of the Decoder Cal and Encoder Cal functions Decoder Cal outputs a full scale sinewave signal with a frequency of 1 kHz Encoder Cal loops back the digital samples from the A D converter output to the D A converter input For loopback measurements and equipment other than the CMU a so called In Out Calibration is provided which determines approximate full scale input and output levels of the transmission path from the saturation
Download Pdf Manuals
Related Search