(RSVVP) User`s Manual
Contents
1. Knowles amp Gear Whole time window and User defined time window Evaluate metrics a Figure 18 Selection of the metrics profile and time interval During the evaluation of the metrics various graphs appear and disappear on the When the metrics evaluation is done the results are shown on the screen Figure 19 Table_Results NCHRP Comparison Metric values Whole time interval 0 0 539751 True and Test curves acceleration True and Test curves velocity True curve Test curve LL LLl15 True curve Te st curve ANOVA Metrics MPC Metrics Value Value 95 Average NA 0 Sprague Geers Magnitude 49 Standard deviation 0 14 Sprague Geers Phase 13 T test ede Sprague Geers Comprehensive 22 07 Acceleration Residuals Residuals time history Residuals histogram Residuals cumulative distribution 30 100 90 25 80 _ 20 J 70 60 T 2 S 3 Cc cC Z S 15 S 50 8 S a o 40 a a 1 30 Residual time history Residuals Mean 5 20 90th percentile upper boundary 10 90th percentile lower boundary 0 0 0 1 0 2 0 3 04 0 5 06 0 7 0 6 0 4 0 2 0 0 2 0 4 0 6 0 8 06 04 02 0 0 2 0 4 0 6 0 8 Residuals Cumulative Residuals Proceed to evalute metrics Figure 19 Output of results for the whole time interval 25 By clicking the Proceed to evaluate metrics button the window shown in Figure 202. GUI_06_09_2 Mano Desktopik SS VI 06 09 e C Users Mario Desktop RSSVP_GUI_08_09 2 No synchronization Ud No synchronization Minimum absolute area of residuals Least Square error Figure 4 Drop down menu of the Sync Options box Figure 5 Selection of a new starting point in case the user is not satisfied by the initial synchronization of the two curves 3 2 3 Shift Drift controls Another preprocessing option supported by RSVVP is the possibility to correct any initial shift and or drift in the curves Experimental data sometimes show shift and or drift effects due to the change of temperature immediately before or during the test The shift effect is the initial vertical shift of the curve due an increase of the temperature after the measurement gauges have been zeroed while the drift effect 1s the linear drift of experimental curve typical of the increase of the temperature during the test The shift and drift controls of RSVVP correct these effects and can be very useful for correcting these data acquisition errors Both the shift and drift controls can be activated independently from each other by checking the respective box Once one or both of them have been checked the user has the choice to apply the selected control s to the true curve the test curve or both the true and test curves Figure 6 By default these controls are inactive 10 RSVVP Software
3. CFC 1000 User defined CFC Figure 3 Filter Options box a drop down menu and b Optional user defined CFC field in case the user defined option has been selected If it is necessary to specify a value for the CFC which is not listed in the menu select the option User defined CFC at the end of the list and input the desired CFC parameters in the Optional user defined CFC field located right below Figure 3b Note this field is active only if the User defined CFC option is selected from the drop down menu 3 2 2 Curve Synchronization RSVVP allows the user to optionally synchronize the two input curves before evaluating the comparison metrics This option can be very useful if the original test and true curves have not been acquired starting at exactly the same instant e g the test and true curve represent respectively a numerical simulation and an experimental test of the same crash test but the instant at which data collection was started is not the same The synchronization of the two input curves is very important as any initial shift in the time of acquisition between the test and true curves could seriously affect the final value of the comparison metrics For example two identical input curves with an initial phase difference due to a different starting point in the acquisition process would probably lead to poor values of most of
4. column indicates the boundaries of the specific time interval Figure 14 shows a typical layout of the Excel output file Whole time interval 0 0 5474 User time interval 1 0 08005 0 19995 User time interval 2 0 12005 0 21995 MPC Metrics Value Value Geers Magnitude 7 1 4 7 Geers Phase 23 9 22 1 Geers Comprehensive 24 9 22 6 Geers CSA Magnitude N A N A Geers CSA Phase N A N A Geers CSA Comprehensive N A N A Sprague Geers Magnitude N A N A Sprague Geers Phase N A N A Sprague Geers Comprehensive N A N A Russell Magnitude 5 6 3 8 Russell Phase 22 5 21 6 Russell Comprehensive 20 5 19 4 Knowles Gear Magnitude 58 101 1 Knowles Gear Phase 1 8 0 Knowles Gear Comprehensive 53 92 3 Single Value Metrics Value Value Whang s inequality metric 38 5 36 5 Theil s inequality metric N A N A Zilliacus error metric 76 8 76 5 RSS error metric metric N A N A WlFac Error N A N A Regression Coefficient 66 7 49 9 Correlation Coefficient N A N A Correlation Coefficient NARD 76 1 77 9 ANOVA Metrics Value Value Average 0 01 0 04 Std 0 15 0 25 T test 7 21 7 39 T T c 2 81 2 88 Value 96 10 5 21 4 23 8 N A N A N A N A N A N A 7 9 21 2 20 1 1573 2 0 1436 2 Value 96 38 1 N A 85 9 N A N A 65 2 N A 78 6 Value 0 05 0 16 14 43 5 63 Figure 14 Layout of the Excel table containing the metrics results for the various time intervals A summary of the input files and preprocessing options for each channel is
5. NCHRP 22 24 Input Curves Load True Curve C UsersiMano Desktool RSSVP GUI 0 Load let Ouve C Users Mario Desktop RSSVP_GUI_ True and Test curves True and Test curves True curve only Test curve only Figure 6 Shift and Drift controls 3 3 Metrics selection and time interval Once the test and true curves have been preprocessed push the Proceed to metric evaluation button to select the desired comparison metrics In case of multichannel input this button will appear once the curves for the last channel have been preprocessed Three main metric profiles can be selected in RSV VP 1 NCHRP 22 24 2 All metrics and 3 User selected metrics For each of the three available profiles the comparison metrics can be evaluated on either the entire time window on which the true and test curves are defined or a smaller user defined local time window These features will be described in the following sections 11 3 3 1 Metrics selection The NCHRP 22 24 profile is the default profile and it is strongly suggested that this profile be used for roadside safety applications like comparing a full scale crash test to a simulation the other profiles can be used to asses other types of curve comparisons such as component level comparison of a material stress strain curve This profile evaluates the metrics suggested in the NCHRP 22 24 Project for comparison of full scale crash test results to num
6. single metric value by calculating a weighted average of the value from each channel In order to start the evaluation of the metrics press the Evaluate metrics button located at the bottom of the window Figure 10 Note that it is possible to go back to the main graphical interface to change any of the selected input curves and or modify any of the preprocessing options by clicking the Back button Metric profile NCHRP 22 24 suggested v Metrics MPC metrics r Single value metrics t hang s inequality ANO A metric 4 Ray Time window for curve comparison Whole time window and User defined time window v Figure 10 Evaluate metrics button starting the metric evaluation 4 1 Whole time window No action is needed to define the time interval in the case where the Whole time window option has been selected option 1 and 2 of time intervals as RSVVP will automatically consider the maximum time interval possible in which both the true and test curves are defined 4 2 Definition of a User defined time window If a User defined time window has been defined i e options 1 and 3 RSVVP will prompt the user to select the upper and lower boundaries of the local time interval on which the 15 comparison metrics will be evaluated It is possible to evaluate the metrics on as many user defined time windows as desired after the results
7. 0 10000000 0 09500000 Ou L200 0000 0 09400000 0 14000000 0 09300000 Figure 1 File format of the test and true curves No limitation is imposed or assumed for the units of both the abscissa and ordinate columns The user must ensure that the physical meaning and the units of the input curves are consistent To input the test and true curves click on the respective buttons located at the top of the graphical interface and Select the file containing the input data Figure 2a Once the curve has been selected a preview will be shown in the graph area of the main graphical window In the case of multichannel input clicking on the Next Ch button located at the bottom of the screen moves on to the input of the next channel The name of the actual channel appears at the top of the window In order to proceed to the next channel it is necessary to input the actual channel and perform the preprocessing It is always possible to come back to the previous input channel by clicking the Prey Ch button Once the last channel is reached e g Pitch rate it is possible to proceed to the selection of metrics by clicking the Proceed to metrics evaluation button In order to proceed to the next step it is necessary to input the true and test curves for each of the six channels mra Oce RSVVP Software NCHRP 22 24 Input Curves Load True Curve C Users Mario Desktop RSSVP_GUI_08_09_2 Load Test Curve C Use
8. 1 4 zi To install MCR perform the following steps 1 Extract the content of the RSVVP zip file in the folder on your PC where you want to install RSVVP for example CARSVVP 2 Open the folder where you extracted the files and double click on the nstaller bat file 3 Follow the instructions of the installation wizard It may take several minutes to install the free Matlab MCR environment that is used in conjunction with RSVVP 4 Reboot your PC At this point RSV VP should be installed and ready to use on your computer 2 9 Starting RSVVP After MCR and RSVVP have been installed simply double click the RSVVP exe file located in the installation folder previously created e g C RSVVP to start the program Once started a series of intuitive and user friendly graphical interfaces will guide the user through the preprocessing the evaluation of the comparison metrics and the saving of the results The following sections describe the features and use of the program 3 Data input and selection of options Each chapter of this manual first gives a brief explanation of the described feature while the text in the boxed frame indicates how to execute that specific operation The first step is the selection of the input curves to be compared and specifying the various preprocessing options At this stage the user can also decide which comparison metrics will be evaluated by RSVVP 3 1 Specifying the curves In RSVVP the tw
9. appears where it 1s possible to define the upper and lower boundaries for the time interval on which the metrics are now calculated again The interval selected for this example 1s 0 05 sec 0 15 sec 2g User Defined Time Interval oe 50 e T 30 o ESEE 4 MEE I NI qo oo cas sad Sone aioe So os Lata T T E E ces nce E E E E E E E E EE TT 4 om UE asasceascacstgewssaecessmeseedaesteesnesesessaclaastseastesss seessscdasebes E zi leg l j 0 05 Lowver limit Figure 20 Setting of the boundaries for the User defined time interval Clicking the Evaluate metrics button causes RSVVP to evaluate the previously selected metrics only on the defined time interval Figure 21 shows the metric values obtained for this time interval 26 Table Results NCHRP Comparison Metric values Time interval 1 0 05005 0 149951 True and Test curves acceleration True and Test curves velocity True curve Test curve True curve Test curve T test 6 88 ANOVA Metrics MPC Metrics Value Value Average 0 04 Sprague Geers Magnitude 3 Standard deviation 0 25 Sprague Geers Phase 4 Sprag
10. deviation 0 27 Sprague Geers Phase 1 6 10 85 Sprague Geers Comprehensive 3 5 i A 0 15 0 16 0 17 0 18 0 19 0 2 0 21 Acceleration Residuals Residuals time history Residuals histogram Residuals cumulative distribution Residual Percentage EN 0 6 0 4 0 2 0 0 2 0 4 0 6 0 8 06 04 02 0 0 2 0 4 0 6 0 8 Residuals Cumulative Residuals Evaluate on a new interval Save results and Exit Figure 23 Output of results for time interval 0 15 sec 0 20 sec 7 3 Save results and exit Assuming the analysis previously performed on the two time intervals is satisfactorily it is decided to save all the results and exit RSVVP After the Save results and Exit button is pressed RSVVP creates a folder called Results in the directory where it was installed and a subfolders for each time interval considered during the metrics analysis In this example three different subfolders are created e Whole_time_ Interval e User defined interval 1 0 05 0 15 and e User defined interval 2 0 15005 0 19995 Also the Excel file Comparison Metrics xls 1s created containing a summary of the metrics value for each interval 28 Comparison Metrics xls Microsoft Excel ox Home Insert Page Layout Formulas Data Review View co TA Ie amp Cut Calibri o A um Wrap Text General A dd Bad Ite BK Ar Fy 43 Copy x E E Bl E Merge amp Ce
11. e Graph of residual time history e Graph of the residual histogram and e Graph of the residual cumulative distribution 17 If the NCHRP 22 24 profile was selected the graph of the velocity time history is also shown Figure 12 and Figure 13 show the typical output screen for the case of the NCHRP 22 24 profile and the other two metric selection profiles 1 e All metrics or User defined profiles respectively If the NCHRP 22 24 profile has been selected a green square beside the value of each metric indicates that the acceptance criterion for that specific metric has been passed while a red square indicates the criterion has not been passed Table Results NCHRP Comparison Metric values Whole time interval 0 0 539751 True and Test curves acceleration True and Test curves velocity C True curve True curve Test curve Test curve ANOVA Metrics MPC Metrics Value Value Average D Sprague Geers Magnitude 1 9 Standard deviation 0 14 Sprague Geers Phase 13 _T test ak Sprague Geers Comprehensive 22 07 07 Acceleration Residuals Residuals time history Residuals histogram Residuals cumulative distribution Residual Percentage D Percentage 3 0 4 Residual time history Residuals Mean 5 ar 0 6 90th percentile upper boundary 90th percentile lower boundary L mM L 0 0 1 02 0 3 0 4 0 5 0 6 0 7 0 6 0 4 0 2 0 0 2 0 4 0 6 0 8 0 6 04
12. inequality Sprague amp Geers Zilliacus error Correlation Coefficient 7 Russell RSS error F Correlation Coefficient NARD Knowles amp Gear ANO A metric n Ray Time window Whole time window and User defined time window Y Evaluate metrics Bit Figure 8 Selection of the available comparison metrics User selected metrics profile 3 3 2 Time interval Metrics can be evaluated on either the whole time window 1 e the complete curve or a user defined window 1 e a portion of the curve If the Whole time window option is 13 selected the metrics are evaluated considering the whole time interval on which both input curves were defined Metrics cannot be evaluated on any time point for which just one of the two input curves was defined as only the maximum time interval common to both the curves is considered If the User defined time window option is selected the metrics are evaluated on one or more time intervals arbitrarily defined by the user Three different options are available for the time intervals 1 Whole time window and User defined time window 2 Whole time window only and 3 User defined time window only The user can select either the Whole Time window option or the User defined time window option or both of them Whole time window and User defined time window option In the
13. latter case RSVVP will proceed to evaluate the comparison metrics first on the Whole Time interval and after showing the results it will prompt the user to define an arbitrary User Defined Time interval on which to evaluate the metrics on a local base By default RSVVP evaluates the selected metrics on both the Whole time interval and the User selected time interval If only the whole time interval or one or more user defined time interval s is desired select the corresponding option in the drop down menu located in the Time window box below the list of metrics Figure 9 Metric profile NCHRP 22 24 suggested Metrics MPC metrics Single value metrics WVhang s inequalit Seers CSA Theil s inequality V Sprague amp Geers vel Russel Whole time window and User defined time window ANO A metric 7 R Whole time window and User defined time window Whale time window only User defined time window only 4 Ray Whole time window only User defined time window only HEN T Figure 9 Selection of the type of time window s 14 4 Metrics evaluation Once the input curves have been pre processed RSVVP proceeds to evaluate the metrics on the time interval selected by the user in the previous step In case of multichannel input RSVVP first calculate the value of the metrics for each channel and then combines them together into a
14. objective quantifiable comparison of the agreement between two curves The comparison metrics calculated by RSVVP can be used to validate computer simulation models using data obtained from experimental tests verify a simulation with another simulation assess the repeatability of two experimental tests or generally speaking perform a comparison of virtually any pair of curves All the comparison metrics evaluated by RSVVP are deterministic meaning they do not specifically address the probabilistic variation of either experiments or calculations i e the calculation results are the same every time given the same input For a description of each metric calculated by the RSVVP see Appendix A In order to ensure a correct comparison of the two curves RSVVP gives the user the option to perform various preprocessing tasks before the metrics are calculated The intuitive and interactive graphical interfaces of RSVVP allow the user to input the two curves to be compared and select all the possible preprocessing options Also a series of automatic warnings alert the user about possible mistakes during the preprocessing phase The interpretation of the results obtained using this program is completely the responsibility of the user The RSVVP program does not presuppose anything about the two curves it simple compares the data and calculates the metrics The user must ensure that the curves are appropriate candidates for a comparison 2 Insta
15. pair of input curves while in the Multiple Channel option the comparison metric is evaluated as a weighted average of the metric values from several pairs of curves each pair representing a different channel The metrics evaluated in the Multiple Channel case are intended to give a general assessment of the comparison using all the available channels simultaneously A typical example of application of the Multiple Channel option is when the user wants to perform a comparison between experimental and numerical results using all the different acquisition channels e g X Y Z accelerations and or roll pitch and yaw rates altogether in order to obtain a comprehensive evaluation of the match between a real test and a numerical simulation that experiences a complex motion requiring multiple data acquisition channels The weighting factors used to combine the values of the comparison metric from each channel are based on the peak of the true curve of the respective channel see Appendix B for more details The input curve files must be in ASCII format and can have any extension or no extension at all The abscissa and ordinate data of the input curves must be tabulated into two respective columns with no headings as shown in Figure Each line in the input file represents one data point e g time and the acceleration at that time 0 00000000 0 10000000 De 2D QuOSSU0000 0 04000000 0 09800000 0 06000000 0 09 790000 000000000 009600000
16. vol ITI Validation Procedure Manual Report No FHWA RD 88 213 Federal Highway Administration Virginia 1988 11 B Whang W E Gilbert and S Zilliacus Two Visually Meaningful Correlation Measures for Comparing Calculated and Measured Response Histories Carderock Division Naval Surface Warfare Center Bethesda Maryland Survivability Structures and Materials Directorate Research and Development Report CARDEROCKDIV U SSM 67 93 15 September 1993 12 H Theil Economic Forecasts and Policy North Holland Publishing Company Amsterdam 1975 13 D M Russell Error Measures for Comparing Transient Data Part II Error Measures Case study Proceedings of the 68th shock and vibration symposium pp 185 198 2006 35
17. written at the end of the Excel file 6 2 Graphs RSVVP creates several graphs during the evaluation of the metrics and saves them as bitmap image files For each time interval considered during the execution of the program the following graphs are created into the folder Results Time histories a Time histories of the true and test curves b Time histories of the metrics and c Residuals time histories histogram and cumulative distribution In the case of multichannel input the time histories of the metrics represent the weighted time histories of the metrics form each channel Similarly the residuals time history histogram 20 and distribution are plotted using the weighted average form the residual histories of each channel A specific directory is created for each time interval In order to save disk space the bitmap files are compressed in zip format 7 Example This example shows how to use RSVVP to compare the acceleration time histories obtained from two full scale crash tests involving a longitudinal rigid barrier Both tests were performed using new vehicles 1 e same model and brand and the same longitudinal barrier Although conceptually the two crash tests should be identical in practice the acceleration curves obtained from each test show some differences In fact in such a complex event as a crash test it 1s practically impossible to completely control all the parameters involved like the exact
18. 02 0 0 2 0 4 0 6 0 8 Residuals Cumulative Residuals Proceed to evalute metrics Figure 12 Screen output for a the NCHRP 22 24 profile and b the All metrics User defined profiles In the case of multichannel input a drop down menu allows the user to select which channel to show in the plot area The metric values do not change when the plotted channel changes as these values represent the weighted average of all the channels and not the value of the specific channel shown in the plot view 18 Table_Results Comparison Metric values Whole time interval 0 0 5474 Metrics values MPC Metrics Single Value Metrics Value 96 Value 96 Geers Magnitude 74 Whana s Inequality metric 38 5 Geers Phase 23 9 Theil s Inequality metric N A Geers Comprehensive 249 Zilliacus error metric 75 8 Geers CSA Magnitude NA W EGRE eror aonig E WI Fac error N A Geers CSA Phase N A Regression coefficient 66 7 Geers CSA Comprehensive ze Correlation coefficient N A Sprague Geers Magnitude N A Correlation coefficient NARD 76 1 Sprague Geers Phase N A z Sprague Geers Comprehensive N A ii Value Russel Magnitude 5 6 Average 0 01 Russel Phase 22 5 Standard deviation 0 15 Russel Comprehensive 20 5 T test 721 Knowles Gear Magnitude 58 f i Knowles Gear Phase 1 8 fug 04 02 03 04 0 5 0 6 07 Knowles Gear Comprehensive 53 Acceleration Residuals Residuals history Residuals hi
19. Roadside Safety Verification and Validation Program RSVVP User s Manual eC ET EET Lee ee ee er TTETTTE 2 2 2 2 2 2 2 2 2 2 2 2 2 Rp T 22 4 4d d AMdW dl3 wae ee ee ee ee ee gt o XAh d X EbB 4 HR JWd u4 F 1 EBNNNNN ANN UG aE wl ZZ LQ1d acc tM eee ee ee ee ee ee ee ee ee ee basi Acceleration g s 0 1 0 1 Time sec Worcester Polytechnic Institute WPI December 2008 Rev 1 4 Malcolm H Ray Mario Mongiardini Contents l 2 UJ a Introduction TOI 9 V VP aos Eae Dot adatta ata es Cono ramen cao a Con E uod JAG ES CUI Luho yo NES Serpe EE Cece LE OE 2al 35S Se MM Te utt PN UNS os ioco e vasa abr ened os ubi Do aco olt uad Scenic To DU eel etic 2 2 Installation of the MATLAB Component Runtime esses 2 2 92 VAIN RSVV P onsooedapat inte da aaa ae eaten Dnus fa Ls icm ALD e aM du ca Sactas 3 Data mput and selection of ODLOTS ui edem aa ei etedese tu Us obe v ope uod lo eed tud 3 Sallis putobthesc uby Sui ou iet hee era bett eodera etat DUE To eee acute Do batab o taba t1 Rue ef 3 I2 Eres 1i DR NOU ME ENT 6 NA PEE SUI ENT TT Tm 6 Fa Cunen VCURONIZ INTO TEE ETT TEE 8 3 2 9 bU DEC COMUONS crassa tt cont A E equ seed eines 10 29 9 Metrics selection and fime IhEeryalun sos ai
20. amp Gear cx Pro ITOA TOAm 10M 2P TOAm 12 where c t T with t TOA TOAm 31 Single value metrics Single value metrics give a single numerical value that represents the agreement between the two curves Seven single value metrics were considered in this work 1 the correlation coefficient metric 2 the NARD correlation coefficient metric NARD 3 Zilliacus error metric 4 RSS error metric 5 Theil s inequality metric 6 Whang s inequality metric and 7 the regression coefficient metric 9 12 The first two metrics are based on integral comparisons while the others are based on a point to point comparison The definition of each metric 1s shown in Table A3 Table A3 Definition of single value metrics Integral comparison metrics n gt c m gt c m Correlation Correlation Coefficient n c Oc In Em Xmj Coefficient NARD 2 y max m2 E _ max 0m 2 Weighted Integrated Factor max mj c 2 72 X max mj c Point to point comparison metrics RENE g ome 0 de RSS error error s E Theil s V 2c m inequality 2 c m l Whang s inequality Ylc l YImil l l Regression i n 1 Gm coefficient n m m ANOVA metric ANOVA metrics are based on the assumption that is two curves do in fact represent the same event then any differences between the curves mus
21. angle and velocity of impact point of impact behavior of the vehicle s mechanical components etc Before evaluating the comparison metrics the RSVVP preprocessing options will be used to correctly prepare the data by filtering and synchronizing the original acceleration curves Next the two curves will be compared evaluating the comparison metrics of the NCHRP 22 24 profile based on both the acceleration time history and their integrals 1 e velocity time histories 7 1 Preprocessing of the original curves In order to show how each preprocessing option 1 e filtering and synchronizing contributes to the improvement of the original input curves the preprocessing operations are applied incrementally step by step in this example All the preprocessing operations could have been applied simultaneously First the two acceleration time histories 1 e curve 1 and 2 are loaded into RSVVP considering the curve 1 as the true curve Figure 15 In this case we only want to compare a single pair of curves so the Single Channel option is selected at the beginning of RSVVP 21 Input curves and Preprocessing options Input Curves C curve 1 dat C curve 2 dat A lie ita True curve Test curve Ecc one amp ARGIIDTRTTT 1 ITI Fez 0 0 3 0 4 Preprocessing Filter Options Sync Options No filtering No synchronization Optional user defined CFC meien Shift I D
22. celeration data or rotational velocities 12 The default metrics profile is NCHRP 22 24 It is also possible to evaluate all the fourteen available comparison metrics or just some of them by selecting respectively the option All metrics or User selected metrics from the drop down menu located at the top of the graphical interface Figure 7 E GUL metrics Metric profile Y NCHRP 22 24 suggested NCHRP 22 24 suggested Metrics All metrics MPC metrics T User selected metrics Geers CSA The qus NCHRP 22 24 suggested V Sprague amp Geers vel Zilliacus error Correlation Coefficient e Russell RSS error Correlation Coefficient NARD NCHRP ao suggested Knowles amp Gear All metrics ANOVA metric E dicil User selected metrics v Ray laare a ornen ea Time window for curve comparison Whole time window and User defined time window Y omm Emm Figure 7 Selection of the metric profiles In case the User selected metrics profile has been selected the checkbox beside each available metric will become active and it will be possible to select which comparison metrics to evaluate by checking the corresponding checkbox Figure 8 Metric profile User selected metrics Metrics MPC metrics Single value metrics V Geers Z Whang s inequality Weighted Integrated Factor Geers CSA 7 Theil s
23. e user a warning message if no filtering and or synchronization options are selected After the test and true curves have been preprocessed it is possible to proceed to the next step the selection of the metrics and the time interval on which to evaluate them 7 2 Metric selection and evaluation In this example the NCHRP 22 24 metrics profile is selected This evaluates the ANOVA metrics and the Sprague and Geers metrics using the acceleration and the velocity time histories respectively The latter are obtained by integrating the former acceleration curves Also the curves are compared both considering their total length Whole time window and on some user defined time intervals User defined time window The metric evaluation is initiated by pushing the Evaluate metrics button Figure 18 24 background These graphs are saved as output files by the RSVVP and represent the time histories of metrics and other curves see chapter 6 for more information about the output files Metric profile NCHRP 22 24 suggested Metrics MPC metrics Single value metrics Geers Whang s inequality Weighted Integrated Factor Geers CSA Theil s inequality Regression coefficient Sprague amp Geers vel Zilliacus error Correlation Coefficient Russell RSS error Correlation Coefficient WARD ANON S metric E Ray Time window
24. ependently from each other Once the desired preprocessing options have been selected press the Preprocess curves button located immediately below the Preprocessing box to create a preview of the preprocessed curves If the preview is not satisfactory any of the previous options can be changed and performed again until a satisfactorily preview graph is obtained In order to proceed to the next step 1 e metrics selection it is necessary to push the Preprocess curves button even if no optional preprocessing options have been selected Regardless of whether any optional preprocessing feature has been selected or not RSVVP still performs a series of basic and necessary preprocessing operations to the original input curves like re sampling of the two curves to the same sampling rate and trimming of the longer curve to the same length as the shorter curve In this case the previewed preprocessed curves would appear exactly the original ones Following is a description of each available pre processing option 3 2 1 Filtering RSVVP gives the user the option of filtering the two input curves This option can be very useful in case the original input curves display some level of noise e g noise created by the transducer during the acquisition process of experimental curves or undesired high frequency vibrations In order to obtain a value of the comparison metrics that is as reliable as possible it is very important to remo
25. erical simulations a Analysis of Variance ANOVA of acceleration signals and b Sprague and Geers metrics for the velocity signals The ANOVA metrics are based on the residuals between the true and test curves while the Sprague and Geers metrics are evaluated using the velocity curves obtained by integrating the test and true curves 1 e the velocity time histories in case the input curves are acceleration time histories Although the ANOVA and the Sprague and Geers metrics are the only metrics included in the NCHRP 22 24 procedure RSVVP still gives the user the ability to evaluate other comparison metrics The second profile All metrics automatically selects all the fourteen different comparison metrics that are available in RSVVP while the third profile 1 e User selected metrics allows the user to select the desired comparison metrics The fourteen different comparison metrics available in RSVVP are described in Appendix A The metrics can be divided into three main categories e MPC metrics e Single value metrics and e ANOVA metrics The NCHRP 22 24 profile is the only one which evaluates the Sprague and Geers metrics using the integrals 1 e velocity time history of the test and true curves instead of the direct test and true curves e g acceleration time history RSVVP automatically integrates the data once to obtain the velocity curves so users using the NCHRP 22 24 profile should specify either ac
26. ethod of the Minimum area of residuals is selected Figure 17 shows the results obtained using both methods The synchronization based on the Minimum area of residuals gives better visual results in this case 23 r gt BD input curves and Pre options RSVVP Software NCHRP 22 24 RSVVP Software NCHRP 22 24 Input Curves Input Curves C curve 1 dat C curve 1 dat C curve 2 dat Load Test Curve C curve 2 dat 60 T T T T T 60 T T T T T i i ep True curve A True curve Test curve j Test curve 40 0 2 0 3 0 4 0 5 E 02 0 3 04 0 5 0 6 Preprocessing Preprocessing Filter Options Sync Options Fitter Options Sync Options CFC 60 Least Square error CFC 60 Minimum absolute ar Shift Shift Drift User value User value Apply to True and Test curves and Test curves Optional user defined CFC SSG iei Optional user defined CFC Correction controls Applyto True Preproces curves True curve A i i Al esgsese True curve Test curve i H i H i Test curve 0 2 0 3 0 4 0 5 0 6 0 7 i 0 1 0 2 0 3 04 0 5 0 6 0 7 Proceed to metrics evaluation Proceed to metrics evaluation Release 1 3 beta Release 1 3 beta mum Figure 17 Filtered and synchronized time histories based on the a Least Square and b the Minimum Are of Residuals method RSVVP gives th
27. he same phase component as the Russell metric 6 Also the magnitude component of the Russell metric is peculiar as it is based on a base 10 logarithm and it is the only MPC metrics among those considered in this paper to be symmetric 1 e the order of the two curves is irrelevant The Knowles and Gear metric 7 8 is the most recent variation of MPC type metrics Unlike the previously discussed MPC metrics it is based on a point to point comparison In fact this metric requires that the two compared curves are first synchronized in 30 time based on the so called Time of Arrival TOA which represents the time at which a curve reaches a certain percentage of the peak value In this work the percentage of the peak value used to evaluate the TOA was 5 which is the typical value found in literature Once the curves have been synchronized using the TOA it is possible to evaluate the magnitude metric Also in order to avoid creating a gap between time histories characterized by a large magnitude and those characterized by a smaller one the magnitude component M has to be normalized using the normalization factor QS Table A2 Definition of MPC metrics MI 9 m os Integral comparison metrics M2 P I2 cimi lXchym sign cimi M s Pasa Pesa 1 Geers CSA Sprague amp Geers Mg sign m Logio 1 m Russell f where Eci Emi p cimi Point to point comparison metrics p 2 TOA TOA 2 2 Knowles
28. le vne r O ju 33 Appendix B Weighting factors This appendix presents a description of how the weighting factors used to combine together the metric values from different channels are evaluated The weighting factors are based on the absolute peak of the acceleration time histories of the True curve from each channel The weighting factors for each channel are calculated in the following way e The peak absolute value of the True curve a i for each channel is determined by scanning the data file e There are six peak values so the maximum peak value is determined by comparing the six peak values from each channel to determine the maximum peak value amax e The local weight of each channel is defined as lw 2 d max lw Y Iw i i Once the weighting factors have been evaluated the time histories of each metric are e The channel weight factor is then determined as w combined together using a weighted average Note that the combination of the time histories 1s performed for each of the metrics selected by the user 34 References 1 M H Ray Repeatability of Full Scale Crash Tests and a Criteria for Validating Finite Element Simulations Transportation Research Record Vol 1528 pp 155 160 1996 2 W L Oberkampf and M F Barone Measures of Agreement Between Computation and Experiment Validation Metrics Journal of Computational Physics Vol 217 No 1 Special issue U
29. llation 2 1 System requirements RSVVP has been written and compiled using Matlab Running RSVVP requires either the full Matlab version 7 or higher software or the freely distributable MATLAB Component Runtime MCR be installed on the system In either case the minimum hardware requirements to run RSVVP are iN 32 bit version 64 bit version Intel amp Pentium 4 and above Intel Celeron Intel amp Pentium 4 and above Intel Intel Xeon Intel Core AMD Athlon 64 AMD Celeron Intel Xeon Intel Core Opteron AMD Sempron AMD64 512 MB 1024 MB 510 MB MATLAB only 510 MB MATLAB only Installation of the MATLAB Component Runtime The source code of RSVVP has been completely written in Matlab version R2007b and then compiled as an executable file for Windows XP Vista in order to create a standalone program which can run on machines that do not have Matlab installed on them Although it is a standalone application if Matlab is not installed RSVVP requires that the MATLAB Component Runtime MCR program be installed on the machine MCR provides all the necessary Matlab functional support for the correct execution of the RSVVP software Hence before running RSVVP on a machine without Matlab it is first necessary to install the MCR environment MCR has to be installed only once The RSVVP exe file and the MCR environment can be downloaded from http civil ws2 wpi edu Documents Roadsafe NCHRP22 2A RSVVP RSVVP
30. ncertainty quantification in simulation science pp 5 36 2006 3 T L Geers An Objective Error Measure for the Comparison of Calculated and Measured Transient Response Histories The Shock and Vibration Bulletin The Shock and Vibration Information Center Naval Research Laboratory Washington D C Bulletin 54 Part 2 pp 99 107 June 1984 4 Comparative Shock Analysis CSA of Main Propulsion Unit MPU Validation and Shock Approval Plan SEAWOLF Program Contract No N00024 90 C 2901 9200 SER 03 039 September 20 1994 5 M A Sprague and T L Geers Spectral elements and field separation for an acoustic fluid subject to cavitation J Comput Phys pp 184 149 Vol 162 2003 6 D M Russell Error Measures for Comparing Transient Data Part I Development of a Comprehensive Error Measure Proceedings of the 68th shock and vibration symposium pp 175 184 2006 7 L E Schwer Validation Metrics for Response Time Histories Perspective and Case Studies Engng with Computers Vol 23 Issue 4 pp 295 309 2007 8 C P Knowles and C W Gear Revised validation metric unpublished manuscript 16 June 2004 revised July 2004 9 J Cohen P Cohen S G West and L S Aiken Applied multiple regression correlation analysis for the behavioral sciences Hillsdale NJ Lawrence Erlbaum 3rd ed 2003 10 S Basu and A Haghighi Numerical Analysis of Roadside Design NARD
31. nter 468 593 Conditional Format Good Neutral _ Insert Delete Format Sort amp Find amp M Formatting as Table m z Filter Select A41 m Alignment fa Number ir Editing N Sprague amp Geers metric velocity time histories Sprague Geers Magnitude Sprague Geers Phase _Sprague Geers Comprehensive ANOVA Metrics acceleration time histories Average _9 Std 10 T test 11 T T_c Nan awn wo 44 gt M Sheet1 lt Sheet2 Sheet3 2 Ready RSVVP metric results NCHRP 22 24 proposed metrics B Value 96 Value 1 9 1 3 2 3 Value Value 0 0 14 2 46 0 96 Value 0 04 0 25 6 88 2 68 Figure 24 Excel file with the summary of the metrics value 20 Whole time interval 0 0 53975 User time interval 1 0 05005 0 14995 User time interval 2 0 15005 0 19995 2 9 1 6 0 09 0 27 10 85 4 23 Appendix A Comparison metrics evaluated by RSVVP A brief description of the metrics evaluated by RSVVP is presented in this section All thirteen metrics available in RSVVP are deterministic shape comparison metrics Details about the mathematical formulation of each metric can be found in the cited literature Conceptually the metrics evaluated can be classified into three main categories 1 magnitude phase composite MPC metrics 11 single value metrics and 111 analysi
32. o curves which have to be compared are called respectively the true curve and the test curve The true curve represents the baseline or the reference curve and is assumed to be the correct response while the test curve represents the model or experiment which has to be verified and or validated As the comparison metrics assess the degree of similarity between any pair of curves in general the input curves may represent various physical entities e g acceleration time histories force deflection plots stress strain plots etc Comparison metrics provide an objective measure of how well two curves match each other and therefore can be applied to virtually any pair of curves where a comparison 1s required A typical application of the metrics evaluated by RSVVP is the validation of a numerical model by comparing the experimental results with the numerical simulation results Another application could be to check the repeatability of one experiment by comparing the results obtained from several repetitions of the same experiment Yet another application is to verify the results of one numerical simulation with the results of another numerical simulation In RSVVP two main types of comparison can be performed e Single Channel i e one test curve and one true curve or e Multiple Channel i e up to six pairs of true and test curves In the Single Channel option the comparison metric is based on the comparison of a single
33. oe teen e E te ti Ed 11 no SNC TICS SC le ON Seria tare T tare ait ett eec DeL anette 12 51 2 MEINE aa A eee 13 Menes evala Oe E E E ue duse tudu ned 15 4l Whole ume WIDdOW scccacesszsssisetsdavesactaxeiedocensteadtsaestadedavagectaxelecaceatipacbeasdiedidavesaataxelaees 15 4 2 Definition of a User defined time Window cccccccccccccceccceceeeeeeeeesseeeeeseeeeeeeeeeeees 15 Sere ooi erm E 17 OUtDUL orres US 26 csinsazsiasenz ut wictensivracienccutanilay dlsidenndutnddeaseeantaacubsuiadsl dadteadebuutensPesitnccuteutaleadnsees 19 6 1 Table of results Excel NO SITS SU E S tando Ruta EEE A E 9 OU MME 5 21 aa a een etn Senn UM 20 IEEE CUOI LT Omm Tals Preprocessing of the ortemal CUIVES cse io Exp Po P Ep tI Du a aaron es 7 2 Metric selection and evaluation cccccecsceccscscsccccecscscaccecscscscscecscsceccscscscnceececs 7 3 Save results and CX1t c cccscsceccscsceccccecscecsccscececscescecscscsccecscecscesescscscecescscseaceseecs Appendix A Comparison metrics evaluated by RSVVP oo cccccccnntttteetsesseseeeeeeeeeeeeeees Appendix Be Wels ntin TaACtOUs soser an O EO References il 1 Introduction to RSVVP The Roadside Safety Verification and Validation Program RSVVP calculates quantitative comparison metrics used in verifying and validating roadside safety crash tests and simulations Comparison metrics are mathematical measures that provide an
34. of the user defined time window have been shown RSVVP will prompt the user for a new User Defined time window The results obtained for each time interval will be saved separately To create a User defined time interval it is necessary to specify the lower and upper time boundaries RSVVP shows a window with a graph of the test and true curves and two blank fields at the bottom which are used to define respectively the time value of the lower and upper boundary fill in the desired values and press the Evaluate metrics button to start the evaluation of the metrics on the defined interval Once values are input into the fields the upper and lower boundaries are graphically shown as vertical lines in the graph area with the test and true curves In case of multichannel input a drop box menu located at the bottom of the window allows to select which channel to show on the plot area Note that the defined upper and lower boundaries do not change when a new channel is plotted as the same time interval must be used for each channel 16 r Bj User Defined Time Interval 50 0 05 0 15 Lower limit Upper limit Figure 11 Selection of the User defined time window 5 Screen output For each of the time intervals on which the comparison metrics were evaluated RSVVP shows various screen outputs to present the results e Graph of the test and true curves e Values of the comparison metrics
35. rift User value Apply to True and Test curves v 0 6 0 4 0 2 0 0 01 02 03 04 05 06 07 Proceed to metrics evaluation Release 1 3 beta Figure 15 Original acceleration time histories loaded into RSVVP As the original acceleration time histories are characterized by a certain level of noise and high frequency vibrations they need to be filtered In this example a CFC 60 filter 1s selected Figure 16 22 rOCes na op RSVVP Software NCHRP 22 24 Input Curves C curve 1 dat C curve 2 dat 60 40 rgo SF pki 5 5 6 64 a Finn E e E E E E E Du pe 40 0 s 0 3 0 4 Preprocessing Fitter Options Sync Options CFC 60 m No synchronization xj Correction controls Optional user defined CFC Shift IM Drift User value Apply to True and Test curves X Preprocess curves p True curve Test curve 0 2 0 3 04 0 5 Proceed to metrics evaluation aay Release 1 3 beta Figure 16 Original and filtered acceleration time histories From the graph it can clearly be seen that the two time histories are not synchronized to each other as the initial time at which the accelerations were being recorded was not the same Initially a synchronization based on the Least Square Method is selected but as the results are not completely satisfactory in a second phase the m
36. rs Mario Desktop RSSVP_GUI_08_09 2 ia 7 Load True Curve N t 1 ee e c EE LE E Load Test Curve TE e E SN Input Curves toasterimario My_Documents Matlab RSSVP_ toastenmario My Documents Matlab RSSVP __ B Select the True curve gt wm v ww Look in hos e tj ey Eg z Name Date modified Type ni 01 02 03 04 O05 06 07 s C Raw curves a 0 26 2008 11 22 PM File Folder e Preprocessing ij Sim_data 6 4 2008 11 46 AM File Filter Options Sync Options _ Test_data 6 1 2008 3 36 PM File No filtering No synchronization v Optional user defined CFC Correction controls Shift Drift E Apply to True and Test curves Preprocess curves 4 m File name Sim_data m Files of type All Files x Release 1 3 beta Figure 2 Input the test and true curves 3 2 Preprocessing After the true and test curves have been selected and the preview graph shown RSVVP is ready to perform some basic and necessary pre processing operations on the original input curves as well as some optional preprocessing operations which can be selected by the user on the basis of the appearance of the original test and true curves Three optional pre processing operations are available e Filtering e Synchronization and e Shift Drift control All three of these pre processing controls are optional and can be selected ind
37. s of variance ANOVA metrics MPC MPC metrics treat the curve magnitude and phase separately using two different metrics i e M and P respectively The M and P metrics are then combined into a single value comprehensive metric C The following MPC metrics were used a Geers original formulation and two variants b Russell and c Knowles and Gear 3 8 Table A2 shows the analytical definition of each metric In this and the following sections the terms mj and c refer 66299 1 to the measured and computed quantities respectively with the 1 subscribe indicating a specific instant 1n time In all MPC metrics the phase component P should be insensitive to magnitude differences but sensitive to differences in phasing or timing between the two time histories similarly the magnitude component M should be sensitive to differences in magnitude but relatively insensitive to differences in phase These characteristics of MPC metrics allow the analyst to identify the aspects of the curves that do not agree For each component of the MPC metrics zero indicates that the two curves are identical Each of the MPC metrics differs slightly in its mathematical formulation The different variations of the MPC metrics are primarily distinguished in the way the phase metric is computed how it is scaled with respect to the magnitude metrics and how it deals with synchronizing the phase In particular the Sprague and Geers metric 5 uses t
38. stogram Residuals cumulative distribution Percentage 95 Percentage 96 e ce Residual time history Residuals Mean 90th percentile upper boundary 90th percentile lower boundary 08 06 04 02 0 0 2 04 0 6 0 8 1 0 5 0 0 5 1 Residuals Cumulative Residuals Proceed to evalute metrics Figure 13 Screen output for a the NCHRP 22 24 profile and b the All metrics User defined profiles According to the profile selected for the time interval the window with the screen output can give the user the option to 1 proceed to the evaluation of a new interval and or 2 to save the results and quit the program Push the button corresponding to the action you want to take and wait till the next message appears 6 Output of results During the curve preprocessing and the evaluation of the metrics RSVVP creates different types of output All the output data are saved into a subfolder named Results located inside the folder where RSVVP was installed e g if the directory where RSVVP was installed is CARSVVP the Output folder is C RSVVP Results 6 1 Table of results Excel worksheet The final values of the comparison metrics are saved in the Excel file Comparison Metrics xls In the spreadsheet the values of the comparison metrics for each time interval 19 considered during the evaluation process are saved in separate columns The label of each
39. t be attributable only to random experimental error The analysis of variance 1 e ANOVA is a standard statistical test that assesses whether the variance between two curves can be attributed to random error 1 2 When 32 two time histories represent the same physical event both should be identical such that the mean residual error e and the standard deviation of the residual errors o are both zero Of course this 1s never the case in practical situations e g experimental errors cause small variations between tested responses even in identical tests The conventional T statistic provides an effective method for testing the assumption that the observed e is close enough to zero to represent only random errors Ray proposed a method where the residual error and its standard deviation are normalized with respect to the peak value of the true curve and came to the following acceptance criteria based on six repeated frontal full scale crash tests 1 e The average residual error normalized by the peak response i e should be less than five percent p 2 46 m M rax n lt 0 05 m e The standard deviation of the normalized residuals 1 e o should be less than 20 percent e The t test on the distribution of the normalized residuals should not reject the null hypothesis that the mean value of the residuals is null for a paired two tail t test at the five percent level to 995 Q 90 percenti
40. the comparison metrics Two different synchronization options are available in RSVVP Both options are based on the minimization of a target function which is respectively 1 the absolute area between the two curves i e the area of the residuals and 2 the squared error between the two curves Although these two methods are similar they sometimes give slightly different results Selecting one of these methods will result in the most probable pairing point for the two curves Once the original curves have been preprocessed the user is given the possibility to refine the synchronization By default RSVVP does NOT synchronize the input curves To apply the synchronization option click on the drop down menu in the Sync Options box Figure 4 and select one of the two desired synchronization method 1 Minimum absolute area of residuals or 2 Least Square error Once the curves have been preprocessed by pushing the Preprocess curves button a pop up window will ask the user if the synchronization is satisfactory If the No button is pushed another pop up window with a slider will appear Figure 5 Moving the slider changes the initial starting point of the minimization algorithm on which the whole synchronization process is based In this way the user can manually adjust the synchronization process of the wo curves options and Preprocessing Opt Ka ie RSVVP Software NCHRP 22 24 Input Curves tus Cuwa C sers
41. ue Geers Comprehensive 5 Acceleration Residuals Residuals time history Residuals histogram Residuals cumulative distribution 8 100 7 90 80 6 70 x 5 E 60 T 2 S 3 oD cC mus m 450 2 5 ao o o a s 40 a 3 a 30 Residual time history 2 Residuals Mean 20 90th percentile upper boundary 1 10 90th percentile lower boundary 0 0 0 0 04 0 06 0 08 0 1 0 12 0 14 0 16 0 6 0 4 0 2 0 0 2 0 4 0 6 0 8 06 04 02 0 0 2 0 4 0 6 0 8 Residuals Cumulative Residuals Evaluate on a new interval Save results and Exit Figure 21 Output of results for the selected time interval By clicking the Evaluate on a new interval button a new time interval is now defined and the metrics are evaluated again following the same procedure used for the first time interval In this case the time interval defined is 0 15 sec 0 20 sec Figure 22 and gives the results shown in Figure 23 j i mx i9 User Defined Time Interval 00 15 0 2 Lower limit Upper limit Figure 22 Time interval 0 15 sec 0 20 sec 27 Table Results NCHRP Comparison Metric values Time interval 2 0 15005 0 199951 True and Test curves velocity True and Test curves acceleration 1 2 True curve True curve 25 Test curve Test curve ANOVA Metrics T test MPC Metrics Value Value 95 15 Average 0 09 Sprague Geers Magnitude 2 5 Standard
42. ve noise from both the test and true curves and to make sure the two curves are filtered in the same way While filtering is optional in RSVVP it is recommended that unfiltered data is used and that both the test and true curves are filtered in RSVVP In this way the user can be sure that both curves were filtered in exactly the same way The filter algorithm used by RSV VP is compliant with the SAE J211 1 specifications The user can select between different SAE Channel Frequency Class CFC filters 60 180 600 and 1000 Table 1 shows the specifications of each CFC value as defined by SAE J211 1 Table 1 Specifications for the usual CFC values CFC value 3 dB limit frequency Hz Stop damping dB 100 30 180 300 1000 1650 While it is not recommended if the user wants to use a filter class different from the standard SAE J211 filters it is possible to specify any other user defined filters parameters in RSVVP By default RSVVP does NOT filter the input curves To apply the filter option click on the drop down menu in the Filter Options box Figure 3a and select the desired CFC filter class RSVVP Software NCHRP 22 24 ce Load True Curve tosd Test Gave C WsersiManolDesktopiRSS 2SSVP GUI 08 09 7 i Test curve NM Filter Options Mo filtering d A No filtering i a User defined CFC CFC 60 l r P Optional user defined CFC CFC 180 BI n enn Po 300 CFC 600
Download Pdf Manuals
Related Search