Home
User Guide for PKgraph Package
Contents
1. Figure 24 Exploratory data analysis The detailed information for this patient is selected for investigation 29 PRED 6 PRED y PRED vs DV Covariates PRED 7 PRED Dv7 BUNC COV 7 AGE y IPRED vs DVjCovariates IPRE 8 IPRE y DV 8 CONC COV 8 AGE General Configure layoutx 5 layout y 5 graphics 0 lattice ggplot2 cancel Clean Figures ox Command area 4 WRES Ed LES OOO GO GOOD O o O 0 AMIAAMDORMDO CM O o O e Oo g 6 D O OOGDGUNNEDOUDPGED GD O DIO IEC DOOR Mog 8T I Pia t OD ON ONXINEOXIDOOTO DO O Oo O GRR OUDOODOO Q o 10 20 30 PRED Figure 25 Structural model diagnostics R Visualization for influence analysi Visualization for influence analysis Visualization for NONMEM runs Target directory path jackknife_dirl modelfit_dirl Simulation folder pattern NM run NONMEM result file name CSI IVIESTFPDF 1 fit Patient ID ID Iz Plot variable CL z General Configure graphics 9 lattice ggplot2 Cancel ciem Figures ox EH Command area a Case deletion ID Figure 26 Influence analysis 30 R mydata Scatterplot yr File Options File Options Brush cor2 simID 20 1901 simID 52 Figure 27 Influence analysis linking results from multidimensional scaling and parallel coordinate plots l
2. a EN Lc 5 jS Histogram comparison Scatter plot comparison ETA2 4 AGE WT 0 023804 34 823000 38 21200 0 250000 13 026000 15 265000 100 0 529830 6 417200 0 227210 0 023804 34 823000 38 21200 0 500000 14 984000 14 953000 100 0 529830 6 417200 0 227210 0 023804 34 823000 38 21200 0 750000 14 160000 14 648000 100 0 529830 6 417200 0 227210 0 023804 34 823000 38 21200 1 000000 19 316000 14348000 100 0 529830 6 417200 0 227210 0 023804 34 823000 38 21200 1 500000 13 146000 13 768000 100 0 529830 6 417200 0 227210 0 023804 34 823000 38 21200 12 921000 13 211000 100 0 529830 6 417200 0 227210 0 023804 34 823000 38 21200 2 500000 8485000 12 677000 100 0 529830 6 417200 0 227210 0 023804 34 823000 38 21200 3 000000 16 437000 12 164000 100 0 529830 6 417200 0 227210 0 023804 34 823000 38 21200 4 000000 10 724000 11 200000 100 0 529830 6 417200 0 227210 0 023804 34 823000 38 21200 6 000000 8 735200 9 495400 100 0 529830 6 417200 0 227210 0 023804 34 823000 38 21200 ood d d d d d d d d Figure 17 Menu items in Model comparison R PKgraph Project Configure Data management Exploratory data analysis PK Models Model validation Model comparison Interactive diagnostics BOX WE amp amp HB open preferences subset clear save help quit Current directory C Projects Phd pkgraph 1130 1 pkdata 2 CS1 IV1ESTFPDF it 3 CS1 IVIESTFPDF fit2 ID items 4 Ha 4 DOSE j CLb tjV ETAL 4 ETA2 4 AGE
3. 529830 6 4 bootl 1 1 000000 19 316000 14 348000 100 0 529830 64 boot2 1 1 500000 13 146000 13 768000 100 0 529830 64 bugReport txt 1 2 000000 12 921000 13 211000 100 0 529830 6 4 C3 s3 1 pdf B 1 2 500000 8 485000 12 677000 100 0 529830 64 cdd raw resultsl csv 1 3 000000 16 437000 12 164000 100 0 529830 64 cdd skipped individualsl csv 1 4 000000 10 724000 11 200000 100 0 529830 6 4 cdd2 1 6 000000 8 735200 9 495400 100 0 529830 64 cdd3 1 8 000000 7 697000 8 050000 100 0 529830 6 4 CS1 IVIESTFPDF fit 1 12 000000 4 479000 5 785800 100 0 529830 64 CS1_IVIESTFPDF fit2 1 16 000000 2 418300 4 158500 100 0 529830 6 4 Figure 10 Menu items in Model validation 5 6 3 Bootstrap summary PsN This function is specifically for PsN boot results Figure 14 It takes two result files from PsN raw_resultsl csv and included_individuals1 csv and generates related plots 5 6 4 Visualization for bootstrap This function is to visualize data from boostrap multiple NONMEM runs Let s use multiple NONMEM run form PsN Figure 15 and find file directory for these runs Then we can select parameters as shown in Figure 16 These parameters include e Target directory path the path for multiple NONMEM runs It isa required parameter e Dootstrap folder pattern the common name style for multiple NONMEM runs For this example it is NM run It is a required parameter e NONMEM result file name the fit result for each NONMEM run In this example it
4. CS1 IVIESTFPDF fit2 1 16 000000 2 4183 Figure 4 Menu items in Configure data users can click the tab of data area and select the proper one as the current working data e Factor factor categorical variables Graphical packages require the vari able to be factor type in order to display the categorical symbol in figures For example in Figure 21 after we make the ISM as a factor the sym bol 0 1 is show as the subtitle on the figure otherwise the name of variable ISM will be shown instead 5 4 Exploratory data analysis This menu item is utilized to explore data and screen patterns The explanation for the basic parameter set is available at section Basic graphical parame ters It has the following functions Figure 6 e Univariates plot univariate varaiables e Bivariates plot bivariate variables e Parallel coordinate plot Parallel coordinate plot for multivariate variables e Scatterplot matrix Scatterplot matrix for multivariate variables 10 Project Configure Exploratory data analysis PK Models S dp u open preferences Factor fclear save help quit Current directory C Projects Phd pkgraph 1130 1 pkdata items 4 ha ID 4 TIME 4 CONC al pdf 1 0 00000 0 0000 bigVar pdf 1 0 250000 13 026 boot included individualsl csv 1 0 500000 14 984 boot raw resultsl csv 1 0 750000 14 160 bootl 1 1 000000 19 316 boot2 1 1 500000 13 146 bugReport txt 1 2 000000 12 921 c3 s3 1 pd
5. Configure model result in PK models 5 5 3 Basic goodness of fit plots Goodness of fit plot is one of key tools to check model fitting These kinds of plots will give an overall perspective of model performance including scatter plot for concentration versus PRED concentration versus IPRED PRED versus IDV time and IPRED versus IDV time 5 5 4 Parameters Generally there are assumptions for distribution of parameters during modeling process The histogram is utilized to check this distribution In addition the correlation of parameters has significant effect on modeling performance and it can be checked by scatter plots or a scatterplot matrix The interface for this function is shown in Figure 9 After users choose proper parameters in the left window the system will produce all figures automatically Users can pick specific figures for diagnosing with functions in the toolbar 5 5 5 Random effects The assumptions for random effects also need to be tested for distribution and correlation by histogram scatter plots or a scatterplot matrix 14 Rhe 00 NENNEN Parameter Distribution of parameters CL Distribution of parameters QQ cL CL M 08 o EL i Scatterplot matrix of parameters 0 7 e e d Se a cd En r1 Parameter vs parameter i e o 0 9 e e Py ee T o Parameters x cL 0 4 e uF E i Se Parameters y V y 0 3 Pt e CA General Configure T e 9e
6. MJ DOSE yx yl 0 Edge brushing XJ My cL 4 Off po El av f 9 Case ID x v era E x v Era 5 i y 4 X Y Ace 0 e 3 X Y WT 0 i i a m X Y CLCR x v conca x v PRED o E d o X Y RES 20 8 l x x wres a a Lil i tmp data 1600 x 17 R data frame tmp data 0 ao 9 9 B Figure 23 Exploratory data analysis Peak is identified with brushing This patient is from light weight and middle age group 28 anms 8 e Bee x x TIME y CONC main xlab CONC 4 ID4 IPRE 4 DOSE CL 4V 4 ETAL 4 ETA2 4 ylab 0 000000 55 101 930000 250 000000 0 137300 2 452700 1 123200 0 98557 117 370003 55 100 510002 250 000000 0 137300 2 452700 1 123200 0 98557 mnt Iz 82848000 55 99 112999 250 000000 0 137300 2452700 1123200 0 98557 _ General Configure E 112970001 55 97 736000 250 000000 0 137300 2452700 1 123200 0 98557 cond iz 81 925003 55 96 377998 250 000000 0 137300 2 452700 1 123200 0 98557 PE 84902000 55 93718002 250 000000 0 137300 2452700 1 123200 038557 103 080002 55 91 130997 250 000000 0 137300 2 452700 1 123200 0 98557 m5 98 953003 88 615997 250 000000 0 137300 2 452700 1 123200 0 98557 graphics lattice ggplot2 91 827003 86 169998 250 000000 0 137300 2 452700 1 123200 0 98557 64 407997 250 000000 0 137300 2 452700 1 123200 0 98557 ona cn a Ao miu
7. ab a etn ten te rre Ee A 14 5 5 5 Random lt eilects 4 Log 28 mue Ros Bbw eS OS GE Genes 14 0 5 0 Structural model s a duae CEE RSE Re 15 5 5 7 Residual error model eoe Rb eoe 15 5 5 8 Govariale model x lo khe ee Sete ee SS wl ees 15 9 0 Model validation 4 5 22 eee so awad d eec bE Rege 16 5 6 1 Influence analysis summary PsN 16 0 6 2 Visualization for influence analysis 16 5 6 3 Bootstrap summary PsN 17 5 6 4 Visualization for bootstrap 17 ox Model comparison ue saa ae a A eee Me 18 Dad elecb bases hus ues e deua desc Voie e 19 5 7 2 Configure mapping naaa Reo OE a atit at 19 959 COMPASSO 2 4 el amp coded ee OP EE See 0 94 19 5 8 Interactive graphics 6 444 8 28 oou ee 20 6 Example 21 1 Introduction Population pharmacokinetic PopPK modeling has become increasing impor tant in drug development because it allows unbalanced design sparse data and the study of individual variation However this complexity of the model makes it a challenge to diagnose the fit Graphics can play an important and unique role in PopPK model diagnostics The software described in this paper PKgraph provides a graphical user interface for PopPK model diagnosis with interactive graphics It also provides an integrated and comprehensive platform for analysis of pharmacokinetic data including exploratory data analysis goodness of model fit model validation and
8. boot 13 146000 100 000000 34 823000 38 212000 0 529830 6 417200 rT 4 479000 100 000000 34 823000 38 212000 0 529830 6 417200 23 Lpdf 2418300 100 000000 34 823000 38 212000 0 529830 6 417200 pe 12 921000 100 000000 34 823000 38 212000 0 529830 6 417200 cdd_skipped_individuals1 csv 8 485000 100 000000 34 823000 38 212000 0 529830 6 417200 cdd2 4 758600 100 000000 34 823000 38 212000 0 529830 6 417200 cdd3 2 365500 100 000000 34 823000 38 212000 0 529830 6 417200 CS1_IVLESTFPDF fit 16437000 100 000000 34 823000 38 212000 0 529830 6 417200 CSI IVIESTFPDF fit2 10 724000 100 000000 34 823000 38 212000 0 529830 6 417200 l 8 735200 100 000000 34 823000 38 212000 0 529830 6 417200 dida 7 697000 100 000000 34 823000 38 212000 0 529830 6 417200 0 000000 100 000000 40 732000 50 878000 0 484020 9 289200 ANACAN AAR nnnnnn AN TINNA EN o70nnn ADANIN MIEN 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 Figure 20 Current data set for Model comparison 25 Model comparison Model 1 2 CS1 IVIESTFPDF fit xl CLx x Model 2 3 CS1 IVIESTFPDF fit2 2 CS1 IVIESTFPDF fit 3 CSI IVIESTFPDF fIt2 x2 CLy number of bins main xlab ylab type percent y General Configure cond ISM 2 layout x 1 layout y 2 graphics 0 lattice ggplot2 Command area 4 r Figure 21 histogram comparison for Model comparison 26 RGui R Co
9. is C 1 IVIESTFPDF 1 fit It is a required parameter e Bootstrap key table path the path for bootstrap key file which is file describing the sampling schema for patient IDs It is a required parameter 17 Influence analysis summary PsN E FE 23 PsN Case deletion diagnostics PsN summary for influence analysis Result file cdd raw resultsl csv Note 1 raw resultsl csv default file Deleted ID file cdd skipped individualsl csv Note 2 skipped individualsl csv default file General Configure graphics 0 lattice ggplot2 Command area Cov ratios 0 4 0 6 Cook scores Figure 11 Influence analysis summary PsN e Bootstrap key table name The file describes the sampling schema for patient IDs In this example it is included_individuals1 csv It is a required parameter e Patient ID the ID for each subject It is a required parameter e Plot variable the variable you use to detect difference among patients For this example we choose CL It is a required parameter e zlabel the name label for each NONMEM run It is optional 5 7 Model comparison In this process there are three main steps 1 select datasets 2 configure mapping 3 comparison Figure 17 The first step is to select datasets for comparison Currently the program only supports comparison of two models Then users proceed to configure mapping by matching column names or vari able nam
10. 000 5 785800 16 000000 2 418300 4 158500 CS1 IVIESTFPDF fit CS1 IVIESTFPDF fit2 items 4 a ID TIME Random effects p al pdf g 1 0 00 umma model 1 e l boot included individualsl csv 1 0 1 boot raw resultsl csv 3 1 0 750000 14160000 14 648000 1 bootl 1 1 000000 19 316000 14 348000 1 boot2 1 1 500000 13 146000 13 768000 1 T 1 2 000000 12921000 13211000 1 C3 s3 1 pdf 1 2 500000 8 485000 12 677000 1 cdd raw resultsl csv 1 3 000000 16 437000 12 164000 1 cdd skipped individualsl csv 1 4 000000 10 724000 11 200000 1 cdd2 1 6 000000 8 735200 9 495400 1 cdd3 1 8 000000 7 697000 8 050000 1 1 1 1 1 Figure 7 Menu items in PK models The interface for this function is shown in Figure 8 The fixed column left is column name from data and the selectable column right is variable name from the default metric system Table 3 By this matching the other seven functions can be performed However these functions work independently and some variables in the default metric system must be matched to those in real data Table 4 Functions Required items to be selected in right column Individual plots ID Basic goodness of fit plots PRED IPRE DV IDV WRES Parameters PARAMETERS Structural model PRED IPRE DV IDV WRES COV Residual error model WRES PRED COV IPRE Covariate model PARAMETERS ETA WRES COV Random effects ETA Table 4 Required variables for different functions 5 5 2 Individual plots Bivariat
11. 17200 0 227210 0 023804 34 823000 38 21200 2 000000 12 921000 13 211000 100 0 529830 6417200 0 227210 0 023804 34 823000 38 21200 33 1paf 2 500000 8 485000 12677000 100 0 529830 6 417200 0 227210 0 023804 34 823000 38 21200 3 000000 16 437000 12 164000 100 0 529830 6 417200 0 227210 0 023804 34 823000 38 21200 4 000000 10 724000 11 200000 100 0 529830 6 417200 0 227210 0 023804 34 823000 38 21200 6 000000 8 735200 9 495400 100 0 529830 6 417200 0 227210 0 023804 34 823000 38 21200 8 000000 7 697000 8 050000 100 0 529830 6 417200 0 227210 0 023804 34 823000 38 21200 12 000000 4 479000 5 785800 100 0 529830 6 417200 0 227210 0 023804 34 823000 38 21200 16 000000 2 418300 4 158500 100 0 529830 6 417200 0 227210 0 023804 34 823000 38 21200 20 000000 4 758600 2 988800 100 0 529830 6 417200 0 227210 0 023804 34 823000 38 21200 24 000000 2 365500 2 148200 100 0 529830 6 417200 0 227210 0 023804 34 823000 38 21200 cdd raw resultsl csv I cdd skipped individualsl csv cdd2 cdd3 CS1 IVIESTFPDF fit CS1_IVIESTFPDF fit2 datal csv data2 csv a N A LB HB HB B HB bp ME REP e LOST l da3 0 000000 0 000000 10 547000 100 0 618210 9 481600 0 381480 0 366570 32 765000 74 83800 f chnnn 7 4 n 5nn an aa ann ACIONA n 301 40n n ccc n 7ctch nn 774 0 20nn f2 E E E E E E E E E E Es E E EJ E EJ E E E E mmm umm Imm Eum pea r mr uu HER peo pecus md Aum Bata types ata types
12. 2 included individualsl csv default file number of bins General Configure graphics lattice 9 ggplot2 tj Command area 0 ii n 0 8 0 i 9 seERR2 Figure 14 Bootstrap summary PsN In ggobi the main operation for brushing data is ctrl b By moving the brushing rectangle the users can select interesting subsets in ggobi More in formation is available at http www ggobi org if needed 6 Example One dataset from NONMEM is utilized to demonstrate PKgraph This data set has 100 patients with covariates ISM gender AGE and WT The data is fitted with one compartment model with zero order absorption and first order elimination As a text file the fitting result from NONMEM is imported into PKgraph for further investigation and analysis In the open dialog we set up file format for reading with default parameters and as a result the input data shows up on the right panel while a message Data is loaded successfully appears in the status bar at bottom of panel Alternatively to make the input process flexible users can input data into R first and then load data from Data from R environment in the open dialog All the fitted results from a wide variety of software including NONMEM SAS etc can be loaded into this package 21 Name Date modified Type Size di jackknife dirl 2 4 2010 10 50 AM File Folder J modelfit_dirl 2 4 2010 10 45 AM File Folder EN boot inc
13. Model comparison Model 1 2 CSI IVIESTFPDF fit xl CLx z Model 2 3 CS1 IVIESTFPDF fit2 x2 CLy y number of bins main xlab ylab type percent y General Configure cond ISM ES layout x 1 layout y 2 graphics 9 lattice ggplot2 Command area Figure 28 Histogram comparison for comparing distributions of CL from two models 42d 2 CS1 IVIESTFPDF fit 3 CSI IV1ESTFPDF fit2 0 0 0 2 0 4 0 6 0 8 32
14. User Guide for PKgraph Package Xiaoyong Sun February 28 2011 Binformatics and Computational Biology Program Department of Statistics Iowa State University Ames Iowa 50010 USA Contents 1 Introduction 2 Installation 3 PKgraph infrastructure 3 1 Graphical user interfaces 222222 l n oll Main interface x x naaa ox DRIED Te a oe Oko GPIO i66 2 2 oru Boe PEE EVE ed eee 3 2 FUNCHONALIMOCUIC serati a oem RABE OE ERE E Y s Quick start NEEDS UI se Yea ves ah oe Sky ee ee E ee a Lll data ame xs 4 4 4 amp ess Sidon RIA ww YS 412 JINONMIMLTOIGeES o e a a eed ee ok RO a ee ee 42 Diagnose model seis ga 3g a Se hk a ee de 4 3 Basic graphical parameters pe sm doe eO PR ee dee da 4 4 Abbreviations in the software Menu items in main interface lk 050 Gt 26 4 os a AAA De ae ee E Dal Coni een e Se He A gt koa A ey e se ed o Data Management se re Geuye Se IIIS See ae YS 54 Exploratory data analysis Gc dos died db Re Rete Ree mene Oll a e gei3 moe oooh eum EES BREE ES OB MEL uU O ee ek OR Re T 5 4 3 Parallel coordinate plots 4 demo ee Dd Scatterplot MAME e dece AOE Oe SES Se Se es 2 95 o Luo anon dedi owe dox Oe qe pU Eee ESSO Ege ded 5 5 1 Configure model result cles 95 2 Mardua POS x mous editi a e ee oed 5 5 3 Basic goodness of fit plots lll sn johnsunx1 gmail com N CU wu C2 D 1 1 1000 DOLO HParainerers x 2 26 e ER o
15. are configured successfully A AAA Figure 1 Main interface of PKgraph 3 1 2 Graph interface Selecting an item from a diagnostic module menu brings up a graph interface Figure 2 The style of the interface is the same for all diagnostic functionality It contains three areas 1 parameter setup area 2 tool bar 3 plot area e The parameter area setup allows choice of variable plot labels layout for trellis or facetted plots A choice of lattice or ggplot2 graphics is provided Note At the bottom of this area there is a module called command area which is for next release It is not fully functional at this point e The tool bar allows users to 1 save plots 2 open plots for interactive graphics ctrl b for brushing data 3 display subset selection from ggobi 4 save subset selection from ggobi and 5 close ggobi save plots this button can save the current plot from plot area The figure can be saved as pdf jpg tiff png formats This format is configured by Set saving format in the Configure menu item For multiple plots generated with one parameter set such as plots for observation concentration versus time conditioned on 50 patients R will only keep the last few patients as one page in the plot area This button will automatically save all pages for all patients with the specified figure format open plots for interactive graphics this button opens two plots in ggobi for interactive graphics The fi
16. baa bigVar pdf boot_included_individualsl csv boot_raw_resultsl csv bootl boot2 bugReport txt c3_s3_l pdf cdd_raw_resultsl csv cdd skipped individualsl csv cdd2 cdd3 CS1 IVIESTFPDF fit CS1 IVIESTFPDF fit2 CS E 4 LoL IVIESI 3_CS1_IVIESTFPDF fit2 Data types are configured successfully Figure 18 Select datasets in Model comparison 24 a BE E Hu Current directory C Projects Phd pkgraph 1130 items 4 al pdf bigVar pdf boot included individualsl csv boot raw resultsl csv bootl boot2 bugReport txt C3 s3 1 pdf cdd raw resultsl csv cdd skipped individualsl csv cdd2 cdd3 CS1_IVIESTFPDF fit CS1_IVLESTFPDF fit2 datal csv Note The goal of this step is to merge two data dd3 Please choose data variables before fit Yes ID TIME DV Covariates f2 Data is ready for model comparison Figure 19 Configure mapping in Model comparison BOX F amp F amp amp HB open preferences subset clear save help quit Current directory C Projects Phd pkgraph 1130 m CONC 4 DOSE 4 AGE 4 WT di Clx 4 Vx 4 sink 0 000000 100 000000 34 823000 38 212000 0 529830 6 417200 a 13 026000 100 000000 34 823000 38 212000 0 529830 6 417200 als E L E cv 14 984000 100 000000 34 823000 38 212000 0 529830 6 417200 14160000 100 000000 34 823000 38 212000 0 529830 6 417200 bootl 19 316000 100 000000 34 823000 38 212000 0 529830 6 417200
17. bootstrap targets for confidence interval case deletion diagnostics identify influential cases and stochastic simulation is utilized to compare mod els PKgraph mainly focuses on case deletion diagnostics and bootstrap It accepts two kinds of model validation data 1 results from PsN 2 results from multiple NONMEM runs For the first type of data PsN has the following functions bootstrap case deletion and stochastic simulation PKgraph pro vides the graphic ability to visualize the final results from PsN For the second type of data PKgraph can handle multiple NONMEM run folders and extract useful information to visualize It provides the following functions Figure 10 e Influence analysis summary PsN analyze PsN cdd results e Visualization for influence analysis apply parallel coordinate plots and multidimensional scaling to visualize data from case deletion diagnostics multiple NONMEM runs e Bootstrap summary PsN analyze PsN boot results e Visualization for bootstrap visualize data from bootstrap multiple NON MEM runs 5 6 1 Influence analysis summary PsN This function is specifically for PsN cdd results Figure 11 It takes two result files from PsN raw resultsi1 csv and skipped individuals1 csv and generates a scatter plot for cov raito versus cov score 5 6 2 Visualization for influence analysis This function is to visualize data from case deletion diagnostics multiple NON MEM runs Let s use
18. e graphics lattice ggplot2 a 10 ee Cancel Clean Figures ok e A 14 a Command area m e se 0 ns T lt Venim o at AN o ME det Lt v ov y e E 0 8 10 14 x Figure 9 Parameters in PK models 5 5 6 Structural model Structural model can be diagnosed by PRED versus concentration conditioned on time IPRED versus concentration conditioned on time WRES versus time WRES versus PRED PRED versus concentration conditioned on covariates IPRED versus concentration conditioned on covariates 5 5 7 Residual error model Two assumptions are related to this submodel 1 homoscedastic variability 2 symmetrically distributed residuals To test these assumptions we applied the following techniques 1 histogram for distributions of WRES 2 histogram for individual distribution of WRES 3 scatterplot of WRES versus PRED to check the shape of residual 4 scatterplot of WRES versus PRED conditioned on covariates to screen the covariate effects 5 autocorrelation of WRES 5 5 8 Covariate model Parameters ETA and WRES are of great use to help screen proper covariates We can utilize the following methods to check covariate models 1 scatter plot for parameters versus covariates ETAs versus covariates WRES versus covariates 2 scatterplot matrix of covariates 15 5 6 Model validation Resampling methods has been extensively employed in the model validation Currently
19. e plot for each individual 13 EE O XM 060m Project Configure Data management Exploratory data analysis PK Models Model validation Model comparison Interactive diagnostics BOX E amp t B A open preferences subset clear save help quit Current directory C Projects Phd pkgraph 1130 1 pkdata items 4 a ID 4 TIME 4 CONC 4 IPRE 4 DOSE 4 CL 4V 4 ETAL 4 ETA2 AGE 4 WI ral al pdf 1 an e eee 92204 24822000 3821200 bigVar pdf 923804 34 823000 38 212001 boot included individualsl csv General Configure 23804 34 823000 38 21200 boot raw resultsl csv E TIME TIME 23804 34 823000 38 21200 bootl gt PRE IPRE 23804 34 823000 38 21200 boot2 23804 34 823000 38 21200 ica No match lz a PARAMETERS Ea apaan c3 s3 1 pdf a PARAMETERS era ETA 23804 34 823000 38 21200 23804 34 823000 38 21200 23804 34 823000 38 21200 23804 34 823000 38 21200t 23804 34 823000 38 21200 23804 34 823000 38 21200 23804 34 823000 38 21200 23804 34 823000 38 21200 23804 34 823000 38 21200 5570 32 765000 74 83800 T com ANY 7cch nn 774 0 0nnm b cdd_raw_resultsl csv ETA AGE cov cdd skipped individualsl csv No match ISM No match cdd2 cdd3 CS1_IVLESTFPDF fit CS1 IVIESTFPDF fit2 datal csv data2 csv dd3 f f2 No match CONC1 No match PRED RES E E E E E El E El II Data types are configured successfully Figure 8
20. es from two data sets These matching variables are generally the variables from original data sets and they are not related to model fitting For example we have to match TIME ID DV WT etc from original data but not match those variables from model fit such as ETA RES WRES etc When all parameters are set the program offers three choices for comparison his togram comparison distribution comparison scatter plot comparison and transform comparison 18 Name Date modified Type Je MM runi 2 4 2010 10 44 AM File Folder di MM run2 2 4 2010 10 44 AM File Folder de MM run3 2 4 2010 10 44 AM File Folder de NM run4 2 4 2010 10 45 AM File Folder de MM run5 2 4 2010 10 45 AM File Folder Je MM run amp 2 4 2010 10 45 AM File Folder de NM run7 2 4 2010 10 45 AM File Folder de MM run amp 2 4 2010 10 45 AM File Folder de NM_run9 2 4 2010 10 45 AM File Folder de MM run10 2 4 2010 10 44 AM File Folder Figure 12 Multiple NONMEM runs for case deletion diagnostics 5 7 1 Select datasets This function is to select datasets available in the PKgraph data area Figure 18 shows there are three data sets available including fit result 2 2 CS1 IVIESTFPDF fit fit with additive error model and fit result 3 3 CS1 IVIESTFPDF fit2 pro portional error model In this example we will compare these two models 5 7 2 Configure mapping This step will join two fit results As a result user
21. f 1 250000 84850 cdd raw resultsl csv 1 35000000 16437 cdd skipped individualsl csv 1 4000000 10 724 cdd2 1 6 000000 8 7352 cdd3 1 8 000000 7 6970 CS1 IVIESTFPDF fit 1 12 000000 4 4790 CS1 IVIESTFPDF fit2 1 16 000000 2 4183 Figure 5 Menu items in Data management 5 4 1 Univariate When clicking this menu item users will generate a graph interface Figure 2 In this interface users can specify all parameters in the left area of window In the right area of window it has five buttons on the top explained in section Graph interface 5 4 2 Bivariate This menu item also generates a graph interface It is similar to the Univariate interface except that users will have two variables instead of one 5 4 3 Parallel coordinate plots This menu item provides access to parallel function from lattice package 5 4 4 Scatterplot matrix This menu item provides access to splom function from lattice package 5 5 PK models This menu item is utilized to check model assumptions and goodness of fit The guideline follows Census menu http census sourceforge net It has the eight functions Figure 7 Configure model result is required for the 11 PK Models 26 gy n Univariates open preferences subset clear sav Bivariates Current directory C Projects Phd pkgr AA Scatterplot matrix items 4 ila ID 4 TIME 4 CONC al pdf 1 0 00000 0 0000 bigVar pdf 1 0 250000 13 026 boot included individ
22. i for interactive diagnostics by clicking second button in the tool bar area on the top right panel All variable names for model 1 will have additional x label and all vari able names for model 2 will have additional y label Let us look at histogram comparison as one example First we need to make sure that current data set is 4 ModelComparison Figure 20 second we click histogram comparison The result is shown in Figure 21 for comparing CL 5 8 Interactive graphics This functional module incorporates a unique feature interactive graphics into every step of model diagnostics It targets to link diverse data sets in one inte erative platform Users can have access to this feature through ggobi button in the graph interface In addition users have flexibility to apply this feature to achieve their specific goals In the toolbar there is option interactive graph ics designed for this purpose It includes three steps select datasets configure mapping and diagnostics By linking diverse data sets with a key variable users can seek patterns by brushing linking and diagnosing patterns conveniently 20 Rs X c Bootstrap summary PsN ree x ix PsN result file boot raw resultsl csv Note 1 raw results1 csv default file Bootstrap results seETA 2 Bootstrap results seERRI SS Bootstrap results seERR2 Bootstrap key file boot_included_individualsl csv v Note
23. ll ggplot2 install packages ggplot2 3 PKgraph infrastructure The software incorporates a key concept interactive graphics to link various datasets and diagnostics plots The framework is programmed using RGtk2 and consists of main formats of interfaces 1 main containing links to all parts of the software and handles the basic data management and links to diagnostic modules and 2 graph which provides tools specifically for each diagnostic module 3 1 Graphical user interfaces 3 1 1 Main interface The main interface Figure 1 of PKgraph provide the links to all components of the software There are four areas 1 tool area tool bar and menu bar top 2 directory area middle left 3 data area middle right and 4 status bar bottom e The tool area has menu items linking to the basic management modules project configuration data management and the diagnostic modules exploratory data analysis PK models model validation model compari son and interactive diagnostics These are menu items containing numer ous functions associated with each of the different types of diagnostics e The directory area shows current directory and all of its files These files might be data files or code depending on the modeling software used e Clicking on any of the data files will open them and display them in the data area 3 Choosing the file also brings up a panel allowing for different formats to be read
24. luded individualsl csv 12 8 2009 2 16 PM Microsoft Office E 15 KB EN boot raw resultsl csv 12 8 2009 2 19 PM Microsoft Office E 24 KB P bootstrap R 12 8 2009 3 25PM Tinn R 8 KB Eb included individualsi csv 12 8 2009 2 16 PM Microsoft Office E 15 KB raw resultsl csv 12 8 2009 2 19 PM Microsoft Office E 24 KB Name Date modified Type X NM runi 2 4 2010 10 44 AM File Folder de NM_run2 2 4 2010 10 44 AM File Folder Jk NM run3 2 4 2010 10 44 AM File Folder di NM run4 2 4 2010 10 45AM File Folder de NM_run5 2 4 201010 45AM File Folder NM_run6 2 4 201010 45AM File Folder Jk NM run7 2 4 2010 10 45 AM File Folder Je NM run amp 2 4 2010 10 45 AM File Folder Y NM_run9 2 4 201010 45AM File Folder J NM runi 2 4 2010 10 44 AM File Folder Figure 15 Multiple NONMEM runs for bootstrap gt library PKgraph gt data pkdata gt PKgraph Figure 22 demonstrates how to load default data in the software To further explore data first we choose Bivariates from Exploratory Data Analysis located at menu bar to check the scatter plots of interested variables Figure 23 Figure 24 The option cond from the functional model interface helps user to draw conditional plots to seek patterns for subgroups Certainly users can also select geplot2 graphic package with different taste of figure Next we can take advantage of interactive techniques to look at maximum concentration by clicking second image bu
25. ly generate diagnosing results After we match data variables to the default names we can proceed to automatically generate rou tine goodness of fit plots for interested models Figure 25 is one of the results for structural model diagnostics To further look at the influential cases from same data set we can link them together by model validation option in menu bar In this process we have 100 NONMEM runs available at directory C Projects ymodelfit dirl using PsN function cdd Let s input the path of these NONM runs and select plot variable as CL After clicking OK we will have the parallel coordinates plot showing the CL variables for all NONMEM runs From Figure 26 we can see some patients have more influential effects on CL when records from these patients are deleted Let s identify these influential cases with interactive graphics Figure 27 clearly demonstrates that these influential cases come from patient 52 and 20 based on multidimensional scaling and parallel coordinate plots In addition we compare additive error model 2 CS1 IVIESTFPDF fit with proportional error model 3 CS1 IVIESTFPDF fit2 by model comparison function in the menu bar By comparing the distribution of two models Figure 28 does not find significant difference between two models for CL In addition using gender as a conditional variable we found first model always gave a higher peak value for both male and female 29
26. model comparison It can be used with a variety of modeling fitting software including NONMEM Monolix SAS and R PKgraph is programmed in R and uses the R packages lattice ggplot2 for static graph ics and rggobi for interactive graphics This R package is supported with a user friendly graphical user interface so that users can easily control diagnosing with simple clicks The PKgraph software serves as a supplement to the existing packages NONMEM Xpose and PsN for diagnosing models PKeraph is an R packaged built on the following R packages RGtk2 gWidgets gWidgetsRGtk2 lattice and ggplot2 It requires R gt 2 0 and GTK and runs under Windows Linux and Mac 2 Installation PKeraph needs to install the following programs and R packages 1 install GTK For Windows you can download the GTK Developer s Pack from http gladewin32 sourceforge net For Unix you can fetch the source files for the different libraries from ftp ftp gtk org pub gtk v2 8 2 Install RGtk2 Please see RGtk2 Installation notes if you have problems install packages RGtk2 3 install rggobi a Download and install ggobi www ggobi org b Install rggobi install packages rggobi 4 Install gWidgets install packages g Widgets 9 Install cairoDevice install packages cairo Device 6 Install gWidgetsRGtk2 install packages gWidgetsRGtk2 7 Install lattice install packages lattice 8 Insta
27. multiple NONMEM run form PsN directly Figure 12 and find file directory for these runs Then we can select parameters as shown in Figure 13 These parameters include e Target directory path the path for multiple NONMEM runs It is a required parameter e Simulation folder pattern the common folder name for multiple NON MEM runs For this example it is NM_run It is a required parameter e NONMEM result file name the file name for NONEM fitted result In each NONMEM run there should be a file with this name It is required parameter e Patient ID the ID for each subject It is a required parameter e Plot variable the variable you use to detect difference among patients For this example we choose CL It is a required parameter 16 PKgraph R Pkarep 4 S o il Project Configure Data management Exploratory data analysis PK Models Model comparison E 26 ee A mal Hi sel Influence analysis summary PsN open preferences subset clear save help quit Visualization for influence analysis Bootstrap summary PsN Current directory C Projects Phd pkgraph 1130 1l pkdata Visualization for bootstrap items 4 ha ID TIME 4 CONC 4 IPRE 4 DOSE 4 CL V al pdf 1 0 000000 0 000000 15 583000 100 0 529830 6 4 bigVar pdf 1 0 250000 13 026000 15 265000 100 0 529830 6 4 boot included individualsl csv 1 0 500000 14 984000 14 953000 100 0 529830 64 boot raw resultsl csv 1 0 750000 14 160000 14 648000 100 0
28. ns to setup the data format start line and separation symbol e Save a file save a file e Save a workspace save a workspace for later usage It generally saves a group of lists for configuration and related data e Clean data clean all loaded data e Restore old workspace restore the workspace from the data and list you saved from previous step e Frit exit from PKgraph Ro GER ee Configure Data management Exploratory data analysis PK Models m Open data A El iH ell E Save data clear save help quit fel Save workspace Phd pkgraph 1130 1_pkdata 4 Clean data 4 4 5 Restore old workspace 1 a ID TIME CONC d Exit 1 0 000000 0 0000 igVar p 1 0 250000 13 026 boot included individualsl csv 1 0 500000 14 984 boot raw resultsl csv 1 0 750000 14 160 bootl 1 1 000000 19 316 boot2 1 1 500000 13 146 bugReport bd 1 2 000000 12 921 c3_s3_l pdf 1 2 500000 8 4850 cdd raw resultsl csv 1 3 000000 16 437 cdd skipped individualsl csv 1 4000000 10 724 cdd2 1 6 000000 8 7352 cdd3 1 8 000000 7 6970 CS1 IVIESTFPDF fit 1 12 000000 4 4790 CS1_IVLESTFPDF fit2 1 16 000000 2 4183 1 nnnnnn A TEL Figure 3 Menu items in Project 5 2 Configure This menu itme is utilized to configure PKgraph It has the following functions Figure 4 e Set data type set the ID TIME DV variables for current PK data This configuration is used for integrative graphics to draw a time series plot automaticall
29. nsole R File Edit View Misc Packages Windows Help R PKgraph Project Configure Data management Exploratory data analysis PK Models Model validation Model comparison Interactive diagnostics e x BE a 9m a open preferences subset clear save help quit data pkdata PKgraph gt Current directory C Projects Phd pkgraph 1130 items GE R Open al pdf Configure eee Choose file types boot included individualsl csv boot raw resultsl csv E eon bootl Data start from line boot2 Data separated by la PE A Data has column names TRUE y cdd_raw_resultsl csv cdd skipped individualsl csv Data from R environment z cdd2 cdd3 CS1 IVIESTFPDF fit CS1_IVLESTFPDF fit2 datal csv data2 csv dd3 f2 Working directory is setup successfully Figure 22 Load default data from Open dialog After loading data with data pkdata users can select pkdata from Data from R environment in the Open dialog Zt File Edit View Misc Packages Windows Help er G R GGobi E gt EACEA R tmp data Scatterplot current N arm FE tmp data Scatterplot File Display View Interaction Tools Help File Options Brush File Options Brush o conc 2 m x y conc 3 Choose color amp glyph M Ip 5 j x xj 1D 15235 2 A ID 5 Persistent E Bersisten Ld mer 35s Point brushing E 4 Color and glyph x X
30. ph we use a lot of popular arguments from R graphics Here is the explanation For those who want to know more about these parameters please check R manual 4 4 Abbreviations in the software The abbreviated variables used in the software are listed as Table 2 Abbreviated terms main xlab ylab type layout x layout y cond loess lowess Description main title of the plot It is the argument in R functions label of the x y axis It is the argument in R functions what type of plot should be drawn the number of columns in a multi panel display the number of rows in a multi panel display conditional variable locally weighted scatterplot smoothing Table 1 Basic graphical parameters Abbreviated terms ID TIME CONC PRED RES WRES IPRED IWRES COV DV IDV Description Patient ID Time after dose Observed concentration of drug in the body Population predicted concentration Residual Weighted residual individual predicted concentration Individual weighted residual Covariates Dependent variables Observed concentration Independent variables Usually time Table 2 Abbreviated terms 5 Menu items in main interface In this section I will go through each function in the menu items of toolbar 5 1 Project This menu item is in charge of input output and save data It has the following functions Figure 3 e Open data open modeling fit result from NONMEM Monolix SAS R or other software It has optio
31. rst plot is a time series plot for this data observed concentration versus time and the second plot is the current plot from plot area These two plots are linked by patient ID A specific feature of interactive graphics is to explore data by brushing In the ggobi users can use ctrl b for brush ing data to link two plots For those who would like to use more advanced features of interactive graphics ggobi manual is a good resource http www ggobi org display subset selection from ggobi this button helps user to visualize and analyze the brushed data from the previous step open plots for interactive graphics The brushed data is shown as a new dialog gave subset selection from ggobi this buttons save all brushed data from previous step open plots for interactive graphics display subset selection from ggobi close ggobi close all related ggobi instances e The plot area displays the figure and multiple figures if more than one are created 3 2 Functional module Functional module matches the menu items in PKgraph toolbar It includes the following menu itmes e Project e Configure e Data management e Exploratory data analysis IR Univarate plot l mE a ee quHEGX 2 CONC 1 A A r Univerate plot AN 1 number of bins 3 ain I lab l ylab pe percent General Configure I gt d i 1 l l yout_x ayouty 1 l I 2 graphics 0 la
32. s have to combine them as one single file including all the interested variables An sample data from NONMEM gt library PKgraph gt data pkdata 4 1 2 NONMEM folders For model validation PKgraph accepts two kinds of model validation data 1 results from PsN 2 results from multiple NONMEM runs For the first type of data PsN has the following functions bootstrap case deletion and stochastic simulation PKgraph provides the graphic ability to visualize the final results from PsN For the second type of data PKgraph can handle multiple NONMEM run folders and extract useful information to visualize Please see details in Model validation 4 2 Diagnose model There are eight function menu items in the main interface Each matches a functional module They can be considered as two categories basic and di agnostic modules The basic module includes Project Configure Data management menu items and the diagnostic module includes Exploratory data analysis PK model Model validation Model comparison Interac tive graphics menu items The five menu items in the diagnostic module can be utilized separately The basic module is utilized to read in data configure data and manage data The diagnostic module aims to test assumptions of population pharmacokinetic models Please see Menu items in main interface section for details 4 3 Basic graphical parameters In PKgra
33. s have to match the original data variables between two fit results For example Figure 19 e Matching variables ID Time Concentration WT AGE etc must be matched in this step These variables do not change with different models e Non matching variables RES PRED WRES etc are fit results and should NOT be matched These variables change with different models After mapping a new dataset joining two fit results will show in data area of main interface 5 7 3 Comparison histogram comparison enables to compare distributions of matching param eters from two models scatter plot comparison provides an environment to 19 Visualization for influence analysis E Ez 2 a Visualization for NONMEM runs B z A 1 Grouped by case deletion run ID Parallel coordinate plot Metric MDS 4 Target directory path jackknife_dirl modelfit_dirl Simulation folder pattern NM run NONMEM result file name CS1 IVIESTFPDF 1 fit Patient ID ID Plot variable cL a General Configure graphics lattice 9 ggplot2 Command area Coordinate 2 0 00 0 02 Coordinate 1 Figure 13 Parameters and results for case deletion diagnostics compare matching parameters by scatter plot transform comparison trans forms data by ratio or log ratio in order to visualize the difference between variables from two models All these models can be linked directly to ggob
34. thus handling all possible modeling software formats The data files might contain raw data and model diagnostics such as parameter estimates fitted values and residuals and these are displayed in the table view of the data area e The stats bar displays the progress of the different functions for example here it says Data is loaded successfully to indicate that there were no problems with opening the data file Project Configure Data management Exploratory data analysis PK Models Model validation Model comparison Interactive diagnostics AAA open preferences subset clear save help quit Ea pres EA pesas Ea arena rem Rm Current directory C Projects Phd pkgraph 11 I 1 pkdata 1 ID 4 TIME 4 CONC 4 IPRE 4 DOSE CL 3 4 ETAL 4 ETA2 4 AGE 4 WT 1 0 000000 0 000000 15 583000 100 0 529830 6 417200 0 227210 0 023804 34 823000 38212000 0 250000 13 026000 15 265000 100 0 529830 6 417200 0 227210 0 023804 34 823000 38 21200 0 500000 14 984000 14 953000 100 0 529830 6 417200 0 227210 0 023804 34 823000 38 21200 0 750000 14 160000 14 648000 100 0 529830 6 417200 0 227210 0 023804 34 823000 38 21200 1 000000 19 316000 14 348000 100 0 529830 6 417200 0 227210 0 023804 34 823000 38 21200 a o d wu a bigVar pdf boot included individualsl csv m m m A AA A A A 8 X o boot_raw_resultsl csv bootl boo 1 500000 13 146000 13 768000 100 0 529830 64
35. ttice ggplot2 l Cancel Clean Figures OK id Command area i 0 50 100 m I Figure 2 Graph interface of PKgraph e PK models e Model validation e Model comparison e Interactive graphics In the next sections I will go through each menu item in detail 4 Quick start PKgraph targets audiences working in population pharmacokinetics models and particularly those professionals who have only basic knowledge of R 4 1 Input data 4 1 1 data frame PKeraph accepts one type of input data data frame It can be model fit re sults from NONMEM Monolix SAS or R This data frame should include ID time observed concentration individual predicated concentration population predicted concentration residuals weighted residuals parameters random ef fects etc Details are as follows Note Abbreviated terms are explained in Table 2 e Exploratory data analysis ID Time DV e PK models See details in Table 4 e Model comparison ID Time DV and interested variables from fit results such as WRES IPRE etc For this module two data frames come from two model fit results are required Dependent on the modeling software users need to convert the model fit results to this single data frame After that PKgraph can read in and diagnose the model fit results For NONMEM the ab file can be considered as this single data frame and read in R directly For Monolix there are a few output files and user
36. tton on the right panel This will start ggobi and load related data GGobi includes two windows console window and plot window In order to link figures together users need to open all interested figures by Display option in the menu bar The following figure clearly shows that max imum concentration comes from male patients value 1 To look at these data in detail we go back to the figure graphical user interface and click third image button to check selected data set in ggobi The selected data set pops up and links to patient with ID 55 We repeat the same procedure for other variables to check patterns Next we utilize PK model option to check model assumptions and diagnose model fitting The program provides default names such as ID TIME COV 22 Visualization for bootstrap i x Target directory path aph bootstrap modelfit_dirl Resampling design grouped by bootstrap run ID variability of parameters for ordered ID Bootstrap fokdei pelles NM run grouped by bootstrap run ID NONMEM result file name CS1 IV ESTFPDF 1 fit Bootstrap key table path ects Phd pkgraph bootstrap Bootstrap key table name included_individualsl csv Patient ID ID Plot variable CL General Configure Density graphics lattice ggplot2 Cancel Clean Figures oK Command area Figure 16 Parameters and results for bootstrap visualization etc in order to automatical
37. ualsl csv 1 0 500000 14 984 boot raw resultsl csv lE 1 0 750000 14 160 bootl 1 1 000000 19 316 boot2 1 1 500000 13 146 bugReport txt 1 2 000000 12 921 c3 s3 1 pdf 1 250000 84850 cdd raw resultsl csv 1 3 000000 16 437 cdd skipped individualsl csv 1 4000000 10 724 cdd2 1 6 000000 8 7352 cdd3 1 8 000000 7 6970 CS1 IVIESTFPDF fit 1 12 000000 4 4790 CS1_IVLESTFPDF fit2 1 16 000000 2 4183 Figure 6 Menu items in Exploratory data analysis other seven functions Users have to configure data variable first before going to specific model diagnostics 5 5 1 Configure model result This is the key step to match data variables to default metric system By this step fit results from any platform NONMEM Monolix SAS R can be inter preted graphically in figures Package variable Description ID Patient ID TIME Time after dose DV Dependent variables Observed concentration IDV Independent variables PRED Population predicted concentration RES Residual WRES Weighted residual IPRED individual predicted concentration IWRES Individual weighted residual COV Covariates Table 3 Package metric system 12 Ru NN Project Configure Data management Exploratory data analysis ESTEE Model validation E 26 gy 2 El ien sel Configure model result open preferences subset clear save help quit Individual plots Current directory C Projects Phd pkgraph 1130 1_pkdata c ra Parameters 12 000000 4 479
38. y e Set working directory change current working directory in R e Set saving format set up saving format for figures including pdf jpg tiff png bmp win metafile and figure width and height If figure width and height is not configured the default one will be used Note to save figure in graph interface users need to configure this menu first e Set figure configuration color and loess can be selected for figures 5 3 Data management This menu item is utilized to manage data It has the following functions Figure 5 e Subset subset current data After this a new subset data will generate in the data area of main interface And it will be the current working data for the following diagnosis If users do not want to work on this Ro Na Project HE 000 Data management Exploratory data analysis PK Models e Set data type El a ell open Set working directory save help quit Set saving format Currer Set figure configuration 4 pkgraph 1130 1_pkdata items 4 2 ID 4 TIME 4 CONC al pdf 1 0 000000 0 0000 bigVar pdf 1 0 250000 13 026 boot included individualsl csv 1 0 500000 14 984 boot raw resultsl csv 1 0 750000 14160 bootl 1 1 000000 19 316 boot2 1 1 500000 13 146 bugReport txt 1 2 000000 12 921 c3 s3 1 pdf 1 250000 84850 cdd raw resultsl csv 1 3 000000 16 437 cdd skipped individualsl csv 1 24000000 10 724 cdd2 1 6 000000 8 7352 cdd3 1 8 000000 7 6970 CS1 IVIESTFPDF fit 1 12 000000 4 4790
Download Pdf Manuals
Related Search
Related Contents
EMS User`s Manual 3300 Installation Manual InLine 59943N cable tie ADC−21XⅡ Maas AHT-28-V/AHT-78-U Handfunkgeräte REMKO RKS 326 H – 371 H W and OPERATOR`S MANUAL MANUEL de L`UTILISATEUR MANUAL del chapter 1 introduction to esa 86/88e CyberPower PDU15SWV16FNET power distribution unit PDU Bedienungsanleitung Operating instructions Copyright © All rights reserved.
Failed to retrieve file