DAME Web Application REsource Plugin Creator User Manual
Contents
1. Fields _ Directory defined at runtime Parameter name er Type Inte gi io Expression _ Addto the first configuration file _ Add to the second configuration file Cancel OK Fig 13 window to add a constant field either as command line parameter or configuration file entry 12 DAMEWARE Plugin Creator User Manual This document contains proprietary information of DAME project Board All Rights Reserved DAta Mining amp Exploration Program 3 6 2 Edit sort and remove a constant field After creation of a new constant parameter it can be modified by choosing Edit field in the panel of Fig 11 Of course in a command line it is important the order in which the parameters are listed otherwise the execution will crash So far to correctly locate the constant parameter position in the command line the buttons Up and Down of Fig 11 can be used Finally the option Delete field in Fig 11 allows user to delete a constant parameter 3 7 Final Steps of the setup procedure At the end of the setup procedure by having configured input and output files constant and parameters of command lines for all the foreseen use cases of the model to be plugged in the panel shown in Fig 14 asks for the final name of the executable program to be launched if the plugin will be selected for an experiment through the DAMEWARE web app GUI Name of file to execute Fig 14 final panel2. Editned narameter2 Version 1 0 Delete fied parameter3 Here you can specify additional field information in terms of command line up parameters as required by the model executable to be plugged in Down You can also insert only the constants for Train use case Press Edit button if you want to edit jan already inserted constant Use l delete button to remove a constant Buttons Up and Down help you to change the original insertion order of parameters If one or more defined parameters ishould be inserted in a configuration file use choose template to load a Ibemplate to process it First Configuration File Template none up Choose Template down pa rameter 4 il Save lt Back Next gt Fig 12 Example Train use case of additional fields and sorting setup panel Configuration Files case If one or two configuration files for each of it you can choose a Template if available to create the configuration file using button Choose Template 3 6 1 Adda constant field By selecting Add new field in the panel of Fig 11 it is possible to add a new command line constant parameter Fig 13 This can be done by describing its name identifier of the parameter type and the expression If the model provides to pass the name of an output directory defined at runtime you can use checkbox Directory defined at runtime
3. of additional fields and sorting setup panel Configuration Files case 12 Fig 13 window to add a constant field either as command line parameter or configuration file entry 12 cile een 13 Fig 15 example of message notifying the successful creation of the compressed plugin package 13 2 DAMEWARE Plugin Creator User Manual This document contains proprietary information of DAME project Board All Rights Reserved DAta Mining amp Exploration Program 1 Introduction he present document is the user guide of the DAMEWARE plugin creator designed to make able end users of the data mining web app to easily integrate new machine learning models within the DAMEWARE infrastructure without any specific knowledge about its internal architecture This manual together with the plugin Java program DAMEWAREPluginWizard jar is part of the web app documentation package and it is available at the address http dame dsf unina it dameware html plugin The plugin creator through a guided procedure allows user to configure all machine learning model parameters use cases and I O information setup Then it generates source code to be physically included into the infrastructure and to be finally exposed as new machine learning resource available to the community as shown in Fig 1Fig 2 Other Services RESOURCE MANAGER Experiment Setup Workspace 4su
4. The Save button pressed here will save all the performed steps of the plugin creation procedure You can press Finish button only after compiling Name of file to execute field to generate the final plugin code to be integrated within the web app After the Finish button is clicked the procedure will create all internal features to be physically integrated into the DAMEWARE web app infrastructure The Finish button will activate a new panel asking for the directory where to store all created files of the plugin package The Fig 15 reports the disclaimer which informs about the compressed archive successful creation i File Classification AutoMLPONA zip created in C7 Fig 15 example of message notifying the successful creation of the compressed plugin package 3 8 Delivery of the plugin creator At the end of the guided procedure for instance when the user selects the button Finish of Fig 14 the plugin wizard produces a compressed package zip called lt functionality gt _ lt model acronym gt zip containing 13 DAMEWARE Plugin Creator User Manual This document contains proprietary information of DAME project Board All Rights Reserved DAta Mining amp Exploration Program A Java file named lt functionality gt _ lt model acronym gt java A Java file named lt model acronym gt java An XML file named lt functionality gt _ lt model acronym gt xml An XML file named Configuration
5. the parameter will be write in a configuration file considered as the first one in case of the presence of two configuration files e Add to the second configuration file if this option is selected the value of the parameter will be write in a second of two foreseen configuration files If none of the two configuration file adding has been selected the created parameter is by default considered in the executable command line list In this case after pressing OK button a popup disclaimer confirming this will appear DAMEWARE Plugin Creator User Manual This document contains proprietary information of DAME project Board All Rights Reserved DAta Mining amp Exploration Program Train Input Fields information Add new Field Train Add a new Field Edit Field Train New Field Delete Field Up Visualized labe arameter 1 ma Ip jeter Description this is the tooltip text Type Double is Optional Default Value 0 05 Constrain RANGE Range description tooltip text for the range left bound right bound Lucci 0 01 1 0 gt tooltip text for the range __ Add to the first configuration file Add to the second configuration file Fig 7 Example Train use case panel for new parameter field editing At the end press OK button to save the parameter configura
6. Command line constant parameters Setup The Additional fields and sorting panel shown in Fig 11 as example for the Train use case allows user to specify additional field information in terms of command line constant parameters or configuration files constant parameters if previously created see Fig 12 as requested by the model executable to be plugged in Command line fields and sorting input dataset parameter 1 __ parameter 2 Sees 3 Here you can specify additional field information in terms of command line Up L parameters as required by the model executable to be plugged in Nou can also insert only the constants for Train use case Press Edit button if you want to edit jan already inserted constant Use delete button to remove Buttons Up and Down help you to ichange the original insertion order of parameters If one or more defined parameters should be inserted in a configuration ifile use choose template to load a template to process it Fig 11 Example Train use case of additional fields and sorting setup panel Command Line case 11 DAMEWARE Plugin Creator User Manual This document contains proprietary information of DAME project Board All Rights Reserved DAta Mining amp Exploration Program Train Command line fields and sorting Add new field linput dataset _ _ parameter 1
7. Creator User Manual This document contains proprietary information of DAME project Board All Rights Reserved DAta Mining amp Exploration Program o Name this field specifies the name of the parameter used as identifier for the produced image Example X 1 Y 2 name plotting1 Then the internal identifier of the image will be named plottingImagel o List of images to create a field where to insert the list of all the images to be generated from a specified output file describing the images e Append to Log File it specify if the file content should also be included in the log file e Generate confusion matrix in output the output values of the experiment table of data will be also saved as a confusion matrix only in case of classification type experiments e Join this file with another output file Join it with columns from another output file If this checkbox 1s selected you can choose the output file to join e Join this file with an input file if selected the user must enter the type of the input file that will be merged with the output file The resulting output file will contain joined input and output columns 3 5 2 Edit and remove an output file By selecting Edit Output File button in the panel of Fig 10 it is possible to modify the saved file configuration By selecting Delete Output File button in the panel of Fig 10 it is possible to delete the saved configuration for an output file 3 6
8. DAta Mining amp Exploration Program fd AE om E O Dipartimento di Scienze Fisiche 8 i smuronazionneaiasnorisice i CALTECH Universita d Napol Federico H Srna AN x A SSERVATORIO ASTRONOMICO di CAPODIMONTE x DL DAME Web Application REsource Plugin Creator User Manual DAMEWARE MAN NA 0016 Issue 2 0 Date September 03 2013 Authors S Cavuoti A Nocella S Riccardi M Brescia Doc ModelPlugin_UserManual DAMEWARE MAN NA 0016 Rel2 0 DAMEWARE Plugin Creator User Manual This document contains proprietary information of DAME project Board All Rights Reserved DAta Mining amp Exploration Program Index PVR OCC 10 Rizla 3 2 System Reguirements and Installation 3 SS Taye a nizan UL 4 3 1 General Pie mi In ormat inizia 5 VIII tic EEA I 5 3 3 Input PA AS Pizia ISO I I pini 6 3 3 1 New parameter SC 6 RA 6 3 3 2 Edit sort and remove input parameter Setup ii 7 SI EER e E E 7 3 4 1 Add new input ERRE I 8 3 4 2 Edit sort and remove an input file Li 8 3 5 ROMO HE Pe RR RR O RR OO TOT 9 3 5 1 Add new gutput MUGS acs cs sncanecssnscateassacanepsennecasacainntestanimesecguanastoasaaadicaeseennt O ae aaae TICO ITC IO iniae sarii besiis 10 3 5 2 Edit and remove an output file ii 1 3 6 Command line constant parameters Setup 1 3 6 1 POC a constant feld sprint di RESET EI a
9. File xml that tracks all the setup work done v Vv v Vv The user must deliver the zip file together with the source code of the model to be recompiled on the worker nodes of DAME computing platform to the DAMEWARE administrators helpdame gmail com asking to complete the model plugin integration task Then the user will be informed about the availability of the new model in the new release of DAMEWARE web app 14 DAMEWARE Plugin Creator User Manual This document contains proprietary information of DAME project Board All Rights Reserved DAta Mining amp Exploration Program 4 Appendix References and Acronyms Abbreviations amp Acronyms A amp A Al ANN ARFF ASCII Bok BP BLL CE CSOM CSV DAL DAME DAMEW ARE DAPL DL DM DMM DMS FITS FL FW GRID GSOM GUI HW Meaning Artificial Intelligence Artificial Neural Network Attribute Relation File Format American Standard Code for Information Interchange Base of Knowledge Back Propagation Business Logic Layer Cross Entropy Clustering SOM Comma Separated Values Data Access Layer DAta Mining amp Exploration DAME Web Application REsource Data Access amp Process Layer Data Layer Data Mining Data Mining Model Data Mining Suite Flexible Image Transport System Frontend Layer FrameW ork Global Resource Information Database Gated SOM Graphical User Interface Hardware A amp A KDD IEEE INAF JPEG LAR
10. MDS MLC MLP MSE NN OAC PC PI REDB RIA SDSS SL SOFM SOM SW UI URI VO XML Meaning Knowledge Discovery in Databases Institute of Electrical and Electronic Engineers Istituto Nazionale di Astrofisica Joint Photographic Experts Group Layered Application Architecture Massive Data Sets Multi Layer Clustering Multi Layer Perceptron Mean Square Error Neural Network Osservatorio Astronomico di Capodimonte Personal Computer Principal Investigator Registry amp Database Rich Internet Application Sloan Digital Sky Survey Service Layer Self Organizing Feature Map Self Organizing Map Software User Interface Uniform Resource Indicator Virtual Observatory eXtensible Markup Language Tab 1 Abbreviations and acronyms 15 DAMEWARE Plugin Creator User Manual This document contains proprietary information of DAME project Board All Rights Reserved DAta Mining amp Exploration Program 000 IRAROAEN 16 DAMEWARE Plugin Creator User Manual This document contains proprietary information of DAME project Board All Rights Reserved
11. al model information panel e Model name the name of the model to be plugged in It is strongly suggested to use an acronym of the model for example MLP for Multi Layer Perceptron Spaces are not allowed here use underscore instead if required e Documentation the URI address where to find model related documentation if absent any reference can be specified as well e Version current model version or release e Functionality the functional domain of experiments to be associated to the plugged model for instance classification or regression for supervised models clustering or feature_extraction for unsupervised The user should be aware about the feasible domain for the model e Owner Name the user name e Owner e mail a correct e mail address of the user to be contacted by DAMEWARE administrator At the end click Next button to proceed 3 2 Use Case Specifications As common practice in the machine learning domain any experiment requires a series of phases use cases to be performed in sequence For instance at the beginning of any experiment a training phase is required to train the model either in supervised and unsupervised cases The test phase is usually the second step sometimes also named as validation amp test phase The last step is the run phase in which the trained and tested model is used as a generic function applied to new data of the focused problem There is also the full step
12. automatically performing both train and test phases considered as a unique workflow Running Modes Information Train Documentation hitpJ game dstunina i Test Documentation ion URI The field running priority can be left lempty available on next releases Fig 5 use case setup panel the asterisk means mandatory option DAMEWARE Plugin Creator User Manual This document contains proprietary information of DAME project Board All Rights Reserved DAta Mining amp Exploration Program Fig 5 shows the use case setup panel in which the user can choose the steps available for the plugged model usually a canonical machine learning includes all of them by also specifying referenced documentation if any If train and test cases checkboxes are selected the full case starting disabled will become available The Next button becomes available when at least one running mode is selected In the figure above only the Train running mode is selected as an example 3 3 Input parameter Setup In Fig 6 the main input data panel is shown appearing after pressing Next button from the previous panel It allows user to enter information about the parameters required by the model for each of the previously selected use cases The figure reports an example for the Train use case it is the same in other use cases Train Input Fields informati
13. d formats But the model usually allows only a particular format as input So far the web app will perform an embedded conversion to the right format as specified by the user in this field Split input columns from target columns if the model requires to separate the rows of the input file For example separate an input dataset file in Inputs and Targets using the number of input neurons At the end click OK button to save the input file configuration 3 4 2 Edit sort and remove an input file By selecting Edit Input File button in the panel of Fig 8 it is possible to modify the saved file configuration By selecting Delete Input File button in the panel of Fig 8 it is possible to delete the saved configuration for an input file Finally the buttons Up and Down in the panel of Fig 8 can be used to sort listed input files already configured as they will appear in the sequence shown in the DAMEWARE web app model configuration tab during an experiment setup process DAMEWARE Plugin Creator User Manual This document contains proprietary information of DAME project Board All Rights Reserved 3 5 Output File Setup Program DAta Mining amp Exploration Each time an experiment is completed involving one or more use cases its outcome consists in a series of output files images plots diagrams tables trained weights logs etc Through the panel shown in Fig 10 the user can configur
14. e button to i j remove an intput file sr DOWA Train Add new Input File Buttons Up and Down help you to ichange the original insertion order of i parameters Description parameter tooltip text Visualized label linput dataset _ _ is optional Convert to fits table _ Split input columns from target columns splitting parameter None _ Add to the first configuration file _ Add to the second configuration file Save lt Back Next gt Fig 8 Example Train use case of input file setup panel Add new input file By selecting New Input File it is allowed to enter the input file information Visualized label the name of the input file type label as it will appear in the DAMEWARE web app setup tab For example dataset or trained weight file Description a brief description of the file content Is optional if selected this input file field is considered not required if optional it is empty no default values will be used in this case Convert to if the model requires a particular format for this file for example the dataset should be given as a CSV file the user can choose the right format by selecting one of the allowed formats listed in the pop up menu for instance ASCII CSV CSV no_header FITS TABLE and VOTABLE NOTE the DAMEW ARE web app allows to arbitrarily submit input files in a quite wide range of supporte
15. e and insert the list of output files as foreseen to be produced for each model use case The figure reports an example in the Train use case the same for the others Only the files specified here will be exposed as output available to be downloaded in the DAMEWARE web app experiment status window see Fig 9 DAME Application User bresciamax amail com Other Services LooOut Je RESOURCE MANAGER Workspace E New Workspace Rename Workspace f primo mipqnaExp f mipexp phat f clashRegr F tx f imgt f bestarsqna f 4survey Fig 9 The DAMEWARE web app experiment status window containing list of outputs a Image Viewer er v File Manager Workspace imipqnaExp q Y My Experiments Ti Upload Op Experiment 3 Delete Workspace mipgnaExp de is sii x Experiment Status Last Access M Delte ca x d classtraini ended 2013 07 02 x d x ta Download gt Addinws File Type Description s x Go e mipana_TRAIN_error txt ASCI ASCI report on training iteration eri D x E mipana_TRAIN_output txt ASCII Training output with input dataset da x id mipana_TRAIN_trainTestGutLog t ASCII Training output vs target values wA x i 3 E mipana_TRAJM_frozen_net txt ASCi Trained network setup file a x id mipqna_TRAIN_weights txt ASCil Trained network weights file as x b classtesti b classrunt b classfullt b regrtraint b regrtesti b regrrunt b regriull
16. e steps described in the next sections The button New is used if user wants to start a plugin configuration from scratch Otherwise it can be selected the Load button to load an already configured partially or completely and saved plugin In this last case after selecting the Load option a new window will let you choose any of the XML configuration files available Fig 3 Cerca in ai DAMEWARE PLUGIN CONFIGURATION FILES ConfigurationFile alb Oggetti recenti i Desktop d Documenti A i Computer I a Nome file ilDocuments DAMEWARE PLUGIN CONFIGURATION FIL Apri Rete Tipo file xml DE Annulla Fig 3 loading XML plugin configuration file already created panel In the following we will focus on the New option starting the new plugin configuration procedure DAMEWARE Plugin Creator User Manual This document contains proprietary information of DAME project Board All Rights Reserved DAta Mining amp Exploration Program 3 1 General Plugin Information The Fig 4 shows the general plugin information to be compiled Plugin Information Model name example Version 1 0 Documentation hitp dame dsf unina it Please insert general plugin information f Version j1 0 Functionality classification Owner Information Owner Name DAME team Owner e mail nelpdame gmall com Fig 4 Gener
17. f Fig 10 the new panel shown allows the user to describe the output file information e Name the name of output file as generated by your own model code e Relative path optional relative path sub directory where output file will be created basically the experiment location used internally by the DAMEWARE web app e Description parameter tooltip text e Format the internal representation of the file it is basically to assign a right extension at the creation time for further use in the web app It must match the real type as used by your model code e Rename in All the files produced by the model will be renamed in lt plugin gt _ lt running mode gt _ lt new name gt In this text area you can choose the final name of output file as it will appear in the web app experiment status panel If this element 1s empty the output file name will be lt plugin gt _ lt running mode gt _ lt original name gt e Convert in the type to be converted the output file Unselect it if none conversion is required e Plot this file select this checkbox only if it is required a plot for the file contents If selected a pop up tab will appear asking to configure plot features Errore L origine riferimento non stata trovata o X file column index to be used as the abscissa oY file column index to be used as the ordinate Here index is the ordinal number referred to the column position in the file not the meta datum 10 DAMEWARE Plugin
18. i DAMEWARE Plugin Creator User Manual 2013 07 02 2013 07 02 2013 07 02 2013 07 02 2013 07 02 2013 07 02 2013 07 02 Xx XxX x x x x x This document contains proprietary information of DAME project Board All Rights Reserved a DAta Mining amp Exploration Program Train Output Files information Add OutputFile y n gt Train Add a new Output file Edit Output File Version 1 0 Delete Output F Train Add output file Insert here the list of output files to be i produced by Train use case Name loutput csv Use Edit button to edit an already r inserted output file Use delete button Relative path myexplexamples ito remove an output file Description hooltip text Format esv Rename in example_Train_ output_sample fits Convert in fits _ Is Optional lx Plot this file x h y 2 name plottite delete List of images to create Plottina columns 1 and 2 in example_Train_plot title jpeg _ Append to Log File _ Generate confusion matrix in output l dain th rou join this file with an Input file Input file input dataset Save lt Back Next gt L 7 Fig 10 Example Train use case of the output file setup panel 3 5 1 Add new output files By selecting Add Output File in the panel o
19. i 12 3 6 2 Edit SOrt and remove a COnNStantie dci 13 a1 Final Steps of the setip PrOC COU S iitncseccsctcetsawantansadaananesvesdantedalneaaverceniadadnisuicanduotenadnedsawentatiadadeinvadeedanuenatnesaecs 13 3 8 DeVry OF TNS pue imera erten r e a a aiai ar EEEE E Aa REE eniin een 13 4 Appendix References and AcronymsS 15 TABLE INDEX Tab 1 Abbreviations and acronyms iii 15 FIGURE INDEX Fig 1 The list left popup menu of machine learning models and related functionalities available in the DAMEWARE Weil 3 Fig 2 introduction panel of the plugin wizard tool after lAUnch iii 4 Fig 3 loading XML plugin configuration file already created panel iii 4 Fig 4 General model information panel ii 5 UO use case selup PONE maia na 5 Fig 6 General panel to build new parameter fields Train use case example M ii 6 Fig 7 Example Train use case panel for new parameter field editing iiii 7 Fig 8 Example Train use case of input file setup panel ii 6 Fig 9 The DAMEWARE web app experiment status window containing list Of QUIputs ii 9 Fig 10 Example Train use case of the output file setup panel iiii 10 Fig 11 Example Train use case of additional fields and sorting setup panel Command Line case 11 Fig 12 Example Train use case
20. om the web address specified in the introduction above Then simply double click on it or execute java jar DAMEWAREPluginWizard jar from a OS command window DAMEWARE Plugin Creator User Manual This document contains proprietary information of DAME project Board All Rights Reserved DAta Mining amp Exploration Program 3 The Java plugin wizard GUI The following sections will drive users through the plugin integration procedure In Fig 2 the intro panel is shown It appears after the execution of the wizard application Version 1 0 Welcome to the DAMEWARE Plugin Creator DAMEWARE This wizard tool will guide you along the integration of your data mining model i dai sie A linto DAMEWARE Web Application REsource of DAME DAta Mining and Exploration the ew buttor starta a plugin DAMEWARE is a Service Oriented Application for data mining configuration from scratch 4 A The Load button resumes an already on massive data sets based on machine learning existing plugin The Back and Next buttons allow to More information available here proceed through the wizard steps The Save button saves your work lwhenever you want New Load Fig 2 introduction panel of the plugin wizard tool after launch At the bottom of the intro panel there are two buttons Back and Next initially disabled useful to proceed forward and backward respectively through the procedur
21. on Add new Field Edit Field Delete Field Up Down Save Fig 6 General panel to build new parameter fields Train use case example 3 3 1 New parameter setup By selecting Add new field in the panel of Fig 6 the panel in Fig 7 allows to configure the input parameter by specifying e Visualized label the label of the parameter as it will appear in the DAMEWARE web app experiment setup tab for example input layer neurons or error threshold e Description a brief description of the parameter appearing as tooltip e Type the internal value allowed for the parameter Integer Double Boolean String In the Boolean case the true false value can be chosen e Is Optional checkbox to be selected if the parameter will be not required for an optional parameter default value will be assigned if left empty e Default Value if the is optional checkbox is selected the default value field will become enabled and the user will have to enter the desired value according to the type specified in the field Type If desired value is not according to the type specified in the field Type an Error message will be shown e Constraint if the parameter has a limited range of allowed values the user can choose one of the three options NO_CONSTRAINTS RANGE VALUES e Add to the first configuration file if this option is selected the value of
22. ryey Select a Running i a Train Experiment trial bode Selecta a F Clustering SOM Functionality sain Pane Classification FMLPGA Classification_MLP configuration file BAI Classification MLPONA Classification _SVM dataset type Clustering_CSOM Clustering_E SOM Clustering_GSOM output rows Clustering_SOM Clustering _SOM_Aute Clustering _SOM_Kmeans output dimension Clustering _SOM_TWL Clustering_SOM_UmatCC normalize data v Field is Required input file il input nodes output columns Feature_Extraction_PPS Regression_FMLPGA Regression MLP epochs Regression_MLPONA Regression_SVM neighbor size initial learning rate final jearning rate Submit Fig 1 The list left popup menu of machine learning models and related functionalities available in the DAMEWARE web application The plugin creator implicitly assumes that the user model to be plugged in is available in the form of an executable provided with a command line parameter list or with external configuration files max 2 external files 2 System Requirements and Installation The only basic requirement to download and execute the Java plugin creator on local user machine is that the Java Runtime Environment JRE is properly installed If not please download and install the JRE from http www oracle com technetwork java javase downloads index html After JRE installation download the plugin application fr
23. tion 3 3 2 Edit sort and remove input parameter setup By selecting Edit Field button in the panel of Fig 6 it is possible to modify the saved parameter configuration By selecting Delete Field button in the panel of Fig 6 it is possible to delete the saved configuration for a parameter Finally the buttons Up and Down in the panel of Fig 6 can be used to sort listed parameters already configured as they will appear in the sequence shown in the DAMEWARE web app model configuration browser tab during an experiment setup process 3 4 Input file Setup After pressing Next button the input file choice panel is shown Fig 8 Any use case of a model foresees of course to specify input files to be used by the model such as dataset trained weight matrix for a neural network external configuration files etc The panel shown reports an example of input file setup pop up window for instance in the Train use case the same for the other use cases DAMEWARE Plugin Creator User Manual This document contains proprietary information of DAME project Board All Rights Reserved 3 4 1 DAta Mining amp Exploration Program Train Input Files information New Input File Edit input File Version 1 0 Delete input File Insert the input files to be used for i Train use case DM Use Edit button to edit an input file Up S Train Add a new Input file lalready inserted Use delet
Download Pdf Manuals
Related Search