Home

Kieker 1.6 User Guide

1. gt java classpath build lib kieker 1 6 emf jar kieker examples userguide ch2bookstore manual BookstoreAnalysisStarter tmp kieker 20120402 163314855 UT C myHost KIEKER SINGLETON Listing 2 14 Commands to compile and run the analysis under UNIX like systems gt mkdir build gt javac src kieker examples userguide ch2bookstore manual x java classpath lib kieker 1 6_emf jar d build gt java classpath build lib kieker 1 6_emf jar kieker examples userguide ch2bookstore manual BookstoreAnalysisStarter C Temp kieker 20120402 163314855 UT C myHost KIEKER SINGLETON Listing 2 15 Commands to compile and run the analysis under Windows You need to make sure that the application gets the correct path from the monitoring run The TeeFilter prints an output message for each record received An example output can be found in Appendix G 1 Kieker 1 6 User Guide 18 3 Kieker Monitoring Component The Java sources of this chapter as well as a pre compiled binary can EE be found in the examples userguide ch3 4 custom components di rectory of the binary release 3 1 Monitoring Controller The MonitoringController constructs and controls a Kieker Monitoring instance As depicted by the class diagram in F igure 3 1 it provides methods for Creating IMonitoringController instances Section 8 1 1 Logging monitoring records with the configured monitoring writer Section 3 1 2
2. registerReader reader AbstractReaderPlugin registerFilter filter AbstractFilterPlugin registerRepository filter AbstractRepository connect plugin AbstractPlugin repositoryPort String repository AbstractRepository connect srcPlugin AbstractPlugin outputPort String dstPlugin AbstractPlugin inputPort String run readers Ts 1 filters repositories 1e AbstractReaderPlugin from kieker analysis plugin reader AbstractFilterPlugin from kieker analysis plugin filter AbstractRepository from kieker analysis repository AbstractReaderPlugin configuration Configuration AbstractFilterPlugin configuration Configuration AbstractRepository configuration Configuration v Vv V interface AbstractPlugin interface interface IReaderPlugin from kieker analysis plugin IFilterPlugin IRepository from kieker analysis plugin reader AbstractPlugin configuration Configuration from kieker analysis plugin filter from kieker analysis repository read boolean Vv init boolean interface IPlugin name String repositoryName String repositoryDescription String getCurrentConfiguration Configuration from kieker analysis plugin name String plugin
3. KIEKER SINGLETON Sampling Controller Periodic Sensor available Current Poolsize 0 Scheduled Tasks 0 Bookstore main Starting request 0 Bookstore main Starting request 1 Bookstore main Starting request 2 Bookstore main Starting request 3 Bookstore main Starting request 4 Apr 2 2012 6 33 14 PM kieker monitoring core controller MonitoringController 1 run INFO ShutdownHook notifies controller to initiate shutdown Listing G 1 Execution of the manually instrumented Bookstore application Section 2 3 Kieker 1 6 User Guide 84 Apr 2 2012 9 30 08 PM kieker analysis plugin reader filesystem FSDirectoryReader run INFO lt Loading tmp kieker 20120402 163314855 UT C avanhoorn thinkpad KIEKER SINGLETON kieker 20120402 163314882 UT C 000 Thread 1 dat OperationExecutionRecord 1333384394881852990 1 public void kieker examples userguide ch2bookstore manual getBook boolean lt no session id gt 1 1333384394878278653 1333384394880370678 myHost 1 1 OperationExecutionRecord 1333384394890981315 1 public void kieker examples userguide ch2bookstore manual getBook boolean lt no session id gt 1 1333384394888877486 1333384394890961341 myHost 1 1 OperationExecutionRecord 1333384394893124745 1 public void kieker examples userguide ch2bookstore manual getBook boolean lt no session id gt 1 1333384394891055138 1333384394893117202 myHo
4. gt lt include name jar file kieker gt Kieker 1 6 User Guide 87 lt fileset gt lt resourcecount gt lt condition gt lt path id classpath gt lt fileset dir lib dir includes x jar gt lt path gt lt target name check files if not all files available gt lt fail message msg_filesNotFound gt lt target gt lt target name run monitoring depends check files build jar monitoring gt lt java fork true classname main class monitoring gt lt classpath gt lt path refid classpath gt lt path location S jar file monitoring gt lt classpath gt lt java gt lt target gt lt target name run analysis depends check files build jar analysis lt java fork true classname main class analysis gt lt arg line analysis directory gt lt classpath gt lt path refid classpath gt lt path location jar file analysis gt lt classpath gt lt java gt lt target gt lt target name build jar monitoring depends clean monitoring compile gt lt jar destfile jar file monitoring basedir build dir gt lt manifest gt lt attribute name Main Class value main class monitoring gt lt manifest gt lt jar gt lt delete dir build dir gt lt target gt
5. C 2 2 Copy Kieker and HornetQ Libraries Kieker Libraries Copy the following file from Kieker s binary distribution to the exam ple s 1ib directory 1 kieker 1 6 emf jar from dist HornetQ Libraries Copy the following files from the HornetQ 1ib folder to the 1ib directory of this example 1 hornetg jms client jar hornetq commons jar if available hornetq core client jar jboss jms api jar jnp client jar netty jar De cn C 2 3 Kieker Monitoring Configuration for Hornet Q The file examples userguide appendix JMS META INF kieker monitoring properties hornetq is already configured to use the AsyncJMSWriter via HornetQ The important properties are the definition of the provider URL the context factory and the queue kieker monitoring writer jms AsyncJMSWriter ProviderUrl jnp localhost 1099 Listing C 6 Excerpt from kieker monitoring properties hornetq configuring the provider URL of the JMS writer via HornetQ kieker monitoring writer jms AsyncJMSWriter ContextFactory Type org jnp interfaces NamingContextFactory Listing C 7 Excerpt from kieker monitoring properties hornetq configuring the context factory of the JMS writer via HornetQ Kieker 1 6 User Guide 67 C 2 4 Running the Example The execution of the example is performed by the following three steps 1 Start the JMS server gt run sh Listing C 8 Start of the JMS server under UNIX like systems gt run bat Listing C
6. Retrieving the current time via the configured time source Section 3 1 3 Scheduling and removing period samplers Section 3 1 4 and Controlling the monitoring state Section 3 1 5 3 1 1 Creating MonitoringController Instances The MonitoringController provides two different static methods for retrieving in stances of IMonitoringController 1 The method MonitoringController getInstance returns a singleton instance As described in Section the configuration is read from a properties file that has been passed to the JVM is located in the classpath or conforms to the default configuration Appendix E interface MonitoringController from kieker monitoring core controller getinstance IMonitoringController E ep 1 1 IMonitoringWriter from kieker monitoring writer getTimeSource ITimeSource writer newMonitoringRecord record IMonitoringRecord boolean diga schedulePeriodicSampler sampler ISampler period long 1 1 ITimeSource from kieker monitoring timer getTime long removeScheduledSampler sampler ScheduledSamplerJob boolean disableMonitoring enableMonitoring terminateMonitoring isMonitoringEnabled boolean isMonitoringTerminated boolean timeSource Figure 3 1 Class diagram of the MonitoringController including selected methods Kieker 1 6 User Guide 19 2 The metho
7. gt lt copy file META INF aop xml tofile build dir META INF aop xml gt lt copy file META INF aop event xml tofile build dir META INF aop event xml gt lt copy file META INF kieker monitoring properties tofile build dir META INF kieker monitoring properties gt lt jar destfile jar file basedir build dir gt lt manifest gt lt attribute name Main Class value main class gt lt manifest gt lt jar gt lt delete dir build dir gt lt target gt lt target name compile depends init lt javac source 1 5 target 1 5 srcdir fsrc dir destdir build dir classpathref classpath includeAntRuntime false gt lt target gt lt target name init depends clean gt Kieker 1 6 User Guide 93 lt delete dir build dir gt lt mkdir dir build dir gt lt target gt lt target name clean gt lt delete file jar file gt lt target gt lt project gt Listing H 7 build xml Kieker 1 6 User Guide 94 Bibliography 1 Hyperic Inc 2011 Hyperic SIGAR API http www hyperic com products 2 G Kiczales J Lamping A Menhdhekar C Maeda C Lopes J M Loingtier and J Irwin 1997 Aspect oriented programming In Proceedings of the 2007 European Conference on Object Oriented Programming ECOOP 97 volume 1241 of Lectur
8. lt weaver options gt lt include within kieker examples userguide ch5bookstore gt lt weaver gt lt aspects gt lt aspect name kieker monitoring probe aspect operationExecution OperationExecutionAspectAnnotation gt lt aspects gt lt aspectj gt Listing 5 2 aop xml Line 5 tells the AspectJ weaver to consider all classes inside the example package As pectJ allows to use wild cards for the definition of classes to include e g lt include within Kieker 1 6 User Guide 42 bookstoreTracing Bookstores to weave all classes with the prefix Bookstore located in a package bookstoreTracing Line 9 specifies the aspect to be woven into the classes In this case the Kieker probe OperationExecutionAspectAnnotation is used It requires that methods intended to be instrumented are annotated by OperationExecutionMonitoringProbe as mentioned be fore Listings and show how to compile and run the annotated Bookstore applica tion The aop xml must be located in a META INF directory in the classpath in this case the build directory The AspectJ weaver has to be loaded as a so called Java agent It weaves the monitoring aspect into the byte code of the Bookstore application Additionally a kieker monitoring properties is copied to the META INF directory This configuration file may be adjusted as desired see Section 3 2 gt mkdir build gt mkdir build META INF gt javac src kie
9. new Configuration configuration setProperty CONFIG_PROPERTY_NAME_TS_NANOS Long toString this rtThresholdNanos return configuration Listing 4 3 Plugin returning its default and current configurations The declaration of the available properties and their default values within a plugin is shown in section 4 2 2 as this is done with annotations Kieker 1 6 User Guide 34 26 27 28 29 30 31 32 33 34 35 36 37 38 60 61 62 63 64 4 2 2 OPlugin Annotation and Output Ports The Plugin class annotation is used to define a plugin name a description and the lists of output ports and configuration properties with default values Listing 4 4 shows the Plugin annotation for the example filter If the Plugin annotation is not present for a plugin the name defaults to the plugin s simple classname the description defaults to the empty string and the list of output ports is empty These default values are also used in case a respective attribute is omitted Note that the name is not required to be a unique among filters it is simply used for descriptive purposes such as in Figure 4 1 Output ports are specified using the nested OutputPort annotation In addition toa name and a description for the output port a list of event types can be specified Note that in this case the name is mandatory and must be unique for a plugin as it is used for connecting input and output ports The list of event
10. 0 00 utilization 0 00 usage 722 0 MB swap utilization 5 35 utilization 1 31 usage 721 0 MB swap utilization 1 80 utilization 0 20 usage 721 0 MB swap utilization 1 40 utilization 0 79 usage 721 0 MB swap utilization 1 80 utilization 0 79 usage 721 0 MB swap utilization 0 40 utilization 0 59 usage 721 0 MB swap Kieker 1 6 User Guide 72 E Kieker Monitoring Default Configuration This is the file kieker monitoring properties from the binary release and constitutes Kieker Monitoring s default configuration Section describes how to use a custom configuration In order to use a custom Kieker Monitoring configuration create a copy of this file and modify it according to your needs tH The location of the file is passed to Kieker Monitoring via the JVM parameter kieker monitoring configuration For example with a configuration file named my kieker monitoring properties in the folder META INF you would pass this location to the JVM when starting your application HH java Dkieker monitoring configuration META INF my kieker monitoring properties If no configuration file is passed Kieker tries to use a configuration file in META INF kieker monitoring properties If this also fails a default configuration is being used according to the values in this default file The name of the Kieker instance k
11. 30 31 32 33 34 35 36 37 38 39 40 41 42 43 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 public MyPipe final String pipeName 1 this pipeName pipeName public String getPipeName 1 return this pipeName d public void put final PipeData data throws InterruptedException this buffer put data y public PipeData poll final long timeout throws InterruptedException return this buffer poll timeout TimeUnit SECONDS Listing F 2 MyPipe java public PipeData final long loggingTimestamp final Object recordData final Class lt extends IMonitoringRecord gt recordType this loggingTimestamp loggingTimestamp this recordData recordData in real settings we would clone this recordType recordType public final long getLoggingTimestamp return this loggingTimestamp I public final Object getRecordData return this recordData in real settings we would clone public Class lt extends IMonitoringRecord gt getRecordType return this recordType Listing F 3 PipeData java Kieker 1 6 User Guide 83 G Example Console Outputs G 1 Quick Start Example Chapter 2 Apr 2 2012 6 33 14 PM kieker monitoring core configuration ConfigurationFactory createSingletonConfiguration INFO Loading properties from properties file in classpath META INF kieker monitoring properties Apr 2 2012 6 33 14 PM kieker monitoring core c
12. Be careful when using the value 1 since then the asynchronous writer is no longer decoupled from the monitored application kieker monitoring writer filesystem AsyncBinaryFsWriter QueueFullBehavior 0 Kieker 1 6 User Guide 78 Maximum time to wait for the writer threads to finish in milliseconds A MarShutdownDelay of 1 means infinite waiting kieker monitoring writer filesystem AsyncBinaryFsWriter MaxShutdownDelay 1 HEHEHE kieker monitoring writer kieker monitoring writer namedRecordPipe Pipe Writer The name of the pipe used must not be empty kieker monitoring writer namedRecordPipe PipeWriter pipeName kieker pipe GERAAK kieker monitoring writer kieker monitoring writer jms AsyncJMS Writer The url of the jndi provider that knows the jms service ActiveMQ tcp 127 0 0 1 61616 HornetQ jnp localhost 1099 OpenJMS tcp 127 0 0 1 30385 kieker monitoring writer jms AsyncJMSWriter ProviderUrl tcp 127 0 0 1 61616 The topic at the jms server which is used in the publisher subscribe communication kieker monitoring writer jms AsyncJMSWriter Topic queuel The type of the jms factory implementation e g ActiveMQ org apache activemqg jndi ActiveMQInitialContextFactory HornetQ org jnp interfaces NamingContextFactory OpenJMS org exolab jms jndi InitialContextFactory kieker monito
13. CALL 1283156498817855493 6488138950668976129 2 1 1283156498817823953 1283156498820007367 SRVO 02 bookstoreTracing CRM getOffers N A gt 6488138950668976129 3 2 1283156498817855493 1283156498819999771 SRV1 01 bookstoreTracing Catalog getBook N A gt lt SYNC RPLY 1283156498819999771 6488138950668976129 3 2 1283156498817855493 1283156498819999771 SRV1 1 bookstoreTracing Catalog getBook N A gt 6488138950668976129 2 1 1283156498817823953 1283156498820007367 SRVO 02 bookstoreTracing CRM getOffers N A gt lt SYNC RPLY 1283156498820007367 6488138950668976129 2 1 1283156498817823953 1283156498820007367 SRV0 2 bookstoreTracing CRM getOffers N A gt 6488138950668976129 0 0 12831564987 70302094 1283156498820012272 SRV0 3 bookstore Tracing Bookstore searchBook N A gt lt SYNC RPLY 1283156498820012272 6488138950668976129 0 0 1283156498770302094 1283156498820012272 SRV0 3 bookstoreTracing Bookstore searchBook N A gt gt Listing 5 11 Textual output of trace 6488138950668976129 s message trace representation Trace Equivalence Classes Deployment assembly level trace equivalence classes are computed and written to output files using the command line options print Deployment Equivalence Classes and print Assembly Equivalence Classes Listings and show the output generated for the monitoring data used in this section Class O cardinality 386 executions 4 representative
14. and how to use Kieker Analysis to analyze the monitored data Chapter provides a more detailed description of Kieker Monitoring and shows how to implement and use custom monitoring records monitoring probes and monitoring writers A more detailed description of Kieker Analysis and how to implement and use custom monitor ing readers and analysis plugins follows in Chapter Chapter 5 demonstrates how to use Kieker TraceAnalysis for monitoring analyzing and visualizing trace information Additional resources are included in the The Java sources presented in this user guide as well as pre compiled binaries are included in the examples userguide directory of the Kieker distribution see Section 2 1 Also the example directories can be imported as Eclipse projects Kieker 1 6 User Guide 6 2 Quick Start Example This chapter provides a brief introduction to Kieker based on a simple Bookstore example application Section explains how to download and install Kieker The Bookstore application itself is introduced in Section while the following sections demonstrate how to use Kieker for monitoring Section 2 3 and analyzing Section 2 4 the resulting monitoring data 2 1 Download and Installation The Kieker download sitd provides archives of the binary and source distribution the Javadoc API as well as additional examples For this quick start guide Kieker s binary distribution e g kieker 1 6 binaries zip is required
15. strumentation Kieker 1 6 User Guide 11 2 3 Monitoring with Kieker Monitoring In the previous Sections 2 1 and 2 2 the Kieker installation and the example application have been introduced In this section the preparations for application monitoring the instrumentation of the application and the actual monitoring are explained In this example the instrumentation is done manually This means that the monitoring probe is implemented by mixing monitoring logic with business logic which is often not desired since the resulting code is hardly maintainable Kieker includes probes based on AOP aspect oriented programming 2 technology as covered by Chapter 5 How ever to illustrate the instrumentation in detail the quick start example uses manual instrumentation The first step is to copy the Kieker jar file kieker 1 6 jar to the lib directory of the example directory see Section 2 2 The file is located in the kieker 1 6 dist directory of the extracted Kieker archive as described in Section 2 1 The final layout of the example directory is illustrated in Figure 2 4 examples userguide ch2 manual instrumentation build EE EE EE a Ae Directory for the Java class files ME EE ER EA EO Directory for the reguired libraries kieker 1 6 jar Ste EE EE N EE The directory for the source code files Figure 2 4 The directory structure of the Bookstore application with Kieker libraries The Java
16. 2011 10 10 01 0000 UTC Wed 11 May 2011 10 10 01 0000 UTC Wed 11 May 2011 10 10 01 0000 UTC usage 0 0 MB Wed 11 May 2011 10 10 06 0000 UTC Wed 11 May 2011 10 10 06 0000 UTC Wed 11 May 2011 10 10 06 0000 UTC usage 0 0 MB Wed 11 May 2011 10 10 11 0000 UTC Wed 11 May 2011 10 10 11 0000 UTC Wed 11 May 2011 10 10 11 0000 UTC usage 0 0 MB Wed 11 May 2011 10 10 16 0000 UTC Wed 11 May 2011 10 10 16 0000 UTC Wed 11 May 2011 10 10 16 0000 UTC usage 0 0 MB Wed 11 May 2011 10 10 21 0000 UTC Wed 11 May 2011 10 10 21 0000 UTC Wed 11 May 2011 10 10 21 0000 UTC usage 0 0 MB Wed 11 May 2011 10 10 26 0000 UTC Wed 11 May 2011 10 10 26 0000 UTC Wed 11 May 2011 10 10 26 0000 UTC usage 0 0 MB CPU host thinkpad cpu id 0 CPU host thinkpad cpu id 1 Mem Swap host thinkpad mem CPU host thinkpad cpu id 0 CPU host thinkpad cpu id 1 Mem Swap host thinkpad mem CPU host thinkpad cpu id 0 CPU host thinkpad cpu id 1 Mem Swap host thinkpad mem CPU host thinkpad cpu id 0 CPU host thinkpad cpu id 1 Mem Swap host thinkpad mem CPU host thinkpad cpu id 0 CPU host thinkpad cpu id 1 Mem Swap host thinkpad mem CPU host thinkpad cpu id 0 CPU host thinkpad cpu id 1 Mem Swap host thinkpad mem utilization
17. 3 toArray 4 1 record initFromArray y 1 1 record new MonitoringRecordImpl is IMonitoringRecord Figure 1 2 Communication among Kieker framework components The monitoring probes create the monitoring records containing the monitoring data and deliver them to the monitoring controller The monitoring controller employs the monitoring writers to write these monitoring records to a monitoring log or stream For analyzing purposes a monitoring reader plugin reads the records from the monitoring log stream These records can then be further processed by a configuration of additional filter plugins inter connected via input and output ports 1 2 Framework Components and Extension Points Figure depicts the possible extension points for custom components as well as the components which are already included in the Kieker distribution and detailed below e Monitoring writers and corresponding readers for file systems and SQL databases for in memory record streams named pipes as well writer and read ers employing Java Management Extensions JMX 6 and Java Messaging Ser vice JMS technology A special reader allows to replay existing persistent monitoring logs for example to emulate incoming monitoring data also in real time e Time sources utilizing Java s System nanoTime default or System current TimeMillis methods Kieker 1 6 User Guide 4 A A A
18. 9 Start of the JMS server under Windows Note that the script must be called from within HornetQ s bin directory 2 Start the analysis part in a new terminal gt ant run analysis hornetq 3 Start the instrumented Bookstore in a new terminal gt ant run monitoring hornetq C 3 OpenJMS C 3 1 Download and Prepare OpenJMS Download an OpenJMS install archive from http openjms sourceforge net and decompress it to the root directory of the example Under UNIX like systems make sure that the executable bit of all scripts within the bin directory are set C 3 2 Copy Kieker and OpenJMS Libraries Kieker Libraries Copy the following files from Kieker s binary distribution to the ex ample s 1ib directory 1 kieker 1 6 emf jar from dist 2 commons logging 1 1 1 jar from 1ib OpenJMS Libraries Copy the following files from the OpenJMS 1ib folder to the lib directory of this example 1 openjms lt version gt jar openjms common lt version gt jar openjms net lt version gt jar jms lt version gt jar concurrent lt version gt jar spice jndikit lt version gt jar DoE N Kieker 1 6 User Guide 68 C 3 3 Kieker Monitoring Configuration for OpenJMS The file examples userguide appendix JMS META INF kieker monitoring properties openjms is already configured to use the AsyncJMSWriter via OpenJMS The important properties are the definition of the provider URL and the context factor
19. IMonitoringController MONITORING CONTROLLER MonitoringController getInstance private final Catalog catalog new Catalog private final CRM crm new CRM this catalog public void searchBook 1 Call Catalog getBook and log its entry and exit timestamps final long tin MONITORING CONTROLLER getTimeSource getTime this catalog getBook false lt the monitored execution final long tout MONITORING_CONTROLLER getTimeSource getTime final OperationExecutionRecord e new OperationExecutionRecord public void this catalog getClass getName getBook boolean OperationExecutionRecord NO_SESSION_ID OperationExecutionRecord NO_TRACEID tin tout myHost OperationExecutionRecord NO EOI ESS OperationExecutionRecord NO EOI ESS MONITORING CONTROLLER newMonitoringRecord e 2 Call the CRM catalog s getOffers method without monitoring this crm getDffers J Listing 2 7 Instrumentation of the getBook call in Bookstore java Kieker 1 6 User Guide 13 The lines 32 and 34 are used to determine the current time in nanoseconds before and after the getBook call In lines 36 to 42 a monitoring record for this measurement is created and initialized passing the method signature the hostname and the two time values as arguments Finally the record is handed over to the monitoring controller line 43 which calls a monitoring writer to persist the record In this exa
20. Kieker 1 6 User Guide 53 Operation Dependency Graphs Deployment assembly level operation dependency graphs are generated using the command line options plot Deployment Operation Dependency Graph and plot Assembly Operation Dependency Graph Figures 5 9 a and 5 9 b show the operation depen dency graphs for the monitoring data used in this section lt lt deployment component gt gt 1635 3 Bookstore 1635 ESA lt lt execution container gt gt SRVO lt lt deployment component gt gt 2 CRM getOffers 573 lt lt deployment component gt gt 1 Catalog getBook searchBook lt lt assembly component gt gt 1635 a deployment level 1635 lt lt execution container gt gt SRV1 lt lt deplgyment component gt gt Q1 Catalog getBook 2 CRM 3 Bookstore 1635 searchBook lt lt assembly component gt gt 1635 lt lt assembly component gt gt 1 Catalog getBook gt C_getorterst b assembly level Figure 5 9 Operation dependency graphs Kieker 1 6 User Guide 54 5 3 5 HTML Output of the System Model Kieker TraceAnalysis writes an HTML representation of the system model reconstructed from the trace data to a file system entities html Figure shows a screen
21. Please note that the database writers are currently in a prototype stage and that they should be used with care The PrintStreamWriter simply sends the String representation of in coming records to the standard output or standard error streams which can be helpful for debugging purposes The AsyncJMSWriter and JMXWriter write records to a JMS Java Messaging Ser vice 5 queue and JMX Java Management Extensions 6 queue respectively The PipeWriter allows to pass records via in memory record streams named pipes These writers allow to implement on the fly analysis in distributed systems i e analysis while continuously receiving new monitoring data from an instrumented application potentially running on another machine A more detailed description of how to use the AsyncJM SWriter can be found in Appendix C interface IMonitoringWriter from kieker monitoring writer terminate boolean interface IMonitoringRecordReceiver from kieker monitoring core newMonitoringRecord record IMonitoringRecord boolean AbstractMonitoringWriter from kieker monitoring writer AbstractMonitoringWriter Configuration configuration init boolean toString String An AA A AsyncFsWriter JMXWriter PipeWriter AsyncDbWriter SyncFsWriter EER GE SyncDbWriter PrintStreamWriter Figure 3 4 Interface IMonitoringWriter and the implementing classes Kie
22. and must be downloaded Af ter having extracted the archive you ll find the directory structure and contents shown in Figure kieker 1 6 bin AE EE ron RR EER SN TAS GE Ee Ee Call scripts for Kieker tools JES dist EE AE EE RP ant Gan RE RE OE EE eN The Kieker framework libraries doe EE eens PRO RAR EE RES E PANE E EE eS kieker 1 onuserguiderpdf EE einen PDF file of this document examples LES userguide n a cee amines Ra Source code of the examples in this document MG EE EE EE EE EE Libraries reguired by Kieker Le EN Me cn A E N N EO Example configuration files kieker monitoring properties kin Figure 2 1 Directory structure and contents of Kieker s binary distribution http kieker monitoring net download Kieker 1 6 User Guide 7 The Java sources presented in this user guide as well as pre compiled binaries are included in the examples userguide directory The file kieker 1 6 jar contains the Kieker Monitoring and Kieker Analysis components as well as the Kieker TraceAnalysis tool The sample Kieker Monitoring configuration file kieker monitoring properties will be detailed in Chapter 3 In addition to the kieker 1 6 jar file the dist direc tory includes variants of this jar files with integrated third party libraries Additional information on these jar files and when to use them will follow later in this document 2 2 Bookstore Example Application The Bookstore application is a small s
23. configuration prop erties Whether the MonitoringController will be available as an MBean kieker monitoring MBean true Kieker 1 6 User Guide 21 FI Java Monitoring amp Management Console YOK tg Java Monitoring amp Management Console YOOX Connection Window Help Connection Window Help pid 6107 bookstoreApplication BookstoreStarter ma pid 6107 bookstoreApplication BookstoreStarter Overview Memory Threads Classes VM Summary MBeans Overview Memory Threads Classes VM Summary MBeans amp C Mimplementation Attribute values amp 5 Mimplementation Operation invocation amp 9 com sun management Name IE Value EF com sun management Cd java lang Experimentid E Cd java lang G java util logging HostName avanhoorn thinkpad G java util logging E kieker monitoring MontonngEnabled frue E kieker monitoring 4 MonitoringController None nate MEXER 9 MonitoringController ittributes Attributes NumberOfinserts o TEE boolean enableMonitoring O HostName iTimeSource Unavailable HostName ES MonitoringTerminated MonitoringTerminated MonitoringEnabled MonitoringEnabled Experimentid Experimentid TimeSource TimeSource boolean disableMonitoring O NumberOfinserts NumberoOfinserts D Operations Operations terminateMonitoring terminateMonitoring enableMonitoring enableMonitoring disableMonitoring disableMonitoring F incE
24. cuting the manually instrumentated Bookstore application and the analysis as described in Chapter 2 The files are included in the directory examples userguide ch2 manual instrumentation In order to run the analysis of the application it is necessary to pass the location of the monitoring log directory This is done via the parameter analysis directory e g gt ant run analysis Danalysis directory tmp kieker 20120402 163314855 UTC myHost KIEKER SINGLETON Listing H 1 Command to compile and run the instrumented Bookstore via ant build dir build src dir src lib dir lib jar file monitoring BookstoreMonitoring jar jar file analysis BookstoreAnalysis jar Jar file kieker kieker _emf jar main class monitoring kieker examples userguide ch2bookstore manual BookstoreStarter main class analysis kieker examples userguide ch2bookstore manual BookstoreAnalysisStarter meta dir META INF msg filesNotFound One or more of the required libraries could not be found Please add the following files to the lib dir directory jar file kieker Listing H 2 build properties lt project name Bookstore Application basedir default run monitoring gt lt property file build properties gt lt condition property not all files available gt lt resourcecount property num requiredLibsFound when ne count 1 gt lt fileset dir lib dir
25. dependency graph dot file Generate and store an assembly level operation dependency graph dot file Generate and store an aggregated deployment level call tree dot files Generate and store an aggregated assembly level call tree dot files Generate and store call trees for the selected traces dot files Save message trace representations of valid traces txt files Save execution trace representations of valid traces txt files Save a execution trace representations of invalid trace artifacts txt files Output an overview about the deployment level trace equivalence classes Output an overview about Kieker 1 6 User Guide 59 select traces lt id0 idn gt ignore invalid traces max trace duration lt duration in ms gt the assembly level trace equivalence classes Consider only the traces identified by the list of trace IDs Defaults to all traces If selected the execution aborts on the occurence of an invalid trace Threshold in milliseconds after which an incomplete trace becomes invalid Defaults to 600000 10 minutes ignore executions before date lt yyyyMMdd HHmmss gt Executions starting before this date UTC timezone are ignored ignore executions after date lt yyyyMMdd HHmmss gt Executions ending after short labels Example this date UTC timezone are ignored If se
26. executed This document assumes that the reader enters the commands in the example directory For this first example this is examples userquide ch2 bookstore application Windows comes with two command line interpreters called cmd exe and command com Only the first one is able to handle wildcards cor rectly So we recommend using cmd exe for these examples gt mkdir build gt javac src kieker examples userguide ch2bookstore java d build gt java classpath build Kieker examples userguide ch2bookstore BookstoreStarter Listing 2 5 Commands to compile and run the Bookstore application The first command compiles the application and places the resulting four class files in the build directory To verify the build process the build directory can be inspected The second command loads the bookstore application and produces the output shown in Listing 2 6 Bookstore main Starting request O Bookstore main Starting request 1 Bookstore main Starting request 2 Bookstore main Starting request 3 Bookstore main Starting request 4 Listing 2 6 Example run of the Bookstore application In this section the Kieker example application was introduced and when everything went well the bookstore is a runnable program Furthermore the composition of the appli cation and its function should now be present The next Section will demonstrate how to monitor this example application employing Kieker Monitoring using manual in
27. fault temporary directory under UNIX like systems this is typically tmp check the environment variables TMPDIR or temp for the location under Mac OS or Windows respectively The exact location of the created monitoring log is reported in Kieker s console output see for example Appendix G 1 The monitoring directory contains two types of files dat files containing text representations of the monitoring records and a file named kieker map which contains information on the types of monitoring records used tmp kieker 20120402 163314855 UTC myHost KIEKER SINGLETON E kieker map kieker 20120402 163314882 UT C 000 Thread 1 dat Figure 2 6 Directory structure after a monitoring run The Listings 2 11 and 2 12 show example file contents The dat file is saved in CSV format Comma Separated Values in this case the values of a monitoring record are separated by semicolons To understand the dat file structure the semantics have to be explained For this quick start example only some of the values are relevant The first value 1 indicates the record type The fourth value indicates the class and method which has been called And the seventh and eighth value are the start and end time of the execution of the called method 0 1333384394881852990 1 public void kieker examples userguide ch2bookstore manual getBook boolean lt no session id gt 1 1333384394878278653 1333384394880370678 myHost 1 1 0 13333843948
28. getInstance acquirePipe this pipeName catch final Exception ex LOG error Failed to acquire pipe this pipeName ex Listing 4 7 Example reader s initialization in the constructor excerpt Listing shows the MyPipeReader s read method In this case the reader polls the pipe for new records and forwards these to its output port public boolean read 1 tryd Wait max 4 seconds for the next data PipeData data this pipe poll 4 while data null Create new record init from received array final IMonitoringRecord record throws MonitoringRecordException AbstractMonitoringRecord createFromArray data getRecordType data getRecordData record setLoggingTimestamp data getLoggingTimestamp and delegate the task of delivering to the super class super deliver MyPipeReader OUTPUT PORT NAME record next turn data this pipe poll 4 catch final Exception e return false signal error i return true Listing 4 8 Example reader s read method 4 2 4 Developing Filter Plugins Custom filters must extend the class AbstractFilterPlugin In addition to providing meta information including output ports via the Plugin annotation Section 4 2 2 as well as implementing a constructor and getters for handling Configurations Sec tion 4 2 1 filters may override the methods init and terminate methods implementing initialization and cleanup
29. gt 1 1333384394906181991 1333384394908337642 myHost 1 1 Apr 2 2012 9 30 08 PM kieker analysis AnalysisController terminate INFO Terminating analysis Apr 2 2012 9 30 08 PM kieker analysis plugin reader filesystem FSReader terminate INFO Shutting down reader Listing G 2 Execution of the example analysis Section 2 4 Kieker 1 6 User Guide 85 G 2 Trace Monitoring Analysis amp Visualization Chapter 5 Bookstore main Starting request O 10 04 2012 13 03 51 kieker monitoring core configuration ConfigurationFactory createSingletonConfiguration INFO Loading properties from properties file in classpath META INF kieker monitoring properties 10 04 2012 13 03 51 kieker monitoring core controller MonitoringController createlnstance INFO Current State of kieker monitoring 1 5 Status enabled Name KIEKER Hostname pc vanhoorn experimentID 1 JMXController JMX disabled RegistryController O strings registered TimeSource kieker monitoring timer SystemNanoTimer Time in nanoseconds with nanoseconds precision since Thu Jan 01 01 00 00 CET 1970 WriterController Number of Inserts 0 Automatic assignment of logging timestamps true Writer kieker monitoring writer filesystem AsyncFsWriter Configuration kieker monitoring writer filesystem AsyncFsWriter QueueFullBehavior 0 kieker monitoring writer filesystem Asyn
30. invoke tools included in kieker 1 6 jar The following sections give a short description of their functionality and list their usage outputs as printed to the standard output stream when called without command line parameters In addition to the standard output stream the file kieker 1log is used for logging output during execution The Windows bat wrapper scripts must be executed from within the bin directory A 1 Script convertLoggingTimestamp sh bat The script converts Kieker Monitoring logging timestamps representing the number of nanoseconds since 1 Jan 1970 00 00 UTC to a human readable textual representation in the UTC and local timezones Main class kieker tools loggingTimestampConverter LoggingTimestampConverterTool Usage usage kieker tools logging TimestampConverter Logging TimestampConverterTool t lt timestampl timestampN gt t timestamps stimestampl timestampN gt List of timestamps UTC timezone to convert Example The following listing shows the command to convert two logging timestamps as well as the resulting output gt bin convertLoggingTimestamp sh timestamps 1283156545581511026 1283156546127117246 1283156545581511026 Mo 30 Aug 2010 08 22 25 0000 UTC Mo 30 Aug 2010 10 22 25 0200 local time 1283156546127117246 Mo 30 Aug 2010 08 22 26 0000 UTC Mo 30 Aug 2010 10 22 26 0200 local time Listing A 1 Execution under UNIX like systems gt convertLo
31. jma domain will be used kieker monitoring writer jmx JMXWriter domain The name of the MonitoringLog in the domain kieker monitoring writer jmx JMXWriter logname MonitoringLog Listing E 1 kieker monitoring properties Kieker 1 6 User Guide 81 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 F Additional Source Code Listings F 1 MyNamedPipeManager and MyPipe Not synchronized private final Map lt String MyPipe gt pipeMap new HashMap lt String MyPipe gt public static MyNamedPipeManager getInstance return MyNamedPipeManager PIPE_MGR_INSTANCE VEK Returns a pipe with name pipeName If a pipe with this name does not exist prior to thel call it will be created param pipeName name of the new pipe return the pipe throws IllegalArgumentException if the given name is null or has length zero public MyPipe acquirePipe final String pipeName throws IllegalArgumentException if pipeName null pipeName length 0 throw new IllegalArgumentException Invalid connection name pipeName Dts Es MyPipe conn synchronized this conn this pipeMap get pipeName if conn null conn new MyPipe pipeName this pipeMap put pipeName conn J return conn Listing F 1 MyNamedPipeManager java Kieker 1 6 User Guide 82 26 27 28 29
32. jvmarg value Dkieker monitoring configuration meta dir properties file gt lt java gt lt target gt lt target name build jar depends compile gt lt jar destfile jar file basedir build dir gt lt manifest gt lt attribute name Main Class value main class gt lt manifest gt lt jar gt lt delete dir build dir gt lt target gt lt target name compile depends init gt lt javac source 1 5 target 1 5 srcdir src dir destdir build dir classpathref classpath includeAntRuntime false gt lt mkdir dir build dir meta dir gt lt copy file meta dir properties file tofile build dir meta dir properties file ie lt target gt lt target name init depends clean gt lt delete dir build dir gt lt mkdir dir build dir gt lt target gt lt target name clean gt lt delete file jar file gt lt target gt lt project gt Listing H 5 build xml Kieker 1 6 User Guide 91 H 3 AspectJ based Trace Monitoring Chapter 5 The following build xml and build properties files can be used for compiling and exe cuting the Bookstore application instrumentated with AspectJ see Chapter 5 The files are included in the directory examples userguide ch5 trace monitoring aspectj build dir build src dir src lib d
33. kieker monitoring properties can be placed in a directory called META INF located in the classpath The available configuration properties can also be passed as JVM arguments e g Dkieker monitoring enabled true Configurations for Non Singleton Instances The class Configuration provides factory methods to create Configuration objects according to the default configuration or loaded from a specified properties file creat eDefaultConfiguration createConfigurationFromFile and createSingletonConfiguration Note that JVM parameters are only evaluated when using the factory method cre ateSingletonConfiguration The returned Configuration objects can be adjusted by setting single property values using the method setProperty 3 3 Monitoring Records Monitoring records are objects that contain the monitoring data as mentioned in the previous chapters Typically an instance of a monitoring record is constructed in a monitoring probe Section 3 4 passed to the monitoring controller Section B 1 seri alized and deserialized by a monitoring writer Section 3 5 and a monitoring reader and provided to analysis filters Section AD Figure ya es this life cycle of a monitoring record In Chapter 2 we ve already introduced and used the monitoring record type Oper ationExecutionRecord Kieker allows to use custom monitoring record types Corre sponding classes must implement the interface IMonitoringRecord shown in Figure 3 3 The meth
34. logging timestamps lt true false gt Replay the original logging timestamps defaults to true n realtime worker threads lt num gt Number of worker threads used in realtime mode defaults to 1 r realtime lt true false gt Replay log data in realtime Example The following command replays the monitoring testdata included in the binary release to another directory gt bin logReplay sh inputdirs examples userguide ch5 trace monitoring aspectj testdata kieker 20100830 082225522 UTC keep logging timestamps true realtime false Listing A 3 Execution under UNIX like systems gt logReplay bat inputdirs examples userguide ch5 trace monitoring aspectj testdata kieker 20100830 082225522 UT C keep logging timestamps true realtime false Listing A 4 Execution under Windows Kieker 1 6 User Guide 57 A 3 Script kax run sh bat Executes a Kieker Analysis pipe and filter configuration file kax file described in Sec tion 4 1 Main class kieker tools KaxRun Usage usage kieker tools KaxRun i lt filename gt i input lt filename gt the analysis project file kax loaded A 4 Script kax viz sh bat Visualizes a Kieker Analysis pipe and filter configuration file kax file described in Sec tion Main class kieker tools KaxViz Usage usage kieker tools KaxViz i lt filename gt svg lt filename gt i input lt filename gt the analysis projec
35. newMonitoringRecord r Listing 3 1 Method sample from MemSwapUsageSampler java 3 1 5 Controlling the Monitoring State The MonitoringController provides methods to temporarily enable or disable moni toring enableMonitoring disableMonitoring as well as to terminate monitoring perma nently terminateMonitoring The current state can be requested by calling the meth ods isMonitoringEnabled and isMonitoring Terminated If monitoring is not enabled i e disabled or terminated no monitoring records retrieved via the method newMonitor ingRecord are passed to the monitoring writer Also probes should be passive or return immediately with respect to the return value of the method isMonitoringEnabled Note that once the MonitoringController is terminated it cannot be enabled later on 3 1 6 JMX MBean Access to MonitoringController The MonitoringController s interface methods see Figure can be accessed as a JMX MBean For example this allows to control the monitoring state using the methods described in the previous Section As a JMX compliant graphical client that is included in the JDK jconsole is probably the easiest way to get started Figure shows two screenshots of the MBean access using jconsole In order to enable JMX MBean access to the MonitoringController the value of the configuration property kieker monitoring MBean must be set to true listing below The kieker monitoring properties includes additional JMX related
36. of the Bookstore application with AspectJ based instrumentation as described in Sec tion In order to illustrate the visualization of distributed traces the hostname of the Catalog s method getBook was probabilistically changed to a second hostname For a more detailed description of the underlying formalisms we refer to our technical report II The output can be found in the directory examples userguide ch5 trace monitoring aspectj testdata kieker 20100830 082225522 UTC example plots 5 3 1 Textual Trace and Equivalence Class Representations Execution Traces Textual execution trace representations of valid invalid traces are written to an out put file using the command line options print Execution Traces and print invalid Execution Traces Listing 5 10 shows the execution trace representation for the valid trace 6129 Traceld 6488138950668976129 minTin 12831564987 70302094 Mon 30 Aug 2010 08 21 38 0000 UTC maxTout 1283156498820012272 Mon 30 Aug 2010 08 21 38 0000 UTC maxEss 2 lt 6488138950668976129 0 0 1283156498770302094 1283156498820012272 SRVO 03 bookstoreTracing Bookstore searchBook N A gt lt 6488138950668976129 1 1 1283156498770900902 1283156498773404399 SRV1 1 bookstoreTracing Catalog getBook N A gt lt 6488138950668976129 2 1 1283156498817823953 1283156498820007367 SRVO 02 bookstoreTracing CRM getOffers N A gt lt 6488138950668976129 3 2 1283156498817855493 1283156498819999
37. pdf and png format gt bin dotPic fileConverter sh out pdf png Listing A 7 Execution under UNIX like systems gt dotPic fileConverter bat out pdf png Listing A 8 Execution under Windows Kieker 1 6 User Guide 61 B Java EE Servlet Container Example Using the sample Java web application MyBatis JPetStore this example demonstrates how to employ Kieker Monitoring for monitoring a Java application running in a Java EE container in this case Jettyf Monitoring probes based on the Java EE Servlet API Spring and AspectJ are used to monitor execution trace and session data see Sec tion 5 The directory examples JavaEEServletContainerExample contains the pre pared Jetty server with the MyBatis JPetStore application B 1 Setting The subdirectory jetty hightide jpetstore includes the Jetty server with the JPet Store application already deployed to the server s webapps directory The example is prepared to use two alternative types of Kieker probes either the Kieker Spring intercep tor default or the Kieker AspectJ aspects Both alternatives additionally use Kieker s Servlet filter Required Libraries and Kieker Monitoring Configuration Both settings require the files aspectjweaver 1 7 0 jar and kieker 1 6 jar which are already included in the webapps s WEB INF 1ib directory Also the Kieker configuration file is already included in the webapps s WEB INF classes META INF directory where it is considered for con
38. set customStoragePath as desired Examples var kieker or CAN KiekerData ensure the folder exists kieker monitoring writer filesystem AsyncBinaryFsWriter customStoragePath The maximal number of entries records per created file 4 Must be greater than zero kieker monitoring writer filesystem AsyncBinaryFsWriter maxEntrieslnFile 25000 The maximal file size of the generated monitoring log Older files will be deleted if this file size is exceeded Given in MiB At least one file will always remain regardless of size Use 1 to ignore this functionality kieker monitoring writer filesystem AsyncBinaryFsWriter maxLogSize 1 The maximal number of log files generated Older files will be deleted if this number is exceeded At least one file will always remain regardless of size Use 1 to ignore this functionality kieker monitoring writer filesystem AsyncBinaryFsWriter maxLogFiles 1 Asynchronous writers need to store monitoring records in an internal buffer This parameter defines its capacity in terms of the number of records kieker monitoring writer filesystem AsyncBinaryFsWriter QueueSize 10000 Behavior of the asynchronous writer when the internal queue is full 0 terminate Monitoring with an error default 1 writer blocks until queue capacity is available 2 writer discards new records until space is available
39. to develop custom plugins Chapter 5 presents the Kieker TraceAnalysis tool which is also based on Kieker Analysis Kieker Analysis has a dependency to the Eclipse Modeling Framework EMF P For this reason we are using the kieker 1 6_emf jar that is a variant of the kieker 1 6 jar additionally including the required EMF dependencies When using the kieker 1 6 jar the org eclipse emf jar files to be found in Kieker s 1ib directory need to be added to the classpath examples L userguide ch2 manual instrumentation build A T E aren aa ee Directory for the Java class files E ER T E acne antec Directory for the required libraries SKEI EE RE re eee ae EE ree The directory for the source code files Jmanual BookstoreAnalysisStarter java Figure 2 7 Directory layout of the example application with the analysis files highlighted http www eclipse org modeling emf Kieker 1 6 User Guide 16 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 receivedEvents x relayedEvents lt lt Filter gt gt TmonitoringRecords Tere Figure 2 8 Example pipe and filter configuration The analysis application developed in this section comprises the file BookstoreAnaly sisStarter java as shown in Figure 2 7 This file can also be found in the directory examples userguide ch2 manual instrumentation The file sets up the basic pipe and filter configuration depicted in Figure
40. 129 b Trace 6130 T Es SRVO SRVO RVT q E SRVO SRVO 3 Bookstore 1 Catalog 2 CRM 1 Catalog 3 Bookstore 1 Catalog 2 CRM T i i searchBook searchBook getBook getBook H K K getOffers getOffers getBook getBook Ee LLE K r K pronome nono c Trace 6131 d Trace 6141 Figure 5 3 Deployment level sequence diagrams of the trace equivalence class representa tives Listing 5 13 Kieker 1 6 User Guide 50 Assembly Level Sequence Diagrams Assembly level sequence diagrams are generated using the command line option plot Assembly Sequence Diagrams Figure shows the sequence diagram for the assembly level trace equivalence representative Section 5 3 1 3 Bookstore m i searchBook _ 1 Catalog 2 CRM getBook getOfters getBook Figure 5 4 Assembly level sequence diagram of trace 6129 5 3 3 Call Trees Trace Call Trees Trace call trees are generated using the command line option plot Call Trees Fig ures 5 5 a 5 5 d show these call trees for each deployment level trace equivalence rep resentative Section 5 3 1 1 1 1 1 SRVO SRVO SRVO SRVO 3 Bookstore 3 Bookstore 3 Bookstore 3 Bookstore searchBook searchBook
41. 2 5 3 4 Dependency Graphs Container Dependency Graphs A container dependency graph is generated using the command line option plot Container Dependency Graph Figure shows the container dependency graph for the monitoring data used in this section 2154 lt lt execution container gt gt SRV1 1635 lt lt execution container gt gt SRVO Figure 5 7 Container dependency graph Component Dependency Graphs Deployment assembly level component dependency graphs are generated using the command line options plot Deployment Component Dependency Graph and plot Assembly Component Dependency Graph Figures 5 8 a and 5 8 b show the component depen dency graphs for the monitoring data used in this section lt lt execution container gt gt SRVO 543 lt lt deployment component gt gt 573 1 Catalog 1635 lt lt deployment component gt gt 1635 lt lt deployment component gt gt P 3 Bookstore gt 2 CRM 1092 1062 E E lt lt execution container gt gt SRV1 lt lt deployment component gt gt 1 Catalog a deployment level 1635 lt lt assembly component gt gt 1635 gt lt lt assembly component gt gt 3 Bookstore 1635 1635 1 Catalog TH lt lt assembly component gt gt gt 2 CRM b assembly level Figure 5 8 Component dependency graphs
42. 3 St Pele 223 E R sjalals a Rc E D Q 2 E Sla gt Bos D e na A e Slalels S 9 e S z SWE 3 9 8 S 3 8 2 a 8 2 N 5 PER Sg S 5 2 2 2 2 S 8 3 g MI IE E a FE 3 8 is E 3 5 8 SIs g S 5 5 imaceanalysis gt 5 is o isl jes es 9 ES E S S S 3 E 5 S Architecture reconst S 7 gt Asynchrwriters v Pipe and fiter framework Analysis E Visualization Plug in Monitoring E Mahaira El O controller Probe Monitoring Monitoring E Analysis 3 Record di Reader Controller Kieker Monitoring Kieker Analysis Figure 1 3 Kieker framework components and extension points for custom components e Monitoring record types allowing to store monitoring data about operation executions including timing control flow and session information CPU and re source utilization memory swap usage as well as a record type which can be used to store the current time e Monitoring probes A special feature of Kieker is the ability to monitor dis tributed traces of method executions and corresponding timing information For monitoring this data Kieker includes monitoring probes employing AspectJ 10 Java EE Servlet 7 Spring 8 and Apache CXF 9 technology Additionally Kieker includes probes for periodic system level resource monitoring employing the Sigar library I e Analysis Visualization plugins can be assembled to pipe and filter ar
43. 58 59 60 public void terminate 61 nothing to do 62 63 Listing 3 5 MyPipeWriter java Kieker 1 6 User Guide 28 The monitoring writer to be used is selected by the Kieker Monitoring configuration prop erty Section 3 kieker monitoring writer Writer specific configuration properties can be provided by properties prefixed by the fully gualified writer classname Listing 3 6 demonstrates how to use the custom writer MyPipeWriter defined above In this exam ple the pipe name is passed as the property value pipeName kieker monitoring writer kieker examples userguide ch3and4bookstore MyPipeWriter kieker examples userguide ch3and4bookstore MyPipeWriter pipeName somePipe Listing 3 6 Configuration of the custom writer MyPipeWriter As the data structure of this kind of monitoring stream we created a class PipeData in order to demonstrate the use of the toArray and initFromArray in Section methods A PipeData object holds a logging timestamp and an Object array containing the serialized record data Appendix includes a source code listing of this class Alternatively we could have used IMonitoringRecord as the data structure used by the pipe This is the way Kieker s PipeWriter works Kieker 1 6 User Guide 29 4 Kieker Analysis Component The Java sources of this chapter as well as a pre compiled binary can be found in the examples userguide ch3 4 custom components di rectory of the b
44. 6488138950668976130 max stack depth 2 Class 1 cardinality 356 executions 4 representative 6488138950668976131 max stack depth 2 Class 2 cardinality 706 executions 4 representative 6488138950668976129 max stack depth 2 Class 3 cardinality 187 executions 4 representative 6488138950668976141 max stack depth 2 Listing 5 12 Textual output of information on the deployment level trace equivalence classes Class O cardinality 1635 executions 4 representative 6488138950668976129 max stack depth 2 Listing 5 13 Textual output of information on the assembly level trace equivalence class Kieker 1 6 User Guide 49 5 3 2 Sequence Diagrams Deployment Level Sequence Diagrams Deployment level sequence diagrams are generated using the command line option plot Deployment Sequence Diagrams Figures 5 3 a H5 3 d show these sequence di agrams for each deployment level trace equivalence representative Section 5 3 1 Fr SRVO RVT RVO T SRVO SAVT RVO RVO 3 Bookstore 1 Catalog 2 CRM 3 Bookstore 1 Catalog 2 CRM 1 Catalog searchBook searchBook getBook getBook lt 4 lt 4 getOtfers getOffers getBook getBook bee ees PEETER r 4 EE a Trace 6
45. 771 SRV1 1 bookstoreTracing Catalog getBook N A gt Listing 5 10 Textual output of trace 6488138950668976129 s execution trace representation Message Traces Textual message trace representations of valid traces are written to an output file using the command line option print Message Traces Listing shows the message trace representation for the valid trace 6129 Trace 6488138950668976129 lt SYNC CALL 1283156498770302094 gt 6488138950668976129 0 0 12831564987 70302094 1283156498820012272 SRVO0 3 bookstore Tracing Bookstore searchBook N A gt lt SYNC CALL 1283156498770900902 6488138950668976129 0 0 1283156498770302094 1283156498820012272 SRVO 03 bookstoreTracing Bookstore searchBook N A gt 6488138950668976129 1 1 1283156498770900902 1283156498773404399 SRV1 01 bookstoreTracing Catalog getBook N A gt lt SYNC RPLY 1283156498773404399 6488138950668976129 1 1 1283156498770900902 1283156498773404399 SRV1 1 bookstoreTracing Catalog getBook N A gt 6488138950668976129 0 0 1283156498770302094 1283156498820012272 SRVO 03 bookstoreTracing Bookstore searchBook N A gt Kieker 1 6 User Guide 48 lt SYNC CALL 1283156498817823953 6488138950668976129 0 0 1283156498770302094 1283156498820012272 SRVO 03 bookstoreTracing Bookstore searchBook N A gt 6488138950668976129 2 1 1283156498817823953 1283156498820007367 SRVO 02 bookstoreTracing CRM getOffers N A gt lt SYNC
46. 90981315 1 public void kieker examples userguide ch2bookstore manual getBook boolean lt no session id gt 1 133338439488887 7486 1333384394890961341 myHost 1 1 Listing 2 11 kieker 20120402 163314882 UTC 000 Thread 1 dat excerpt Kieker 1 6 User Guide 15 The second file is a simple mapping file referencing keys to monitoring record types In Listing the key 1 is mapped to the type of operation execution records which were used in the monitoring The key value corresponds to the key values in the dat file 0 kieker common record controlflow OperationExecutionRecord Listing 2 12 kieker map By the end of this section two Java classes of the Bookstore application have been manually instrumented using Kieker Monitoring and at least one run of the instrumented application has been performed The resulting monitoring log written to the dat file in CSV format could already be used for analysis or visualization by any spreadsheet or statistical tool The following Section will show how to process this monitoring data with Kieker Analysis 2 4 Analysis with Kieker Analysis In this section the monitoring data recorded in the previous section is analyzed with Kieker Analysis For this quick example guide the analysis tool is very simple and does not show the full potential of Kieker For more detail read Chapter 4 to learn which plugins i e readers and filters are included in Kieker how to use them and how
47. Kieker s file system reader FSReader reads monitoring records from a file system monitoring log as produced in the previous Section 2 3 and passes these to the TeeFilter plugin the TeeFilter plugin reads events of arbitrary type i e Java Object prints them to a configured output stream and also relays them to filters connected to the filter s output port relayedEvents Currently Kieker Analysis pipe and filter configurations can only be created programmatically i e by configuring instantiating registering and connecting the plugins in a Java program For the example this is demonstrated in Listing 2 13 which shows an excerpt from the BookstoreAnalysisStarter s main method Create Kieker Analysis instance final AnalysisController analysisInstance new AnalysisController Set filesystem monitoring log input directory for our analysis final Configuration fsReaderConfig new Configuration fsReaderConfig setProperty FSReader CONFIG_PROPERTY_NAME_INPUTDIRS args 0 final FSReader reader new FSReader fsReaderConfig analysisInstance registerReader reader Create and register a simple output writer final Configuration teeFilterConfig new Configuration teeFilterConfig setProperty TeeFilter CONFIG PROPERTY NAME STREAM TeeFilter CONFIG PROPERTY VALUE STREAM STDOUT final TeeFilter teeFilter new TeeFilter teeFilterConfig analysisInstance registerFilter teeFilter Connect
48. Listing shows the commands to compile and run the application under UNIX like systems Listing shows the same commands for Windows The expected working directory is the base directory of this example i e examples userguide ch2 manual instrumentation gt mkdir build gt javac src kieker examples userguide ch2bookstore manual x java classpath lib kieker 1 6 jar d build gt java classpath build lib kieker 1 6 jar kieker examples userguide ch2bookstore manual BookstoreStarter Listing 2 9 Commands to compile and run the instrumented Bookstore under UNIX like systems Kieker 1 6 User Guide 14 Under Windows it is necessary to separate the classpath elements by a semicolon instead of a colon Also we recommend to use the Windows shell cmd exe for this tutorial since problems have been reported for the Windows PowerShell gt mkdir build gt javac src kieker examples userguide ch2bookstore manual x java classpath lib kieker 1 6 jar d build gt java classpath build lib kieker 1 6 jar kieker examples userguide ch2bookstore manual BookstoreStarter Listing 2 10 Commands to compile and run the instrumented Bookstore under Windows If everything worked correctly a new directory for the monitoring data with a name sim ilar to kieker 20120402 163314855 UTC myHost KIEKER SINGLETON is created see Figure 2 6 In Kieker s default configuration the log directory can be found in the de
49. N http kieker monitoring net Kieker 1 6 User Guide Kieker Project October 17 2012 Christian Albrechts Universitat zu Kiel Software Engineering Group http se informatik uni kiel de Kiel University Dept Computer Science Kiel Germany For guide lines on how to cite Kieker and this document please see Section 1 4 Contents 3 11 WhatiskKieker 2 ee 3 1 2 Framework Components and Extension Points 4 1 3 Licensing cccclccll A 5 fo eee ER Be See RES AA AREA OE OE RE 6 1 5 Structure of this User Guide 2 ei 6 7 EE GR AE BU HE ER EE HA 7 2 2 Bookstore Example Application EE EE Es 8 2 3 Monitoring with Kieker Monitoring a a EE EE 12 MEE ER HERE HARE AE ME OE be 16 19 3 1 Monitoring Controller EE EE EE Ee 19 EE EET EER EER N 23 3 3 Monitoring Records SS SS SS 23 3 4 Monitoring Probes 2 SE ee 26 3 5 Monitoring Writers EE EE e 27 4 Kieker Analysis Component 30 4 1 Pipe and Filter Framework and Included Plugins 30 ad ware nearest ae te 33 5 Kieker TraceAnalysis Tool 40 5 1 Monitoring Trace Information EE EE 0000200 eee 41 Deb behead RR ae bod ee Da Gd 46 5 3 Example Kieker TraceAnalysis Outputs 2 205 48 56 Bibliography 95 Kieker 1 6 User Guide 2 1 Introduction Modern software applications are often complex and have to fulfill a large set of functional
50. Name String pluginDescription String getCurrentConfiguration Configuration terminate error boolean Figure 4 2 Class diagram showing important Kieker Analysis classes and their relationship Figure 4 2 shows the class diagram with the important Kieker Analysis classes and their relationships Note that only the most important methods are included An analysis with Kieker Analysis is set up and executed employing the class AnalysisController Setting up and running an analysis with Kieker Analysis requires the following steps to be performed as sketched in Section 2 4 already 1 Creating an instance of the AnalysisController class 2 Creating and registering monitoring readers registerReader filters register Fil ter and repositories registerRepository 3 Connecting plugins to other plugins and to repositories connect 4 Starting the analysis instance run On invocation of the run method the AnalysisController calls the init method of all filter plugins allowing them to initialize Then it starts the configured monitoring readers by calling its read method Plugins send data via their output ports to con nected input ports of other plugins Being the source in a pipe and filter architecture readers don t have input ports Plugins can be connected to repositories which may provide shared services such as managed access to a common architectural model of the analyzed system As soon as th
51. T C virus KIEKER outputdir out plot Deployment Sequence Diagrams plot Call Trees Listing 5 8 Commands to produce the diagrams under Windows The Windows bat wrapper scripts including trace analysis bat must be executed from within the bin directory Kieker 1 6 User Guide 46 The resulting contents of the out directory should be similar to the following tree out deploymentSeguenceDiagram 6120391893596504065 pic call Tree 6120391893596504065 dot system entities html The pic and dot files can be converted into other formats such as pdf by using the Graphviz and PlotUtils tools dot and pic2plot The following Listing 5 9 demonstrates this gt dot callTree 6120391893596504065 dot Tpng o callTree png gt pic2plot deploymentSequenceDiagram 6120391893596504065 pic Tpng gt sequenceDiagram png Listing 5 9 Commands to convert the diagrams The scripts dotPic fileConverter sh and dotPic LS fileConverter bat convert all pic and dot in a specified directory See Appendix A 6 for details Examples of all available visualization are presented in the following Section 5 3 Kieker 1 6 User Guide 47 5 3 Example Kieker TraceAnalysis Outputs The examples presented in this section were generated based on the monitoring data which can be found in the directory examples userguide chb trace monitoring aspect j testdata kieker 20100830 082225522 UTC It consists of 1635 traces
52. ables kieker monitoring writer database AsyncDbWriter TablePrefix kieker Drop already existing tables or terminate monitoring with an error kieker monitoring writer database AsyncDbWriter Drop Tables false Kieker 1 6 User Guide 80 The number of concurrent Database connections kieker monitoring writer database AsyncDbWriter numberOfConnections 4 Asynchronous writers need to store monitoring records in an internal buffer This parameter defines its capacity in terms of the number of records kieker monitoring writer database AsyncDbWriter QueueSize 10000 Behavior of the asynchronous writer when the internal queue is full 0 terminate Monitoring with an error default 1 writer blocks until queue capacity is available 2 writer discards new records until space is available Be careful when using the value 1 since then the asynchronous writer is no longer decoupled from the monitored application kieker monitoring writer database AsyncDbWriter QueueFullBehavior 0 Maximum time to wait for the writer threads to finish in milliseconds A MarShutdownDelay of 1 means infinite waiting kieker monitoring writer database AsyncDbWriter MaxShutdownDelay 1 HHH kieker monstoring writer kieker monatoring writer jma JMX Writer The domain used to register the MonitoringLog If empty the value of kieker monitoring
53. ail below examples userguide ch2 bookstore application ste EE EE EE EE The directory for the source code files ch2bookstore Bookstore java BookstoreStarter java Catalog java CRM java build properties buildixmiE er Optional built script for the application Figure 2 3 The directory structure of the Bookstore application The Java sources and a pre compiled binary of the uninstrumented Bookstore application can be found in the examples userguide ch2 bookstore application directory The class BookstoreStarter contains the application s main method shown in List ing 2 1 i e the program start routine It initializes the Bookstore and issues five search requests by calling the searchBook method of the bookstore object 23 public static void main final String args 24 final Bookstore bookstore new Bookstore 25 for int i 0 i lt 5 i 26 System out println Bookstore main Starting request i 27 bookstore searchBook 28 29 Listing 2 1 main method from BookstoreStarter java Kieker 1 6 User Guide 9 The Bookstore shown in Listing 2 2 contains a catalog and a CRM object representing the catalog of the bookstore and a customer relationship management system which can provide offers for books out of the catalog The business method of the bookstore is searchBook which will first check the catalog for books and then check for offers In a real applicatio
54. al ISigarSamplerFactory sigarFactory SigarSamplerFactory INSTANCE final CPUsDetailedPercSampler cpuSampler sigarFactory createSensorCPUsDetailedPerc final MemSwapUsageSampler memSwapSampler sigarFactory createSensorMemSwapUsage final long offset 2 start after 2 seconds final long period 5 monitor every 5 seconds monitoringController schedulePeriodicSampler cpuSampler offset period TimeUnit SECONDS monitoringController schedulePeriodicSampler memSwapSampler offset period TimeUnit SECONDS Listing D 1 Excerpt from MonitoringStarter java Based on the existing samplers users can easily create custom Sigar based samplers by extending the class AbstractSigarSampler For example Listing 3 1 in Section shows the MemSwapUsageSampler s sample method Typically it is also required to define a corresponding monitoring record type as explained in Section 3 3 When implementing custom Sigar based samplers the SigarSamplerFactory s getSigar method should be used to retrieve a Sigar instance This example uses a stand alone Java application to set up a Sigar based monitoring process When using servlet containers users may consider implementing this routine as a ServletContextListener which are executed when the container is started and shutdown As an example Kieker includes a CPUMemUsageServletContextListener D 3 Executing the Example The execution of the example is performed by the following t
55. ample application resembling a simple bookstore with a market place facility where users can search for books in an online catalog and subsequently get offers from different book sellers Figure 2 2 shows a class diagram de scribing the structure of the bookstore and a sequence diagram illustrating the dynamics of the application bookstore Bookstore catalog Catalog BookstoreStarter main searchBook getBook BookstoreStarter eee ees bookstore getOffers Bookstore getBook searchBook getOffers catalog oo do o TT gt Catal catalog Wen ea ie getBook dees T a b Figure 2 2 UML class diagram a and sequence diagram b of the Bookstore application Kieker 1 6 User Guide 8 The bookstore contains a catalog for books and a customer relationship management system CRM for the book sellers To provide this service the different classes provide operations to initialize the application search for books and get offers or searched books In this example the methods implementing these operations are merely stubs However for the illustration of Kieker they are sufficient and the inclined reader may extend the application into a real bookstore The directory structure of the Bookstore example is shown in Figure 2 3Jand comprises four Java classes in its source directory src ch2bookstore which are explained in det
56. and non functional requirements The internal behavior of such large systems cannot easily be determined on the basis of the source code Furthermore existing applications often lack sufficient documentation which makes it cumbersome to extend and change them for future needs A solution to these problems can be dynamic analysis based on application level monitoring which allows to log the behavior of the application and to discover for example application internal control flows calling dependencies and method response times Dynamic analysis can help in detecting performance problems and faulty behavior capacity planning and many other areas The Kieker framework provides the necessary monitoring capabilities and comes with tools and libraries for the analysis of monitored data Kieker has been designed for continuous monitoring in production systems inducing only a very low overhead Further information on the overhead caused by Kieker is provided at http kieker monitoring net overhead evaluation 1 1 What is Kieker Kieker is a Java based application performance monitoring and dynamic software analysis framework 12 Monitoring adapters for other platforms such as Visual Basic 6 VB6 NET and COBOL are currently under development Figurell 1Jshows the framework s composition based on the two main components Kieker Monitoring and Kieker Analysis Kieker Monitoring Kieker Analysis Monitoring Probe Ana
57. ava Configure and register the reader final Configuration readerConfig new Configuration readerConfig setProperty MyPipeReader CONFIG_PROPERTY_NAME_PIPE_NAME somePipe final MyPipeReader reader new MyPipeReader readerConfig analysisController registerReader reader Configure register and connect the response time filter final Configuration filterConfig new Configuration final long rtThresholdNanos TimeUnit NANOSECONDS convert 1900 TimeUnit MICROSECONDS filterConfig setProperty configure threshold of 1 9 milliseconds MyResponseTimeFilter CONFIG PROPERTY NAME TS NANOS Long toString rtThresholdNanos final MyResponseTimeFilter filter new MyResponseTimeFilter filterConfig analysisController registerFilter filter analysisController connect reader MyPipeReader OUTPUT PORT NAME filter MyResponseTimeFilter INPUT PORT NAME RESPONSE TIMES Listing 4 1 Initializing and connecting the example reader and filter Starter java 4 1 2 Monitoring Reader Plugins The monitoring readers are the direct counterpart to the monitoring writers While writ ers receive records and write them into files or other kinds of monitoring logs streams readers deserialize monitoring data and provide it as IMonitoringRecord instances There are already some readers implemented in Kieker as shown in the class diagram in Figure 4 3 The FSReader has already been used in Section 2 4 A brief desc
58. cFsWriter flush true kieker monitoring writer filesystem AsyncFsWriter QueueSize 10000 kieker monitoring writer filesystem AsyncFsWriter customStoragePath kieker monitoring writer filesystem AsyncFsWriter MaxShutdownDelay 1 kieker monitoring writer filesystem AsyncFsWriter storelnJavaloT mpdir true kieker monitoring writer filesystem AsyncFsWriter maxEntriesInFile 25000 Records lost O Writer Threads 1 Finished false Writing to Directory tmp kieker 20110428 142829399 UT C Kaapstad KIEKER Sampling Controller Periodic Sensor available Current Poolsize 0 Scheduled Tasks 0 10 04 2012 13 03 51 kieker monitoring core registry ControlFlowRegistry lt clinit gt INFO First threadld will be 7752665283541598209 10 04 2012 13 03 51 kieker monitoring core controller MonitoringController 1 run INFO ShutdownHook notifies controller to initiate shutdown 10 04 2012 13 03 51 kieker monitoring core controller MonitoringController cleanup INFO Shutting down Monitoring Controller KIEKER 10 04 2012 13 03 51 kieker monitoring writer AbstractAsyncWriter terminate INFO Shutting down writers Listing G 3 Execution of the Bookstore with AspectJ trace instrumentation Section 5 1 1 Kieker 1 6 User Guide 86 H Ant Scripts H 1 Quick Start Example Chapter 2 The following build xml and build properties files can be used for compiling and exe
59. chitec tures based on input and output ports The Kieker TraceAnalysis tool Figure 1 1 is itself implemented based on a re usable set of Kieker Analysis plugins allowing to reconstruct and visualize architectural models of the monitored systems e g as dependency graphs sequence diagrams and call trees 1 3 Licensing Kieker is licensed under the Apache License Version 2 0 You may obtain a copy of the license at The Kieker source and binary release archives include a number of third party libraries The 1ib directory of the release archives contains a LICENSE file for each third party library pointing to the respective license text Kieker 1 6 User Guide 5 1 4 Citing Kieker When referencing Kieker resources in your publications we would be happy if you re spected the following guide lines e When referencing the Kieker project please cite our ICPE 2012 paper and or our 2009 technical report II Also you might want to add a reference to our web site http kieker monitoring net like 4 e When referencing this user guide e g when reprinting contents please use a citation like 3 At http kieker monitoring net documentation we provide entries for BIBTEX and other bibliography systems 1 5 Structure of this User Guide Based on a simple example Chapter 2 demonstrates how to manually instrument Java programs with Kieker Monitoring in order to monitor timing information of method ex ecutions
60. d MonitoringController createInstance Configuration config can be uti lized to create an instance that is configured according to the passed Configura tion object as described in Section 3 2 3 1 2 Logging Monitoring Records Monitoring records are sent to the configured monitoring writers by passing these records in form of IMonitoringRecord objects to the MonitoringController s newMonitor ingRecord method Note that this is not the case if monitoring is disabled or terminated Section 3 1 5 3 1 3 Retrieving the Current Time and Using Custom Time Sources The current time is maintained by a so called time source The MonitoringController s method get TimeSource returns an ITimeSource whose method get Time returns a times tamp in nanoseconds Kieker s default time source SystemNanoTimer returns the cur rent system time as the number of nanoseconds elapsed since 1 Jan 1970 00 00 UTC The easiest way to use a custom time source is to extend the AbstractTimeSource and to implement the method getTime Custom time sources make sense for example in simulations where not the current system time but the current simulation time is relevant The configuration needs to be adjusted to use a custom time source class 3 1 4 Scheduling and Removing Periodic Samplers For certain applications it is required to monitor runtime data periodically e g the utilization of system resources such as CPUs For this purpose Kieker supports specia
61. d method providing the obtained records to connected filters via the output port s declared in the P1ugin annotation Section 4 2 2 The read method should be implemented synchronously i e it should return after reading is finished or has been aborted via an invocation of the terminate method Listing 4 6 shows the Plugin annotation of the example reader MyPipeReader Read ing monitoring records from the monitoring pipe introduced in the previous Chapter 3 5 the reader provides received monitoring records via its output port Plugin name Pipe reader description Reads records from a configured pipe outputPorts OOutputPort name MyPipeReader OUTPUT PORT NAME description Outputs any received record eventTypes IMonitoringRecord class gt configuration OProperty name MyPipeReader CONFIG PROPERTY NAME PIPE NAME defaultValue kieker pipe Listing 4 6 Plugin annotation for the example reader Listing 4 7 shows an excerpt of the MyPipeReader s constructor In this case the reader reads the pipe name from the configuration and connects to the named pipe Optionally the reader can override the init method Kieker 1 6 User Guide 36 54 55 56 57 58 59 60 61 63 64 65 66 67 68 69 70 71 72 73 74 75 76 TT 78 79 80 81 82 83 this pipeName config getStringProperty MyPipeReader CONFIG PROPERTY NAME PIPE NAME try this pipe MyNamedPipeManager
62. e Notes in Computer Science pages 220 242 Springer 3 Kieker Project Oct 2012 Kieker 1 6 User Guide Software Engineer ing Group Kiel University Kiel Germany http kieker monitoring net documentation 4 Kieker Project 2012 Kieker web site http kieker monitoring net 5 Oracle 2011 Java Messaging Service JMS http www oracle com technetwork java jms 6 OracleJava management extensions jmx technology http www oracle com technetwork java javase tech javamanagement 140525 html 7 Oracle 2011 Java Servlet Technology http www oracle com technetwork java index jsp 135475 html 8 SpringSource 2011 Spring http www springsource org 9 The Apache Foundation 2011 Apache CXF http cxf apache org 10 The Eclipse Foundation 2011 The AspectJ Project http www eclipse org 11 A van Hoorn M Rohr W Hasselbring J Waller J Ehlers S Frey and D Kiesel horst Nov 2009 Continuous monitoring of software services Design and appli cation of the Kieker framework Technical Report TR 0921 Department of Com puter Science University of Kiel Germany http www informatik uni kiel de uploads tx publication vanhoorn tr0921 pdf Kieker 1 6 User Guide 95 12 A van Hoorn J Waller and W Hasselbring Apr 2012 Kieker A framework for application performance monitoring and dynamic software analysis In Proceed ings of the 8rd ACM SPEC International Con
63. e analysis architecture reconstruction and visualization are included as part of the Kieker TraceAnalysis tool presented in Chapter 5 Like each plugin filters are configured via properties as used in Section 4 1 1 and detailed in Section 4 2 1 4 1 4 Repositories Currently Kieker includes a single repository SystemModelRepository which is used by the Kieker TraceAnalysis filters to update and query a component based system model representing architectural entities and structures discovered while processing the incom ing monitoring data The development and use of repositories is detailed in Section 4 2 5 4 2 Developing Analysis Plugins and Repositories When implementing analysis plugins i e readers or filters and repositories the classes AbstractReaderPlugin AbstractFilterPlugin or respectively AbstractReposi tory need to be extended Figure 4 2 Section 4 2 1 describes how plugins and repos itories can be configured via properties Section describes how to declare meta information for plugins using dedicated annotations Specific information on the devel opment of custom filters readers and repositories are given in Sections 4 2 5 Kieker 1 6 User Guide 33 43 44 45 46 47 48 49 50 51 67 68 69 TO 71 72 4 2 1 Configuration According to the configuration of the Kieker Monitoring components see Section 3 2 plugins and repositories are configured via Configuration objects Classes must p
64. e files kieker 1 6_emf jar and sigar 1 6 4 jar from the binary distri bution to the example s 1ib directory 2 Additionally depending on the underlying system platform corresponding Sigar native libraries need to be placed in the example s 1ib directory Kieker s lib sigar native libs folder already includes the right libraries for 32 and 64 bit Lin ux Windows platforms Native libraries for other platforms can be downloaded from I D 2 Using the Sigar Based Samplers The Sigar API 1 provides access to a number of system level inventory and monitoring data e g regarding memory swap cpu file system and network devices Kieker in cludes Sigar based samplers for monitoring CPU utilization CPUsDetailedPercSampler CPUsCombinedPercSampler and memory swap usage MemSwapUsageSampler When registered as a periodic sampler Section 3 1 4 these samplers collect the data of interest employing the Sigar API and write monitoring records of types CPUUtilizationRecord ResourceUtilizationRecord and MemSwapUsageRecord respectively to the configured monitoring log stream Listing D 1 shows an excerpt from this example s MonitoringStarter which creates and registers two Sigar based peridioc samplers For reasons of performance and thread safety the SigarSamplerFactory should be used to create instances of the Sigar based Samplers Kieker 1 6 User Guide 70 38 39 40 41 42 43 44 45 46 47 48 49 50 51 fin
65. e reader returns from the execution of its read method the method terminate of each registered plugin is called by the AnalysisController Kieker Analysis configurations can be saved to a kax file by calling the AnalysisCon troller s saveToFile method The AnalysisController provides a constructor which accepts the file system location of a kax file to load the configuration from See Ap pendix A 3 and A 4 for included tools scripts which execute and visualize kax files In order to support the asynchronous execution of the AnalysisController instance we provide the AnalysisControllerThread class Kieker 1 6 User Guide 31 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 4 1 1 Programmatic Creation of Pipe and Filter Architectures To give a first impression of the programmatic instantiation configuration and con nection of plugins Listing 4 1 demonstrates this procedure for the example using MyP ipeReader and MyResponseTimeFilter according to Figure 4 1 The configuration for the MyPipeReader is created in lines 50 52 Using this config uration the reader is created in line 53 and registered to the AnalysisController in line 54 Similarly lines 57 64 initialize and register the MyResponseTimeFilter The reader s output is connected to the filter s input in line 65 The entire programmatic creation of the pipe and filter architecture shown in Figure can be found in the example file Starter j
66. eRecord values Object 3 initFromArray values Object 3 toArray Object 3 getValueTypes Class 3 Figure 3 3 Class diagram with the IMonitoringRecord and IMonitoringRecord Factory interfaces the abstract class AbstractMonitoringRecord and a custom mon itoring record type MyResponseTimeRecord In order to use the abstract class for implementing your own monitoring record type you need to 1 Create a class that extends AbstractMonitoringRecord 2 and a Override the methods initFromArray toArray get Value Types b For immutable record types implement IMonitoringRecord Factory a con structor with a single Class lt gt argument and a public static TYPES field In this case initFromArray which is not called by the framework then should throw an UnsupportedDperationException The class MyResponseTimeRecord shown in the class diagram in Figure and in Listing 3 2 is an example of a custom monitoring record type that can be used to mon itor response times of method executions Implementing IMonitoringRecord Factory MyResponseTimeRecord is an immutable type i e it includes only final fields Kieker 1 6 User Guide 24 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 public class MyResponseTimeRecord extends AbstractMonitoringRecord implements IMonit
67. ecords before finally any writing is done kieker monitoring writer filesystem SyncFsWriter flush true HHH kieker monitoring writer kieker monitoring writer filesystem AsyncF s Writer When flushing is disabled it could require a lot of records before finally any writing is done kieker monitoring writer filesystem AsyncFsWriter flush true In order to use the default temporary directory set the property value of storeInJavaloTmpdir to true kieker monitoring writer filesystem AsyncFsWriter storelnJavaloT mpdir true In order to use a custom directory set storeInJavaloTmpdir false and set customStoragePath as desired Examples var kieker or CAN KiekerData ensure the folder exists kieker monitoring writer filesystem AsyncFsWriter customStoragePath The maximal number of entries records per created file 4 Must be greater than zero kieker monitoring writer filesystem AsyncFsWriter maxEntriesInFile 25000 The maximal file size of the generated monitoring log Older files will be deleted if this file size is exceeded Given in MiB At least one file will always remain regardless of size Use 1 to ignore this functionality kieker monitoring writer filesystem AsyncFsWriter maxLogSize 1 The maximal number of log files generated Older files will be deleted if this number is exceeded At least one file w
68. er deliver OUTPUT PORT NAME RT EXCEED rtRecord else super deliver OUTPUT PORT NAME RT VALID rtRecord Listing 4 5 Plugin sending events to output ports Kieker 1 6 User Guide 35 29 30 31 32 33 34 35 36 37 38 39 40 Listing 1 4 shows also how the properties are declared Using the QProperty annota tion it is possible to declare the existing properties Each property has a default value which should be sufficient to initialize the plugin The Plugin annotation as well as the later introduced Repository annotation contains furthermore the two fields dependencies and programmaticOnly The first one offers the possibility to give a description of the needed dependencies for a plugin other libraries e g The latter marks whether the current plugin or repository is for pro grammatic purposes only i e they are of little use in graphical analysis tools 4 2 3 Developing Monitoring Reader Plugins Custom readers must extend the class AbstractReaderPlugin see Figure 4 2 and implement the methods init read and terminate which are called by the Analysis Controller to trigger the reader s initialization reading and termination Like each plugin Section 4 2 1 readers are configured via a constructor accepting a Configu ration object as its only argument they must provide the current configuration via the implemented getCurrentConfiguration method Readers start reading on invocation of the rea
69. ference on Performance Engineering ICPE 2012 pages 247 248 ACM Kieker 1 6 User Guide 96
70. figuration by Kieker Monitoring in both modes Servlet Filter Default The file web xml is located in the webapps s WEB INF direc tory Kieker s Servlet filters are already enabled lt filter gt lt filter name gt sessionAnd TraceRegistrationFilter lt filter name gt lt filter class gt kieker monitoring probe servlet SessionAndTraceRegistrationFilter lt filter class gt lt init param gt lt param name gt logFilterExecution lt param name gt lt param value gt true lt param value gt lt init param gt lt filter gt Kieker 1 6 User Guide 62 lt filter mapping gt lt filter name gt sessionAndTraceRegistrationFilter lt filter name gt lt url pattern gt lt url pattern gt lt filter mapping gt Listing B 1 Enabling the Servlet filter in web xml This filter can be used with both the Spring based and the AspectJ based instrumenta tion mode Spring based Instrumentation Default Kieker s Spring interceptor are already en abled in the file applicationContext xml located in the webapps s WEB INF direc tory lt bean id opEMII class kieker monitoring probe spring executions OperationExecutionMethodInvocationInterceptor gt lt aop config gt lt aop advisor advice ref opEMII pointcut execution public org mybatis jpetstore x x gt lt aop config gt Listing B 2 Enabling the Spring intercep
71. g adaptive Monitoring conf in the META INF directory of the binary release By default Kieker s adaptive monitoring feature is deactivated It can be enabled by setting the value of the configuration property kieker monitoring adaptiveMonitoring enabled in the kieker monitoring properties file to true The kieker monitoring properties includes additional properties to configure the adaptive monitoring e g the location of the afore mentioned file with include exclude patterns and the polling interval for this file Kieker 1 6 User Guide 22 3 2 Kieker Monitoring Configuration Kieker Monitoring instances can be configured by properties files Configuration objects and by passing property values as JVM arguments If no configuration is specified a default configuration is being used Appendix E lists this default configuration with a documentation of all available properties The default configuration properties file which can be used as a template for custom configurations is provided by the file kieker monitoring properties in the directory kieker 1 6 META INF of the binary release see Section 2 1 Configurations for Singleton Instances In order to use a custom configuration file its location needs to be passed to the JVM using the parameter kieker monitoring configuration as follows gt java Dkieker monitoring configuration lt ANY DIR gt my kieker monitoring properties Alternatively a file named
72. g white spaces C 1 ActiveMQ C 1 1 Download and Prepare ActiveMQ Download an ActiveMQ archive from http activemg apache org download html and decompress it to the base directory of the example Note that there are two different distributions one for Unix Linux Cygwin and another one for Windows Under UNIX like systems you ll need to set the executable bit of the start script gt chmod x bin activemq Also under UNIX like systems make sure that the file bin activemq includes UNIX line endings e g using your favorite editor or the dos uniz tool C 1 2 Copy Kieker and ActiveMQ Libraries Kieker Libraries Copy the following file from Kieker s binary distribution to the exam ple s 1ib directory 1 kieker 1 6_emf jar from dist Kieker 1 6 User Guide 65 ActiveMQ Libraries Copy the following files from the ActiveMQ release to the 1ib directory of this example 1 activemg all lt version gt jar from ActiveMQ s base directory 2 slf4j log4j lt version gt jar from ActiveMQ s lib optional directory 3 log4j lt version gt jar from ActiveMQ s 1ib optional directory C 1 3 Kieker Monitoring Configuration for ActiveMQ The file examples userguide appendix JMS META INF kieker monitoring properties activeMQ is already configured to use the AsyncJMSWriter via ActiveMQ The important properties are the definition of the provider URL and the context factory kieker monitoring writer jms As
73. gController Listing 5 6 demonstrates how to integrate the SessionAndTraceRegistrationFilter in the web xml file of a web application The Java EE Servlet container example described in Appendix B employs the Ses sionAndTraceRegistrationFilter Kieker 1 6 User Guide 44 50 51 52 53 54 55 56 57 58 59 60 61 lt filter gt lt filter name gt sessionAndTraceRegistrationFilter lt filter name gt lt filter class gt kieker monitoring probe servlet SessionAndTraceRegistrationFilter lt filter class gt lt init param gt lt param name gt logFilterExecution lt param name gt lt param value gt true lt param value gt lt init param gt lt filter gt lt filter mapping gt lt filter name gt sessionAnd TraceRegistrationFilter lt filter name gt lt url pattern gt lt url pattern gt lt filter mapping gt Listing 5 6 OperationExecutionRegistrationAndLoggingFilter in a web xml file 5 1 3 Spring The Spring framework provides interfaces for intercepting Spring services and web requests Kieker includes the probes OperationExecutionMethodInvocationInter ceptor and OperationExecutionWebRequestRegistrationInterceptor The Opera tionExecutionMethodInvocationInterceptor is similar to the AspectJ based probes described in the previous section and monitors method executions as well as corre sponding trace and session information The OperationExecutionWebReq
74. ggingTimestamp bat timestamps 1283156545581511026 1283156546127117246 1283156545581511026 Mo 30 Aug 2010 08 22 25 0000 UTC Mo 30 Aug 2010 10 22 25 0200 local time 1283156546127117246 Mo 30 Aug 2010 08 22 26 0000 UTC Mo 30 Aug 2010 10 22 26 0200 local time Listing A 2 Execution under Windows Kieker 1 6 User Guide 56 A 2 Script logReplay sh bat Replays filesystem monitoring logs created by Kieker Monitoring Example applications are e Merging multiple directories containing monitoring data into a single output di rectory e Importing a filesystem monitoring log to another monitoring log e g a database Therefore an appropriate Kieker Monitoring configuration file must be passed to the script see Section 3 2 e Replaying a recorded filesystem monitoring log in real time in order to simulate incoming monitoring data from a running system e g via JMS see also Ap pendix C Main class kieker tools logReplayer FilesystemLogReplayerStarter Usage usage kieker tools logReplayer FilesystemLogReplayerStarter i inputdirs lt dirl dirN gt Log directories to read data from ignore records after date lt yyyyMMdd HHmmss gt Records logged after this date UTC timezone are ignored disabled by default ignore records before date lt yyyyMMdd HHmmss gt Records logged before this date UTC timezone are ignored disabled by default k keep
75. h3 4 custom components build dir build src dir src lib dir lib meta dir META INF jar file BookstoreApplication jar properties file kieker monitoring properties jar file kieker kieker _emf jar main class kieker examples userguide ch3and4bookstore Starter meta dir META INF msg filesNotFound One or more of the required libraries could not be found Please add the following files to the lib dir directory jar file kieker Listing H 4 build properties lt project name Bookstore Application basedir default run gt lt property file build properties gt lt path id classpath gt lt fileset dir S lib dir includes x jar gt lt path gt lt condition property not all files available gt lt resourcecount property num requiredLibsFound when ne count 1 gt lt fileset dir lib dir gt lt include name jar file kieker 1 gt lt fileset gt lt resourcecount gt lt condition gt lt target name check files if not all files available gt lt fail message msg filesNotFound gt lt target gt lt target name run depends check files build jar gt lt java fork true classname main class gt lt classpath gt lt path refid classpath gt Kieker 1 6 User Guide 90 lt path location S jar file gt lt classpath gt lt
76. hich is often not desired since the resulting code is hardly maintainable Many middleware technologies such as Java EE Servlet 7 Spring 8 and Apache CXF 9 provide interception AOP 2 interfaces Kieker 1 6 User Guide 26 which are well suited to implement monitoring probes AspectJ allows to instrument Java applications without source code modifications Chapter 5 describes the Kieker probes based on these technologies allowing to monitor trace information in distributed applications 3 5 Monitoring Writers Monitoring writers serialize monitoring records to the monitoring log stream and must implement the interface IMonitoringWriter The monitoring controller passes the re ceived records to the writer by calling the method newMonitoringRecord Writers can use the methods to serialize the record contents as described in Section 3 3 Figure shows the monitoring writers already implemented in Kieker Monitoring The available properties for the included writers are well documented in the example configuration file see Appendix E The writers AsyncFsWriter SyncFsWriter AsyncDbWriter and SyncDbWriter can be used to store monitoring records to filesystems and databases respectively The vari ants with the prefix Async are implemented using asynchronous threads that decouple the I O operations from the control flow of the instrumented application The AsyncF sWriter is the default writer which has already been used in Section
77. ieker monitoring name KIEKER 7 Whether a debug mode is activated This changes a few internal id generation mechanisms to enable easier debugging Additionally it is possible to enable debug logging in the settings of the used logger This setting should usually not be set to true kieker monitoring debug false Enable disable monitoring after startup truel false default true If monitoring is disabled the MonitoringController simply pauses Furthermore probes should stop collecting new data and monitoring writers stop should stop writing existing data kieker monitoring enabled true The name of the VM running Kieker If empty the name will be determined automatically else it will be set to the given value kieker monitoring hostname The initial ID associated with all experiments currently not used kieker monitoring initialExperimentld 1 Kieker 1 6 User Guide 73 Enables disable the automatic assignment of each records logging timestamp A truelfalse default true kieker monitoring setLogging Timestamp true Whether a shutdown hook should be registered This ensures that necessary cleanup steps are finished and no information is lost due to asynchronous writers This should usually not be set to false kieker monitoring useShutdownHook true Whether any JMX functionality is available kieker monitoring jmx false kieker monitoring
78. ieker 1 6 User Guide 63 3 Browse through the application to generate some monitoring data This data can be analyzed and visualized using Kieker TraceAnalysis as described in Chapter 5 JPetStore Demo Mozilla Firefox Jog File Edit View History Bookmarks Tools Help 3 JPetStore Demo EI lt localhost acti atal 4 y o JPetStore Fish Saltwater Freshwater Dogs Various Breeds Various Breeds Exotic Varieties Reptiles Lizards Turtles Snakes Birds Exotic Varieties Figure B 1 MyBatis JPetStore Kieker 1 6 User Guide 64 C Using the JMS Writer and Reader This chapter gives a brief description on how to use the AsyncJMSWriter and JMSReader classes The directory examples userguide appendix JMS contains the sources ant scripts etc used in this example It is based on the Bookstore application with manual instrumentation presented in Chapter 2 The following sections provide step by step instructions for the JMS server imple mentations ActiveMQ Section C 1 HornetQ C 2 and OpenJMS C 3 The general procedure for each example is the following 1 Download and prepare the respective JMS server implementation Copy required libraries to the example directory Start the JMS server Start the analysis instance which receives records via JMS Start the monitoring instance which sends records via JMS ork Ww N Due to a bug in some JMS servers avoid paths includin
79. ill always remain regardless of size Use 1 to ignore this functionality kieker monitoring writer filesystem AsyncFsWriter maxLogFiles 1 Asynchronous writers need to store monitoring records in an internal buffer This parameter defines its capacity in terms of the number of records kieker monitoring writer filesystem AsyncFsWriter QueueSize 10000 Behavior of the asynchronous writer when the internal queue is full 0 terminate Monitoring with an error default 1 writer blocks until queue capacity is available Kieker 1 6 User Guide TT 2 writer discards new records until space is available Be careful when using the value 1 since then the asynchronous writer is no longer decoupled from the monitored application kieker monitoring writer filesystem AsyncFsWriter QueueFullBehavior 0 Maximum time to wait for the writer threads to finish in milliseconds A MaxShutdownDelay of 1 means infinite waiting kieker monitoring writer filesystem AsyncFsWriter MaxShutdownDelay 1 HHH kieker monitoring writer kieker monitoring writer filesystem AsyncBinaryF s Writer In order to use the default temporary directory set the property value of storeInJavaloTmpdir to true kieker monitoring writer filesystem AsyncBinaryFsWriter storelnJavaloT mpdir true In order to use a custom directory set storeInJavaloTmpdir false and
80. inary release 4 1 Pipe and Filter Framework and Included Plugins Kieker Analysis provides a framework to define and execute pipe and filter architectures of analysis plugins i e monitoring readers and analysis filters as well as repositories This section describes how to use and develop readers filters and repositories The description is based on the example pipe and filter architecture shown in Figure 4 1 The custom monitoring reader MyPipeReader which corresponds to the writer developed in Section sends records to the connected custom filter MyResponseTimeFilter This filter accepts only events of the record type MyResponseTimeRecord developed in Section The MyResponseTimeFilter classifies incoming MyResponseTimeRecords based on whether they satisfy or exceed a configured threshold and passes them to the respective output ports validResponseTimes or invalidResponseTimes Two instances of a second custom filter MyResponseTimeDutputPrinter print the received records to the standard output stream newEvent validResponse Times lt lt Filter gt gt Print valid Reponse time printer lt lt Filter gt gt Response time filter outputPort newResponse Time invalidResponse Times lt lt Filter gt gt LI print invalid Reponse time printer newEvent Figure 4 1 Example pipe and filter configuration Kieker 1 6 User Guide 30 AnalysisController from kieker analysis
81. ing writer PrintStreamWriter Stream STDOUT HARE kieker monitoring writer kieker monitoring writer filesystem SyncFs Writer In order to use the default temporary directory set the property value of storelnJavalo Tmpdir to true kieker monitoring writer filesystem SyncFsWriter storelnJavaloT mpdir true In order to use a custom directory set storeInJavaloTmpdir false and set customStoragePath as desired Examples var kieker or CAN KiekerData ensure the folder exists kieker monitoring writer filesystem SyncFsWriter customStoragePath The maximal number of entries records per created file 4 Must be greater than zero kieker monitoring writer filesystem SyncFsWriter maxEntrieslnFile 25000 The maximal file size of the generated monitoring log Older files will be deleted if this file size is exceeded Given in MiB At least one file will always remain regardless of size Use 1 to ignore this functionality kieker monitoring writer filesystem SyncFsWriter maxLogSize 1 Kieker 1 6 User Guide 76 The maximal number of log files generated Older files will be deleted if this number is exceeded At least one file will always remain regardless of size Use 1 to ignore this functionality kieker monitoring writer filesystem SyncFsWriter maxLogFiles 1 When flushing is disabled it could require a lot of r
82. ionExecutionRecord from kieker common record componentName String opName String sessionld String traceld long tin long tout long eoi int ess int Figure 5 1 The class diagram of the operation execution record Section 5 1 describes how to instrument Java applications for monitoring trace informa tion It presents the technology specific probes provided by Kieker for this purpose with a focus on AspectJ Additional technology specific probes can be implemented based on the existing probes Section presents the tool which can be used to analyze and visualize the recorded trace data Examples for the available analysis and visualization outputs provided by Kieker TraceAnalysis are presented in Section 5 3 Kieker 1 6 User Guide AO 5 1 Monitoring Trace Information The following Sections describe how to use the monitoring probes based on AspectJ Sec tion 5 1 1 the Java Servlet API Section 5 1 2 the Spring Framework Section gt and Apache CXF Section 5 1 4 provided by Kieker 5 1 1 AspectJ Based Instrumentation AspectJ allows to weave code into the byte code of Java applications and libraries without requiring manual modifications of the source code Kieker includes the AspectJ based monitoring probes OperationExecutionAspectAnnotation OperationExecu tionAspectAnnotationServlet OperationExecutionAspectFull and OperationEx ecutionAspectFullServ
83. ir lib meta dir META INF jar file BookstoreApplication jar properties file kieker monitoring properties jar file kieker kieker 1 6_aspectj jar main class kieker examples userguide ch5bookstore BookstoreStarter main class hostname rewriter kieker examples userguide ch5bookstore BookstoreHostnameRewriter meta dir META INF msg filesNotFound One or more of the required libraries could not be found Please add the following files to the lib dir directory jar file kieker Listing H 6 build properties lt project name Tutorial Example basedir default run gt lt property file build properties gt lt path id classpath gt lt fileset dir lib dir includes x jar gt lt path gt lt condition property not all files available gt lt resourcecount property num requiredLibsFound when ne count 1 gt lt fileset dir S lib dir gt lt include name jar file kieker gt lt fileset gt lt resourcecount gt lt condition gt lt target name check files if not all files available gt lt fail message msg_filesNotFound gt lt target gt lt target name run depends check files build jar lt java fork true classname main class gt lt classpath gt lt path refid classpath gt lt path location S jar file gt lt classpath gt
84. is responseTimeNanos Listing 3 2 MyResponseTimeRecord java 3 4 Monitoring Probes The probes are responsible for collecting the monitoring data and passing it to the monitoring controller In Chapter 2 3 we have already demonstrated how to manually instrument a Java application Listing shows a similar manual monitoring probe which uses the monitoring record type MyResponseTimeRecord defined in the previous Section final long tin MONITORING CONTROLLER getTimeSource getTime this catalog getBook false final long tout MONITORING CONTROLLER getTimeSource getTime Create a new record and set values final MyResponseTimeRecord e new MyResponseTimeRecord mySimpleKiekerExample bookstoreTracing Catalog getBook tout tin Pass the record to the monitoring controller MONITORING CONTROLLER newMonitoringRecord e Listing 3 3 Excerpt from Bookstore java In order to avoid multiple calls to the getInstance method of the MonitoringController class singleton instances should be stored in a final static variable as shown in List ing 3 4 private static final IMonitoringController MONITORING CONTROLLER MonitoringController getInstance Listing 3 4 Singleton instance of the monitoring controller stored in a final static variable excerpt from Bookstore java When manually instrumenting an application the monitoring probe is implemented by mixing monitoring logic with business logic w
85. isting 4 10 shows the Repository annotation of the SystemModelRepository which as included in Kieker as part of the Kieker TraceAnalysis tool Repository name System model repository description Model manager for Kieker s component model public class SystemModelRepository extends AbstractRepository Listing 4 10 Respository annotation of Kieker s SystemModelRepository Plugins specify the list of required repositories in their Plugin annotation Reposito ries are connected to filter provided repository ports A plugin s repository ports are specified using the nested RepositoryPort annotation as depicted for a Kieker Trace Analysis filter in Listing Like for input and output port names this name must be unique for the plugin and should be provided as a public static constant with prefix REPOSITORY_PORT_NAME_ in order to ease the programmatic connection of repositories to readers and filters Plugin repositoryPorts RepositoryPort mame AbstractTraceAnalysisFilter REPOSITORY_PORT_NAME_SYSTEM_MODEL repositoryType SystemModelRepository class public abstract class AbstractTraceAnalysisFilter extends AbstractFilterPlugin Listing 4 11 Declaration of required repositories in the Respository annotation Kieker 1 6 User Guide 38 Plugins can access their connected repositories via the getRepository method provided by the super class as shown in Listing 4 12 147 this systemEntityFact
86. jmx domain kieker monitoring Enable Disable the MonitoringController MBean kieker monitoring jmx MonitoringController true kieker monitoring jmx MonitoringController name MonitoringController Controls JMX remote access kieker monitoring jmx remote false kieker monitoring jmx remote port 59999 kieker monitoring jmx remote name JMXServer If the SUN JMX Bootstrap class is not available a fallback to the default implementation can be used The fallback solution prevents the VM from terminating A graceful shutdown is done by connecting to the JMXServer there to kieker monitoring JMXServer and using the operation stop kieker monitoring jmx remote fallback true These properties will be forwarded to configure the JMX server com sun management jmxremote local only false com sun management jmxremote authenticate false com sun management jmxremote ssl false The size of the thread pool used to execute registered periodic sensor jobs The thread pool is also used to periodically read the config file for adaptive ZH monitoring xx Set to O to deactivate scheduling kieker monitoring periodicSensorsExecutorPoolSize 1 Enable or disable adaptive monitoring kieker monitoring adaptiveMonitoring enabled false Default location of the adaptive monitoring configuration File kieker monitoring adaptiveMonitoring configFile META INF kieker monitoring adaptiveMonitori
87. ker examples userguide ch5bookstore java d build classpath lib kieker 1 6_aspectj jar gt cp META INF aop xml build META INF gt cp META INF kieker monitoring properties build META INF gt java javaagent lib kieker 1 6_aspectj jar classpath build kieker examples userguide ch5bookstore BookstoreStarter Listing 5 3 Commands to compile and run the Bookstore under UNIX like systems gt mkdir build gt mkdir build META INF gt javac src kieker examples userguide ch5bookstore java d build classpath lib kieker 1 6_aspectj jar gt copy META INF aop xml build META INF gt copy META INF kieker monitoring properties build META INFY gt java javaagent lib kieker 1 6_aspectj jar classpath build kieker examples userguide ch5bookstore BookstoreStarter Listing 5 4 Commands to compile and run the annotated Bookstore under Windows After a complete run of the application the monitoring files should appear in the same way as mentioned in Section 2 3 including the additional trace information An example log of a complete run can be found in Appendix 6 2 Instrumentation without annotations AspectJ based instrumentation without using annotations is quite simple It is only necessary to modify the file aop xm1 as shown in Listing 5 5 Kieker 1 6 User Guide 43 Ei GO ONOU APUN 10 11 lt DOCTYPE aspect PUBLIC AspectJ DTD EN http www aspec
88. ker 1 6 User Guide 27 Listing 3 5 shows a custom writer MyPipeWriter which uses a named pipe to write the given records into a buffer located in the memory The source code of the class MyPipe is listed in Appendix F 1 23 public class MyPipeWriter extends AbstractMonitoringWriter 1 24 25 public static final String CONFIG PROPERTY NAME PIPE NAME 26 MyPipeWriter class getName pipeName 27 28 private volatile MyPipe pipe 29 private final String pipeName 30 31 public MyPipeWriter final Configuration configuration 32 super configuration 33 this pipeName 34 configuration getStringProperty CONFIG PROPERTY NAME PIPE NAME 35 36 37 public boolean newMonitoringRecord final IMonitoringRecord record 38 aay dl 39 Just write the content of the record into the pipe 40 this pipe put new PipeData record getLoggingTimestamp 41 record toArray record getClass 42 catch final InterruptedException e 43 return false signal error 44 45 return true 46 47 48 Override 49 protected Configuration getDefaultConfiguration 1 50 final Configuration configuration new Configuration super getDefaultConfiguration 51 configuration setProperty CONFIG PROPERTY NAME PIPE NAME kieker pipe 52 return configuration 53 54 55 Override 56 protected void init throws Exception 57 this pipe MyNamedPipeManager getInstance acquirePipe this pipeName
89. l monitoring probes called samplers Samplers must implement the interface ISampler which includes a single method sample IMonitoringController monitoringController This method is called in periodic time steps as specified by the MonitoringController s registration function schedulePeriodicSampler Periodic samplers can be stopped by calling the MonitoringController s method removeScheduledSampler Listing 3 1 shows the sample method of the MemSwapUsageSampler which can be used to monitor memory and swap usage employing the Sigar library I Likewise to other monitoring probes described in this user guide see for example Sections and 2 3 it collects the data of interest lines 53 54 creates a monitoring record lines 55 58 and passes this monitoring record to the monitoring controller line 59 The available Sigar based samplers for monitoring system level monitoring data such as CPU and memory usage are discussed in Appendix D Kieker 1 6 User Guide 20 public void sample final IMonitoringController monitoringCtr throws SigarException if monitoringCtr isMonitoringEnabled return i final Mem mem this sigar getMem final Swap swap this sigar getSwap final MemSwapUsageRecord r new MemSwapUsageRecord monitoringCtr getTimeSource getTime monitoringCtr getHostname mem getTotal mem getActualUsed mem getActualFree swap getTotal swap getUsed swap getFree monitoringCtr
90. lected abbreviated labels e g package names are used in the visualizations The following commands generate a deployment level operation dependency graph and convert it to pdf format gt bin trace analysis sh inputdirs examples userguide ch5 trace monitoring aspect testdata kieker 20100830 082225522 UTC outputdir plot Deployment Operation Dependency Graph gt dot T pdf deploymentOperationDependencyGraph dot gt deploymentOperationDependencyGraph pdf Listing A 5 Execution under UNIX like systems gt trace analysis bat inputdirs examples userguide ch5 trace monitoring aspectj testdata kieker 20100830 082225522 UT C outputdir plot Deployment Operation Dependency Graph gt dot T pdf deploymentOperationDependencyGraph dot gt deploymentOperationDependencyGraph pdf Listing A 6 Execution under Windows Additional examples can be found in Chapter 5 A 6 Script dotPic fileConverter sh bat Converts each dot and pic file e g diagrams generated by Kieker TraceAnalysis Sec tion B located in a directory into desired graphic output formats This scripts simply calls the Graphviz and PlotUtils tools dot and pic2plot Kieker 1 6 User Guide 60 Usage Usage dotPic fileConverter sh lt output directory gt lt file type 1 file type N gt Example The following command converts each dot and pic file located in the directory out to files in
91. let which can be woven into Java applications at compile time and load time These probes monitor method executions and corresponding trace and timing information The probes with the postfix Servlet additionally store a session identifier within the OperationExecutionRecord When the probes with name element Annotation are used methods to be monitored must be annotated by the Kieker anno tation OperationExecutionMonitoringProbe This section demonstrates how to use the AspectJ based probes to monitor traces based on the Bookstore application from Chapter The Java sources of the example presented in this section as well as a pre compiled binary can be found in the examples userguide ch5 trace monitoring aspectj directory of the binary release examples userguide ch5 trace monitoring aspectj WES build AE EE EE tei Directory for the Java class files META INF ISMETA INE RR pa Directory for the configuration files L aop xm kieker monitoring properties NEEN D A E E NE A ER eA ren era eae Directory for the needed libraries kieker 1 6_aspectj jar Este AE OE O ree eee Directory for the source code files bookstoreTracing Bookstore java BookstoreStarter java Catalog java CRM java Figure 5 2 The new directory structure of the Bookstore application Kieker 1 6 User Guide A1 21 22 23 24 25 26 27 28 29 30 31 Ha oono AUN 11 Figure 5 2 shows the directory used by the example
92. lt arg line num requests gt Kieker 1 6 User Guide 92 lt jvmarg value javaagent lib dir jar file kieker gt lt jvmarg value Dorg aspectj weaver showWeavelnfo true gt lt jvmarg value Da weaving verbose true gt lt java gt lt target gt lt target name run event depends check files build jar gt lt java fork true classname main class gt lt classpath gt lt path refid classpath gt lt path location S jar file gt lt classpath gt lt arg line num requests gt lt jvmarg value javaagent lib dir jar file kieker gt lt jvmarg value Dorg aspectj weaver showWeavelnfo true gt lt jvmarg value Daj weaving verbose true gt lt jvmarg value Dkieker monitoring writer filesystem AsyncFsWriter QueueSize 30000 gt lt jvmarg value Dorg aspectj weaver loadtime configuration META INF aop event xml gt lt java gt lt target gt lt target name run hostname rewriter depends check files build jar gt lt java fork true classname main class hostname rewriter gt lt arg line analysis directory gt lt classpath gt lt path refid classpath gt lt path location S jar file gt lt classpath gt lt java gt lt target gt lt target name build jar depends compile
93. lt target name build jar analysis depends clean analysis compile gt lt jar destfile jar file analysis basedir build dir W gt lt manifest gt lt attribute name Main Class value main class analysis gt lt manifest gt lt jar gt lt delete dir build dir gt lt target gt lt target name compile depends init gt Kieker 1 6 User Guide 88 lt javac source 1 5 target 1 5 srcdir fsrc dir destdir build dir classpathref classpath includeAntRuntime false gt lt mkdir dir build dir meta dir gt lt target gt lt target name init delete dir build dir gt lt mkdir dir build dir gt lt target gt lt target name clean monitoring delete file jar file monitoring gt lt target gt lt target name clean analysis gt lt delete file fjar file analysis gt lt target gt lt target name clean depends clean monitoring clean analysis gt lt target gt lt project gt Listing H 3 build xml Kieker 1 6 User Guide H 2 Custom Components Chapters 3 and 4 The following build xml and build properties files can be used for compiling and executing the manually instrumentated Bookstore application with the custom com ponents as described in Chapters 3 and The files are included in the directory examples userguide c
94. lysis 4 Plugin Kieker TraceAnalysis Analysis Controller Monitoring Controller Monitoring Record Monitoring Log Stream Monitoring Reader Monitoring Writer Figure 1 1 Overview of the framework components directly if you are interested in Kieker support for other platforms Kieker 1 6 User Guide 3 The Kieker Monitoring component is responsible for program instrumentation data col lection and logging Its core is the MonitoringController The component Kieker Analysis is responsible for reading analyzing and visualizing the monitoring data Its core is the AnalysisController which manages the life cycle of the pipe and filter ar chitecture of analysis plugins including monitoring readers and analysis filters The monitoring and analysis parts of the Kieker framework are composed of subcom ponents which represent the different functionalities of the monitoring and analysis tasks The important interaction pattern among the components is illustrated in Figure 1 2 but will be explained furthermore throughout the course of this user guide 1 2 newMonitoringRecord record MonitoringController AnalysisController 1 7 7 1 E EET 4 2 inputRecord record EE IMonitoringProbe 2 newMonitoringRecord record l TRE IFilterPlugin 1 1e j IMonitoringWriter IMonitoringReaderPlugin 1 1 i
95. mple the filesystem writer is used Kieker uses this writer by default when no other writer is specified as detailed in Section 3 5 In addition to the instrumentation in the Bookstore class the getOffers method of the CRM class is instrumented as well Similar to Listing measurements are taken before and after the call of the catalog s getBook method as shown in lines 36 and 38 of Listing 2 8 Not shown in the listing is the instantiation of the monitoring controller However it is done in the same way as illustrated in Listing 2 7 Finally a record is created see lines 40 46 and stored by calling the monitoring controller see line 47 public void getOffers 1 1 Call Catalog getBook and log its entry and exit timestamps final long tin MONITORING CONTROLLER getTimeSource getTime this catalog getBook false lt the monitored execution final long tout MONITORING CONTROLLER getTimeSource getTime final OperationExecutionRecord e new OperationExecutionRecord public void this catalog getClass getName getBook boolean OperationExecutionRecord NO SESSION ID OperationExecutionRecord NO_TRACEID tin tout myHost OperationExecutionRecord NO_EOI_ESS OperationExecutionRecord NO_EOI_ESS MONITORING CONTROLLER newMonitoringRecord e i Listing 2 8 Instrumentation of the getBook call in CRM java The next step after instrumenting the code is running the instrumented application
96. n these methods would pass objects to ensure the results of the catalog search will be available to the offer collecting method However for our example we omitted such code 19 public class Bookstore 20 21 private final Catalog catalog new Catalog 22 private final CRM crm new CRM this catalog 23 24 public void searchBook 25 this catalog getBook false 26 this crm getOffers 27 28 Listing 2 2 Bookstore java The customer relationship management for this application is modeled in the CRM class shown in Listing It only provides a business method to collect offers which uses the catalog for some lookup The additional catalog lookup is later used to illustrate different traces in the application 19 public class CRM 20 private final Catalog catalog 21 22 public CRM final Catalog catalog 23 this catalog catalog 24 25 26 public void getOffersO 27 this catalog getBook false 28 29 5 Listing 2 3 CRM java Finally the class Catalog is shown in Listing 2 4 It resembles the catalog component in the application 19 public class Catalog 1 20 21 public void getBook final boolean complexQuery 22 nothing to do here 23 24 5 Listing 2 4 Catalog java Kieker 1 6 User Guide 10 After this brief introduction of the application and its implementation the next step is to see the example running To compile and run the example the commands in Listing 2 5 can be
97. name Examples ZH MySQL com mysgl jdbc Driver DerbyDB org apache derby jdbc EmbeddedDriver kieker monitoring writer database SyncDbWriter DriverClassname org apache derby jdbc EmbeddedDriver Connection string Examples MySQL jdbe mysgl HOSTNAME DBNAME user DBUSER amp password DBPASS DerbyDB jdbe derby DBNAME user DBUSER password DBPASS kieker monitoring writer database SyncDbWriter ConnectionString jdbc derby tmp KIEKER user DBUSER password DBPASS create true Prefix for the names of the database tables kieker monitoring writer database SyncDbWriter TablePrefix kieker Drop already existing tables or terminate monitoring with an error kieker monitoring writer database SyncDbWriter DropTables false HEHE kieker monitoring writer kieker monitoring writer database AsyncDb Writer Database driver classname HH MySQL com mysgl jdbc Driver DerbyDB org apache derby jdbc EmbeddedDriver kieker monitoring writer database AsyncDbWriter DriverClassname org apache derby jdbc EmbeddedDriver Connection string Examples MySQL jdbc mysql HOSTNAME DBNAME user DBUSER amp password DBPASS DerbyDB jdbc derby DBNA ME user DBUSER password DBPASS kieker monitoring writer database AsyncDbWriter ConnectionString jdbc derby tmp KIEKER user DBUSER password DBPASS create true HH Prefix for the names of the database t
98. ng conf Enable disable the updating of the pattern file by activating or deactivating probes through the api kieker monitoring adaptiveMonitoring updateConfigFile false Kieker 1 6 User Guide 74 The delay in seconds in which the pattern file is checked for changes Requires kieker monitoring periodicSensorsExecutorPoolSize gt 0 xx Set to O to disable the observation kieker monitoring adaptiveMonitoring readlnterval 30 HPPA TIMER HAHAHAH Selection of the timer used by Kieker classname The value must be a fully qualified classname of a class implementing kieker monitoring timer ITimeSource and providing a constructor that accepts a single Configuration kieker monitoring timer kieker monitoring timer SystemNano Timer HHH kieker monitoring timer kieker monitoring timer SystemMilli Timer A timer with millisecond precision The offset of the timer The time returned is since 1970 1 1 minus this offset If the offset is empty it is set to the current time The offset must be specified in milliseconds kieker monitoring timer SystemMilliTimer offset 0 The timeunit used to report the timestamp Accepted values 0 nanoseconds 1 microseconds 2 milliseconds 3 seconds kieker monitoring timer System MilliTimer unit 0 HHH kieker monitoring timer kieker monitoring timer Sy
99. ods initFromArray toArray getValueTypes are used for serialization and de serialization of the monitoring data contained in the record Alternatively in order Kieker 1 6 User Guide 23 to support the definition of immutable record types the marker interface IMonitor ingRecord Factory needs to be implemented requiring the implementation of i the toArray method as before ii a constructor accepting a values array and iii a pub lic static TYPES field The method setLoggingTimestamp is used by the monitoring controller to store the date and time when a record is received by the controller The method getLoggingTimestamp can be used during analysis to retrieve this value Kieker Monitoring provides the abstract class AbstractMonitoringRecord Figure which already implements the methods to maintain the logging timestamp interface AbstractMonitoringRecord IMonitoringRecord from kieker common record from kieker common record setLoggingTimestamp timestamp long setLoggingTimestamp timestamp long getLoggingTimestamp long getLoggingTimestamp long initFromArray values Object 3 MyResponseTimeRecord toArray Object getValueTypes Class j TYPES Class lt gt className String interface methodName String IMonitoringRecord Factory responseTimeNanos long from kieker common record MyResponseTim
100. of this section The jar file kieker 1 6 aspectj jar already includes the AspectJ weaver which is registered with the JVM and weaves the monitoring instrumentation into the Java classes It will be configured based on the configuration file aop xm1 for which a working sample file is provided in the example s META INF directory Instead of registering the kieker 1 6 aspectj jar as an agent to the JVM the aspectjweaver 1 7 0 jar can be used In this case the kieker 1 6 jar needs to be added to the classpath Once the necessary files have been copied to the example directory the source code can be instrumented with the annotation OperationExecutionMonitoringProbe List ing 5 1 shows how the annotation is used public class Bookstore private final Catalog catalog new Catalog private final CRM crm new CRM this catalog COperationExecutionMonitoringProbe public void searchBook this catalog getBook false this crm getOffers Listing 5 1 Bookstore java As a first example each method of the Bookstore application will be annotated The annotation can be used to instrument all methods except for constructors The aop xml1 file has to be modified to specify the classes to be considered for instru mentation by the AspectJ weaver Listing 5 2 shows the modified configuration file lt DOCTYPE aspect PUBLIC AspectJ DTD EN http www aspectj org dtd aspectj_1_5_0 dtd gt lt aspect gt
101. onfiguration ConfigurationFactory loadConfigurationFromResource WARNING File META INF kieker monitoring properties not found in classpath Apr 2 2012 6 33 14 PM kieker monitoring core controller MonitoringController createlnstance INFO Current State of kieker monitoring 1 5 Status enabled Name KIEKER SINGLETON Hostname myHost experimentID 1 JMXController JMX disabled RegistryController O strings registered TimeSource kieker monitoring timer SystemNanoTimer Time in nanoseconds with nanoseconds precision since Thu Jan 01 01 00 00 CET 1970 WriterController Number of Inserts 0 Automatic assignment of logging timestamps true Writer kieker monitoring writer filesystem AsyncFsWriter Configuration kieker monitoring writer filesystem AsyncFsWriter QueueFullBehavior 0 kieker monitoring writer filesystem AsyncFsWriter flush true kieker monitoring writer filesystem AsyncFsWriter QueueSize 10000 kieker monitoring writer filesystem AsyncFsWriter customStoragePath kieker monitoring writer filesystem AsyncFsWriter MaxShutdownDelay 1 kieker monitoring writer filesystem AsyncFsWriter storelnJavaloT mpdir true kieker monitoring writer filesystem AsyncFsWriter maxEntriesInFile 25000 Records lost O Writer Threads 1 Finished false Writing to Directory tmp kieker 20120402 163314855 UT C myHost
102. oringRecord Factory private static final long serialVersionUID 1775L private static final Class lt gt TYPES String class String class long Glassmys Attributes storing the actual monitoring data private final String className private final String methodName private final long responseTimeNanos public MyResponseTimeRecord final String clazz final String method final long rtNano 1 this className clazz this methodName method this responseTimeNanos rtNano public MyResponseTimeRecord final Object values AbstractMonitoringRecord checkArray values MyResponseTimeRecord TYPES this className String values 0 this methodName String values 1 this responseTimeNanos Long values 2 Deprecated Will not be used because the record implements IMonitoringRecord Factory public final void initFromArray final Object values throw new UnsupportedOperationException public final Object toArray 1 return new Object this className this methodName this responseTimeNanos public Class lt gt getValueTypes 1 return MyResponseTimeRecord TYPES clone public String getClassName return this className Kieker 1 6 User Guide 25 64 65 66 67 68 69 TO 71 72 73 32 33 34 35 36 37 38 39 24 25 public String getMethodName return this methodName public long getResponseTimeNanos return th
103. ory SystemModelRepository 148 this getRepository REPOSITORY PORT NAME SYSTEM MODEL Listing 4 12 Accessing a repository within in a plugin Kieker 1 6 User Guide 39 5 Kieker TraceAnalysis Tool Kieker TraceAnalysis implements the special feature of Kieker allowing to monitor ana lyze and visualize distributed traces of method executions and corresponding timing information For this purpose it includes monitoring probes employing AspectJ 10 Java EE Servlet 7 Spring 8 and Apache CXF 9 technology Moreover it allows to reconstruct and visualize architectural models of the monitored systems e g as sequence and dependency diagrams Section 2 already introduced parts of the monitoring record type OperationExecu tionRecord Kieker TraceAnalysis uses this record type to represent monitored executions and associated trace and session information Figure 5 1 shows a class diagram with all attributes of the record type OperationExecutionRecord The attributes className operationName tin and tout have been introduced before The attributes traceld and sessionld are used to store trace and session information eoi and ess contain control flow information needed to reconstruct traces from monitoring data For details on this please refer to our technical report 11 interface IMonitoringRecord from kieker common record A L AbstractMonitoringRecord from kieker common record Leenen Operat
104. phviz A graph visualization software which can be down loaded from http www graphviz org 2 GNU PlotUtils A set of tools for generating 2D plot graphics which can be downloaded from http www gnu org software plotutils for Linux and from http gnuwin32 sourceforge net packages plotutils htm for Windows 3 ps2pdf The ps2pdf tool is used to convert ps files to pdf files Under Windows it is recommended to add the bin directories of both tools to the path environment variable A af Once both programs have been installed the Kieker TraceAnalysis tool can be used It can be accessed via the wrapper script trace analysis sh or trace analysis bat Windows in the bin directory Non parameterized calls of the scripts print all possible options on the screen as listed in Appendix A 5 The commands shown in Listings and generate a seguence diagram as well as a call tree to an existing directory named out The monitoring data is assumed to be located in the directory tmp kieker 20110428 142829399 UTC Kaapstad KIEKER or htemphlkieker 20100813 121041532 UTC virus KIEKER under Windows gt trace analysis sh inputdirs tmp kieker 20110428 142829399 UT C Kaapstad KIEKER outputdir out plot Deployment Seguence Diagrams plot Call Trees Listing 5 7 Commands to produce the diagrams under UNIX like systems gt trace analysis bat inputdirs Yotemp kieker 20100813 121041532 U
105. r 1 6 User Guide 58 i inputdirs lt dirl dirN gt o outputdir lt dir gt p output filename prefix lt prefix gt plot Deployment Sequence Diagrams plot Assembly Seguence Diagrams Log directories to read data from Directory for the generated file s Prefix for output filenames Generate and store deployment level sequence diagrams pic files Generate and store assembly level sequence diagrams pic files plot Deployment Component Dependency Graph Generate and store a plot Assembly Component Dependency Graph plot Container Dependency Graph plot Deployment Operation Dependency Graph plot Assembly O peration Dependency Graph plot Aggregated Deployment Call Tree plot Aggregated Assembly Call Tree plot Call Trees print Message Traces print Execution Traces print invalid Execution Traces print Deployment Equivalence Classes print Assembly Equivalence Classes deployment level component dependency graph dot file Generate and store an assembly level component dependency graph dot file Generate and store a container dependency graph dot file Generate and store a deployment level operation
106. ring writer jms AsyncJMSWriter ContextFactory Type org apache activema jndi ActiveMOlnitialContextFactory The service name for the jms connection factory kieker monitoring writer jms AsyncJMSWriter FactoryLookupName ConnectionFactory The time that a jms message will be kept alive at the jms server before it is automatically deleted kieker monitoring writer jms AsyncJMSWriter MessageT ime ToLive 10000 Asynchronous writers need to store monitoring records in an internal buffer This parameter defines its capacity in terms of the number of records kieker monitoring writer jms AsyncJMSWriter QueueSize 10000 Behavior of the asynchronous writer when the internal queue is full 0 terminate Monitoring with an error default 1 writer blocks until queue capacity is available 2 writer discards new records until space is available Be careful when using the value 1 since then the asynchronous writer is no longer decoupled from the monitored application kieker monitoring writer jms AsyncJMSWriter QueueFullBehavior 0 Kieker 1 6 User Guide 79 Maximum time to wait for the writer threads to finish in milliseconds A MarShutdownDelay of 1 means infinite waiting kieker monitoring writer jms AsyncJMSWriter MaxShutdownDelay 1 GERAAK kieker monitoring writer kieker monitoring writer database SyncDb Writer Database driver class
107. ription of how to use the JMSReader can be found in Appendix C Please note that the database reader DBReader is currently in a prototype stage and that it should be used with care Like each plugin readers are configured via properties as used in Section and detailed in Section 4 2 1 Kieker 1 6 User Guide 32 interface AbstractReaderPlugin IReaderPlugin from kieker analysis plugin reader from kieker analysis plugin reader ES geet ee read boolean FSReader PipeReader JMSReader JMXReader DBReader terminate error boolean Figure 4 3 Monitoring reader plugins included with Kieker 4 1 3 Filter Plugins Filter plugins receive events Java objects via input ports from other plugins and im plement analyses or visualizations based on these events Kieker already includes some basic filter plugins For example the CountingFilter and TeeFilter forward incoming events to their output ports The CountingFilter additionally provides the current number of received records via a second output port The TeeFilter additionally prints incoming events to an output stream which may be the standard output standard er ror a logger or a file A TimestampFilter and a TypeFilter filter incoming records by timestamp and by type respectively A TraceIdFilter filters incoming trace events e g OperationExecutionRecords see Section 2 by trace ID Additional filters for trac
108. rovide a public constructor accepting a Configuration object as its only argument It is important to invoke the constructor of the super class The configuration properties accepted by a plugin or repository should be provided via public static constants with prefix CONFIG PROPERTY NAME in order to ease the programmatic initialization of plugins Section 4 1 1 For the example filter MyResponseTimeFilter Listing 4 2 shows the constructor the configuration property and the corresponding member value obtained from the configuration public static final String CONFIG PROPERTY NAME TS NANOS thresholdNanos private final long rtThresholdNanos the configured threshold for this filter instance public MyResponseTimeFilter final Configuration configuration 1 super configuration this rtThresholdNanos configuration getLongProperty CONFIG PROPERTY NAME TS NANOS Listing 4 2 Plugin constructor accepting a Configuration object Additionally the current configuration must be provided via the method getCurrentCon figuration Please note that the returned configuration should be sufficient to initialize the plugin or repository via the mentioned constructor The AnalysisController uses the getCurrentConfiguration to save the pipe and filter configuration Listing 4 3 shows how the methods are implemented for the example filter MyResponseTimeFilter public Configuration getCurrentConfiguration final Configuration configuration
109. searchBook searchBook 3 3 3 3 SRV1 SRVO SRV1 SRVO SRVO SRVO SRVO SRVO 1 Catalog 2 CRM 1 Catalog 2 CRM 1 Catalog 2 CRM 1 Catalog 2 CRM getBook getOffers getBook getOffers getBook getOffers getBook getOffers SRV1 SRVO SRV1 SRVO 1 Catalog 1 Catalog 1 Catalog 1 Catalog getBook getBook getBook getBook a Trace 6129 b Trace 6130 c Trace 6131 d Trace 6141 Figure 5 5 Calls trees of the trace equivalence class representatives Listing 5 13 Kieker 1 6 User Guide 51 Aggregated Call Trees Aggregated deployment assembly level call trees are generated using the command line options plot Aggregated Deployment Call Tree and plot Aggregated Assembly Call Tree Figures 5 6 a and 5 6 b show these aggregated call trees for the traces contained in the monitoring data used in this section 1635 SRVO 3 Bookstore 1635 searchBook 3 Bookstore 1092 1635 543 searchBook SRVI SRVO SRVO ses as 1 Catalog 2 CRM 1 Catalog getBook getOffers getBook 1 Catalog 2 CRM getBook getOffers 1062 573 1635 SRV1 SRVO 1 Catalog 1 Catalog 1 Catalog getBook getBook getBook a deployment level b assembly level Figure 5 6 Aggregated call trees generated from the 1635 traces Kieker 1 6 User Guide 5
110. shot of this file rendered by a web browser System Model Reconstructed by Kieker TraceAnalysis Mozilla Firefox yan Datei Bearbeiten Ansicht Chronik Lesezeichen Extras Hilfe di dv e Q ta l example plots system entities html yy Q le System Model Reconstructed by Kie d we a Component Types ID Package Name Operations 3 bookstoreTracing Bookstore searchBoolk N A 2 bookstoreTracing CRM e getOffers N A 1 bookstoreTracing Catalog aetBook boolean N A Operations ID Component type Name Parameter types Return type 3 bookstoreTracing Bookstore searchBook N A 2 bookstoreTracing CRM getOffers N A 1 bookstoreTracing Catalog getBook e boolean N A Assembly Components ID Name Component type 3 3 bookstoreTracing Bookstore 2 2 bookstoreTracing CRM 2 jm bookstoreTracing Catalog Execution Containers ID Name 2 SRVO 1 SRV1 Deployment Components ID Assembly component Execution container 4 3 bookstoreTracing Bookstore SRVO 3 M2 bookstoreTracing CRM SRVO 2 Ml bookstoreTracing Catalog SRVO 1 Ml bookstoreTracing Catalog SRV1 a v Fertig 3 FoxyProxy Default q Figure 5 10 HTML output of the system model reconstructed from the traces Kieker 1 6 User Guide 55 A Wrapper scripts The bin directory of Kieker s binary release contains some sh and bat scripts to
111. sources and pre compiled binaries of the manually instru mented Bookstore application described in this section can be found in the examples userguide ch2 manual instrumentation directory Kieker maintains monitoring data as so called monitoring records Section 3 3 describes how to define and use custom monitoring record types The monitoring record type used in this example is an operation execution record which is included in the Kieker distribution Figure 2 5 shows the attributes which are relevant to this example The record type will be detailed in Chapter 5 Kieker 1 6 User Guide 12 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 interface IMonitoringRecord from kieker common record AbstractMonitoringRecord from kieker common record OperationExecutionRecord from kieker common record operationSignature String hostname String tin long tout long Fus Figure 2 5 The class diagram of the operation execution record The attributes relevant to this part are operationSignature and hostname as well as tin and tout for the timestamps before and after the call of the instrumented method Listing 2 7 shows the instrumentation of the Bookstore class and its method search Book In the lines 25 and 26 the monitoring controller is instantiated It provides the monitoring service for the instrumentation private static final
112. st 1 1 OperationExecutionRecord 1333384394895258815 1 public void kieker examples userguide ch2bookstore manual getBook boolean lt no session id gt 1 1333384394893155964 1333384394895252319 myHost 1 1 OperationExecutionRecord 1333384394897431926 1 public void kieker examples userguide ch2bookstore manual getBook boolean lt no session id gt 1 1333384394895325513 1333384394897423685 myHost 1 1 OperationExecutionRecord 1333384394899547069 1 public void kieker examples userguide ch2bookstore manual getBook boolean lt no session id gt 1 1333384394897449596 1333384394899541691 myHost 1 1 OperationExecutionRecord 1333384394901701603 1 public void kieker examples userguide ch2bookstore manual getBook boolean lt no session id gt 1 1333384394899602104 1333384394901695946 myHost 1 1 OperationExecutionRecord 1333384394903896086 1 public void kieker examples userguide ch2bookstore manual getBook boolean lt no session id gt 1 1333384394901715571 1333384394903886517 myHost 1 1 OperationExecutionRecord 1333384394906161388 1 public void kieker examples userguide ch2bookstore manual getBook boolean lt no session id gt 1 1333384394904036047 1333384394906153566 myHost 1 1 OperationExecutionRecord 1333384394908345045 1 public void kieker examples userguide ch2bookstore manual getBook boolean lt no session id
113. stemNano Timer A timer with nanosecond precision The offset of the timer The time returned is since 1970 1 1 minus this offset If the offset is empty it is set to the current time The offset must be specified in milliseconds kieker monitoring timer System NanoT imer offset 0 The timeunit used to report the timestamp Accepted values 0 nanoseconds 1 microseconds 2 milliseconds 3 seconds kieker monitoring timer System NanoTimer unit 0 Kieker 1 6 User Guide 75 Hdr dad WRITER EER Selection of monitoring data writer classname The value must be a fully qualified classname of a class implementing kieker monitoring writer MonitoringWriter and providing a constructor that accepts an IMonitoringController and a single Configuration kieker monitoring writer kieker monitoring writer filesystem AsyncFsWriter HEHE kieker monitoring writer kieker monitoring writer Dummy Writer Configuration Properties of the Dummy Writer kieker monitoring writer DummyWriter key value HEHEHE kieker monitoring writer kieker monitoring writer PrintStream Writer The PrintStream used to print the monitoring records Either STDOUT or STDERR Other values are used as a filenames for a target log file You should use AsyncFsWriter instead of this writer for logging to files kieker monitor
114. t file kax loaded svg lt filename gt name of svg saved on close A 5 Script trace analysis sh bat Calls Kieker TraceAnalysis to analyze and visualize monitored trace data as described in Chapter Main class kieker tools traceAnalysis TraceAnalysisTool Usage usage kieker tools traceAnalysis TraceAnalysisTool i lt dirl dirN gt o lt dir gt p lt prefix gt plot Deployment Sequence Diagrams plot Assembly Sequence Diagrams plot Deployment Component Dependency Graph plot Assembly Component Dependency Graph plot Container Dependency Graph plot Deployment Operation Dependency Graph plot Assembly O peration Dependency Graph plot Aggregated Deployment Call Tree plot Aggregated Assembly Call Tree plot Call Trees print Message Traces print Execution Traces print invalid Execution Traces print Deployment Equivalence Classes print Assembly Equivalence Classes select traces lt id0 idn gt ignore invalid traces max trace duration lt duration in ms gt ignore executions before date lt yyyyMMdd HHmmss gt ignore executions after date lt yyyyMMdd HHmmss gt short labels Kieke
115. tasks The Plugin annotation of the example filter MyRe sponseTimeFilter was shown in Listing 4 4 already Kieker 1 6 User Guide 37 55 56 57 58 59 39 40 41 42 40 41 42 43 Filters receive events via methods marked with the InputPort annotation These methods must accept a single argument which has to be a super type of the set of accepted event types declared in the respective InputPort annotation s eventTypes In addition to an optional description each InputPort must have a name which is unique for this filter The input port names should be provided as a public static constants with prefix INPUT_PORT_NAME_ im order to ease the programmatic connection of readers and filters as described in Section 4 1 1 Listing 4 9 shows the declaration of the input port provided by the example plugin MyResponseTimeFilter The body of this method was shown in Listing 4 4 already InputPort name MyResponseTimeFilter INPUT PORT NAME RESPONSE TIMES description Filter the given record depending on the response time eventTypes 1 MyResponseTimeRecord class public void newResponseTime final MyResponseTimeRecord rtRecord Listing 4 9 InputPort annotation for the example plugin s input method 4 2 5 Developing and Accessing Required Repositories Custom repositories must extend the class AbstractRepository The Repository annotation is used to provide a name and a description for a repository type L
116. the output of the reader with the input of the filter analysisInstance connect reader FSReader OUTPUT_PORT_NAME_RECORDS teeFilter TeeFilter INPUT_PORT_NAME_EVENTS Start the analysis analysisInstance run Listing 2 13 BookstoreAnalysisStarter java excerpt from main method Kieker 1 6 User Guide 17 The BookstoreAnalysisStarter follows a simple scheme Each analysis tool has to create at least one AnalysisController which can be seen in Listing 2 13 in line 37 Then the plugins which may be readers or filters are configured instantiated and registered with the analysis instance Lines 40 44 configure instantiate and register the file system monitoring log reader which uses the command line argument value as the input directory The application expects the output directory from the earlier monitoring run see Section as the only argument value which must be passed manually Lines 47 51 configure instantiate and register the TeeFilter which outputs received events to the standard output Lines 54 and 55 connect the TeeFilter s input port to the filesystem reader s output port The analysis is started by calling its run method line 57 The Listings 2 14 and 2 15 describe how the analysis application can be compiled and executed under UNIX like systems and Windows gt mkdir build gt javac src kieker examples userguide ch2bookstore manual x java classpath lib kieker 1 6_emf jar d build
117. tj org dtd aspectj 1 5 0 dtd gt lt aspectj gt lt weaver options gt lt include within kieker examples userguide ch5bookstore gt lt weaver gt lt aspects gt lt aspect name kieker monitoring probe aspect operationExecution OperationExecutionAspectFull gt lt aspects gt lt aspectj gt Listing 5 5 aop xml The alternative aspect OperationExecutionAspectFull is being activated in line 9 As indicated by its name this aspect makes sure that every method within the included classes packages will be instrumented and monitored Listing 5 5 demonstrates how to limit the instrumented methods to those of the class BookstoreStarter The commands shown in the Listings and can again be used to compile and execute the example Note that the annotations within the source code have no effect when using this aspect When using a custom aspect it can be necessary to specify its class name in the include directives of the aop xml 5 1 2 Servlet Filters The Java Servlet API 7 includes the javax servlet Filter interface It can be used to implement interceptors for incoming HTTP requests Kieker includes the probe Ses sionAndTraceRegistrationFilter which implements the javax servlet Filter in terface It initializes the session and trace information for incoming requests If desired it additionally creates an OperationExecutionRecord for each invocation of the filter and passes it to the Monitorin
118. tor in applicationContext xml AspectJ based Instrumentation In order to use AspectJ based instrumentation the following changes need to be performed The file start ini located in Jetty s root directory allows to pass various JVM arguments JVM system properties and other options to the server on startup When using AspectJ for instrumentation the respective JVM argument needs to be activated in this file javaagent webapps jpetstore WEB INF lib aspectjweaver 1 7 1 jar Listing B 3 Enabling the AspectJ weaver in Jetty s start ini The AspectJ configuration file aop xm1 is already located in the webapps s WEB INF classes META INF directory and configured to instrument the JPetStore classes with Kieker s Opera tionExecutionAspectFull aspect Section 5 When using the AspectJ based instrumentation make sure to disable the Spring in terceptor in the file applicationContext xml mentioned above 1 Start the Jetty server using the start jar file You should make sure that the server started properly by taking a look at the console output that appears during server startup 2 Now you can access the JPetStore application by opening the URL localhost 8080 jpetstore Figure B 1 Kieker initialization messages should appear in the console output The output includes the information where the mon itoring data is written to should be a kieker lt DATE TIME gt directory located in the default temporary directory K
119. types defaults to a list including only Object class The output port names should be provided as a public static constant with prefix OUTPUT PORT NAME in order to ease the programmatic connection of readers and filters as described in Section 4 1 1 Repositories required by filters are also specified as part of the Plugin annotation This is detailed in Section 4 2 5 Plugin name Response time filter description Filters incoming response times based on a threshold outputPorts OQutputPort name MyResponseTimeFilter 0OUTPUT PORT NAME RT VALID description Qutputs response times satisfying the threshold eventTypes MyResponseTimeRecord class OutputPort name MyResponseTimeFilter 0OUTPUT PORT NAME RT EXCEED description Qutputs response times exceeding the threshold eventTypes MyResponseTimeRecord class configuration Property name MyResponseTimeFilter CONFIG PROPERTY NAME TS NANOS defaultValue 1000000 Listing 4 4 Plugin annotation for the example plugin MyResponseTimeFilter Plugins can send events to their output ports by calling the deliver method provided by the super class The method expects the output port name and the event to be sent as arguments Listing 4 5 shows how the example filter plugin MyResponseTimeFilter delivers records to its two output ports declared in the P1ugin annotation if rtRecord getResponseTimeNanos gt this rtThresholdNanos 1 sup
120. uestRegis trationInterceptor intercepts incoming Web requests and initializes the trace and ses sion data for this trace If you are not using the OperationExecutionWebRequestReg istrationInterceptor you should use one of the previously described Servlet filters to register session information for incoming requests Section 5 1 2 See the Spring documentation for instructions how to add the interceptors to the server configuration 5 1 4 CXF SOAP Interceptors The Apache CXF framework 9 allows to implement interceptors for web service calls for example based on the SOAP web service protocol Kieker includes the probes Opera tionExecutionSUAPRequestOutInterceptor OperationExecutionSUAPRequest InIn terceptor OperationExecutionSOAPResponseOutInterceptor and OperationExe cutionSOAPResponseInInterceptor which can be used to monitor SOAP based web service calls Session and trace information is written to and read from the SOAP header of service requests and responses allowing to monitor distributed traces See the CXF documentation for instructions how to add the interceptors to the server configuration Kieker 1 6 User Guide 45 5 2 Trace Analysis and Visualization Monitoring data including trace information can be analyzed and visualized with the Kieker TraceAnalysis tool which is included in the Kieker binary as well a gt In order to use this tool it is necessary to install two third party pro grams 1 Gra
121. wo steps 1 Monitoring CPU utilization and memory usage for 30 seconds class Monitor ingStarter gt ant run monitoring Kieker s console output lists the location of the directory containing the file system monitoring log The following listing shows an excerpt Writer kieker monitoring writer filesystem AsyncFsWriter Configuration kieker monitoring writer filesystem AsyncFsWriter QueueFullBehavior 0 kieker monitoring writer filesystem AsyncFsWriter QueueSize 10000 kieker monitoring writer filesystem AsyncFsWriter customStoragePath kieker monitoring writer filesystem AsyncFsWriter storelnJavaloTmpdir true Writer Threads 1 Finished false Writing to Directory tmp n kieker 20110511 10095928 UT C avanhoorn thinkpad KIEKER SINGLETON Kieker 1 6 User Guide 71 A sample monitoring log can be found in the directory examples userguide appendix Sigar testdata kieker 20110511 10095928 UTC avanhoorn thinkpad KIEKER SINGLETON 2 Analyzing the monitoring data class AnalysisStarter gt ant run analysis Danalysis directory lt path to monitoring log gt You need to replace lt path to monitoring log gt by the location of the file sys tem monitoring log You can also use the above mentioned monitoring log included in the example The AnalysisStarter produces a simple console output for each monitoring record as shown in the following excerpt Wed 11 May
122. xperimentid incExperimentid incExperimentid _ si a newMonitoringRecord newMonitoringRecord schedulePeriodicSampler schedulePeriodicSampler removeScheduledSampler removeScheduledSampler boolean boolean newMonitoringRecord pl ScheduledSamplerjob chedulePeriodicSampler pl boolean renomi veduledsampler pl Refresh a Attributes b Operations Figure 3 2 Screenshots of the jconsole JMX client accessing the MonitoringCon troller s attributes and operations via the MBean interface 3 1 7 Adaptive Monitoring The MonitoringController provides an API to activate and deactivate probes at run time By passing a method signature e g public void Bookstore getBook to the method isProbeActivated probes can check whether or not monitoring for the method with the given signature is active Monitoring can be de activated for single signature patterns e g public void Bookstore via the methods activateProbe and deactivateProbe The current list of de activated patterns can be obtained via the method getProbePatternList The entire list can be replaced using the method set ProbePatternList Alternatively a file with include and exclude patterns can be used This file can be polled in regular intervals A default configuration file including a de scription of the pattern syntax is provided by the file kieker monitorin
123. y kieker monitoring writer jms AsyncJMSWriter ProviderUrl tcp 127 0 0 1 3035 Listing C 10 Excerpt from kieker monitoring properties openjms configuring the provider URL of the JMS writer via OpenJMS kieker monitoring writer jms AsyncJMSWriter ContextFactory Type org exolab jms jndi InitialContextFactory Listing C 11 Excerpt from kieker monitoring properties openjms configuring the context factory of the JMS writer via OpenJMS C 3 4 Running the Example The execution of the example is performed by the following three steps 1 Start the JMS server you may have to set your JAVA HOME and OPENJMS_HOME variables first gt openjms lt version gt bin startup sh Listing C 12 Start of the JMS server under UNIX like systems gt openjms lt version gt bin startup bat Listing C 13 Start of the JMS server under Windows 2 Start the analysis part in a new terminal gt ant run analysis openjms 3 Start the instrumented Bookstore in a new terminal gt ant run monitoring openjms Kieker 1 6 User Guide 69 D Sigar Based Samplers for System Level Monitoring This chapter gives a brief description on how to use the included periodic samplers Section 3 1 4 for monitoring CPU utilization and memory swap usage The directory examples userguide appendix Sigar contains the sources ant scripts etc used in this example These samplers employ the Sigar API I D 1 Preparation 1 Copy th
124. yncJMSWriter ProviderUrl tcp 127 0 0 1 61616 Listing C 1 Excerpt from kieker monitoring properties activemq configuring the provider URL of the JMS writer via ActiveMQ kieker monitoring writer jms AsyncJMSWriter ContextFactory Type org apache activemq jndi ActiveMOlnitialContextFactory Listing C 2 Excerpt from kieker monitoring properties activemq configuring the context factory of the JMS writer via ActiveMQ C 1 4 Running the Example The execution of the example is performed by the following three steps 1 Start the JMS server you may have to set your JAVA_HOME variable first gt bin activemq start Listing C 3 Start of the JMS server under UNIX like systems gt bin activemq Listing C 4 Start of the JMS server under Windows 2 Start the analysis part in a new terminal gt ant run analysis activemq 3 Start the instrumented Bookstore in a new terminal gt ant run monitoring activemq Kieker 1 6 User Guide 66 C 2 HornetQ C 2 1 Download and Prepare HornetQ Download a HornetQ archive from http www jboss org hornetg downloads html and decompress it to the root directory of the example You need to create a queue in the HornetQ configuration file config stand alone non clustered hornetq jms xml as shown in Listing C 5 lt queue name queuel gt lt entry name queue queuel gt lt queue gt Listing C 5 Queue definition to be added to the HornetQ configuration file

Kieker 1.6 User Guide

Contents

Download Pdf Manuals

Related Search

Related Contents