Home

Design and implementation of a web framework for the analysis and

1. 29 3 2 2 6 Saving the simulation in database 30 3 2 2 7 Analysing the 31 3 2 2 8 Downloading the graph 32 3 2 2 9 Network visualization 33 3 2 3 Developer use cases 34 3 2 3 1 New user behaviour 35 3 2 3 2 New network building algorithm 36 3 2 4 Admin Use cases 37 3 2 4 1 Manage the database 38 3 2 4 2 Manage the user s permissions 39 3 2 5 COnDCLIUSIOBDS un 8 descen rcrum Ta Hoge AE Is 4 Architecture and implementation 41 Introduction xeu PRICE SOE ee E UO Re AD Architecture ov te Roo es e dedo da ELS de 0 ia e Ho 4 3 MASON engine and its implementation 4 4 Neo4J database and its implementation 4 5 SigmaJS graph visualizator and its implementation 4 6 Gephi and export to GEXF file implementation 4 7 User web interface and Servlet and their implementation AS TOoncl sidM A en Sa do A der ee eh te thee yi 5 Prototype and example usage 5 1 JIntrod ction 5 Weeds Aes ee Dr EN een an 5 2 Random network esee 5 3 Creating network with Neo4J 5 4 Loading network from
2. 5 5 Running the simulation of the loaded 5 6 Results screen and final uu avv ee en Due 6 Conclusions and future lines 6 11 GConcLlusl DS uera Sina jene a Henn Ns ctr A test ee Sete wR 6 2 Achieved goals 2 0a x e Dean 6 3 Buture works unde ure A eed N a A Installing and running a BigMarket server AJ Installation pad Soe BA ee Eurum eS Bid ang Wem A1 43 43 AT 49 54 95 56 68 65 67 67 68 71 72 73 75 77 79 79 80 81 A AA Requirenmients gar a Ber WOR Gate Odes ge Eod eit 83 A 1 2 Downloading the source code 83 A 1 3 Importing the project in Eclipse 84 A 1 4 Converting the project into an Eclipse project 2 2 2 2 88 A 1 5 Running the Neo4J database 91 A 2 Runa BigMarket 91 A 2 1 Introduction 4 439 0 428 RAD eee Bad a 91 A 2 2 Building the WAR Web application ARchive 91 A 2 3 Running a the application 2 2 2 2 CE rn nn 93 B User manual 97 B 1 Run new random network 99 B 2 Load network 4 23 c 2 2 28 a li 102 Create a network s soo s suce macs ale Su maca aoea pe e a a a 103 Bibliography 105 1 1 1 2
3. XDvclet Revert Apply Figure A 9 Convert the project into Eclipse project step 1 Now you have the project converted into a Eclipse project specifically a Dynamic Web Project but there are errors in our project In order to solve this you have to import the library servlet api To import a determined library in Eclipse you have to follow the next steps 88 A 1 INSTALLATION 1 Right click on our project and select Build path Configure build path 2 In the new window select Add external JARs New Go Into Open in New Window Open Type Hierarchy Show In Copy Copy Qualified Name Paste Delete Remove from Context Build Path Source Refactor Import Export Refresh Alt Shift W gt Ctrl C Ctrl V Delete Ctri Alt Shift Down Link Source New Source Folder Use as Source Folder Add External Archives Add Libraries Configure Build Path Figure A 10 Import libraries in an Eclipse project step 1 Java Build Path b Java Code Style Java Compiler Java Editor Javadoc Location JavaScript JSP Fragment Project Facets Project References Run Debug Settings Server Mir Figure A 11 Java Build Path 8 Source Projects BA Libraries Order and Export JARs and class folders on the build path gt EAR Libraries b We
4. 2 Grupo de Sistemas Inteligentes About BigMarket Set up Simulation Runni Time step 21 User type Number of users Number of tweets writed Broadcaster 4 37 Acquaintances 0 0 Odd users 720 72 Figure 5 6 Running the simulation As we can see in the figure 5 6 in the step 21 we have 4 broadcasters that have written 37 tweets and 720 odd users that have written 72 tweets In this point we can resume the simulation or finish it 5 6 Results screen and final actions The last screen of our simulation shows the results of four centrality measures betweeness closeness in degree and out degree like we can see in the figure 5 7 We can reach this screen by pushing the Stop button in the Running screen In this screen showed in the figure 5 7 we have the option to see the network created during the simulation and download it for analyze with a SNA tool 73 CHAPTER 5 PROTOTYPE AND EXAMPLE USAGE a y A gsi 2 Sim gsi 2 Grupo de Sistemas Inteligentes UPM About BigMarket Set up Simulation Running Actions Contact Node Betweeness See the graph of the network 1510 0 0 759 784 6996031746049 E Export the graph in a gexf file 766 1237 7555555555564 E Download graph 768 2853 9353174603216 758 4996 8456349206335 771 1370 3365079365078 765 4140 168253968279 760 3435 Figure 5 7 Results of the simulation If we click on the See network button we can see the network at the end of the
5. 3 2 2 5 Running the simulation This use case represents the execution of a simulation using a network created with some of the previous use cases So at the beginning of the use case we have a network and the Running screen displayed The table also represents the flow of events of the use case Use Case Name Running the simulation Use Case ID UC1 5 Pre Condition The network has been created and the running screen has been displayed Post Condition The results screen is displayed and the simulation stored in the database Flow of Events Actor Input System Response The user clicks on the Run The simulation starts 1 one step Run button The user clicks on Pause M 2a The simulation is paused button A The user clicks on Stop The simulation is stopped button and it is stored in the 3a The user repeats the step 1 N A The user clicks on Stop button 3b The simulation is stopped and stored in the database Table 3 6 Running the simulation use case 29 CHAPTER 3 REQUIREMENT ANALYSIS 3 2 2 6 Saving the simulation in database This use case represents the storing of the simulation in database pre condition we have the simulation must be running At the end of this use case the simulation is stored in the database The table also represents the flow of events of the use
6. Post Condition N A Flow of Events Actor Input System Response The admin logs in the The actions for managing the 1 database interface as database become enabled superuser The admin performs an 20 problems of the database 2 action in order to solve are solved database problems Table 3 13 Manage the database use case 38 3 2 USE CASES 3 2 4 2 Manage the user s permissions This use case represents the possibility to manage the user s permissions In the future the users will have different permissions that allow them to use BigMarket in a way or other Use Case Name Manage the user s permissions Use Case ID UC3 2 An user needs to change its permissions or user have to be Pre Condition banned for using BigMarket Post Condition N A Flow of Events Actor Input System Response A The admin log in BigMarket The administration windows as superuser are opened The admin selects the user The user is updated with the and his her new permissions new permissions Table 3 14 Manage the user s permissions use case 3 2 5 Conclusions With the use cases described we have introduced the basic functionalities that have been implemented in this project They help us to understand the different actors that can interact They can serve as a base for further development and different use cases that can come to mind
7. MA a Figure 2 1 MASON Logo MASON Stands for Multi Agent Simulator Of Networks It is a fast discrete event multiagent simulation library core in Java designed to be the foundation for large custom purpose Java simulations and also to provide more than enough functionality for many lightweight simulation needs MASON contains both a model library and an optional suite of visualization tools in 2D and 3D MASON is a joint effort between George Mason University s Evolutionary Computation Laboratory and the GMU Center for Social Complexity MASON features e 100 Java 1 3 or higher e Fast portable and fairly small e Models are completely independent from visualization which can be added removed or changed at any time e Models may be checkpointed and recovered and dynamically migrated across plat forms e Can produce results that are identical across platforms e Models are self contained and can run inside other Java frameworks and applications e 2D and 3D visualization e Can generate PNG snapshots Quicktime movies charts and graphs and output data streams These features make MASON a good choice for coding the simulation In our project we will need to simulate the behaviour of a complex society so we must code a simulation MASON will facilitate us these task because it is coded in Java so it adapts perfectly to our purpose For the visualization and build the graph the nodes and their
8. 13 CHAPTER 2 ENABLING TECHNOLOGIES Our project is focus in Big Data so we will need store a big amount of nodes and its relationships As we see in the second feature of Neo4J it supports a lot of relationships and transactions becoming Neo4J the best choice for store ours graphs Neo4J also offers a graphic interface in order to see the data store in the database and allow us to create a new graph using Neo4J commands in general do any action So that this feature will add more functions to our tool So in sight of our needs working with graphs and a lot of nodes and relationships we consider Neo4J a good choice for our project In chapter 4 we will speak more about the way that BigMarket store the graph in the database and how the user can recover the information Figure 2 5 Neo4J Framework 2 4 GraphStream MGS Figure 2 6 GraphStream Logo GraphStream is a graph handling Java library that focuses on the dynamics aspects of graphs Its main focus is on the modelling of dynamic interaction networks of various sizes http graphstream project org 14 2 4 GRAPHSTREAM The goal of the library is to provide a way to represent graphs and work on it To this end GraphStream proposes several graph classes that allow us to model directed and undirected graphs 1 graphs or p graphs a k a multigraphs that are graphs that can have several edges between two nodes GraphStream allows us to
9. accept mtHeader setValue application json mPost addRequestHeader mtHeader StringRequestEntity requestEntity new StringRequestEntity application json UTF 8 mPost setRequestEntity requestEntity client executeMethod mPost mPost getResponseBodyAsString Header locationHeader mPost getResponseHeader location location locationHeader getValue mPost releaseConnection String data sim getSimDataset nodeUris put n getId location this saveNodeRelations location sim getUsers get n getIndex getFollowers this addProperty location dataset data this addLabel location data 51 CHAPTER 4 ARCHITECTURE AND IMPLEMENTATION jcatch Exception System out println Exception in creating node in neo4j Tote 803 e Load network from database at the same way like save network in database we can load a network stored previously in the database To do this we follow a method similar to that used to save a network in the database Listing 4 9 Load network public void getNodesPerLabel String labelName try String response String nodePointUrl http localhost 7474 db data label labelName nodes HttpClient client new HttpClient GetMethod mGet new GetMethod nodePointUrl Header mtHeader new Header mtHeader setName accept mtHeader setValue application json mGet addRequest
10. n an lisis de redes sociales Big Data Twitter V Abstract The aim of this project is the development and implementation of a web framework which allows the creation and simulation of social networks in order to analyse the diffusion of hearsay in accordance to the relationship between the different types of users With this purpose a web framework has been developed capable of generating networks from an initial number of nodes inserted by the user a possibility of creating an own network inserting each node and its relationship with each other or loading a network created previously The generated network can be saved for future simulations behaviour will be assigned to each agent users of the network depending on the number of its relations with the others agents As last functionality the user can download the red in a GEXF Graph Exchange XML Format file in order to process it with a SNA software A last functionality added to the project is the possibility of download the code in order to extend the behaviour of the agents to adapt it to the user s needs This report presents the build up of the framework piecemeal and subsequent connection between them in order to shape the final framework An analysis of the different tools used and why they are chosen or not is included too Finally some appendixes are included to extend some aspects and facilitate the understanding of these and some tutorials to provide the need
11. simulation like we show in the figure 5 8 mal al Tom To Se CC NN Figure 5 8 Network visualization Finally we can download the graph in GEXF file and analyse it with Gephi 74 5 7 CONCLUSION 5 7 Conclusion In this chapter we have explained and extended use case for BigMarket so that the user can have a global view of the BigMarket framework In the next chapter we will resume the conclusions and the results of this master thesis 75 CHAPTER 5 PROTOTYPE AND EXAMPLE USAGE 76 Conclusions and future lines This chapter summarizes the conclusions extracted from this master thesis and the objectives achieved are evaluated After that we describe thinking about future work 77 CHAPTER 6 CONCLUSIONS AND FUTURE LINES 78 6 1 CONCLUSIONS 6 1 Conclusions using different papers that represent a variety of mathematics models 3 4 that allows modelling different types of networks we have created a tool that allows the user to build a network from zero and make it evolve in time simply selecting the number of initial nodes Due to integration with a database the user is able to store the network that he she has been created in order to recover the network for future uses for restarting it or simply change the conditions and see how the simulation and the network evolves in the new situations The tool also provides a functionality for creating a
12. 2 1 2 2 2 3 2 4 2 5 2 6 2 7 2 8 2 9 2 10 2 11 2 12 3 1 3 2 3 3 4 1 4 2 List of Figures OhalLogo 4 22 c c RB RO doe SOROR OR RON URS d 5 ThinkVine LOgO amp 2 2 0 bue komo a a Rs 6 MASON Logo hh horn 11 ium elm ae a oe x Ge s 12 Gephi Framework 13 Neo4J Logo 4 222 e lee e eee Ro 13 Neo4J Framework 14 GraphStream 14 GraphStream Graph Example 16 Apache Tomcat Logo 16 Java LOEO 332 3 9945 2 0 3o p EUR ee eub PUR E ded RU RUE A 17 HTMLS5 Logo 17 553 LOBOL au wa Bew naeh ie 18 SigmaJS Logo 22 4246 eb c9 re ne 18 User use Cases sasur an Say wee E A a nn 24 Developer use cases 34 Admin use cases 37 UML class diagram 44 UML component 45 5 1 100 Node Random network setup 68 5 2 NeoAJ interfaces sx cR em ER XE des Be EE Peu 69 Bid Nodescrestedt sn tern B e e Dye ez d UR een e dos 70 5 4 Final networks nr u Sm a Le eoe eus gr go
13. Constants SIM sim request getRequestDispatcher Constants RUNNING PAGE forward request response 58 4 7 USER WEB INTERFACE AND SERVLET AND THEIR IMPLEMENTATION First of all the servlet checks if the form submitted is a Setup simulation form Then check if the user wants to create a random network or load a network Depending of the user s choice the servlet execute a code block or other Once the initial network has been setup the user can run the simulation To do this BigMarket enables in the Running page a form similar to the Setup form The code of this form is the following Listing 4 15 Servlet handle setup form div class action buttons gt form action BigMarketServlet method POST name running gt lt input type hidden name formName value runningForm gt lt input type submit class btn btn primary value Run one step id runOneStepButton onCLick clickROS gt input type submit class btn btn primary value Run id runButton onClick clickRun gt lt input type submit class btn btn primary value Pause jd pauseButton onClick clickPause gt lt input type submit class btn btn primary value Stop id StopButton onClick clickStop gt lt input name actionSelected type hidden value default id actionSelected gt lt form gt lt div gt This form has a hidden field which store the value of the option s
14. SNSA social network simulation and analysis Exposes the goals of the master thesis structure of this master thesis and a short representation of the state of art CHAPTER 1 INTRODUCTION 1 1 1 1 Context Agent based social simulation ABSS computer assisted simulation technique used to model artificial societies populated with multiple autonomous entities called agents which act autonomously by employing some knowledge or representation of their beliefs desires and intentions ABSS is an innovative approach to open questions in a wide range of scientific domains including economics biology chemistry ecology and sociology 1 On the other hand the social network analysis SNA in advance has emerged like a key methodology in fields like social sciences in which are included sociology social psychology economy Moreover it has also gained a significant support in other fields like biology or physics 2 This project deals with SNSA social network simulation and analysis which joins these two research fields ABSS and SNA Currently the social networks have reached a great impact in the relationships between persons and enterprises thanks to applications like T witter Facebook or similar These have a great importance in order to know the people opinion about various topics and how the relations among these persons change depends of their opinion The analysis of this behavior through tradi
15. composed by the Neo4J tools is apart of its use like database create and modify the network user can use BigMarket to open a new window with the Neo4J interface which allows us to build a new network from scratch or modify a previous stored network using Neo4J queries The Neo4J interface also allows the user to see the networks stored in the database Actually there is not any security like sessions it will be implemented in future networks so an user can watch all the networks stored in the database 49 CHAPTER 4 ARCHITECTURE AND IMPLEMENTATION Once we have created or modify a network we can use BigMarket to load it and run a simulation using this loaded network For do this the user web interface simplify the method of write queries for Neo4J we will explain it in user web interface section Neo4J implements a REST API that allows us to send JSON objects with the informa tion of the network in order to store it in the database Now we have introduce each part of the Neo4J used in BigMarket we will explain it in more detail We show too the implementation of each part in order to make this module more comprehensible by the users e Create network with Neo4J queries as we comment at the start of this section Neo4J allows us to create or modify a network using its queries Like Neo4J is a graph database its queries are oriented to the building of graphs so we can create a net work or modify and e
16. gt lt td gt lt td lt broad Tweets gt lt td gt lt tr gt lt tr gt lt th gt Acquaintances lt td gt lt td lt acqUsers gt lt td gt lt td lt acq Tweets gt lt td gt lt tr gt lt tr gt lt th gt Odd users lt td gt 4 7 USER WEB INTERFACE AND SERVLET AND THEIR IMPLEMENTATION lt td gt lt oddUsers gt lt td gt lt td gt lt oddT weets gt lt td gt lt tr gt lt table gt lt div gt The first part extract from the response of the servlet request seen from the JSP file the information about the simulation and represent it in a table to make it easy to understand by the user How the servlet manage all of this Easy with a block of code similar to the block of code of the previous part Listing 4 17 Servlet handle setup form yelse if formName equals Constants RUNNING FORM NAME ACTION SELECTED if actionSelected equals Constants STEP clickRunOneStep request response telse if actionSelected equals Constants RUN clickRun request response yelse if actionSelected equals Constants PAUSE clickPause request response telse clickStop request response String actionSelected request getParameter Constants First the servlet checks if the form submitted is a Running form In case of success it checks if the button selected is a run one step run pause or stop button But
17. have downloaded the exe execute it and follow the instructions 3 When the installer ends the installation you have to run the database server To do this execute Neo4J Community application 4 In the new window click on Start and you will have finished this section A 2 Runa BigMarket Server A 2 1 Introduction In the past section we saw how to import the project from Github convert it into an Eclipse project and finally how to solve the errors that appeared Now it is time to run the server In order to achieve this we guide you through the next sections A 2 2 Building the WAR Web application ARchive Apache Tomcat requires a WAR file to publishes a web application so build a WAR file from the project will be our first step To achieve this follow the next steps 91 APPENDIX A INSTALLING AND RUNNING A BIGMARKET SERVER 1 Firstly right click on our project in Eclipse and select Export New Go Into Open in New Window I Package Open Type Hierarchy F4 gt Show In Alt Shift W gt Ei Copy Ctrl C Copy Qualified Name E Paste Ctrl V Delete Delete Remove from Context Ctri Alt Shift Down Build Path gt Source Alt Shift S gt Refactor Alt Shift T gt s Export Figure A 14 Building the WAR step 1 2 In the new window click on Web folder and select WAR file then click in Next Select an export destination type filter text gt Gen
18. in our own computer we comment in the past section MASON follows these points in each execution e Network growth first of all MASON introduce a number of new users in the network This number depends of the number of actual users in the network and the steps from the beginning of the simulation This growth follows the next expression extracted from 3 e 00 239 1 67 The implementation of this network growth can see in the following block Listing 4 1 Network growth long t simulation schedule getSteps double exponent 1190 0 5000 0 double r Math pow t exponent double integralResult 1565xr 937 double n initialPopulation Math exp integralResult int nt int Math round n e Connecting the new users once the new users are added is time to connect them with the old users of the network To do this we will use an algorithm that makes more possible that new users connect with other that has a lot of followers instead of an user that has less followers In order to help to the users to understand this function we attach the implementation of the connecting new users function Listing 4 2 Connecting new users public void lookForNewUsers Simulation sim Graph graph sim getGraphManager getGraph for User u sim getUsers if u getFollowed size 0 amp amp u getFollowers size 0 Node n2 graph getNode u getId int random int Mat
19. jhow the servlet give to the JSP the information about the simulation To answer this question here it goes an example Listing 4 18 Servlet handle setup form request setAttribute broadUsers getBroadUsers sim request setAttribute aqUsers getAqUsers sim request setAttribute oddUsers getOddUsers sim request setAttribute Constants STEPS sim schedule getSteps CHAPTER 4 ARCHITECTURE AND IMPLEMENTATION request setAttribute Constants SIM sim request setAttribute oddTweets sim getEventManager getStatistics getOddTweets request setAttribute broadTweets sim getEventManager getStatistics getBroadTweets request setAttribute acqTweets sim getEventManager getStatistics getAcqTweets request getRequestDispatcher Constants RUNNING PAGE forward request response The servlet stores in variables the information about the simulation once all the necessary variables are filled the servlet sends to the JSP page all the information in last line of code Run one step run and pause button have a similar behaviour but Stop button is a bit special If the user clicks this button the servlets executes the code necessary for store the simulation in the database and calculate the centrality of the network The code that manage this is the following Listing 4 19 Servlet handle setup form private void clickStop HttpServletRequest
20. name of the simulation that we want to load and in the New name for simulation field we will introduce the name which the network will be saved at the end of the simulation 71 CHAPTER 5 PROTOTYPE AND EXAMPLE USAGE gsi EUMD ums 2 Grupo de Sistemas Inteligentes UPM About BigMarket Set up Simulation Running Actions Contact Select an option New random network network from DataBase New random network E Load network from DataBase Number of initial nodes i Dataset identifier 100 CreateNetwork Simulation name New name for simulation Random network i LoadNetwork Figure 5 5 Load network setup Finally we click on Setup button and the network will be loaded In the next section we will run a simulation with this network 5 5 Running the simulation of the loaded network Once we have clicked on the Setup button BigMarket leads us to the Running screen Meanwhile the MASON step engine is booted In this screen we have the option to run one step of the simulation or simply let it running indefinitely until we want to pause it In this screen we can also see the information of the simulation number of users and its type the tweets and the steps like we can see in the figure 5 6 Once we want to finish the simulation we have to click the Stop button and Neo4J will save the simulation for us 72 5 6 RESULTS SCREEN AND FINAL ACTIONS gsi 2 di Ep
21. store any kind of data attribute on the graph elements numbers strings or any object Moreover in addition GraphStream provides a way to handle the graph evolution in time This means handling the way nodes and edges are added and removed and the way data attributes may appear disappear and evolve In order to handle dynamic graphs the library defines in addition to graph structures the notion of stream of graph events which as you guessed is at the origin of the library name The number of events is restricted they are e node addition e node removal e edge addition e edge removal e graph node edge attribute addition e graph node edge attribute change e graph node edge attribute removal e step Inside the library a lot of components can generate such streams of events These components are called sources Other components can receive these events and process them they are in fact very comparable to listeners a concept widely used in the Java world We call such components sinks When a component is able to both receive graph events sink and produce them source we call it a pipe The graph structures in GraphStream are pipes There are many kinds of pipes that can act as filter removing some events or adding more events or allow to cross the network or communicate between threads At the start of the chapter we talked about MASON Although MASON has mecha nisms to build graphs it is
22. the figure B 2 Then you have to introduce the number of initial nodes of your network Finally choose a name for your simulation this name is important because it will be the ID of your simulation when you have to store it in the database Once you have configured your simulation click on Set up 99 APPENDIX B USER MANUAL G 4 La gsi de Sistemas Inteligentes About BigMarket Set up Simulation New random network Number of initial nodes 4 Simulation name MySim Running Select an option New random network Load network from DataBase gs Actions Contact Load network from DataBase Dataset identifier New name for simulation Create network Figure B 2 New random network simulation step 2 5 In the running screen you can start the simulation and establish if it runs without 100 stopping or just step by step You have the following options Run with this option the simulation will run until you press stop or pause button Run one step as its own name means with this option you will run the simu lation one step Pause this button allows you to pause the simulation if it is running Stop this button ends the simulation and leads you to the next screen 1 RUN NEW RANDOM NETWORK SIMULATION gsi E en Grupo de Sistemas Inteligentes About BigMarket Set up Simulation Ru Time step 0 User type Number of users Number of tweets writed Bro
23. 39 CHAPTER 3 REQUIREMENT ANALYSIS 40 Architecture and implementation This chapter describes in depth how the system is structured in different modules and how the users interact with them In order to make the chapter more understandable to the users we will attach to each module its implementation We will describe each one of these modules describing its main purpose structure and function includ ing the module implementation After reading this chapter the user will know how the application and each of its modules work and how BigMarket implements these functions A1 CHAPTER 4 ARCHITECTURE AND IMPLEMENTATION 42 4 1 INTRODUCTION 4 1 Introduction In this chapter we show two detailed diagrams First we can see a class diagram that represents the class structure of BigMarket this diagram is represented by figure 4 1 Fol lowing this diagram we can see the diagram 4 2 that represents the complete architecture of BigMarket In the first section we introduce both schemes and the behaviour and the main function of each of the modules and components After this in the following subsec tions we describe each module in depth showing specific diagrams screenshots and detailing their particular operation also to help the user to understand how each module works we attach the implementation of each function 4 2 Architecture To define the architecture of BigMarket we have built two
24. Header mtHeader client executeMethod mGet response mGet getResponseBodyAsString JsonArray root JsonArray new JsonParser parse response for int i 0 i lt root size i JsonElement e root get i JsonObject obj e getAsJsonObject String s obj get self toString String finalS s substring 1 s length 1 nodesRetrieve add finalS 52 4 4 NEO4J DATABASE AND ITS IMPLEMENTATION catch Exception e System out println Exception in creating node in neo4j System out println nodesRetrieve public void getNodeRelations for String s nodesRetrieve try String response String nodePointUrl s relationships out HttpClient client new HttpClient GetMethod mGet new GetMethod nodePointUrl Header mtHeader new Header mtHeader setName accept mtHeader setValue application json mGet addRequest Header mtHeader client executeMethod mGet response mGet getResponseBodyAsString JsonArray root JsonArray new JsonParser parse response for int i 0 i lt root size 1 JsonElement e root get i JsonObject obj e getAsJsonObject String st obj get end toString String finalS st substring 1 st length 1 relationsRetrieve put s finalS Exception e System out println Exception in creating node in neo4j e System out
25. It has been structured as follows Chapter 1 provides an introduction to the problem which will be approached in this project It provides an overview of the benefits of SNSA framework Furthermore a deeper description of the project and its environment is also given Chapter 2 contains an overview of the existing technologies on which the development of the project will rely Chapter 3 describes one of the most important stages in software development the requirement analysis using different scenarios For this a detailed analysis of the possible use cases is made using the Unified Modelling Language UML This language allows us to specify build and document a system using graphic language The result of this evaluation will be a complete specification of the requirements which will be matched by each module in the design stage This helps us also to focus on key aspects and take apart other less important functionalities that could be implemented in future works Chapter 4 describes the architecture of the system divided in several modules with its own purpose and functions Chapter 5 describes a selected use cases It is going to be explained the running of all the tools involved and its purpose It allows us to test the application and give us some feedback to improve our system and repair bugs and errors Chapter 6 sums up the findings and conclusions found throughout the document and gives a hint about future develop
26. PROYECTO FIN DE CARRERA Titulo Titulo ingl s Autor Tutor Ponente Departamento Diseno e implementaci n de una herramienta web para el an lisis y simulaci n con MASON de redes sociales Design and implementation of a web framework for the anal ysis and simulation with MASON of social networks Daniel Lara Diezma Emilio Serrano Fern ndez Carlos A Iglesias Fern ndez Ingenier a de Sistemas Telem ticos MIEMBROS DEL TRIBUNAL CALIFICADOR Presidente Vocal Secretario Suplente Mercedes Garijo Ayestar n Marifeli Sedano Ru z Carlos ngel Iglesias Fern ndez Jos Carlos Gonz lez Crist bal FECHA DE LECTURA CALIFICACI N UNIVERSIDAD POLIT CNICA DE MADRID ESCUELA T CNICA SUPERIOR DE INGENIEROS DE TELECOMUNICACION Departamento de Ingenier a de Sistemas Telem ticos Grupo de Sistemas Inteligentes PROYECTO FIN DE CARRERA DESIGN AND IMPLEMENTATION OF A WEB FRAMEWORK FOR THE ANALYSIS AND SIMULATION WITH MASON OF SOCIAL NETWORKS Daniel Lara Diezma Junio de 2015 Resumen El objetivo de este proyecto fin de carrera es el desarrollo e implementaci n de una aplicaci n web que permita la simulaci n de redes sociales para poder analizar la difusi n de rumores de acuerdo a la conexi n entre los distintos tipos de usuarios este prop sito se ha desarrollado un framework web capaz de generar redes a partir de un numero inicial de nodos introducido por el usuario
27. Structure of this Master Thesis 4 L4 State ofarU i2 238 ok ke wes Xe3 dm RS RUSO X ded XY Ae omo XU en 5 WAG eve cim dog Ehe o A ce Beh 5 1 4 2 ThinkVine e e we x Be ae 6 2 Enabling technologies 9 2 4 MASON ceea sadiay a ka pap daa sS ss ss 11 2 2 AGODA Eo Go Roger ge EO Sins HE ensis 12 20 Neol cocineros oko wu om A o E e e 13 2 4 GraphStream 14 2 5 Other technologles 5 cuore a ara ra en rg ego 16 2 5 l Apache Tomcat exe ala nl Wed xo nd Rue a 16 2 0 2 JAVA sies sinh Ss vs a RES quadr dne A awe Hes 17 2 5 9 ya ea Sh Spe ae EU p 17 2 94 EDIT TEA cud Au utis En a et 18 2 9 5 RU Ue eer OUR row Santi torio ge sae 18 3 Requirement Analysis 19 3 1 Overview eg ere Rue P E a eee X RD dou 21 9 2 Use cases uec wa Pu Eg ES UE hp qr LIRE RE Xs 21 3 21 Actors dictionary 2er cece eine Rue 21 3 2 2 BigMarket user use cases 23 3 2 2 1 25 3 2 2 2 Creating new random network 26 3 2 2 3 Creating network with 4 27 3 2 2 4 Loading network from database 28 3 2 2 5 Running
28. TATION In the next paragraphs we will do a simply introduction of how BigMarket works and in the following sections we will explain each part with more detail Since the main purpose of this master thesis is to develop an HTML5 Framework to build analyse and represent the evolve of a social network in the time the whole actions that the users can do will be done through a web interface that will connect with a servlet that receives the request of the user and will do the properly action depending the request In order to create the initial network non technical users can use the automatic Big Market network builder engine in order to create their own networks this engine builds a network with the number of the initial nodes using an algorithm based on the popularity of each user it means that a new user is more probably that follows an old user that has a lot of followers instead of an user that has less followers The more technical users can build their networks using the querys provided by Neo4J with this method the user of BigMarket is who establish how many nodes are in the begin of the simulation and how they are connected Once the network is created BigMarket will use in each step the same algorithm explained with the non technical users to introduce and connect the new nodes Once the network is created and the user starts the simulation the MASON step engine takes the control In each step MASON step engine introduce a
29. Tomcat 7 0 e Eclipse Neo4J 4 Gephi Git A 1 2 Downloading the source code For downloading the source code from Git you have to follow these steps 1 Create the file system where you are going to clone the git repository 2 Open Git Bash and go to the path created in the previous step 3 Once you are in the path execute the next command git clone https github com gsi upm BigMarket git If all has gone well you have to see a screen like this Figure 1 Git Bash console capture Now you have the source code in your own computer Http www oracle com http tomcat apache org https eclipse org http neo4j com http gephi github io Shttp git scm com 83 APPENDIX A INSTALLING AND RUNNING A BIGMARKET SERVER A 1 3 Importing the project in Eclipse In this section we will explain how to import the project in Eclipse In the previous section we obtained the source code from GitHub so we will continue from that point In order to import the project in Eclipse you have to follow the next steps 1 Open Eclipse 2 Right click on the Project explorer view and select Import 3 In the new window select Git Projects from Git Select x 24 Import more projects from Git Repository AJ Select an import source type filter text Git E Project fom Git amp Install amp Java EE gt amp Maven b amp Plug i
30. a script code for network visualization script src js sigma min js gt lt script gt lt script src js sigma layout forceAtlas2 min js gt lt script gt lt script src js sigma parsers json min js gt lt script gt lt script gt sigma parsers json networkGraph json container container settings defaultNodeColor 337AB7 y function s s startForceAtlas2 5 lt script gt 4 6 Gephi and export to GEXF file implementation Despite of Gephi is no part of BigMarket we will talk about it because is the tool that we have use to open the GEXF files generated by BigMarket 95 CHAPTER 4 ARCHITECTURE AND IMPLEMENTATION Once the simulation has finished BigMarket offers us the option to download the graph in GEXF file in order to make network analysis with more detail To achieve this BigMarket converts the graph to Graph Exchange XML Format GEXF and saves it in a file This file will be download to the user s computer The code that allows this conversion is the following Listing 4 12 Save graph in GEXF file private void exportGraphGEXF String path getServletContext getRealPath grafoInicial gexf FileSinkGEXF file new FileSinkGEXF try file writeAll sim getGraphManager getGraph path catch IOException e e printStackTrace 4 7 User web interface and Servlet and their implementation In this section
31. adcaster 0 null Acquaintances null null Odd users 0 null Grupo de Sistemas Inteligentes GSI UPM Figure B 3 New random network simulation step 3 Once you press stop button BigMarket will start to store the data in the database if you run BigMarket in your own computer pay attention that Neo4J server is up like we explained you in the previous appendix 6 The next screen is the Actions screen This screen is divided in two parts e SNA results this part contains a table that represents the results from the analysis of the simulation of the network that you have been created in the previous steps e Actions this part have two buttons The first of them the network button allows you to see the network in a new window The second button Download graph button let you to download the graph in order to analyse it with a SNA tool like Gephi 101 APPENDIX B USER MANUAL di 1 5515 e 53 Grupo de Sistemas Inteligentes About BigMarket Set up Simulation Running Actions Contact i See the graph of the network Node Meassure E 58 0 0 Export the graph in a gexf file 30 20 E Figure B 4 New random network simulation step 4 B 2 Load network In this section we will explain you how to load a network from a previous simulation If you run BigMarket in your own computer pay attention to have the Neo4J server up Now to load a network follow the next steps 1 In the index s
32. atabase new random network etc At the begin of this use case the servlet should be started and the Setup simulation screen must be displayed At the end of the use case the network will be created This table also represent the flow of events that allows creating the network Use Case Name Setup the network Use Case ID UCI 1 Pre Condition The Setup simulation screen has been displayed Post Condition The network which will be used in the simulation has been created successfully Flow of Events Actor Input System Response The user clicks on the Start button in the index screen The Setup simulation screen is displayed The user selects how he she want to create the network The fields to setup the network in order to the selection of the user are enabled The user fills the mandatory fields The Setup button becomes enabled The user clicks on the Setup button The network is created Table 3 2 Setup the network use case 25 CHAPTER 3 REQUIREMENT ANALYSIS 3 2 2 2 Creating new random network This use case is an extend use case of the previous use case In this case the action will be to create a new random network Like the previous use case at the begin we have the servlet started and the Setup simulation screen displayed At the end of the use case a new random network will have be
33. b App Libraries Import libraries in an Eclipse project step 2 gt BA JRE System Library JavaSE 1 7 89 APPENDIX A INSTALLING AND RUNNING A BIGMARKET SERVER 3 In the JAR selection window go to the path where you installed Apache Tomcat 7 Once you are in it go into folder lib and double click on servlet api CJ J Apache apache tomcat 1061 lib Buscar ub Organizar Nueva carpeta m0 Fass 2 Nombre Fecha de modifica Tipo al di Descargas annotations api 27 03 201514 04 Executable Jar File BE Escritorio lil catalina 27 03 201514 04 Executable Jar File El Sitios recientes catalina ant 27 03 2015 14 00 Executable Jar File J Datos catalina ha 27 03 2015 14 04 Executable Jar File E everdrive la catalina tribes 27 03 2015 14 04 Executable Jar File lil ecj 4 4 2 27 03 201514 04 Executable Jar File GB Bibliotecas lil el api 27 03 20151404 Executable Jar File E Documentos 8 jasper 27 03 201514 04 Executable Jar File E everdrive E jasper el 27 03 2015 14 04 Executable Jar File EE Imagenes la jsp api 27 03 201514 04 Executable Jar File a M sica servlet api 27 03 20151444 Executable Jar File El Subversion l tomcat7 websocket 27 03 20151444 Executable Jar File Videos 4 m Nombre jar zip Figure 12 Import libraries in an Ecl
34. built the WAR Now is time A 2 RUN A BIGMARKET SERVER to deploy it in our Apache Tomcat Server To do this go to the path where you saved the WAR copy the WAR generated and finally paste it in the following path TOMCATHOME webapps 2 Now for running the server open a shell in Windows a cmd and go to TOM CATHOME bin Once you are in this path execute the next command startup bat If all has gone well a new window will be opened do not close this windows and few seconds later you should see this line Server startup in X ms 3 Finally in order to check that all is alrigth open a browser and go to http localhost 8080 BigMarket Now you know how to import the code in your own computer how to import it in Eclipse in order to adapt the project to your own purposes and finally you have learnt how to run the application in a server in order to test it and probe the changes that you will make to the code In the next Appendix we will teach you how to use the application on the scenarios exposed in the use cases from the Chapter 3 Change localhost to the IP address of the server machine if the client is in another host 95 APPENDIX A INSTALLING AND RUNNING A BIGMARKET SERVER 96 APPENDIX User manual This user manual goes through the most important features for users Like we men tioned in the last appendix the code of the project is available at https
35. case Use Case Name Running the simulation Use Case ID UC1 6 Pre Condition The simulation is running Post Condition The simulation is stored in the database Flow of Events Actor Input System Response The simulation is stopped The user clicks on the Stop and stored in the database button The Results screen is displayed Table 3 7 Saving the simulation in database use case 30 3 2 USE CASES 3 2 2 7 Analysing the network This use case represents the analysis of the network At the beginning of the use case the simulation has been stored and the Results screen has been displayed The table also represents the flow of events of the use case Use Case Name Analysing the network Use Case ID UC1 7 Pre Condition The simulation has been stopped and stored in the database The Results screen has been displayed Post Condition N A Flow of Events Actor Input System Response The user can analyze the results and restart a new N A simulation going to the Setup simulation screen Table 3 8 Analysing the network use case 31 CHAPTER 3 REQUIREMENT ANALYSIS 3 2 2 8 Downloading the graph This use case represents the option to download the graph that contains the information of the network in order to analyse it with a SNA tool At the beginning of the use case the simulation
36. creen press the Start button like in the step 1 of the previous section 2 Now in the Set up screen you have to select the Load network from DataBase Then write the identifier of the simulation that you want to load and finally establish a new name for the simulation in order to store it in the database Once you have fill all the fields press the Setup button 102 B 3 CREATE A NETWORK amp dit Y gsi E il Sow gsi Grupo de Sistemas Inteligentes UPM About BigMarket Set up Simulation Running Actions Contact Select an option New random network 9 network from DataBase New random network E Load network from DataBase Number of initial nodes i Dataset identifier E MySim Simulation name New name for simulation 2 Create network Figure B 5 Load network step 1 In the following section we will explain you the use of the Create network button 3 The next screen is the Running screen and its use is the same as we explained in the step 5 of the previous section 4 The use of the last screen Actions screen is the same of the step 6 of the previous section B 3 Create a network This section explain you how to create a network The creation of a network is based on the Neo4J database that allows you create nodes and its relationships using the Neo4J commands You can find the API in the Neo4J web page To create a network sele
37. ct Load a network from database on Setup screen and click on Create network It will open you a new window with a visualization of Neo4J server In this new window you have to use the Cypher query language of Neo4J to create the network You can find a complete tutorial in this link http neo4j com developer cypher query language Ihttp neo4j com developer cypher 103 APPENDIX B USER MANUAL 104 Bibliography E Serrano G Poveda and M Garijo Towards a Holistic Framework for the Evaluation of Emergency Plans in Indoor Environments Sensors vol 14 no 3 pp 4513 4535 2014 E Otte and R Rousseau Social network analysis a powerful strategy also for the information sciences Journal of Information Science vol 28 no 6 pp 441 453 2002 B L Said T Bouron and A Drogoul Multi Agent Based Simulation of Consumer Behaviour Towards a New Marketing Approach in International Congress On Modelling and Simulation MODSIM 2001 Canberra Australie d 2001 A Hummel Kern S K hne and D hler agent based simulation of viral marketing effects in social networks 26th European Simulation and Modelling Conference ESM 2012 212 219 2012 105 BIBLIOGRAPHY 106
38. diagrams The first of them is a class diagram built following the specifications of UML Class Diagrams UML 2 class diagrams are the mainstay of object oriented analysis and design UML 2 class diagrams show the classes of the system their interrelationships including inheritance aggregation and association and the operations and attributes of the classes Class diagrams are used for a wide variety of purposes including both conceptual domain modeling and detailed design modelling Following this diagram we can find a component diagram that give us a global vision of the architecture of BigMarket to make the component diagram we have following the specifications of UML Component Diagrams http www agilemodeling com artifacts classDiagram htm http agilemodeling com artifacts componentDiagram htm 43 CHAPTER 4 ARCHITECTURE AND IMPLEMENTATION SimulationGUI uses GraphManager GraphJSONParser Neo4JManageTool UserModel Figure 4 1 UML class diagram 44 4 2 ARCHITECTURE 5 2 Gephi Manage SigmaJS tool Export graph to GEXF file Network visualization Request of the users Tomcat server User web interface Store network to database Neo4J Response of the system Evolve of the network in time Simulation running options Load network from database Figure 4 2 UML component diagram CHAPTER 4 ARCHITECTURE AND IMPLEMEN
39. e possibility to develop a new network algorithm To do this the developer should download the code to his her computer Then he she can develop his her own network building algorithm Use Case Name New network building algorithm Use Case ID UC2 2 Pre Condition The developer has download the code Post Condition N A Flow of Events Actor Input System Response The developer can code his her own class in which The developer imports the 1 the developer defines the new project in Eclipse algorithm in order to build the network Table 3 12 New network building algorithm use case 36 3 2 USE CASES 3 2 4 Admin use cases This use case package collects the admin functionalities of BigMarket as shown in 3 3 The use cases presented in this section are as shown in the Figure 3 3 The functions of the admin actor will be implemented in future works e Manage the database detailed in sub section 3 2 4 1 e Manage the user s permissions detailed in sub section 3 2 4 2 BigMarket Manage x Mr Fi Admin Manage database Figure 3 3 Admin use cases 37 CHAPTER 3 REQUIREMENT ANALYSIS 3 2 4 1 Manage the database In this use case the possibility to manage the database by an admin is represented Use Case Name Manage the database Use Case ID UC3 1 Pre Condition The database needs to be repair delete registers resolve problems etc
40. e she has a normal number of followers he she is catalogued like acquaintance and finally if the user has a few number of followers he she is and odd user Because of 48 4 44 NEO4J DATABASE AND ITS IMPLEMENTATION this BigMarket modify the behaviour of the users at the end of each simulation step once the new users and their relationships are added and the old users relations are modified Finally this is the code that allows BigMarket to modify the users behaviour Listing 4 4 Modifying behaviour private void setUserType Simulation sim User user Graph graph sim getGraphManager getGraph double enteredEdges 0 0 double totalEdges graph getEdgeCount double percentage 0 0 enteredEdges user getFollowers size percentage enteredEdges totalEdges if percentage gt 0 3 user set Type Constants USER TYPE BROADCASTER System out println EL usuario user getUserName es un Constants USER TYPE BROADCASTER telse if 0 3 gt percentage amp amp percentage gt 0 1 user set Type Constants USER TYPE ACQUAINTANCES System out println EL usuario user getUserName es un Constants USER TYPE ACQUAINTANCES telse user set Type Constants USER_TYPE_ODDUSERS System out println EL usuario user getUserName es un Constants USER TYPE ODDUSERS 4 4 Neo4J database and its implementation The main purpose of the part
41. eated MATCH n CreateNetwork REIURN n LIMIT 25 69 CHAPTER 5 PROTOTYPE AND EXAMPLE USAGE In the figure 5 3 we can see a representation of the previous query 3 2 9 CreateNetwork 3 Graph Displaying 3 nodes 0 relationships AUTO COMPLETE Figure 5 3 Nodes created Now is time to create the relations between nodes We will join the node 1 with the node 2 and the node 3 with the node 1 Note the direction of the relationship is important SO e Nodel Node2 e Node3 Nodel To do this we write in the Neo4J interface the next query Listing 5 5 Nodes created MMATCH a CreateNetwork b CreateNetwork WHERE a name Nodel AND b name Node2 CREATE a r RELTYPE b To see the final network created we can execute the query used in the previous pint to see the nodes created So the final network would be 70 5 4 LOADING NETWORK FROM DATABASE MATCH n CreateNetwork RETURN n LIMIT 25 amp o e Y CreateNetwork 3 e Graph 2 ELTYPE 2 Sy Pa Figure 5 4 Final network 5 4 Loading network from database In the lasts sections we had created two networks Now is time to load one of this and execute a simulation with it We will load the network created using Neo4J To do this in the Setup simulation we have to select the options and fills the fields like in the figure 5 5 In the Dataset identifier field we will introduce the
42. ecification of the uses of the system and therefore define the complete list of requisites to match First we will present a list of the actors in the system and a UML diagram representing all the actors participating in the different use cases This representation al lows apart from specifying the actors that interact in the system showing the relationships between them These use cases will be described the next sections including each one a table with their complete specification Using these tables we will be able to define the requirements to be established 3 2 1 Actors dictionary The list of primary and secondary actors is presented in table 3 1 These actors participate in the different use cases which are presented later 21 CHAPTER 3 REQUIREMENT ANALYSIS 22 Actor identifier Role Description ACT 1 User End user that uses BigMarket in order to make a new simulation or load a previous simulation to study possible marketing strategies ACT 2 Developer Technical developer which code their own behaviours and networks for use them in BigMarket ACT 3 Admin Administrator of BigMarket this actor will be implemented in future works its principal purpose will be to manage the users accounts maintenant the service and lookup that all services server database etc are up Table 3 1 Actors list 3 2 USE CASES 3 2 2 BigMarket user use cases Th
43. ed information to maintain and develop future improvements Keywords Social simulation social network user graph behaviour relationship social network analysis Big Data Twitter VII Agradecimientos Me gustar a darle las gracias a todas aquellas personas que han estado d a a d a d ndome nimos durante los a os de carrera y que han estado ah en los momentos dif ciles Y en concreto a mi madre y a mi padre por darme la posibilidad de poder estudiar una carrera a mi novia por hacer que cada d a me levante con ganas de comerme el mundo a los Signoritos por hacer de mi paso por esta escuela una gran experiencia al club de teatro NECN por conseguir que recuerde con mucho carino los ltimos anos de carrera al resto de companeros de universidad por acompanarme en este camino a todos los profesores de la ETSIT porque sin ellos no habr a llegado hasta aqu y en concreto a Carlos Iglesias por darme la oportunidad de entrar pronto en el DIT y descubrir que me encantaba la programaci n a mi tutor Emilio por guiarme en este y por ltimo a mis amigos de toda la vida por apoyarme en los momentos duros Finalmente gracias una vez m s a todas aquellas personas que me han estado a mi lado hasta llegar hasta aqu Gracias IX Contents Resumen Abstract VII Agradecimientos IX Contents XI List of Figures XV List of Tables XIX 1 Introduction 1 3 1 2 Master thesis goals s 3 1 3
44. elected by the user This field will be read by the servlet in order to make an action according to the user choice The running page also show dynamically the information about the simulation steps number and types of users and number of tweets of each user type The code that allows this is the following Listing 4 16 Servlet handle setup form lt 59 CHAPTER 4 ARCHITECTURE AND IMPLEMENTATION 60 Integer broadUsers Integer request getAttribute broadUsers Integer acqUsers Integer request getAttribute aqUsers Integer oddUsers Integer request getAttribute oddUsers Integer oddTweets Integer request getAttribute oddTweets Integer acqTweets Integer request getAttribute acqTweets Integer broadTweets Integer request getAttribute broadTweets Long steps Long request getAttribute steps Simulation sim Simulation getAttribute sim lt div class jumbotron gt lt table class table table striped id tableA gt lt tr gt lt th gt Time step lt th gt lt td lt steps gt lt td gt lt tr gt lt div gt lt div class row gt lt table class table table striped id tableB gt lt tr gt lt th gt User type lt th gt lt th gt Number of users lt th gt lt th gt Number of tweets writed lt th gt lt tr gt lt tr gt lt th gt Broadcaster lt td gt lt td gt lt broad Users
45. en created The table also represent the flow of the events of the use case Use Case Name Creating new random network Use Case ID UC1 2 Pre Condition The Setup simulation screen has been displayed TT The network which will be use in the simulation has been Post Condition created successfully Flow of Events Actor Input System Response The fields for creating new The user selects New 1 random network become random network enabled 5 The user fills the new The Setup button random network fields becomes enabled The user clicks on the 3 The network is created Setup button Table 3 3 Creating new random network use case 26 3 2 USE CASES 3 2 2 3 Creating network with Neo4j This use case represents the creation of a network using the Neo4J web interface It extends from Setup the network use case Like the previous use case at the begin we have the servlet started and the Setup simulation screen displayed At the end of the use case the network will have been created and stored in the database The table also represent a flow of events of the use case Use Case Name Creating network with Neo4j Use Case ID UC1 3 Pre Condition The Setup simulation screen has been displayed Post Condition The network which will be used in the simulation has been created successfully Flow of Events Actor Input System Resp
46. ependen cles as possible It is intended to let application developers write once run anywhere WORA meaning that compiled Java code can run on all platforms that support Java without the need for recompilation Java applications are typically compiled to bytecode that can run on any Java virtual machine regardless of computer architecture 2 5 3 HTML5 HTML Figure 2 10 HTML5 Logo HTML5 is a core technology markup language of the Internet used for structuring and presenting content for the World Wide Web of October 2014 this is the final and complete fifth revision of the HTML standard of the World Wide Web Consortium W3C The previous version HTML 4 was standardised in 1997 Its core aims have been to improve the language with support for the latest multimedia while keeping it easily readable by humans and consistently understood by computers and 5http tomcat apache org Shttps www java com en 17 CHAPTER 2 ENABLING TECHNOLOGIES devices web browsers parsers etc HTML5 is intended to subsume not only HTML 4 but also XHTML 1 and DOM Level 2 HTML 2 5 4 CSS3 Figure 2 11 CSS3 Logo Cascading Style Sheets CSS is a style sheet language used for describing the look and formatting of a document written in a markup language While most often used to change the style of web pages and user interfaces written in HTML and XHTML the language can be applied to any kind of XML docume
47. er to work with it in our workspace To do this you have to complete this step Select the project and click on Project Properties Project Facets Once you are in this windows select the boxes like in the figure A 9 When you have marked the boxes click on Apply and Ok A 0 CN NN Re type Filler text Project Facets Resource r n Builders Configuration custom Seve Delete ee Project Facet Version Detsiis Runtimes Java Build Path Application Chent module iT Application Client module 6 0 hava Code Style Web Services La Web Seres 12 Enables the project to be deployed es a Java EE Application ha Comites Maid gt Client module Java Editor J Co Dynamic Web Module Javadoc Location EAR gt Requires the following facet Javascript E pe A i e j lave 16 ot newer ioclet XDoclet 2 Ji D deve ert Conflicts with the following facets n act E J i JavaSenpt ci Proyect References Application Client module lavaServer Faces 22 Run Debug Settings Dymamie Web Module 1 RS REST Web Services 11 Server e Service Policies Ws 22 y FIR Module Targeted Runtimes JCA Module 16 JCA Module Task Repository PA 23 Static Web Module a Stets Web Module ask Tags Utility Module Vidit Utility Module wats aud i eb Fragment Module Web Content Settings dh Modde Web Pege Editor WebDoclet XDectet 123 Web Project Settings
48. eral gt 2 gt amp Install gt Es Java gt Java EE gt amp Plug in Development gt Remote Systems gt Run Debug gt amp Tasks gt E Team 4 amp Web WAR file gt Web Services b amp XML Figure A 15 Building the WAR step 2 92 A 2 RUN A BIGMARKET SERVER Export Web project to the local file system Web project BigMarket Destination CA Users dlaradie Desktop BigMarket war Target runtime C Optimize for a specific server runtime Export source files Overwrite existing file Figure A 16 Building the WAR step 3 A 2 3 Running a the application There are two ways for running the application e Running the application in Eclipse this way allows you to run Big Market in Eclipse To do this follow the next steps 1 Select the project and open the run icon Select Run As and finally click on Run on server FLOTA gc PIE no launch history Run As gt 8 1RunonServer Alt Shift X R Run Configurations 2 Java Applet Alt Shift X A 3 3Java Application Alt Shift X J Organize Favorites Figure A 17 Running Big Market in Eclipse step 1 2 In the new window open the Apache folder and select Tomcat v7 0 server then click on Next 93 APPENDIX A INSTALLING AND RUNNING A BIGMARKET SERVER 94 Wannsee iy Run On Server Se
49. etwork simulation step 1 99 2 New random network simulation 2 100 New random network simulation step 3 101 B 4 New random network simulation step 4 102 B 5 Load network 1 108 3 1 3 2 3 3 3 4 3 5 3 6 3 7 3 8 3 9 3 10 3 11 3 12 3 13 3 14 List of Tables Actors list Oe don TL peus a 22 Setup the network use case 25 Creating new random network use case 26 Creating network with Neo4j use case 27 Loading network from database use case 28 Running the simulation use case 29 Saving the simulation in database use case 30 Analysing the network use case 2 CC om onen 31 Downloading the graph use case 32 Network visualization use case 33 New user behaviour use case 35 New network building algorithm use case 36 Manage the database use case 38 Manage the user s permissions use case 39 XIX Introduction This chapter provides an introduction to the problem which will be approached in this project It provides the context and the importance of the software
50. gMarket do the SNA analyse in combination with the Gephi API For more information about the integration between BigMarket and Gephi API please refer to the chapter 4 http gephi github io 12 2 3 NEO4J tin Figure 2 3 Gephi Framework 2 3 Neo4J Neo4j o Figure 2 4 Neo4J Logo Neo4J is an open source graph database implemented in Java The developers describe Neo4j as embedded disk based fully transactional Java persistence engine that stores data structured in graphs rather than in table Neo4J features e Performance Neo4j s native graph engine is engineered to let navigate hyper connectivity at speed Built from the bottom up to support property graphs Neo4j allows you to connect the nodes easily and with unparalleled performance and reliability e Scalability Neo4j scales up and out supporting tens of billions of nodes and their relationships and hundreds of thousands of ACID Atomicity Consistency Isolation and Durability transactions per second In our project we will need a database in order to store the simulations so that the user can recover a simulation that he have made in the past BigMarket uses graphs to represent a social network or a society representing the people like nodes and the relations between them like edges Thus if we have a graph to represent the society we will need a graph database in order to store it 3http neo4j com
51. github com gsi upm BigMarket 97 APPENDIX B USER MANUAL 98 1 RUN NEW RANDOM NETWORK SIMULATION B 1 Run new random network simulation In this section we will explain you how to create a new random network simulation This is the simplest way to use BigMarket In order to create a new random network simulation you have to follow the next steps 1 First of all you have to have deployed the WAR in an Apache Tomcat Server and run the server like we explained in the Appendix A 2 Once you have completed the step 1 open a browser and go to http localhost 8080 BigMarket 3 Now you should see the index page of the application In this screen click on start in order to go to the set up screen e A puta gsi 2 Grupo de Sistemas Inteligentes UPM About BigMarket Set up Simulation Running Actions Contact BigMarket is a web application that allows you to run simulations based on different types of networks Tha aim of BigMarket is to provide an open source tool to assist on Agent Based Social Simulation ABSS The project have been developed by the Grupo de Sistemas Inteligentes GSI UPM from the Universidad Polit cnica de Madrid UPM Grupo de Sistemas Inteligentes GSI UPM Figure B 1 New random network simulation step 1 4 Like the objective of this section is to teach you how to run a simulation for a random network you have to select New random network in the Set up screen like you can see in
52. h random popularity size CHAPTER 4 ARCHITECTURE AND IMPLEMENTATION Node nl graph getNode popularity get random connectNewUsers sim nl n2 public void connectNewUsers Simulation sim Node nl Node n2 Graph graph sim getGraphManager getGraph graph addEdge Integer toString graph getEdgeCount 1 n2 nl true Follow f new Follow Follow graph getEdgeCount 1 TS graph getEdgeCount 1 sim getUsers get nl getIndex sim getUsers get n2 getIndex e Establishing new relations when the new users and their relationships are incorporate in the network the next step is establish new relations between the old users We will use the same algorithm of the last point to do this You can see the implementation of this function in the following block Listing 4 3 Establishing new relations public void lookForNewUsers Simulation sim Graph graph sim getGraphManager getGraph for User u sim get Users if u getFollowed size 0 amp amp u getFollowers size 0 Node n2 graph getNode u getId int random int Math random popularity size Node nl graph getNode popularity get random connectNewUsers sim nl n2 e Modifying the behaviour BigMarket assign the behaviour to each user according their followers If the users have a lot of followers we assume that the user is a broadcaster if h
53. hard to build dynamic graph and modify the visualization in 15 CHAPTER 2 ENABLING TECHNOLOGIES real time With GraphStream we do not have these problems In GraphStream we count with layouts too that allows us to represent the graphs in a way that is easier to identify the elements So these features makes GraphStream a good graph builder and visualizator for our purposes removing the visualization part of MASON Figure 2 7 GraphStream Graph Example 2 5 Other technologies In this section we talk about other technologies that help us to build our project but they are well known or they have contributed in a lesser way to our project so we will extend less explaining these technologies 2 5 1 Apache Tomcat Apache Tomcat Figure 2 8 Apache Tomcat Logo Apache Tomcat is an open source web server and servlet container developed by the Apache Software Foundation ASF Tomcat implements several Java EE specifications including Java Servlet JavaServer Pages JSP Java EL and WebSocket and provides a 16 2 5 OTHER TECHNOLOGIES pure Java HTTP web server environment for Java code to run in In our project we will use Tomcat to publish the service and make it accessible from Internet 2 5 2 Java Java Figure 2 9 Java Logo Java is a general purpose computer programming language that is concurrent class based object oriented and specifically designed to have as few implementation d
54. has been stored and the Results screen has been displayed Finally the user will have in his her computer the GEXF file that represents the network Use Case Name Downloading the graph Use Case ID UC1 8 Pre Condition The simulation has been stopped and stored in the database The Results screen has been displayed Post Condition The user has in his her computer the GEXF file with the network Flow of Events Actor Input System Response The user clicks on A new window appear asking where the file will be saved 1 to the user where he she wat Download graph button to save the file 5 The user selects the path The file is downloaded to the user 32 Table 3 9 Downloading the graph use case 3 2 USE CASES 3 2 2 9 Network visualization This use represents the possibility to see the network online without the necessity of down loading the graph At the beginning of the use case the simulation has been stored and the Results screen has been displayed Finally a new window with the network visualization is displayed Use Case Name Network visualization Use Case ID UC1 9 Pre Condition The simulation has been stopped and stored in the database The Results screen has been displayed Post Condition A new window is displayed with a representation of the network Flow of Events Actor In
55. ication more accessible and that it can reach more users Implement user session The idea is to make a simple login in order to allow the user to store their networks and simulations with privacy so only him her can load these simulations in the future Modify the behaviours in real time Now once the network has been created the user have not control over it the BigMarket engine is the one which controls the behaviour assignment So in the future the idea is that the user can stop the simulation and modify it introducing new agents or modifying whose exist yet List with examples of simulations Give to the user group of example networks Simply accessible by deploying a list in the tool GUI Implement the role of administrator Create the role of administrator of BigMarket who manages the permissions of the different users manage the database solve the problems of the users etc APPENDIX Installing and running a BigMarket server This tutorial goes through the process of installing and running BigMarket in any computer with a Windows OS Project s code is available at https github com gsi upm BigMarket git After the installation the user will be able to run the application itself or to modify the source files in order to introduce his own changes 8l APPENDIX A INSTALLING AND RUNNING A BIGMARKET SERVER 82 A 1 INSTALLATION A 1 Installation A 1 1 Requirements e JDK 1 7 Apache
56. iour of the agents according real buying habits Watching the data of previous years it assigns to each agent a buying frequency It introduces the data provided by the enterprise about its marketing strategies and the consumers and its habits Finally it recreates the sales using its own rules e Agent type Consumers they establish relationships between them and with the environment variables Ihttp www ohal group com 1 4 STATE OF ART e Agent Properties The agents are based on real demographic statistics The models are created taking samples from the society whose simulation is wanted Buying habits and social media use behaviours are assigned to the consumers variation in a single person does not change the behaviour of the whole society e Implementation available like in the previous case the software is not available e Comments it is a very interesting framework for building the model of each agent 2 2http www thinkvine com CHAPTER 1 INTRODUCTION A Enabling technologies This chapter introduces which technologies have made possible this project First of all we must introduce MASON an ABSS tool After that we speak about Gephi a SNA tool Then we comment Neo4J a graph database Finally we present the other technologies that have helped us to develop this project CHAPTER 2 ENABLING TECHNOLOGIES 10 2 1 MASON 2 1 MASON
57. ipse project step 3 4 Finally click on Ok and see how the errors are solved 18 Properties for B type filter text Java Build Path ce Resource Builders 19 Source amp Projects Eh Libraries i Orderand Export Deployment Assembly JARs and class folders on the build path ER a gt ea servet apijar C Users dlaradie Datos Apache apach Add JARS Java Build Path BA CAR Libraries li AOR _ gt Java Code Style m JRE System Library JavaSE 1 7 Add ExternalJARs Java Compiler ep p Java Editor a eb in Add Variable Javadoc Location JavaScript Add Class Folder Project Facets Project References Add External Class Folder Run Debug Settings Server Edit Seniice Policies Targeted Runtimes Task Repository Task Tags Migrate JAR File Validation Web Content Settings Web Page Editor Web Project Settings WikiText XDoclet m Add Library Remove J Figure A 13 Import libraries in an Eclipse project step 4 90 A 2 RUN A BIGMARKET SERVER A 1 5 Running the Neo4J database In this subsection we will teach you how to download the database and run it in order to store your simulations 1 First of all you have to download the Neo4J database exe from this link http neo4j com download 2 Once you
58. is use case package collects the user functionalities of BigMarket as shown in 3 1 The use cases presented in this section are as shown in the Figure 3 1 e Setup the network detailed in sub section 3 2 2 1 e Creating new random network detailed in sub section 3 2 2 2 e Creating network with Neo4j detailed in sub section 3 2 2 3 e Loading network from database detailed in sub section 3 2 2 4 e Running the simulation detailed in sub section 3 2 2 5 e Saving the simulation in database detailed in sub section 3 2 2 6 e Analysing the network detailed in sub section 3 2 2 7 e Downloading the graph detailed in sub section 3 2 2 8 e Network visualization detailed in sub section 3 2 2 9 23 CHAPTER 3 REQUIREMENT ANALYSIS BigMarket lt lt extends gt gt _ Setup the lt lt extends gt gt network nae lt lt extends gt gt RS Run the simulation Save simulation in database extends 7 User Analyze the network M lt lt extends gt gt s lt lt uses gt gt Calculate centrality Figure 3 1 User use cases 24 Create new random network Create network with Neo4J Load network from database Download graph See network 3 2 USE CASES 3 2 2 1 Setup the network This use case represent the action of creating a network In this case we will not enter in details about the way to create the network loading from d
59. ively Once the user fills the mandatory fields he she clicks on the Setup button and submit the form to the servlet The servlet has a doGet method to handle this form specifically the part corresponding to this form is the following Listing 4 14 Servlet handle setup form String formName request getParameter Constants FORMNAME if formName equals Constants SETUP FORM NAME String radioButtons request getParameter Constants RADIO BUTTONS if radioButtons equals Constants RANDOM SELECTED int numberOfNodes Integer parseInt request getParameter Constants NUMBER OF NODES String randomNetworkName request getParameter Constants RANDOM NETWORK NAME launchSimulation request response numberOfNodes randomNetworkName telse if radioButtons equals Constants LOAD SELECTED Neo4JManageTool n new Neo4JManageTool n launchLoad request getParameter datasetIdentifier sim new Simulation System current TimeMillis sim setDataBase n sim setFlag 2 String data request getParameter newLoadName sim setSimDataSet data Launcher launcher new Launcher sim launcher start request setAttribute broadUsers getBroadUsers sim request setAttribute acqUsers getAqUsers sim oddUsers getOddUsers sim Constants STEPS sim schedule getSteps request setAttribute ya Vom A IE request setAttribute 0 request setAttribute
60. lect which server to use How do you want to select the server Choose an existing server 9 Manually define a new server Download additional server adapters Select the server type type filter text 5 Tomcat v3 2 Server B Tomcat v4 0 Server E Tomcat v4 1 Server B Tomcat v5 0 Server Tomcat v5 5 Server B Tomcat v6 0 Server m iB Tomcat v7 0 Server Me ur non Publishes and runs J2EE and Java EE Web projects and server configurations to a local Tomcat server I Server s host name localhost j Server name Tomcat v7 0 Server at localhost Always use this server when running this project Figure A 18 Running Big Market in Eclipse step 2 3 In the last screen select the path where you installed Apache Tomcat and finally click on Finish Name Apache Tomcat v7 0 Tomcat installation directory C Users dlaradie Datos Apache apache tomcat 7 0 61 apache tomcat 7 0 47 Download and Install JRE Workbench default JRE Z Installed JREs Figure A 19 Running Big Market in Eclipse step 3 e Deploying the WAR file in an Apache Tomcat server by this way you can deploy the application in a Tomcat Server and run this server in order to make the application accessible from the Internet or simply to test it by yourself To achieve this objective follow the next steps 1 In the previous subsection Building the WAR we
61. lt h4 gt lt p gt Number of initial nodes lt input type text class form control id numNodes name numNodes disabled true gt lt p gt lt p gt lt p gt lt p gt Simulation name lt input type text class form control id nameRandom name nameRandom disabled true gt lt p gt lt div gt div class col lg 4 id col right gt lt h4 gt Load network from DataBase lt h4 gt lt p gt Dataset identifier lt input type text class form control id datasetIdentifier name dataset disabled true gt lt p gt lt p gt New name for simulation lt input type text class form control id newLoadName name newLoadName disabled true gt lt p gt lt p gt lt input type button class btn btn primary value Create network id createButton disabled true onclick createNetwork gt lt p gt lt div gt lt div gt lt div class setup button gt lt p align center gt lt button type submit class btn btn primary id submitButton gt Setup lt button gt lt p gt lt div gt lt form gt The form has an option to mark if the user wants to create a random network or load 97 CHAPTER 4 ARCHITECTURE AND IMPLEMENTATION a network the option of create a new network using Neo4J has not a call to the servlet because it just open a new window with the Neo4J interface Then depending of the option marked create random network or load network fields are enabled respect
62. ment to continue the work done for this master thesis Finally the appendix provides useful related information especially covering the instal lation and configuration of the tools used in this thesis 4 1 4 STATE OF ART 1 4 State of art In this section we are going to explain how the SNSA software works actually For this we have chosen two frameworks that give us a start point to begin our project 1 4 1 Ohal ohal sir analytics Figure 1 1 Ohal Logo 1 Interesting aspects it adapts to real models using networks and social media with its users and relations between them Each agent has individual characteristics The response of each scenario change according the characteristics of the agents and the messages It allows two types of transmission viral and social pressure It also allows if scenarios 2 Simulated market social networks in general any social media 3 Objectives studying the best way to propagate hearsays in Twitter and Facebook 4 How it works it creates the initial network with its characteristics topology prop agation mode agents and the seeding Then it starts the simulation and analyses how the hearsays are propagated When the simulation ends data is extracted and a anew simulation is started modifying the initial characteristics Data of the second simulation is compared with the first one 5 Agent type e Initial agent it generates the initial mes
63. n Development Remote Systems b amp Run Debug ES SVN B Tasks Team amp Web E Web services XML 2 m 2 Net Finish Cancel A N Figure A 2 Import project from Git step 3 84 A 1 INSTALLATION 4 Select Existing local repository Select a location of Git Repositories type filter text j Clone URI Finish Figure A 3 Import project from Git step 4 5 Click in Add in order to add new Git repository Figure A 4 Import project from Git step 5 85 APPENDIX A INSTALLING AND RUNNING A BIGMARKET SERVER 6 Select the path where you downloaded the BigMarket code in the previous section mark the repository and click in finish Add Git es Search and select Git repositories on your local file system Search for local Git repositories on the file system all Search criteria Directory C Users dlaradie Datos PFC Codigo Browse Look for nested repositories Search results type filter text 0 CAUsers dlaradie Datos PFC Codigo BigMarket git D Finish Cancel Figure 5 Import project from Git step 6 7 Once you have chosen the repository click in Next You can also clone a repository or add local repositories to the list KS _ _ Add 0 BigMarket C Users dlaradie Datos PFC Codigo BigMa
64. n specifv network using the Neo4J query commands Thanks to this the user is able to build a network that satisfies his her needs like a specific start situation the relationships between the users at the beginning of the simulation establish the broadcasters the acquaintances and the odd users etc Finally the tool allows the user to see the network that he she has been created on the browser or download it in a file in order to analyse it with and SNA tool Although BigMarket gives to the user some SNA data like closeness or betweeness the user maybe wants to achieve their own analysis 6 2 Achieved goals In this section we will analyse the goals established at the beginning of this master thesis and see if them has been achieved e Developing a free web framework to facilitate the access to SNSA social network simulation and analysis tools for any user independently of his her computer skills this goal has been achieved successfully We have developed a tool that allows the user to build a network from scratch and run a simulation without any programing knowledge simply following the steps in the GUI and all of it in his her browser Also he she can obtain the results of the SNA directly in the browser BigMarket allows analyse the following results betweeness closeness in degree and out degree e Saving time to the users when they want to make a SNSA implementing a framework that allows an easy configuration of the net
65. new number of users depending on the actual number of users and the time that the simulation is running we understand the time like the number of steps since the simulation began connect the new users with the oldies using the algorithm explained in the previous paragraph makes new relationships between the old users and finally modify the behaviour of the users according to the new network structure When the user stops the simulation Neo4J enter in action First of all the network is parser into JSON format in order to adapt it to Neo4J query format This query is made by an http request that store the simulation in the database Once the simulation is completely stored in the database BigMarket represent in the screen four centrality analysis betweeness closeness in degree and out degree If the user wants to analyse the network with more detail BigMarket enables the possibility to download the graph that contains the network in a GEXF file in order to analyse it with a SNA tool like Gephi In the results screen the user have the possibility to see the network too This visualization will be showed in a new window and the nodes and their relationships will be represented 46 4 3 MASON ENGINE AND ITS IMPLEMENTATION 4 3 MASON engine and its implementation As we explained in the past section BigMarket uses the MASON step engine in order to make the network evolve in time MASON also allows a GUI if we execute BigMarket
66. nt including plain XML SVG and XUL Along with HTML and JavaScript CSS is a cornerstone technology used by most websites to create visually engaging webpages user interfaces for web applications and user interfaces for many mobile applications 2 5 5 SigmaJS sigmajs Figure 2 12 SigmaJS Logo Sigma is a JavaScript library dedicated to graph drawing It makes easy to publish networks on Web pages and allows developers to integrate network exploration in rich Web applications Thttp www w3schools com html default asp 5http www w3schools com css default asp http sigmajs org 18 Requirement Analysis This chapter describes ome of the most important stages in software development the requirement analysis using different scenarios For this a detailed analysis of the possible use cases is performed using the Unified Modelling Language UML This language allows us to specify build and document a system using graphic language 19 CHAPTER 3 REQUIREMENT ANALYSIS 20 3 1 OVERVIEW 3 1 Overview The result of this chapter will be a complete specification of the requirements which will be matched by each module in the design stage This also helps us to focus on key aspects and take apart other less important functionalities that could be implemented in future works 3 2 Use cases These sections identify the use cases of the system This helps us to obtain a complete sp
67. om network with a initial number of nodes of 100 The BigMarket engine will be the responsible of establishing the relationships between each node e Creating network with Neo4J we will create a new network using the Neo4J frame work In the past section we will show a few queries that allows us to create nodes and relationships between them Now we will use them to create a full network e Loading network from database the last network selected will be a network stored in the database we will use the network created in the previous point 5 2 Random network Once we are in the setup screen we will select the option of create a new random network We will introduce an initial number of nodes of 100 and we will call the network Random network So when we have completed these steps we will have a Setup simulation screen like the figure 5 1 67 CHAPTER 5 PROTOTYPE AND EXAMPLE USAGE amp lit E A gs Edi mie gs 2 Grupo de Sistemas Inteligentes UPM About BigMarket Set up Simulation Running Actions Contact Select an option random network DLoad network from DataBase New random network i Load network from DataBase Number of initial nodes Dataset identifier 100 Simulation name New name for simulation Random network Figure 5 1 100 Node Random network setup Finally we click on Setup button and the network will be created This action does not save the network in
68. onse i The user selects load Create simulation button network becomes enabled The system opens a new The user clicks on Create 2 window with the Neo4J network interface The user creates the network Neo4j store the new network using Neo4j commands in database Table 3 4 Creating network with Neo4j use case 27 CHAPTER 3 REQUIREMENT ANALYSIS 3 2 2 4 Loading network from database This use case also extends from Setup network use case In this case a network will be loaded from the database Like the previous use case at the begin we have the servlet started and the Setup simulation screen displayed At the end of the use case we have a network loaded from the database The table also represents the flow of events of the use case Use Case Name Loading network from database Use Case ID UC1 4 Pre Condition The Setup simulation screen has been displayed Post Condition The network which will be used in the simulation has been loaded successfully Flow of Events Actor Input System Response i The user selects Load The fields for loading network from database network become enabled 5 The user fills the load The Setup button network fields becomes enabled The user clicks on the The network is loaded from Setup button database Table 3 5 Loading network from database use case 28 3 2 USE CASES
69. ph saves the GEXF file in the user s computer as we explained in section 4 6 4 8 Conclusion In this chapter we have seen how each part of BigMarket works and how all interact between them to build a simply SNSA framework In the next chapter we select an use case and we will use it to explain in detail how the user interface works 63 CHAPTER 4 ARCHITECTURE AND IMPLEMENTATION 64 Prototype and example usage In this chapter we are going to describe a selected use case by explaining the running of all the tools involved and its purpose and responses Thank to these use cases we will show the overall performance of the application and all the main functions available to the user 65 CHAPTER 5 PROTOTYPE AND EXAMPLE USAGE 66 5 1 INTRODUCTION 5 1 Introduction In this chapter we will explain in detail a use case For explain with more detail the use of BigMarket we will create three networks at the begin in order to explain the three ways to create and use a network in the simulation Then we will choose one of the three networks created and we will use it for run a simulation After we will see the results of the centrality analysis and the network using the SigmaJS visualizator Finally we will download the graph in a GEXF file and we will analyse it with the Gephi framework These three networks will be the next e Random network with this option we will create a new rand
70. println relationsRetrieve 53 CHAPTER 4 ARCHITECTURE AND IMPLEMENTATION E 4 5 SigmaJS graph visualizator and its implementation In this section we will explain how the user can see the structure of the network at the end of the simulation simply with one click This representation helps the user to see a final photography of the network SigmaJS allows us to represent a graph in an HTML file using JavaScript and JSON For a correct visualization of the network first of all we have to parse the network to JSON Once the JSON is built and save in a file we can attach it to the HTML that includes the JavaScript code for SigmaJS To understand it better we provide here an example of a little network in JSON format and the lines of JavaScript necessary in our HTML file First we can see the implementation of the function than extract the information of the network in a JSON file Then we show the code necessary to visualise the network in an HTML file Listing 4 10 Network in JSON nodes dig Hoe lapel WO Wise Ms 1 2 size 6 Pek jg Er Busen i OP 0 2 size 6 Ist ig nM Label Ba 1 ge 0 54 4 6 GEPHI AND EXPORT TO FILE IMPLEMENTATION size 6 E edges ig iy T target nN ju 2 Qn 2 gt UsourceM Ha target my Listing 4 11 Jav
71. put System Response new window appear with a The user clicks on See 1 representation of the network button network Table 3 10 Network visualization use case 33 CHAPTER 3 REQUIREMENT ANALYSIS 3 2 3 Developer use cases This use case package collects the developer functionalities of BigMarket as shown in 3 2 The use cases presented in this section are as shown in the Figure 3 2 e New user behaviour detailed in sub section 3 2 3 1 e Neu network bwilding algorithm detailed in sub section 3 2 3 2 BigMarket New user behaviour Admin New building network algorithm Figure 3 2 Developer use cases 34 3 2 USE CASES 3 2 3 1 New user behaviour This use case represents the possibility that a developer user codes his her own user be haviour First of all the user should have download the code in his her computer Finally he she can develop the behaviour Use Case Name New user behaviour Use Case ID UC2 1 Pre Condition The developer has downloaded the code Post Condition N A Flow of Events Actor Input System Response The developer can code The developer imports the his her own class in which project in Eclipse the developer defines the new behaviour Table 3 11 New user behaviour use case 35 CHAPTER 3 REQUIREMENT ANALYSIS 3 2 3 2 New network building algorithm This use case represent th
72. re a 71 5 5 Load network setup a E ee 72 5 6 Running the simulation 73 5 7 Results of the simulation nn 74 5 8 Network visualization 2 2 2 22 cn nn ee 74 1 Git Bash console capture 83 A 2 Import project from Git 84 Import project from Git 4 85 A 4 Import project from Git 5 85 A 5 Import project from Git step 6 nn 86 A 6 Import project from Git step 7 0 86 A 7 Import project from Git step 8 oaoa a e 87 A 8 Import project from Git step 9 22e 87 A 9 Convert the project into Eclipse project 1 88 A 10 Import libraries in an Eclipse project step 1 0 89 A 11 Import libraries in an Eclipse project step 2 89 A 12 Import libraries in an Eclipse project step 90 A 13 Import libraries in an Eclipse project step 4 less 90 A 14 Building the WAR step 1 92 A 15 Building the WAR 2 92 A 16 Building the WAR 3 93 A 17 Running Big Market in Eclipse step 1 2 2 2 o 93 A 18 Running Big Market in Eclipse step 2 2 2 2 En nn nn 94 A 19 Running Big Market in Eclipse 8 94 B 1 New random n
73. relationships Inttp cs gmu edu eclab projects mason 11 CHAPTER 2 ENABLING TECHNOLOGIES we will choose other tool than fill in better way to our needs In coming chapters we will explain with more detail how MASON is used to build the simulation 2 2 Gephi Co Gephi Figure 2 2 Gephi Logo Gephi is an interactive visualization and exploration platform for all kinds of networks and complex systems dynamic and hierarchical graphs Gephi has been used in number of research projects in the university journalism and elsewhere for instance in visualizing the global connectivity of New York Times content and examining Twitter network traffic during social unrest along with more traditional network analysis topics The Gephi Consortium is a French non profit corporation which supports development of future releases of Gephi Members include SciencesPo Linkfluence WebAtlas and Quid Gephi inspired the LinkedIn InMaps and was used for the network visualizations for Truthy We will use Gephi in this project in two ways e BigMarket allows the user to download an gexf file Once he she has download this file it can analyse it by using Gephi Framework so the user must have Gephi installed in its own computer e The other way that we use Gephi in our project is using its Java API for analyse the graph in the web framework In this way the user do not need to have installed Gephi in his computer Bi
74. request HttpServletResponse response throws ServletException IOException sim getGui getConsole pressStop sim finish neoDB setSim sim neoDB launchDatabaseTool GraphJSONParser g new GraphJSONParser sim getGraphManager getGraph String path getServletContext getRealPath networkGraph json g launchParser path exportGraphGEXF calculateCloseness request response calculateBetweenness request response request getRequestDispatcher Constants ACTIONS forward request response Like we can see in the code first of all the simulation is finished then the Neo4J tool is 62 4 8 CONCLUSION initialized and launched it stores the network in the database Then the JSON file for the network visualization is generated Next step is saving the graph in a GEXF file and finally the centrality is calculated and the user interface goes to the next screen Actions page In this last screen of the simulation called Actions we can see four tables with the information about the centrality and two buttons that allows us to see the network and y download the graph in a GEXF file The way to represent the centrality in the tables is similar to the way showed for the simulation information so we have no copy here the code The See network visualization simply opens a new window with the visualization showed at 4 5 and the Download gra
75. rket git E Finish Cancel Figure A 6 Import project from Git step 7 A 1 INSTALLATION 8 Now select the option Import as general project and click in Next Select a wizard to use for importing projects Depending on the wizard you may select a directory to determine the wizard s scope Wizard for project import Import existing projects Use the New Project wizard 9 Import as general project gt Working Directory C Users dlaradie Datos PFC Codigo BigMarket b E git gt src gt E WebContent gitignore B gitignore LICENSE tt README HELP tet 5 README md Figure A 7 Import project from Git step 8 9 Finally choose a name for your project in our case BigMarket and click in Finish Import projects from a Git repository Project name BigMarket Directory C Users diaradie Datos PFC Codigo BigMarket Figure A 8 Import project from Git step 9 87 APPENDIX A INSTALLING AND RUNNING A BIGMARKET SERVER Now you have the project imported in Eclipse But it is not an Eclipse project yet so in the next lines we will explain you how to convert the project imported into an Eclipse project A 1 4 Converting the project into an Eclipse project In the last section we saw how to import the project downloaded from Git in Eclipse Now is time to convert this project into an Eclipse project in ord
76. sage e Propagator agent it has a lot of contacts and a great capacity for influence its contacts it propagates the message generated by the initial agent e User agent final user that receives the message it can propagate the message too but only to its inner contacts circle 6 Agent properties it establishes different types of relationships CHAPTER 1 INTRODUCTION e Two way both agents can influence each other e One way one agent influences another but this second agent cannot influence the first one 7 Implementation available contact with them and then they realize the study but the software is not available 8 Comments its mechanics are interesting but they have not software available 1 4 2 ThinkVine Figure 1 2 ThinkVine Logo e Interesting aspects it generates future ideas based on marketing strategies It allows watching the impact of different marketing strategies on different users groups This framework also has the capacity of manage a lot of if scenarios Finally it allows self learning e Simulated market marketing in a heterogeneous society e Objectives it allows watching how the social media influence the buying habits in heterogeneous societies and foresee the impact in these habits according the society evolution e How it works firstly it creates a mathematical model of consumers agents based on a demographic census then it adapts the behav
77. tambi n se le ofrece la posibilidad de disenar gr ficamente una red propia o bien cargar una red creada anteriormente La red generada se puede guardar para posteriores simulaciones A los agentes los usuarios de la red se les asigna un comportamiento dependiendo del n mero de conexiones con el resto de agentes C mo ltima funcionalidad de la herramienta se le permite al usuario realizar distintos tipos de an lisis de red as como obtener la topolog a de la red analizada en lenguaje GEXF Graph Exchange XML Format para exportar la informaci n a otras herramientas de an lisis de redes sociales Un ltimo anadido al proyecto es la posibilidad de descargarse el c digo y ejecutarlo en una m quina propia de modo que pueda adaptar o extender el comportamiento de los agentes de la red a sus necesidades En la memoria se muestran la elaboraci n de la herramienta parte por parte y la pos terior conexi n entre todas ellas para conformar la herramienta final Tambi n se incluye un an lisis de las distintas herramientas utilizadas indicando el porqu de su elecci n o su desestimaci n as como un estudio del estado del arte Finalmente se incluyen unos anexos con el fin de ampliar algunos aspectos y as facilitar la comprensi n de estos y algunos man uales para proveer la informaci n necesaria para el mantenimiento y desarrollo de posibles mejoras Palabras clave simulaci n social red social usuario grafo comportamiento relaci
78. the database just create the network to use it in the simulation 5 3 Creating network with Neo4J For creating a new network using the queries from Neo4J first we have to go to the Setup simulation screen and select the option Load network from database Once we have selected this option the button for creating a new network or modify an existing network will be enabled If we click this button a new window displaying the Neo4J web interface will be opened In this new window represented in the figure 5 2 we will use the Neo4J queries to create the nodes and establish relationships between them network with three nodes and two relationships will be enough 68 5 3 CREATING NETWORK WITH NEO4J CREATE n CreateNetwork name Nodel Figure 5 2 Neo4J interface First of all we will create the three nodes To do this we write these queries in the Neo4J interface Listing 5 1 Creating node 1 CREATE n CreateNetwork name Node1 Listing 5 2 Creating node 2 CREATE n CreateNetwork name Node2 Listing 5 3 Creating node 3 CREATE n CreateNetwork name Node3 The label CreateNetwork will be used to know to what simulation belongs the node and the name Nodel Node2 and Node3 will identify the node inside the simulation Once we have created the three nodes we can see them using the next query Listing 5 4 Nodes cr
79. tional methods has a large cost in time and money At this point the SNA software has a great importance because it reduces the cost of the analysis to a large degree Nowadays there are variety of frameworks that allow making SNSA The disadvantage is that these frameworks are closed sourced or programmer is needed to program and configure the simulation With this project the possibility of accessing a social network analysis framework through the web is offered with a simple front end and open source so the disadvantages mentioned in the previous paragraph are solved Apart from the framework the user can download the code too in order to code new behaviours that improve the tool 1 2 Master thesis goals The principal objectives of the project are the followings e Developing a free web framework to facilitate the access to SNSA tools for any user independently of his computer skills CHAPTER 1 INTRODUCTION e Saving time to the users when they want to make a SNSA implementing a framework that allows an easy configuration of the network and of the simulation e Integrating Big Data technologies into the framework More specifically with a noSQL graph database e Facilitating familiarization of new developers with the SNSA tools because it offers a base for supply new developments 1 3 Structure of this Master Thesis In this section we will provide a brief overview of all the chapters of this Master Thesis
80. we will explain how works the user web interface and the BigMarket servlet We explain this at the end of the chapter because these parts are the glue that tie the parts explained until now In this section we explain the code leaving the explanation of the GUI for the chapter 5 First of all we have the index page in which we find a bit introduction of what is BigMarket Once we click on Start we enter to the Setup simulation screen In this screen we can choose how we are going to build our initial network This page contains a form that allows the servlet knows what options the user has chosen create random network load network or create network with Neo4J The implementation of the form is the following Listing 4 13 Save graph in GEXF file form action BigMarketServlet method POST name setup form gt lt input type hidden name formName value setupForm gt lt div class jumbotron gt 56 4 7 USER WEB INTERFACE AND SERVLET AND THEIR IMPLEMENTATION lt h4 gt Select an option lt h4 gt lt p gt lt input type radio name networks id random value random onclick randomNetworkSelected gt New random network lt p gt lt p gt lt input type radio name networks id load value load onclick loadNetworkSelected gt Load network from DataBase lt p gt lt div gt lt div class row gt lt div class col 1g 4 id col left gt lt h4 gt New random network
81. work and of the simulation this goal has been achieved successfully BigMarket allows the user to build a network and configure a simulation with few steps so it saves the user time e Integrating Big Data technologies into the framework More specifically with a noSQL 79 CHAPTER 6 CONCLUSIONS AND FUTURE LINES graph database this goal has been achieved successfully We have integrating into the framework Neo4J database it allows the user to store his her simulation and the network in a database Facilitating familiarization of new developers with the SNSA tools because it offers a base for supply new developments this goal has been achieved successfully BigMarket has been programmed following the rules of modularity The behaviours of the users are independent it means that you can program your own behaviour and assign it to an user of the network You can also program how the network evolve in time In view of these results we can say that all the goals marked at the beginning of this master thesis has been achieved succesfully 6 3 Future work There are several lines than can be followed to continue and extend features of this work In the following points some fields of study or improvement are presented to continue the development 80 In order to make the framework more accessible it will be necessary to make it responsive so the user can use BigMarket from a tablet or smartphone in order to make the appl
82. xisting network using few queries note that if you want to do more complex networks maybe you should study with more detail the possibilities of Neo4J So the basics queries that allows us to interact with the graphs that store our networks are the following Listing 4 5 Query get data MATCH n REIURN n LIMIT 100 Listing 4 6 Create node CREATE n name World RETURN hello n name Listing 4 7 Query relationship MATCH martin name Martin Sheen gt movie e Save network in database to do this we will use the Neo4J REST with this we can send a request to Neo4J with the information of the network in a JSON object The code that allow us to do this is Listing 4 8 Save network private void createNodes 1 50 4 44 NEO4J DATABASE AND ITS IMPLEMENTATION String location null nodeUris clear System out println SIMULATION NODES sim getGraphManager getGraph getNodeCount for org graphstream graph Node n sim getGraphManager getGraph getNodeSet 1 System out println NODE n getId try String nodePointUrl this SERVER ROOT URI db data node String dataset dataset HttpClient client new HttpClient PostMethod mPost new PostMethod nodePointUrl Header mtHeader new Header mtHeader setName content type mtHeader setValue application json mtHeader setName

Design and implementation of a web framework for the analysis and

Contents

Download Pdf Manuals

Related Search

Related Contents