Scientific Visualization and Parallel Workflows
Contents
1. Spatial Bounds europe HB Image Un V 63524566 o 63540522 124 103 Unavailable 0 832e403 ExtractSubset 7054740 7060058 Unavailable 0 8 32e 03 ExtractSurface1 wy Polygonal 7054740 o 7060058 454 837 Unavailable 08326403 4 Include Boundary Warp scalar 1 Np Polygonal ag 7054740 7060058 372 102 565 048 0 8 32e 03 CSCS 2013 25 CSCS 2013 Swiss National Supercomputing Centre Data Streaming in VTK Data larger than memory can be easily treated Piece by piece Releasing or re using memory after each subset e Optionally accumulating sub object representations for the final image The upstream filters should be prepared to handle piece requests of any size Each filter can translate the piece request lt x Centro Svizzero di Calcolo Scientifico o Swiss National Supercomputing Centre Reminder VTK pipeline Data Filters Data Mappers First pass Get general bounds information without reading the data Second pass Decide how much to sub divide The Extent Translator and process pieces CSCS 2013 27 The pipeline enables a two way exchange of data amp information he renderer drives the request for data updates The Extent Translator The Extent Translator does a binary subdivision of the data and let the user access pieces one at a time CSCS 2013 CSC s SGD co siz Cao Senti en is g2. A 64x64x64x4 block of floats 10485 6 10485 6 10485 6 10485 6 10485 6 10485 6 2013 11 25 2013 11 25 2013 11 25 2013 11 25 2013 11 25 2013 11 25 14 4 benchmark benchmark benchmark benchmark benchmark benchmark Each file can also be gzipped 221 22094 22398 21958 21838 22220 422 2013 11 25 2013 11 25 2013 11 25 2013 11 25 2013 11 25 2013 11 25 2013 11 25 Qaod Qaod 20002 20003 00004 Qao 0004 0004 0004 0004 0004 0004 bot bot bot bot bot bot benchmark benchmark benchmark benchmark benchmark benchmark benchmark Qaod Qaod 00002 WOOOS 00004 WOOOS 0004 0004 0004 0004 0004 0004 0004 bot bof bof bof bof bof gz gz gt gz gz 02 47 st CSCS SGD soe siz cae cen 2 Swi Sid seria tre Data Parallelism by example BOV format VisIt can put the pieces together serially or in parallel with the following meta file benchmark 0004 bov BOV version 1 0 I O benchmark program DATA FILE benchmark 05d 04 bof gz DATA SIZE 192 128 64 DATA BRICKLETS 64 64 64 DATA FORMAT FLOAT VARIABLE node data BRICK ORIGIN 0 0 0 0 0 0 BRICK SIZE 3 0 2 0 1 0 48 vo x Centro Svizzero di Calcolo Scientifico ig gt Swiss National Supercomputing Centre Volume rendering uses a hybrid approach Object space partitioning Image space pal Vi
3. AMR extract part Integrate flow AMR surface Surface vectors D Glyph CSCS 2013 lt x Centro Svizzero di Calcolo Scientifico o Swiss National Supercomputing Centre The VTK visualization pipeline 2 Data Filters Data Mappers ParaView s Rendering drives the execution view StillRender VisIt defines its own meta data package called Contracts CSCS 2013 10 e VTK extends the data flow paradigm VTK acts as an event flow environment where data flow downstream and events or information flow upstream Svizzero d Calcolo Scientifico Nation li inse omputing Centre Ge om CS inde Swi Example of a VisIt Contract Spatial extents are examined and the visualization pipeline is by passed for those outside the range O CSCS 2013 Example of a VisIt Contract Selection of partitions and assignments to processors Data extents min amp max are examined and the data filtering visualization pipeline is by passed for those outside the range CSCS 2013 12 Swiss National Supercomputing Centre The VTK visualization pipeline 3 Large data when dividable ien can be treated by pieces The Source will distribute data pieces to multiple execution engines Parallel pipelines will be instantiated to treat all pieces and create the graphics output This is hidden from the user O CSCS 2013
4. lt us di Calcolo Scientifico Gei rcomputing Centre Visualization without a client ParaView and VisIt can run a server only job Ideal for batch processing A python script is necessary While creating a new visualization with the client one can save the corresponding python commands to construct the pipelines Python programs do not include any explicit parallel programming that s easy CSCS 2013 64 CSCS x Centro Svizzero di Calcolo Scientifico awd Swiss National Supercomputing Centre In situ Visualization e Clients runs locally and Server runs remotely in parallel display results computed on handling data processing for client the server Local VisIt Clients Parallel Cluster Files Data processed in data flow networks Filters in data flow networks can be implemented as plug ins Data Flow Network CSCS 2013 65 CSCS GD us di Calcolo Scientifico Coupling of Simulations and VisIt Libsim is a VisIt library that simulations use to enable couplings between simulations and VisIt Not a special package It Is part of VisIt Libsim Source Front g End E a Filter Data Ty c di Access Code M Filter CSCS 2013 CSC s SGD cato siz ae Sete en is ve ge ul entre Data model for in situ visualization me Materials Species Mesh Types e Structured meshes Point meshes CSG meshes AMR
5. parallel visualization server thru a firewall ParaView and VisIt use ssh tunnels to establish connections Local remote Clients Parallel server at CSCS O CSCS 2013 Centro Svizzero di Calcolo Scientifico Swiss National Supercomputing Centre Supercomputer or graphics cluster CSCS 2013 CSC 5 SGD cae six S get eiert entre VisIt Launcher ZSBATCH nodes 12 Creating HW based display _ Creating HW based display SBATCH ntasks 96 02 Creating Mesa SW based display SBATCH gres gpu 2 03 Creating Mesa SW based display ZSBATCH exclusive 04 Creating Mesa SW based display 05 Creating Mesa SW based display _ _ 06 Mesa SW display mpirun np 96 engine par sshtunneling Creating HW based display hw accel display 0 95 Creating HW based display E 10 Creating Mesa SW based display host castorO _ 11 Creating Mesa SW based display port 15129 key 709adcfbf 12 Creating Mesa SW based display 13 Creating Mesa SW based display CSCS 2013 60 Se CSCS Centro Svizzero di Calcolo Scientifico Swiss National Supercomputing Centre ParaView Launcher SBATCH nodes z 12 SBATCH ntasks 96 SBATCH gres gpu 2 SBATCH distribution c SBATCH exclusive mpiexec binding rr ppn 0 0 pvserver rc ch 148 n 48 env DISPLAY 0 1 ch 148 187 19 45 sp 1 CSCS 2013 B Memory Inspec
6. 13 lt Centro Svizzero di Calcolo Scientifico Swiss National Supercomputing Centre First Rendering option The client GUI collects all objects to be liil rendered peli Data Filters Each pipeline creates rendering primitives from ____ its partial data p e The client does a heavy rendering CSCS 2013 14 lt Centro Svizzero di Calcolo Scientifico Swiss National Supercomputing Ce Second Rendering option Sort last rendering Each pipeline does a full frame rendering of its partial data An image compositor merges all somma images by comparing Z depth of a all pixels Final Image Final Image 5 AC CSCS 2013 15 Node O sends its frame buffer to the client Node 0 collects composits all frames buffers CSCS 2013 4 CSCS gt Calcolo Sc Centro Svizzero di Calc ientifico ry Swiss National Supercomputing Centre Sort last rendering pixel compositing e N rendering tasks Depth of the tree is log N CSCS 2013 17 CSCS GD di Calcolo Scientifico Arbitrary or adaptive 3 D data partitioning Is the final image order independent A sort last compositing enables complete freedom in data partitioning Each pixel carries its color amp depth CSCS 2013 18 x Centro Svizzero di Calcolo Scientifico De Swiss National Supercomputing Centre Third Render
7. CSCS ETH Centro Svizzero di Calcolo Scientifico Eidgen ssische Technische Hochschule Z rich Swiss National Supercomputing Centre Swiss Federal Institute of Technology Zurich Open source Scientific Visualization VisIt and ParaView Jean M Favre Visualization Task Leader November 2013 O CSCS 2013 CS cs lt cano EE Swi us et Gre gerett Agenda Visualization pipelines Parallel pipelines Rendering modes Data formats parallel I O and parallel visualization Remote client server parallel viz Demonstrations with ParaView e Demonstrations with Visit O CSCS 2013 CSCS 939 us Calcolo Scientifico gt Swiss Nati irene ng Centre Visualization Pipelines Introduction From a survey article by Ken Moreland IEEE Transactions on Visualizations and Computer Graphics vol 19 no 3 March 2013 Read Isosurface Render A visualization pipeline embodies a dataflow network in which computation is described as a collection of executable modules that are connected in a directed graph representing how data moves between modules There are three types of modules sources filters and sinks CSCS 2013 3 CSC gt D contro sy d ntifico Swi ia Supercomputing Centre ve Visualization Pipelines Definitions e Modules are functional units with or more inputs ports and 0 or more output ports Read Connections are dire
8. a shared resource or a distributed set of machines An advanced topic see demo CSCS 2013 53 CS cs UM cano Swi us spa gerett Scientific Visualization VisIt and ParaView support two modes of execution 1 Interactive imaging analysis query Requires a GUI to test and try multiple visual representations 2 A batch oriented movie making process Requires a script python to enable reproducible visualizations and the support of time series or multiple experiments CSCS 2 Swiss National Supercomputing Centre Visualization Client Server ParaView and Visit use the client server concept client optional runs the GUI server embedded and local by default or remote and or parallel does the real work O Data analysis Image generation CSCS 2013 55 Swiss National Supercomputing Centre Parallel Visualization Parallelism Is a must for big data Parallelism is the source of many problems CSCS 2013 CS cs SGD cato 2 Swi us spa gerett Parallel Visualization Should we bother Yes Interactive visualization is necessary to gain insight from exploration Yes Parameter tuning should be fast CSCS 2013 57 e CSCS Centro Svizzero di Calcolo Scientifico NC gt Swiss National Supercomputing Centre Client and Remote Servers Direct connections chent app can request a direct connection to a
9. can use MPI IO or HDF5 or NetCDF or ADIOS CSCS 2013 42 CSCS SGD so sz sca int Swiss Nation gerett SILO Data Format e https wci llnl gov codes silo A very versatile data format The Getting Data Into VisIt manua covers how to create files of this type In addition there are many code examples here e http ortal nersc gov svn visit trunk src tools DataManualExamples CreatingComp atible CSCS 2013 43 CS cs SG cano EE Swi us et Gre gerett SILO Data Format From the User Manual Silo is a serial library Nevertheless it as well as the tools that use it like VisIt has several features that enable its effective use in parallel with excellent scaling behavior O CSCS 2013 44 GP co CS Cen d Calcolo Scientifico Swi get af omputing Centre Pixie HDF5 Data Format radial toroidal Modes poloidal kd tree sub divisions CSCS 2013 45 a CSCS Centro Svizzero di Calcolo Scientifico Swiss National Supercomputing Centre Data Parallelism by example BOV format n Read a single block in a single file but split the block in pieces Cube dimension 640x640x640 Bricklets 80x80x80 Divide brick true Modes stride 8 random block CSCS SGD soe si hien olo Scientifico Supercomputing Centre Data Parallelism by example BOV format Alternatively each process writes its own file independently
10. cientifico Swiss National Supercomputing Centre de MPI tasks ghost cells hyperslabs Example a 4 processor run Mesh var mesh Mesh var mesh n Height cB Height cm Height cm A A 0 2 0 10 0 20 0 30 0 40 0 50 0 60 0 70 0 80 0 90 e Width cm Width cm Width cm user fame user user Ittre Thu Aug 2 13 49 35 2012 Mu Aug 2 14 12103 2012 Thu Aug 2 14 15 45 2012 CSCS 2013 38 CS cs SGD cato Swi get soie nios MPI tasks ghost cells hyperslabs The goal of parallel I O Present a uniform grid storage display The Visualization software ParaView or VisIt will do its own subdivision and re construct ghost zones when necessary CSCS 2013 39 CS cs lt coos ge Swi an si MPI tasks ghost cells hyperslabs Pseuctoc olor Ver procki 23 00 The Visualization software should know how to re construct ghost zones when necessary O CSCS 2013 CS cs SGD cato 2 Swi us spa gerett MPI tasks ghost cells hyperslabs Def a hyperslab is a subset tn n D of a larger grid Parallel I O is a composition superposition of multiple hyperslabs Each processor must know where each piece fits in the global mesh CSCS 2013 41 CS cs Ge gt Swi get oie nios Data formats Parallelism Once you know the IJK extents of all your hyperslabs you
11. city 4 Air at 25 C Volume Fraction W Pressure 4 Water at 25 C Superficial Velocity M Water at 25 C Velocity Water at 25 C Volume Fraction Alr at 25 C Water at 25 C Surface Tension Coefficient 2 Air at 25 C Conservative Volume Fraction 3 Air at 25 C Eddy Viscosity j Air at 25 C Shear Strain Rate I Air at 25 C Solver Yplus 1 Air at 25 C Vplus 2 Rotation Velocity J Total Pressure 2 Total Pressure in Stn Frame j Water at 25 C Conservative Volume Fraction Water at 25 C Eddy Viscosity 1 Water at 25 C Shear Strain Rate _s Water at 25 C Solver Vplus I Water at 25 C Velocity in Sin Frame 4 Water at 25 C plus 24 Air at 25 C Water at 25 C Curvature Pelton turbine simulation courtesy of 2 Air at 25 C Buoyancy Force Bforce Dh EPFL LMH and VATech Hydro ParaView Figure 2 Displacement pattern of Ti atoms in tetragonal BaTiO a Average pic shiftings along the direction of tetragonal distortion b Local instantaneous Ti sh directions close to the body diagonal Ti displacements are represented as bold blu SGD co CS V Cen fo di Calcolo Scientifico Swi a sais omputing Centre Scientific Visualization Two modes e Post mortem This does not mean you can start thinking about it The Visualization after the simulations are done You should plan it before running your code Live a k a In situ visualization Simulation and visualization codes run at the same time on
12. ctional attachments EE between input and output ports Render Execution management is inherent in the pipeline Event driven Demand driven O CSCS 2013 CSC s SGD usus Cao Senti en is gen ul entre ve Source al Whole es Region 1 Visualization Pipelines Filter 1 a Whole 2 Region 2 Filter 2 7 Whole 2 Region 3 Sink a Update Information Source J Update Region 3 g Metadata Filter 1 ha Update Region 2 g Filter2 ha Update Region 1 g Sink b Update Region Source Dat Set 1 Filter 1 dba Set 2 Filter 2 Er Set 3 Sink c Update Data 1st pass Sources describe the region they can generate pass The application decides which region the sink should process 3 d pass The actual data flow thru the pipeline CSCS Centro Svizzero di Calcolo Scientifico ig gt Swiss National Supercomputing Centre Visualization Pipelines Data Parallelism Process 1 Process 2 Process 3 Process 4 Reader Reader Reader Reader Filter ft Filter lt gt Filter gt Filter 1 Filter 2 Filter 2 Filter 2 Filter 2 Renderer gt Renderer 3 Renderer d Renderer Data parallelism partitions the input data into a set number of pieces and replicates the pipeline for each piece Some fi
13. e ul entre ve Streaming the data _ mapper vtkPolyDataMapper mapper SetNumberOfPieces 64 PolyDataMapper mapper SetPiece 0 CSCS 2013 29 CSCS Se Swiss National Supercomputing Centre The Vis pipeline is under the hood 112 Data Parallelism e data are divided automatically da based on the number of servers Data Filters available Data Mappers Transient Data time dependent data requests are also managed similarly via Rendering the two way pipeline data exchange CSCS 2013 30 lt Centro Svizzero di Calcolo Scientifico Swiss National Supercomputing Centre Summary VisIt and ParaView hide all the pipeline management Meta data are paramount to let the pipeline schedule the most efficient processing The real questions are Can you provide data that can be distributed Is the distribution piece invariant CSCS 2013 31 CSCS ETH Centro Svizzero di Calcolo Scientifico Eidgen ssische Technische Hochschule Z rich Swiss National Supercomputing Centre Swiss Federal Institute of Technology Zurich do Data formats parallel I O and visualization CSCS 2013 32 CS cs GD Swi us id Sae socie aisi Prelude Data formats Interface between simulations and visualization Many formats exist Pick the most appropriate High level libraries HDF5 netCDF Make up your own Pa
14. g or multi pass e The snow removal was done in about 5 passes Data Streaming Divide and conquer Load datasets of any size by splitting the volumes in pieces Process the split data O CSCS 2013 lt x Centro Svizzero di Calcolo Scientifico o Swiss National Supercomputing Centre Example Digital Elevation Model The VTK file header gt vtk DataFile Version 3 0 European DEM File BINARY DATASET STRUCTURED POINTS DIMENSIONS 8319 7638 1 ORIGINOOO SPACING111 POINT DATA 63540522 O CSCS 2013 23 SGD ce get Centre CS V Cen ro di Calcolo Scientifico Swi Use sub sampling when data are too big Warning 64 millions points are first read in memory then sub sampled The memory footprint can still be huge PolyDataMapper CSCS 2013 24 CSCS Centro Svizzero di Calcolo Scientifico Swiss National Supercomputing Centre te Memory usage blows up down the pipeline Kitware ParaView 3 4 0 nx File Edit View Sources Filters Animation Tools Help 29997 Kar bMS met FH Saba Geo HFICLIEVOTL19 Pipeline Browser ax CH builtin europe vtk ExtractSubset Eh ExtractSurface a Warp scalar 1 Object Inspector ex Properties _ Display Information SS statistics View ff N Name Data Type No of Cells No of Points Memory MB ieometry Size
15. ing option Tiled Display Each renderer does a partial frame rendering of the full data CSCS Centro Svizzero di Calcolo Scientifico Swiss National Supercomputing Centre do When there is too much data Adaptive resolution processing should be used DB mvframe037 1 3d hdf5 e DB mvframe037 1 3d hdf5 DB mvframe037 1 3d hdf5 Cycle 371 Time 0 236984 Cycle 371 Time 0 236984 Cycle 371 Time 0 236984 Ver dey e SCH Var density Ver density TRAI 0 004154 gar 0 004154 0 0001925 SE E certe 0 0001925 j 0 0001925 8 9232 06 ue D D EE 8 923 4 1362 07 4 136 07 Mew alles e if zi 1 91 76 08 1 9175 08 Min 9175 087 fe s 0 004154 Mac 0 004154 adus i Min 1 9172 08 Mn 1 9172 08 4 Spa 3 CSCS Centro Svizzero di Calcolo Scientifico Swi ional ss National Supercomputing Centre When large data require distributed processing Sub sampling can help prototype a visualization As long as the data format reader supports it see the Xdmf reader in ParaView Plece wise processing on a single node Data streaming when the whole visualization will not fit in memory Distributed processing on multiple nodes Parallel file I O Parallel processing Parallel rendering CSCS 2013 21 CS cs SGD cato ateneo ntifico Swi an omputing Centre Sub sampling streamin
16. lters will have to exchange information e g streamlines A special rendering phase will be needed CS cs SG cano Swi us spa gerett VisIt and ParaView are based on VTK The Visualization ToolKit VTK is an open source freely available software system for 3D computer graphics image processing and visualization VisIt and ParaView are end user applications based on VTK with support for VisIt ParaView Parallel Data Archiving VTK Parallel Reading Parallel Processing Parallel Rendering MPI Single node client server MPI cluster rendering CSCS 2013 CSCS Centro Svizzero di Calcolo Scientifico a gt Swiss National Superc ting Cent The VTK visualization pipeline 1 Data Filters Data Mappers ExtractHistogram IExtractEdges Gg RTAnalyticSourcel Renderin g Filter SetInputConnection Source GetOutputPort Mapper SetInputConnection Filter GetOutputPort CSCS VTK s main execution paradigm is the data flow i e the concept of a downstream flow of data Pipeline Browser e x I builtin 8 multicomb D ts contour _ CS cs SGD cato ateneo ntifico Swi an li inse omputing Centre Examples of Filters Sources Contour Calculator wavelet 9 Cut Pick cell 2 Clip Probe Fractal a Threshold Group Sphere ha Extract grid Ungroup Superquadric Warp vector AMR outline Stream lines
17. meshes Wu e Unstructured amp Polyhedral mesh Variables 1 to N components e Zonal and Nodal poo d CSCS 2013 67 CSCS 2013 SCS Centro Svizzero di Calcolo Scientifico a Swiss National Supercomputing Centre Summary Visit and ParaView support visualization with a focus on large data typically output by simulations in HPC Remote client server parallel interactive and batch oriented executions are used daily A data format which supports distributed access is essential CSCS 2013 68 Se CSCS Centro Svizzero di Col Scenic National Supercomputing Centre VisIt and ParaView tutorials on line We will now follow with several demonstrations of the VisIt and ParaView applications http visitusers org index php title VisIt Tutorial ewTutorial41 pdf CSCS 2013 69 Thank you for your attention CSCS 2013
18. rallel I O CSCS 2013 33 CS cs SGD ceto Swi us spa gerett Data formats Purpose of I O Archive results to file s Provide check point restart files Analysis Visualization Debugging simulations Requirements Fast parallel selective Independent off of processors Self documented CSCS 2013 34 lt x Centro Svizzero di Calcolo Scientifico o Swiss National Supercomputing Centre Data formats e Community specific CGNS CCSM NEK5000 H5Part Ad hoc Make up your own No Many formats exist Choose the most appropriate High level libraries HDF5 netCDF CSCS 2013 35 lt x Centro Svizzero di Calcolo Scientifico o Swiss National Supercomputing Centre Data formats and Parallelism e MPI IO Raw data parallelism The BOV format can be read by VisIt and ParaView e ADIOS Raw data but complexity is hidden HDF5 NetCDF content discovery is possible but semantic is left as an exercise SILO Poor man s parallelism 1 file per process metafile but strong semantic O CSCS 2013 Centro Svizzero di Calcolo Scientifico Swiss National Supercomputing Centre MPI tasks ghost cells hyperslabs Grids are sub divided with ghost regions Ghost cells nodes are usually not archived The User You is responsible for managing the subdivisions and know what to archive Example a 12 processor run CSCS 2012 Fe CSCS Centro Svizzero di Calcolo S
19. sIt employs a hybrid approach that acts as a partition but identifies regions of imbalance and handles those regions using an Image space partition IEH SE 33 R amp SE num ond CSCS 2013 CS cs SG cano Swi get soie nios Scientific Visualization Why visualization e How to Remote Visualization Client server Parallel Visualization In situ Visualization CSCS 2013 50 CSCS Centro Svizzero di Calcolo Scientifico Swiss National Supercomputing Centre CM Visualization is many complementary things noise silo sh Point 0 58807 8 09064 0 146425 Zone 59805 ncident Nodes 62225 62226 62275 62276 04725 64726 64775 64776 radial nodal 62225 2 14 2915 m E M 62226 14 0149 PES 62275 14 239 eum E Presentation Graphics Variables Visual Debugging 00 00 01 CSCS Centro Svizzero di Calcolo Scientifico Swiss National Supercomputing Centre do Kitware ParaView Eile Edit View Select Source Filler Window Help ax 99 1 eis ExtractParts 1 ExtractParts2 ReflectD qp Reflect rotor Contour dle ere ontoui Lio Pam spia mater Swiss National Supercomputing Centre Name cfxt Class vikCFXSReader Accept Reset of Filename liscratch mvalie Pelton pe Browse jest Point Arrays x All On All Off 2 Alr at 25 C Superficial Velo
20. tor client System Total 3 16 GiB 13 47 castor31 paraview 207 52 MiB 0 86 n y server System Total f 21 GiB 5 87 eee E qe jm remm 3 jo Pi System Total i am pyserver GiB 40 77 addi m E Im _ E e ban 790 36 MiB 3 36 pe i cene DI 20128 829 27 MiB 3 52 System Total e E 25 69 castor1 DO GIB 41 76 75 545 System Total 15 pyserver System Total 1 6 pwserver WF Acre CSCS Centro Svizzero di Calcolo Scientifico awd Swiss National Supercomputing Centre From the Supercomputing 13 Showcase In silico Modeling for Fracture Fixation in Osteoporotic Bone Juri Steiner Harry van Lenthe Stephen J Ferguson Jean M Favre ETH Zurich CSCS DHEST Department of 4 s g SEN EE 4 V T Crin 4 Ta 4 SC e 70100 LU dii SH SE e ee om Pr s 4 CSCS 2013 http www youtube com watch v fQ4pRrNPDSg 62 CSCS Centro Svizzero di Calcolo Scientifico SD swiss nat wiss National Supercomputing Centre No interactivity for 150 millions cells 160 140 E Render 120 Normals 100 Smoother 80 B Extract 60 Surface 40 Distribute 20 Reader 4 8x8 12x8 63 CSCS
Download Pdf Manuals
Related Search