Home

Intel® Math Kernel Library User's Guide

1. cceeeeeeeeeee sees 5 15 Specifying Makefile Parameters cccecceceeee eee eeeeeeeeeeeeeeeaeeaeenas 5 16 Specifying List Of FUNCtIONS ccceceeee ects eee eeeeeeeeeeeneeeeeeae naan 5 16 Chapter 6 Managing Performance and Memory Using Intel MKL Parallelism ceceeeeee scene eee nena eee eeeeeenaeeaeaes 6 1 Techniques to Set the Number of Threads ccccceeeeeeeeeeeeeeenees 6 3 Avoiding Conflicts in the Execution Environment eeeeeeee es 6 3 Setting the Number of Threads Using OpenMP Environment Chapter 7 Chapter 8 Chapter 9 Contents Variable rrarena noraa quid Y a aAA eee NAARAAT AEAN 6 4 Changing the Number of Threads at Run TiMe sssssssssssrrrrrrrrsrn 6 5 Using Additional Threading Control s ssssssssssssreresrerrnserrnrrsrereners 6 8 Tips and Techniques to Improve PerfOrmance cscceceeeeeeeeeeeeneeeees 6 13 Coding TEChAiQues iraisser a tanran biased dia devvaneede ata eeiia es 6 13 Hardware Configuration TipS ccceeeeeeeeee eee eee eeeeeeeeeeeeeeeaeaes 6 15 Managing Multi core Performance ccceeeeeeeeee eee eeeeeeeeeeaeees 6 15 Operating on Denormals ccceeeeee eens ee eee eee eee ee eae ee teeta tenes 6 17 FFT Optimized Radices ssssssssersessssnsrrrrrrrrnsnnsensenserrnrrennannnes 6 17 Using Intel MKL Memory Management ceeeeeeeeeeeeeeeaeeeeae es 6 17 Redefining Memory FUNCtiONS cc ccceee cece eect ee ee eee e een
2. When calling LAPACK routines from C also mind that LAPACK routine names can be both upper case or lower case with trailing underscore or not For example these names are equivalent dgetrf DGETRF dgetrf_ DGETRF_ BLAS BLAS routines are Fortran style routines If you call BLAS routines from a C language program you must follow the Fortran style calling conventions e Pass variables by address as opposed to passing by value e Store data Fortran style that is in column major rather than row major order Refer to the LAPACK section for details of these conventions See Example 7 2 on how to call BLAS routines from C 7 5 7 Intel Math Kernel Library User s Guide When calling BLAS routines from C also mind that BLAS routine names can be both upper case and lower case with trailing underscore or not For example these names are equivalent dgemm DGEMM dgemm_ DGEMM_ CBLAS An alternative for calling BLAS routines from a C language program is to use the CBLAS interface CBLAS is a C style interface to the BLAS routines You can call CBLAS routines using regular C style calls When using the CBLAS interface the header file mk1 h will simplify the program development as it specifies enumerated values as well as prototypes of all the functions The header determines if the program is being compiled with a C compiler and if it is the included file will be correct for use with C compilation Example 7 3 illustrates t
3. MKL ALL 2 MKL BLAS 1 MKL FFT 4 MKL AIL 2 MKL BLAS 1 MKL FFT 4 MKL ALL 2 MKL BLAS 1 MKL FFT 4 The global variables MKL_ ALL MKL_ BLAS MKL_ FFT and MKL_VML as well as the interface for the Intel MKL threading control functions can be found in the mk1 h header file Table 6 3 illustrates how values of MKL _ DOMAIN NUM_THREADS are interpreted 6 11 6 Intel Math Kernel Library User s Guide Table 6 3 Interpretation of MKL_DOMAIN_NUM_THREADS values Value of MKL DOMAIN NUM THREADS Interpretation MKL ALL 4 All parts of Intel MKL are suggested to try using 4 threads The actual number of threads may be still different because of the MKL_DYNAMIC setting or system resource issues The setting is equivalent to MKL NUM_THREADS 4 MKL ALL 1 MKL BLAS 4 All parts of Intel MKL are suggested to use 1 thread except for BLAS MKL_ VML 2 which is suggested to try 4 threads VML is suggested to try 2 threads The setting affects no other part of Intel MKL NOTE The domain specific settings take precedence over the overall ones For example the MKL_BLAS 4 value of MKL_DOMAIN_NUM_THREADS suggests to try 4 threads for BLAS regardless of later setting MKL_NUM_ THREADS and a function call mkl domain _set_num_ threads 4 MKL BLAS suggests the same regardless of later calls to mkl_set_num threads However pay attention to that a function call with input MKL_ ALL such as mkl_domain_s
4. Monospace lowercase Monospace lowercase mixed with uppercase UPPERCASE MONOS PACE Monospace italic items item item Italic is used for emphasis and also indicates document names in body text for example see Intel MKL Reference Manual Indicates filenames directory names and pathnames for example libmkl_core a opt intel mk1 10 1 0 004 Indicates commands and command line options for example icc myprog c LSMKLPATH ISMKLINCLUDE lmkl lguide lpthread C C code fragments for example a new double SIZE SIZE Indicates system variables for example SMKLPATH Indicates a parameter in discussions routine parameters for example 1da makefile parameters for example functions_list etc When enclosed in angle brackets indicates a placeholder for an identifier an expression a string a symbol or a value for example lt mk1 directory gt Substitute one of these items for the placeholder Square brackets indicate that the items enclosed in brackets are optional Braces indicate that only one of the items listed between braces should be selected A vertical bar separates the items 1 4 Getting Started This chapter helps you start using the Intel Math Kernel Library Intel MKL for the Linux OS by giving you some basic information and describing post installation steps Checking Your Installation After installing Intel MKL verify that the library has been properly in
5. Search scope for completion proposals Search current file and included files J Search current project Insertion X inser single proposals automatically X insert common prefixes automatically _ Present proposals in alphabetical order Content Assist parsing timeout ms 3000 Auto actvation X Enable as tigger X Enable gt as tigger Xi Enable as tigger delay ms 500 Background for completion proposals Foreground for completion proposals Completion Proposal Filter kDefault Fiter gt 7 Restore Defaults Apply OK Cancel 10 9 LINPACK and MP LINPACK Benchmarks This chapter describes the Intel Optimized LINPACK Benchmark for the Linux OS and Intel Optimized MP LINPACK Benchmark for Clusters Intel Optimized LINPACK Benchmark for Linux OS Intel Optimized LINPACK Benchmark is a generalization of the LINPACK 1000 benchmark It solves a dense real 8 system of linear equations Ax b measures the amount of time it takes to factor and solve the system converts that time into a performance rate and tests the results for accuracy The generalization is in the number of equations N it can solve which is not limited to 1000 It uses partial pivoting to assure the accuracy of the results This benchmark should not be used to report LINPACK 100 performance as that is a compiled code only benchmark This is a shared memory SMP implementation which ru
6. When the installation of Intel MKL for the Linux OS is complete you can use three scripts mklvars32 mklvarsem64t and mklvars64 with two flavors each sh and csh in the tools environment directory to set the environment variables INCLUDE LD_LIBRARY_PATH MANPATH LIBRARY PATH CPATH and FPATH in the user shell Section Automating the Process explains how to automate setting of these variables at startup For information on how to set up environment variables for threading see Setting the Number of Threads Using OpenMP Environment Variable in Chapter 6 Automating the Process To automate setting of the environment variables INCLUDE LD_LIBRARY_PATH MANPATH CPATH FPATH and LIBRARY_PATH at startup add mklvars sh to your shell profile so that each time you login the script will execute and set the path to the appropriate Intel MKL directories With the local user account you should edit the following files by adding the appropriate script to the path manipulation section right before exporting variables e bash bash_profile bash_login or profile setting up MKL environment for bash lt absolute path_to_installed_MKL gt tools environment mklvars lt arch gt sh e sh 4 Intel Math Kernel Library User s Guide profile setting up MKL environment for sh lt absolute_path_to_installed_MKL gt tools environment mklvars lt arch gt sh e csh login setting up MKL environment for csh l
7. e INTEGER KIND 8 for Fortran e long long int for C C Note that code written this way will not work for the LP64 interface Table 3 4 summarizes usage of the integer types Table 3 4 Integer types 3 8 Fortran C or C 32 bit integers INTEGER 4 int or INTEGER KIND 4 Universal integers INTEGER MKL_INT e 64 bit for ILP64 without specifying KIND 32 bit otherwise Intel Math Kernel Library Structure 3 Table 3 4 Integer types continued Fortran C or C Universal type for the FFT INTEGER MKL_LONG interface parameters without specifying KIND Browsing the Intel MKL include files Given a function with integer parameters the Reference Manual does not explain which parameters become 64 bit and which remain 32 bit for ILP64 To find out this information you need to browse the include files examples or tests You are encouraged to start with browsing the include files as they contain prototypes for all Intel MKL functions Then you may see the examples and tests for better understanding of the function usage All include files are located in the lt mk1 directory gt include directory Table A 2 in Appendix A shows the include files to browse Some function domains that support only a Fortran interface see Table A 1 still provide header files for C C in the include directory Such h files enable using a Fortran binary interface from C C code and so describe the C interface including its ILP64
8. GMP arithmetic functions B 1 GNU Multiple Precision Arithmetic Library B 1 H Help for Intel MKL in Eclipse CDT 10 1 HT Technology see Hyper Threading technology hybrid version of MP LINPACK 11 4 Hyper Threading Technology configuration tip 6 15 I ILP64 programming support for 3 5 installation checking 2 1 interface layer 3 3 J Java examples 7 12 Index 2 L language interfaces support A 1 Fortran 95 interfaces 7 2 language specific interfaces 7 1 LAPACK calling routines from C 7 4 Fortran 95 interfaces to 7 2 packed routines performance 6 14 layer compiler support RTL 3 4 computational 3 4 interface 3 3 RTL 3 3 threading 3 4 layered model 3 2 Legacy OpenMP run time compiler library 5 11 library run time Compatibility OpenMP 5 11 run time Legacy OpenMP 5 11 library structure 3 1 link command examples 5 8 syntax 5 3 link libraries computational 5 13 for Intel 64 architecture 5 13 interface for the Absoft compilers 5 11 threading 5 12 linkage models comparison 5 2 linking 5 1 dynamic 5 1 layered model 5 4 legacy model 5 4 recommendations 5 2 static 5 1 with Cluster FFT 9 1 with ScaLAPACK 9 1 LINPACK benchmark 11 1 M memory functions redefining 6 18 memory management 6 17 memory renaming 6 18 mixed language programming 7 4 module Fortran 95 7 4 MP LINPACK benchmark 11 4 hybrid version 11 4 multi cor
9. Math Kernel Library Intel MKL basically provides support for Fortran and C C programming However not all function domains support both Fortran and C interfaces see Table A 1 in Appendix A For example LAPACK has no C interface Still you can call functions comprising these domains from C using mixed language programming Moreover even if you want to use LAPACK or BLAS which basically support Fortran in the Fortran 95 environment additional effort is initially required to build language specific interface libraries and modules being delivered as source code The chapter mainly focuses on mixed language programming and the use of language specific interfaces It expands upon the use of Intel MKL in C language environments for function domains that basically support Fortran as well as explains usage of language specific interfaces and in particular Fortran 95 interfaces to LAPACK and BLAS In this connection compiler dependent functions are discussed to explain why Fortran 90 modules are supplied as sources A separate section guides you through the process of running examples of invoking Intel MKL functions from Java Using Language Specific Interfaces with Intel MKL The following interface libraries and modules may be generated as a result of operation of respective makefiles located in the interfaces directory Table 7 1 Interface libraries and modules File name Comment libmkl_blas95 a Contains Fortran 95 wrappers for BLAS
10. Processor numbers differentiate features within each processor family not across different processor families See http www intel com products processor_number for details BunnyPeople Celeron Celeron Inside Centrino Centrino Atom Centrino Atom Inside Centrino Inside Centrino logo Core Inside FlashFile i960 InstantIP Intel Intel logo Intel386 Intel486 IntelDX2 IntelDX4 IntelSX2 Intel Atom Intel Atom Inside Intel Core Intel Inside Intel Inside logo Intel Leap ahead Intel Leap ahead logo Intel NetBurst Intel NetMerge Intel NetStructure Intel SingleDriver Intel SpeedStep Intel StrataFlash Intel Viiv Intel vPro Intel XScale IPLink Itanium Itanium Inside MCS MMX Oplus OverDrive PDCharm Pentium Pentium Inside skoool Sound Mark The Journey Inside VTune Xeon and Xeon Inside are trademarks of Intel Corporation in the U S and other countries Other names and brands may be claimed as the property of others Copyright 2006 2008 Intel Corporation All rights reserved Contents Chapter 1 Overview Technical SUPPOrt eniin a A edd le E A A 1 1 About This DOCUMENE erisa errete aran erra REEL EREA at TEAREN RASER 1 1 aea O 358 AA E A A A a ences 1 2 AUO nE E o A EEE E A EEE E AOA E ard 1 2 Document Organization s ssssssssessrrrrrrrrssseurrrnnrnrrnnrnnnunununrrrnnnn 1 2 Term and Notational ConventionS sssssssrssrrrrrrrrresssssrrrrnrrrrrs 1 3 Chapter 2 Getting Started Checking
11. conj trans m2 prod trans conj m1 conj trans m2 7 11 7 Intel Math Kernel Library User s Guide prod conj trans m1 conj trans m2 These expressions are substituted in the release mode only with NDEBUG preprocessor symbol defined Supported uBLAS versions are Boost 1 34 1 and Boost 1 35 0 To get them visit www boost org A code example provided in the lt mkl_directory gt examples ublas source sylvester cpp file illustrates usage of the Intel MKL uBLAS header file for solving a special case of the Sylvester equation To run the Intel MKL ublas examples specify the BOOST_ROOT parameter in the make command for instance make lib32 BOOST_ROOT lt your_path gt boost_1_35_0 Invoking Intel MKL Functions from Java Applications 1 This section describes examples that are provided with the Intel MKL package and illustrate calling the library functions from Java Intel MKL Java examples Java was positioned by its inventor the Sun Microsystems Corporation as Write Once Run Anywhere WORA language Intel MKL may help to speed up Java applications the WORA philosophy being partially supported as Intel MKL editions are intended for wide variety of operating systems and processors covering most kinds of laptops and desktops many workstations and servers To demonstrate binding with Java Intel MKL includes the set of Java examples found in the following directory lt mkl directory gt examp
12. core a libmk1_core a Dynamic libmkl_lapack so libmk1_core so libmkl_core so libmkl_core so libmkl_core so libmk1_core so libmkl_core so See below libmk1_scalapack _1p64 so libmk1_lapack so libmk1_core so libmk1l_scalapack_ ilp64 so libmk1_lapack so libmk1_core so n a 1 Not applicable Linking Your Application with Intel Math Kernel Library 5 2 Add also the library with BLACS routines corresponding to the used MPI For details see Linking with ScaLAPACK and Cluster FFTs in chapter 9 Notes on Linking Updating LD_LIBRARY_PATH When using the Intel MKL shared libraries do not forget to update the shared libraries environment path that is a system variable LD_LIBRARY_PATH to include the libraries location For example if the Intel MKL libraries are in the lt mk1_directory gt lib 32 directory the following command line can be used assuming a bash shell export LD LIBRARY PATH lt mkl1_directory gt 1ib 32 LD LIBRARY PATH Linking with threading libraries If you link with libiomp statically discouraged e and use the Intel compiler then link in the libiomp version that comes with the compiler that is use openmp option e but do not use the Intel compiler then link in the 1ibiomp version that comes with Intel MKL If you use dynamic linking libiomp5 so of the threading library recommended make sure the LD_LIBRARY_PATH is defined so that exactly this
13. you can call the same function using the CBLAS interface as follows cblas_cdotu n x 1 y 1 amp result NOTE NOTE The complex value comes back expressly in this case The following example illustrates a call from a C program to the complex BLAS Level 1 function zdotc This function computes the dot product of two double precision complex vectors In this example the complex dot product is returned in the structure c 7 7 7 Intel Math Kernel Library User s Guide Example 7 1 Calling a complex BLAS Level 1 function from C include mkl h define N 5 void main int n inca 1 incb 1 i typedef struct double re double im complex16 complex16 a N bIN c void zdotc n N for i 0 i lt n i a i re double i a i im double i 2 0 b i re double n i b i im double i 2 0 zdote amp c amp n a amp inca b amp incb printf The complex dot product is 6 2f 6 2f n c re c im 7 8 Language specific Usage Options 7 Below is the C implementation Example 7 2 Calling a complex BLAS Level 1 function from C include lt complex gt include lt iostream gt define MKL Complex1 6 std complex lt double gt include mkl h define N 5 int main int n inca 1 incb 1 i std complex lt double gt a N bIN c n N for i 0 i lt n i afi std complex lt double gt i i 2 0 b
14. 1m See Selecting Libraries to Link for details of this syntax usage and specific recommendations on which libraries to link depending on your Intel MKL usage scenario See also e Fortran 95 Interfaces and Wrappers to LAPACK and BLAS in chapter 7 for information on the libraries that you should build prior to linking e Working with Intel Math Kernel Library Cluster Software on linking with cluster components To link with Intel MKL you can choose layered model or legacy model which is backward compatible on link line except cluster components The syntax above incorporates both models For the layered model you need to choose one library from the Interface layer one library from the Threading layer the Computational layer library no choice here and add run time libraries In case of the legacy model you need not change the link line with respect to the one used with Intel MKL 9 x see the Dummy Libraries section in chapter 3 for details Figure 5 1 compares linking for the Intel MKL version 10 0 or higher which uses layers and Intel MKL 9 x Linking Your Application with Intel Math Kernel Library 5 Figure 5 1 Linking with Layered Intel Math Kernel Library Intel MKL 10 0 Intel MKL 9 x E choose one In case of employing the layered model for static linking the cluster component Interface layer Threading layer and Computational layer libraries must be enclosed in grouping symbols
15. 4 nodes Make an HPL run using compile options such as ASYOUGO or ASYOUGO2 or ENDEARLY to aid in your search These options enable you to gain insight into the performance sooner than HPL would normally give this insight When doing so follow these recommendations Use the MP LINPACK patched version of HPL to save time in the searching Using a patched version of HPL should not hinder your performance That s why features that could be performance intrusive are compile optional and it is called out below in MP LINPACK That is if you don t use any of the new options explained in section Options to reduce search time then these changes are disabled The primary purpose of the additions is to assist you in finding solutions HPL requires long time to search for many different parameters In the MP LINPACK the goal is to get the best possible number Given that the input is not fixed there is a large parameter space you must search over In fact an exhaustive search of all possible inputs is improbably large even for a powerful cluster This patched version of HPL optionally prints information on performance as it proceeds or even terminates early depending on your desires Save time by compiling with DENDEARLY DASYOUGO2 described in the Options to reduce search time section and using a negative threshold Do not to use a negative threshold on the final run that you intend to submit if you are doing a Top500 entry Y
16. ASYOUGO2_DISPLAY see details in the New Features section if it was captured HPL 1 0a code modified to capture DGEMM information if desired from ASYOUGO2_DISPLAY HPL 1 0a code modified to do additional grid experiments originally not in HPL 1 0 HPL 1 0a code modified to do ASYOUGO and ENDEARLY modifications Bugfix added to allow for 64 bit address computation HPL 1 0a code modified to do ASYOUGO ASYOUGO2 and ENDEARLY modifications HPL 1 0a sample HPL dat modified New Sample architecture make for processors using IA 32 architecture and Linux New Sample architecture make for processors using Intel 64 architecture and Linux New Sample architecture make for IA 64 architecture and Linux A repeat of testing ptest HPL dat in the top level directory 1 1 Intel Math Kernel Library User s Guide Table 11 2 Contents of the MP LINPACK Benchmark benchmarks mp_linpack Next three files are prebuilt executables readily available for simple performance testing bin _intel ia32 xhpl_ia32 New Prebuilt binary for IA 32 architecture Linux and Intel MPI 3 0 bin _intel em64t xhpl em64t New Prebuilt binary for Intel 64 architecture Linux and Intel MPI 3 0 bin _intel ipf xhpl_ipf New Prebuilt binary for IA 64 architecture Linux and Intel MPI 3 0 Next three files are prebuilt hybrid executables bin _intel ia32 xhpl_hybrid_ New Prebuilt hybrid binary for IA 32 architecture ia32 Linux a
17. Kernel Library Language Interfaces Support Reason In case your function domain does not directly support the needed environment you can use mixed language programming see Mixed langquage programming with Intel MKL For a list of language specific interface libraries and modules and an example how to generate them see also Using Language Specific Interfaces with Intel MKL Select among the following options how you are going to thread your application e Your application is already threaded e You may want to use the Intel threading capability that is Compatibility OpenMP run time library Libiomp or Legacy OpenMP run time library Libguide or a threading capability provided by a third party compiler You do not want to thread your application Reason By default the OpenMP software sets the number of threads that Intel MKL uses If you need a different number you have to set it yourself using one of the available mechanisms For more information and especially how to avoid conflicts in the threaded execution environment see Using Intel MKL Parallelism Additionally the compiler that you use to thread your application determines which threading library you should link with your application see Linking Examples Decide which linking model is appropriate for linking your application with Intel MKL libraries Static Dynamic Reason For information on the benefits of each linking model link
18. Library User s Guide Example 8 1 Aligning addresses at 16 byte boundaries RKKKKK C language x include lt stdlib h gt void darray int workspace Allocate workspace aligned on 16 bit boundary darray MKL_ malloc sizeof double workspace 16 call the program using MKL mkl_app darray Free workspace MKL free darray e e kk k Fortran language double precision darray pointer p wrk darray 1 integer workspace Allocate workspace aligned on 16 bit boundary p_wrk mkl_malloc 8 workspace 16 call the program using MKL call mkl_app darray Free workspace call mkl_free p_wrk 8 2 Working with Intel Math Kernel Library Cluster Software This chapter discusses usage of Intel MKL ScaLAPACK and Cluster FFTs mainly describing linking your application with these domains and including C and Fortran specific linking examples gives information on the supported MPI See Table 3 7 for detailed Intel MKL directory structure in chapter 3 For information on the available documentation and the doc directory see Table 3 7 in the same chapter For information on MP LINPACK Benchmark for Clusters see section Intel Optimized MP LINPACK Benchmark for Clusters in chapter 11 Intel MKL ScaLAPACK and FFTs support MPICH 1 2 x and Intel MPI To link a program that calls ScaLAPACK you need to know how to link an MPI application first
19. MKL may actually use the number of threads that differs from the one suggested the controls will also enable you to instruct the library to try using the suggested number in the event of an undetectable threading behavior in the application calling the library NOTE NOTE Intel MKL does not always have a choice on the number of threads for certain reasons such as system resources Employing Intel MKL threading controls in your application is optional If you do not use them the library will mainly behave the same way as Intel MKL 9 1 in what relates to threading with the possible exception of a different default number of threads See Note on FFT Usage for the usage differences 6 8 Managing Performance and Memory 6 Table 6 2 lists the Intel MKL environment variables for threading control their equivalent functions and OMP counterparts Table 6 2 Intel MKL environment variables for threading controls Equivalent OMP Environment Environment Variable Service Function Comment Variable MKL NUM THREADS mkl_set_num threads Suggests the number of OMP_NUM_THREADS threads to use MKL_ DOMAIN NUM_ mkl_ domain _set_num Suggests the number of THREADS threads threads for a particular function domain MKL DYNAMIC mk1_set_dynamic Enables Intel MKL to OMP_DYNAMIC dynamically change the number of threads NOTE The functions take precedence over the respective environment variables In particular if in your application you w
20. MKLVersion structure which also contains the version information For the function descriptions and calling syntax see the Support Functions chapter in the Intel MKL Reference Manual Sample programs for extracting version information are provided in the examples versionquery directory A makefile is also provided to automatically build the examples and output summary files containing the version information Compiler Support Intel supports Intel MKL for use only with compilers identified in the Release Notes However the library has been successfully used with other compilers as well When using the CBLAS interface the header file mk1 h will simplify program development since it specifies enumerated values as well as prototypes for all the functions The header determines if the program is being compiled with a C compiler and if so the included file will be correct for use with C compilation Starting with Intel MKL 9 1 full support is provided for the GNU gfortran compiler which differs from the Intel Fortran Compiler in calling conventions for functions that return complex data Absoft Fortran compilers are supported as well For usage specifics of the Absoft compilers see Linking with the Absoft compilers in chapter 5 Using Intel MKL Code Examples 2 2 Intel MKL package includes code examples located in the examples subdirectory of the Intel MKL installation directory The examples provide the most direct and imme
21. Table A 1 in Appendix A This document assumes you have completed the installation of Intel MKL on your system If you have not completed the installation see Intel Math Kernel Library Installation Guide file Install txt 1 1 1 Intel Math Kernel Library User s Guide This guide should be used in conjunction with the latest version of the Inte Math Kernel Library for Linux Release Notes document to reference how to use the library in your application Purpose Intel Math Kernel Library for Linux User s Guide is intended to assist you in mastering the usage of the Intel MKL on Linux In particular it e Describes post installation steps to help you start using the library e Shows you how to configure the library with your development environment e Acquaints you with the library structure e Explains how to link your application to the library and provides simple usage scenarios e Describes how to code compile and run your application with Intel MKL for Linux Audience The guide is intended for Linux programmers with beginner to advanced experience in software development Document Organization The document contains the following chapters and appendices Chapter 1 Chapter 2 Chapter 3 Chapter 4 Chapter 5 1 2 Overview Introduces the concept of the Intel MKL usage information describes the document s purpose and organization as well as explains notational conventions Getting
22. Typically this involves using mpi scripts mpicc or mpif77 C or FORTRAN 77 scripts that use the correct MPI header files and others If for instance you are using MPICH installed in opt mpich then typically opt mpich bin mpicc and opt mpich bin mpif77 will be the compiler scripts and opt mpich 1lib libmpich a will be the library used for that installation Linking with ScaLAPACK and Cluster FFTs To link to ScaLAPACK and or Cluster FFTs in Intel MKL basically use the following general form lt lt MPI gt linker script gt lt files to link gt L lt MKL path gt Wl start group lt MKL Cluster Library gt lt BLACS gt lt MKL Core Libraries gt Wl end group where lt MPI gt is one of several MPI implementations MPICH Intel MPI 2 x Intel MPI 3 x 9 1 9 Intel Math Kernel Library User s Guide lt BLACS gt is one of BLACS libraries for the appropriate architecture which are listed in Table 3 6 for example for IA 32 architecture it is one of lmk1l_blacs 1lmkl_blacs_intelmpi and 1mk1_blacs_openmpi lt MKL Cluster Library gt is one of ScaLAPACK or Cluster FFT libraries for the appropriate architecture which are listed in Table 3 6 for example for IA 32 architecture it is one of lmkl_scalapack_core or lmkl_cdft_core lt MKL Core Libraries gt iS lt MKL LAPACK amp MKL kernel libraries gt for ScaLAPACK and lt MKL kernel libraries gt for Cluster FFTs lt MKL kernel libraries gt are p
23. Your Installatio Nsu a e a a a e 2 1 Obtaining Version Information sssssssssssssrrsssrrrnsrurrnnnnerrnnnuennnnnuran 2 2 Compiler S pport ria anra a tera i eh ea eed AAE Ea 2 2 Using Intel MKL Code ExampleS ssssssssssssssrrsrsrnrrnrsrrnrnnrrrnrrsrrrrners 2 2 Before You Begin Using Intel MKL sssssssssssrrssrrrrrsrerrrrnurrrnrrsrerrners 2 3 Chapter 3 Intel Math Kernel Library Structure High level Directory Structure cceeceeeee eee eee eee eee eens i 3 1 Layered Model COnCept sssssssrsrrsrrsersessenrerrsrnrrnnesnennerrorrrenennnee 3 2 LAY GIS canes aTi EE E VAN AAN EINNA ARARA aa Aa at 3 3 Sequential Version of the Library sssssssssssssssssrrrrrerrrrssensrerenrrrns 3 4 Support for ILP64 PrograMMing csceceeee eset eee ee teeta teat neta eee eees 3 5 Intel MKL Versions tiee Re idee cetera TEE nite ean ia 3 10 Directory Structure in Detail ssssssssssrsessesssrrernrrnnnnnsssssnerrnernnn 3 10 DUMMY Libraries cccce cece ese eee eeee ii i a a aa 3 20 Accessing the Intel Math Kernel Library Documentation 3 20 Intel Math Kernel Library User s Guide Contents of the Documentation DireCtOry cccceceeeeeeeeene eee eenees 3 20 Accessing Man Pages arer oiai cpu bade wou E bade vantery 3 21 Chapter 4 Configuring Your Development Environment Setting Environment Variables ccceeeeeeeeeee eens eee cena eeeeeeeeneees 4 1 Automating the Processerna cr eee a dc
24. a shared memory machine For that the Intel Optimized LINPACK Benchmark should be used instead This benchmark should be used on a distributed memory machine Intel is providing optimized versions of the LINPACK benchmarks to make it easier than using HPL for you to obtain high LINPACK benchmark results on your systems based on genuine Intel processors Use this package to benchmark your cluster The prebuilt binaries require Intel MPI 3 x be installed on the cluster The run time version of Intel MPI is free and can be downloaded from www intel com software products cluster NOTE If you wish to use a different version of MPI you can do so by using the MP LINPACK source provided The package includes software developed at the University of Tennessee Knoxville Innovative Computing Laboratories and neither the University nor ICL endorse or promote this product Although HPL 1 0a is redistributable under certain conditions this particular package is subject to the MKL license Intel MKL 10 0 Update 3 has introduced a new functionality into MP LINPACK which is called a hybrid build while continuing to support the older version The term hybrid refers to special optimizations added to take advantage of mixed OpenMP MPI parallelism If you want to use one MPI process per node and to achieve further parallelism via OpenMP use of the hybrid build If you want to rely exclusively on MPI for parallelism and use one MPI per core u
25. appropriate the radio button 10 7 1 0 Intel Math Kernel Library User s Guide 10 8 To insert an element if it is the only item in the list when Content Assist is invoked check Insert single proposals automatically To display proposals in alphabetical order rather than by relevance check Present proposals in alphabetical order To change the amount of time Content Assist is permitted to parse proposals type the value in the Content Assist parsing timeout text box area To enable alternative triggers for Content Assist check appropriate check boxes under Auto activation To change the delay before Content Assist is automatically invoked for the triggers see above type the new delay in the delay text box area under Auto activation To change the background color of the Content Assist dialog box click the color palette button next to Background for completion proposals To change the foreground color of the Content Assist dialog box click the color palette button next to Foreground for completion proposals Click OK Getting Assistance for Programming in the Eclipse IDE 1 0 Figure 10 7 Customizing Code Content Assist Preferences paeme type filter text b General b Ant 7 CiC Appearance Build Console Code Formatter b Debug V Editor Templates File Types Indexer Make Managed Build Parser PathEntry Vanable b Help b install Update b Java b Plugin Development Mi 818 Code Assist
26. as MKL NUM_THREADS and equivalent Intel MKL functions for thread management see Using Additional Threading Control for details The Intel MKL variables are always inspected first then the OpenMP variables are examined and if neither are used the OpenMP software chooses the default number of threads This is a change with respect to Intel MKL versions 9 x or earlier which used a default value of one as the Intel compiler OpenMP software uses the default number of threads equal to the number of processors in your system NOTE NOTE In Intel MKL 10 1 the OpenMP software determines the default number of threads The default number of threads is equal to the number of logical processors in your system for Intel OpenMP libraries To achieve higher performance you are recommended to set the number of threads to the number of real processors or physical cores Do this by any available means which are summarized in section Techniques to Set the Number of Threads 1 Except LAPACK deprecated routines lacon lasq3 and lasq4 6 2 Managing Performance and Memory 6 Techniques to Set the Number of Threads You can employ different techniques to specify the number of threads to use in Intel MKL e Set OpenMP or Intel MKL environment variable OMP_ NUM THREADS MKL NUM THREADS MKL DOMAIN NUM THREADS e Call OpenMP or Intel MKL function omp_set_num_threads mkl_set_num_threads mkl_domain_set
27. attention to the iterative steps 3 and 4 They make up a loop that searches for HPL parameters specified in HPL dat which the top performance of you cluster is reached with 11 7 1 1 Intel Math Kernel Library User s Guide Get HPL installed and functional on all the nodes You may run nodeperf c included in the distribution to see the performance of DGEMM on all the nodes Compile nodeperf c in with your MPI and Intel MKL For example mpicc 03 nodeperf c lt mk1_directory gt lib em6 4t libmkl_em 4t a lt mk1l_directory gt lib em6 4t libguide a lpthread o nodeperf Launching nodeperf c on all the nodes is especially helpful in a very large cluster Indeed there may be a stray job on a certain node for example 738 which is running 5 slower than the rest MP LINPACK will then run as slow as the slowest node In this case nodeperf enables quick identifying of the potential problem spot without lots of small MP LINPACK runs around the cluster in search of the bad node It is common that after a bunch of HPL runs there may be zombie processes and nodeperf facilitates finding the slow nodes It goes through all the nodes one at a time and reports the performance of DGEMM followed by some host identifier Therefore the higher the penultimate number then the faster that node was performing Edit HPL dat to fit your cluster needs Read through the HPL documentation for ideas on this However you should try on at least
28. coding data alignment 8 1 mixed language calls 7 7 techniques to improve performance 6 13 Compatibility OpenMP run time compiler library 5 11 compiler support 2 2 compiler support RTL layer 3 4 compiler Absoft linking with 5 11 compiler dependent function 7 3 computational layer 3 4 configuration file for OOC DSS PARDISO 4 4 configuring development environment 4 1 Eclipse CDT 4 2 content assist see code assist context sensitive Help for Intel MKL in Eclipse CDT 10 4 custom shared object 5 15 building 5 15 specifying list of functions 5 16 specifying makefile parameters 5 16 D data alignment 8 2 denormal performance 6 17 development environment configuring 4 1 directory structure documentation 3 20 high level 3 1 in detail 3 10 documentation 3 20 for Intel MKL viewing in Eclipse ID 10 1 dummy library 3 20 dynamic linking 5 1 Index 1 Intel Math Kernel Library User s Guide Eclipse CDT code content assist with Intel MKL 10 6 configuring 4 2 searching the Intel Web site 10 3 Eclipse CDT Intel MKL Help 10 1 0 1 context sensitive 10 4 environment variables setting 4 1 examples code 2 2 linking general 5 8 ScaLAPACK Cluster FFT linking with 9 3 F FFT functions data alignment 6 14 FFT interface MKL_LONG type 3 6 optimized radices 6 17 threading tip 6 13 FFTW interface support B 1 Fortran 95 interfaces to LAPACK and BLAS 7 2 G
29. double data initialization However this optimization requires a change in the FFT usage Suppose you create threads in the application yourself after initializing all FFT descriptors In this case threading is employed for the parallel FFT computation only the descriptors are released upon return from the parallel region and each descriptor is used only within the corresponding thread Starting with Intel MKL 10 0 you must explicitly instruct the library before the commit stage to work on one thread To do this set MKL_NUM_THREADS 1 or MKL DOMAIN NUM THREADS MKL FFT 1 or call the corresponding pair of service functions Otherwise the actual number of threads may be different because the D tiCommitDescriptor function is not in a parallel region See Example C 27a Using Parallel Mode with Multiple Descriptors Initialized in One Thread in the Intel MKL Reference Manual Tips and Techniques to Improve Performance To obtain the best performance with Intel MKL follow the recommendations given in the subsections below Coding Techniques To obtain the best performance with Intel MKL ensure the following data alignment in your source code e arrays are aligned at 16 byte boundaries 6 13 6 Intel Math Kernel Library User s Guide 6 14 e leading dimension values n element_size of two dimensional arrays are divisible by 16 e for two dimensional arrays leading dimension values divisible by 2048 are avoided LAPACK packed
30. fi mkl_lapack f90 mkl_lapack fi mkl_solver f90 mkl_pardiso f77 mkl_pardiso 90 mkl_dss mkl_dss 77 90 mkl_rci fi mkl_rci fi mkl _ vml mkl_vml fi Include files Cor C mkl h mkl_blas h mkl_trans h mkl_cblas h mkl_spblas h mkl_lapack h mkl_scalapack h mkl_solver h mkl_pardiso h mkl_dss h mkl_rci h mkl_rci h mkl _vml h C C interface Table A 2 Function domain Vector Statistical Functions Fourier Transform Functions Cluster Fourier Transform Functions Partial Differential Equations Support Routines Trigonometric Transforms Poisson Solvers GMP interface Service routines Memory allocation routines MKL examples interface Intel Math Kernel Library Language Interfaces Support l Intel MKL include files continued Include files Fortran mkl_vml 77 mkl_vsl fi mkl_dfti f90 mkl_cdft f90 mkl_ trig transforms f 90 mk1_ poisson 90 Cor C mkl_vsl h mkl_dfti h mkl_cdft h mkl_ trig transforms h mk1l_poisson h mkl_gmp h mkl_service h i_malloc h mkl_example h A 3 Support for Third Party Interfaces This appendix describes in brief certain interfaces that Intel Math Kernel Library Intel MKL supports GMP Functions Intel MKL implementation of GMP arithmetic functions includes arbitrary precision arithmetic operations on integer numbers The interfaces of such functions fully match the GNU Multipl
31. for Programming in the Eclipse IDE 1 0 Place the cursor to the function name 2 Press Fi This causes two lists to display The list of links to the relevant topics in the product documentation displays in the Related Topics page under See also The Intel MK Help Index establishes the relevance see Figure 10 4 Typically one link displays in this list for each function The list of search results for the function name displays in the Related Topics page under Dynamic Help see Figure 10 5 3 Click a needed link to open the Help topic Figure 10 4 F1 Help in the Eclipse IDE test_3 cpp Eclipse Platform rid C s Ji x 3 NT TT uxisetVersfion d baz u version 10 5 1 0 Intel Math Kernel Library User s Guide Figure 10 5 F1 Help Search in the Eclipse IDE CDT est cpp Eclipse Platform Search Project Run Window Hep iver eEr amp rSrl HrOraqe lep u se gilia c RNE B S outi Mak 7 BY Hep 2n l a ar eee a Related Topics About C C Editor Click below to see help See also Editor view Dynamic Help Search results Verson Information Functions MKLGetVersion Support Functions IF More resus is 23 A Tasks E console Properties eyes Using Code Content Assist in the Eclipse IDE CDT Code or Content Assist feature in Eclipse means automatic prompt for possible completion of a code line being written in
32. gf ilp6 4 a libmkl_ gf lp64 a Threading layer libmk1l_intel_thread a libmkl_gnu_thread a libmkl_sequential a Computational layer libmkl_cdft a libmkl_cdft_core a libmkl_core a libmkl_ipf a libmkl_lapack a libmk1l_scalapack a libmk1l_scalapack_ ilp 4 a libmkl_scalapack_ lp64 a Contents ILP64 version of BLACS routines supporting Intel MPI 2 0 and 3 0 and MPICH2 LP64 version of BLACS routines supporting Intel MPI 2 0 and 3 0 and MPICH2 Libraries for IA 64 architecture ILP64 interface library for Intel compiler LP64 interface library for Intel compiler SP2DP interface library for Intel compiler ILP64 interface library for GNU Fortran compiler LP64 interface library for GNU Fortran compiler Parallel drivers library supporting Intel compiler Parallel drivers library supporting GNU compiler Sequential drivers library Dummy library Contains a reference to 1ib 64 libmk1 cdft_core a Cluster version of FFTs Kernel library for IA 64 architecture Dummy library Contains references to Intel MKL libraries 1ib 64 libmk1l intel lp64 a 1ib 64 libmkl intel thread a and 1ib 64 libmk1_ core a Dummy library Contains references to Intel MKL libraries 1ib 64 libmk1l intel lp64 a 1ib 64 libmk1l intel thread a and 1ib 64 libmk1_ core a Dummy library Contains a reference to 1ib 64 libmk1 scalapack_1lp 64 a ScaLAPACK routines library supporting ILP64 interface ScaLAPACK routines library supportin
33. i std complex lt double gt n 1i i 2 0 zdotc amp c amp n a amp inca b amp incb std cout lt lt The complex dot product is lt lt c lt lt std endl return 0 7 9 7 Intel Math Kernel Library User s Guide The implementation below uses CBLAS Example 7 3 Using CBLAS interface instead of calling BLAS directly from C include mk1 h typedef struct double re double im complex16 extern C void cblas_zdotc_sub const int const complexi6 const int const complex16 const int const complexl16 define N 5 void main int n inca 1 incb 1 i complex16 a N bIN c n N for i 0 i lt n i a i re double i a i im double i 2 0 b i re double n i b i im double i 2 0 cbhlas_zdotc_sub n a inca b incb amp c printf The complex dot product is 6 2f 6 2f n c re c im Support for Boost uBLAS Matrix Matrix Multiplication If you have got used to uBLAS you can perform BLAS matrix matrix multiplication in C using Intel MKL substitution of Boost uBLAS functions uBLAS pertains to the Boost C open source libraries and provides BLAS functionality for dense packed and sparse Language specific Usage Options 7 matrices The library uses an expression template technique for passing expressions as function arguments which enables evaluating vector and matrix expressions in one pass witho
34. intel 1p64 so libmkl_ intel thread so libmkl_lapack so libmkl_core so ScaLAPACK static case Legacy libmkl_scalapack a libmkl_blacs a libmkl_lapack a libmkl_em6 4t a Layered LP64 libmkl_scalapack_1p64 a libmkl_blacs_ lp64 a libmkl_intel_1p64 a libmkl_intel_thread a libmkl_lapack a libmkl_core a ScaLAPACK dynamic case Layered LP64 libmk1_scalapack_1p64 so libmkl_blacs_ 1p64 so libmkl_intel_1p64 so libmkl_intel_thread so libmkl1_lapack so libmkl_core so Iterative Sparse Solver static case Legacy libmkl_solver a libmkl_lapack a libmkl_em 4t a Layered LP64 libmkl_ solver _1p64 a libmkl_intel lp64 a libmkl_intel_ thread a libmkl_core a Layered ILP64 libmkl_ solver _ilp64 a libmkl_intel ilp 4 a libmkl_intel_ thread a libmkl_core a DSS PARDISO static case Legacy libmkl_lapack a libmkl_em 4t a Layered LP64 libmkl_ intel _1lp6 4 a libmkl_ intel thread a libmkl_core a Linking Your Application with Intel Math Kernel Library 5 Layered ILP64 libmkl_ intel ilp6 4 a libmkl_intel thread a libmkl_core a DSS PARDISO dynamic case Legacy libmkl_lapack so libmkl_em6 4t so Layered LP64 libmkl_ intel _1p6 4 so libmkl_intel_ thread so libmkl_lapack so libmk1l_core so Layered ILP64 libmkl intel _ilp6 4 so libmkl_intel_ thread so libmkl_lapack so libmk1l_core so When linking see Link Command Syntax and Linking Examples note that The Iterative Sparse Solver and Trust Region Solver routine librar
35. interface ifort myprog f LSMKLPATH ISMKLINCLUDE 1lmkl_solver_1p64 Wl start group MKLPATH libmkl_ intel _l1p64 a SMKLPATH libmkl_intel_thread a S MKLPATH libmkl_core a W1 end group liomp5 lpthread Static linking of user s code myprog f sequential version of an iterative sparse solver and sequential Intel MKL supporting LP64 interface ifort myprog f LSMKLPATH ISMKLINCLUDE lmkl_solver_1lp64 sequential Wl start group MKLPATH libmkl_intel_ 1p64 SMKLPATH libmk1_ sequential a SMKLPATH libmkl_core a W1 end group lpthread See Fortran 95 Interfaces and Wrappers to LAPACK and BLAS in chapter 7 for information on how to build Fortran 95 LAPACK and BLAS interface libraries Linking Your Application with Intel Math Kernel Library 5 Linking with Interface Libraries Linking with the Absoft compilers You can use Intel MKL with the Absoft compilers on systems based on Intel 64 or IA 32 architecture Table 5 2 explains which Interface layer library must be included in the link line to link with the Absoft compilers Table 5 2 Interface layer library for linking with the Absoft compilers Programming Architecture Interface Static Linking Dynamic Linking IA 32 Does not matter libmkl_intel a libmkl_intel so Intel 64 ILP64 libmkl_ gf _ilp6 4 a libmkl_ gf _ilp64 so Intel 64 LP64 libmkl_gf lp6 4 a libmkl_gf_1p64 so Linking with Threading Libraries Several compilers that Intel MKL supports have recently s
36. routines The routines with the names that contain the letters HP OP PP SP TP UP in the matrix type and storage position the second and third letters respectively operate on the matrices in the packed format see LAPACK Routine Naming Conventions sections in the Intel MKL Reference Manual Their functionality is strictly equivalent to the functionality of the unpacked routines with the names containing the letters HE OR PO SY TR UN in the corresponding positions but the performance is significantly lower If the memory restriction is not too tight use an unpacked routine for better performance Note that in such a case you need to allocate N2 2 more memory than the memory required by a respective packed routine where N is the problem size the number of equations For example solving a symmetric eigenproblem with an expert driver can be speeded up through using an unpacked routine call dsyevx jobz range uplo n a lda vl vu il iu abstol m w z ldz work lwork iwork ifail info where a is the dimension 1da by n which is at least N elements instead of call dspevx jobz range uplo n ap vl vu il iu abstol m w Z ldz work iwork ifail info where ap is the dimension N N 1 2 FFT functions There are additional conditions which improve performance of the FFT functions Applications based on IA 32 or Intel 64 architecture The addresses of the first elements of arrays and the leading d
37. s Guide Chapter 10 Getting Assistance for Programming in the Eclipse IDE Chapter 11 Viewing the Intel MKL Reference Manual in the Eclipse IDE 10 1 Searching the Intel Web Site from the Eclipse IDE 008 10 3 Using Context Sensitive Help in the Eclipse IDE CDT 006 10 4 Using Code Content Assist in the Eclipse IDE CDT ccecce 10 6 LINPACK and MP LINPACK Benchmarks Intel Optimized LINPACK Benchmark for Linux OS ecese 11 1 Contentia aeni tea Eea E tia dt Soda cA eee eaten ake 11 1 Running the Software c cece eee ee eect eee eee eee ee eee nena nea teeta ees 11 2 KNOWN LIMMAT ON Sct ticeten cee cdeets p E ATTA 11 3 Intel Optimized MP LINPACK Benchmark for Clusters ssec 11 4 CONTENTS 755 asia Maske Missa ka A TTAN E aaa Adee Aaa ae 11 5 Building MP LINPACK 3 csi cces ser sit eas aria iota anita ted 11 6 NEW FOACUIES cca cectavuens cecdwacetsaevasleevs devincecteakecdavdNhccivaaedaweerdteey 11 7 Benchmarking a CIUStED ecceee eee eect eee eee eee aN 11 7 Appendix A Intel Math Kernel Library Language Interfaces Support Appendix B Support for Third Party Interfaces Index GMP FUNCTIONS toyadi ei an EEEE uni le EAE OTONA B 1 FFTW Interface Support ssssessssssrrsrrrerrernennssnesnsrnrerrnnnannneseessrrnes B 1 List of Tables vii Table 1 1 Notational CONVENTIONS ssssssssssrsrrrrssrsserrerrrrrrrrrresrens 1 4 Table 2 1 What you need to know before you
38. shared objects for IA 32 architecture Contains static libraries and shared objects for IA 64 architecture Itanium processor family Contains static libraries and shared objects for Intel 64 architecture formerly Intel EM64T Contains man pages for Intel MKL functions Contains source and data files for tests Contains tools for creating custom dynamically linkable libraries Contains shell scripts to set environmental variables in the user shell Contains an Eclipse IDE plug in with Intel MKL Reference Manual in WebHelp format See mk1_documentation htm for comments Contains a utility for reporting the package ID and license key information to Intel Premier Support Layered Model Concept The Intel Math Kernel Library has long had a structure that is not visible to the user except for the 32 bit version for Windows OS In that case two interface libraries are provided and you must select one of them at run time Both libraries are relatively small and independent of a particular processor based on IA 32 architecture and the major part of Intel MKL which is independent of the interface is not duplicated Thus by means of the interface libraries Intel MKL supports two different compiler interface standards without 3 2 greatly increasing the library size Starting with release 10 0 Intel MKL is extending this approach to support a richer set of circumstances compilers and threading Intel Math Kernel Lib
39. source code is distributed in this case To use the Intel MKL FFT Trigonometric Transform or Poisson Laplace and Helmholtz Solver routines link in the math support system library by adding 1m to the link line In products for Linux it is necessary to link the pthread library by adding lpthread The pthread library is native to Linux and libguide makes use of this library to support multi threading Any time libguide is required add lpthread at the end of your link line link order is important 5 7 5 Intel Math Kernel Library User s Guide See also Linking Examples Linking with Interface Libraries Linking with Threading Libraries Linking with Computational Libraries Linking Examples 1 The section provides specific examples of linking supporting Intel compilers on systems based on IA 32 and Intel 64 architectures In these examples lt MKL path gt and lt MKL include gt placeholders are replaced with user defined environment variables SMKLPATH and SMKLINCLUDE respectively See also examples on linking with ScaLAPACK and Cluster FFT in chapter 9 For more linking examples see the Intel MKL support website at http www intel com support performancetools libraries mkl Linking on systems based on IA 32 architecture 1 Static linking of user s code myprog f and parallel Intel MKL ifort myprog f LSMKLPATH ISMKLINCLUDE Wl start group MKLPATH libmkl_intel a SMKLPATH libmk1l intel th
40. the i8 or DMKL_ILP64 option to the LP64 libraries may result in unpredictable consequences and erroneous output 3 7 3 Intel Math Kernel Library User s Guide Table 3 3 summarizes the compiler options Table 3 3 Compiler options for the ILP64 interface Fortran C or C Compiling for the ILP64 interface ifort i8 icc DMKL_ILP64 Compiling for the LP64 interface ifort ice Coding for ILP64 Although the 90 i and h files in the Intel MKL include directory were changed to meet requirements of the ILP64 interface the LP64 interface was not changed That is all function parameters that were 32 bit integers still retain the 32 bit integer type and all function parameters that were standard long integers still retain the standard long type So you do not need to change existing code if you are not using the ILP64 interface To migrate to ILP64 or write new code for ILP64 you need to use appropriate types for parameters of the Intel MKL functions and subroutines For the parameters that must be 64 bit integers in ILP64 you are encouraged to use the universal integer types namely e INTEGER for Fortran e MKL_INT for C C e MKL LONG for the parameters of the C C FFT interface This way you make your code universal for both ILP64 and LP64 interfaces You may alternatively use other 64 bit types for the integer parameters that must be 64 bit in ILP64 For example with Intel compilers you may use types
41. the editor window For a software library Eclipse looks up possible completion of a function name or a named constant in a header file So to enable Code Content Assist with Intel MKL 1 Specify paths to include files as explained in Configuring the Eclipse IDE CDT to Link with Intel MKL in chapter 4 or follow the recommendations given in the respective User s Guide on configuring the Eclipse CDT shell to link with the library 2 Add the include statement with the needed header file lt header h gt to your source file 10 6 Getting Assistance for Programming in the Eclipse IDE 1 0 To be prompted for the completion of the name of an Intel MKL function or a named constant in the code window 1 Type the first few characters of the name in your code line 2 Press Ctrl SPACEBAR The prompt info appears in a popup Figure 10 6 Code Content Assist Java test c Eclipse SDK az defftidc double r jnt m int sign double ws ve o cefft2d double r jnt mint n void o defft2de double r jnt m nt void Customizing Code Content Assist To customize Code Content Assist set Content Assist preferences 1 Click Window gt Preferences 2 Expand C C and click C C Editor 3 Click the Content Assist tab 4 Do the following see Figure 10 7 To change the scope from which Content Assist retrieves its proposals select either Search current file and included files or Search current project by checking the
42. this link to see the results L Pr 38 NCE TE aa O amp rors O warnings Onfos L r Description amp Go To Al Topics Related Topics W Bookmarks L3 Index a 10 3 1 0 Intel Math Kernel Library User s Guide Using Context Sensitive Help in the Eclipse IDE CDT You can get context sensitive help in the Eclipse CDT editor using Infopop windows and F1 Help Infopop window Infopop window is a popup description of a C function To obtain the description of an Intel MKL function whose name is typed in the editor place the cursor over the function name Figure 10 3 Infopop Window with an Intel MKL function description est_3 cpp Eclipse Platform Search Project Run Window Help ri y C a a a a S a a e lt s 1 fa Sel mil e CPP MKLGetVersli Name d_backward_trig_transform Protoype void d_backward_trig_transform double f DFTI_ ipar double dpar int stat Description This function computes the backward Trigonometric Transform d_init_trig_transform routine and passed to d_backward_ size of the problem n which determines sizes of the array para routine with the ipar array and defined in the previously called Ma ww ele wee dees F1 Help F1 Help basically displays the list of relevant documentation topics for a keyword To get Fi Help for an Intel MKL function whose name is typed in the editor window 10 4 Getting Assistance
43. using the environment variable set MKL_ DISABLE FAST MM to any value which will cause memory to be allocated and freed from call to call Disabling this feature will negatively impact performance of routines such as the level 3 BLAS especially for small problem sizes Using one of these methods to release memory will not necessarily stop programs from reporting memory leaks and in fact may increase the number of such reports in case you make multiple calls to the library thereby requiring new allocations with each call Memory not released by one of the methods described will be released by the system when the program ends Redefining Memory Functions 6 18 Starting with MKL 9 0 you can replace memory functions that the library uses by default with your own ones The memory renaming feature enables this replacement Memory renaming In general if users try to employ their own memory management functions instead of similar system functions malloc free calloc and realloc actually the memory gets managed by two independent memory management packages which may cause memory issues To prevent such issues the memory renaming feature was introduced in certain Intel libraries and in particular in Intel MKL This feature enables users to redefine memory management functions Redefining is possible because Intel MKL actually uses pointers to memory functions i_malloc i free i_calloc i_realloc rather than the functions themselv
44. 0 and MPICH2 LP64 version of BLACS routines supporting Intel MPI 2 0 and 3 0 and MPICH2 A soft link to lib 64 libmkl_blacs_intelmpi_ilp64 a A soft link to lib 64 libmkl_blacs_intelmpi_lp64 a LP64 version of BLACS routines supporting the following MPICH versions Myricom MPICH version 1 2 5 10 e ANL MPICH version 1 2 5 2 ILP64 version of BLACS routines supporting OpenMPI LP64 version of BLACS routines supporting OpenMPI ILP64 version of BLACS routines supporting SGI MPT Intel Math Kernel Library Structure 3 Detailed directory structure continued Directory file libmkl_blacs_ sgimpt_lp6 4 a Dynamic Libraries Interface layer libmk1l_gf_ilp64 so libmkl_ gf _1p64 so libmk1l_intel_ilp64 so libmkl_intel_1p64 so libmk1l_intel_sp2dp so Threading layer libmk1_gnu_thread so libmkl_intel_ thread so libmk1_sequential so Computational layer libmkl so libmkl_core so libmk1l_i2p so libmkl_lapack so libmk1_scalapack_ ilp64 so libmkl_scalapack_ 1p64 so libmkl_vml_i2p so RTL layer libguide so libiomp5 so libmkl_blacs_ intelmpi_ilp6 4 so Contents LP64 version of BLACS routines supporting SGI MPT ILP64 interface library for GNU Fortran compiler LP64 interface library for GNU Fortran compiler ILP64 interface library for Intel compiler LP64 interface library for Intel compiler SP2DP interface library for Intel compiler Parallel drivers library supporting GNU compiler Paral
45. 4 interface library for GNU Fortran compiler LP64 interface library for GNU Fortran compiler ILP64 interface library for Intel compiler LP64 interface library for Intel compiler SP2DP interface library for Intel compiler Parallel drivers library supporting GNU compiler Parallel drivers library supporting Intel compiler Parallel drivers library supporting PGI compiler Sequential drivers library 3 13 3 Intel Math Kernel Library User s Guide Table 3 6 Detailed directory structure continued Directory file Computational layer libmkl_ cdft a libmkl_cdft_core a libmkl_core a libmkl_em6 4t a libmkl_lapack a libmk1l_scalapack a libmk1_scalapack_ ilp64 a libmk1l_scalapack_ lp6 4 a libmkl_solver a libmkl_solver_ ilp64 a libmkl_solver_ilp64_ sequential a libmk1l_solver_1p64 a libmkl_solver_1p64_ sequential a Contents Dummy library Contains a reference to lib em 4t libmkl_ cdft_core a Cluster version of FFTs Kernel library for Intel 64 architecture Dummy library Contains references to Intel MKL libraries lib eme4t libmkl intel 1p64 a lib em 4t libmkl_ intel thread a and lib em 4t libmkl_core a Dummy library Contains references to Intel MKL libraries lib em 4t libmkl intel lp64 a lib em 4t libmkl intel thread a and lib em 4t libmkl_ core a Dummy library Contains a reference to lib em 4t libmkl scalapack_1lp 64 a ScaLAPACK routines library supporting ILP64 int
46. ALLOC ALLOC SIZE 128 ALPHA 1 1 BETA 1 2 DO I 1 N DO J 1 N A I d I d B I J I j C I J 0 0 END DO END DO CALL DGEMM N N N N N ALPHA A N B N BETA C N print Row A CY DO i 1 10 write 14 F20 8 F20 8 I A 1 1I C 1 1 END DO CALL OMP_SET_NUM_THREADS 1 DO I 1 N DO J 1 N A I d I d B I J I j C I J 0 0 END DO END DO CALL DGEMM N N N N N ALPHA A N B N BETA C N print Row A Cc DO i 1 10 write 14 F20 8 F20 8 I A 1 1I C 1 1 END DO 6 7 6 Intel Math Kernel Library User s Guide Example 6 1 Changing the number of processors for threading continued CALL OMP_SET_NUM_THREADS 2 DO I 1 N DO J 1 N A I d I d B I J I j C I J 0 0 END DO END DO CALL DGEMM N N N N N ALPHA A N B N BETA C N print Row A Cc DO i 1 10 write 14 F20 8 F20 8 I A 1 1I C 1 1 END DO STOP END Using Additional Threading Control Intel MKL 10 0 introduces new optional threading controls that is the new environment variables and service functions They behave similar to their OpenMP equivalents but take precedence over them By using these controls along with OpenMP variables you can thread the part of the application that does not call Intel MKL and the library independently from each other These controls enable you to specify the number of threads for Intel MKL independently of the OpenMP settings Although Intel
47. Ass St ssssssssssssssresrrrrrrrrrrrsrrssrrrens Figure 10 7 Customizing Code Content ASSist sssssssssrreserreresres Overview Intel Math Kernel Library Intel MKL offers highly optimized thread safe math routines for science engineering and financial applications that require maximum performance Technical Support Intel provides a support web site which contains a rich repository of self help information including getting started tips known product issues product errata license information user forums and more Visit the Intel MKL support website at http www intel com software products support About This Document Intel MKL Reference Manual provides reference information on routine functionalities parameter descriptions interfaces calling syntaxes and return values The Intel MKL User s Guide contains usage information which goes into more technical detail than reference information Usage information explains more specifics of routine calls especially in mixed language programming and additionally covers the organization configuration performance and accuracy of Intel MKL This guide focuses on the usage information needed to call Intel MKL routines from user s applications running on the Linux OS The document describes features particular to Linux usage of Intel MKL along with OS independent features This guide contains usage information for all Intel MKL function domains listed in
48. BLAS95 libmkl_lapack95 a Contains Fortran 95 wrappers for LAPACK LAPACK95 libfftw2xc_intel a Contains interfaces for FFTW version 2 x C interface for Intel compiler to call Intel MKL FFTs libfftw2xc_gnu a Contains interfaces for FFTW version 2 x C interface for GNU compiler to call Intel MKL FFTs 7 1 7 Intel Math Kernel Library User s Guide Table 7 1 Interface libraries and modules continued File name Comment libfftw2xf_intel a Contains interfaces for FFTW version 2 x Fortran interface for Intel compiler to call Intel MKL FFTs libfftw2xf_gnu a Contains interfaces for FFTW version 2 x Fortran interface for GNU compiler to call Intel MKL FFTs libfftw3xc_intel a Contains interfaces for FFTW version 3 x C interface for Intel compiler to call Intel MKL FFTs libfftw3xc_gnu a Contains interfaces for FFTW version 3 x C interface for GNU compiler to call Intel MKL FFTs libfftw3xf_intel a Contains interfaces for FFTW version 3 x Fortran interface for Intel compiler to call Intel MKL FFTs libfftw3xf_gnu a Contains interfaces for FFTW version 3 x Fortran interface for GNU compiler to call Intel MKL FFTs libfftw2x_cdft_SINGLE a Contains single precision interfaces for MPI FFTW version 2 x C interface to call Intel MKL cluster FFTs libfftw2x_cdft_DOUBLE a Contains double precision interfaces for MPI FFTW version 2 x C interface to call Intel MKL cluster FFTs mk195_blas mod Contains Fortran 95 inter
49. CK Benchmark ccccceeeeeeees 11 5 List of Examples Example 6 1 Changing the number of processors for threading 6 5 Example 6 2 Setting the number of threads to ONE eeeeee eee e eae 6 9 Example 6 3 Setting an affinity mask by operating system means using AN Intel compiler ssssssssersssrrrrsrrrrresrerrrsrerrresrrrrnserren 6 16 Example 6 4 Redefining memory fUNCTtIONS sssssssssssrnrrsrrrrrrsrrrrs 6 19 Example 7 1 Calling a complex BLAS Level 1 function from C 7 8 Example 7 2 Calling a complex BLAS Level 1 function from C 7 9 Example 7 3 Using CBLAS interface instead of calling BLAS directly lino a n SE AN AE E T 7 10 Example 8 1 Aligning addresses at 16 byte boundaries 0008 8 2 viii Intel Math Kernel Library User s Guide List of Figures Figure 5 1 Linking with Layered Intel Math Kernel Library Figure 7 1 Column major order vs row major Order eeeeee es Figure 10 1 Intel Math Kernel Library Help in the Eclipse IDE Figure 10 2 Hits to the Intel Web Site in the Eclipse IDE Help a acces tain cacs eevee wt ts cae hh teak Saad E dans Masham ese Reua les Figure 10 3 Infopop Window with an Intel MKL function description seere enre iaai aene e AAEE AEEA SEE EUAN Figure 10 4 F1 Help in the Eclipse IDE sssssssssssssssrrrrrrrrrsrrssrssnn Figure 10 5 F1 Help Search in the Eclipse IDE CDT cc cea Figure 10 6 Code Content
50. Different systems may have different number of processors or amount of memory and require new input files The extended help can be used for insight into proper ways to change the sample input files Each input file requires at least the following amount of memory lininput_itanium 16GB lininput_xeon32 2 GB lininput_xeon64 16 GB If the system has less memory than the above sample data inputs require you may have to edit or create your own data input files as directed in the extended help Each sample script in particular uses the OMP_NUM_THREADS environment variable to set the number of processors it is targeting To optimize performance on a different number of physical processors change that line appropriately If you run the Intel Optimized LINPACK Benchmark without setting the number of threads it will default to the number of cores according to the OS You can find the settings for this environment variable in the runme_ sample scripts If the settings do not already match the situation for your machine edit the script Known Limitations The following limitations are known for the Intel Optimized LINPACK Benchmark for Linux e Intel Optimized LINPACK Benchmark is threaded to effectively use multiple processors So in multi processor systems best performance will be obtained with Hyper Threading technology turned off which ensures that the operating system assigns threads to physical processors only e Ifan incomplete
51. EADS Intel MKL 10 0 has also introduced other mechanisms to set the number of threads such as MKL NUM THREADS Or MKL_ DOMAIN NUM_ THREADS see section Using Additional Threading Control in chapter 6 Make certain that the relevant environment variable has the same and correct value on all the nodes Intel MKL version 10 0 and higher also no longer sets the default number of threads to 1 but depends on the compiler to set the default number For the threading layer based on the Intel compiler libmk1_intel_thread a this Working with Intel Math Kernel Library Cluster Software 9 value is the number of CPUs according to the OS Be cautious to avoid over prescribing the number of threads which may occur for instance when the number of MPI ranks per node and the number of threads per node are both greater than one The best way to set for example the environment variable OMP_NUM_THREADS is in the login environment Remember that mpirun starts a fresh default shell on all of the nodes and so changing this value on the head node and then doing the run which works on an SMP system will not effectively change the variable as far as your program is concerned In bashrc you could add a line at the top which looks like this OMP NUM _THREADS 1 export OMP_NUM_THREADS It is possible to run multiple CPUs per node using MPICH but the MPICH must be built to allow it Be aware that certain MPICH applications may not work perfec
52. IC FALSE does not ensure that Intel MKL will use the number of threads that you request mainly because the library may have no choice on this number for such reasons as system resources Moreover the library may examine the problem and pick a different number of threads than the value suggested For example if you attempt to do a size 1 matrix matrix multiply across 8 threads the library may instead choose to use only one thread because it is impractical to use 8 threads in this event Note also that if Intel MKL is called in a parallel region it will use only one thread by default If you want the library to use nested parallelism and the thread within a parallel region is compiled with the same OpenMP compiler as Intel MKL is using you may experiment with setting MKL_ DYNAMIC to FALSE and manually increasing the number of threads In general you should set MKL_DYNAMIC to FALSE only under circumstances that Intel MKL is unable to detect for example when nested parallelism is desired where the library is called already from a parallel section MKL DYNAMIC being TRUE in particular provides for optimal choice of the number of threads in the following cases e If the requested number of threads exceeds the number of physical cores perhaps because of hyper threading and MKL_DYNAMIC is not changed from its default value of TRUE Intel MKL will scale down the number of threads to the number of physical cores Managing Performance and Mem
53. IPF_fma unroll w tpp2 DASYOUGO2 Gives detailed single node DGEMM performance information It captures all DGEMM calls if you use Fortran BLAS and records their data Because of this the routine has a marginal intrusive overhead Unlike DASYOUGO which is quite non intrusive DASYOUGO2 is interrupting every DGEMM call to monitor its performance You should beware of this overhead although for big problems it is for sure less than 1 10th of a percent Here is a sample ASYOUGO2 output the first 3 non intrusive numbers can be found in ASYOUGO and ENDEARLY so it suffices to describe these numbers here Col 001280 Fract 0 050 Mflops 42454 99 DT 9 5 DF 34 1 DMF 38322 78 The problem size was N 16000 with a blocksize of 128 After 10 blocks that is 1280 columns an output was sent to the screen Here the fraction of columns completed is 1280 16000 0 08 Only about 20 outputs are printed at various places through the matrix decomposition fractions 0 005 0 010 0 015 0 02 0 025 0 03 0 035 0 04 0 045 0 05 0 055 0 06 0 065 0 07 0 075 0 080 0 085 0 09 0 095 10 195 295 395 895 However this problem size is so small and the block size so big by comparison that as soon as it printed the value for 0 045 it was already through 0 08 fraction of the columns On a really big problem the fractional LINPACK and MP LINPACK Benchmarks 1 1 number will be more accurate It never prints more than the 46 numbers above So small
54. Intel Math Kernel Library for the Linux OS User s Guide August 2008 Document Number 314774 007US World Wide Web http developerintel com intel Version Version Information Date 001 Original issue Documents Intel Math Kernel Library Intel MKL 9 0 gold release September 2006 002 Documents Intel MKL 9 1 beta release Getting Started LINPACK and MP LINPACK Benchmarks chapters and Support for Third Party and Removed Interfaces appendix added Existing chapters extended Document restruc tured List of examples added January 2007 003 Documents Intel MKL 9 1 gold release Existing chapters extended Docu ment restructured More aspects of ILP64 interface discussed Section Config uring the Eclipse IDE CDT to Link with Intel MKL added to chapter 3 Cluster content is organized into one separate chapter 9 Working with Intel Math Kernel Library Cluster Software and restructured appropriate links added June 2007 004 Documents Intel MKL 10 0 Beta release Layered design model has been described in chapter 3 and the content of the entire book adjusted to the model Automation of setting environment variables at startup has been described in chapter 4 New Intel MKL threading controls have been described in chapter 6 The User s Guide for Intel MKL merged with the one for Intel MKL Cluster Edition to reflect consolidation of the respectiv
55. Intel MKL which did not use layered libraries Dummy libraries do not contain any functionality but only dependencies on a set of layered libraries Placed in a link line dummy libraries enable omitting dependent layered libraries which will be linked automatically Dummy libraries contain dependency on the following layered libraries default principle e Interface Intel LP64 e Threading Intel compiled e Computational the computation library So if you employ the above interface and use OpenMP threading provided by the Intel compiler you may not change your link lines Accessing the Intel Math Kernel Library Documentation This section details the contents of the Intel MKL documentation directory and explains how to access man pages for the library Contents of the Documentation Directory Table 3 7 shows the contents of the doc subdirectory in the Intel MKL installation directory Table 3 7 Contents of the doc directory File name Comment Install txt Intel MKL Installation Guide mk1l_documentation htm Index of the Intel MKL documentation 3 20 Intel Math Kernel Library Structure 3 Table 3 7 Contents of the doc directory continued File name Comment mk1lEULA txt Intel MKL license mklman pdf Intel MKL Reference Manual mklman90_j pdf Intel MKL Reference Manual in Japanese mklsupport txt Information on package number for customer support reference Readme txt Intel MKL Initial User Information Rel
56. Library Cluster Software Discusses usage of ScaLAPACK and Cluster FFTs in particular describes linking of your application with these function domains including C and Fortran specific linking examples gives information on the supported MPI Getting Assistance for Programming in the Eclipse IDE Discusses Intel MKL features that software engineers can benefit from when working in the Eclipse IDE LINPACK and MP LINPACK Benchmarks Describes the Intel Optimized LINPACK Benchmark for Linux and Intel Optimized MP LINPACK Benchmark for Clusters Intel Math Kernel Library Language Interfaces Support Summarizes information on language interfaces that Intel MKL provides for each function domain including the respective header files Support for Third Party Interfaces Describes in brief some interfaces that Intel MKL supports The document also includes an Index Term and Notational Conventions The following term is used in the manual in reference to the operating system Linux OS This term refers to information that is valid on all supported Linux operating systems The following notation is used in reference to Intel MKL directories 1 3 1 Intel Math Kernel Library User s Guide lt mk1_directory gt The main directory to which Intel MKL is installed Should be substituted with the specific pathname in the configuring linikng and building instructions Table 1 1 Notational conventions Italic
57. MKL set your environment variables using the script corresponding to your architecture see Setting Environment Variables Identify all Intel MKL function domains that problems you are solving require BLAS Sparse BLAS LAPACK PBLAS ScaLAPACK Sparse Solver routines Vector Mathematical Library functions Vector Statistical Library functions Fourier Transform functions FFT Cluster FFT PBLAS Trigonometric Transform routines Poisson Laplace and Helmholtz Solver routines Optimization Trust Region Solver routines GMP arithmetic functions Reason The function domain you intend to use narrows the search in the Reference Manual for specific routines you need Additionally the link line that you use to link your application with Intel MKL cluster software depends on the function domains you intend to employ see Working with Intel Math Kernel Library Cluster Software Coding tips may also depend on the function domain see Tips and Techniques to Improve Performance 2 3 2 Intel Math Kernel Library User s Guide Table 2 1 What you need to know before you get started continued Programming language Threading model Linking model MPI used Though Intel MKL provides support for both Fortran and C C programming not all the function domains support a particular language environment for example C C or Fortran90 95 Identify the language interfaces that your function domains support see Intel Math
58. MPICH2 libmkl_blacs_ A soft link to Lib 32 libmk1l_blacs_intelmpi a intelmpi20 a libmkl_blacs_ BLACS routines supporting OpenMPI openmpi a 3 11 3 Intel Math Kernel Library User s Guide Detailed directory structure continued Directory file Dynamic Libraries Interface layer libmk1_gf so libmkl_intel so Threading layer libmk1l_gnu_thread so libmk1l_intel_ thread so libmk1_pgi_thread so libmkl_ sequential so Computational layer libmk1l so libmkl_core so libmkl_ def so libmkl_lapack so libmk1_p4 so libmkl_p4m so libmkl_p4p so Contents Interface library for GNU Fortran compiler Interface library for Intel compiler Parallel drivers library supporting GNU compiler Parallel drivers library supporting Intel compiler Parallel drivers library supporting PGI compiler Sequential drivers library Dummy library Contains references to Intel MKL libraries 1ib 32 libmk1_ intel so 1ib 32 libmk1l_ intel thread so and 1ib 32 libmk1l_core so Library dispatcher for dynamic load of processor specific kernel library Default kernel library Intel Pentium Pentium Pro and Pentium II processors LAPACK and DSS PARDISO routines and drivers Pentium 4 processor kernel library Kernel library for processors based on the Intel Core microarchitecture except Intel Core Duo and Intel Core Solo processors for which mkl_p4p so is intended Kernel library for Intel Pent
59. NCLUDING LIABILITY OR WARRANTIES RELATING TO FITNESS FOR A PARTICULAR PURPOSE MERCHANTABILITY OR INFRINGEMENT OF ANY PATENT COPYRIGHT OR OTHER INTELLECTUAL PROPERTY RIGHT UNLESS OTHERWISE AGREED IN WRITING BY INTEL THE INTEL PRODUCTS ARE NOT DESIGNED NOR INTENDED FOR ANY APPLICATION IN WHICH THE FAILURE OF THE INTEL PRODUCT COULD CREATE A SITUATION WHERE PERSONAL INJURY OR DEATH MAY OCCUR Intel may make changes to specifications and product descriptions at any time without notice Designers must not rely on the absence or characteristics of any features or instructions marked reserved or undefined Intel reserves these for future definition and shall have no responsibility whatsoever for conflicts or incompatibilities arising from future changes to them The information here is subject to change without notice Do not finalize a design with this information The products described in this document may contain design defects or errors known as errata which may cause the product to deviate from published specifications Current characterized errata are available on request Contact your local Intel sales office or your distributor to obtain the latest specifications and before placing your product order Copies of documents which have an order number and are referenced in this document or other Intel literature may be obtained by calling 1 800 548 4725 or by visiting Intel s Web Site Intel processor numbers are not a measure of performance
60. NI Specification versions 1 1 and 5 0 and should work with virtually every modern implementation of Java The examples and the Java part of the wrappers are written for the Java language described in The Java Language Specification First Edition and extended with the feature of inner classes this refers to late 1990s This level of language version is supported by all versions of Sun JDK developer toolkit and compatible implementations starting from version 1 1 5 that is by all modern versions of Java Language specific Usage Options 7 The level of C language is Standard C that is C89 with additional assumptions about integer and floating point data types required by the Intel MKL interfaces and the JNI header files That is the native float and double data types are required to be the same as JNI jfloat and jdouble data types respectively and the native int is required to be 4 byte long Running the examples The Java examples support all the C and C compilers that the Intel MKL does The makefile intended to run the examples also needs the make utility which is typically provided with the Linux OS To run Java examples the JDK developer toolkit is required for compiling and running Java code A Java implementation must be installed on the computer or available via the network You may download the JDK from the vendor website The examples should work for all versions of JDK However they were tested only with th
61. P64 command line option to the compiler to enforce MKL_ INT and MKL_LONG being 64 bit And vice versa if your code is compiled with DMKL_ILP64 option you can bind it only with the ILP64 interface because the LP64 binary interface requires MKL_INT to be 32 bit and MKL_LONG to be the standard long type Note that certain MKL functions have parameters explicitly declared as int or int Such integers are always 32 bit regardless of whether the code is compiled with the DMKL_ILP64 option Table 3 2 summarizes how the Intel MKL ILP64 concept is implemented Table 3 2 Intel MKL ILP64 concept Fortran C or C The same include directory for lt mk1 directory gt include ILP64 and LP64 interfaces Type used for parameters that INTEGER 4 int are always 32 bit Type used for parameters that INTEGER MKL_INT are 64 bit integers for the ILP64 interface and 32 bit integers for LP64 Type used for all integer INTEGER MKL_ LONG parameters of the FFT functions Command line option to i8 DMKL_ILP64 control compiling for ILP64 Compiling for ILP64 The same copy of the Intel MKL include directory is used for both ILP64 and LP64 interfaces So the compilation for the ILP64 interface looks like this Fortran ifort i8 I lt mkl drectory gt include Cor C icc DMKL_ILP64 I lt mkl directory gt include To compile for the LP64 interface just omit the i8 or DMKL _ILP64 option Notice that linking of the application compiled with
62. Started Describes post installation steps and gives information needed to start using Intel MKL after its installation Intel Math Kernel Library Structure Discusses the structure of the Intel MKL directory after installation at different levels of detail as well as the library versions and parts Configuring Your Development Environment Explains how to configure Intel MKL with your development environment Linking Your Application with Intel Math Kernel Library Compares static and dynamic linking models describes the general link line syntax to be used for linking with Intel MKL libraries Chapter 6 Chapter 7 Chapter 8 Chapter 9 Chapter 10 Chapter 11 Appendix A Appendix B Overview 1 explains which libraries should be linked with your application for your particular platform and function domain discusses how to build custom dynamic libraries Managing Performance and Memory Discusses Intel MKL threading shows coding techniques and gives hardware configuration tips for improving performance of the library explains features of the Intel MKL memory management and in particular shows how to replace memory functions that the library uses by default with your own ones Language specific Usage Options Discusses mixed language programming and the use of language specific interfaces Coding Tips Presents coding tips that may be helpful to your specific needs Working with Intel Math Kernel
63. This error handler will be added to the library and used instead of the Intel MKL error handler xerbla By default this parameter is not specified and the native Intel MKL xerbla is used Note that if the user s error handler has the same name as the Intel MKL handler the name of the user s handler must be upper case that is XERBLA o All parameters are not mandatory In the simplest case the command line could be make ia32 and the values of the remaining parameters will default As a result mk1l_custom so library for processors using IA 32 architecture will be created the functions list will be taken from the user_list file and the native Intel MKL error handler xerbla will be used Another example for a more complex case is as follows make ia32 export my_ func _list txt name mkl_small xerbla my_xerbla o In this case mk1_small so library for processors using IA 32 architecture will be created the functions list will be taken from my_func_list txt file and user s error handler my _xerbla o will be used The process is similar for processors using Intel 64 or IA 64 architecture Specifying List of Functions 5 16 Entry points in functions list file should be adjusted to interface For example Fortran functions get an underscore character _ as a suffix when added to the library dgemm_ ddot_ dgetrf_ If selected functions have several processor specific versions they all will be included into the custom library an
64. _num_threads When choosing the appropriate technique take into account the following rules e If you employ the OpenMP techniques OMP_NUM_THREADS and omp_set_num threads only which was the case with earlier Intel MKL versions the library will still respond to them e The Intel MKL threading controls take precedence over the OpenMP techniques e A function call takes precedence over any environment variables The exception is the OpenMP subroutine omp_set_num_threads which does not have precedence over Intel MKL environment variables such as MKL_ NUM THREADS e The environment variables cannot be used to change run time behavior in the course of the run as they are read only once Avoiding Conflicts in the Execution Environment There are situations in which conflicts can exist in the execution environment that make the use of threads in Intel MKL problematic They are listed here with recommendations for dealing with them First a brief discussion of why the problem exists is appropriate If the user threads the program using OpenMP directives and compiles the program with Intel compilers Intel MKL and the program will both use the same threading library Intel MKL tries to determine if it is in a parallel region in the program and if it is it does not spread its operations over multiple threads unless the user specifically requests Intel MKL to do so via the MKL_ DYNAMIC functionality see Using Additional Threading Contr
65. ant Intel MKL to use a given number of threads and do not want users of your application to change this via environment variables set this number of threads by a call to mkl_set_num_threads which will have full precedence over any environment variables being set The example below illustrates the use of the Intel MKL function mkl_set_num threads to mimic the Intel MKL 9 x default behavior that is running on one thread Example 6 2 Setting the number of threads to one RRKKKR C language x include lt omp h gt include lt mkl h gt mkl_set_num threads 1 6 9 6 Intel Math Kernel Library User s Guide Example 6 2 Setting the number of threads to one 6 10 k k k Fortran language x call mkl_set_num_threads 1 The section further expands on the Intel MKL environment variables for threading control See the Intel MKL Reference Manual for the detailed description of the threading control functions their parameters calling syntax and more code examples MKL_DYNAMIC The default value of MKL DYNAMIC is TRUE regardless of OMP_DYNAMIC whose default value may be FALSE MKL DYNAMIC being TRUE means that Intel MKL will always try to pick what it considers the best number of threads up to the maximum specified by the user MKL DYNAMIC being FALSE means that Intel MKL will normally try not to deviate from the number of threads the user requested Note however that setting MKL_DYNAM
66. appens in LU decomposition The ASYOUGO performance estimate is usually an overestimate because LU slows down as it goes but it gets more accurate as the problem proceeds The greater the lookahead step the less accurate the first number may be ASYOUGO tries to estimate where one is in the LU decomposition that MP LINPACK performs and this is always an overestimate as compared to ASYOUGO2 which measures actually achieved DGEMM performance Note that the ASYOUGO output is a subset of the information that ASyouGO2 provides So refer to the description of the DASYOUGO2 option below for the details of the output DENDEARLY Terminates the problem after a few steps so that you can set up 10 or 20 HPL runs without monitoring them see how they all do and then only run the fastest ones to completion DENDEARLY assumes DASYOUGO You do not need to define both although it doesn t hurt Because the problem terminates early it is recommended setting the threshold parameter in HPL dat to a negative number when testing ENDEARLY There is no point in doing a residual check if the problem ended early It also sometimes gives a better picture to compile with DASYOUGO2 when using DENDEARLY You need to know the specifics of DENDEARLY 1 1 Intel Math Kernel Library User s Guide 11 10 DENDEARLY stops the problem after a few iterations of DGEMM on the blocksize the bigger the blocksize the further it gets It prints only 5 or 6 update
67. aspect Limitations Note that not all components support the ILP64 feature Table 3 5 shows which function domains support ILP64 interface Table 3 5 ILP64 support in Intel MKL Function domain Support for ILP64 3 9 3 Intel Math Kernel Library User s Guide Table 3 5 ILP64 support in Intel MKL continued Function domain Support for ILP64 GMP No Intel MKL Versions Intel MKL for the Linux OS distinguishes the following versions e JA 32 architecture located in the 1ib 32 directory e Intel 64 architecture located in the 1ib em64t directory e JA 64 architecture located in the 1ib 64 directory See a detailed structure of these directories in Table 3 7 Directory Structure in Detail The information in the table below shows a detailed structure of the architecture specific directories of the library For the contents of the doc directory see Contents of the Documentation Directory For the contents of subdirectories in the benchmarks directory see LINPACK and MP LINPACK Benchmarks Table 3 6 Detailed directory structure Directory file Contents lib 321 Libraries for IA 32 architecture Static Libraries Interface layer libmkl_gf a Interface library for GNU Fortran compiler libmkl_intel a Interface library for Intel compiler Threading layer libmkl_gnu_thread a Parallel drivers library supporting GNU compiler libmk1l_intel_thread a Parallel drivers library supporting Intel compiler I
68. atibility library 1ibiomp is an extension of libguide that provides support for one additional threading compiler on Linux gnu That is a 5 11 5 Intel Math Kernel Library User s Guide program threaded with a gnu compiler can safely be linked with Intel MKL and libiomp and execute efficiently and effectively So you are encouraged to use libiomp rather than libguide Table 5 3 shows different scenarios depending on the threading compiler used and the possibilities for each scenario to choose the Threading layer and RTL layer when using the current version of Intel MKL static cases only Table 5 3 Selecting the Threading Layer Application RTL Layer Compiler Threaded Threading Layer Recommended Comment Intel Does not libmkl_intel_thread a libiomp5 so or matter libguide so PGI Yes libmkl_pgi_thread a or PGI supplied Use of libmk1_ libmkl_ sequential a sequential a removes threading from Intel MKL calls PGI No libmkl_intel_thread a libiomp5 so or libguide so PGI No libmkl_pgi_thread a PGI supplied PGI No libmk1l_sequential a None gnu Yes libmkl gnu _thread a libiomp5 so or libiomp5 offers GNU OpenMP superior scaling run time library performance gnu Yes libmkl_sequential a None gnu No libmkl_intel_thread a libiomp5 so or libguide so other Yes libmk1_sequential a None other No libmkl_intel_thread a libiomp5 so or libguide so Linking with Computational Libraries Typically if you are using the
69. c linking of user s code myprog f parallel version of an iterative sparse solver and parallel Intel MKL ifort myprog f LSMKLPATH ISMKLINCLUDE 1mkl1_solver Wl start group MKLPATH libmkl_intel a SMKLPATH libmk1l_ intel thread a MKLPATH libmkl_core a W1 end group liomp5 lpthread Static linking of user s code myprog f sequential version of an iterative sparse solver and sequential Intel MKL ifort myprog f LSMKLPATH ISMKLINCLUDE lmkl_solver_ sequential Wl start group MKLPATH libmkl_intel a SMKLPATH libmkl_sequential a SMKLPATH libmkl_core a W1 end group lpthread Linking on systems based on Intel 64 architecture 1 Static linking of user s code myprog f and parallel Intel MKL supporting LP64 interface ifort myprog f LSMKLPATH ISMKLINCLUDE Wl start group MKLPATH libmkl_ intel _1p64 a SMKLPATH libmk1l intel thread a MKLPATH libmkl_core a W1 end group liomp5 lpthread Dynamic linking of user s code myprog f and parallel Intel MKL supporting LP64 interface ifort myprog f LSMKLPATH ISMKLINCLUDE lmk1l_intel_ 1p 6 4 lmkl_intel_ thread lmkl_core liomp5 lpthread Static linking of user s code myprog f and sequential version of Intel MKL supporting LP64 interface ifort myprog f LSMKLPATH ISMKLINCLUDE Wl start group MKLPATH libmkl_ intel _1p64 a SMKLPATH libmkl_sequential a S MKLPATH libmkl_core a W1l end group lpthread 5 9 5 Intel Math Kernel Library User s Guid
70. chapter 5 Note that with the Standard Make the above settings are needed for the CDT internal functionality only The compiler linker will not automatically pick up these settings and you will still have to specify them directly in the makefile For Managed Make projects you can specify settings for a particular build To do this 1 Goto the Tool Settings tab of the C C Build property page All the settings you need to specify are on this page Names of the particular settings depend upon the compiler integration and therefore are not given below 2 If the compiler integration supports include path options set the Intel MKL include path that is lt mkl_directory gt include 4 3 4 Intel Math Kernel Library User s Guide 3 If the compiler integration supports library path options set a path to the Intel MKL libraries depending upon the target architecture for example lt mk1 directory gt lib em 4t 4 Specify names of the Intel MKL libraries to link with your application for example mk1l_lapack and mk1_ia32 As compilers typically require library names rather than library file names the lib prefix and a extension are omitted To learn how to choose the libraries see Selecting Libraries to Link in chapter 5 Note on the Configuration file for Out of Core OOC DSS PARDISO Solver When using the configuration file for the OOC DSS PARDISO Solver mind that the maximum length of the OOC path in
71. chapter 7 focuses on general language specific programming options this one presents coding tips that may be helpful to meet certain specific needs Currently the only tip advising how to achieve numerical stability is given You can find other coding tips relevant to performance and memory management in chapter 6 Aligning Data for Numerical Stability If linear algebra routines LAPACK BLAS are applied to inputs that are bit for bit identical but the arrays are differently aligned or the computations are performed either on different platforms or with different numbers of threads the outputs may not be bit for bit identical though they will deviate within the appropriate error bounds The Intel MKL version may also affect numerical stability of the output as the routines may be implemented differently in different versions With a given Intel MKL version the outputs will be bit for bit identical provided all the following conditions are met e the outputs are obtained on the same platform e the inputs are bit for bit identical e the input arrays are aligned identically at 16 byte boundaries Unlike the first two conditions which are under users control the alignment of arrays by default is not For instance arrays dynamically allocated using malloc are aligned at 8 byte boundaries but not at 16 byte If you need the numerically stable output use MKL_malloc to get the properly aligned workspace 8 1 8 Intel Math Kernel
72. clude directory If you do not have administrator rights then do the following 1 Copy the entire directory lt mk1_directory gt interfaces blas95 or lt mk1_directory gt interfaces lapack95 into a user defined directory lt user_dir gt 2 Copy the corresponding file mkl_blas 90 or mkl_lapack f 90 from lt mk1_directory gt include into the user defined directory lt user_dir gt blas95 or lt user_dir gt lapack95 respectively 3 Run one of the above make commands in lt user_dir gt blas95 or lt user_dir gt lapack95 with an additional variable for instance make PLAT 1nx32 INTERFACE mkl1_blas f90 lib make PLAT 1nx32 INTERFACE mkl_lapack f 90 lib Now the required library and the mod file will be built and installed in the lt user_dir gt blas95 or lt user_dir gt lapack95 directory respectively By default the ifort compiler is assumed You may change it with an additional parameter of make FC lt compiler gt For instance make PLAT 1nx64 FC lt compiler gt lib There is also a way to use the interfaces without building the libraries To delete the library from the building directory use the following commands make PLAT 1nx32 clean for IA 32 architecture make PLAT 1nx32e clean for Intel 64 architecture make PLAT 1nx64 clean for IA 64 architecture Compiler dependent Functions and Fortran 90 Modules Compiler dependent functions arise whenever the compiler places into the object code function call
73. command syntax and examples link libraries and other linking topics like how to save disk space by creating a custom dynamic library see Linking Your Application with Intel Math Kernel Library Reason To link your application with ScaLAPACK and or Cluster FFT the libraries corresponding to your particular MPI should be included in the link line see Working with Intel Math Kernel Library Cluster Software 2 4 Intel Math Kernel Library Structure The chapter discusses the structure of the Intel Math Kernel Library Intel MKL including the Intel MKL directory structure as well as the library versions and parts Starting with version 10 1 Intel MKL employs a layered model to streamline the library structure reduce its size and add usage flexibility See also Layered Model Concept High level Directory Structure Table 3 1 shows a high level directory structure of Intel MKL after installation Table 3 1 High level directory structure Directory Comment lt mkl directory gt Intel MKL main directory lt mk1 directory gt benchmarks linpack Contains an OMP version of the LINPACK benchmark lt mk1 Contains an MPI version of the LINPACK benchmark directory gt benchmarks mp_linpack lt mkl directory gt doc Contains the Intel MKL documentation lt mkl directory gt examples Contains source code data and makefiles for examples lt mk1 directory gt include Contains INCLUDE files for library routin
74. computational layer Run time library RTL The last layer provides RTL support Not all RTLs are delivered with Intel MKL The only RTLs provided except those that are relevant to the Intel MKL cluster software are Intel compiler based RTLs Intel Compatibility OpenMP run time compiler library libiomp and Intel Legacy OpenMP run time compiler library libguide To thread using third party threading compilers you can employ Threading layer libraries or use the compatibility library in the appropriate circumstances Layers There are four essential parts of the library 1 Interface layer 2 Threading layer 3 Computational layer 4 Compiler Support RTL layer RTL layer for brevity Interface Layer This layer provides matching between compiled code of your application and the threading and or computational parts of the library This layer provides e An LP64 interface to Intel MKL ILP64 software see Support for ILP64 Programming for details e A means of dealing with the way different compilers return function values 3 3 3 Intel Math Kernel Library User s Guide e A means of mapping between single precision names and double precision names in applications that employ ILP64 such as Cray style naming Threading Layer This layer provides a way for threaded Intel MKL to share supported compiler threading The layer also provides for a sequential version of the library What was internal to the library previ
75. d managed by dispatcher Managing Performance and Memory This chapter features different ways to obtain the best performance with the Intel Math Kernel Library Intel MKL primarily it discusses threading see Using Intel MKL Parallelism then shows coding techniques and gives hardware configuration tips for improving performance The chapter also discusses the Intel MKL memory management and shows how to redefine memory functions that the library uses by default Using Intel MKL Parallelism Intel MKL is threaded in a number of places e Direct sparse solver e LAPACK Linear equations computational routines factorization getrf gbtrf potrf pptrf sytrf hetrf sptrf hptrf solving gbtrs gttrs pptrs pbtrs pttrs sytrs sptrs hptrs tptrs tbtrs Orthogonal factorization computational routines geqrf ormqr unmqr ormlg unmlg ormql unmgql ormrg unmrg Singular Value Decomposition computational routines gebrd bdsqr Symmetric Eigenvalue Problems computational routines sytrd hetrd sptrd hptrd steqr stedc Note that a number of other LAPACK routines which are based on threaded LAPACK or BLAS routines make effective use of parallelism gesv posv gels gesvd syev heev etc e All Level 3 BLAS Sparse BLAS matrix vector and matrix matrix multiply routines for the compressed sparse row and diagonal formats e VML 6 1 6 Intel Math K
76. d new_mask 08X was_mask 08X n tid unsigned int amp new_mask unsigned int amp was_mask Call Intel MKL FFT function Managing Performance and Memory 6 Example 6 3 Setting an affinity mask by operating system means using an Intel compiler continued return 0 See the Linux Programmer s Manual in man pages format for particulars of the sched_setaffinity function used in the above example Operating on Denormals If an Intel MKL function operates on denormals that is non zero numbers that are smaller than the smallest possible non zero number supported by a given floating point format or produces denormals during the computation for instance if the incoming data is too close to the underflow threshold you may experience considerable performance drop The CPU state may be set so that floating point operations on denormals invoke the exception handler that slows down the application To resolve the issue before compiling the main program turn on the ftz option if you are using the Intel compiler or any other compiler that can control this feature In this case denormals are treated as zeros at processor level and the exception handler is not invoked Note however that setting this option slightly impacts the accuracy Another way to bring the performance back to norm is proper scaling of the input data to avoid numbers near the underflow threshold FFT Optimized Radices You can i
77. data input file is given the binaries may either hang or fault See the sample data input files and or the extended help for insight into creating a correct data input file 1 1 Intel Math Kernel Library User s Guide Intel Optimized MP LINPACK Benchmark for Clusters The Intel Optimized MP LINPACK Benchmark for Clusters is based on modifications and additions to HPL 1 0a from Innovative Computing Laboratories ICL at the University of Tennessee Knoxville UTK The benchmark can be used for Top 500 runs see http www top500 org The use of the benchmark requires that you are already intimately familiar with the HPL distribution and usage This package adds some additional enhancements and bug fixes designed to make the HPL usage more convenient The benchmarks mp_linpack directory adds techniques to minimize search times frequently associated with long runs The Intel Optimized MP LINPACK Benchmark for Clusters is an implementation of the Massively Parallel MP LINPACK benchmark HPL code was used as a basis It solves a random dense real 8 system of linear equations Ax b measures the amount of time it takes to factor and solve the system converts that time into a performance rate and tests the results for accuracy You can solve any size N system of equations that fit into memory The benchmark uses full row pivoting to ensure the accuracy of the results This benchmark should not be used to report LINPACK performance on
78. de Content Assist in the Eclipse CDT The first three features are provided through the Intel MKL plugin for Eclipse Help See Table 3 1 in chapter 3 for the plugin location after installation To use the plugin place it into the plugins folder of your Eclipse directory The last feature is native to the Eclipse CDT Viewing the Intel MKL Reference Manual in the Eclipse IDE To view the Reference Manual in Eclipse 1 Select Help gt Help Contents from the menu 2 Inthe Help tab click Intel R Math Kernel Library Help under All Topics 3 In the Help tree that expands click Intel Math Kernel Library Reference Manual see Figure 10 1 The Intel MKL Help Index is also available in Eclipse and the Reference Manual is included in the Eclipse Help search 10 1 1 0 Intel Math Kernel Library User s Guide Figure 10 1 Intel Math Kernel Library Help in the Eclipse IDE workbench User Guide amp Java Development User Guide amp Platform Plug in Developer Guide E JDT Plug in Developer Guide PDE Guide E APT in Edipse S E Intel R Math Kernel Library Heb _ Intel Math Kernel Library Reference Manual 0 Legal Information E U Overview amp 4 BLAS and Sparse BLAS Routines amp 4 LAPACK Routines Linear Equations E U LAPACK Routines Least Squares and Eigenvalue Problems amp 4 LAPACK Auxiiary and Utility Routines E U ScaLAPACK Routines G4 ScaLAPACK Auxiliary and Utility Routines U4 Sparse Solv
79. diate way for you to find out e Whether Intel MKL is working on your system e How you should call the library e How to link the library The examples are grouped in subdirectories mainly by Intel MKL function domains and programming languages For instance subdirectory examples spblas contains Sparse BLAS examples and subdirectory examples vmlc contains VML examples in C Source code of the examples is in the next level sources subdirectory To compile build and run the examples use the makefile provided For information on how to use it refer to the makefile header See also Getting Started 2 High level Directory Structure in chapter 3 Before You Begin Using Intel MKL Before you get started using the Intel MKL sorting out a few important basic concepts will greatly help you get off to a good start The table below summarizes some important things to think of before you start using Intel MKL Table 2 1 Target platform Mathematical problem What you need to know before you get started Identify the architecture of your target machine e IA 32 e Intel 64 IA 64 Itanium processor family Reason Intel MKL libraries which you should link with your application are located in directories corresponding to your particular architecture see Intel MKL Versions So you should provide proper paths in your link lines see Linking Examples To configure your development environment for the use with Intel
80. e 1 10 Dynamic linking of user s code myprog f and sequential version of Intel MKL supporting LP64 interface ifort myprog f LSMKLPATH ISMKLINCLUDE lmk1_intel_1p64 lmkl_sequential lmkl_core lpthread Static linking of user s code myprog f and parallel Intel MKL supporting ILP64 interface ifort myprog f LSMKLPATH ISMKLINCLUDE Wl start group MKLPATH libmkl_ intel _ilp 4 a SMKLPATH libmk1l_ intel thread a MKLPATH libmkl_core a W1 end group liomp5 lpthread Dynamic linking of user s code myprog f and parallel Intel MKL supporting ILP64 interface ifort myprog f LSMKLPATH ISMKLINCLUDE lmk1l_intel_ilp 64 lmkl_intel_ thread lmkl_core liomp5 lpthread Static linking of user s code myprog Fortran 95 LAPACK interfacet and parallel Intel MKL supporting LP64 interface ifort myprog f LSMKLPATH ISMKLINCLUDE 1lmkl_lapack95 Wl start group MKLPATH libmkl_intel_l1p64 a SMKLPATH libmk1l intel thread a MKLPATH libmkl_core a W1 end group liomp5 lpthread Static linking of user s code myprog Fortran 95 BLAS interfacet and parallel Intel MKL supporting LP64 interface ifort myprog f LSMKLPATH ISMKLINCLUDE 1mk1l_blas95 Wl start group MKLPATH libmkl_ intel _1p64 a SMKLPATH libmkl_intel_thread a S MKLPATH libmkl_core a W1 end group liomp5 lpthread Static linking of user s code myprog f parallel version of an iterative sparse solver and parallel Intel MKL supporting LP64
81. e following Java implementations e J2SE SDK 1 4 2 JDK 5 0 and 6 0 from Sun Microsystems Inc http sun com e JRockit JDK 1 4 2 and 5 0 from BEA Systems Inc http bea com NOTE The implementation from the Sun Microsystems Corporation supports only processors using IA 32 and Intel 64 architectures The implementation from BEA Systems supports Intel Itanium 2 processors as well Also note that the Java run time environment JRE system which may be pre installed on your computer is not enough You need the JDK developer toolkit that supports the following set of tools e java e javac e javah e javadoc To make these tools available for the examples makefile set up the JAVA _ HOME environment variable and add the JDK binaries directory to the system PATH for example export JAVA_HOME home lt user name gt jdk1 5 0_09 export PATH JAVA_HOME bin PATH You may also need to clear the JDK_HOME environment variable if it is assigned a value 7 15 7 Intel Math Kernel Library User s Guide unset JDK HOME To start the examples use the makefile found in the Intel MKL Java examples directory make so32 soem64t so64 1ib32 libem64t 1ib64 function compiler If started without specifying a target any of the choices like so32 the makefile prints the help info which explains the targets as well as the function and compiler parameters For the examples list see the examples 1st file i
82. e for example lt mk1 directory gt lib em 4t For a particular build go to the Tool Settings tab of the C C Build gt Settings property page and specify names of the Intel MKL libraries to link with your application for example mkl1_solver_1p64 and mk1l_core As compilers typically require library names rather than library file names the lib prefix and a extension are omitted To learn how to choose the libraries see Selecting Libraries to Link in chapter 5 The name of the particular setting where libraries are specified depends upon the compiler integration Note that the compiler linker will automatically pick up the include and library paths settings only in cases where the automatic makefile generation is turned on otherwise you will have to specify the include and library paths directly in the makefile to be used Configuring the Eclipse IDE CDT 3 x To configure Eclipse CDT 3 x to link with Intel MKL follow the instructions below For Standard Make projects 1 Goto C C Include Paths and Symbols property page and set the Intel MKL include path that is lt mk1 directory gt include 2 Go to the Libraries tab of the C C Project Paths property page and set the Intel MKL libraries to link with your applications for example lt mk1 directory gt lib em6 4t libmkl_lapack a and lt mk1 directory gt lib em64t libmkl_core a To learn how to choose the libraries see Selecting Libraries to Link in
83. e Precision GMP Arithmetic Library For specifications of these functions please see http www intel com software products mkl docs gnump WebHel If you currently use the GMP library you need to modify INCLUDE statements in your programs to mkl_gmp h FFTW Interface Support Intel MKL offers two collections of wrappers being the FFTW interface www fftw org superstructure to be used for calling the Intel MKL Fourier transform functions These collections correspond to the FFTW versions 2 x and 3 x and the Intel MKL versions 7 0 and later The purpose of these wrappers is to enable developers whose programs currently use FFTW to gain performance with the Intel MKL Fourier transforms without changing the program source code See the FFTW to Intel MKL Wrappers appendix in the Intel MKL Reference Manual file mklman pdf for details on the use of the wrappers B 1 Index A Absoft compiler linking with 5 11 affinity mask 6 16 aligning data 8 2 audience 1 2 benchmark 11 1 BLAS calling routines from C 7 5 Fortran 95 interfaces to 7 2 C C calling LAPACK BLAS CBLAS from 7 4 calling BLAS functions in C 7 7 complex BLAS Level 1 function from C 7 8 complex BLAS Level 1 function from C 7 9 Fortran style routines from C 7 4 CBLAS 7 6 CBLAS code example 7 10 Cluster FFT linking with 9 1 cluster software 9 1 linking examples 9 3 linking syntax 9 1 code assist with Intel MKL in Eclipse CDT 10 6
84. e discussed for LAPACK and BLAS in the subsections below require skills in manipulating the descriptor of a deferred shape array which is the Fortran 90 type Moreover BLAS95 LAPACK95 routines contain links to a Fortran RTL WARNING Avoid calling BLAS95 LAPACK95 from C C Such calls LAPACK As LAPACK routines are Fortran style when calling them from C language programs make sure that you follow the Fortran style calling conventions e Pass variables by address as opposed to pass by value Function calls in Example 7 2 and Example 7 3 illustrate this e Store your data Fortran style that is in column major rather than row major order Language specific Usage Options 7 With row major order adopted in C the last array index changes most quickly and the first one changes most slowly when traversing the memory segment where the array is stored With Fortran style column major order the last index changes most slowly whereas the first one changes most quickly as illustrated by Figure 7 1 for a 2D array Figure 7 1 Column major order vs row major order 1 2 3 q 0 1 2 3 1 0 2 1 3 2 A Column major order Fortran style B Row major order C style For example if a two dimensional matrix A of size m x n is stored densely in a one dimensional array B you can access a matrix element like this Alil j Bli n j inc i 0 m 1 j 0 n 1 A i j B j m i in Fortran i 1 m j l n
85. e performance 6 15 notational conventions 1 3 number of threads changing at run time 6 5 Intel MKL choice particular cases 6 10 setting for cluster 9 2 setting with OpenMP environment variable 6 4 techniques to set 6 3 numerical stability 8 1 0 OpenMP Compatibility run time compiler library 5 11 Legacy run time compiler library 5 11 OpenMP run time compiler library 5 11 P parallel performance 6 4 parallelism 6 1 PARDISO OOC configuration file 4 4 performance 6 1 coding techniques to gain 6 13 hardware tips to gain 6 15 multi core 6 15 of LAPACK packed routines 6 14 with denormals 6 17 R RTL 7 3 RTL layer 3 3 run time library 7 3 Compatibility OpenMP 5 11 Legacy OpenMP 5 11 S ScaLAPACK linking with 9 1 sequential version of the library 3 4 stability numerical 8 1 static linking 5 1 support technical 1 1 syntax linking cluster software 9 1 linking general 5 3 T technical support 1 1 thread safety of Intel MKL 6 2 threading avoiding conflicts 6 4 environment variables and functions 6 8 Intel MKL behavior particular cases 6 10 Intel MKL controls 6 8 see also number of threads threading layer 3 4 U uBLAS matrix matrix multiplication substitution with Intel MKL functions 7 10 unstable output numerically getting rid of 8 1 usage information 1 1 Index 3
86. e products September 2007 005 Documents Intel MKL 10 0 Gold release Configuring of Eclipse CDT 4 0 to link with Intel MKL has been described in chapter 3 Intel Compatibility OpenMP run time compiler library 1ibiomp has been described October 2007 006 Documents Intel MKL 10 1 beta release Information on dummy libraries in Table High level directory structure has been further detailed Information on the Intel MKL configuration file removed Section Accessing Man Pages has been added to chapter 3 Section Support for Boost uBLAS Matrix Matrix Multiplication has been added to chapter 7 Chapter Getting Assistance for Programming in the Eclipse IDE has been added May 2008 007 Documents Intel MKL 10 1 gold release Linking examples for IA 32 archi tecture and section Linking with Computational Libraries have been added to chapter 5 Integration of DSS PARDISO into the layered structure has been documented Two Fortran code examples have been added August 2008 intel INFORMATION IN THIS DOCUMENT IS PROVIDED IN CONNECTION WITH INTEL PRODUCTS NO LICENSE EXPRESS OR IMPLIED BY ESTOPPEL OR OTHERWISE TO ANY INTELLECTUAL PROPERTY RIGHTS IS GRANTED BY THIS DOCUMENT EXCEPT AS PROVIDED IN INTEL S TERMS AND CONDITIONS OF SALE FOR SUCH PRODUCTS INTEL ASSUMES NO LIABILITY WHATSOEVER AND INTEL DISCLAIMS ANY EXPRESS OR IMPLIED WARRANTY RELATING TO SALE AND OR USE OF INTEL PRODUCTS I
87. ease Notes htm Intel MKL Release Notes HTML format Release Notes txt Intel MKL Release Notes text format userguide pdf Intel MKL User s Guide this document vmlnotes htm General discussion of VML vslnotes pdf General discussion of VSL tables Directory that contains tables referenced in vmlnotes htm Accessing Man Pages During installation the man pages for the Intel MKL functions are copied to subdirectory man man3 of the Intel MKL installation directory To make the man pages accessible through the man command in your command shell add the directory with the man pages to the MANPATH environment variable see Setting Environment Variables in chapter 4 Once the environment variable is set to view the man page for an Intel MKL function enter the following command in your command shell man lt function base name gt In this release lt function base name gt is the function name with omitted prefixes denoting data type precision or function domain Examples e For the BLAS function ddot enter man dot e For the ScaLAPACK function pzgeql2 enter man pgeql2 e For the FFT function DEtiCommitDescriptor enter man CommitDescriptor NOTE Function names in the man command are case sensitive 3 21 Configuring Your Development Environment This chapter explains how to configure your development environment for the use with the Intel Math Kernel Library Intel MKL Setting Environment Variables
88. ence between a problem size 700000 and 701000 for instance Another factor that influences the performance drop off is the grid dimensions P and Q For big problems the performance tends to fall off less from the first few steps when P and Q are roughly equal in value You can make use of a large number of parameters such as broadcast types and change them so that the final performance is determined very closely by the first few steps Using these tools will greatly assist the amount of data you can test 11 11 Intel Math Kernel Library Language Interfaces Support Table A 1 shows language interfaces that Intel Math Kernel Library Intel MKL provides for each function domain and Table A 2 lists the respective header files However Intel MKL routines can be called from other languages using mixed language programming For example see section Mixed language programming with Intel MKL in chapter 7 on how to call Fortran routines from C C Table A 1 Intel MKL language interfaces support FORTRAN 77 Fortran 90 95 C C Function Domain interface interface interface Basic Linear Algebra Subprograms BLAS via CBLAS BLAS like extension transposition routines Sparse BLAS Level 1 via CBLAS Sparse BLAS Level 2 and 3 LAPACK routines for solving systems of linear equations LAPACK routines for solving least squares problems eigenvalue and singular value problems and Sylvester s equation
89. er Routines E U Vector Mathematical Functions Statistica Functions EU Fourier Transform Functions O4 Interval Linear Solvers 4 Partial Differential Equations Support amp C Optimization Solvers Routines E U Support Functions _ 04 BLACS Routines E Appendices 8 Bibliography E O dossa amp CDT Plug in Developer Guide E C C Development User Guide 10 2 Getting Assistance for Programming in the Eclipse IDE 1 0 Searching the Intel Web Site from the Eclipse IDE The Intel MKL plugin tunes Eclipse Help search to target http www intel com so that when you run a search from the Eclipse Help pane the search hits at the site are shown through a separate link Figure 10 2 shows search results for VML Functions in Eclipse Help 1 hit means an entry hit to the respective site Click Intel com 1 hit to open the list of actual hits to the Intel Web site Figure 10 2 Hits to the Intel Web Site in the Eclipse IDE Help search ava Eclipse SDK 5 Edit Navigate Search Project Run Window Hep Je rOrariSBeor IOP liv a Elg Java a g N Saj gt Ge wekome La An outine is not available 3 Search x Search expression 7 YML Functions el s Search scope Default gt Local Help 1 10 of 93 hits Google 1 hit 4 Web Search Click on this link to see the results Edlipse org 1 hit Web Search Click on this link to see the results Web Search Click on
90. er problems will have fewer than 46 updates and the biggest problems will have precisely 46 updates The Mf lops is an estimate based on 1280 columns of LU being completed However with lookahead steps sometimes that work is not actually completed when the output is made Nevertheless this is a good estimate for comparing identical runs The 3 numbers in parenthesis are intrusive ASYOUGO2 addins The DT is the total time processor 0 has spent in DGEMM The DF is the number of billion operations that have been performed in DGEMM by one processor Hence the performance of processor 0 in Gflops in DGEMM is always DF DT Using the number of DGEMM flops as a basis instead of the number of LU flops you get a lower bound on performance of our run by looking at DMF which can be compared to Mflops above It uses the global LU time but the DGEMM flops are computed under the assumption that the problem is evenly distributed amongst the nodes as only HPL s node 0 0 returns any output Note that when using the above performance monitoring tools to compare different HPL dat inputs you should beware that the pattern of performance drop off that LU experiences is sensitive to some of the inputs For instance when you try very small problems the performance drop off from the initial values to end values is very rapid The larger the problem the less the drop off and it is probably safe to use the first few performance values to estimate the differ
91. erface ScaLAPACK routines library supporting LP64 interface A dummy library Contains a reference to lib eme 4t libmkl solver _1p64 a Iterative Sparse Solver and GMP routine library supporting ILP64 interface Sequential version of Iterative Sparse Solver and Trust Region Solver routine library supporting ILP64 interface Iterative Sparse Solver Trust Region Solver and GMP routine library supporting LP64 interface Sequential version of Iterative Sparse Solver Trust Region Solver and GMP routine library supporting LP64 interface Intel Math Kernel Library Structure 3 Detailed directory structure continued Directory file RTL layer libguide a libiomp5 a libmkl_blacs_ilp64 a libmkl_blacs_ intelmpi_ilp64 a libmkl_blacs_ intelmpi_ lp6 4 a libmkl_blacs_ intelmpi20_ ilp64 a libmkl_ blacs_ intelmpi20 lp 4 a libmkl_blacs_ 1p64 a libmk1_blacs_ openmpi_ilp 4 a libmk1l_blacs_ openmpi_ lp 4 a libmkl_ blacs_ sgimpt_ilp6 4 a libmkl blacs_ sgimpt_lp6 4 a Dynamic Libraries Interface layer libmkl_gf_ilp64 so libmkl_gf_1p64 so libmk1_intel_ilp64 so libmk1_intel_1p64 so libmk1_intel_sp2dp so Contents Intel Legacy OpenMP run time library for static linking Intel Compatibility OpenMP run time library for static linking ILP64 version of BLACS routines supporting the following MPICH versions Myricom MPICH version 1 2 5 10 e ANL MPICH version 1 2 5 2 ILP64 version of BLACS routines
92. ernel Library User s Guide e All FFTs except 1D transformations when DFTI_NUMBER_OF_ TRANSFORMS 1 and sizes are not power of two NOTE NOTE For power of two data in 1D FFTs Intel MKL provides parallelism for all the three supported architectures For Intel 64 architecture the parallelism is provided for double complex out of place FFTs only Being designed for multi threaded programming Intel MKL is thread safe which means that all Intel MKL functions work correctly during simultaneous execution by multiple threads In particular any chunk of threaded Intel MKL code provides access of multiple threads to the same shared data while permitting only one thread at any given time to access a shared piece of data Due to thread safety you can call Intel MKL from multiple threads and not worry about the function instances interfering with each other The library uses OpenMP threading software which responds to the environmental variable OMP_NUM_THREADS that sets the number of threads to use Notice that there are different means to set the number of threads In Intel MKL releases earlier than 10 0 you could use the environment variable OMP_NUM_THREADS see Setting the Number of Threads Using OpenMP Environment Variable for details or the equivalent OpenMP run time function calls detailed in section Changing the Number of Threads at Run Time Starting with version 10 0 Intel MKL also offers variables that are independent of OpenMP such
93. ers Interfaces for the ESSL like functions are described in the generated documentation for the com intel mk1 ESSL class Each wrapper consists of the interface part for Java and JNI stub written in C You can find the sources in the following directory lt mkl directory gt examples java wrappers Both Java and C parts of the wrapper for CBLAS and VML demonstrate the straightforward approach which you may easily employ to cover additional CBLAS functions The wrapper for FFT is more complicated because of supporting the lifecycle for FFT descriptor objects To compute a single Fourier transform an application needs to call the FFT software several times with the same copy of the native FFT descriptor The wrapper provides the handler class to hold the native descriptor while virtual machine runs Java bytecode The wrapper for VSL RNG is similar to the one for FFT The wrapper provides the handler class to hold the native descriptor of the stream state The wrapper for the convolution and correlation functions mitigates the same difficulty of the VSL interface which assumes similar lifecycle for task descriptors The wrapper utilizes the ESSL like interface for those functions which is simpler for the case of 1 dimensional data The JNI stub additionally enwraps the MKL functions into the ESSL like wrappers written in C and so packs the lifecycle of a task descriptor into a single call to the native method The wrappers meet the J
94. ers for CBLAS FFT VML VSL RNG and ESSL like convolution and correlation functions do not depend on each other To build the wrappers just run the examples see the Running the examples section for details The makefile builds the wrapper binaries and the examples invoked after that double check if the wrappers are built correctly As a result of running the examples the following directories will be created in lt mk1 directory gt examples java docs include classes bin e results The directories docs include classes and bin will contain the wrapper binaries and documentation the directory results will contain the testing results For a Java programmer the wrappers look like the following Java classes com intel mk1l CBLAS com intel mk1 DFTI com intel mk1l ESSL com intel mk1l VML 7 13 7 Intel Math Kernel Library User s Guide com intel mk1l VSL Documentation for the particular wrapper and example classes will be generated from the Java sources during building and running the examples To browse the documentation start from the index file in the docs directory which will be created by the build script lt mkl directory gt examples java docs index html The Java wrappers for CBLAS VML VSL RNG and FFT establish the interface that directly corresponds to the underlying native functions and you can refer to the Intel MKL Reference Manual for their functionality and paramet
95. ers library supporting Intel compiler Parallel drivers library supporting PGI compiler Sequential drivers library Dummy library Contains references to Intel MKL libraries lib em 4t libmkl intel 1p64 s0 lib em 4t libmkl intel thread so and lib em 4t libmkl_ core so Library dispatcher for dynamic load of processor specific kernel Default kernel library Kernel library for processors based on the Intel Core microarchitecture Kernel library for processors based on the next generation Intel microarchitecture Nehalem LAPACK and DSS PARDISO routines and drivers ScaLAPACK routines library supporting ILP64 interface ScaLAPACK routines library supporting LP64 interface VML VSL part of default kernels VML VSL for processors based on the Intel Core microarchitecture VML VSL for processors based on the next generation Intel microarchitecture Nehalem VML VSL for Intel Xeon processor using Intel 64 architecture VML VSL for 45nm Hi k Intel Core 2 and Intel Xeon pro cessor families Intel Legacy OpenMP run time library for dynamic linking Intel Compatibility OpenMP run time library for dynamic linking Intel Math Kernel Library Structure 3 Table 3 6 Detailed directory structure continued Directory file libmkl_intelmpi_ ilp64 so libmkl_intelmpi_ 1p64 so 1ib 64 Static Libraries Interface layer libmkl_intel_ilp6 4 a libmkl_intel_ lp64 a libmkl_intel_sp2dp a libmkl_
96. es Quick comparison of Intel MKL linkage models Static Linkage Automatic All processors Link to static libraries Regular names Small Small Yes Custom Dynamic Linkage Recompile and redistribute All processors Build separate dynamic libraries and link to them Modified names Small Smallest Yes 1 Except for LAPACK deprecated routines lacon lasq3 and lasq4 Intel MKL specific Linking Recommendations You are strongly encouraged to dynamically link in Intel Compatibility OpenMP run time library 1ibiomp and Intel Legacy OpenMP run time library libguide Linking to static OpenMP run time library is not recommended because it is very easy with layered 5 2 Linking Your Application with Intel Math Kernel Library 5 software to link in more than one copy of the library This causes performance problems too many threads and may cause correctness problems if more than one copy is initialized You are advised to link with 1ibiomp and 1libguide dynamically even if other libraries are linked statically Link Command Syntax To link libraries having filenames libzzz a or libzzz so with your application two options are available In the link line list library filenames using relative or absolute paths for example lt ld gt myprog o lt mk1 directory gt 1lib 32 libmkl1_solver a lt mkl directory gt lib 32 libmkl_intel a lt mkl directory gt lib 32 libmkl_ intel thread a lt mkl di
97. es These pointers initially hold addresses of respective system memory management functions malloc free calloc realloc and are visible at the application level So the pointer values can be redefined programmatically Once a user has redirected these pointers to their own respective memory management functions the memory will be managed with user defined functions rather than system ones As only one user defined memory management package is in operation the issues are avoided Managing Performance and Memory 6 Intel MKL memory management by default uses standard C run time memory functions to allocate or free memory These functions can be replaced using memory renaming How to redefine memory functions To redefine memory functions you may use the following procedure 1 Include the i_malloc h header file in your code The header file contains all declarations required for an application developer to replace the memory allocation functions This header file also describes how memory allocation can be replaced in those Intel libraries that support this feature 2 Redefine values of pointers i_malloc i free i_calloc i_realloc prior to the first call to MKL functions Example 6 4 Redefining memory functions include i _malloc h i_malloc my_malloc i_calloc my_calloc i_realloc my_realloc i_free my free Now you may call Intel MKL functions 6 19 Language specific Usage Options Intel
98. es as well as for test and example programs lt mk1 directory gt interfaces blas95 Contains Fortran 95 wrappers for BLAS and a makefile to build the library lt mkl directory gt interfaces Contains wrappers for MPI FFTW version 2 x for com fftw2x cdft plex 1D transforms to call Intel MKL Cluster FFT inter z face 3 1 3 Intel Math Kernel Library User s Guide Table 3 1 High level directory structure continued Directory Comment lt mkl directory gt interfaces fftw2xc lt mk1 lt mk1 lt mk1 lt mk1 lt mk1 lt mk1 lt mk1 lt mk1 lt mk1 lt mk1 lt mk1 lt mk1 directory gt interfaces fftw2xf directory gt interfaces fftw3xc directory gt interfaces fftw3xf directory gt interfaces lapack95 directory gt lib 32 directory gt 1lib 64 directory gt lib em6 4t directory gt man man3 directory gt tests directory gt tools builder directory gt tools environment directory gt tools plugins com intel mkl help lt mk1 directory gt tools support Contains wrappers for FFTW version 2 x C interface to call Intel MKL FFTs Contains wrappers for FFTW version 2 x Fortran interface to call Intel MKL FFTs Contains wrappers for FFTW version 3 x C interface to call Intel MKL FFTs Contains wrappers for FFTW version 3 x Fortran interface to call Intel MKL FFTs Contains Fortran 95 wrappers for LAPACK and a makefile to build the library Contains static libraries and
99. et_num_threads 4 MKL ALL is equivalent to mkl_ set _num_ threads 4 and thus it will be overwritten by later calls to mkl_set_num_threads Similarly the environment setting of MKL_DOMAIN_NUM_THREADS with MKL _ALL 4 will be overwritten with MKL NUM THREADS 2 Whereas the MKL_ DOMAIN NUM_THREADS environment variable enables you set several variables at once for example MKL_BLAS 4 MKL_FFT 2 the corresponding function does not take string syntax So to do the same with the function calls you may need to make several calls which in this example are as follows mkl_domain_set_num threads 4 MKL BLAS mkl _ domain _set_num threads 2 MKL_ FFT Setting the Environment Variables for Threading Control To set the environment variables used for threading control in the command shell in which the program is going to run enter export lt VARIABLE NAME gt lt value gt for certain shells such as bash Managing Performance and Memory 6 For example export MKL NUM THREADS 4 export MKL DOMAIN NUM THREADS MKL ALL 1 MKL BLAS 4 export MKL DYNAMIC FALSE For other shells such as csh or tcsh enter set lt VARIABLE NAME gt lt value gt For example set MKL NUM THREADS 4 set MKL DOMAIN NUM THREADS MKL ALL 1 MKL BLAS 4 set MKL DYNAMIC FALSE Note on FFT Usage Introduction of additional threading control made it possible to optimize the commit stage of the FFT implementation and get rid of
100. face module for BLAS BLAS95 mk195 lapack mod Contains Fortran 95 interface module for LAPACK E LAPACK95 mkl95_precision mod Contains Fortran 95 definition of precision parameters for BLAS95 and LAPACK95 Section Fortran 95 Interfaces and Wrappers to LAPACK and BLAS shows by example how these libraries and modules are generated Fortran 95 Interfaces and Wrappers to LAPACK and BLAS Fortran 95 interfaces are provided for pure procedures and along with wrappers are delivered as sources For more information see Compiler dependent Functions and Fortran 90 Modules The simplest way to use them is building corresponding libraries and linking them as user s libraries To do this you must have administrator rights Provided the product directory is open for writing the procedure is simple 1 Goto the respective directory lt mk1_directory gt interfaces blas95 or lt mkl_directory gt interfaces lapack95 2 Type one of the following commands make PLAT 1nx32 lib for IA 32 architecture make PLAT 1nx32e lib for Intel 64 architecture make PLAT 1nx64 lib for IA 64 architecture 7 2 Language specific Usage Options 7 As a result the required library and a respective mod file will be built and installed in the standard catalog of the release The mod files can also be obtained from files of interfaces using the compiler command ifort c mkl_lapack f 90 or ifort c mkl_blas f90 These files are in the in
101. for example W1 start group MKLPATH libmkl_cdft_core a SMKLPATH 1libmkl_blacs_intelmpi_ilp64 a S MKLPATH libmkl_ intel ilp64 a SMKLPATH 1libmkl_intel_ thread a SMKLPATH libmkl_core a W1 end group See specific examples in the Linking Examples section If you use dummy libraries e The path to Intel MKL libraries must be added to the list of paths that the linker will search for archive libraries for example aS L lt MKL path gt e No Interface layer or Threading layer libraries should be included in the link line e No grouping symbols must be employed The order of listing libraries in the link line is essential except for the libraries enclosed in the grouping symbols 5 5 5 Intel Math Kernel Library User s Guide Selecting Libraries to Link Below are several simple examples of link libraries for the layered and legacy link models on 64 bit Linux based on Intel 64 architecture for different components using Intel compiler interface 5 6 BLAS FFT VML VSL components static case Legacy libmkl_em 4t a Layered libmkl intel 1p64 a libmkl_intel_thread a libmkl_core a BLAS FFT VML VSL components dynamic case Legacy libmkl so Layered libmkl_ intel 1p6 4 so libmkl_intel thread so libmkl_core so LAPACK static case Legacy libmkl_lapack a libmkl_em 4t a Layered libmkl intel 1p64 a libmkl_intel_thread a libmkl_core a LAPACK dynamic case Legacy libmkl_lapack so libmkl so Layered libmkl_
102. g LP64 interface 3 17 3 Intel Math Kernel Library User s Guide Table 3 6 Directory file libmkl_solver a libmkl_solver_ ilp6 4 a libmkl_ solver_ilp6 4_ sequential a libmkl_solver_1p64 a libmkl_solver_1p64_ sequential a RTL layer libguide a libiomp5 a libmkl_blacs_ilp64 a libmkl_blacs_ intelmpi_ilp 4 a libmkl_blacs_ intelmpi_lp64 a libmkl_blacs_ intelmpi20 ilp 4 a libmkl_blacs_ intelmpi20 lp64 a libmkl_blacs_ l1p64 a libmkl_blacs_ openmpi_ilp64 a libmkl_blacs_ openmpi_lp64 a libmkl_blacs_ sgimpt_ilp6 4 a Detailed directory structure continued Contents Dummy library Contains a reference to 1ib 64 libmk1 solver _1p64 a Iterative Sparse Solver and Trust Region Solver routine library supporting ILP64 interface Sequential version of Iterative Sparse Solver and Trust Region Solver routine library supporting ILP64 interface Iterative Sparse Solver Trust Region Solver and GMP routine library supporting LP64 interface Sequential version of Iterative Sparse Solver Trust Region Solver and GMP routine library supporting LP64 interface Intel Legacy OpenMP run time library for static linking Intel Compatibility OpenMP run time library for static linking ILP64 version of BLACS routines supporting the following MPICH versions Myricom MPICH version 1 2 5 10 e ANL MPICH version 1 2 5 2 ILP64 version of BLACS routines supporting Intel MPI 2 0 and 3
103. gemm CblasRowMajor CblasNoTrans CblasNoTrans m n k alpha a lda b ldb beta c ldc printf row ta tc n for 1 0 i lt 10 i printf d t f t f n i ali SIZE c i SIZE omp _ set_num threads 1 for i 0 aie i for j 0 j lt SIZE j a i SIZE j double i j b i SIZE j double i j c i SIZE j double 0 cblas_dgemm CblasRowMajor CblasNoTrans CblasNoTrans m n k alpha a lda b ldb beta c ldc printf row ta tc n for i 0 1 lt 10 i printf Sd t t n i ali SIZE c i SIZE omp_ set _num_threads 2 for i 0 i lt SIZE i for j 0 j lt SIZE j i SIZE j double i i SIZE j double i 0 a b c i SIZE j double 0 oe cblas_dgemm CblasRowMajor CblasNoTrans CblasNoTrans m n k alpha a lda b ldb beta c ldc printf row ta tc n for i 0 1 lt 10 i printf Sd t E tsF n i alix SIZE c i SIZE delete a delete b delete c kKkKKKKK Fortran language KkKKKKKK PROGRAM DGEMM DIFF THREADS 6 6 Managing Performance and Memory 6 Example 6 1 Changing the number of processors for threading continued INTEGER N L J PARAMETER N 1000 REAL 8 A N N B N N C N N REAL 8 ALPHA BETA INTEGER 8 MKL MALLOC integer ALLOC SIZE integer NTHRS ALLOC SIZE 8 N N A PTR MKL MALLOC ALLOC_SIZE 128 B PTR MKL MALLOC ALLOC_SIZE 128 C_PTR MKL M
104. get started 2 3 Table 3 1 High level directory structure sssssssssssssssrrrrrrrrrrrrssrss 3 1 Table 3 2 Intel MKL ILP64 CONCept sssssssssssrrrrrrrrrssrssrrrrrrrrrrrrrrss 3 7 Table 3 3 Compiler options for the ILP64 interface cccceeeeees 3 8 Table 3 4 Integer tyPeS ssesssrrssrssssesssrrrrrnrnnnnnssnsenserrerrnnnnennes 3 8 Table 3 5 ILP64 support in Intel MKL ssssssssssssrrssssssrrrrrrrrrrrersss 3 9 Table 3 6 Detailed directory StruCtUre ccccecceseeeeeeeeeeneeeeeeaeeaas 3 10 Contents Table 3 7 Contents of the doc directory ssssssssssssssssrsrrrrrrrrrssrssere 3 20 Table 5 1 Quick comparison of Intel MKL linkage models 5 2 Table 5 2 Interface layer library for linking with the Absoft COMPIerS s ede a aaa ea eee ain a ea ae ct eeve 5 11 Table 5 3 Selecting the Threading Layer cccccecesceeeeeeeeeeeesaeeaees 5 12 Table 5 4 Computational libraries to link by function domain 5 13 Table 6 1 How to avoid conflicts in the execution environment for YOUr threading Modelar karintin e ceed eee na seers eee er eb aaa i 6 4 Table 6 2 Intel MKL environment variables for threading controls 6 9 Table 6 3 Interpretation of MKL_DOMAIN_NUM_THREADS values 6 12 Table 7 1 Interface libraries and modules ccc eceeee cee ue eeeeeeeeeeeeenens 7 1 Table 11 1 Contents of the LINPACK Benchmark c scceceeeeeeeeeees 11 2 Table 11 2 Contents of the MP LINPA
105. h Kernel Library User s Guide C code presented in Example 6 3 solves the problem The code example calls the system function sched_setaffinity to bind the threads to the cores on different sockets After that the Intel MKL FFT function is called Compile your application with the Intel compiler using the following command icc test_application c openmp where test_application c is the filename for the application Build the application and run it in 2 threads env OMP_NUM THREADS 2 a out Example 6 3 Setting an affinity mask by operating system means using an Intel compiler SetThreadAffinityMask GetCurrentThread mask include lt stdio h gt define __USE_GNU Set affinity mask include lt sched h gt include lt unistd h gt include lt omp h gt int main void int NCPUs sysconf _SC_NPROCESSORS_CONF printf Using thread affinity on i NCPUs n NCPUs pragma omp parallel default shared cpu_set_t new_mask cpu_set_t was_mask int tid omp_get_thread_num CPU_ZERO amp new_mask 2 packages x 2 cores pkg x 1 threads core 4 total cores CPU_SET tid 0 0 2 amp new_mask if sched_getaffinity 0 sizeof was_mask amp was_mask 1 printf Error sched_getaffinity d sizeof was_mask amp was_mask n tid if sched_setaffinity 0 sizeof new_mask amp new_mask 1 printf Error sched_setaffinity d sizeof new mask amp new_mask n tid printf tid
106. he use of CBLAS interface Using Complex Types in C C 7 6 As described in the Building Applications document for the Intel Visual Fortran Compiler 10 1 C C does not directly implement the Fortran types COMPLEX 4 and COMPLEX 8 However you can write equivalent structures The type COMPLEX 4 has two fields each of which is a 4 byte floating point number The first field contains the real number component and the second one contains the imaginary number component The type COMPLEX 8 is similar to COMPLEX 4 except that each field contains an 8 byte floating point number Intel MKL provides complex types MKL_Complex8 and MKL_Complex16 which are structures equivalent to the Fortran complex types COMPLEX 4 and COMPLEX 8 respectively These types are defined in the mkl_types h header file You can use these types to define complex data You can also redefine the types with your own types before including the mkl_types h header file The only requirement is that the types must be compatible with the Fortran complex layout that is the complex type must be a pair of real numbers for the values of real and imaginary parts For example you can use the following definitions in your C code define MKL Complex8 std complex lt float gt and define MKL Complex16 std complex lt double gt See Example 7 2 for details You can also define these types in the command line DMKL_Complex8 std complex lt float gt DMKL_Comp
107. ial a routines LP64 libmkl_core a interface Iterative n a n a libmkl solver n at Sparse ilp 4 a Solvers or Trust Region libmkl_solver_ Solver and ilp64_ GMP sequential a routines ILP64 libmkl_core a interface 5 13 5 Intel Math Kernel Library User s Guide Table 5 4 Function domain Interface Direct Sparse Solver PARDISO Solver Vector Math Library Vector Statistical Library Fourier Transform Functions Trigono metric Transform Functions Poisson Library ScaLAPACK2 ScaLAPACK LP64 interface ScaLAPACK ILP64 interface Cluster Fourier Transform Functions Computational libraries to link by function domain continued IA 32 Architecture Static libmkl_core a libmkl_core a libmkl_core a libmkl_core a libmkl_core a libmkl_core a libmkl_scalapack _core a libmk1_core a n a n a libmkl_cdft_ COre a libmkl_core a Dynamic libmkl_lapack so libmkl_core so libmkl_core so libmkl_core so libmkl_core so libmkl_core so libmkl_core so libmk1_scalapack _core so libmk1_lapack sO libmk1_core so n a n a n a Intel 64 or IA 64 Architecture Static libmkl_core a libmkl_core a libmkl_core a libmkl_core a libmkl_core a libmkl_core a See below libmkl_scalapack _lp64 a libmkl_core a libmkl_scalapack _ilp64 a libmkl lapack sO 7 libmk1_core a libmk1_cdft
108. imension values in bytes n element_size of two dimensional arrays should be divisible by cache line size which equals e 32 bytes for Pentium III processor e 64 bytes for Pentium 4 processor e 128 bytes for processor using Intel 64 architecture Applications based on IA 64 architecture The sufficient conditions are as follows e For the C style FFT the distance L between arrays that represent real and imaginary parts is not divisible by 64 The best case is when L k 64 16 Managing Performance and Memory 6 e Leading dimension values in bytes n element_size of two dimensional arrays are not power of two Hardware Configuration Tips Dual Core Intel Xeon processor 5100 series systems To get the best Intel MKL performance on Dual Core Intel Xeon processor 5100 series systems you are advised to enable the Hardware DPL streaming data Prefetcher functionality of this processor Configuration of this functionality is accomplished through appropriate BIOS settings where supported Check your BIOS documentation for details The use of Hyper Threading Technology Hyper Threading Technology HT Technology is especially effective when each thread is performing different types of operations and when there are under utilized resources on the processor Intel MKL fits neither of these criteria as the threaded portions of the library execute at high efficiencies using most of the available resources and perform identical opera
109. ironment variables for example MKL_NUM THREADS Changing the Number of Threads at Run Time It is not possible to change the number of processors during run time using the environment variables However you can call OpenMP API functions from your program to change the number of threads during run time The following sample code demonstrates changing the number of threads during run time using the omp_set_num_threads routine See also Techniques to Set the Number of Threads To run this example use the omp h header file from the Intel compiler package If you do not have the Intel compiler but wish to explore the functionality in the example use Fortran API for omp_set_num threads rather than the C version Example 6 1 Changing the number of processors for threading KKKKKKEK C language kkkkkkk include omp h include mkl h include lt stdio h gt define SIZE 1000 void main int args char argv double a b c a new double SIZE SIZE b new double SIZE SIZE c new double SIZE SIZE double alpha 1 beta 1 int m SIZE n SIZE k SIZE lda SIZE ldb SIZE ldc SIZE i 0 j 0 char transa n transb n for i 0 i lt SIZE i for j 0 j lt SIZE j ali SIZE j double i j b i SIZE j double i j c i SIZE j double 0 6 5 6 Intel Math Kernel Library User s Guide Example 6 1 Changing the number of processors for threading continued cblas_d
110. it is 1000 4 4 Linking Your Application with Intel Math Kernel Library This chapter discusses linking your applications with Intel Math Kernel Library Intel MKL for the Linux OS The chapter compares static and dynamic linking models describes the general link line syntax to be used for linking with Intel MKL libraries features information in a tabular form on the libraries that should be linked with your application for your particular platform and function domain provides linking examples Building custom shared objects is also discussed Selecting Between Linkage Models You can link your applications with Intel MKL statically using static libraries or dynamically using shared libraries Static Linking Static linking resolves all symbolic references at link time The behavior of statically built executables is predictable because there are no run time dependencies The main disadvantage is that having to relink new versions of the library to your application may be error prone and time consuming because you have to relink the entire application Moreover static linking results in large executables and uses memory less efficiently If several executables are linked with the same library each of them must load it into memory independently This matters most for executables having data sizes that are small and comparable with the size of the executable Dynamic Linking Dynamic linking postpones the resolution of
111. ith more than 2 billion elements e To enable compiling your Fortran code with the i8 compiler option 3 5 3 Intel Math Kernel Library User s Guide 3 6 The Intel Fortran Compiler supports the i8 option for changing behavior of the INTEGER type By default the standard INTEGER type is 4 byte The i8 option makes the compiler treat INTEGER constants variables function and subroutine parameters as 8 byte The ILP64 binary interface uses 8 byte integers for function parameters that define array sizes indices strides etc At the language level that is in the 90 and 3 files located in the Intel MKL include directory such parameters are declared as INTEGER To bind your Fortran code with the ILP64 interface you must compile your code with the i8 compiler option And vice versa if your code is compiled with i8 you can bind it only with the ILP64 interface because the LP64 binary interface requires the INTEGER type to be 4 byte Note that some Intel MKL functions and subroutines have scalar or array parameters of type INTEGER 4 or INTEGER KIND 4 which are always 4 byte regardless of whether the code is compiled with the i8 option For the languages of C C Intel MKL provides the MKL_INT type as a counterpart of the INTEGER type for Fortran MKL_INT is a macro defined as the standard C C type int by default However if the MKL_ILP64 macro is defined for the code compilation MKL_ INT is defined as a 64 bit i
112. ium 4 processor with Streaming SIMD Extensions 3 SSE3 Intel Math Kernel Library Structure 3 Table 3 6 Detailed directory structure continued Directory file libmk1_scalapack_ core so libmkl_vml_def so libmkl_vml_ia so libmk1l_vml_p4 so libmk1_vml_p4m so libmk1_vml_p4m2 so libmk1_vml_p4m3 so libmk1_vml_p4p so RTL layer libguide so libiomp5 so libmkl_ blacs_ intelmpi so lib em64t Static Libraries Interface layer libmkl_ gf ilp64 a libmkl_ gf lp64 a libmkl_intel_ilp6 4 a libmkl_intel_ lp64 a libmkl_intel_sp2dp a Threading layer libmkl_gnu_thread a libmk1l_intel_thread a libmkl_pgi_thread a libmkl_sequential a Contents ScaLAPACK routines VML VSL part of default kernel for old Intel Pentium pro cessors VML VSL default kernel for newer Intel architecture pro cessors VML VSL part of Pentium 4 processor kernel VML VSL for processors based on the Intel Core microarchitecture VML VSL for 45nm Hi k Intel Core 2 and Intel Xeon pro cessor families Kernel library for processors based on the next generation Intel microarchitecture Nehalem VML VSL for Pentium 4 processor with Streaming SIMD Extensions 3 SSE3 Intel Legacy OpenMP run time library for dynamic linking Intel Compatibility OpenMP run time library for dynamic linking BLACS routines supporting Intel MPI 2 0 and 3 0 and MPICH2 Libraries for Intel 64 architecture ILP6
113. kaee Cs 4 1 Configuring the Eclipse IDE CDT to Link with Intel MKL 4 2 Configuring the Eclipse IDE CDT 4 0 0 cccccccceseeeeeeeee ener eaeeaeenas 4 2 Configuring the Eclipse IDE CDT 3 X ssssssssssssrrsrrunrrrrrrnnrrrrrererens 4 3 Note on the Configuration file for Out of Core OOC DSS PARDISO Solve favs otecwe davis rw Hie Rotana arcane ott AEN eae tate ceala mena age eee ean 4 4 Chapter 5 Linking Your Application with Intel Math Kernel Library Selecting Between Linkage Models cceeeeeeeeeeeeeeeeeeeeeeeeeeeeneees 5 1 Static LINKING eenia ae ariaa raTa ane dehy os EITEN E ae ideas ele 5 1 DyNamMic LINKING ieia a eaa een a Ea Ka EAN EAEE tenets 5 1 Making the CHO e a e eaa aaa a ETE AAEE EAA AE aAa DEN 5 2 Intel MKL specific Linking Recommendations sssssssssressrrrrrsrerres 5 2 Link Command Synta osiris iai pa aani a A aeaa eNd 5 3 Sel cti g Hbraries to Linki r r cack aaa a aise eee aa E 5 6 Linking Examples anaa a a a E A a a a Na 5 8 Linking with Interface Libraries ccceeeeeeee eee ee teeta seat eeeeees 5 11 Linking with Threading Libraries cceceeeee eee neces eee eeeeee es 5 11 Linking with Computational Libraries ccceeeeeeeeeee eter eens 5 12 Notes On LINKING sseee saree eee kien feiss Aguabiadeibsseade awardee 5 15 Building Custom Shared Objects cceceeeeee teen eee eee eeeeeeeeeenaes 5 15 Intel MKL Custom Shared Object Builder
114. layered model of linking the Computational layer contains only one library However certain Intel MKL function domains still require several computational link libraries For each Intel MKL function domain Table 5 4 lists Computational layer libraries that you must include in the link line for the layered model of linking For more information on linking with ScaLAPACK and Cluster FFTs see also Linking with ScaLAPACK and Cluster FFTs in chapter 9 Linking Your Application with Intel Math Kernel Library 5 Table 5 4 Computational libraries to link by function domain Function IA 32 Architecture Intel 64 or IA 64 Architecture domain Interface Static Dynamic Static Dynamic BLAS libmkl_core a libmkl_core so libmkl_core a libmkl_core so Sparse BLAS libmkl_core a libmkl_core so libmkl_core a libmkl_core so BLAS95 Libmk1_ n at Libmk1_ n a Interface blas95 a blas95 a libmkl_core a libmkl_core a CBLAS libmkl_core a libmkl_core so libmkl_core a libmkl_core so LAPACK libmkl_core a libmkl_lapack libmkl_core a libmkl_lapack so so libmkl_core so libmkl_core so LAPACK95 libmk1_ n a libmk1_ n a Interface lapack95 a lapack95 a libmkl_core a libmkl_core a Iterative libmkl_solver a n a See below n a Sparse or Solvers libmkl_solver_ Trust Region sequential a Solver and GMP routines libmkl_core a Iterative n a n a libmkl solver n at Sparse lp64 a Solvers or Trust Region libmkl_solver_ Solver and lp64_ GMP sequent
115. lel drivers library supporting Intel compiler Sequential drivers library Dummy library Contains references to Intel MKL libraries 1ib 64 libmk1l intel 1p64 so 1ib 64 libmk1l intel thread so and 1ib 64 libmk1_core so Library dispatcher for dynamic load of processor specific kernel library Kernel library for IA 64 architecture LAPACK and DSS PARDISO routines and drivers ScaLAPACK routines library supporting ILP64 interface ScaLAPACK routines library supporting LP64 interface VML kernel for IA 64 architecture Intel Legacy OpenMP run time library for dynamic linking Intel Compatibility OpenMP run time library for dynamic linking ILP64 version of BLACS routines supporting Intel MPI 2 0 and 3 0 and MPICH2 3 19 3 Intel Math Kernel Library User s Guide Table 3 6 Detailed directory structure continued Directory file Contents libmkl_blacs_ LP64 version of BLACS routines supporting Intel MPI intelmpi_1p64 so 2 0 and 3 0 and MPICH2 1 Additionally a number of interface libraries may be generated as a result of respective makefile operation in the interfaces directory see Using Lanquage Specific Interfaces with Intel MKL in chapter 7 Dummy Libraries Layered libraries give more flexibility to choose the appropriate combination of libraries but do not have backward compatibility by library names in link lines Dummy libraries are introduced to provide backward compatibility with earlier version of
116. les java The examples are provided for the following MKL functions e the gemm gemv and dot families from CBLAS e complete set of non cluster FFT functions e ESSL like functions for 1 dimensional convolution and correlation e VSL Random Number Generators RNG except user defined ones and file subroutines e VML functions except GetErrorCallBack SetErrorCallBack and ClearErrorCallBack You can see the example sources in the following directory IBM Engineering Scientific Subroutine Library ESSL Language specific Usage Options 7 lt mkl directory gt examples java examples The examples are written in Java They demonstrate usage of the MKL functions with the following variety of data e 1 and 2 dimensional data sequences e real and complex types of the data e single and double precision However note that the wrappers used in examples do not e demonstrate the use of huge arrays gt 2 billion elements e demonstrate processing of arrays in native memory e check correctness of function parameters e demonstrate performance optimizations To bind with Intel MKL the examples use the Java Native Interface JNI developer framework The JNI documentation to start with is available from http java sun com j2se 1 5 0 docs guide jni index html The Java example set includes JNI wrappers which perform the binding The wrappers do not depend on the examples and may be used in your Java applications The wrapp
117. lex16 std complex lt double gt Language specific Usage Options 7 Calling BLAS Functions That Return the Complex Values in C C Code You must be careful when handling a call from C to a BLAS function that returns complex values The problem arises because these are Fortran functions and complex return values are handled quite differently for C and Fortran However in addition to normal function calls Fortran enables calling functions as though they were subroutines which provides a mechanism for returning the complex value correctly when the function is called from a C program When a Fortran function is called as a subroutine the return value shows up as the first parameter in the calling sequence This feature can be exploited by the C programmer The following example shows how this works Normal Fortran function call result cdotc n x 1 y 1 A call to the function as a subroutine call cdotc result n x 1 y 1 A call to the function from C notice that the hidden parameter gets exposed cdotc amp result amp m x amp one y amp one 775 NOTE Intel MKL has both upper case and lower case entry points in BLAS with trailing underscore or not So all these names are acceptable cdotc CDOTC cdotc_ CDOTC_ Using the above example you can call from C and thus from C several level 1 BLAS functions that return complex values However it is still easier to use the CBLAS interface For instance
118. mprove the performance of Intel MKL FFT if the length of your data vector permits factorization into powers of optimized radices In Intel MKL the list of optimized radices depends upon the architecture e 2 3 4 5 for IA 32 architecture e 2 3 4 5 for Intel 64 architecture e 2 3 4 5 7 11 for IA 64 architecture Using Intel MKL Memory Management Intel MKL has memory management software that controls memory buffers for the use by the library functions New buffers that the library allocates when certain functions Level 3 BLAS or FFT are called are not deallocated until the program ends To get the amount of 6 17 6 Intel Math Kernel Library User s Guide memory allocated by the memory management software call the MKL_MemStat function If at some point your program needs to free memory it may do so with a call to MKL FreeBuffers If another call is made to a library function that needs a memory buffer then the memory manager will again allocate the buffers and they will again remain allocated until either the program ends or the program deallocates the memory This behavior facilitates better performance However some tools may report the behavior as a memory leak You can release memory in your program through the use of a function made available in Intel MKL or you can force memory releasing after each call by setting an environment variable The memory management software is turned on by default To disable the software
119. n However you are required to be familiar with building HPL and picking appropriate values for these variables New Features The toolset is basically identical with the HPL 1 0a distribution There are a few changes which are optionally compiled in and are disabled until you specifically request them These new features are ASYOUGO Provides non intrusive performance information while runs proceed There are only a few outputs and this information does not impact performance This is especially useful because many runs can go hours without any information ASYOUGO2 Provides slightly intrusive additional performance information because it intercepts every DGEMM ASYOUGO2_DISPLAY Displays the performance of all the significant DGEMMs inside the run ENDEARLY Displays a few performance hints and then terminates the run early FASTSWAP Inserts the LAPACK optimized DLASWP into HPL s code This may yield a benefit for Itanium 2 processor You can experiment with this to determine best results HYBRID Establishes the Hybrid OpenMP MPI mode of MP LINPACK providing the possibility to use threaded Intel MKL and prebuilt MP LINPACK hybrid libraries MPI library version 3 1 or higher You are also recommended to use the WARNING Use this option only with an Intel compiler and the Intel compiler version 10 0 or higher Benchmarking a Cluster To benchmark a cluster follow the sequence of steps maybe optional below Pay special
120. n the same directory Known limitations There are three kinds of limitations e functionality e performance e known bugs Functionality It is possible that some MKL functions will not work if called from the Java environment via a wrapper like those provided with the Intel MKL Java examples Only those specific CBLAS FFT VML VSL RNG and the convolution correlation functions listed in the Intel MKL Java examples section were tested with the Java environment So you may use the Java wrappers for these CBLAS FFT VML VSL RNG and convolution correlation functions in your Java applications Performance The functions from Intel MKL must work faster than similar functions written in pure Java However note that performance was not the main goal for these wrappers The intent was giving code examples So an Intel MKL function called from Java application will probably work slower than the same function called from a program written in C C or Fortran Known bugs There are a number of known bugs in Intel MKL identified in the Release Notes and there are incompatibilities between different versions of JDK The examples and wrappers include workarounds for these problems to make the examples work anyway Source codes of the examples and wrappers include comments which describe the workarounds Coding Tips This is another chapter whose contents discusses programming with Intel Math Kernel Library Intel MKL Whereas
121. nd Intel MPI 3 0 bin intel em 6 4t xhpl New Prebuilt hybrid binary for Intel 64 hybrid em64t E architecture Linux and Intel MPI 3 0 bin _intel ipf xhpl_ New Prebuilt hybrid binary for IA 64 architecture hybrid ipf Linux and Intel MPI 3 0 lib hybrid 32 libhpl_ hybrid a New Prebuilt library with the hybrid version of MP LINPACK for IA 32 architecture lib hybrid eme4t libhpl _ New Prebuilt library with the hybrid version of MP hybrid a LINPACK for Intel 64 architecture lib hybrid 64 libhpl hybrid a New Prebuilt library with the hybrid version of MP E E LINPACK for IA 64 architecture nodeperf c New Sample utility that tests the DGEMM speed across the cluster Building MP LINPACK There are a few included sample architecture makes It is recommended that you edit them to fit your specific configuration In particular e Set TOPdir to the directory MP LINPACK is being built in e You may set MPI variables that is MPdir MPinc and MPlib e Specify the location of Intel MKL and of files to be used LAdir LAinc LAlib e Adjust compiler and compiler linker options e Specify the version of MP LINPACK you are going to build hybrid or non hybrid by setting the version parameter for the make for example make arch em64t version hybrid install LINPACK and MP LINPACK Benchmarks 1 1 For some sample cases like Linux systems based on Intel 64 architecture the makes contain values that seem to be commo
122. ns on a single platform and should not be confused with MP LINPACK which is a distributed memory version of the same benchmark This benchmark should not be confused with LINPACK the library which has been expanded upon by the LAPACK library Intel is providing optimized versions of the LINPACK benchmarks to make it easier than using HPL for you to obtain high LINPACK benchmark results on your systems based on genuine Intel processors Use this package to benchmark your SMP machine Additional information on this software as well as other Intel software performance products is available at http developer intel com software products Contents The Intel Optimized LINPACK Benchmark for Linux contains the following files located in the benchmarks linpack subdirectory in the Intel MKL directory see Table 3 1 11 1 1 1 Intel Math Kernel Library User s Guide Table 11 1 Running the Software To obtain results for the pre determined sample problem sizes on a given system type one of the following as appropriate linpack_itanium linpack_xeon32 linpack_xeon64 runme_itanium runme_xeon32 runme_xeon64 lininput_itanium lininput_xeon32 lininput_xeon64 lin_itanium txt lin _xeon32 txt lin_xeon64 txt help lpk xhelp lpk Contents of the LINPACK Benchmark benchmarks linpack The 64 bit program executable for a system based on Intel Itanium 2 processor The 32 bit program executable for a
123. nteger type To define the MKL_ILP64 macro you may call the compiler with the DMKL_ILP64 command line option Intel MKL also defines the type MKL_LONG for maintaining ILP64 interface in the specific case of FFT interface for C C The MKL_LONG macro is defined as the standard C C type long by default and if the MKL_ILP64 macro is defined for the code compilation MKL LONG is defined as a 64 bit integer type NOTE NOTE The type int is 32 bit for the Intel C compiler as well as for most of modern C C compilers The type long is 32 or 64 bit for the Intel C and compatible compilers depending on the particular OS In the Intel MKL interface for the C or C languages that is in the h header files located in the Intel MKL include directory such function parameters as array sizes indices strides etc are declared as MKL_INT The FFT interface for C C is the specific case The header file mk1_dfti h uses the MKL_LONG type for both explicit and implicit parameters of the interface functions Specifically the type of the explicit parameter dimension of the function Df tiCreateDescriptor is MKL_LONG and the type of the implicit parameter length is MKL_ LONG for a one dimensional transform and MKL_LONG that is an array of numbers having type MKL_LONG for a multi dimensional transform Intel Math Kernel Library Structure 3 To bind your C C code with the ILP64 interface you must provide the DMKL_IL
124. ntel Math Kernel Library Structure 3 Table 3 6 Detailed directory structure continued Directory file Contents libmkl_pgi_thread a Parallel drivers library supporting PGI compiler libmkl_sequential a Sequential drivers library Computational layer libmkl_cdft a Dummy library Contains a reference to 1ib 32 libmk1_cdft_core a libmkl_cdft_core a Cluster version of FFTs libmkl_core a Kernel library for IA 32 architecture libmkl_ia32 a Dummy library Contains references to Intel MKL libraries 1ib 32 libmk1_intel a 1ib 32 libmk1l_ intel thread a and 1lib 32 libmk1l_core a libmkl_lapack a Dummy library Contains references to Intel MKL libraries 1ib 32 libmk1_intel a 1ib 32 libmkl_ intel thread a and 1lib 32 libmk1_core a libmkl_scalapack a Dummy library Contains a reference to 1lib 32 libmk1_scalapack_core a libmk1_scalapack_ ScaLAPACK routines core a libmkl_solver a Iterative Sparse Solver Trust Region Solver and GMP routines libmkl_solver_ Sequential version of Iterative Sparse Solver Trust Region sequential a Solver and GMP routines RTL layer libguide a Intel Legacy OpenMP run time library for static linking libiomp5 a Intel Compatibility OpenMP run time library for static linking libmkl_blacs a BLACS routines supporting the following MPICH versions Myricom MPICH version 1 2 5 10 e ANL MPICH version 1 2 5 2 libmkl_blacs_ BLACS routines supporting Intel MPI 2 0 and 3 0 and intelmpi a
125. ntel_thread 1lmkl_lapack lmkl_core liomp5 lpthread To link with Cluster FFT for a cluster of systems based on the IA 64 architecture use the following libraries opt intel mpi 3 0 bin mpiifort lt user files to link gt SMKLPATH libmk1l_cdft_core a SMKLPATH libmkl_blacs_intelmpi_ilp6 4 a SMKLPATH libmkl_ intel _ilp6 4 a SMKLPATH 1libmkl_intel_thread a SMKLPATH 1libmkl_core a liomp5 lpthread aa ae A binary linked with ScaLAPACK runs in the same way as any other MPI application For information refer to the documentation that comes with the MPI implementation For instance the script mpirun is used in case of MPICH2 and OpenMPI and a number of MPI processes is set by np In case of MPICH 2 0 and all Intel MPIs you should start the daemon before running an application the execution is driven by the script mpiexec For further linking examples see the Intel MKL support website at http www intel com support performancetools libraries mkl 9 5 Getting Assistance for Programming in the Fclipse IDE This chapter discusses features of the Intel Math Kernel Library Intel MKL which software engineers can benefit from when working in the Eclipse IDE The following features assist programming in the Eclipse IDE e The Intel MKL Reference Manual viewable from within the IDE e Eclipse Help search tuned to target the Intel Web sites e Context sensitive help in the Eclipse C C Development Tools CDT e Co
126. nthreaded code However it is thread safe which means that you can use it in a parallel region from your own OpenMP code You should use sequential version only if you have a particular reason not to use Intel MKL threading The sequential version layer may be helpful when using Intel MKL with programs threaded with non Intel compilers or in other situations where you may for various reasons need a non threaded version of the library To obtain sequential version of Intel MKL in the Threading layer choose the sequential library to link Note that the sequential library depends on the POSIX threads library pthread which is used to make the Intel MKL software thread safe and should be included in the link line See also Directory Structure in Detail Using Intel MKL Parallelism in chapter 6 1 Except for LAPACK deprecated routines lacon lasq3 and lasq4 Intel Math Kernel Library Structure 3 Avoiding Conflicts in the Execution Environment in chapter 6 Linking Examples in chapter 5 Support for ILP64 Programming The terms LP64 and ILP64 are used for certain historical reasons and due to the programming models philosophy described here http www unix org version2 whatsnew p64_wp html Intel MKL ILP64 libraries do not completely follow the programming models philosophy However the general idea is the same use 64 bit integer type for indexing huge arrays arrays with more than 23 1 elements It i
127. ol for details However Intel MKL can be aware that it is in a parallel region only if the threaded program and Intel MKL are using the same threading library If the user s program is threaded by some other means Intel MKL may operate in multithreaded mode and the performance may suffer due to overuse of the resources 6 3 6 Intel Math Kernel Library User s Guide Here are several cases with recommendations depending on the threading model you employ Table 6 1 model Threading model You thread the program using OS threads pthreads on the Linux OS You thread the program using OpenMP directives and or pragmas and compile the program using a compiler other than a compiler from Intel There are multiple programs running on a multiple cpu system as in the case of a parallelized program running using MPI for communication in which each processor is treated as a node How to avoid conflicts in the execution environment for your threading Discussion If more than one thread calls the library and the function being called is threaded it may be important that you turn off Intel MKL threading Set the number of threads to one by any of the available means see Techniques to Set the Number of Threads This is more problematic in that setting of OMP_NUM_THREADS in the environment affects both the compiler s threading library and Libiomp Libguide In this case you should try to choose the Threading layer libra
128. ory 6 e Ifyou are able to detect the presence of MPI but cannot determine if it has been called in a thread safe mode it is impossible to detect this with MPICH 1 2 x for instance and MKL_ DYNAMIC has not been changed from its default value of TRUE Intel MKL will run one thread MKL_DOMAIN_NUM_THREADS MKL DOMAIN NUM THREADS accepts a string value lt MKL env string gt which must have the following format lt MKL env string gt lt MKL domain env string gt lt delimiter gt lt MKL domain env string gt lt delimiter gt lt space symbol gt lt space symbol gt lt comma symbol gt lt semicolon symbol gt lt colon symbol gt lt space symbol gt lt MKL domain env string gt lt MKL domain env name gt lt uses gt lt number of threads gt lt MKL domain env name gt MKL_ALL MKL BLAS MKL_ FFT MKL VML lt uses gt lt space symbol gt lt space symbol gt lt equality sign gt lt comma symbol gt lt space symbol gt lt number of threads gt lt positive number gt lt positive number gt lt decimal positive number gt lt octal number gt lt hexadecimal number gt In the syntax above MKL_ BLAS indicates the BLAS function domain MKL_FFT indicates non cluster FFTs and MKL_VML indicates the Vector Mathematics Library For example MKL ALL 2 MKL BLAS 1 MKL FFT 4 MKL ALL 2 MKL BLAS 1 MKL FFT 4 MKL ALL 2 MKL BLAS 1 MKL FFT 4
129. ou can set the threshold in line 13 of the HPL 1 0a input file HPL dat LINPACK and MP LINPACK Benchmarks 1 1 If you are going to run a problem to completion do it with DASYOUGO see Options to reduce search time section 5 Using the quick performance feedback return to step 3 and iterate until you are sure that the performance is as good as possible Options to reduce search time Running huge problems to completion on large numbers of nodes can take many hours The search space for MP LINPACK is also huge not only can you run any size problem but over a number of block sizes grid layouts lookahead steps using different factorization methods etc It can be a large waste of time to run a huge problem to completion only to discover it ran 0 01 slower than your previous best problem There are 3 options you might want to experiment with to reduce the search time e DASYOUGO e DENDEARLY DASYOUGO2 Use cautiously as it does have a marginal performance impact To see DGEMM internal performance compile with DASYOUGO2 and DASYOUGO2_DISPLAY This will give lots of useful DGEMM performance information at the cost of around 0 2 performance loss If you want the old HPL back simply don t define these options and recompile from scratch try make arch lt arch gt clean_arch all DASYOUGO Gives performance data as the run proceeds The performance always starts off higher and then drops because this actually h
130. ously now is essentially exposed in the threading layer This layer is compiled for different environments threaded or sequential and compilers Intel gnu and so on Computational Layer This is the heart of Intel MKL and has only one variant for any processor operating system family such as 32 bit Intel processors on a 32 bit operating system The computational layer accommodates multiple architectures through identification of the architecture or architectural feature and chooses the appropriate binary code at execution Intel MKL may be thought of as the large computational layer that is unaffected by different computational environments Then as it has no RTL requirements RTLs refer not to the computational layer but to one of the layers above it the Interface layer or Threading layer The most likely case is matching the threading layer with the RTL layer RTL Layer This layer has run time library support functions For example 1ibiomp and 1libguide are RTLs providing threading support for the OpenMP threading in Intel MKL See also the Linking Examples section in chapter 5 Sequential Version of the Library Starting with release 9 1 the Intel MKL package provides support for sequential non threaded version of the library It requires no RTL layer that is no Compatibility OpenMP or Legacy OpenMP run time library and does not respond to the environment variable OMP_NUM_THREADS This version of Intel MKL runs u
131. pt mpich e SMKLPATH is a user defined variable containing lt mk1_directory gt lib 32 e You use the Intel C Compiler 9 1 or higher and the main module is in C To link with ScaLAPACK for a cluster of systems based on the IA 32 architecture use the following libraries opt mpich bin mpicce lt user files to link gt LSMKLPATH lmk1_scalapack_core lmkl_blacs_intelmpi lmkl_lapack lmkl_intel lmkl_intel_ thread 1lmkl_lapack lmkl_core liomp5 lpthread POG gO gO To link with Cluster FFT for a cluster of systems based on the IA 32 architecture use the following libraries opt mpich bin mpicce lt user files to link gt SMKLPATH libmkl_cdft_core a SMKLPATH libmkl_blacs_intelmpi a SMKLPATH libmkl_intel a SMKLPATH libmkl_intel_thread a SMKLPATH libmkl_core a liomp5 lpthread a an an an an E Examples for Fortran Module Suppose the following conditions are met e Intel MPI 3 0 is installed in opt intel mpi 3 0 e SMKLPATH is a user defined variable containing lt mk1_directory gt lib 64 e You use the Intel Fortran Compiler 9 1 or higher and the main module is in Fortran To link with ScaLAPACK for a cluster of systems based on the IA 64 architecture use the following libraries opt intel mpi 3 0 bin mpiifort lt user files to link gt LSMKLPATH lmkl_scalapack_1p64 lmkl1_blacs_intelmpi_ 1p64 Bo ee ge Working with Intel Math Kernel Library Cluster Software 9 lmk1_lapack lmkl_intel_1p64 lmkl_i
132. rary Structure 3 Interfaces On Linux systems based on IA 64 architecture the Intel Fortran Compiler returns complex values differently than gnu and some other compilers Rather than duplicate the library for these differences separate interface libraries are provided to support compiler differences while constraining the size of the library Similarly LP64 can be supported on top of ILP64 through an interface Moreover interface libraries are provided to support legacy supercomputers where single precision means 64 bit arithmetic Threading For efficiency reasons Intel MKL employs function level threading throughout the library rather than loop level threading Consequently all threading can be constrained to a relatively small set of functions and collected into a library All references to compiler specific run time libraries are generated in these functions By compiling them with different compilers and providing a threading library layer Intel MKL can work in programs threaded with Intel compilers and other supported threading compilers A non threaded library version can also be obtained by turning off threading when compiling the threading library layer because all threading is provided through OpenMP technology Computation For any given processor family processors based on IA 32 IA 64 or Intel 64 architecture a single computational library is used for all interfaces and threading layers because there is no parallelism in the
133. read a MKLPATH libmkl_core a W1 end group liomp5 lpthread 2 Dynamic linking of user s code myprog f and parallel Intel MKL ifort myprog f LSMKLPATH ISMKLINCLUDE lmkl_intel 1lmkl_intel_ thread lmkl_core liomp5 lpthread 3 Static linking of user s code myprog f and sequential version of Intel MKL ifort myprog f LSMKLPATH ISMKLINCLUDE Wl start group MKLPATH libmkl_intel a SMKLPATH libmkl_sequential a SMKLPATH libmkl_core a W1 end group lpthread 4 Dynamic linking of user s code myprog f and sequential version of Intel MKL ifort myprog f LSMKLPATH ISMKLINCLUDE lmk1l_intel 1lmkl_sequential lmkl_core lpthread 5 Static linking of user s code myprog f Fortran 95 LAPACK interfacet and parallel Intel MKL See Fortran 95 Interfaces and Wrappers to LAPACK and BLAS in chapter 7 for information on how to build Fortran 95 LAPACK and BLAS interface libraries 5 8 Linking Your Application with Intel Math Kernel Library 5 ifort myprog f LSMKLPATH ISMKLINCLUDE 1lmk1l_lapack95 Wl start group MKLPATH libmkl_intel a SMKLPATH libmk1l intel thread a MKLPATH libmkl_core a W1 end group liomp5 lpthread Static linking of user s code myprog f Fortran 95 BLAS interfacet and parallel Intel MKL ifort myprog f LSMKLPATH ISMKLINCLUDE lmkl_blas95 Wl start group MKLPATH libmkl_intel a SMKLPATH libmk1l intel thread a MKLPATH libmkl_core a W1 end group liomp5 lpthread Stati
134. rectory gt lib 32 libmkl_core a lt mkl directory gt lib 32 libiomp5 so lpthread where lt 1ds gt is a linker myprog o is the user s object file Appropriate Intel MKL libraries are listed first and followed by the system library libpthread In the link line list library names with absolute or relative paths if needed preceded with L lt paths which indicates where to search for binaries and I lt include gt which indicates where to search for header files Discussion of linking with Intel MKL libraries employs this option To link with Intel MKL libraries specify paths and libraries in the link line as shown below NOTE NOTE The syntax below is provided for dynamic linking For static linking replace each library name preceded with 1 with the path to the library file for example replace 1mk1_core with SMKLPATH 1libmkl_core a where SMKLPATH is the appropriate user defined environment variable See specific examples in the Linking Examples section lt files to links L lt MKL path gt I lt MKL include gt lmkl_lapack95 1mk1_blas95 cluster components 5 3 5 Intel Math Kernel Library User s Guide lmkl_ intel intel_ilp 4 intel _1p 64 intel _sp2dp gf gf _ilp 64 gf_lp64 lmkl_ intel_thread gnu_thread pgi_thread sequential lmkl_solver lmkl_solver_1p64 1lmkl_solver_il1p64 lmkl_lapack lmkl_ 1ia32 em64t ipf 1lmk1_core liomp5 lguide lpthread
135. rocessor optimized kernels threading library and system library for threading support linked as described at the beginning of section Link Command Syntax in Chapter 5 lt MKL LAPACK amp kernel libraries gt are the LAPACK library and lt MKL kernel libraries gt grouping symbols W1 start group and W1 end group are required in case of static linking For example if you are using Intel MPI 3 x wish to statically use the LP64 interface with ScaLAPACK and to have only one MPI process per core and thus do not employ threading provide the following linker options LSMKLPATH ISMKLINCLUDE W1 start group SMKLPATH libmk1l_scalapack_1p64 a SMKLPATH libmkl blacs_intelmpi_lp 4 a SMKLPATH 1libmkl_ intel 1lp6 4 a MKLPATH libmkl1_sequential a SMKLPATH libmkl_core a static_mpi Wl end group lpthread 1m For more examples see Examples for Linking with ScaLAPACK and Cluster FFT Note that lt lt MPI gt linker script gt and lt BLACS gt library should correspond to the MPI version For instance if it is Intel MPI 2 x then lt Intel MPI 2 x linker script gt and libmkl_blacs_intelmpi libraries are used To link with Intel MPI 3 0 or 3 1 libmkl_blacs_intelmpi should also be used For information on linking with Intel MKL libraries see Chapter 5 Linking Your Application with Intel Math Kernel Library Setting the Number of Threads 9 2 The OpenMP software responds to the environmental variable OMP_NUM_THR
136. ry that matches the layered Intel MKL with the OpenMP compiler you employ see Linking Examples on how to do this If this is impossible the sequential version of Intel MKL can be used as the Threading layer To do this you should link with the appropriate Threading layer library libmk1_sequential aor libmk1l_ sequential so see the High level Directory Structure section in chapter 3 The threading software will see multiple processors on the system even though each processor has a separate MPI process running on it In this case set the number of threads to one by any of the available means see Techniques to Set the Number of Threads To avoid correctness and performance problems you are also strongly encouraged to dynamically link with the Intel Compatibility OpenMP run time library 1ibiomp and Intel Legacy OpenMP run time library libguide Setting the Number of Threads Using OpenMP Environment Variable You can set the number of threads using the environment variable OMP_NUM_THREADS To change the number of threads in the command shell in which the program is going to run enter 6 4 Managing Performance and Memory 6 export OMP NUM THREADS lt number of threads to use gt for certain shells such as bash or set OMP NUM THREADS lt number of threads to use gt for other shells such as csh or tcsh See Using Additional Threading Control on how to set the number of threads using Intel MKL env
137. s whereas DASYOUGO prints about 46 or so outputs before the problem completes Performance for DASYOUGO and DENDEARLY always starts off at one speed slowly increases and then slows down toward the end because that is what LU does DENDEARLY is likely to terminate before it starts to slow down DENDEARLY terminates the problem early with an HPL Error exit It means that you need to ignore the missing residual results which are wrong as the problem never completed However you can get an idea what the initial performance was and if it looks good then run the problem to completion without DENDEARLY To avoid the error check you can set HPL s threshold parameter in HPL dat toa negative number Though DENDEARLY terminates early HPL treats the problem as completed and computes Gflop rating as though the problem ran to completion Ignore this erroneously high rating The bigger the problem the more accurately the last update that DENDEARLY returns will be close to what happens when the problem runs to completion DENDEARLY is a poor approximation for small problems It is for this reason that you are suggested to use ENDEARLY in conjunction with ASYOUGO2 because ASYOUGO2 reports actual DGEMM performance which can be a closer approximation to problems just starting The best known compile options for Itanium 2 processor are with the Intel compiler and look like this 02 ipo ipo_obj ftz IPF fltacc
138. s Auxiliary and utility LAPACK routines Parallel Basic Linear Algebra Subprograms PBLAS ScaLAPACK routines DSS PARDISO solvers Other Direct and Iterative Sparse Solver routines Vector Mathematical Library VML functions Vector Statistical Library VSL functions Fourier Transform functions FFT Cluster FFT functions A 1 A Intel Math Kernel Library User s Guide Table A 1 Intel MKL language interfaces support continued FORTRAN 77 Fortran 90 95 Function Domain interface interface Trigonometric Transform routines Fast Poisson Laplace and Helmholtz Solver Poisson Library routines Optimization Trust Region Solver routines GMP arithmetic functions Service routines including memory allocation Supported using a mixed language programming call See Table A 2 for the respective header file Table A 2 A 2 Table A 2 lists available header files for all Intel MKL function domains Function domain All function domains BLAS Routines BLAS like Extension Transposition Routines CBLAS Interface to BLAS Sparse BLAS Routines LAPACK Routines ScaLAPACK Routines All Sparse Solver Routines PARDISO DSS Interface e RCI Iterative Solvers e ILU Factorization Optimization Solver Routines Vector Mathematical Functions Intel MKL include files Fortran mkl fi mkl_blas f90 mkl _blas fi mkl_trans fi mkl_spblas
139. s ea eeaeaes 6 18 Language specific Usage Options Using Language Specific Interfaces with Intel MKL cceeeeeeeaes 7 1 Mixed language programming with Intel MKL ceseee ener eee 7 4 Calling LAPACK BLAS and CBLAS Routines from C Language Environments viii th evade tenet ita E tite nave a A a it dy eine heave 7 4 Using Complex Types in C CH H vo ceeceececceseeeeeeeeeeeeeeeeeeeeeaeeaeennenas 7 6 Calling BLAS Functions That Return the Complex Values in C C COren nein OE toes doko evils Gag ta cae wer eaneu nee anes te 7 7 Support for Boost UBLAS Matrix Matrix Multiplication 0 7 10 Invoking Intel MKL Functions from Java Applications 7 12 Coding Tips Aligning Data for Numerical Stability 0 0 0 ccececceeeeeeee eee eee eee eeeenaees 8 1 Working with Intel Math Kernel Library Cluster Software Linking with ScaLAPACK and Cluster FFTS ecceeeeeeeeeeeeeeeeeaeees 9 1 Setting the Number of Threads cceeceeeeeee cnet ee eee eee eeeeeeae tana 9 2 Using Shared Libraries sss stedene eeni eee eee ee eee eee teen eens eens nena 9 3 ScaLAPACK Teste veiii ai dni te deen mesa eh eet ee 9 3 Examples for Linking with ScaLAPACK and Cluster FFT 20 0 9 3 Examples for C MOdUI cccccceceeeee eee e eee ee eens tees eee ee eee seat ean enanias 9 4 Examples for Fortran MOdule ccccccceseeeeeeeeeeeeeeeeeaeeeeenaeeaeenaenas 9 4 vi Intel Math Kernel Library User
140. s that are resolved in its run time library RTL Linking of such code without the appropriate RTL will result in undefined symbols MKL has been designed to minimize RTL dependencies 7 3 7 Intel Math Kernel Library User s Guide Where the dependencies do arise supporting RTL is shipped with Intel MKL The only examples of such RTLs except those that are relevant to the Intel MKL cluster software are libiomp and libguide which are the libraries for the OpenMP code compiled with an Intel compiler 1ibiomp and libguide support the threaded code in Intel MKL In other cases where RTL dependencies might arise the functions are delivered as source code and it is the responsibility of the user to compile the code with whatever compiler employed In particular Fortran 90 modules result in the compiler specific code generation requiring RTL support so Intel MKL delivers these modules as source code Mixed language programming with Intel MKL Appendix A lists the programming languages supported for each Intel MKL function domain However you can call Intel MKL routines from different language environments This section explains how to do this using mixed language programming Calling LAPACK BLAS and CBLAS Routines from C Language Environments Not all Intel MKL function domains support both C and Fortran environments To use Intel MKL Fortran style functions in C C environments you should observe certain conventions which ar
141. s up to you to choose which interface to use You should definitely choose LP64 interface for compatibility with the previous Intel MKL versions because LP64 is just a new name for the only interface that the Intel MKL versions lower than 9 1 provided You should definitely choose the ILP64 interface if your application uses Intel MKL for calculations with huge data arrays of more than 231 1 elements or the library may be used so in future The LP64 and ILP64 interfaces are supported in the Interface layer Once the appropriate library in the Interface layer is selected see Directory Structure in Detail all libraries below the Interface layer are compiled using the chosen interface As the differences between the ILP64 and LP64 interfaces are out of scope of the Intel MKL Reference Manual you are encouraged to browse the include files examples and tests for the ILP64 interface details To do this see the following directories respectively lt mkl directory gt include lt mkl directory gt examples lt mkl directory gt tests This section shows e How the ILP64 concept is implemented specifically for Intel MKL e How to compile your code for the ILP64 interface e How to code for the ILP64 interface e How to browse the Intel MKL include files for the ILP64 interface This section also explains limitations of the ILP64 support Concept ILP64 interface is provided for the following two reasons e To support huge data arrays w
142. se of the non hybrid build In addition to supplying certain hybrid prebuilt binaries Intel MKL supplies certain hybrid prebuilt libraries to take advantage of the additional OpenMP optimizations LINPACK and MP LINPACK Benchmarks 1 1 Note that the non hybrid version may be used in a hybrid mode but it would be missing some of the optimizations added to the hybrid version Non hybrid builds are the default In many cases the use of the hybrid mode is required for system reasons but if there is a choice the non hybrid code may be faster although that may change in future releases To use the non hybrid code in a hybrid mode use the threaded MPI and Intel MKL link with a thread safe MPI and call function MPI_init_thread so as to indicate a need for MPI to be thread safe Contents The Intel Optimized MP LINPACK Benchmark for Clusters includes the HPL 1 0a distribution in its entirety as well as the modifications delivered in the files listed in Table 11 2 and located in the benchmarks mp_linpack subdirectory in the Intel MKL directory see Table 3 1 Table 11 2 Contents of the MP LINPACK Benchmark benchmarks mp_linpack testing ptest HPL pdtest c src blas HPL dgemm c src grid HPL grid_init c src pgesv HPL_ pdgesvK2 c include hpl_misc h and hpl_pgesv h src pgesv HPL_ pdgesv0 c testing ptest HPL dat Make ia32 Make em64t Make ipf HPL dat HPL 1 0a code modified to display captured DGEMM information in
143. some undefined symbolic references until run time Dynamically built executables contain those symbols along with a list of libraries that provide definitions of the symbols When the executable is loaded the final linking is done before the application runs If several dynamically built executables reference the same 5 1 5 Intel Math Kernel Library User s Guide library it is loaded into memory only once and the executables share it thereby saving memory Dynamic linking enables you to separately update the libraries on which applications depend and does not require relinking the applications The development advantages of dynamic linking are achieved at some cost to performance because every unresolved symbol has to be looked up in a dedicated table and resolved at run time Making the Choice It is up to you to select whether to link in Intel MKL libraries dynamically or statically when building your application Table 5 1 If you are developing just a single application and want to ship only that executable use static linking To reduce the size of executables shipped you can also build custom dynamic libraries see Building Custom Shared Objects Table 5 1 compares the linkage models Feature Processor Updates Optimization Build Calling Total Binary Size Executable Size Multi threaded thread safe Dynamic Linkage Automatic All processors Link to dynamic libraries Regular names Large Smallest Y
144. stalled and configured 1 Check that the directory you chose for the installation has been created The Intel MKL default installation directory may be one of the following opt intel mk1 RR r y xxx where RR r is the version number y is the release update number and xxx is the package number for example opt intel mk1 10 1 0 004 lt Intel Compiler Pro directory gt mk1 where lt Intel Compiler Pro directory gt is the installation directory for Intel C Compiler Professional Edition or Intel Fortran Compiler Professional Edition for example opt intel Compiler 11 0 015 mk1 If you choose to keep multiple versions of Intel MKL installed on your system update build scripts so that they point to the desired version Check that the following six files are placed in the tools environment directory kivars32 sh kilvars32 csh kivarsem6 4t sh kivarsem6 4t csh kilvars64 sh m m m m m m klvars64 csh You can use these files to set environmental variables such as INCLUDE LD_LIBRARY_PATH MANPATH LIBRARY PATH CPATH and FPATH in the current user shell 2 1 2 Intel Math Kernel Library User s Guide Obtaining Version Information Intel MKL provides Intel MKL provides two methods for obtaining information about the current library for example the version number The function MKLGet VersionString extracts a version string and the function MKLGet Version can be used to obtain the
145. supporting Intel MPI 2 0 and 3 0 and MPICH2 LP64 version of BLACS routines supporting Intel MPI 2 0 and 3 0 and MPICH2 A soft link to lib em64t libmkl_blacs_intelmpi_ilp64 a A soft link to lib em64t libmkl_blacs_intelmpi_lp64 a LP64 version of BLACS routines supporting the following MPICH versions Myricom MPICH version 1 2 5 10 e ANL MPICH version 1 2 5 2 ILP64 version of BLACS routines supporting OpenMPI LP64 version of BLACS routines supporting OpenMPI ILP64 version of BLACS routines supporting SGI MPT LP64 version of BLACS routines supporting SGI MPT ILP64 interface library for GNU Fortran compiler LP64 interface library for GNU Fortran compiler ILP64 interface library for Intel compiler LP64 interface library for Intel compiler SP2DP interface library for Intel compiler 3 15 3 Intel Math Kernel Library User s Guide Table 3 6 Directory file Detailed directory structure continued Threading layer libmk1_gnu_thread so libmk1l_intel_ thread so libmk1_pgi_thread so libmk1_sequential so Computational layer libmk1l so libmkl_core so libmk1_def so libmk1l_mc so libmk1_mc3 so libmkl_lapack so libmk1_scalapack_ ilp64 so libmk1l_scalapack_ 1p64 so libmk1_vml_def so libmk1_vml_mc so libmk1_vml_mc3 so libmk1l_vml_p4n so libmk1l_vml_mc2 so RTL layer libguide so libiomp5 so Contents Parallel drivers library supporting GNU compiler Parallel driv
146. system based on Intel Xeon processor or Intel Xeon processor MP with or without Streaming SIMD Extensions 3 SSE3 The 64 bit program executable for a system with Intel Xeon processor using Intel 64 architecture A sample shell script for executing a pre determined problem set for linpack_itanium OMP_NUM_THREADS set to 8 processors A sample shell script for executing a pre determined problem set for linpack_xeon32 OMP_NUM_THREADS set to 2 processors A sample shell script for executing a pre determined problem set for linpack_xeon64 OMP_NUM_THREADS set to 4 processors Input file for pre determined problem for the runme_itanium script Input file for pre determined problem for the runme_xeon32 script Input file for pre determined problem for the runme_xeon64 script Result of the runme_itanium script execution Result of the runme_xeon32 script execution Result of the runme_xeon 64 script execution Simple help file Extended help file runme_itanium runme_xeon32 runme_xeon64 LINPACK and MP LINPACK Benchmarks 1 1 To run the software for other problem sizes please refer to the extended help included with the program Extended help can be viewed by running the program executable with the e option xlinpack_itanium e xlinpack_xeon32 e xlinpack_xeon64 e The pre defined data input files Lininput_itanium lininput_xeon32 and lininput_xeon 4 are provided merely as examples
147. t absolute path to installed MKL gt tools environment mklvars lt arch gt csh In the above commands mklvars lt arch gt stands for each of mklvars32 mklvarsem64t or mklvars64 If you have super user permissions you can add the same commands to a general system file in etc profile for bash and sh or in etc csh login for csh Before uninstalling Intel MKL remove the above commands from all profile files where the script execution was added to avoid problems during logging in Configuring the Eclipse IDE CDT to Link with Intel MKL This section describes how to configure the Eclipse IDE C C Development Tools CDT 3 x and 4 0 to link with Intel MKL TIP After linking your CDT with Intel MKL you can benefit from the Eclipse provided code assist feature See Code Context Assist description in Eclipse Help Configuring the Eclipse IDE CDT 4 0 4 2 To configure Eclipse CDT 4 0 to link with Intel MKL follow the instructions below Configuring Your Development Environment 4 If the tool chain compiler integration supports include path options go to the Includes tab of the C C General gt Paths and Symbols property page and set the Intel MKL include path that is lt mkl directory gt include If the tool chain compiler integration supports library path options go to the Library Paths tab of the C C General gt Paths and Symbols property page and set the Intel MKL library path depending upon the target architectur
148. tarted employing the OpenMP technology for threading Starting with version 10 0 Intel MKL supports implementations of OpenMP that those compilers provide If an application compiled with such a threading compiler used OpenMP threading and called threaded parts of Intel MKL versions lower than 10 0 there might be difficulties They may arise because Intel MKL is threaded using Intel compilers and threading libraries from different compilers are not compatible This can lead to performance issues and perhaps even failures when incompatible threading is used within the same application Starting with Intel MKL 10 0 several solutions are available in certain cases Those solutions are provided both from the Threading Layer and the supplied run time libraries found in the RTL Layer Threading Layer Starting with version 10 0 Intel MKL is structured as layers One of those layers is a Threading Layer Because of the internal structure of the library all of the threading represents a small amount of code This code is compiled by different compilers Intel gnu and PGI compilers on Linux and the appropriate layer linked in with the threaded application RTL Layer The second relevant component is the Compiler Support RTL Layer Prior to Intel MKL 10 0 this layer included only the Intel Legacy OpenMP run time compiler library libguide Now you have a new choice to use the Intel Compatibility OpenMP run time compiler library libiomp The Comp
149. tions on each thread You may obtain higher performance when using Intel MKL without HT Technology enabled See Using Intel MKL Parallelism for information on the default number of threads changing this number and other relevant details If you run with HT enabled performance may be especially impacted if you run on fewer threads than physical cores Moreover if for example there are two threads to every physical core the thread scheduler may assign two threads to some cores and ignore the other ones altogether If you are using the OpenMP library of the Intel Compiler read the respective User Guide on how to best set the affinity to avoid this situation For Intel MKL you are recommended to set KMP_AFFINITY granularity fine compact 1 0 Managing Multi core Performance You can obtain best performance on systems with multi core processors by requiring that threads do not migrate from core to core To do this bind threads to the CPU cores by setting an affinity mask to threads You can do it using any of the following options e OpenMP facilities recommended if available for instance the KMP_AFFINITY environment variable using the Intel OpenMP library e A system function as in the example below Suppose e The system has two sockets with two cores each e 2 threads parallel application that calls the Intel MKL FFT happens to run faster than in 4 threads but the performance in 2 threads is very unstable 6 15 6 Intel Mat
150. tly in a threaded environment see the Known Limitations section in the Release Notes The safest thing for multiple CPUs although not necessarily the fastest is to run one MPI process per processor with OMP_NUM_ THREADS set to one Always verify that the combination with OMP_NUM_THREADS 1 works correctly Using Shared Libraries All needed shared libraries must be visible on all the nodes at run time One way to accomplish this is to point these libraries by the LD_LIBRARY_PATH environment variable in the bashrc file If Intel MKL is installed only on one node you should link statically when building your Intel MKL applications The Intel compilers or GNU compilers can be used to compile a program that uses Intel MKL However make certain that MPI implementation and compiler match up correctly ScaLAPACK Tests To build NetLib ScaLAPACK tests e for IA 32 architecture add libmk1_scalapack_core a to your link command e for IA 64 and Intel 64 architectures add libmk1_scalapack_1p64 aor libmkl_scalapack_ilp64 a depending upon the desired interface Examples for Linking with ScaLAPACK and Cluster FFT For information on detailed MKL structure of the architecture specific directories of the cluster libraries see section Directory Structure in Detail in Chapter 3 9 3 9 Intel Math Kernel Library User s Guide Examples for C Module Suppose the following conditions are met e MPICH2 1 0 7 or higher is installed in o
151. ut temporary matrices uBLAS distinguishes two modes e Debug safe mode default Type and conformance checking is performed e Release fast mode Turned on with NDEBUG preprocessor symbol The documentation for the latest version of the Boost uBLAS is available at www boost org doc libs 1_35_0 libs numeric ublas doc index htm Intel MKL provides overloaded prod functions for substituting uBLAS dense matrix matrix multiplication with the Intel MKL gemm calls Though these functions break uBLAS expression templates and introduce temporary matrices the performance advantage can be considerable for matrix sizes that are not too small roughly over 50 You do not need to change your source code to use the functions To call them e Include the header file mk1_ boost _ublas matrix _prod hpp in your code e Add appropriate Intel MKL libraries to the link line refer to the Linking Your Application with Intel Math Kernel Library chapter for details Only the following expressions are substituted prod ml m2 prod trans m1 m2 prod trans conj m1 m2 prod conj trans m1 m2 prod ml trans m2 prod trans m1 trans m2 prod trans conj m1 trans m2 prod conj trans m1 trans m2 prod ml trans conj m2 prod trans m1 trans conj m2 prod trans conj m1 trans conj m2 prod conj trans m1 trans conj m2 prod ml conj trans m2 prod trans m1
152. version of libiomp is found and used at run time Building Custom Shared Objects Custom shared objects enable reducing the collection of functions available in Intel MKL libraries to those required to solve your particular problems which helps to save disk space and build your own dynamic libraries for distribution Intel MKL Custom Shared Object Builder Custom shared object builder is targeted for creation of a dynamic library shared object with selected functions and located in tools builder directory The builder contains a makefile and a definition file with the list of functions The makefile has three targets ia32 ipf and em64t ia32 target is used for processors using IA 32 architecture ipf is used for IA 64 architecture and em64t is used for Intel Xeon processor using Intel 64 architecture 5 15 5 Intel Math Kernel Library User s Guide Specifying Makefile Parameters There are several macros parameters for the makefile export user list specifies the full name of the file that contains the list of entry point functions to be included into shared object This file is used for definition file creation and then for export table creation The default name is user_list no extension name mkl_custom specifies the name of the created library By default the library mk1_ custom so is built xerbla user_xerbla o specifies the name of the object file that contains the user s error handler
153. y currently does not comply with the layered model So it is not changed internally with respect to the Intel MKL 9 x However to support LP64 ILP64 interfaces two libraries were introduced in the unified structure libmkl_solver_1p 64 a for the LP64 interface and libmkl_solver_ilp64 a for the ILP64 interface For backward link line compatibility libmkl_solver a has become a dummy library There is still only static version of the solver library as it was for previous releases To link with the Iterative Sparse Solver and Trust Region Solver routine library using the layered model include the library libmk1_solver_1p64 a or libmk1l_solver_ilp64 a in the link line depending upon the interface you need i NOTE In MKL 10 1 Gold the DSS PARDISO solver functionality was excluded from libmk1_solver a libraries and integrated into the Intel MKL layered structure So to use DSS PARDISO it is no longer necessary to link with Libmk1_solver a but the former link line is still working Note that both static and dynamic libraries are now available for DSS PARDISO libmk1_lapack95 a and libmk1_blas95 a libraries contain LAPACK95 and BLAS95 interfaces respectively They are not included into the original distribution and should be built before using the interface See Fortran 95 Interfaces and Wrappers to LAPACK and BLAS and Compiler dependent Functions and Fortran 90 Modules in chapter 7 for details on building the libraries and on why

Intel® Math Kernel Library User's Guide

Contents

Download Pdf Manuals

Related Search

Related Contents

Intel&reg; Math Kernel Library User's Guide

Contents

Download Pdf Manuals

Related Search

Related Contents

Intel® Math Kernel Library User's Guide