Perl Extension Building with SWIG
Contents
1. Therefore it is impossible to take a Perl array and pass it in unmodified form to an arbitrary C function The difference between Perl and C datatypes often arises with C functions such as the following Plot some points void plotpts gdImagePtr im int x int npts int c int yll for int i 0 i lt npts itt gdImageSetPixel im x i yl il c Ideally a user might want to pass Perl arrays as argu ments as follows a 10 20 30 40 b 50 70 60 200 gd plotpts im a b 4 1 Error However this script generates a type error instead of acting as one might expect While such behavior may seem restrictive or bizarre SWIG has been deliberately designed to operate in this manner In fact there are even benefits to this approach If Perl arrays were to be used as C arrays a copy would be made verified for type correctness and deallocated every time an array was passed to a C function For large arrays this would introduce a substantial performance overhead Space re quirements are also a concern for some C programs For example a numerical application might manipulate ar rays with millions of elements Converting such arrays to and from a Perl representation would clearly introduce substantial memory and performance overhead In con trast manipulating pointers to such arrays is easy and efficient It should also be noted that SWIG provides a variety of customization options that can2. for input parameter processing out for returning values check for checking values and so forth Typemaps al low a user to re implement these operations for specific datatypes by supplying small fragments of C code that SWIG inserts into the resulting wrapper code To illustrate consider the gd example In the origi nal interface file two functions were included to open and close files These were required because SWIG nor mally maps all pointers including files into blessed ref erences Since a blessed reference is not the same as a Perl file handle it is not possible to pass Perl files to functions expecting a FILE However this is easily changed with a typemap as follows stypemap perl5 in FILE target IolFP sv_2io Ssource This declaration tells SWIG that whenevera FILE appears as a function parameter it should be converted using the supplied C code When generating wrap pers the typemap code is inserted into all wrapper func tions where a FILE is involved In the process the Ssource and target tokens are replaced by the names of C local variables corresponding to the Perl and C representations of an object respectively As a result this typemap allows Perl files to be used in a natural manner For example open OUT gt test gif die error n Much better than before gd gdImageGif Sim OUT Certain operations such as output values are imple mented using
3. SWIG by default han dles all pointers in a uniform manner Thus datatypes such as FILE are represented as blessed references even though such types may appear remarkably similar to other Perl datatypes such as file handles 3 3 2 Arrays SWIG maps all arrays into pointers where the value of an array is simply a pointer to the first element in the ar ray This is the same model used by C compilers and like C SWIG performs no bounds or size checking Thus a function such as void foo double a 4 4 would accept any object of type double It is up to the user to ensure that the pointer is valid and that it points to memory that has been properly allocated 3 3 3 Structures and Objects Finally all structures and objects are represented as pointers This includes cases where objects are manipu lated by value For example the functions double dot_product Vector a Vector b Vector cross_product Vector a Vector b are transformed by SWIG into the following wrappers double wrap_dot_product Vector a Vector b return dot_product a b Vector wrap_cross_product Vector a Vector b Vector r r Vector malloc sizeof Vector r cross_product a b return r The representation of objects by reference avoids the problem of marshaling objects between a C and Perl representation a process that would be extremely dif ficult for very complicated C datatypes It also pro v
4. Spt gt x 20 Spt gt y 50 Spt gt output Create an array of points Spts new gdPoint 10 for i 0 i lt 10 i Sp Spts gt get i Sp gt x i Sp gt y 10 i Pass the points to a function gd gdImagePolygon S im Spts 10 1 The class extension mechanism is also a powerful way to repackage existing functionality For example the gdImage structure and various functions in the gd li brary could be combined into a Perl class as follows Saddmethods gdImage gdImage int w int h return gdImageCreate w h gdImage gdImageDestroy self int colorAllocate int r int g int b return gdImageColorAllocate self r g b void line int xl int yl int x2 int y2 int c gdImageLine self xl yl x2 y2 c hi Users can now write scripts as follows usr bin perl use gd Sim new gdImage 400 400 Sblack im gt colorAllocate 0 0 0 Swhite im gt colorAllocate 255 255 255 Sim gt line 20 50 180 140 white With these simple modifications our interface is al ready looking remarkably similar to that used in the GD module on CPAN However more improvements will be described shortly 3 8 Access Control and Naming In certain instances it may be useful to restrict access to certain variables and class members Hiding objects is easy simply remove them from the interface file Pro viding read only access can be accomplished using the Sr
5. can be extremely troublesome to incorporate into Perl This is especially the case for C programs that over ride the standard behavior of pointers and deferenc ing operations operations that are used extensively by SWIG generated wrapper code In addition SWIG does not provide quite as much flexibility as xsubpp and other Perl specific extension building tools In order to be general purpose SWIG hides many of the internal implementation details of each scripting language As a result it can be difficult to accomplish certain tasks For example one such sit uation is the handling of functions where arguments are implicitly related to each other as follows void foo char str int len str string data len length of string data Ideally it might be desirable to pass a single Perl string to such a function and have it expanded into a data and length component Unfortunately SWIG has no way to know that the arguments are related to each other in this manner Furthermore the current typemap mechanism only applies to single arguments so it can not be used to combine arguments in this manner XS on the other hand is more closely tied to the Perl inter preter and consequently provides more power in the way that arguments can be converted and passed to C func tions Finally SWIG is still somewhat immature with re spect to its overall integration with Perl For example SWIG does not fully support Perl s package and
6. cases can be implemented as Perl scripts As a result the compile execute cycle is no longer a problem and Perl scripts can be used to implement common parts of various test cases This section describes the integration of Perl with an API that is part of a HP OpenCall telecom product de veloped at HP Grenoble The API provides access to the TCAP and SCCP layers of the SS7 protocol and con sists of about 20 function and 60 structure declarations Furthermore most function parameters are pointers to deeply nested structures such as follows typedef enum pa leases hatira typedef struct be addres e eee nature te a aie ree typedef struct tc_address_struct te grobal titie gt to addrass From a Perl users point of view the functionality of fered by the SWIG generated module must be not be very different from the underlying C API Otherwise test writers may be confused by the Perl API and testing will be unnecessarily complicated Fortunately SWIG addresses this problem because Perl interfaces are spec ified using C syntax and the resulting interface closely resembles the original API 5 2 1 Creating the SWIG Interface To wrap the C API there were three choices copy and modify the header files into a SWIG interface file feed the header files directly to SWIG or write an interface file that includes some parts of the header files The first choice requires the duplication of C definitions a task that is
7. pSV PUTBACK int n perl_call_sv this gt pPerl5Fcn G_SCALAR SPAGAIN if n 1 n POPi PUTBACK FREETMPS LEAVE return n 0 false true 5 1 3 Benefits And Limitations The benefits that SWIG provides to BADGER are enor mous e Not counting custom code e g language specific callbacks an extension language interface can be developed in a day compared with weeks for a hand crafted approach e SWIG supports the use of multiple extension lan guages with ease e The resulting solution is flexible and the results can be tailored to meet the needs of complex applica tions e g callbacks smart references and so on SWIG does have limitations but so far none of these limitations has proven to be a real impediment It also appears that most of these limitations will be eradicated once SWIG has its own extension language interface see Section 7 5 2 TCAP and SCCP from HP OpenCall One of the well known pitfalls of systematic library testing is the creation of a huge number of small C programs each designed to perform a single test More often than not these C programs have a lot of common code that is copied from one test case to the other Test ing is further complicated by the tedious process of edit ing compiling and executing each of these programs To solve this problem SWIG can be used to incor porate libraries into Perl extension modules where test
8. char temp 512 Starget temp Copy output into a new Perl scalar Stypemap perl5 argout char msg if argvi gt items EXTEND sp 1 target sv_newmortal sv_setpv target Ssource argvitt 5 Applications SWIG is currently being used in an increasing vari ety of applications This section describes some of the ways in which has been used A number of advanced SWIG Perl interfacing techniques such as typemaps and callback functions are also described 5 1 Electronic CAD SWIG plays a pivotal role in the development process of BADGER an electronic computer aided design sys tem being developed by Fusion MicroMedia used in the design of integrated circuits and other electronic compo nents BADGER is a fully object oriented modular and highly extensible system running under various flavors of the UNIX operating system as well as Windows NT The core components in BADGER are constructed in C and are delivered as a set of shared dynamically loaded libraries The libraries are not directly linked into an executable program Instead each library comes with an extension language EL interface that is gener ated by SWIG allowing the library to be used within a Perl program The combination of a powerful EL and well tuned application specific software results in a sys tem that is potent flexible and easy to use For the most part SWIG is used in a normal fash ion a desc
9. difficult to manage as the API evolves since it is hard to maintain consistency between the interface file and header files The second choice may work if the header files are written in a very clean way However it can break down if header files are too complicated Therefore a mix of header files and interfaces was uti lized As part of the interface building process header files were to be included directly into interface files This is easily done using the include directive but a num ber of problematic nested structure declarations had to be fixed For example struct tcStat union struct stat_p_abort int value tc_p_abort_cause abort p_abort p te_stat To make this structure more manageable in SWIG it can be split into smaller pieces and rewritten as follows typedef struct int value tc_p_abort_cause tc_stat_abort p_abort struct TcStat tc_stat_abort abort Such changes have no impact on user code but they simplify the use of SWIG In addition to splitting a number of structures in the header files were to be hidden from the SWIG com piler While this could be done using a simple ifndef SWIG in the code this could potentially result in a huge customer problem if they also defined a SWIG macro in their compilation process Therefore conditional com pilation was implemented using some clever C com ments that were parsed by vpp See the Text Vpp mod ule during t
10. event and the object that received the event are supplied as arguments BADGER provides a number of classes derived from Trigger that specialize its behavior for certain exten sion languages for C or for an object request bro ker For example the Perl 5Trigger class is derived from Trigger and it specializes its base class by stor ing a pointer to a Perl function reference an SV and by providing the machinery needed to invoke that Perl function For example consider the following Perl fragment sub MyFcn my SEventName shift my S Object shift rest of function here my SObject BadgerFunction my SName Can t find file 6For now callbacks only work with Perl Support for callbacks with Tcl and Java will be added later Badger RegisterByObject Name SObject amp MyFcn SObject gt ReadFile Bogus name The MyFcn Perl function is the callback trig ger function and it is registered with S Object using the event name called Can t find file Now suppose that the SObject gt ReadFile operation fails Internally BADGER will note the failure deter mine the appropriate event name attempt to find a Trig ger object associated with that event and if found will invoke the Trigger by calling the appropriate member function For the example above this means that the MyFcn function will be called with SObject and Can t find file supplied as argume
11. provided many of the ideas used in the shadow class mechanism We would also like to thank John Buckman Scott Bolte and Sriram Srinivasan for their support of SWIG We also thank the University of Utah and Los Alamos Na tional Laboratory for their continued support 9 Availability SWIG is freely available on CPAN at www perl com CPAN authors Dave_Beazley Additional information is also available on the SWIG homepage at www swig org An active mailing list of several hundred subscribers is also available References 1 Sriram Srinivasan Advanced Perl Programming O Reilly and Associates 1997 2 Scott Bolte SWIG The Perl Journal 2 4 26 31 Winter 1997 3 D M Beazley SWIG and automated C C script ing extensions Dr Dobb s Journal 282 30 36 Feb 1998 4 D M Beazley SWIG users manual Technical Re port UUCS 98 012 University of Utah 1998 5 E Gamma R Helm R Johnson and J Vlissides Design Patterns Addison Wesley 1995
12. so forth To fix these problems interface files can be enhanced with helper functions typemaps exception handlers and other declarations Since interfaces are easily re generated making such changes is a relatively straight forward process However as a result SWIG interfaces tend to be built in an evolutionary and iterative manner rather than being formally specified in advance 4 An alternative approach to copying header files is to modify the header files using conditional compilation to add SWIG directives or to remove unnecessary functions 4 3 Traps and Pitfalls Finally there are a number of subtle problems that sometimes arise when transforming a C C program into a Perl extension module One of these problems is the issue of implicit execution order dependencies and reentrant functions From the Perl interpreter users will be able to execute functions at any time and in any order However in many C programs execution is precisely defined For example a precise sequence of function calls might be performed to properly initialize program data Likewise it may only be valid to call certain func tions once during a single execution From Perl it is easy for a user to violate these constraints resulting in a potential program crash or incorrect behavior To fix these problems applications can sometimes be modified by introducing additional state variables For example to prevent repeated execution a function can be m
13. this behavior In particular it was decided that testers should be able to set and get this value using BCD en coded strings such as follows my Sdialog new tc_dialog_portion Sdialog gt abort_reason 0f456A Or print User abort reason is Sdialog gt abort_reason n To do this a typemap for converting BCD Perl strings into an appropriate byte sequence were developed In addition the typemap performs a few sanity checks to prevent invalid values Stypemap perl5 in tc_u_abort Sbasetype temp int i STRLEN len short tmp char Str Starget amp temp convert scalar to char str SvPV Ssource len check if even of char if len 2 0 croak Uneven of char set length field Starget gt length len 2 if len 2 gt sizeof Sbasetype 1 croak Too many bytes n for i 0 i lt Starget gt length i if sscanf str S2hx amp tmp 1 croak sscanf failed on s is it hexa n str Starget gt datas i tmp str 2 To return the byte buffer back to Perl as a string a somewhat simpler typemap is used Stypemap perl5 out tc_u_abort Tne As Starget newSVpvf Sx Ssource gt datas 0 for i l i lt Ssource gt length i sv_catpvf Starget x Ssource gt datas i argvitt SWIG typemaps were also used to fix a few other functions For example some functions required an ad dres
14. 3 1 Pointers SWIG maps C pointers and C references into Perl blessed references These references contain both the value of the pointer itself plus a type signature In the gd example pointers were used to manage both images and files If one were to print out the value a pointer it would appear as follows gdImagePt r SCALAR 0x80b9914 SWIG uses the type signature to perform run time checking of all pointer values These checks emulate many of the checks that would have been performed by a C compiler When an invalid Perl datatype or pointer of invalid type is used a run time error is generated For example o perl use gd Sf gd fopen test gif w gd gdImageLine f 20 50 180 140 0 Type error in argument 1 of gdImageLine Expected gdImagePtr at line 3 Type checking is based on the name of each datatype However the type checker also keeps track of C in heritance hierarchies and typedef definitions Thus an acceptable pointer type includes any alternate names that might have been created with a typedef declara tion as well as any derived datatypes in C When pointers are manipulated in Perl they are opaque values That is pointers can be created and passed around to other C functions but they can not be dereferenced directly Thus in the example it is difficult or impractical for a user to directly manip ulate the internal representation of an image from the Perl interpreter Furthermore
15. For example suppose that you had the following C function int fact int n To wrap this function into a Perl module with xsubpp you would write the following XS file file example xs extern int fact int n MODULE Example PACKAGE Example int fact n int n When processed with xsubpp the following wrapper file is produced include EXTERN h include perl h include XSUB h extern int fact int n XS XS_Example_fact dXSARGS if items 1 croak Usage Example fact n int n int SvIV ST 0 int RETVAL RETVAL fact n ST 0 sv_newmortal sv_setiv ST 0 IV RETVAL XSRETURN 1 XS boot_Example dXSARGS char file __FILE_ XS_VERSION_BOOTCHECK newXsS Example fact XS_Example_fact file ST 0 amp sv_yes XSRETURN 1 To use the module the wrapper code must be com piled and linked into a shared library that can be dynam ically loaded into the Perl interpreter The easiest way to do this is with the MakeMaker utility by writing a script as follows file Makefile PL use ExtUtils MakeMaker WriteMakefile NAME gt Example OBJECT gt example o fact o This script is then used to create a Makefile and module as follows unix gt perl Makefile PL unix gt make unix gt make install Finally in addition to cre
16. GIF images and can be obtained at www boutell com gd gd html A Perl module to gd developed by Lincoln Stein is also available on CPAN so interested readers are encouraged to compare the results of using SWIG against an existing Perl extension File I O explained shortly FILE fopen char name char mode void fclose FILE In this file the ANSI C prototypes for every func tion that we would like to access from Perl are listed In addition a number of SWIG directives which are always preceded by a appear The module di rective specifies the name of the extension module The block is used to insert literal code into the out put wrapper file In this case we simply include the gd h header file Finally a few file I O functions also appear While not part of gd these functions are needed to manufacture file handles used by several gd functions To run SWIG the following command is executed unix gt swig perl5 gd i Generating wrappers for Perl 5 This produces two files gd wrap c and gd pm The first file contains C wrapper functions that appear sim ilar to the output that would have been generated by xsubpp The pm file contains supporting Perl code needed to load and use the module To build the module the wrapper file is compiled and linked into a shared library This process varies on every machine consult the man pages but the following steps are performed on Linux
17. Perl Extension Building with SWIG David M Beazley Dept of Computer Science University of Chicago Chicago IL 60637 Abstract SWIG Simplified Wrapper and Interface Generator is a freely available tool that integrates Perl Python Tcl and other scripting languages with programs writ ten in C C and Objective C This paper provides an introduction to SWIG and shows how it can be used to construct Perl extension modules In addition a num ber of applications in which SWIG has been utilized are described While SWIG is similar to other Perl exten sion building tools such as xsubpp and h2xs SWIG has a number of unique features that help simplify the task of creating Perl extension modules Many of these fea tures are described as well as limitations and future di rections This paper is primarily intended for developers who are interested in combining Perl with applications written in C or C as well as current SWIG users who are interested in learning more about some of SWIG s advanced features 1 Introduction One of Perl s greatest strengths is its ability to sim plify hard programming tasks as well as being able to solve the odd and varied computing problems that oc cur on a day to day basis While it would be nice to use Perl or other high level languages for everything this is simply not practical for many applications In fact performance critical tasks low level systems pro gramming and complex data struct
18. a combination of typemaps as follows Stypemap per1l5 ignore int OUTPUT int temp Starget amp temp Stypemap per1l5 argout int OUTPUT if argvi gt items EXTEND sp 1 target sv_newmortal sv_setiv target IV Ssource argvitt In this case the ignore typemap tells SWIG that a parameter is going to be ignored and that the Perl in terpreter will not be supplying a value Since the un derlying C function still needs a value the typemap sets the value of the parameter to point to a temporary vari able temp The argout typemap is used to return a value held in one of the function arguments In this case the typemap extends the Perl stack if needed and creates a new return value The argvi variable is a SWIG specific variable containing the number of values returned to the Perl interpreter so it is incremented for each return value The C code supplied in each typemap is placed in a private scope that is not visible to any other typemaps or other parts of a wrapper function This allows differ ent typemaps to be used simultaneously even if they de fine variables with the same names This also allows the same typemap to be used more once in the same wrap per function For example the previous section used the int OUTPUT typemap twice in the same function without any adverse side effects 3 10 3 Typemap Libraries Writing new typemaps is a somewhat magical a
19. at they allow Perl to perform a limited form of automatic memory management for C C objects If an object is created from Perl using a shadow class the DESTROY method of that class automatically invokes the C de structor when the objectis destroyed As a result C C objects wrapped by shadow classes can be managed us ing the same reference counting scheme utilized by other Perl datatypes 3 7 Class Extension When building object oriented Perl interfaces it is sometimes useful to modify or extend objects with new capabilities For example the gd library defines the fol lowing data structure for defining points typedef struct int x y gdPoint To make this structure more useful we can add construc tors destructors and various methods to it regardless of whether it is implemented in C or C To do this the SWIG addmethods directive can be used as follows Add some methods to points Saddmethods gdPoint Create a point or array of points gdPoint int npts 1 return gdPoint malloc sizeof gdPoint npts Destroy a point gdPoint free self Array indexing gdPoint get int i return selfti A debugging function void output printf d d n self gt x self gt y Now in the Perl interface gdPoint will appear just like a class with constructors destructors and methods For example Create a point Spt new gdPoint
20. ating the C component of the extension module it is necessary to write a pm file that is used to load and initialize the module For example file Example pm package Example require Exporter require DynaLoader QISA gqw Exporter DynaLoader bootstrap Example 1 At this point you should have a working Perl exten sion module In principle building a Perl extension re quires an XS specification for every C function that is to be accessed To simplify the process of creating these specifications Perl includes h2xs a tool that converts C header files to XS descriptions While useful h2xs is somewhat limited in its ability to handle global vari ables structures classes and more advanced C C fea tures As aresult h2xs can be somewhat difficult to use with more complex applications 3 SWIG Overview In a nutshell SWIG is a specialized compiler that transforms ANSI C C declarations into scripting lan guage extension wrappers While somewhat similar to h2xs SWIG has a number of notable differences First SWIG is much less internals oriented than XS In other words SWIG interfaces can usually be constructed with out any knowledge of Perl s internal operation Second SWIG is designed to be extensible and general purpose Currently wrappers can be generated for Perl Python Tcl and Guile In addition experimental modules for MATLAB and Java have been developed Finally SWIG supports a larger s
21. be used to change its behavior In fact one can even make SWIG map Perl arrays into C arrays if desired Therefore most restric tions can be eliminated with a little extra work Some of these customization techniques are described shortly 3 4 Helper Functions Sometimes the Perl interface constructed by SWIG is lacking in functionality or is difficult to use For exam ple in the previous section a function operating on C arrays was presented To construct C arrays from Perl it is necessary to add some additional functions to the SWIG interface This can be done using the inline directive as follows Add some helper functions for C arrays Sinline int int_array int size return int malloc sizeof int size void int_destroy int a free a void int_set int a int i int val a i val int int_get int a int i return ali a Ww When SWIG builds the scripting interface these func tions become part of the extension module and can be used as follows Convert a Perl array into a C int array sub create_array Slen scalar _ Sia gd int_array S len for i 0 i lt S len i val shift gd int_set S ia i val return Sia a 10 20 30 40 b 50 70 60 200 Sia create_array a Create C arrays Sib create_array b gd plotpts Sim ia ib 4 1 gd int_destroy ia gd int_destroy ib 3 5 Classes and Structures Whil
22. e SWIG represents all objects as opaque point ers the contents of an object can be examined and mod ified through the use of accessor functions as follows Extract data from an object double Point_x_get Point p return p gt x Invoke a C member function int Foo_bar Foo f return f gt bar From a Perl script a user simply passes an object pointer to accessor functions to extract internal information or invoke member functions While it is possible to write accessor functions manu ally SWIG automatically creates them when it is given structure and class definitions For example in the gd library the following structure is used to contain image information typedef struct gdImageStruct unsigned char pixels int sx int sy int colorsTotal gdImage If this structure definition were placed in the SWIG interface file accessor functions would automatically be created These could then be used to extract information about images as follows usr bin perl use gd Sim gd gdImageCreate 400 300 Print out the image width print gd gdImage_sx_get im n Accessor functions are also created for C classes and Objective C interfaces For example the class defi nition class List public List ELSE N void insert Object Object get int i int length hi is translated into the following accessor functions List new_List retur
23. eadonly and readwrite directives For exam ple Create read only variables sreadonly int foo Read only double bar Read only Sreadwrite Create read only class members class List sreadonly int length Sreadwrite Read only member When read only mode is used attempts to modify a value from Perl result in a run time error Another common problem is changing the name of various C declarations For example a C function name may conflict with an existing Perl keyword or subrou tine To fix this problem the sname directive can be used For example sname cpack void pack Object creates a new command cpack If name conflicts oc cur repeatedly the rename directive can be used to change all future occurrences of a particular identifier as follows Srename pack cpack The renaming operations can also be applied to C C class and structure names as needed For exam ple Sname Image class gdImage 3 9 Exception handling To catch errors SWIG allows users to create user defined exception handlers using the except direc tive These handlers are responsible for catching and converting C C runtime errors into Perl errors As an example the following error handler can be used to catch errors in the standard C library except perl5 errno 0 Sfunction if errno die strerror errno When defined the exception handling code is placed into all of the wrapper functio
24. eference and destroy integer pointers These func tions might be used as follows Swpetr new_integer Create an int Shptr new_integer imagesize Sim Swptr Shptr Sw integer_value Swptr Dereference Sh integer_value S hptr delete_integer Swptr delete_integer S hptr Clean up A more elegant solution is to use the SWIG typemap library in the interface file as follows Sinclude typemaps i void imagesize gdImagePtr im int OUTPUT int OUTPUT Now in the Perl script it is possible to do this w h imagesize S im In a similar spirit it is also possible to use Perl refer ences For example Sinclude typemaps i void imagesize gdImagePtr im int REFERENCE int REFERENCE Now in Perl Return values in w and h imagesize Sim S w h To implement this behavior the file typemaps i defines a collection of typemap rules that are attached to specific datatypes such as int OUTPUT and int REFERENCE The creation of these rules is now dis cussed 3 10 2 Creating New Typemaps All wrapper functions perform a common sequence of internal operations For example arguments must be converted from Perl into a C representation a function s return value must be converted back into Perl argument values might be checked and so forth SWIG gives each of these operations a unique name such as in
25. ew datatypes This has been done without having to exam ine the implementation of those typemaps or having to look at any Perl internals Currently SWIG includes a number of libraries that operate in this manner 3 11 Other SWIG Features SWIG has a number of other features that have not been discussed In addition to producing wrapper code SWIG also produces simple documentation files These describe the contents of a module In addition C com ments can be used to provide descriptive text in the doc umentation file SWIG is also packaged with a library of useful modules that include typemaps and interfaces to common libraries These libraries can simplify the construction of scripting interfaces 3 12 Putting it All Together In the first part of this section a minimal interface to the gd library was presented Now let s take a look at a more substantial version of that interface gd i Smodule gd include gd h 2 Make FILE work Stypemap perl5 in FILE target IolFP sv_2io Ssource Grab the gd h header file Sinclude gd h Extend the interface a little bit Saddmethods gdImage gdImage int w int h return gdImageCreate w h gdImage gdImageDestroy self ete Saddmethods gdPoint etc Wrap the fonts readonly variables Sreadonly Sinclude gdfontt h Sinclude gdfonts h Sinclude gdfontmb h Sinclude gdfontl h Sinclude
26. gcc fpic c gd_wrap c Dbool char TI usr lib per15 i586 linux 5 004 COR gcc shared gd_wrap o lgd o gd so Fl At this point the module is ready to use For example the earlier C program can be directly translated into the following Perl script usr bin perl use gd Create an image Sim gd gdImageCreate 200 200 Allocate some colors Sb gd gdImageColorAllocate Sim 0 0 0 Sw gd gdImageColorAllocate Sim 255 255 255 Draw a line gd gdImageLine Sim 20 50 180 140 S w Output the image Sout gd fopen test gif wb gd gdImageGif Sim Sout This syntax is derived from lex and yacc gd fclose Sout Clean up gd gdImageDestroy Sim 3 2 Input Files In the gd example SWIG was given a special interface file containing a list of the C declarations to be included in the Perl module When working with a large C library interface files can often be constructed by copying an existing header file and modifying it slightly However in some cases it is possible to include a header file as follows Smodule ca include gd h s Grab the declarations from gd h Sinclude gd h Some file I O functions FILE fopen char name char mode void fclose FILE The include directive tells SWIG to include a file and parse all of the declarations it contains In this case the interface would now wrap every function in the gd library as opposed to
27. gdfontg h Sreadwrite Finally here is a simple script that uses the module Aside from a few minor differences the script is remark ably similar to the first example given in the standard GD module documentation use gd Sim new gdImage 100 100 Swhite S im gt colorAllocate 255 255 255 Sblack S im gt colorAllocate 0 0 0 Sred Sim gt colorAllocate 255 0 0 Sblue Sim gt colorAllocate 0 0 255 Sim gt transparentcolor white Sim gt interlaced 1l Sim gt rectangle 0 0 99 99 Swhite Sim gt arc 50 50 95 75 0 360 Sblue Sim gt fi11 50 50 S red open IMG gt test gif Sim gt gif IMG close IMG 4 Interface Building Strategies SWIG simplifies the construction of Perl extensions because it hides Perl specific implementation details and allows programmers to incorporate C C applications into a Perl environment using familiar ANSI C C syn tax rules In addition SWIG interfaces are generally specified in a less formal manner than that found in XS or component architectures such as CORBA and COM As a result many users are surprised to find out how rapidly they can create Perl interfaces to their C C ap plications However it is a misperception to think that SWIG can magically take an arbitrary C C header file and instantly turn it into a useful Perl module This sec tion describes some of the issues and solution strategies for effectively using SWIG 4 1 Wrapping an Ex
28. he build of the SWIG interface For exam ple HP reserved comment if not _hp_reserved_t Ef typedef struct int length unsigned char datas MAX_ABORT_LEN tc_u_abort endif Ey 5 2 2 Shadow Classes By default SWIG converts structure definitions into ac cessor functions such as tc_global_title tc_address_gt_get tc_address tc_address_nature tc_global_title_nature_set tc_global_title t tc_address_nature val Unfortunately using such functions is somewhat un friendly from Perl For example to set a single value it would be necessary to write the following Sparam new_tc_address tc_global_title_nature_set tc_address_gt_get S param Svalue Fortunately shadow classes solve this problem by providing object oriented access to the underlying C structures As a result it is possible to rewrite the above Perl code as follows Sparm new tc_address Sparam gt gt nature value Needless to say this approach is much easier for users to grasp 5 2 3 Customization With Typemaps To improve the Perl interface a number of typemaps were defined for various parts of the interface One use of typemaps was in structures such as the following typedef struct tc_u_abort abort_reason tc_dialog_portion Since tc_u_abort is defined by the structure shown earlier SWIG normally tries to manipulate it through pointers However a typemap can be defined to change
29. heSsn SV thePc thePc newSViv Ssource 0 theSsn newSViv Ssource 1 passedHash newHV hv_store passedHash ssn 3 theSsn 0 hv_store passedHash pc 2 thePc 0 target newRV_noinc SV passedHash argvi 5 2 4 Statistics Table 1 shows the amount of code associated with i files and header files as well as the amount of code gen erated by SWIG C and pm files While it was nec essary to write a few i files the size of these files is small in comparsion to the generated output files ihe 1 TCAP and SCPP Modules i files cs 76098 3561 SCPP 364 13060 2246 5 2 5 Results Overall SWIG saved time when providing Perl access to the TCAP and SCCP libraries While it took some time and hard work to write the typemaps the SWIG approach has several advantages compared to XS or the pure C approach The interface files are quite short so if they are well documented a new SWIG user should not have any major problems maintaining them A new version of the API is wrapped with a make command so there is no need to edit any file In most cases the interface files can remain unmodi fied provided there are no weird constructs intro duced in the new version of the API New comments added in the header files will be au tomatically added in the documentation files gener ated by SWIG If necessary new helper functions may be added in the i files without impacti
30. ides better performance since manipulating references is more efficient than copying object data back and forth between languages Finally the use of references closely matches the way in which most C C programs already handle objects The downside to this approach is that objects are opaque in Perl This prevents users from examining their contents directly In addition SWIG wrappers occa sionally need to perform implicit memory allocations as shown above It is up the user to free the resources used by such functions or learn to live with a memory leak Of course this naturally brings us to the next topic 3When C is used SWIG uses the default copy constructor in stead of malloc 3 3 4 Memory Management SWIG maintains a strict separation between the manage ment of Perl and C objects While Perl uses reference counting to keep track of its own objects this scheme is not extended to C C extensions created with SWIG Thus when Perl destroys a blessed reference containing the value of a C pointer only the pointer value disap pears not the underlying C data that it points to From a user standpoint SWIG generated C C ex tensions follow the same memory management rules as the underlying application Thus if a program relies on malloc and free to allocate and deallocate objects these will also be used from the Perl interpreter Like wise a C extension typically requires explicit invo cation of constructors and dest
31. isting Program Building a Perl interface to an existing application generally involves the following steps 1 Locate header files and other sources of C declara tions 2 Copy header files to interface files 3 Edit the interface file and add SWIG directives 4 Remove or rewrite the application s main func tion if necessary 5 Run SWIG compile and link into a Perl extension module While it is theoretically possible to run SWIG directly on aC header file this rarely results in the best scripting interface First a raw header file may contain problem atic declarations that SWIG doesn t understand Second it is usually unnecessary to wrap every function and vari able in a large library More often than not there are in ternal functions that make little sense to use from Perl By copying header files to a separate interface file it is possible to eliminate these functions and clean things up with a little editing 4 Finally the underlying application may require a few slight modifications For example Perl supplies its own main function so if an applica tion also contains main it will have to be removed rewritten or not linked into the extension module 4 2 Evolutionary Interface Building After a Perl interface is first built its use will ex pose any problems and limitations These problems in clude functions that are awkward to use poor integration with Perl datatypes missing functionality and
32. mod ule naming system In other words SWIG can create a module Foo but can t create a module Foo Bar Likewise SWIG does not currently utilize MakeMaker and other utilities although users have successfully used SWIG with such tools In addition some users have reported occasional problems when SWIG modules are used with the Perl debugger and other tools 7 Future Directions Future development of SWIG is focused on three pri mary areas First improved parsing and support for more advanced C are being added These addi tions include support for overloaded functions and C namespaces Limited support for wrapping C tem plates may also be added Second SWIG s code gen eration abilities are being improved Additions include more flexible typemaps and better access to scripting language specific features Finally an extension API is being added to the SWIG compiler This API will allow various parts of the SWIG compiler such as the preprocessor parser and code generators to be accessed through a scripting language interface In fact this inter face will even allow new parsers and code generators to be implemented entirely in Perl 8 Acknowledgments SWIG would not be possible without the feedback and contributions of its users While it is impossible to acknowledge everyone individually a number of peo ple have been instrumental in promoting and improving SWIG s Perl support In particular Gary Holt
33. n new List void delete_List List 1l delete 1 void List_insert List l Object o l gt insert o 3 6 Shadow Classes and Perl Objects As an optional feature the accessor functions created by SWIG can be used to write Perl wrapper classes this is enabled by running SWIG with the shadow option While all the gory details can be found in the SWIG Users Manual the general idea is that the accessor func tions can be encapsulated in a Perl class that mimics the behavior of the underlying object For example package List QISA qw example sub new my self shift my args _ Sself new_List args return undef if defined Sself bless self List my sretval tie retval List Sself return bless retval List sub DESTROY delete_List _ sub insert return result List_insert _ This class provides a wrapper around the underly ing object and is said to shadow the original object Shadow classes allow C and C objects to be used from Perl in a natural manner For example Sl new List l gt insert 0 1 gt DESTROY For C structures access to various attributes are pro vided through tied hash tables For the gd library mem bers of the image data structure could be accessed as follows Sim gd gdImageCreate 400 400 Swidth im gt sx height im gt sy The other significant aspect of shadow classes is th
34. ng other parts of the code or typemaps This allows a new user to do it without reading the whole SWIG manual Typemaps that deal with basic types or simple structures are reusable and can be used with other APIs For those who are considering SWIG s advanced fea tures the learning curve is a little steep at first but the rewards are great because SWIG advanced features will enable you to provide an improved interface to the Perl user 6 Limitations Currently SWIG is being used by hundreds of users in conjunction with a variety of applications However the current implementation of SWIG has a number of limitations Some of these limitations are due to the fact that SWIG is not a full C C parser In particular the following features are not currently supported e Variable length arguments e Pointers to functions e Templates e Overloaded functions and operators e C Namespaces e Nested class definitions When these features appear in a SWIG input file a syntax error or warning message is generated To elim inate these warnings problematic declarations can ei ther be removed from the interface hidden with condi tional compilation or wrapped using helper functions and other SWIG directives A closely related problem is that certain C C pro grams are not easily scripted For example programs that make extensive use of advanced C features such as templates smart pointers and overloaded operators
35. ns In the process the function token is replaced by the actual C function call For the example shown the exception handler re sets the errno variable and calls the C function If the value of errno is modified to a non zero value an er ror message is extracted from the C library and reported back to Perl While catching errors in the C library has been illus trated exception handlers can also be written to catch C exceptions or to use any special purpose error han dling code that might be present in an application 3 10 Typemaps Typemaps are one of SWIG s most powerful features and the primary means of customization Simply stated a typemap is a small bit of C code that can be given to SWIG to modify the way that it processes specific datatypes For instance Perl arrays can be converted into C arrays Perl references can be substituted for pointers and so forth This section briefly introduces typemaps and their use However typemaps are a complicated topic so it is impossible to cover all of the details here and interested readers are strongly advised to consult the SWIG documentation 3 10 1 Example Output Values As a first typemap example consider a function that re turns values through its parameters as follows void imagesize gdImagePtr im int w int h w gdImageSX im h gdImageSyY im As is this function would be difficult to use because the user must write helper functions to manufacture der
36. nterpreter This paper provides an introduction and overview of SWIG a tool designed to integrate C code with a variety of scripting languages including Perl Python and Tcl Currently SWIG can construct Perl extension modules on Unix and Windows NT systems It also supports the ActiveState Perl for Windows and Perl4 SWIG has been freely available since February 1996 and has been pre viously described in Advanced Perl Programming The Perl Journal and Dr Dobb s Journal 2 3 In ad dition SWIG is packaged with a 300 page user manual describing its use 4 The goal of this paper is not to repeat all of this information but to provide an overview of SWIG demonstrate the use of some of its more ad vanced features and describe some of the ways that it is currently being used The authors include the developer of SWIG and two of SWIG s foremost Perl experts who have made substantial contributions to SWIG s develop ment 2 Perl Extension Building To interface Perl with code written in C or C it is necessary to write wrappers that serve as a glue layer between the Perl interpreter and the underlying C code These wrappers are responsible for converting data be tween Perl and C reporting errors and other tasks Perl is packaged with several tools for creating these wrap pers One such tool is xsubpp a compiler that takes interface definitions written in a special language known as XS and converts them into wrappers
37. nts The function may require more information such as the file name that could not be opened and it might find this information by pulling data from the external library using the functions wrapped by SWIG The RegisterByObject function is respon sible for creating an object of the Perl5Trigger class and for creating the association between the Perl5Trigger the event name and the object re ceiving the event There is a bit of typemap trickery involved when intercepting the arguments from Perl Stypemap perl5 in SV pFcn if SvROK S source croak Expected a referenc Starget SvRV Ssource void RegisterByObject const char pcEventName Ref pRef SV pFcn ENA TYY The final portion of the system left to describe is the implementation of the Perl5Trigger Invoke member function which is responsible for calling the Perl function from the C side of the world The im plementation of this taken nearly verbatim from the Ad vanced Perl Programming book 1 pg 353 looks like this bool Perl5Trigger Invoke const char pcEventName void pObject const char pcTypeName aSP ENTER SAVETMPS PUSHMARK sp SV pSV sv_newmortal sv_setpv pSV char pcEventName XPUSHS pSV pSV sv_newmortal sv_setref_pv pSV char pcTypeName pObject XPUSHs pSV pSV sv_newmortal sv_setpv pSV char pcTypeName XPUSHs
38. odi fied as follows void foo static int called 0 if called return called 1 It is also possible to catch such behavior using exception handlers For example sexcept perl5 static int called 0 if called croak Already executed n Sfunction called 1 List all non reentrant functions void foo Clear the exception handler Sexcept perl5 Another common problem is that of improper mem ory management As previously mentioned SWIG ex tensions use the same memory management techniques as C Therefore careless use may result in memory leaks dangling pointers and so forth A somewhat more obscure memory related problem is caused when a C program overwrites Perl data This can be caused by a function such as the following void geterror char msg strcpy msg strerror errno This function copies a string into memory pointed to by msg However in the wrapper function the value of msg is really a pointer to data buried deep inside a Perl scalar value When the function overwrites the value it corrupts the value of the Perl scalar value and can cause the Perl interpreter to crash with a memory addressing error or obscure run time error Again this sort of prob lem can usually be fixed with the use of typemaps For example it is possible to turn the msg parameter into an output value as follows Use a temporary array for the result Stypemap per1l5 ignore char msg
39. or a variety of reasons we are not able to always utilize modern compilers Hence we have cre ated the implementations of the smart references man ually which is a tedious process Fortunately this task can be mostly automated by creating our own code gen erator as part of SWIG This is a simple matter as SWIG is a modular software system 5 1 2 Callbacks The extension language interface produced by SWIG al lows functions defined in the external system to be called from within an extension language Unfortunately the interface produced by SWIG does not support the call ing of extension language functions within C C or Objective C The ability to invoke functions bidirection ally is needed by BADGER so support for callbacks from C to Perl has been developed The basic approach is this e Define a function Register the function e Perform some operation that causes the registered function to be invoked To make this work BADGER provides an abstract base class in C called Trigger so called because a function associated with objects of this class is invoked when an event of some kind occurs BADGER also pro vides the machinery to associate Trigger objects with an event name and with one or more objects internal to the system When an internal object receives an event it examines the set of registered functions looking for a match If a match is found then the Trigger object is invoked and the name of the
40. ription of the classes contained within a li 5For now Perl is the only supported extension language Tcl and Java will be supported in the future brary is presented to SWIG and it generates an EL in terface that allows the code within that library to be ac cessed from an EL There are two interesting facets to the use of SWIG within BADGER the use of smart ref erences and the use of callbacks from C to the EL 5 1 1 Smart References Suppose a Perl program calls a function defined by BADGER and wrapped with SWIG in order to create and return some object Any Perl variable used to refer to that object really holds a handle to the object im plemented as a blessed reference containing the object s type and its memory address Although the implementa tion is a bit more involved the handle in effect acts like a pointer in C Now suppose another function within BADGER is called that causes the original object to be destroyed Severe problems will occur if the Perl vari able is supplied to another BADGER function because the variable refers to a non existent object The reason for the difficulty is that the extension language expects to have control over the lifetime of the object but the ex ternal system BADGER cannot meet this expectation It is possible to design BADGER so that the extension language has complete control over the lifetime of all the objects within the system Unfortunately this ap proach resul
41. rt that requires knowledge of Perl s internal operation SWIG and the underlying application Books such as Advanced Perl Programming and the man pages on extending and embedding the Perl interpreter will prove to be quite useful However since writing typemaps from scratch is difficult SWIG provides a way for typemaps to be placed in a library and utilized without knowing their in ternal implementation details To illustrate suppose that you wanted to write some generic typemaps for check ing the value of various input parameters This could be done as follows check i typemaps for checking argument values Stypemap per1l5 check Number POSITIVE if Starget lt 0 croak Expected a positive value stypemap perl5 check Pointer NONNULL if Starget NULL croak Received a NULL pointer To use these typemaps a user could include the file check i and use the Sapp1y directive The apply directive simply takes existing typemaps and makes them work with new datatypes For example Sinclude check i Force double px to be positive Sapply Number Positive double px Force these pointers to be NON NULL sapply Pointer NONNULL FILE Vector Matrix gdImage Now some functions double log double px px positive double dot_product Vector Vector In this case the typemaps we defined for checking different values have been applied to a variety of n
42. ructors Furthermore for functions that implicitly allocate memory as in the pre vious section it is up to the user to explicitly destroy the result using free or a C destructor While such a scheme may seem problematic it is no less problematic than memory management in C which may or may not be a good thing depending on your point of view Even if it were possible to have Perl automatically manage C C objects this would be an inherently dangerous affair especially since Perl has no way to know how an underlying C application really operates Furthermore it would be a fatal error for Perl to deallocate objects that were still in use Therefore SWIG leaves memory management largely up the user 3 3 5 Pointers Arrays and Perl A common confusion among some novice users is the difference between C datatypes and similar Perl datatypes In particular Perl references are not the same as a C pointers and Perl arrays are not the same as C ar rays Differences also apply to other datatypes such as files this is the reason that the simple example included prototypes for fopen and fclose The primary rea son for these differences is that objects in Perl have a different internal representation than objects in C For example a Perl array is represented as a collection of references to Perl objects which may be of mixed types The internal representation of this array is entirely dif ferent than what would be used for a normal C array
43. s parameter encoded as a two element array By de fault SWIG wraps this parameter as a pointer but this leaves the Perl writer with the painful tasks of creat ing and filling a C array with sensible values using the SWIG pointer library or helper functions Fortunately with typemaps it was possible to create and set this pa rameter using Perl hashes as follows Saddress is an ordinary perl hash Saddress will be used as an array Saddress gt pc 10 Saddress gt ssn 12 SCCP_oamcemd S cnxId time undef Saddress Scommand Scmd_parms The typemap implementing this behavior is as fol lows Stypemap perl5 in HV passedHash SV valuePP SccpOamAddress tempAddress if SvOK Ssource we were passed undef tempAddress 0 tempAddress 1 else if SvROK source croak Not a reference n if SvTYPE SvRV Ssource SVt_PVHV croak Not a hash ref n SccpOamAddress 0 0 passedHash HV SvRV source valuePP hv_fetch passedHash ssn 3 0 if valuePP NULL croak Missing ssn key n tempAddress 1 SvIV valuePP valuePP hv_fetch passedHash pc 2 0 if valuePP NULL croak Missing pc key n tempAddress 0 SvIV valuePP target amp tempAddress SccpOamAddress is returned as ssn gt ssn_value pc gt pc_value Stypemap perl5 out SccpOamAddress HV passedHash SV t
44. the half dozen functions listed in the first example SWIG also includes a C preprocessor that can be used for macro expansion and conditional compilation If a new application is being written with SWIG in mind header files can be written as follows ifdef SWIG Smodule gd s include gd h 3 endif C declarations With this approach the file can serve as both a valid C header file and as an interface specification The SWIG symbol is only defined when SWIG is parsing so special directives can be easily hidden from the C compiler as needed Finally for the truly lazy SWIG can sometimes be run directly on C header and source files For example oe modul modul swig per15 swig per15 gd gd h xampl oe xample c Most users however use a mix of dedicated interface files and header files 3 3 Data Model The most critical part of interfacing Perl to C pro grams is the management of data Since Perl and C uti lize a different set of internal datatypes wrapper gener ators are responsible for producing code that marshals data and objects between languages For fundamental types such as int and double the conversion pro cess is straightforward However pointers arrays struc tures and objects complicate the process Furthermore since most C C programs make extensive use of these datatypes it is important for wrapper generators to sup port as many of these datatypes as possible 3
45. ts in a system that is too closely tied to the implementation of a particular language and adding a new extension language to the mix is difficult An alter nate solution that is simple to implement and is portable is to introduce smart references also called proxies into the design 5 pg 207 In effect a smart refer ence is an object that has the same set of operations as a real object but the smart reference s implementation consists solely of a single pointer to a real object of the appropriate type The extension language interfaces within BADGER have been crafted so that the extension language manip ulates smart references and that the lifetime of a smart reference is completely under the control of the exten sion language Under most circumstances the extension language performs an operation on the smart reference and the smart reference then attempts to transfer the op eration to the real object If the real object has been de stroyed then the smart reference will have been invali dated it points to nil In this case the operation is aborted and if possible an exception is raised in the ex tension language BADGER contains the necessary ma chinery to invalidate any smart references that point to an object being destroyed Modern C compilers with their support for tem plates run time type identification and so forth pro vide the means to automatically construct smart refer ence classes F
46. ubset of C and C including struc tures classes global variables and inheritance This section provides a tour of SWIG and describes many of its interesting features 3 1 A Small Taste As a first example suppose that you wanted to build a Perl interface to Thomas Boutell s gd graphics library Since gd is a C library images are normally created by writing C code such as follows include gd h int main gdImagePtr im FILE KOUE int blk wht Create an image im gdImageCreate 200 200 Allocate some colors b gdImageColorAllocate im 0 0 0 w gdImageColorAllocate im 255 255 255 Draw a line gdImageLine im 20 50 180 140 w Output the image out fopen test gif whb gdImageGif im out fclose out Clean up gdImageDestroy im By building a Perl interface to gd our goal is to write similar code in Perl Thus the functionality of the gd library must be exposed to the Perl interpreter To do this a SWIG interface file can be written as follows File Smodule gd include gd h gd i typedef gdImage gdImagePtr gdImagePtr gdImageCreate int sx int sy void gdImageDestroy gdImagePtr im void gdImageLine gdImagePtr im int xl int yL int x2 int y2 int color int gdImageColorAllocate gdImagePtr im int r int g int b void gdImageGif gdImagePtr im FILE o 1 gd is a freely available graphics library for producing
47. ures are likely to be implemented in a compiled language such as C or C and may be easier to manage in such languages Fur thermore developers often need to work with a wide va riety of existing applications and legacy systems that are written in such languages The integration of Perl and code written in compiled languages has a number of practical benefits First it al lows existing C C applications to be incorporated into a high level interpreted environment This environment David Fletcher Fusion MicroMedia Corp Longmont CO 80501 Dominique Dumont Hewlett Packard Lab TID 5 Ave Raymond Chanas 38053 Grenoble cedex 9 France provides greater flexibility and often simplifies develop ment since debugging and testing can be performed us ing Perl scripts Second Perl can serve as a powerful user interface In other words rather than writing a user interface from scratch it is possible to use a Perl inter preter instead This also allows other for other possibil ities such as graphical user interface development with Perl Tk Finally Perl provides developers with a mech anism for assembling and controlling software compo nents Rather than creating a huge monolithic package C C programs can be packaged as collections of Perl extension modules As a result programs become more modular and easier to maintain Furthermore it is even possible to combine entirely different programs together within a shared Perl i
Download Pdf Manuals
Related Search