Home

CFFI User Manual

1. 103 defcenum name and options amp body enum list erg nee 37 defcfun name and options return type amp body docstring arguments amp rest gt 11sp namer is 86 defcstruct name and options amp body doc and slots gt name 33 defctype name base type amp optional documentation c4 3 9 eR eds 36 defcunion name amp body doc and slots name e veteris beh end qp o D chris 35 defcvar name and options type amp optional documentation lisp name 82 define wi cso dal v Mie ora E e etek are heed 107 define foreign library name and options load clause gt name 94 define foreign type class name supers slots amp rest options class name 38 define parse method name lambda list amp body body gt name cs e pk Rr E I ER En 39 defining callbacks so corre net 16 defining type translation compiler macros 26 dynamic extent esri atate aW Re RET YS 12 E enumeration Gi see ceded conde 7 F FILE and streams 000 c cece cece cece eee 9 foreign arguments 0 eee eee eee T foreign functions and data 222202 gt 4 Index foreign library load eee 6 foreign values with dynamic extent 12 foreign alloc type amp key initial element initial contents count 1 null terminated p pointer 58 foreign bitfield symbols type value SymbOl8S n xsasss ig e ri
2. simplified downcased etc defun bar x let g3182 if x 1 0 not zerop foreign funcall bar int g3182 int No generic function overhead Let s go back to our my string type The expansion interface has no equivalent of free translated object you must instead define a method on expand to foreign dyn the third generic function in this interface This is especially useful when you can allocate something much more efficiently if you know the object has dynamic extent as is the case with function calls that don t save the relevant allocated arguments This exactly what we need for the my string type Chapter 6 Foreign Types 27 defmethod expand from foreign form type my string type foreign string to lisp form defmethod expand to foreign dyn value var body type my string type let encoding string type encoding type with foreign string var value encoding encoding body So let s look at the macro expansion CFFI gt macroexpand 1 defcfun foo my string x my string simplified downcased etc defun foo x with foreign string G2021 X encoding utf 8 foreign string to lisp foreign funcall foo pointer g2021 pointer Again no generic function overhead Other details To short circuit expansion and use the translate functions instead simply call the next method Return its result in cases where your method cannot ge
3. I in package oos eaae EU ee 106 inc pointer pointer offset new pointer hedet tugs vie Eb LX blc pP PNIS I PopS 61 incf pointer place amp optional offset 1 gt new pointef e secure pe eb nS Conr 62 anciud cite tid tenes ed ERN RIGUARDA 106 114 L library foreign ve owe are OSEE 6 limitations of type translators 20 lisp string to foreign string buffer bufsize amp key start end offset encoding buffer ET EEEE EET LEENE RUE M E 78 Lispy C funclonB ils ni RE RR ER oes 10 load foreign library library handler 97 load foreign library error 99 loading CENT cauenasa rrr RR secs nig Rese estes 5 looks like it worked 0000005 7 M make pointer address ptr 63 mem aref ptr type amp optional index 0 64 mem ref ptr type amp optional offset object EUER 65 minimal bindings od nen ea 4 N null pointer pointer 66 null pointer p ptr boolean 67 P Berln auna asien el 4 pointer address ptr address 69 pointer eq ptri ptr2 boolean 70 pointerp ptr gt boolean eee 68 pointers in LISP sis cis ee eh nn era re p RE Rp 7 premature deallocation 2222222 12 PLOEN MERE 106 Pybhlotiz 22i ert e prr e dug due dea RENE 4 R requiring OEREI 21sesdics ec serere bi redes 5 S SLIME nes 20000 dde eso nec rake ederent rs 4 st
4. define curl option setter set curl option errorbuffer curl option errorbuffer pointer define curl option setter set curl option url curl option url pointer curl option Macroexpanding one of the new define curl option setter forms yields the following progn defcfun curl_easy_setopt set curl option nosignal curl code easy handle easy handle option curl option new value long curry curl option setter set curl option nosignal nosignal Finally let s try this out CFFI USER gt set curl option nosignal easy handle 1 gt 0 Looks like it works just as well This interface is now reasonably high level to wash out some of the ugliness of the thinnest possible curl easy setopt FFI without obscuring the remaining C bookkeeping details we will explore 4 8 Memory management According to the documentation for curl easy setopt the type of the third argument when option is CURLOPT_ERRORBUFFER is char Above we ve defined set curl option errorbuffer to accept a pointer as the new option value However there is a CFFI type string which translates Lisp strings to C strings when passed as arguments to foreign function calls Why not then use string as the CFFI type of the third argument There are two reasons both related to the necessity of breaking abstraction described in Section 4 6 Breaking the abstraction page 9 The first reason also applies to CURLOPT URL which we will use to illustrate the
5. names Function Arguments and Values type A foreign struct type names A list Description The function foreign slot names returns a potentially shared list of slot names for the given structure type This list has no particular order Examples defcstruct timeval tv secs long tv usecs long CFFI gt foreign slot names timeval TV SECS TV USECS See Also defcstruct page 33 foreign slot offset page 45 foreign slot value page 47 foreign slot pointer page 46 Chapter 6 Foreign Types 45 foreign slot offset Syntax foreign slot offset type slot name gt offset Function Arguments and Values type A foreign struct type slot name A symbol offset An integer Description The function foreign slot offset returns the offset in bytes of a slot in a foreign struct type Examples defcstruct timeval tv secs long tv usecs long CFFI gt foreign slot offset timeval tv secs gt 0 CFFI gt foreign slot offset timeval tv usecs gt 4 See Also defcstruct page 33 foreign slot names page 44 foreign slot pointer page 46 foreign slot value page 47 Chapter 6 Foreign Types 46 foreign slot pointer Syntax foreign slot pointer ptr type slot name pointer Function Arguments and Values ptr A pointer to a structure type A foreign structure type slot names A slot name in the type pointer A pointer to the slot slot name Desc
6. 4 4 Initializing libcurl After the introductory matter the tutorial goes on to present the first function you should use CURLcode curl global init long flags Let s pick this apart into appropriate Lisp code A CURLcode is the universal error code curl curl h says no return code will ever be removed and new ones will be added to the end defctype curl code int Initialize libcurl with FLAGS defcfun curl global init curl code flags long Implementor s note By default CFFI assumes the UNIX viewpoint that there is one C symbol namespace containing all symbols in all loaded objects This is not so on Windows and Darwin but we emulate UNIX s behaviour there defcfun page 66 for more details Note the parallels with the original C declaration We ve defined curl code as a wrap ping type for int right now it only marks it as special but later we will do something more interesting with it The point is that we don t have to do it yet Looking at curl h CURL GLOBAL NOTHING a possible value for flags above is defined as 0 So we can now call the function CFFI USER gt curl global init 0 Chapter 4 An Introduction to Foreign Interfaces and CFFI 7 gt 0 Looking at curl h again 0 means CURLE_OK so it looks like the call succeeded Note that CFFI converted the function name to a Lisp friendly name You can specify your own name if you want use curl global init you
7. Implementor s note size_t is almost always an unsigned int You can get this and many other types using feature tests for your system by using cffi grovel The above signature trivially translates into a CFFI defcallback form as follows Alias in case size_t changes defctype size unsigned int To be set as the CURLOPT WRITEFUNCTION of every easy handle defcallback easy write size ptr pointer size size nmemb size stream pointer let data size size nmemb handler case We use the dynamically bound easy write procedure to call a closure with useful lexical context progn funcall symbol value easy write procedure foreign string to lisp ptr data size nil data size indicates success The WRITEFUNCTION should return something other than the bytes available to signal an error error if zerop data size 1 0 First note the correlation of the first few forms used to declare the C function s signa ture with the signature in C syntax We provide a Lisp name for the function its return type and a name and type for each argument In the body we call the dynamically bound easy write procedure with a finished translation of pulling together the raw data and size into a Lisp string rather than deal Chapter 4 An Introduction to Foreign Interfaces and CFFI 17 with the data directly As part of calling curl easy perform later we ll bind that variable to a cl
8. When a translate to foreign method requires allocation of foreign memory you must also define a free translated object method to free the memory once the foreign object is no longer needed otherwise you ll be faced with memory leaks This generic function is called automatically by CFFI when passing objects to foreign functions Let s do that Free strings allocated by translate to foreign defmethod free translated object pointer type my string type param declare ignore param foreign string free pointer In this specific example we don t need the param argument so we ignore it See free translated object page 51 for an explanation of its purpose and how you can use it A type translator does not necessarily need to convert the value For example one could define a typedef for pointer that ensures in the translate to foreign method that the value is not a null pointer signalling an error if a null pointer is passed This would prevent some pointer errors when calling foreign functions that cannot handle null pointers Please note these methods are meant as extensible hooks only and you should not call them directly Use convert to foreign convert from foreign and free converted object instead See Section 4 11 Defining new types page 18 for another example of type translators Chapter 6 Foreign Types 26 6 5 Optimizing Type Translators Being based on generic functions the type translation mechanis
9. incf pointer page 62 make pointer page 63 pointerp page 68 null pointer page 66 null pointer p page 67 Chapter 7 Pointers 62 incf pointer Syntax incf pointer place amp optional offset 1 new pointer Macro Arguments and Values place A setf place new pointer A foreign pointer offset An integer Description The incf pointer macro takes the foreign pointer from place and creates a new pointer incremented by offset bytes and which is stored in place Examples CFFI gt defparameter two words foreign string alloc Common Lisp gt TWO WORDS CFFI gt defparameter one word two words NE WORD CFFI gt incf pointer one word 7 gt SB SYS INT SAP X00600457 CFFI gt foreign string to lisp one word gt Lisp CFFI gt foreign string to lisp two words gt Common Lisp See Also inc pointer page 61 make pointer page 63 pointerp page 68 null pointer page 66 null pointer p page 67 Chapter 7 Pointers make pointer Syntax make pointer address ptr Arguments and Values address An integer ptr A foreign pointer Description The function make pointer will return a foreign pointer pointing to address Examples CFFI gt make pointer 42 gt lt FOREIGN ADDRESS x0000002A gt CFFI gt pointerp gt T CFFI gt pointer address gt 42 CFFI gt inc pointer 42 gt lt FOREIGN ADDRESS x000000
10. Ifa list the meaning depends on first library framework The second list element is taken to be a Darwin framework name which is then searched in darwin framework directories and loaded when found ror Each remaining list element itself a library designator is loaded in order until one succeeds default The name is transformed according to the platform s naming convention to shared libraries and the resultant string is loaded as a library designator For example on UNIX the name is suffixed with so If the load fails signal a load foreign library error Please note For system libraries you should not need to specify the directory containing the library Each operating system has its own idea of a default search path and you should rely on it when it is reasonable Implementation specific Notes On ECL platforms where its dynamic FFI is not supported ie when dffi is not present in features cffi load foreign library does not work and you must use ECL s own ffi load foreign library with a constant string argument Examples See Section 4 3 Loading foreign libraries page 5 See Also close foreign library page 92 darwin framework directories page 93 Chapter 11 Libraries define foreign library page 94 foreign library directories page 95 load foreign library error page 99 use foreign library page 100 98 Chapter 11 Libraries 99 load foreign library error Syntax
11. int initial contents 1 2 3 lt A Mac Pointer x102950 gt CFFI gt loop for i from 0 below 3 collect mem aref int i gt 1 23 CFFI gt foreign alloc int initial contents 1 2 3 lt A Mac Pointer x102960 gt CFFI gt loop for i from 0 below 3 collect mem aref int i gt 1 2 3 Allocate a char pointer that points to newly allocated memory 333 by the string type translator for the string foo CFFI gt foreign alloc string initial element foo lt A Mac Pointer x102C40 gt Allocate a null terminated array of strings 333 Note FOREIGN STRING TO LISP returns NIL when passed a null pointer CFFI gt foreign alloc string initial contents foo bar baz null terminated p t lt A Mac Pointer x102D20 gt CFFI gt loop for i from 0 below 4 collect mem aref string i gt foo bar baz NIL CFFI gt progn dotimes i 3 foreign free mem aref pointer i foreign free gt nil See Also foreign free page 57 with foreign object page 71 with foreign pointer page 72 Chapter 7 Pointers 60 foreign symbol pointer Syntax foreign symbol pointer foreign name amp key library Function pointer Arguments and Values foreign name A string pointer A foreign pointer or nil library A Lisp symbol or an instance of foreign library Description The function foreign symbol pointer will return a foreign pointer corresponding
12. slot value handle error buffer defun free easy handle handle Free CURL easy interface HANDLE and any C strings created to be its options 10 Tt might be better to return values than curle ok in real code but this is good for illustration Chapter 4 An Introduction to Foreign Interfaces and CFFI 20 with slots pointer error buffer c strings handle curl easy cleanup pointer foreign free error buffer mapc foreign string free c strings define foreign type easy handle type O actual type pointer simple parser easy handle defmethod translate to foreign handle type easy handle type Extract the pointer from an easy HANDLE slot value handle pointer While we changed some of the Lisp functions defined earlier to use CLOS slots rather than hash tables the foreign functions work just as well as they did before The greatest strength and the greatest limitation of the type translator comes from its generalized interface As stated previously we could define all foreign function calls in terms of the primitive foreign types provided by CFFI The type translator interface allows us to cleanly specify the relationship between Lisp and C data independent of where it appears in a function call This independence comes at a price for example it cannot be used to modify translation semantics based on other arguments to a function call In these cases you should rely on other features of Lisp
13. 42 foreign enum key word Syntax foreign enum keyword type value amp key errorp keyword Function Arguments and Values type An enum type value An integer errorp If true the default signal an error if value is not defined in type If false foreign enum keyword returns nil keyword A keyword symbol Description The function foreign enum keyword returns the keyword symbol that corresponds to value in type An error is signaled if type doesn t contain such value and errorp is true Examples defcenum boolean no yes CFFI gt foreign enum keyword boolean 1 gt YES See Also defcenum page 37 foreign enum value page 43 Chapter 6 Foreign Types 43 foreign enum value Syntax foreign enum value type keyword amp key errorp value Function Arguments and Values type An enum type keyword A keyword symbol errorp If true the default signal an error if keyword is not defined in type If false foreign enum value returns nil value An integer Description The function foreign enum value returns the value that corresponds to keyword in type An error is signaled if type doesn t contain such keyword and errorp is true Examples defcenum boolean no yes CFFI gt foreign enum value boolean yes 1 See Also defcenum page 37 foreign enum key word page 42 Chapter 6 Foreign Types 44 foreign slot names Syntax foreign slot names type
14. Ba 73 D War AGS eis donna Ex DER ati xo EN aO SORORE 81 10 Functions een 85 IL AWG ATES ic eo casein tinned mE ra em ver ga en trien 91 11 1 Defining lbrary uur eer RR sert DRE ERES 91 11 2 Library definition style ssssseeeeeeeeee eene 91 12 Calipers esse 101 13 The Groveller sues 106 13 1 Building FFIs with CFFI Grovel 220005 106 13 2 Specification File Syntax 0 cece eee eee 106 13 3 ASDF Integration can sense 108 13 4 Implementation Notes sseseseseeeee see 108 14 Limitations sess 110 Appendix A Platform specific features 111 Appendix B Glossary 112 Chapter 1 Introduction 1 1 Introduction CFFI is the Common Foreign Function Interface for ANSI Common Lisp systems By foreign function we mean a function written in another programming language and having different data and calling conventions than Common Lisp namely C CFFI allows you to call foreign functions and access foreign variables all without leaving the Lisp image We consider this manual ever a work in progress If you have difficulty with anything CFFI specific presented in the manual please contact the developers with details Motivation See Section 4 1 What makes Lisp different page 4 for an argument in favor of FFI in general CFFT s primary role in any image is to mediate between Lisp developers and the widel
15. This function is available mostly for inspection of the type translation process and possibly optimization of special cases of your foreign function calls Its behavior is better described under translate from foreign s documentation Examples CFFI USER gt convert to foreign a boat string gt lt FOREIGN ADDRESS x097ACDCO gt gt T CFFI USER gt convert from foreign string gt a boat See Also convert to foreign page 30 free converted object page 50 translate from foreign page 52 Chapter 6 Foreign Types 30 convert to foreign Syntax convert to foreign value type foreign value Function alloc params Arguments and Values value The Lisp object to be translated to a foreign object type A CFFI type specifier foreign value The primitive C value ready to be passed to a primitive foreign function alloc params Something of a translation state you must pass it to free converted object along with the foreign value for that to work Description This is an external interface to the type translation facility In the implementation all for eign functions are ultimately defined as type translation wrappers around primitive foreign function invocations This function is available mostly for inspection of the type translation process and possibly optimization of special cases of your foreign function calls Its behavior is better described under translate to foreign s documentat
16. base type A symbol denoting a foreign type keyword A keyword symbol value An index value for a keyword Description The defcenum macro is used to define foreign types that map keyword symbols to integer values similar to the C enum type If value is omitted its value will either be 0 if it s the first entry or it it will continue the progression from the last specified value Keywords will be automatically converted to values and vice versa when being passed as arguments to or returned from foreign functions respectively The same applies to any other situations where an object of an enum type is expected Types defined with defcenum canonicalize to base type which is int by default Examples defcenum boolean no yes CFFI gt foreign enum value boolean no gt 0 defcenum numbers one 1 two four 4 CFFI gt foreign enum keyword numbers 2 TWO See Also foreign enum value page 43 foreign enum key word page 42 Chapter 6 Foreign Types 38 define foreign type Syntax define foreign type class name supers slots amp rest options Macro class name options actual type type simple parser symbol regular defclass option Arguments and Values class name A symbol naming the new foreign type class supers A list of symbols naming the super classes slots A list of slot definitions passed to defclass Description The macro define foreign type defines a new clas
17. different page 4 we mentioned that writing an FFI sometimes requires depending on information not provided as part of the interface The easy option CURLOPT_WRITEDATA which we will not provide as part of the Lisp interface illustrates this issue Strictly speaking the curl option enumeration is not necessary we could have used int 99 instead of curl option nosignal in our call to curl_easy_setopt above We defined it anyway in part to hide the fact that we are breaking the abstraction that the C enum provides If the cURL developers decide to change those numbers later we must change the Lisp enumeration because enumeration values are not provided in the compiled C library libcurl so 3 CFFI works because the most useful things in C libraries non static functions and non static variables are included accessibly in libcurl so 3 A C compiler that violated this would be considered a worthless compiler The other thing define curl options does is give the type of the third argument passed to curl easy setopt Using this information we can tell that the nosignal option should accept a long integer argument We can implicitly assume t 1 and nil 0 as it is in C which takes care of the fact that CURLOPT_NOSIGNAL is really asking for a boolean The type of CURLOPT WRITEDATA is objectpoint However it is really looking for a FILE CURLOPT_ERRORBUFFER is looking for a char so there is no obvious CFFI type but
18. for the slot count Used to declare an array of size count inside the structure Defaults to 1 as such an array and a single element are semantically equivalent offset Overrides the slot s offset The next slot s offset is calculated based on this one Description This defines a new CFFI aggregate type akin to C structs In other words it specifies that foreign objects of the type structure name are groups of different pieces of data or slots of the slot types distinguished from each other by the slot names Each structure is located in memory at a position and the slots are allocated sequentially beginning at that point in memory with some padding allowances as defined by the C ABI unless otherwise requested by specifying an offset from the beginning of the structure offset 0 In other words it is isomorphic to the C struct giving several extra features There are two kinds of slots for the two kinds of CFFI types Simple Contain a single instance of a type that canonicalizes to a built in type such as long or pointer Used for simple CFFI types Aggregate Contain an embedded structure or union or an array of objects Used for aggregate CFFI types The use of CLOS terminology for the structure related features is intentional structure definitions are very much like classes with far fewer features Examples defcstruct point Point structure x int y int CFFI gt with foreign object ptr
19. foreign library or an instance of foreign library as returned by load foreign library See Also define foreign library page 94 load foreign library page 97 use foreign library page 100 Chapter 11 Libraries 93 darwin framework directories Syntax darwin framework directories Special Variable Value type A list in which each element is a string a pathname or a simple Lisp expression Initial value A list containing the following in order an expression corresponding to Darwin path Library Frameworks P Library Frameworks and P System Library Frameworks Description The meaning of simple Lisp expression is explained in foreign library directories page 95 In contrast to that variable this is not a fallback search path the default value described above is intended to be a reasonably complete search path on Darwin systems Examples CFFI gt load foreign library framework OpenGL gt P System Library Frameworks OpenGL framework OpenGL See also foreign library directories page 95 define foreign library page 94 Chapter 11 Libraries 94 define foreign library Syntax define foreign library name and options load clause Macro name name and options name name amp key convention load clause feature library amp key convention Arguments and Values name A symbol feature A feature expression library A library designator conve
20. handle gt lt FOREIGN ADDRESS x09844EE0 gt CFFI USER gt set curl option nosignal easy handle 1 gt 0 CFFI USER gt set curl option url easy handle http www cliki net CFFI gt 0 8 There are advantages and disadvantages to each approach I chose to setf symbol function earlier because it entailed generating fewer magic function names Chapter 4 An Introduction to Foreign Interfaces and CFFI 15 For fun let s inspect the Lisp value of the C string that was created to hold http www cliki net CFFI By virtue of the implementation of add curl handle cstring it should be accessible through the hash table defined CFFI USER gt foreign string to lisp car gethash easy handle easy handle cstrings gt http www cliki net CFFI Looks like that worked and libcurl now knows what URL we want to retrieve Finally we turn back to the errorbuffer option mentioned at the beginning of this section Whereas the abstraction added to support string inputs works fine for cases like CURLOPT URL it hides the detail of keeping the C string for errorbuffer however we need that C string In a moment we ll define something slightly cleaner but for now remember that you can always hack around anything We re modifying handle creation so make sure you free the old handle before redefining free easy handle defvar easy handle errorbuffers make hash table Hashtable of easy handles to C strin
21. later can write a C string of up to 255 characters there We the callers of the function are expected to read the C string at a later time exactly the opposite of what string implies With the semantics for an input string in mind namely that the string should be kept around until we curl easy cleanup the easy handle we are ready to extend the Lisp interface defvar easy handle cstrings make hash table Hashtable of easy handles to lists of C strings that may be safely freed after the handle is freed defun make easy handle Answer a new CURL easy interface handle to which the lifetime of C strings may be tied See add curl handle cstring let easy handle curl easy init setf gethash easy handle easy handle cstrings easy handle defun free easy handle handle T But I thought Lisp was supposed to protect me from all that buggy C crap Before asking a question like that remember that you are a stranger in a foreign land whose residents have a completely different set of values Chapter 4 An Introduction to Foreign Interfaces and CFFI 14 Free CURL easy interface HANDLE and any C strings created to be its options curl easy cleanup handle mapc foreign string free gethash handle easy handle cstrings remhash handle easy handle cstrings defun add curl handle cstring handle cstring Add CSTRING to be freed when HANDLE is answering CSTRING car push c
22. library directories Syntax foreign library directories Special Variable Value type A list in which each element is a string a pathname or a simple Lisp expression Initial value The empty list Description You should not have to use this variable Most if not all Lisps supported by CFFI have a reasonable default search algorithm for foreign libraries For example Lisps for UNIX usually call dlopen 3 which in turn looks in the system library directories Only if that fails does CFFI look for the named library file in these directories and load it from there if found Thus this is intended to be a CFFI only fallback to the library search configuration provided by your operating system For example if you distribute a foreign library with your Lisp package you can add the library s containing directory to this list and portably expect CFFI to find it A simple Lisp expression is intended to provide functionality commonly used in search paths such as ASDF s and is defined recursively as follows 1 A list whose first is a function designator and whose rest is a list of simple Lisp expressions to be evaluated and passed to the so designated function The result is the result of the function call 2 A symbol whose result is its symbol value 3 Anything else evaluates to itself Examples 1s liblibli so libli lisp In libli lisp pushnew P home sirian lisp libli foreign l
23. load foreign library error Condition Type Class precedence list load foreign library error error serious condition condition t Description Signalled when a foreign library load completely fails The exact meaning of this varies depending on the real conditions at work but almost universally the implementation s error message is useless However CFFI does provide the useful restarts retry and use value invoke the retry restart to try loading the foreign library again or the use value restart to try loading a different foreign library designator See also load foreign library page 97 Chapter 11 Libraries 100 use foreign library Syntax use foreign library name Macro Arguments and values name A library designator unevaluated Description See load foreign library page 97 for the meaning of library designator This is intended to be the top level form used idiomatically after a define foreign library form to go ahead and load the library Finally on implementations where the regular evaluation rule is insufficient for foreign library loading it loads it at the required time Examples See Section 4 3 Loading foreign libraries page 5 See also load foreign library page 97 1 Namely CMUCL See use foreign library in libraries lisp for details Chapter 12 Callbacks 101 12 Callbacks Chapter 12 Callbacks 102 callback Syntax callback symbol gt pointer Macro
24. not available as a foreign type However on such platforms CFFI provides its own implementation of the long long type for all of operations in chapters Chapter 6 Foreign Types page 22 Chapter 7 Pointers page 55 and Chapter 9 Variables page 81 The function ality described in Chapter 10 Functions page 85 and Chapter 12 Callbacks page 101 will not be available 32 bit Lispworks 5 0 is an exception In addition to the CFFI implementation described above Lispworks itself implements the long long type for Chapter 10 Functions page 85 Chapter 12 Callbacks page 101 are still missing long long support though cffi sys no stdcall This Lisp doesn t support the stdcall calling convention Note that it only makes sense to support stdcall on 32 bit x86 platforms Appendix B Glossary 112 Appendix B Glossary aggregate type A CFFI type for C data defined as an organization of data of simple type in structures and unions which are themselves aggregate types they are repre sented by value foreign value This has two meanings in any context only one makes sense When using type translators the foreign value is the lower level Lisp value derived from the object passed to translate to foreign see translate to foreign page 53 This value should be a Lisp number or a pointer satisfies pointerp and it can be treated like any general Lisp object it only completes the transformation to a true foreign value when
25. point Assuming we have changed the type of the third argument underlying set curl option url to string look at these two equivalent forms set curl option url easy handle http www cliki net CFFI with foreign string url http www cliki net CFFI foreign funcall curl easy setopt easy handle easy handle curl option url pointer url curl code The latter in fact is mostly equivalent to what a foreign function call s macroexpansion actually does As you can see the Lisp string http www cliki net CFFI is copied into char array and null terminated the pointer to beginning of this array now a C string is passed as a CFFI pointer to the foreign function Unfortunately the C abstraction has failed us and we must break it While string works well for many char arguments it does not for cases like this As the curl easy setopt documentation explains The string must remain present until curl no longer needs it as it doesn t copy the string The C string created by with foreign string however only has dynamic extent it is deallocated when the body above containing the foreign funcall form exits Chapter 4 An Introduction to Foreign Interfaces and CFFI 13 If we are supposed to keep the C string around but it goes away what happens when some libcurl function tries to access the URL string We have reentered the dreaded world of C undefined behavior In some Lisps it will probably get a ch
26. the semantic that C imposes natively C values are accessed as the setf able places defined by mem aref and mem ref Given a pointer and a CFFI type see Chapter 6 Foreign Types page 22 either of these will dereference the pointer translate the C data there back to Lisp and return the result of 1 The definition of memory includes the CPU registers Chapter 7 Pointers 56 said translation performing the reverse operation when setf ing To decide which one to use consider whether you would use the array index operator n or the pointer dereference in C use mem aref for array indexing and mem ref for pointer dereferencing Chapter 7 Pointers 5T foreign free Syntax foreign free ptr undefined Function Arguments and Values ptr A foreign pointer Description The foreign free function frees a ptr previously allocated by foreign alloc The con sequences of freeing a given pointer twice are undefined Examples CFFI gt foreign alloc int lt A Mac Pointer x1022E0 gt CFFI gt foreign free NIL See Also foreign alloc page 58 with foreign pointer page 72 Chapter 7 Pointers 58 foreign alloc Syntax foreign alloc type amp key initial element initial contents Function count 1 null terminated p pointer Arguments and Values type A foreign type initial element A Lisp object initial contents A sequence count An integer Defaults to 1 or the length of initial cont
27. to the foreign symbol denoted by the string foreign name If a foreign symbol named foreign name doesn t exist nil is returned ABI name manglings will be performed on foreign name by foreign symbol pointer if necessary eg adding a leading underscore on darwin ppc library should name a foreign library as defined by define foreign library default which is the default or an instance of foreign library as returned by load foreign library Important note do not keep these pointers across saved Lisp cores as the foreign library may move across sessions Examples CFFI gt foreign symbol pointer errno gt lt A Mac Pointer xA0008130 gt CFFI gt foreign symbol pointer strerror lt A Mac Pointer x9002D0F8 gt CFFI gt foreign funcall pointer int mem ref int string No such file or directory CFFI gt foreign symbol pointer inexistent symbol NIL See Also defcvar page 82 Chapter 7 Pointers inc pointer Syntax inc pointer pointer offset gt new pointer Arguments and Values pointer new pointer A foreign pointer offset An integer Description 61 Function The function inc pointer will return a new pointer pointing offset bytes past pointer Examples CFFI gt foreign string alloc Common Lisp lt A Mac Pointer x102EA0 gt CFFI gt inc pointer 7 lt A Mac Pointer x102EA7 gt CFFI gt foreign string to lisp gt Lisp See Also
28. type pointer the actual pointer object varies from Common Lisp implementation to implementation needing only to satisfy pointerp and be returned from make pointer and friends One solution that would allow us to define a new Lisp structure to represent easy handles would be to write a wrapper around every function that currently takes an easy handle the wrapper would extract the pointer and pass it to the foreign function However we can use type translators to more elegantly integrate this translation into the foreign function calling framework using translate to foreign defclass easy handle pointer initform curl easy init documentation Foreign pointer from curl easy init error buffer initform foreign alloc char count curl error size initial element 0 documentation C string describing last error c strings initform documentation C strings set as options documentation I am a parameterization you may pass to curl easy perform to perform a cURL network protocol request defmethod initialize instance after self easy handle amp key set curl option errorbuffer self slot value self error buffer defun add curl handle cstring handle cstring Add CSTRING to be freed when HANDLE is answering CSTRING car push cstring slot value handle c strings defun get easy handle error handle Answer a string containing HANDLE s current error message foreign string to lisp
29. until max chars is reached or a NULL character is found If ptr is a null pointer returns nil Note that the string type will automatically convert between Lisp strings and foreign strings Examples CFFI gt foreign funcall getenv string HOME pointer gt lt FOREIGN ADDRESS xBFFFFFD5 gt CFFI gt foreign string to lisp gt Users luis See Also lisp string to foreign page 78 foreign string alloc page 75 Chapter 8 Strings 78 lisp string to foreign Syntax lisp string to foreign string buffer bufsize amp key start Function end offset encoding buffer Arguments and Values string A Lisp string buffer A foreign pointer bufsize An integer start end Bounding index designators of string 0 and nil by default offset An integer greater than or equal to 0 Defauls to 0 encoding Foreign encoding Defaults to default foreign encoding Description The lisp string to foreign function copies at most bufsize 1 octets from a Lisp string using the specified encoding into buffer offset The foreign string will be null terminated Start specifies an offset into string and end marks the position following the last element of the foreign string Examples CFFI gt with foreign pointer as string str 255 lisp string to foreign Hello foreign world str 6 Hello See Also foreign string alloc page 75 foreign string to lisp page 77 with foreign pointer as string page
30. 00 gt CFFI gt null pointer p gt T CFFI gt typep foreign pointer gt T See Also inc pointer page 61 null pointer page 66 null pointer p page 67 pointerp page 68 pointer address page 69 pointer eq page 70 mem ref page 65 63 Function Chapter 7 Pointers 64 mem aref Syntax mem aref ptr type amp optional index 0 Accessor setf mem aref ptr type amp optional index 0 new value Arguments and Values ptr A foreign pointer type A foreign type index An integer new value A Lisp value compatible with type Description The mem aref function is similar to mem ref but will automatically calculate the offset from an index mem aref ptr type n is identical to mem ref ptr type n foreign type size type Examples CFFI gt with foreign string str Hello foreign world mem aref str char 6 gt 32 CFFI gt code char gt Space CFFI gt with foreign object array int 10 loop for i below 10 do setf mem aref array int i random 100 loop for i below 10 collect mem aref array int i gt 22 7 22 52 69 1 46 93 90 65 See Also mem ref page 65 Chapter 7 Pointers 65 mem ref Syntax mem ref ptr type amp optional offset object Accessor Arguments and Values ptr A pointer type A foreign type offset An integer in byte units object The value ptr points to Description Examples CFFI gt wit
31. 80 Chapter 8 Strings 79 with foreign string with foreign strings Syntax with foreign string var or vars string amp rest args amp body Macro body with foreign strings bindings amp body body Macro var or vars var var amp optional octet size var bindings var or vars string amp rest args Arguments and Values var byte size var A symbol string A Lisp string body A list of forms to be executed Description The with foreign string macro will bind var to a newly allocated foreign string contain ing string Args is passed to the underlying foreign string alloc call If octet size var is provided it will be bound the length of foreign string in octets in cluding the null terminator Examples CFFI gt with foreign string foo 12345 foreign funcall strlen pointer foo int gt 5 CFFI gt let array coerce 84 117 114 97 110 103 97 array unsigned byte 8 with foreign string foreign string array foreign string to lisp foreign string Turanga See Also foreign string alloc page 75 with foreign pointer as string page 80 Chapter 8 Strings 80 with foreign pointer as string Syntax with foreign pointer as string var size amp optional size var Macro amp rest args amp body body string Arguments and Values var A symbol string A Lisp string body List of forms to be executed Description The with foreign pointer as string macro is
32. Arguments and Values symbol A symbol denoting a callback pointer new value lt A pointer Description The callback macro is analogous to the standard CL special operator function and will return a pointer to the callback denoted by the symbol name Examples CFFI gt defcallback sum int a int b int a b gt SUM CFFI gt callback sum gt lt A Mac Pointer x102350 gt See Also get callback page 105 defcallback page 103 Chapter 12 Callbacks 103 defcallback Syntax defcallback name and options return type arguments amp body Macro body name name and options name name amp key convention arguments 1 arg name arg type Arguments and Values name A symbol naming the callback created return type The foreign type for the callback s return value arg name A symbol arg type A foreign type convention One of cdecl default or stdcall Description The defcallback macro defines a Lisp function that can be called from C The arguments passed to this function will be converted to the appropriate Lisp representation and its return value will be converted to its C representation This Lisp function can be accessed by the callback macro or the get callback function Portability note defcallback will not work correctly on some Lisps if it s not a top level form Examples defcfun qsort void base pointer nmemb int size int fun compar poin
33. CFFI User Manual Copyright 2005 James Bielman lt jamesjb at jamesjb com gt Copyright c 2005 2010 Lu s Oliveira lt loliveira at common lisp net gt Copyright c 2006 Stephen Compall s11 at member fsf org Permission is hereby granted free of charge to any person obtaining a copy of this software and associated documentation files the Software to deal in the Software without restriction including without limitation the rights to use copy modify merge publish distribute sublicense and or sell copies of the Software and to permit persons to whom the Software is furnished to do so subject to the following conditions The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software THE SOFTWARE IS PROVIDED AS IS WITHOUT WARRANTY OF ANY KIND EX PRESS OR IMPLIED INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MER CHANTABILITY FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM DAMAGES OR OTHER LIABILITY WHETHER IN AN ACTION OF CONTRACT TORT OR OTHERWISE ARISING FROM OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE Table of Contents 1 JHPBOOHODIOTE coena Jaco ie een sn as 1 2 Installation naar P aa 2 3 Implementation Support 3 3 1 Limutatilons coepere aetates Respice aient dram eam 3 An
34. Introduction to Foreign Interfaces and CERE eeen e a e dde neas des Iba s psi 4 4 1 What makes Lisp different 00 cece eens 4 4 2 Getting URL cete dee ia 5 4 3 Loading foreign libraries 0000 5 AA Initializing bibcurl uc ve eem RIPE as ana 6 4 5 Setting download options 2 0 e cece eee eee een eee eee T 4 6 Breaking the abstraction 0c eect eens 9 4 7 Option functions in Lisp 0 00 0 cece eee eee 10 AS Memory management rennen 12 4 9 Calling Lisp from u ru ke e Re riksin E reni ES 15 AlO A complete BRL 5o erre edP an 17 4 11 Defining new types 0 cece cece eee nee ees 18 4 12 What s netten sis 20 Wrapper generators ueesss 21 Foreign Types eise RE RR RR Os 22 6 1 Built In Py ex duse pos binos Sates wel e Ro ar foem ate d OR ae et 22 6 2 Other Types seem Reel Ere Rp ERE DE LECCE ERR RE 23 6 3 Defining Foreign Types 0 0 cece eee eee ee 24 6 4 Foreign Type Translators sees 25 6 5 Optimizing Type Translators sese 26 6 6 Foreign Structure Types 0 0 c cece eee eee eee 2 6 7 Allocating Foreign Objects 28 Pointers qus sace soos eat bots Pang UT atop 55 7 1 Basic Pointer Operations esses eee eee 55 7 2 Allocating Foreign Memory sssssssesss e eee ee ee 55 7 3 Accessing Foreign Memory 00 cece e eee eee eee eee 55 S Ole nen Pai SAVEURS doe qua dedu
35. Types 25 define parse method my string amp key encoding utf 8 make instance my string type encoding encoding The next section describes how make this type actually translate between C and Lisp strings 6 4 Foreign Type Translators Type translators are used to automatically convert Lisp values to or from foreign values For example using type translators one can take the my string type defined in the previous section and specify that it should e convert C strings to Lisp strings e convert Lisp strings to newly allocated C strings e free said C strings when they are no longer needed In order to tell CFFI how to automatically convert Lisp values to foreign values define a specialized method for the translate to foreign generic function Define a method that converts Lisp strings to C strings defmethod translate to foreign string type my string type foreign string alloc string encoding string type encoding type From now on whenever an object is passed as a my string to a foreign function this method will be invoked to convert the Lisp value To perform the inverse operation which is needed for functions that return a my string specialize the translate from foreign generic function in the same manner Define a method that converts C strings to Lisp strings defmethod translate from foreign pointer type my string type foreign string to lisp pointer encoding string type encoding type
36. a not a good idea to USE when names may clash with say CL symbols defpackage example internal use nicknames exampleint defpackage example software export use cl cffi exampleint The internal package is created by Lisp code output from the C program written by CFFI Grovel if your specification file is exampleint lisp the exampleint cffi lisp file will contain the CFFI definitions needed by the rest of your project See Section 13 2 Groveller Syntax page 106 13 4 Implementation Notes Implementor s note This info might not be up to date For foo internal lisp the resulting foo internal c foo internal and foo internal cffi lisp are all platform specific either because of possible reader macros Chapter 13 The Groveller 109 in foo internal lisp or because of varying C environments on the host system For this reason it is not helpful to distribute any of those files end users building CFFI Grovel based software will need cffi Grovel anyway If you build with multiple architectures in the same directory e g with NFS AFS home directories it is critical to remove these generated files or the resulting constants will be very incorrect Implementor s note Maybe we should tag the generated names with something host or OS specific Implementor s note For now after some experimentation with CLISP having no long long it seems appropriate to assert that the generated c files are arc
37. ations where an object of a bitfield type is expected Types defined with defbitfield canonicalize to base type which is int by default Examples defbitfield open flags rdonly x0000 wronly x0001 rdwr aus nonblock append creat x0200 elC CFFI gt foreign bitfield symbols open flags b1101 RDONLY WRONLY NONBLOCK APPEND CFFI gt foreign bitfield value open flags rdwr creat gt 514 x0202 defcfun open unix open int Chapter 6 Foreign Types path string flags open flags mode uint16 unportable CFFI gt unix open tmp foo wronly creat 0644 gt lt an fd Consider also the following lispier wrapper around open defun lispier open path mode amp rest flags unix open path flags mode See Also foreign bitfield value page 41 foreign bitfield symbols page 40 32 Chapter 6 Foreign Types 33 defcstruct Syntax defcstruct name and options amp body doc and slots name Macro name and options structure name structure name amp key size doc and slots docstring slot name slot type amp key count offset Arguments and Values structure name The name of new structure type docstring A documentation string ignored slot name A symbol naming the slot It must be unique among slot names in this structure size Use this option to override the size in bytes of the struct slot type The type specifier
38. cept NAME is a keyword TYPE OFFSETS is a plist of TYPEs to their integer offsets as defined by the CURLOPTTYPE LONG et al constants in curl h Also define functions for each option named set TYPE NAME OPTION NAME where OPTION NAME is the NAME from the above destructuring flet enumerated value type offset getf type offsets type offset map PROCEDURE destructuring each of ENUM ARGS map enum args procedure mapcar lambda arg apply procedure arg enum args build a name like SET CURL OPTION NOSIGNAL make setter name option name intern concatenate string SET symbol name type name symbol name option name progn defcenum type name map enum args lambda name type number list name enumerated value type number map enum args lambda name type number declare ignore number define curl option setter make setter name name type name name ecase type long long objectpoint pointer functionpoint pointer off t long gt type name Macroexpanding our define curl options form once more we see something different progn defcenum curl option noprogress 43 nosignal 99 errorbuffer 10010 url 10002 define curl option setter set curl option noprogress curl option noprogress long define curl option setter set curl option nosignal curl option nosignal long Chapter 4 An Introduction to Foreign Interfaces and CFFI 12
39. cfun curl_easy_perform curl code handle easy handle CFFI USER gt with output to string contents let keasy write procedure lambda string write string string contents declare special easy write procedure curl easy perform easy handle gt lt DOCTYPE HTML PUBLIC W3C DTD HTML 4 01 EN Now fear comprehensively lt P gt 7 Unfortunately we can t protect against all non local exits such as returns and throws because unwind protect cannot be used to short circuit a non local exit in Common Lisp due to proposal minimal in ANSI issue EXIT EXTENT Furthermore binding an error handler prevents higher up code from invoking restarts that may be provided under the callback s dynamic context Such is the way of compromise Chapter 4 An Introduction to Foreign Interfaces and CFFI 18 Of course that itself is slightly unwieldy so you may want to define a function around it that simply retrieves a URL I will leave synthesis of all the relevant REPL forms presented thus far into a single function as an exercise for the reader The remaining sections of this tutorial explore some advanced features of CFFI the definition of new types will receive special attention Some of these features are essential for particular foreign function calls some are very helpful when trying to develop a Lispy interface to C 4 11 Defining new types We ve occasionally used the defctype macro in previous secti
40. cializing type name on a symbol naming a foreign type defined with defctype This method should return the appropriate foreign value to use in place of the Lisp value In cases where CFFI can determine the lifetime of the foreign object returned by this method it will invoke free translated object on the foreign object at the appropriate time If translate to foreign returns a second value it will be passed as the param argu ment to free translated object This can be used to establish communication between the allocation and deallocation methods The results are undefined if the type name parameter is specialized in any way except an EQL specializer on a foreign type defined with defctype Specifically translations may not be defined for built in types See Also Section 6 4 Foreign Type Translators page 25 translate from foreign page 52 free translated object page 51 Chapter 6 Foreign Types 54 with foreign slots Syntax with foreign slots vars ptr type amp body body Macro Arguments and Values vars A list of symbols ptr A foreign pointer to a structure type A structure type body A list of forms to be executed Description The with foreign slots macro creates local symbol macros for each var in vars to refer ence foreign slots in ptr of type It is similar to Common Lisp s with slots macro Examples defcstruct tm sec int min int hour int mday int mon int year int wday
41. cific Notes e Corman Lisp does not support foreign funcall On implementations that don t support foreign funcall cffi sys no foreign funcall will be present in features Note in these Lisps you can still use the defcfun interface Examples CFFI gt foreign funcall pointer foreign symbol pointer abs int 42 int gt 42 See Also defcfun page 86 foreign funcall page 88 Chapter 11 Libraries 91 11 Libraries 11 1 Defining a library Almost all foreign code you might want to access exists in some kind of shared library The meaning of shared library varies among platforms but for our purposes we will consider it to include so files on UNIX frameworks on Darwin and derivatives like Mac OS X and d1l files on Windows Bringing one of these libraries into the Lisp image is normally a two step process 1 Describe to CFFI how to load the library at some future point depending on platform and other factors with a define foreign library top level form 2 Load the library so defined with either a top level use foreign library form or by calling the function 1oad foreign library See Section 4 3 Loading foreign libraries page 5 for a working example of the above two steps 11 2 Library definition style Looking at the libcurl library definition presented earlier you may ask why we did not simply do this define foreign library libcurl t default libcurl Indeed this would wo
42. ction to Foreign Interfaces and CFFI 16 WRITEFUNCTION to be called by libcurl rather than the reverse in order to receive data as it is downloaded A binding writer without the aid of FFI usually approaches this problem by writing a C function that accepts C data converts to the language s internal objects and calls the callback provided by the user again in a reverse of usual practices The CFFI approach to callbacks precisely mirrors its differences with the non FFI ap proach on the calling C from Lisp side which we have dealt with exclusively up to now That is you define a callback function in Lisp using defcallback and CFFI effectively creates a C function to be passed as a function pointer Implementor s note This is much trickier than calling C functions from Lisp as it literally involves somehow generating a new C function that is as good as any created by the compiler Therefore not all Lisps support them See Chapter 3 Implementation Support page 3 for information about CFFI support issues in this and other areas You may want to consider changing to a Lisp that supports callbacks in order to continue with this tutorial Defining a callback is very similar to defining a callout the main difference is that we must provide some Lisp forms to be evaluated as part of the callback Here is the signature for the function the writefunction option takes size t function void ptr size t size size t nmemb void stream
43. een the calling conventions of the language and C Common Lisp Bindings are written in Lisp They can be created at will by Lisp programs Lisp programmers can write new bindings and add them to the image using a listener such as SLIME as easily as with regular Lisp definitions The only foreign library to load is the one being wrapped the one with the pure C interface no C or other non Lisp compilation is required We believe the advantages of the Common Lisp approach far outweigh any disadvantages Incremental development with a listener can be as productive for C binding development as it is with other Lisp development Keeping it in the Lisp family as it were makes it much easier for you and other Lisp programmers to load and use the bindings Common Lisp implementations such as CMUCL freed from having to provide a C interface to their own objects are thus freed to be implemented in another language as CMUCL is while still allowing programmers to call foreign functions Perhaps the greatest advantage is that using an FFI doesn t obligate you to become a professional binding developer Writers of bindings for other languages usually end up maintaining or failing to maintain complete bindings to the foreign library Using an FFI however means if you only need one or two functions you can write bindings for only those functions and be assured that you can just as easily add to the bindings if need be Chapter 4 An Introductio
44. ents if supplied null terminated p A boolean false by default pointer A foreign pointer to the newly allocated memory Description The foreign alloc function allocates enough memory to hold count objects of type type and returns a pointer This memory must be explicitly freed using foreign free once it is no longer needed If initial element is supplied it is used to initialize the count objects the newly allocated memory holds If an initial contents sequence is supplied it must have a length less than or equal to count and each of its elements will be used to initialize the contents of the newly allocated memory If count is omitted and initial contents is specified it will default to length initial contents initial element and initial contents are mutually exclusive When null terminated p is true 1 max count length initial contents ele ments are allocated and the last one is set to NULL Note that in this case type must be a pointer type ie a type that canonicalizes to pointer otherwise an error is signaled Examples CFFI gt foreign alloc char gt lt A Mac Pointer x102D80 gt A pointer to 1 byte of memory CFFI gt foreign alloc char count 20 gt lt A Mac Pointer x1024A0 gt A pointer to 20 bytes of memory CFFI gt foreign alloc int initial element 12 gt lt A Mac Pointer x1028B0 gt Chapter 7 Pointers 59 CFFI gt mem ref int 12 CFFI gt foreign alloc
45. face writing job Reports of the amount of work done by generators vary from 30 to 9096 The incremental development style enabled by CFFI generally reduces this proportion below that for languages like Python Chapter 6 Foreign Types 6 Foreign Types 22 Foreign types describe how data is translated back and forth between C and Lisp CFFI provides various built in types and allows the user to define new types 6 1 Built In Types char unsigned char short unsigned short int unsigned int long unsigned long long long unsigned long long Foreign Type Foreign Type Foreign Type Foreign Type Foreign Type Foreign Type Foreign Type Foreign Type Foreign Type Foreign Type These types correspond to the native C integer types according to the ABI of the Lisp implementation s host system long long and unsigned long long are not supported natively on all implementa tions However they are emulated by mem ref and mem set When those types are not available the symbol cffi sys no long long is pushed into features uchar ushort uint ulong llong ullong Foreign Type Foreign Type Foreign Type Foreign Type Foreign Type Foreign Type For convenience the above types are provided as shortcuts for unsigned char unsigned short unsigned int unsigned long long long and unsigned long long respectively int8 uint8 int16 uint16 int32 uint32 Foreign Type Fo
46. finds first Similarly for af packet but no error will be signalled if the platform supports neither AF_PACKET nor PF_PACKET constantenum address family af inet AF INET PF INET documentation IPv4 Protocol family af local AF UNIX AF LOCAL PF UNIX PF LOCAL documentation File domain sockets Chapter 13 The Groveller 108 af inet6 AF INET6 PF INET6 documentation IPv6 Protocol family af packet AF PACKET PF PACKET documentation Raw packet access optional t 13 3 ASDF Integration An example software project might contain four files an ASDF file a package definition file an implementation file and a CFFI Grovel specification file The ASDF file defines the system and its dependencies Notice the use of eval when to ensure CFFI Grovel is present and the use of cffi grovel grovel file name amp key cc flags instead of file name CFFI Grovel is needed for processing grovel file components cl eval when load toplevel execute asdf operate asdf load op cffi grovel asdf defsystem example software depends on cffi serial t components file package cffi grovel grovel file example grovelling file example The package lisp file would contain several defpackage forms to remove circular de pendencies and make building the project easier Note that you may or may not want to use your internal package Implementor s note Mention that it s
47. gned long long m ek rna 22 unsigned shOrt eiilalescsac e d cena 22 rh 22 VOU 23 wrapper base type amp key to c from c 24 A abstraction breaking 0 000 9 abstractions in Cis creed e veo ses eg obs 4 advantages of FEL eek RE REIR 4 B benehts of PPT u RR Rr ERROR E ee ed 4 breaking the abstraction 9 C C abstractions ie 3 eer RE a 4 callback definition eee t ee Rene 16 callback symbol pointer 102 calling foreign functions 0 6 CC flagSBi ins l4 ua es ARA AREA AUR ERU ERA RE 107 CONUM PTT 107 113 close foreign library library success 92 compiler macros for type translation 26 C nStahb ier nd een 106 const ntenum i euoaa ke eee unde ecaee des 107 convert from foreign foreign value type gt VALUS istics e rn UR Rx nes eee A CR ERE 29 convert to foreign value type foreign value alloc params 30 CSEIUCE sec ek e Ree e en ARTE OE KE 107 estruct and class oe e ee wee 107 GLYDG LspLIIteIm een POPE EP Det 106 G nlOBi 1 5 lessesetur Hast we EU Rue eere 107 CUR cas 6cemcnupeer b eene BT ERE RE Dd Id 5 CVAE Leere P TVIYE Ya p Hepa sede Y Ya bd eae es 107 data in Lisp and Cu et in 18 defbitfield name and options amp body masks 31 defcallback name and options return type arguments amp body body name
48. gs serving as error writeback buffers An extra byte is very little to pay for peace of mind defparameter curl error size 257 Minimum char size used by cURL to report errors defun make easy handle Answer a new CURL easy interface handle to which the lifetime of C strings may be tied See add curl handle cstring let easy handle curl easy init setf gethash easy handle easy handle cstrings setf gethash easy handle easy handle errorbuffers foreign alloc char count curl error size initial element 0 easy handle defun free easy handle handle Free CURL easy interface HANDLE and any C strings created to be its options curl easy cleanup handle foreign free gethash handle easy handle errorbuffers remhash handle easy handle errorbuffers mapc foreign string free gethash handle easy handle cstrings remhash handle easy handle cstrings defun get easy handle error handle Answer a string containing HANDLE s current error message foreign string to lisp gethash handle easy handle errorbuffers Be sure to once again set the options we ve set thus far You may wish to define yet another wrapper function to do this 4 9 Calling Lisp from C If you have been reading curl easy setopt 3 you should have noticed that some options accept a function pointer In particular we need one function pointer to set as CURLOPT_ Chapter 4 An Introdu
49. h foreign string ptr Saluton setf mem ref ptr char 3 char code a loop for i from 0 below 8 collect code char mem ref ptr char i gt GENS a 1 a t o n Null CFFI gt setq ptr to int foreign alloc int lt A Mac Pointer x1047D0 gt CFFI gt mem ref ptr to int int 1054619 CFFI gt setf mem ref ptr to int int 1984 1984 CFFI gt mem ref ptr to int int 1984 See Also mem aref page 64 Chapter 7 Pointers null pointer Syntax null pointer pointer Arguments and Values pointer A NULL pointer Description The function null pointer returns a null pointer Examples CFFI gt null pointer gt lt A Null Mac Pointer CFFI gt pointerp gt T See Also null pointer p page 67 make pointer page 63 66 Function Chapter 7 Pointers 67 null pointer p Syntax null pointer p ptr boolean Function Arguments and Values ptr A foreign pointer that may be a null pointer boolean T or NIL Description The function null pointer p returns true if ptr is a null pointer and false otherwise Examples CFFI gt null pointer p null pointer gt T defun contains str p big little not null pointer p foreign funcall strstr string big string little pointer CFFI gt contains str p Popcorns corn gt T CFFI gt contains str p Popcorns salt NIL See Also null pointer page 66 pointerp
50. he encapsulation anyway placing the burden of correct library interface definition on the library 2 w Specifically UFFI an older FFI that takes a somewhat different approach compared to CFFI I believe that these days December 2005 CFFI is more portable and actively developed though not as mature yet Consensus in the free UNIX Common Lisp community seems to be that CFFI is preferred for new development though UFFI will likely go on for quite some time as many projects already use it CFFI includes the UFFI COMPAT package for complete compatibility with UFFI Chapter 4 An Introduction to Foreign Interfaces and CFFI 6 asdf oos asdf load op cffi Nothing special about the CFFI USER package We re just using it as a substitute for your own CL package defpackage cffi user use common lisp cffi in package cffi user define foreign library libcurl unix or libcurl so 3 libcurl so t default libcurl use foreign library libcurl Using define foreign library and use foreign library we have loaded libcurl into Lisp much as the linker does when you start a C program or common lisp load does with a Lisp source file or FASL file We special cased for UNIX machines to always load a particular version the one this tutorial was tested with for those who don t care the define foreign library clause t default libcurl should be satisfactory and will adapt to various operating systems
51. hitecture and operating system dependent but lisp implementation independent This way the same c file and so the same grovel tmp lisp file will be shareable between the implementations running on a given system Chapter 14 Limitations 110 14 Limitations These are CFFI s limitations across all platforms for information on the warts on particular Lisp implementations see Chapter 3 Implementation Support page 3 e The tutorial includes a treatment of the primary intractable limitation of CFFI or any FFI that the abstractions commonly used by C are insufficiently expressive See Section 4 6 Breaking the abstraction page 9 for more details e C structs cannot be passed by value Appendix A Platform specific features 111 Appendix A Platform specific features Whenever a backend doesn t support one of CFFI s features a specific symbol is pushed onto common lisp features The meanings of these symbols follow cfh sys flat namespace This Lisp has a flat namespace for foreign symbols meaning that you won t be able to load two different libraries with homograph functions and successfully differentiate them through the library option to defcfun defcvar etc cfh sys no foreign funcall The macro foreign funcall is not available On such platforms the only way to call a foreign function is through defcfun See foreign funcall page 88 and defcfun page 86 cffi sys no long long The C long long type is
52. ibcurl is a library for downloading files over protocols like HTTP We will use libcurl with CFFI to download a web page Please note that there are many other ways to download files from the web not least the CL CURL project to provide bindings to libcurl via a similar FFI libcurl tutorial 3 is a tutorial for libcurl programming in C We will follow that to develop a binding to download a file We will also use curl h easy h and the man pages for the libcurl function all available in the curl dev package or equivalent for your system or in the cURL source code package If you have the development package the headers should be installed in usr include curl and the man pages may be accessed through your favorite man facility 4 3 Loading foreign libraries First of all we will create a package to work in You can save these forms in a file or just send them to the listener as they are If creating bindings for an ASDF package of yours you will want to add cffi to the depends on list in your asd file Otherwise just use the asdf oos function to load CFFI l Admittedly this is an advanced issue and we encourage you to leave this text until you are more familiar with how CFFI works This does not apply to structs whose contents are intended to be part of the public library interface In those cases a pure Lisp struct definition is always preferred In fact many prefer to stay in Lisp and break t
53. ibrary directories test equal load foreign library default liblibli The following example would achieve the same effect See Section Using asdf to load systems in asdf another system definition facility for information on asdf central registry See mini eval in libraries lisp for the source of this definition As is always the case with a Lisp eval it s easier to understand the Lisp definition than the english Chapter 11 Libraries 96 pushnew merge pathnames p lisp libli user homedir pathname foreign library directories test equal MERGE PATHNAMES P lisp libli USER HOMEDIR PATHNAME load foreign library default liblibli See also darwin framework directories page 93 define foreign library page 94 Chapter 11 Libraries 97 load foreign library Syntax load foreign library library handler Function Arguments and Values library A library designator handler An instance of foreign library Description Load the library indicated by library A library designator is defined as follows 1 If a symbol is considered a name previously defined with define foreign library 2 Ifastring or pathname passed as a namestring directly to the implementation s foreign library loader If that fails search the directories in foreign library directories with cl probe file if found the absolute path is passed to the implementation s loader 3
54. ing the in memory representation as needed to create an array of type objects Examples defcstruct foo a double c char CFFI gt foreign type size double gt 8 CFFI gt foreign type size char gt 1 CFFI gt foreign type size foo gt 16 See Also foreign type alignment page 48 Chapter 6 Foreign Types 50 free converted object Syntax free converted object foreign value type params Function Arguments and Values foreign value The C object to be freed type A CFFI type specifier params The state returned as the second value from convert to foreign used to implement the third argument to free translated object Description The return value is unspecified This is an external interface to the type translation facility In the implementation all foreign functions are ultimately defined as type translation wrappers around primitive foreign function invocations This function is available mostly for inspection of the type translation process and possibly optimization of special cases of your foreign function calls Its behavior is better described under free translated object s documentation Examples CFFI USER gt convert to foreign a boat string gt lt FOREIGN ADDRESS x097ACDCO gt T CFFI USER gt free converted object string t gt NIL See Also convert from foreign page 29 convert to foreign page 30 free translated object page 51 Cha
55. int yday int isdst boolean zone string gmtoff long CFFI gt with foreign object time int setf mem ref time int foreign funcall time pointer null pointer int foreign funcall gmtime pointer time tm lt A Mac Pointer x102A30 gt CFFI gt with foreign slots sec min hour mday mon year tm format nil A A A A A A hour min sec 1900 year mon mday gt 7 22 47 2005 8 2 See Also defcstruct page 33 defcunion page 35 foreign slot value page 47 Chapter 7 Pointers 55 7 Pointers All C data in CFFI is referenced through pointers This includes defined C variables that hold immediate values and integers To see why this is consider the case of the C integer It is not only an arbitrary representation for an integer congruent to Lisp s fixnums the C integer has a specific bit pattern in memory defined by the C ABI Lisp has no such constraint on its fixnums therefore it only makes sense to think of fixnums as C integers if you assume that CFFI converts them when necessary such as when storing one for use in a C function call or as the value of a C variable This requires defining an area of memory represented through an effective address and storing it there Due to this compartmentalization it only makes sense to manipulate raw C data in Lisp through pointers to it For example while there may be a Lisp representation of a struct that is converted to C at st
56. int function set would be in addition setting a new option in an old libcurl can generate a run time error rather than breaking the compile Lisp can just as concisely generate functions as compare values and the undefined function error is just as useful as any explicit error we could define here might be 4 7 Option functions in Lisp We could use foreign funcall directly every time we wanted to call curl easy setopt However we can encapsulate some of the necessary information with the following We will use this type later in a more creative way For now just consider it a marker that this isn t just any pointer defctype easy handle pointer defmacro curl easy setopt easy handle enumerated name value type new value Call curl easy setopt on EASY HANDLE using ENUMERATED NAME as the OPTION VALUE TYPE is the CFFI foreign type of the third argument and NEW VALUE is the Lisp data to be translated to the third argument VALUE TYPE is not evaluated foreign funcall curl easy setopt easy handle easy handle curl option enumerated name value type new value curl code Now we define a function for each kind of argument that encodes the correct value type in the above This can be done reasonably in the define curl options macroexpansion after all that is where the different options are listed We could make cl defun forms in the expansion that simply call curl easy setopt however it is probably ea
57. ion Examples CFFI USER gt convert to foreign t boolean gt 1 NIL CFFI USER gt convert to foreign hello world string gt lt FOREIGN ADDRESS x097C5F80 gt gt T CFFI USER gt code char mem aref char 5 gt See Also convert from foreign page 29 free converted object page 50 translate to foreign page 53 Chapter 6 Foreign Types 31 defbitfield Syntax defbitfield name and options amp body masks Macro masks docstring symbol value name and options name name amp optional base type int Arguments and Values name The name of the new bitfield type docstring A documentation string ignored base type A symbol denoting a foreign type symbol A Lisp symbol value An integer representing a bitmask Description The defbitfield macro is used to define foreign types that map lists of symbols to integer values If value is omitted it will be computed as follows find the greatest value previously used including those so computed with only a single 1 bit in its binary representation that is powers of two and left shift it by one This rule guarantees that a computed value cannot clash with previous values but may clash with future explicitly specified values Symbol lists will be automatically converted to values and vice versa when being passed as arguments to or returned from foreign functions respectively The same applies to any other situ
58. ition to Lisp strings this type will accept foreign pointers and pass them unmod ified A method for free translated object page 51 is specialized for this type So for exam ple foreign strings allocated by this type and passed to a foreign function will be freed after the function returns CFFI gt foreign funcall getenv string SHELL string bin bash CFFI gt with foreign string str abcdef foreign funcall strlen string str int gt 6 string tptr Foreign Type Like string but returns a list with two values when convert from C to Lisp a Lisp string and the C string s foreign pointer CFFI gt foreign funcall getenv string SHELL stringtptr bin bash SB SYS INT SAP XBFFFFC6F Chapter 6 Foreign Types 24 boolean amp optional base type int Foreign Type The boolean type converts between a Lisp boolean and a C boolean It canonicalizes to base type which is int by default convert to foreign nil boolean gt 0 convert to foreign t boolean 1 convert from foreign 0 boolean gt nil convert from foreign 1 boolean t wrapper base type amp key to c from c Foreign Type The wrapper type stores two symbols passed to the to c and from c arguments When a value is being translated to or from C this type funcalls the respective symbol wrapper types will be typedefs for base type and will inherit its translators if any Here s an example of how the boo
59. lean type could be defined in terms of wrapper defun bool c to lisp value not zerop value defun bool lisp to c value if value 1 0 defctype my bool wrapper int from c bool c to lisp to c bool lisp to c convert to foreign nil my bool gt 0 convert from foreign 1 my bool gt t 6 3 Defining Foreign Types You can define simple C like typedefs through the defctype macro Defining a typedef is as simple as giving defctype a new name and the name of the type to be wrapped Define MY INT as an alias for the built in type INT defctype my int int With this type definition one can for instance declare arguments to foreign functions as having the type my int and they will be passed as integers More complex types CFFI offers another way to define types through define foreign type a thin wrapper macro around defclass As an example let s go through the steps needed to define a my string amp key encoding type First we need to define our type class define foreign type my string type encoding reader string type encoding initarg encoding actual type pointer The actual type class option tells CFFI that this type will ultimately be passed to and received from foreign code as a pointer Now you need to tell CFFI how to parse a type specification such as my string encoding utf8 into an instance of my string type We do that with define parse method Chapter 6 Foreign
60. lot value and its setf expansion that open code the memory access when type and slot names are constant at compile time Examples defcstruct point Pointer structure x int y int CFFI gt with foreign object ptr point Initialize the slots setf foreign slot value ptr point x 42 foreign slot value ptr point y 42 Return a list with the coordinates with foreign slots x y ptr point list x y gt 42 42 See Also defestruct page 33 foreign slot names page 44 foreign slot offset page 45 foreign slot pointer page 46 with foreign slots page 54 Chapter 6 Foreign Types 48 foreign type alignment Syntax foreign type alignment type alignment Function Arguments and Values type A foreign type alignment An integer Description The function foreign type alignment returns the alignment of type in bytes Examples CFFI gt foreign type alignment char 1 CFFI gt foreign type alignment short gt 2 CFFI gt foreign type alignment int gt 4 defcstruct foo a char CFFI gt foreign type alignment foo gt 1 See Also foreign type size page 49 Chapter 6 Foreign Types 49 foreign type size Syntax foreign type size type size Function Arguments and Values type A foreign type size An integer Description The function foreign type size return the size of type in bytes This includes any padding within and follow
61. m described above can add a bit of overhead This is usually not significant but we nevertheless provide a way of getting rid of the overhead for the cases where it matters A good way to understand this issue is to look at the code generated by defcfun Consider the following example using the previously defined my string type CFFI gt macroexpand 1 defcfun foo my string x my string simplified downcased etc defun foo x multiple value bind G2019 PARAM3149 translate to foreign x lt MY STRING TYPE 11ED5A79 gt unwind protect translate from foreign foreign funcall foo pointer G2019 pointer lt MY STRING TYPE 11ED5659 gt free translated object G2019 lt MY STRING TYPE 11ED51A79 gt 1t PARAM3149 In order to get rid of those generic function calls CFFI has another set of extensible generic functions that provide functionality similar to CL s compiler macros expand to foreign dyn expand to foreign and expand from foreign Here s how one could define a my boolean with them define foreign type my boolean type O actual type int simple parser my boolean defmethod expand to foreign value type my boolean type if value 1 0 defmethod expand from foreign value type my boolean type not zerop value And here s what the macroexpansion of a function using this type would look like CFFI gt macroexpand 1 defcfun bar my boolean x my boolean
62. mplementations that don t support foreign funcall cffi sys no foreign funcall will be present in features Note in these Lisps you can still use the defcfun interface Examples CFFI gt foreign funcall strlen string foo int gt 3 Given the C code void print_number int n printf N d n n 1 CFFI gt foreign funcall print number int 123456 4 N 123456 NIL Or equivalently Chapter 10 Functions CFFI gt foreign funcall print number int 123456 void 4 N 123456 NIL CFFI gt foreign funcall printf string format nil s d string So long and thanks for all the fish int 42 int So long and thanks for all the fish 42 gt 41 See Also defcfun page 86 foreign funcall pointer page 90 89 Chapter 10 Functions 90 foreign funcall pointer Syntax foreign funcall pointer pointer options amp rest arguments Macro return value arguments arg type arg return type options amp key convention Arguments and Values pointer A foreign pointer arg type A foreign type arg An argument of type arg type return type A foreign type void by default return value A lisp object convention One of cdecl default or stdcall Description The foreign funcall macro is the main primitive for calling foreign functions Note The return value of foreign funcall on functions with a void return type is still undefined Implementation spe
63. n to Foreign Interfaces and CFFI 5 The removal of the C compiler or C interpretation of any kind creates the main disad vantage some of C s abstractions are not available violating information encapsulation For example structs that must be passed on the stack or used as return values without corresponding functional abstractions to create and manage the structs must be declared explicitly in Lisp This is fine for structs whose contents are public but is not so pleasant when a struct is supposed to be opaque by convention even though it is not so defined Without an abstraction to create the struct Lisp needs to be able to lay out the struct in memory so must know its internal details In these cases you can create a minimal C library to provide the missing abstractions without destroying all the advantages of the Common Lisp approach discussed above In the case of structs you can write simple pure C functions that tell you how many bytes a struct requires or allocate new structs read and write fields of the struct or whatever operations are supposed to be public Implementor s note cffi grovel a project not yet part of CFFI automates this and other processes Another disadvantage appears when you would rather use the foreign language than Lisp However someone who prefers C to Lisp is not a likely candidate for developing a Lisp interface to a C library 4 2 Getting a URL The widely available l
64. nclude amp rest files Grovel Form Include the specified files specified as strings in the generated C source code in package symbol Grovel Form Set the package to be used for the final Lisp output ctype lisp name size designator Grovel Form Define a CFFI foreign type for the string in size designator e g ctype pid pid_ t constant lisp name amp rest c names amp key type documentation Grovel Form optional Search for the constant named by the first c name string found to be known to the C preprocessor and define it as lisp name Chapter 13 The Groveller 107 The type keyword argument specifies how to grovel the constant either integer the default or double float If optional is true no error will be raised if all the c names are unknown If lisp name is a keyword the actual constant will be a symbol of the same name interned in the current package define name amp optional value Grovel Form Defines an additional C preprocessor symbol which is useful for altering the behavior of included system headers cc flags amp rest flags Grovel Form Adds cc flags to the command line arguments used for the C compiler invocation cstruct lisp name c name slots Grovel Form Define a CFFI foreign struct with the slot data specfied Slots are of the form lisp name c name amp key type count signed t cunion lisp name c name slots Grovel Form Identical to cstruct but defines a CFFI foreign union cstruc
65. nerate an appropriate replacement for it This analogous to the amp whole form mechanism compiler macros provide The expand methods have precedence over their translate counterparts and are guaranteed to be used in defcfun foreign funcall defcvar and defcallback If you define a method on each of the expand generic functions you are guaranteed to have full control over the expressions generated for type translation in these macros They may or may not be used in other CFFI operators that need to translate between Lisp and C data you may only assume that expand methods will probably only be called during Lisp compilation expand to foreign dyn has precedence over expand to foreign and is only used in defcfun and foreign funcall only making sense in those contexts Important note this set of generic functions is called at macroexpansion time Methods are defined when loaded or evaluated not compiled You are responsible for ensuring that your expand methods are defined when the foreign funcall or other forms that use them are compiled One way to do this is to put the method definitions earlier in the file and inside an appropriate eval when form another way is to always load a separate Lisp or FASL file containing your expand definitions before compiling files with forms that ought to use them Otherwise they will not be found and the runtime translators will be used instead 6 6 Foreign Structure Types For more involved C
66. ng unsigned long base type CFFI gt canonicalize foreign type boolean INT CFFI gt canonicalize foreign type boolean long LONG CFFI gt canonicalize foreign type boolean float Terror signalled by ECASE See Also define foreign type page 38 Chapter 6 Foreign Types 40 foreign bitfield symbols Syntax foreign bitfield symbols type value gt symbols Function Arguments and Values type A bitfield type value An integer symbols A potentially shared list of symbols nil Description The function foreign bitfield symbols returns a possibly shared list of symbols that correspond to value in type Examples defbitfield flags flag a 1 flag b 2 flag c 4 CFFI gt foreign bitfield symbols boolean b101 FLAG A FLAG C See Also defbitfield page 31 foreign bitfield value page 41 Chapter 6 Foreign Types 41 foreign bitfield value Syntax foreign bitfield value type symbols value Function Arguments and Values type A bitfield type symbol A Lisp symbol value An integer Description The function foreign bitfield value returns the value that corresponds to the symbols in the symbols list Examples defbitfield flags flag a 1 flag b 2 flag c 4 CFFI gt foreign bitfield value flags flag a flag c b b101 See Also defbitfield page 31 foreign bitfield symbols page 40 Chapter 6 Foreign Types
67. ng type name on a symbol naming a foreign type defined with defctype This method should return the appropriate Lisp value to use in place of the foreign value The results are undefined if the type name parameter is specialized in any way except an EQL specializer on a foreign type defined with defctype Specifically translations may not be defined for built in types See Also Section 6 4 Foreign Type Translators page 25 translate to foreign page 53 free translated object page 51 Chapter 6 Foreign Types 53 translate to foreign Syntax translate to foreign lisp value type name Generic Function foreign value alloc param Arguments and Values lisp value The Lisp value to convert to foreign representation type name A symbol naming a foreign type defined by defctype foreign value The foreign value to pass in place of lisp value to foreign code alloc param If present this value will be passed to free translated object Description This generic function is invoked by CFFI to convert a Lisp value to a foreign value such as when passing arguments to a foreign function returning a value from a callback or setting a foreign variable A foreign value is one appropriate for passing to the next lowest translator including the low level translators that are ultimately invoked invisibly with CFFI To extend the CFFI type system by performing custom translations this method may be specialized by EQL spe
68. ntic similarity to Lisp such as Perl and Python are accustomed to having access to popular C libraries such as GTK by way of bindings In Lisp we do something similar but take a fundamentally different approach This tutorial first explains this difference then explains how you can use CFFI pow erful system for calling out to C and C and access C data from many Common Lisp implementations The concept can be generalized to other languages at the time of writing only CFFI s C support is fairly complete but C support is being worked on Therefore we will interchangeably refer to foreign functions and foreign data and C functions and C data At no time will the word foreign carry its usual non programming meaning This tutorial expects you to have a working understanding of both Common Lisp and C including the Common Lisp macro system 4 1 What makes Lisp different The following sums up how bindings to foreign libraries are usually implemented in other languages then in Common Lisp Perl Python Java other one implementation languages Bindings are implemented as shared objects written in C In some cases the C code is generated by a tool such as SWIG but the result is the same a new C library that manually translates between the language implementation s objects such as PyObject in Python and whatever C object is called for often using C functions provided by the implementation It also translates betw
69. ntion One of cdecl default or stdcall Description Creates a new library designator called name The load clauses describe how to load that designator when passed to load foreign library or use foreign library When trying to load the library name the relevant function searches the load clauses in order for the first one where feature evaluates to true That happens for any of the following situations 1 If feature is a symbol present in common lisp features 2 If feature is a list depending on first feature a keyword and All of the feature expressions in rest feature are true or At least one of the feature expressions in rest feature is true not The feature expression second feature is not true 3 Finally if feature is t this load clause is picked unconditionally Upon finding the first true feature the library loader then loads the library The meaning of library designator is described in load foreign library page 97 Functions associated to a library defined by define foreign library e g through defcfun s library option will inherit the library s options The precedence is as follows 1 defcfun foreign funcall specific options 2 load clause options 3 global library options the name and options argument Examples See Section 4 3 Loading foreign libraries page 5 See Also close foreign library page 92 load foreign library page 97 Chapter 11 Libraries 95 foreign
70. numeration of the kinds of options we want to set The enum CURLoption isn t the most straightforward but reading the CINIT C macro definition should be enlightening defmacro define curl options name type offsets amp rest enum args As with CFFI DEFCENUM except each of ENUM ARGS is as follows NAME TYPE NUMBER Where the arguments are as they are with the CINIT macro defined in curl h except NAME is a keyword TYPE OFFSETS is a plist of TYPEs to their integer offsets as Chapter 4 An Introduction to Foreign Interfaces and CFFI 8 defined by the CURLOPTTYPE LONG et al constants in curl h flet enumerated value type offset getf type offsets type offset progn defcenum name loop for name type number in enum args collect list name enumerated value type number gt name for REPL users sanity define curl options curl option long 0 objectpoint 10000 functionpoint 20000 off t 30000 noprogress long 43 nosignal long 99 errorbuffer objectpoint 10 url objectpoint 2 a or FN With some well placed Emacs query replace regexps you could probably similarly define the entire CURLoption enumeration I have selected to transcribe a few that we will use in this tutorial If youre having trouble following the macrology just macroexpand the curl option definition or see the following macroexpansion conveniently downcased and reformatted progn defcenum curl option nopr
71. o the same memory address and false otherwise Implementation specific Notes The representation of foreign pointers varies across the various Lisp implementations as does the behaviour of the built in Common Lisp equality predicates Comparing two pointers that point to the same address with EQ Lisps will return true on some Lisps others require more general predicates like EQL or EQUALP and finally some will return false using any of these predicates Therefore for portability you should use POINTER EQ Examples This is an example using SBCL see the implementation specific notes above CFFI gt eql null pointer null pointer NIL CFFI gt pointer eq null pointer null pointer gt T See Also inc pointer page 61 Chapter 7 Pointers 71 with foreign object with foreign objects Syntax with foreign object var type amp optional count amp body body Macro with foreign objects bindings amp body body Macro bindings var type amp optional count Arguments and Values var A symbol type A foreign type evaluated count An integer Description The macros with foreign object and with foreign objects bind var to a pointer to count newly allocated objects of type type during body The buffer has dynamic extent and may be stack allocated if supported by the host Lisp Examples CFFI gt with foreign object array int 10 dotimes i 10 setf mem aref array int i random 100 loo
72. ogress 43 nosignal 99 errorbuffer 10010 url 10002 curl option That seems more than reasonable You may notice that we only use the type to compute the real enumeration offset we will also need the type information later First however let s make sure a simple call to the foreign function works CFFI USER gt foreign funcall curl_easy_setopt pointer easy handle curl option nosignal long 1 curl code gt 0 foreign funcall despite its surface simplicity can be used to call any C function Its first argument is a string naming the function to be called Next for each argument we pass the name of the C type which is the same as in defcfun followed by a Lisp object representing the data to be passed as the argument The final argument is the return type for which we use the curl code type defined earlier defcfun just puts a convenient facade on foreign funcall Our earlier call to curl global init could have been written as follows CFFI USER gt foreign funcall curl_global_init long 0 This isn t entirely true some Lisps don t support foreign funcall so defcfun is implemented without it defcfun may also perform optimizations that foreign funcall cannot Chapter 4 An Introduction to Foreign Interfaces and CFFI 9 curl code gt 0 Before we continue we will take a look at what CFFI can and can t do and why this is so 4 6 Breaking the abstraction In Section 4 1 What makes Lisp
73. on is a structure in which all slots have an offset of zero It is isomorphic to the C union Therefore you should use the usual foreign structure operations for accessing a union s slots Examples defcunion uint32 bytes int value unsigned int bytes unsigned char count 4 See Also foreign slot pointer page 46 foreign slot value page 47 Chapter 6 Foreign Types defctype Syntax defctype name base type amp optional documentation Arguments and Values name The name of the new foreign type base type A symbol or a list defining the new type documentation A documentation string currently ignored Description 36 Macro The defctype macro provides a mechanism similar to C s typedef to define new types The new type inherits base type s translators if any There is no way to define translations for types for types defined with defctype For that you should use define foreign typel page 38 Examples defctype my string string My own string type defctype long bools boolean long Booleans that map to C longs See Also define foreign type page 38 Chapter 6 Foreign Types 3T defcenum Syntax defcenum name and options amp body enum list Macro enum list docstring keyword keyword value name and options name name amp optional base type int Arguments and Values name The name of the new enum type docstring A documentation string ignored
74. ons as a kind of documen tation much what you d use typedef for in C We also tried one special kind of type definition the defcenum type See defcstruct page 33 for a definition macro that may come in handy if you need to use C structs as data However all of these are mostly sugar for the powerful underlying foreign type interface called type translators You can easily define new translators for any simple named foreign type Since we ve defined the new type curl code to use as the return type for various libcurl functions we can use that to directly convert cURL errors to Lisp errors defctype s purpose is to define simple typedef like aliases In order to use type trans lators we must use the define foreign type macro So let s redefine curl code using it define foreign type curl code type O actual type int simple parser curl code define foreign type is a thin wrapper around defclass For now all you need to know in the context of this example is that it does what defctype curl code int would do and additionally defines a new class curl code type which we will take advantage of shortly The CURLcode enumeration seems to follow the typical error code convention of 0 meaning all is well and each non zero integer indicating a different kind of error We can apply that trivially to differentiate between normal exits and error exits define condition curl code error error code initarg curl code reader c
75. ons return type amp body docstring Macro arguments amp rest lisp name name and options name name amp key library convention name lisp name foreign name foreign name lisp name arguments arg name arg type Arguments and Values foreign name A string denoting a foreign function lisp name A symbol naming the Lisp function to be created arg name A symbol return type arg type A foreign type convention One of cdecl default or stdcall library A symbol designating a foreign library docstring A documentation string Description The defcfun macro provides a declarative interface for defining Lisp functions that call foreign functions When one of lisp name or foreign name is omitted the other is automatically derived using the following rules e Foreign names are converted to Lisp names by uppercasing and replacing underscores with hyphens e Lisp names are converted to foreign names by lowercasing and replacing hyphens with underscores If you place the symbol amp rest in the end of the argument list after the fixed arguments defcfun will treat the foreign function as a variadic function The variadic arguments should be passed in a way similar to what foreign funcall would expect Unlike foreign funcall though defcfun will take care of doing argument promotion Note that in this case defcfun will generate a Lisp macro instead of a function and will only work for Lisps that
76. ore time you may only manipulate its raw data through a pointer The C compiler does this also albeit informally 7 1 Basic Pointer Operations Manipulating pointers proper can be accomplished through most of the other operations defined in the Pointers dictionary such as make pointer pointer address and pointer eq When using them keep in mind that they merely manipulate the Lisp representation of pointers not the values they point to foreign pointer Lisp Type The pointers representations differ from implementation to implementation and have different types foreign pointer provides a portable type alias to each of these types 7 2 Allocating Foreign Memory CFFI provides support for stack and heap C memory allocation Stack allocation done with with foreign object is sometimes called dynamic allocation in Lisp because memory allocated as such has dynamic extent much as with let bindings of special variables This should not be confused with what C calls dynamic allocation or that done with malloc and friends This sort of heap allocation is done with foreign alloc creating objects that exist until freed with foreign free 7 3 Accessing Foreign Memory When manipulating raw C data consider that all pointers are pointing to an array When you only want one C value such as a single struct this array only has one such value It is worthwhile to remember that everything is an array though because this is also
77. osure with more useful lexical bindings than the top level defcallback form Finally we make a halfhearted effort to prevent non local exits from unwinding the C stack covering the most likely case with an error handler which is usually triggered unexpectedly The reason is that most C code is written to understand its own idiosyncratic error condition implemented above in the case of curl easy perform and more undefined behavior can result if we just wipe C stack frames without allowing them to execute whatever cleanup actions as they like Using the CURLoption enumeration in curl h once more we can describe the new option by modifying and reevaluating define curl options define curl options curl option long 0 objectpoint 10000 functionpoint 20000 off t 30000 noprogress long 43 nosignal long 99 errorbuffer objectpoint 10 url objectpoint 2 writefunction functionpoint 11 new item here QS ae En m Finally we can use the defined callback and the new set curl option writefunction to finish configuring the easy handle using the callback macro to retrieve a CFFI pointer which works like a function pointer in C code CFFI USER gt set curl option writefunction easy handle callback easy write gt 0 4 10 A complete FFI With all options finally set and a medium level interface developed we can finish the defi nition and retrieve http www cliki net CFFI as is done in the tutorial def
78. p for i below 10 collect mem aref array int i gt 22 7 22 52 69 1 46 93 90 65 See Also foreign alloc page 58 Chapter 7 Pointers 72 with foreign pointer Syntax with foreign pointer var size amp optional size var amp body Macro body Arguments and Values var size var A symbol size An integer body A list of forms to be executed Description The with foreign pointer macro binds var to size bytes of foreign memory during body The pointer in var is invalid beyond the dynamic extend of body and may be stack allocated if supported by the implementation If size var is supplied it will be bound to size during body Examples CFFI gt with foreign pointer string 4 size setf mem ref string char 1 size 0 lisp string to foreign Popcorns string size loop for i from 0 below size collect code char mem ref string char i gt P o p Null See Also foreign alloc page 58 foreign free page 57 Chapter 8 Strings T3 8 Strings As with many languages Lisp and C have special support for logical arrays of characters going so far as to give them a special name strings In that spirit CFFI provides special support for translating between Lisp and C strings The string type and the symbols related below also serve as an example of what you can do portably with CFFI were it not included you could write an equally functional strings lisp without referring to any im
79. page 68 Chapter 7 Pointers 68 pointerp Syntax pointerp ptr boolean Function Arguments and Values ptr An object that may be a foreign pointer boolean T or NIL Description The function pointerp returns true if ptr is a foreign pointer and false otherwise Implementation specific Notes In Allegro CL foreign pointers are integers thus in this implementation pointerp will return true for any ordinary integer Examples CFFI gt foreign alloc 32 gt lt A Mac Pointer x102D20 gt CFFI gt pointerp gt T CFFI gt pointerp this is not a pointer NIL See Also make pointer page 63 null pointer p page 67 Chapter 7 Pointers 69 pointer address Syntax pointer address ptr address Function Arguments and Values ptr A foreign pointer address An integer Description The function pointer address will return the address of a foreign pointer ptr Examples CFFI gt pointer address null pointer gt 0 CFFI gt pointer address make pointer 123 gt 123 See Also make pointer page 63 inc pointer page 61 null pointer page 66 null pointer p page 67 pointerp page 68 pointer eq page 70 mem ref page 65 Chapter 7 Pointers 70 pointer eq Syntax pointer eq ptri ptr2 gt boolean Function Arguments and Values ptrl ptr2 A foreign pointer boolean T or NIL Description The function pointer eq returns true if ptr and ptr2 point t
80. passed through low level code in the Lisp implementation such as the foreign function caller or indirect memory addressing combined with a data move In other contexts this refers to a value accessible by C but which may only be accessed through CFFI functions T he closest you can get to such a foreign value is through a pointer Lisp object which itself counts as a foreign value in only the previous sense simple type A CFFI type that is ultimately represented as a builtin type CFFI only provides extra semantics for Lisp that are invisible to C code or data Index Index boolean amp optional base type int 24 Heu Pt 22 LRL T ad 23 gr i niga eier keinen 23 Di TD 22 INEIG p ee en nenn 22 ANC och nennen aan nenn 22 int64 a nalen tere 23 bad a ana ae aan einen 22 ILong uses iit e EEEn OEE EIEEE 22 long nase KOREEN du e Y PS 22 long dOuble i ne erre pannei 23 Qlong long xx anche de ue ende RESAS 22 pointer amp optional type sssssss 23 short nenn ee 22 DSUPIDBE rein a aed wu te dee 23 iStringtptr nannten int 23 r char cic rr e rAr IEn E ba ed dead aeons 22 VDE ner een ran 22 UInE 16 see ee his 22 UINE32 20 sans bike 22 int64 este ee neu 23 pam mms 22 Ullong belt eve DRE IMG attends 22 PAULO G nenn waste dace anat 22 unsigned char iiesee de duende EERE NSSS 22 unsigned int i d ls sace saure 22 unsigned long 25 cad e t ee rape e 22 unsi
81. plementation specific symbols Chapter 8 Strings 74 default foreign encoding Syntax default foreign encoding Special Variable Value type A keyword Initial value utf 8 Description This special variable holds the default foreign encoding Examples CFFI gt default foreign encoding utf 8 CFFI gt foreign funcall strdup string encoding utf 16 foo string f CFFI gt let default foreign encoding utf 16 foreign funcall strdup string encoding utf 16 foo string foo See also Section 6 2 Other Types page 23 string type foreign string alloc page 75 foreign string to lisp page 77 lisp string to foreign page 78 with foreign string page 79 with foreign pointer as string page 80 Chapter 8 Strings 75 foreign string alloc Syntax foreign string alloc string amp key encoding Function null terminated p start end pointer Arguments and Values string A Lisp string encoding Foreign encoding Defaults to default foreign encoding null terminated p Boolean defaults to true start end Bounding index designators of string 0 and nil by default pointer A pointer to the newly allocated foreign string Description The foreign string alloc function allocates foreign memory holding a copy of string converted using the specified encoding Start specifies an offset into string and end marks the position following the last element of the foreign s
82. point Chapter 6 Foreign Types 34 Initialize the slots setf foreign slot value ptr point x 42 foreign slot value ptr point y 42 Return a list with the coordinates with foreign slots x y ptr point list x y gt 42 42 Using the size and offset options to define a partial structure this is useful when you are interested in only a few slots of a big foreign structure defcstruct foo size 32 Some struct with 32 bytes lt 16 bytes we don t care about x int offset 16 an int at offset 16 y int another int at offset 16 sizeof int lt a couple more bytes we don t care about z char offset 24 a char at offset 24 lt 7 more bytes ignored since size is 32 CFFI gt foreign type size foo gt 32 Using count to define arrays inside of a struct defcstruct video_tuner name char count 32 See Also foreign slot pointer page 46 foreign slot value page 47 with foreign slots page 54 Chapter 6 Foreign Types 35 defcunion Syntax defcunion name amp body doc and slots name Macro doc and slots docstring slot name slot type amp key count Arguments and Values name The name of new union type docstring A documentation string ignored slot name A symbol naming the slot slot type The type specifier for the slot count Used to declare an array of size count inside the structure Description A uni
83. pointer The first thing to note is that nowhere in the C interface includes this information it can only be found in the manual We could disjoin these clearly different types ourselves by splitting objectpoint into filepoint and charpoint but we are still breaking the abstraction because we have to augment the entire enumeration form with this additional information The second is that the CURLOPT_WRITEDATA argument is completely incompatible with the desired Lisp data a stream It is probably acceptable if we are controlling every file we might want to use as this argument in which case we can just call the foreign function fopen Regardless though we can t write to arbitrary streams which is exactly what we want to do for this application Finally note that the curl_easy_setopt interface itself is a hack intended to work around some of the drawbacks of C The definition of Curl_setopt while long is far less 5 Another possibility is to allow the caller to specify the desired C type of the third argument This is essentially what happens in a call to the function written in C 6 See Section Other Kinds of Streams in GNU C Library Reference for a GNU only way to extend the FILE type You could use this to convert Lisp streams to the needed C data This would be quite involved and far outside the scope of this tutorial Chapter 4 An Introduction to Foreign Interfaces and CFFI 10 cluttered than the equivalent disjo
84. pter 6 Foreign Types 51 free translated object Syntax free translated object value type name param Generic Function Arguments and Values pointer The foreign value returned by translate to foreign type name A symbol naming a foreign type defined by defctype param The second value if any returned by translate to foreign Description This generic function may be specialized by user code to perform automatic deallocation of foreign objects as they are passed to C functions Any methods defined on this generic function must EQL specialize the type name pa rameter on a symbol defined as a foreign type by the defctype macro See Also Section 6 4 Foreign Type Translators page 25 translate to foreign page 53 Chapter 6 Foreign Types translate from foreign Syntax translate from foreign foreign value type name lisp value Arguments and Values foreign value The foreign value to convert to a Lisp object type name A symbol naming a foreign type defined by defctype 52 Generic Function lisp value The lisp value to pass in place of foreign value to Lisp code Description This generic function is invoked by CFFI to convert a foreign value to a Lisp value such as when returning from a foreign function passing arguments to a callback function or accessing a foreign variable To extend the CFFI type system by performing custom translations this method may be specialized by EQL specializi
85. r name here as the name argument to defcfun The tutorial goes on to have us allocate a handle For good measure we should also include the deallocator Let s look at these functions CURL curl easy init void curl easy cleanup CURL handle Advanced users may want to define special pointer types we will explore this possibility later For now just treat every pointer as the same defcfun curl easy init pointer defcfun curl easy cleanup void easy handle pointer Now we can continue with the tutorial CFFI USER gt defparameter easy handle curl easy init gt EASY HANDLE CFFI USER easy handle gt lt FOREIGN ADDRESS x09844EE0 gt Note the print representation of a pointer It changes depending on what Lisp you are using but that doesn t make any difference to CFFI 4 5 Setting download options The libcurl tutorial says we ll want to set many options before performing any download actions This is done through curl_easy_setopt CURLcode curl easy setopt CURL curl CURLoption option We ve introduced a new twist variable arguments There is no obvious translation to the defcfun form particularly as there are four possible argument types Because of the way C works we could define four wrappers around curl easy setopt one for each type in this case however we ll use the general purpose macro foreign funcall to call this function To make things easier on ourselves we ll create an e
86. rather than the powerful yet domain specific type translator interface 4 12 What s next CFFI provides a rich and powerful foundation for communicating with foreign libraries as we have seen it is up to you to make that experience a pleasantly Lispy one This tutorial does not cover all the features of CFFI please see the rest of the manual for details In particular if something seems obviously missing it is likely that either code or a good reason for lack of code is already present Implementor s note There are some other things in CFFI that might deserve tutorial sections such as free translated object or structs Let us know which ones you care about Chapter 5 Wrapper generators 21 5 Wrapper generators CFFI s interface is designed for human programmers being aimed at aesthetic as well as technical sophistication However there are a few programs aimed at translating C and C header files or approximations thereof into CFFI forms constituting a foreign interface to the symbols in those files These wrapper generators are known to support output of CFFI forms Verrazano Designed specifically for Common Lisp Uses GCC s parser output in XML format to discover functions variables and other header file data This means you need GCC to generate forms on the other hand the parser employed is mostly compliant with ANSI C SWIG A foreign interface generator originally designed to generate Python bindings it ha
87. reams and ncnpetrtebey e pere pete aaa sages 9 SULIN GS Lupe Mrs RUNE EE ER EE 14 BWIG X m 4 T translate from foreign foreign value type name lisp value 52 translate to foreign lisp value type name foreign value alloc param 53 translating types u a 18 tutorial OFEI 2 2 ae la 4 type definition is sent 18 type translators optimizing 26 Index U use foreign library name 100 V VAEBtES eur rede RP Rp bed red she vir he 4 W with foreign object var type amp optional Count amp body body iioc e e es 71 with foreign objects bindings amp body body 115 with foreign pointer var size koptional size var amp body body 12 with foreign pointer as string var size amp optional size var amp rest args amp body body string lle prb d 80 with foreign slots vars ptr type amp body body omm 54 with foreign string var or vars string amp rest args amp body body 00005 19 with foreign strings bindings amp body body A cS 79 workaround for Or een ne 5
88. reign Type Foreign Type Foreign Type Foreign Type Foreign Type Chapter 6 Foreign Types 23 int64 Foreign Type uint64 Foreign Type Foreign integer types of specific sizes corresponding to the C types defined in stdint h float Foreign Type double Foreign Type On all systems the float and double types represent a C float and double respec tively On most but not all systems float and double represent a Lisp single float and double float respectively It is not so useful to consider the relationship between Lisp types and C types as isomorphic as simply to recognize the relationship and relative precision among each respective category long double Foreign Type This type is only supported on SCL pointer amp optional type Foreign Type A foreign pointer to an object of any type corresponding to void You can optionally specify type of pointer e g pointer char Although CFFI won t do anything with that information yet it is useful for documentation purposes void Foreign Type No type at all Only valid as the return type of a function 6 2 Other Types CFFI also provides a few useful types that aren t built in C types string Foreign Type The string type performs automatic conversion between Lisp and C strings Note that in the case of functions the converted C string will have dynamic extent i e it will be automatically freed after the foreign function returns In add
89. ription Returns a pointer to the location of the slot slot name in a foreign object of type type at ptr The returned pointer points inside the structure Both the pointer and the memory it points to have the same extent as ptr For aggregate slots this is the same value returned by foreign slot value Examples defcstruct point Pointer structure x int y int CFFI gt with foreign object ptr point foreign slot pointer ptr point x lt FOREIGN ADDRESS xBFFF6E60 gt Note the exact pointer representation varies from lisp to lisp See Also defcstruct page 33 foreign slot value page 47 foreign slot names page 44 foreign slot offset page 45 Chapter 6 Foreign Types 47 foreign slot value Syntax foreign slot value ptr type slot name object Accessor Arguments and Values ptr A pointer to a structure type A foreign structure type slot name A symbol naming a slot in the structure type object The object contained in the slot specified by slot name Description For simple slots foreign slot value returns the value of the object such as a Lisp integer or pointer In C this would be expressed as ptr gt slot For aggregate slots a pointer inside the structure to the beginning of the slots data is returned In C this would be expressed as amp ptr gt slot This pointer and the memory it points to have the same extent as ptr There are compiler macros for foreign s
90. rk just as well on the computer on which I tested the tutorial There are a couple of good reasons to provide the so s current version number however Namely the versionless so is not packaged on most UNIX systems along with the actual fully versioned library instead it is included in the development package along with C headers and static a libraries The reason CFFI does not try to account for this lies in the meaning of the version num bers A full treatment of shared library versions is beyond this manual s scope see Section Library interface versions in GNU Libtool for helpful information for the unfamiliar For our purposes consider that a mismatch between the library version with which you tested and the installed library version may cause undefined behavior Implementor s note Maybe some notes should go here about OS X which I know little about stephen 1 Windows programmers may chafe at adding a UNIX specific clause to define foreign library Instead ask why the Windows solution to library incompatibility is include your own version of every library you use with every program Chapter 11 Libraries 92 close foreign library Syntax close foreign library library success Function Arguments and Values library A symbol or an instance of foreign library success A Lisp boolean Description Closes library which can be a symbol designating a library define through define
91. rtant for libraries which are implemented in different ways by different vendors such as the UNIX POSIX functions The CFFI declarations are usually quite different from platform to platform but the information you give to CFFI Grovel is the same Hence much less work is required If you use ASDF CFFI Grovel is integrated so that it will run automatically when your system is building This feature was inspired by SB Grovel a similar SBCL specific project CFFI Grovel can also be used without ASDF 13 1 Building FFIs with CFFI Grovel CFFI Grovel uses a specification file lisp describing the features that need groveling The C compiler is used to retrieve this data and write a Lisp file cffi lisp which contains the necessary CFFI definitions to access the variables structures constants and enums mentioned in the specification CFFI Grovel provides an ASDF component for handling the necessary calls to the C compiler and resulting file management 13 2 Specification File Syntax The specification files are read by the normal Lisp reader so they have syntax very similar to normal Lisp code In particular semicolon comments and reader macros will work as expected There are several forms recognized by CFFI Grovel progn amp rest forms Grovel Form Processes a list of forms Useful for conditionalizing several forms For example freebsd progn constant ev enable EV_ENABLE constant ev disable EV_DISABLE i
92. rts various free and commercial Lisp implementations Allegro CL Corman CL CLISP CMUCL ECL LispWorks Clozure CL SBCL and the Scieneer CL In general you should work with the latest versions of each implementation since those will usually be tested against recent versions of CFFI more often and might include necessary features or bug fixes Reasonable patches for compatibility with earlier versions are welcome nevertheless 3 1 Limitations Some features are not supported in all implementations Allegro CL e Does not support the long long type natively e Unicode support is limited to the Basic Multilingual Plane 16 bit code points CMUCL e No Unicode support 8 bit code points Corman CL e Does not support foreign funcall ECL e On platforms where ECL s dynamic FFI is not supported ie when dffi is not present in features cffi load foreign library does not work and you must use ECL s own ffi load foreign library with a constant string argument e Does not support the long long type natively e Unicode support is not enabled by default Lispworks e Does not completely support the long long type natively in 32 bit platforms e Unicode support is limited to the Basic Multilingual Plane 16 bit code points SBCL e Not all platforms support callbacks Chapter 4 An Introduction to Foreign Interfaces and CFFI 4 4 An Introduction to Foreign Interfaces and CFFI Users of many popular languages bearing sema
93. s been ported to many other systems including CFFI in version 1 3 28 Includes its own C declaration munger not intended to be fully compliant with ANSI C First this manual does not describe use of these other programs they have documen tation of their own If you have problems using a generated interface please look at the output CFFI forms and verify that they are a correct CFFI interface to the library in ques tion if they are correct contact CFFI developers with details keeping in mind that they communicate in terms of those forms rather than any particular wrapper generator Oth erwise contact the maintainers of the wrapper generator you are using provided you can reasonably expect more accuracy from the generator When is more accuracy an unreasonable expectation As described in the tutorial see Section 4 6 Breaking the abstraction page 9 the information in C declarations is in sufficient to completely describe every interface In fact it is quite common to run into an interface that cannot be handled automatically and generators should be excused from generating a complete interface in these cases As further described in the tutorial the thinnest Lisp interface to a C function is not always the most pleasant one In many cases you will want to manually write a Lispier interface to the C functions that interest you Wrapper generators should be treated as time savers not complete automation of the full foreign inter
94. s class name It is a thin wrapper around defclass Among other things it ensures that class name becomes a subclass of foreign type what you need to know about that is that there s an initarg actual type which serves the same purpose as defctype s base type argument Examples Taken from CFFT s boolean type definition define foreign type boolean amp optional base type int Boolean type Maps to an int by default Only accepts integer types ecase base type char unsigned char int unsigned int long unsigned long base type CFFI gt canonicalize foreign type boolean INT CFFI gt canonicalize foreign type boolean long LONG CFFI gt canonicalize foreign type boolean float gt signalled by ECASE See Also defctype page 36 define parse method page 39 Chapter 6 Foreign Types 39 define parse method Syntax define parse method name lambda list amp body body name Macro Arguments and Values type name A symbol naming the new foreign type lambda list A lambda list which is the argument list of the new foreign type body One or more forms that provide a definition of the new foreign type Description Examples Taken from CFFT s boolean type definition define foreign type boolean amp optional base type int Boolean type Maps to an int by default Only accepts integer types ecase base type char unsigned char int unsigned int lo
95. sier and clearer to use defcfun define curl options was becoming unwieldy so I defined some helpers in this new definition defun curry curl option setter function name option keyword Wrap the function named by FUNCTION NAME with a version that curries the second argument as OPTION KEYWORD This function is intended for use in DEFINE CURL OPTION SETTER setf symbol function function name let c function symbol function function name lambda easy handle new value funcall c function easy handle option keyword new value defmacro define curl option setter name option type option value foreign type Define with DEFCFUN a function NAME that calls curl easy setopt OPTION TYPE and OPTION VALUE are the CFFI foreign type and value to be passed as the second argument to easy setopt and FOREIGN TYPE is the CFFI foreign type to be used for the resultant function s third argument This macro is intended for use in DEFINE CURL OPTIONS progn defcfun curl easy setopt name curl code Chapter 4 An Introduction to Foreign Interfaces and CFFI 11 easy handle easy handle option option type new value foreign type curry curl option setter name option value defmacro define curl options type name type offsets amp rest enum args As with CFFI DEFCENUM except each of ENUM ARGS is as follows NAME TYPE NUMBER Where the arguments are as they are with the CINIT macro defined in curl h ex
96. similar to with foreign pointer ex cept that var is used as the returned value of an implicit progn around body after being converted to a Lisp string using the provided args Examples CFFI gt with foreign pointer as string str 6 str size encoding ascii lisp string to foreign Hello foreign world str str size Hello See Also foreign string alloc page 75 with foreign string page 79 Chapter 9 Variables 9 Variables 8l Chapter 9 Variables 82 defcvar Syntax defcvar name and options type amp optional documentation gt Macro lisp name name and options name name amp key read only library default name lisp name foreign name foreign name lisp name Arguments and Values foreign name A string denoting a foreign function lisp name A symbol naming the Lisp function to be created type A foreign type read only A boolean documentation A Lisp string not evaluated Description The defcvar macro defines a symbol macro lisp name that looks up foreign name and dereferences it acording to type It can also be setfed unless read only is true in which case an error will be signaled When one of lisp name or foreign name is omitted the other is automatically derived using the following rules e Foreign names are converted to Lisp names by uppercasing replacing underscores with hyphens and wrapping around asterisks e Lisp names are converted to foreign names by lowerca
97. sing replacing hyphens with underscores and removing asterisks if any Examples CFFI gt defcvar errno int ERRNO CFFI gt foreign funcall strerror int errno string Inappropriate ioctl for device CFFI gt setf errno 1 gt 1 CFFI gt foreign funcall strerror int errno string gt Operation not permitted Trying to modify a read only foreign variable CFFI gt defcvar errno error number read only t int ERROR NUMBER CFFI gt setf error number 12 pa Trying to modify read only foreign var ERROR NUMBER Note that accessing errno this way won t work with every implementation of the C stan dard library Chapter 9 Variables See Also get var pointer page 84 83 Chapter 9 Variables 84 get var pointer Syntax get var pointer symbol pointer Function Arguments and Values symbol A symbol denoting a foreign variable defined with defcvar pointer A foreign pointer Description The function get var pointer will return a pointer to the foreign global variable symbol previously defined with defcvar Examples CFFI gt defcvar errno int read only t gt ERRNO CFFI gt errno gt 25 CFFI gt get var pointer errno lt A Mac Pointer xA0008130 gt CFFI gt mem ref int gt 25 See Also defcvar page 82 Chapter 10 Functions 10 Functions 85 Chapter 10 Functions 86 defcfun Syntax defcfun name and opti
98. string gethash handle easy handle cstrings Here we have redefined the interface to create and free handles to associate a list of allocated C strings with each handle while it exists The strategy of using different function names to wrap around simple foreign functions is more common than the solution implemented earlier with curry curl option setter which was to modify the function name s function slot Incidentally the next step is to redefine curry curl option setter to allocate C strings for the appropriate length of time given a Lisp string as the new value argument defun curry curl option setter function name option keyword Wrap the function named by FUNCTION NAME with a version that curries the second argument as OPTION KEYWORD This function is intended for use in DEFINE CURL OPTION SETTER setf symbol function function name let c function symbol function function name lambda easy handle new value funcall c function easy handle option keyword if stringp new value add curl handle cstring easy handle foreign string alloc new value new value A quick analysis of the code shows that you need only reevaluate the curl option enu meration definition to take advantage of these new semantics Now for good measure let s reallocate the handle with the new functions we just defined and set its URL CFFI USER gt curl easy cleanup easy handle NIL CFFI USER gt setf easy handle make easy
99. support foreign funcall Examples defcfun strlen int Calculate the length of a string n string Chapter 10 Functions 8T CFFI gt strlen 123 EE defcfun abs c abs int n int CFFI gt c abs 42 gt 42 Function without arguments defcfun rand int CFFI gt rand 1804289383 Variadic function example defcfun sprintf int str pointer control string amp rest CFFI gt with foreign pointer as string s 100 sprintf s Ac d 4 2f 4s char 90 short 42 float pi string super locrian A 42 3 14 super locrian See Also foreign funcall page 88 foreign funcall pointer page 90 Chapter 10 Functions 88 foreign funcall Syntax foreign funcall name and options amp rest arguments Macro return value arguments arg type arg return type name and options name name amp key library convention Arguments and Values name A Lisp string arg type A foreign type arg An argument of type arg type return type A foreign type void by default return value A lisp object library A lisp symbol not evaluated convention One of cdecl default or stdcall Description The foreign funcall macro is the main primitive for calling foreign functions Note The return value of foreign funcall on functions with a void return type is still undefined Implementation specific Notes e Corman Lisp does not support foreign funcall On i
100. t and class c name slots Grovel Form Defines a CFFI foreign struct as with cstruct and defines a CLOS class to be used with it This is useful for mapping foreign structures to application layer code that shouldn t need to worry about memory allocation issues cvar namespec type amp key read only Grovel Form Defines a foreign variable of the specified type even if that variable is potentially a C preprocessor pseudo variable e g cvar errno errno errno values assum ing that errno values is an enum or equivalent to type int The namespec is similar to the one used in defcvar page 82 cenum name and opts amp rest elements Grovel Form Defines a true C enum with elements specified as lisp name amp rest c names amp key optional documentation name and opts can be either a symbol as name or a list name amp key base type define constants If define constants is non null a Lisp constant will be defined for each enum member constantenum name and opts amp rest elements Grovel Form Defines an enumeration of pre processor constants with elements specified as lisp name amp rest c names amp key optional documentation name and opts can be ei ther a symbol as name or a list name amp key base type define constants If define constants is non null a Lisp constant will be defined for each enum member This example defines af inet to represent the value held by AF_INET or PF_INET whichever the pre processor
101. ta eb be dis 40 foreign bitfield value type symbols value OUT een 41 foreign enum keyword type value amp key errorp keyword nee 42 foreign enum value type keyword amp key errorp zPoyalue lsekaeseme an RE RIS 43 foreign free ptr undefined 57 foreign funcall name and options amp rest arguments return value 88 foreign funcall pointer pointer options amp rest arguments return value 90 foreign pointer i pde eb bp dA ed 55 foreign slot names type names 44 foreign slot offset type slot name offset Huard ee ce na coa 45 foreign slot pointer ptr type slot name gt po3nter eiee ke e be er PE EDEN 46 foreign slot value ptr type slot name ODP CCE cessi at PRSE i epa sins 4T foreign string alloc string amp key encoding null terminated p start end pointer a ee er re ee eee dp rig 75 foreign string free pointer 76 foreign string to lisp ptr optional offset count max chars encoding string 77 foreign symbol pointer foreign name amp key Library pointer G ace 60 foreign type alignment type gt alignment 48 foreign type size type size 49 free converted object foreign value type params s ilie ia p pe ex ever S 50 free translated object value type name param S 51 function definition ies au 6 G get callback symbol pointer 105 get var pointer symbol pointer 84
102. ter defcallback lt int a pointer b pointer let x mem ref a int y mem ref b int cond x y 1 lt x y 1 t 0 CFFI gt with foreign object array int 10 Initialize array loop for i from 0 and n in 7 210435169 8 do setf mem aref array int i n Chapter 12 Callbacks 104 Sort it qsort array 10 foreign type size int callback Return it as a list loop for i from 0 below 10 collect mem aref array int i gt 123456789 10 See Also callback page 102 get callback page 105 Chapter 12 Callbacks 105 get callback Syntax get callback symbol gt pointer Accessor Arguments and Values symbol A symbol denoting a callback pointer A pointer Description This is the functional version of the callback macro It returns a pointer to the callback named by symbol suitable for example to pass as arguments to foreign functions Examples CFFI gt defcallback sum int a int b int a b gt SUM CFFI gt get callback sum gt lt A Mac Pointer x102350 gt See Also callback page 102 defcallback page 103 Chapter 13 The Groveller 106 13 The Groveller CFFI Grovel is a tool which makes it easier to write CFFI declarations for libraries that are implemented in C That is it grovels through the system headers getting information about types and structures so you don t have to This is especially impo
103. tring This string must be freed with foreign string free If null terminated p is false the string will not be null terminated Examples CFFI gt defparameter str foreign string alloc Hello foreign world gt lt FOREIGN ADDRESS x00400560 gt CFFI gt foreign funcall strlen pointer str int gt 21 See Also foreign string free page 76 with foreign string page 79 Chapter 8 Strings 76 foreign string free Syntax foreign string free pointer Function Arguments and Values pointer A pointer to a string allocated by foreign string alloc Description The foreign string free function frees a foreign string allocated by foreign string alloc Examples See Also foreign string alloc page 75 Chapter 8 Strings 77 foreign string to lisp Syntax foreign string to lisp ptr amp optional offset count Function max chars encoding gt string Arguments and Values ptr A pointer offset An integer greater than or equal to 0 Defauls to 0 count Either nil the default or an integer greater than or equal to 0 max chars An integer greater than or equal to 0 1 array total size limit by default encoding Foreign encoding Defaults to default foreign encoding string A Lisp string Description The foreign string to lisp function converts at most count octets from ptr into a Lisp string using the defined encoding If count is nil the default characters are copied
104. types than simple aliases to built in types such as you can make with defctype CFFI allows declaration of structures and unions with defcstruct and defcunion For example consider this fictional C structure declaration holding some personal infor mation Chapter 6 Foreign Types 28 struct person int number char reason F The equivalent defcstruct form follows defcstruct person number int reason string Please note that this interface is only for those that must know about the values contained in a relevant struct If the library you are interfacing returns an opaque pointer that needs only be passed to other C library functions by all means just use pointer or a type safe definition munged together with defctype and type translation defestruct page 33 for more details 6 7 Allocating Foreign Objects See Section 7 2 Allocating Foreign Memory page 55 Chapter 6 Foreign Types 29 convert from foreign Syntax convert from foreign foreign value type value Function Arguments and Values foreign value The primitive C value as returned from a primitive foreign function or from convert to foreign type A CFFI type specifier value The Lisp value translated from foreign value Description This is an external interface to the type translation facility In the implementation all for eign functions are ultimately defined as type translation wrappers around primitive foreign function invocations
105. unk of the Lisp C stack You may segfault You may get some random piece of other data from the heap Maybe in a world where dynamic extent is defined to be infinite extent everything will turn out fine Regardless results are likely to be almost universally unpleasant Returning to the current set curl option url interface here is what we must do let easy handle unwind protect with foreign string url http www cliki net CFFI setf easy handle curl easy init set curl option url easy handle url do more with the easy handle like actually get the URL when easy handle curl easy cleanup easy handle That is fine for the single string defined here but for every string option we want to pass we have to surround the body of with foreign string with another with foreign string wrapper or else do some extremely error prone pointer manipulation and size calculation in advance We could alleviate some of the pain with a recursively expanding macro but this would not remove the need to modify the block every time we want to add an option anathema as it is to a modular interface Before modifying the code to account for this case consider the other reason we can t simply use string as the foreign type In C a char is a char not necessarily a string The option CURLOPT_ERRORBUFFER accepts a char but does not expect anything about the data there However it does expect that some libcurl function we call
106. url error code report lambda c stream format stream libcurl function returned error A curl error code c documentation Signalled when a libcurl function answers a code other than CURLE OK defmethod translate from foreign value type curl code type Raise a CURL CODE ERROR if VALUE a curl code is non zero if zerop value curle ok error curl code error curl code value Chapter 4 An Introduction to Foreign Interfaces and CFFI 19 The heart of this translator is new method translate from foreign By specializing the type parameter on curl code type we immediately modify the behavior of every function that returns a curl code to pass the result through this new method To see the translator in action try invoking a function that returns a curl code You need to reevaluate the respective defcfun form so that it picks up the new curl code definition CFFI USER gt set curl option nosignal easy handle 1 CURLE OK As the result was 0 the new method returned curle ok just as specified I will leave disjoining the separate CURLcodes into condition types and improving the report function as an exercise for you The creation of easy handle cstrings and easy handle errorbuffers as prop erties of easy handles is a kluge What we really want is a Lisp structure that stores these properties along with the C pointer Unfortunately easy handle is currently just a fancy name for the foreign
107. y varying FFIs present in the various Lisp implementations it supports With CFFI you can define foreign function interfaces while still maintaining portability between implementa tions It is not the first Common Lisp package with this objective however it is meant to be a more malleable framework than similar packages Design Philosophy e Pointers do not carry around type information Instead type information is supplied when pointers are dereferenced e A type safe pointer interface can be developed on top of an untyped one It is difficult to do the opposite e Functions are better than macros When a macro could be used for performance use a compiler macro instead Chapter 2 Installation 2 2 Installation CFFI can be obtained through one of the following means available through its website e official release tarballs e git repository In addition you will need to obtain and install the following dependencies e Dabel a charset encoding decoding library e Alexandria a collection of portable public domain utilities e trivial features a portability layer that ensures consistent features across multiple Common Lisp implementations Furthermore if you wish to run the testsuite RT is required You may find mechanisms such as clbuild recommended or ASDF Install not as rec ommendable helpful in getting and managing CFFI and its dependencies Chapter 3 Implementation Support 3 3 Implementation Support CFFI suppo

CFFI User Manual

Contents

Download Pdf Manuals

Related Search

Related Contents