Foreign Function Calls in OpenMCL


Table of Contents

Overview
Argument/return value type designators
External entrypoints and named external entrypoints.
Functional Reference
ccl:%ff-call [Function]
ff-call [Macro]
external [Macro]
ccl:%reference-external-entry-point [Function]
ccl:external-call [Macro]
ccl:foreign-symbol-entry [Function]
ccl:foreign-symbol-address [Function]
defcallback [Macro]
#_ [Reader macro]
#? [Reader macro]
C structure return conventions

Overview

OpenMCL provides a number of constructs for calling foreign functions from Lisp code (all of them based on the function CCL:%FF-CALL). In many cases, OpenMCL's interface translator provides information about the foreign function's entrypoint name and argument and return types; this enables the use of the #_ reader macro (described below), which may be more concise and/or more readable than other constructs.

OpenMCL also provides a mechanism for defining callbacks: lisp functions which can be called from foreign code.

There's no supported way to directly pass lisp data to foreign functions: scalar lisp data must be coerced to an equivalent foreign representatation, and lisp arrays (notably strings) must be copied to non-GCed memory.

Argument/return value type designators

The types of foreign argument and return values in foreign function calls and callbacks can be specified by any of the following keywords:

:UNSIGNED-BYTE

The argument/return value is of type (UNSIGNED-BYTE 8)

:SIGNED-BYTE

The argument/return value is of type (SIGNED-BYTE 8)

:UNSIGNED-HALFWORD

The argument/return value is of type (UNSIGNED-BYTE 16)

:SIGNED-HALFWORD

The argument/return value is of type (SIGNED-BYTE 16)

:UNSIGNED-FULLWORD

The argument/return value is of type (UNSIGNED-BYTE 32)

:SIGNED-FULLWORD

The argument/return value is of type (SIGNED-BYTE 32)

:UNSIGNED-DOUBLEWORD

The argument/return value is of type (UNSIGNED-BYTE 64)

:SIGNED-DOUBLEWORD

The argument/return value is of type (SIGNED-BYTE 64)

:SINGLE-FLOAT

The argument/return value is of type SINGLE-FLOAT

:DOUBLE-FLOAT

The argument/return value is of type DOUBLE-FLOAT

:ADDRESS

The argument/return values is a MACPTR

:VOID

or NIL Not valid as an argument type specifier; specifies that there is no meaningful return value

Under Darwin (only), an small positive integer N can also be used as an argument specifier; it indicates that the corresponding argument is a pointer to an N-word structure or union which should be passed by value to the foreign function.

External entrypoints and named external entrypoints.

PowerPC machine instructions are always aligned on 32-bit boundaries, so the two least significant bits of the first instruction ("entrypoint") of a foreign function are always 0. OpenMCL often represents an entrypoint address as a fixnum that's binary-equivalent to the entrypoint address: if E is an entrypoint address expressed as a signed 32-bit integer, then (ash E -2) is an equivalent fixnum representation of that address. An entrypoint address can also be encapsulated in a MACPTR, but that's somewhat less efficient.

Although it's possible to use fixnums or macptrs to represent entrypoint addresses, it's somewhat cumbersome to do so. OpenMCL can cache the addresses of named external functions in structure-like objects of type CCL:EXTERNAL-ENTRY-POINT (sometimes abbreviated as EEP). Through the use of LOAD-TIME-VALUE, compiled lisp functions are able to reference EEPs as constants; the use of an indirection allows OpenMCL runtime system to ensure that the EEP's address is current and correct.

Functional Reference

All of the symbols mentioned below are defined in the CCL package; most (but not all) are exported from that package.

ccl:%ff-call [Function]

Syntax

ff-call entrypoint {arg-type-keyword arg}* &optional result-type-keyword

Description

Calls the foreign function at address entrypoint passing the values of each arg as a foreign argument of type indicated by the corresponding arg-type-keyword. Returns the foreign function result (coerced to a Lisp object of type indicated by result-type-keyword), or NIL if result-type-keyword is :VOID or NIL

Arguments

 

entrypoint

A fixnum or MACPTR

arg-type-keyword

One of the foreign argument-type keywords, described above

arg

A lisp value of type indicated by the corresponding arg-type-keyword

result-type-keyword

One of the foreign argument-type keywords, described above

ff-call [Macro]

Syntax

ff-call entrypoint {arg-type-specifier arg}* &optional result-type-specifier

Description

Calls the foreign function at address entrypoint passing the values of each arg as a foreign argument of type indicated by the corresponding arg-type-specifier. Returns the foreign function result (coerced to a Lisp object of type indicated by result-type-specifier), or NIL if result-type-specifer is :VOID or NIL

Arguments

 

entrypoint

A fixnum or MACPTR

arg-type-specifer

One of the foreign argument-type keywords, described above, or an equivalent foreign type specifier.

arg

A lisp value of type indicated by the corresponding arg-type-specifier

result-type-specifier

One of the foreign argument-type keywords, described above, or an equivalent foreign type specifier.

external [Macro]

Syntax

external name

Description

Maps name to an EXTERNAL-ENTRY-POINT object; tries to resolve the entrypoint address of the foreign symbol name if it's not already resolved.

Arguments
name

A lisp string. Note that, under Darwin, the names of C-callable external functions have underscores prepended to them.

ccl:%reference-external-entry-point [Function]

Syntax

ccl:%reference-external-entry-point eep

Description

Tries to resolve the address of the EXTERNAL-ENTRY-POINT eep; returns a fixnum representation of that address if successful, else signals an error.

Arguments

 

eep

An EXTERNAL-ENTRY-POINT, as obtained by the EXTERNAL macro.

ccl:external-call [Macro]

Syntax

ccl:external-call name {arg-type-specifier arg}* &optional result-type-specifier

Description

Calls the foreign function at the address obtained by resolving the external-entry-point associated with name, passing the values of each arg as a foreign argument of type indicated by the corresponding arg-type-specifier. Returns the foreign function result (coerced to a Lisp object of type indicated by result-type-specifier), or NIL if result-type-specifer is :VOID or NIL

Arguments

 

name

A lisp string. See external, above.

arg-type-specifer

One of the foreign argument-type keywords, described above, or an equivalent foreign type specifier.

arg

A lisp value of type indicated by the corresponding arg-type-specifier

result-type-specifier

One of the foreign argument-type keywords, described above, or an equivalent foreign type specifier.

ccl:foreign-symbol-entry [Function]

Syntax

ccl:foreign-symbol-entry name

Description

Tries to resolve the address of the foreign symbol name. If successful, returns a fixnum representation of that address, else returns NIL.

Arguments

 

name

A lisp string.

ccl:foreign-symbol-address [Function]

Syntax

ccl:foreign-symbol-address name

Description

Tries to resolve the address of the foreign symbol name. If successful, returns that address encapsulated in a MACPTR, else returns NIL.

Arguments

 

name

A lisp string.

defcallback [Macro]

Syntax

defcallback name ({arg-type-specifier var}* &optional result-type-specifier) &body body

Description

Proclaims name to be a special variable; sets its value to a MACPTR which, when called by foreign code, calls a lisp function which expects foreign arguments of the specified types and which returns a foreign value of the specified result type. Any argument variables which correspond to foreign arguments of type :ADDRESS are bound to stack-allocated MACPTRs.

If name is already a callback function pointer, its value is not changed (though it's arranged that an updated version of the lisp callback function will be called.) This feature allows for incremental redefinition of callback functions.

DEFCALLBACK returns the callback pointer, e.g., the value of name.

Arguments

 

name

A symbol which can be made into a special variable

arg-type-specifer

One of the foreign argument-type keywords, described above, or an equivalent foreign type specifier. In addition, if the keyword :WITHOUT-INTERRUPTS is specified, the callback will be executed with lisp interrupts disabled if the corresponding var is non-NIL. If :WITHOUT-INTERRUPTS is specified more than once, the rightmost instance wins.

var

A symbol (lisp variable), which will be bound to a value of the specified type.

body

A sequence of lisp forms, which should return a value which can be coerced to the specified result-type.

#_ [Reader macro]

The #_ reader macro:

  1. Reads a symbol from the current input stream, with *PACKAGE* bound to the "OS" package and with readtable-case preserved.

  2. Does a lookup on that symbol in the OpenMCL interface database, signalling an error if no foreign function information can be found for the symbol in any active interface directory.

  3. Notes the foreign function information, including the foreign function's return type, the number and type of the foreign function's required arguments, and an indication of whether or not the function accepts additional arguments (via e.g., the "varargs" mechanism in C).

  4. Defines a macroexpansion function on the symbol, which expand macro calls involving the symbol into EXTERNAL-CALL forms where foreign argument type specifiers for required arguments and the return value specifer are provided from the information ind the database.

  5. Returns the symbol.

The effect of these steps is that it's possible to call foreign functions that take fixed numbers of arguments by simply providing argument values, as in:

(#_isatty fd)
(#_read fd buf n)
   

and to call foreign functions that take variable numbers of arguments by specifying the types of non-required args, as in:

(with-cstrs ((format-string "the answer is: %d"))
  (#_printf format-string :int answer))
   

#? [Reader macro]

In OpenMCL 0.14.2 and later, the #? reader macro can be used to access foreign variables; this functionality depends on the presence of "vars.cdb" files in the interface database. The current behavior of the #? reader macro is to:

  1. Read a symbol from the current input stream, with *PACKAGE* bound to the "OS" package and with readtable-case preserved.

  2. Use that symbol's pname to access the OpenMCL interface database, signalling an error if no appropriate foreign variable information can be found with that name in any active interface directory.

  3. Use type information recorded in the database to construct a form which can be used to access the foreign variable, and return that form.

Note that the set of foreign variables declared in header files may or may not match the set of foreign variables exported from libraries (we're generally talking about C and Unix here ...). When they do match, the form constructed by the #? reader macro manages the details of resolving and tracking changes to the foreign variable's address.

Future extensions (via prefix arguments to the reader macro) may offer additional behavior; it might be convenient (for instance) to be able to access the address of a foreign variable without dereferencing that address.

Foreign variables in C code tend to be platform- and packge-specific (the canonical example - "errno" - is typically not a variable when threads are involved. )

In LinuxPPC,

? #?stderr

returns a pointer to the stdio error stream ("stderr" is a macro under OSX/Darwin).

On both LinuxPPC and DarwinPPC,

? #?sys_errlist

returns a pointer to a C array of C error message strings.

C structure return conventions

On both LinuxPPC and DarwinPPC, C functions that are defined to return structures or unions do by reference: they actually accept a first parameter of type "pointer to returned struct/union" - which must be allocated by the caller - and don't return a meaningful value. OpenMCL's interface translator make this explicit: it effectively prepends an :ADDRESS to the foreign function's argument list and changes its return type to :VOID.

DarwinPPC C functions actually follow a slightly different convention: if the size of the returned structure is 4 bytes or less, the "structure" is returned by value. This irregularity isn't handled by the interface translator; fortunately, it seems to very rarely occur in practice (nameservice functions that return a (:STRUCT :IN_ADDR) being a notable exception.)