TIP #66: Stand-alone and Embedded Tcl/Tk Applications

TIP:66
Title:Stand-alone and Embedded Tcl/Tk Applications
Version:$Revision: 1.15 $
Author:Arjen Markus <arjen dot markus at wldelft dot nl>
State:Draft
Type:Informative
Vote:Pending
Created:Tuesday, 02 October 2001
Keywords:installation, initialisation, embedded, resources

Abstract

This TIP describes the development and deployment of Tcl/Tk applications, with particular attention on how to embed the interpreter into executables written in C or C++.

Introduction and Background

Usually, an application that uses Tcl/Tk in some way uses an independent installation and the application itself is started via a standard shell, like tclsh or wish. There are numerous occasions when such a set-up is not convenient:

Another reason to document the resources used by Tcl/Tk is that this provides better insight in how to tune Tcl/Tk for a particular application.

Two examples may illustrate the need for such stand-alone applications and what is involved:

  1. When we were building an installation script for an MS Windows application using one of the commercial tools that are available for this arcane job, we ran into a bizarre limitation: text replacement was possible for the so-called Windows INI-files only, but not for other types of files. The text to be replaced was the name of the installation directory. After several trials with the programming constructs the tool allowed, we chose a much better solution: a small Tcl script wrapped into a stand-alone program using Freewrap. (The application itself now actually uses another stand-alone Tcl script to take care of the file management that was too complicated for ordinary DOS batch files.)

  2. The second example involves a small program that proves the usefulness of Tcl/Tk in on-line visualisation. The idea there is that large computational programs can send their data at regular steps during the computation to a separate program that plots these results in some meaningful way. To achieve this the program exports the results to the Tcl interpreter which uses the socket command to send them to a (primitive) viewer. For demonstration purposes you must be able to copy the program along with some files it needs on an arbitrary computer and, later, remove it with just a little effort.

Applications that use Tcl/Tk as an embedded library to achieve their goals, rather than exist as extensions or applications written in Tcl, can be quite useful. Examples include on-line visualisation in large computational programs, network applications that can be deployed as a single file etc. There is, however, little documentation on how to build such applications and what is required for their installation.

The aim of this TIP is to provide guidelines that make this development easier:

Related TIPs and Discussions

The are several TIPs at the moment of this writing that are in some way related to the subject:

Contents of the Planned Document

The document that should help programmers with the issues discussed here will have the following (tentative) table of contents:

Discussion

Issues that arise are:

This TIP is meant to be a document that enables programmers who do not have intimate knowledge of the Tcl core to build such application and deploy them in the way they want.

Should it turn out that some automated tool would be nice to help the programmers, then this TIP will also cover such a tool.

Using the Tcl library

There are numerous ways an application written mainly in a language like C can use the Tcl and Tk libraries (in short: Tcl):

* The application can use Tcl in a more complicated mixture

Tcl scripts get evaluated that require binary extensions (both defined outside the application and as an integral part of the application).

Note: due to the fact that the author is mostly familiar with the UNIX/LINUX and Windows platforms, no comments will be made about the Macintosh. This is completely due to ignorance, not to arrogance.

In principle, using the Tcl/Tk libraries is very simple: just create a Tcl interpreter, fill it with variables, commands and so on and feed it scripts, either as a file or as a string. It gets more complicated in the following situations:

The key to a successful implementation is: understanding how to properly initialise Tcl.

The application with which we will illustrate the various options is a simple program without any virtues of its own. It will read some data from a file, perform an insanely complicated but further unspecified computation on these data and output them into some convenient format to file.

The application will have two versions, a simple one consisting of the three separate steps and a more complicated one consisting of a preliminary step and then a loop involving both the computation and the output.

Let us assume that the application is written in some convenient programming language like C. The reasons for using Tcl are:

The simplest way: create a bare interpreter

With the Tcl routine Tcl_CreateInterp() you can create an interpreter that is capable of all the basic commands:

  Tcl_Interp * interp         ;
  char       * input_filename ;
  char       * buffer         ;
  double       x, y, z        ;

  /* Create the interp, use it to read the given input file,
     Note:
     Using the string API for simplicity, no error checking
  */
  interp = Tcl_CreateInterp() ;
  Tcl_SetVar( interp, "input_file", input_filename, TCL_GLOBAL_ONLY ) ;
  Tcl_EvalFile( interp, startup_script ) ;

  /* Extract the input data
  */
  buffer = Tcl_GetVar( interp, "x", TCL_GLOBAL_ONLY ) ;
  Tcl_GetDouble( interp, buffer, &x ) ;
  buffer = Tcl_GetVar( interp, "y", TCL_GLOBAL_ONLY ) ;
  Tcl_GetDouble( interp, buffer, &y ) ;
  buffer = Tcl_GetVar( interp, "z", TCL_GLOBAL_ONLY ) ;
  Tcl_GetDouble( interp, buffer, &z ) ;

  /* Destroy the interp - if you do not need it any longer
  */
  Tcl_DestroyInterp( interp ) ;

The output routine contains a similar fragment (note, we assume the Tcl interpreter was stored somewhere):

  Tcl_Interp * interp                   ;
  char       * output_filename          ;
  char         buffer[TCL_DOUBLE_SPACE] ;
  double       a, b                     ;

  /* Export the results to the interpreter
  */
  Tcl_PrintDouble( interp, a, buffer ) ;
  Tcl_SetVar( interp, "a", buffer, TCL_GLOBAL_ONLY ) ;
  Tcl_PrintDouble( interp, b, buffer ) ;
  Tcl_SetVar( interp, "b", buffer, TCL_GLOBAL_ONLY ) ;

  Tcl_SetVar( interp, "output_file", input_filename, TCL_GLOBAL_ONLY ) ;
  Tcl_EvalFile( interp, report_script ) ;

To add error checking (always do!), use code like this:


  Tcl_Channel errChannel ;

  if ( Tcl_EvalFile( ... ) != TCL_OK ) {
     errChannel = Tcl_GetStdChannel( TCL_STDERR ) ;
     if ( errChannel != NULL ) {
        TclWriteObj( errChannel, Tcl_GetObjResult(interp) ) ;
        TclWriteChars( errChannel, "\n", -1 ) ;
        ... /* Quit the program or other error handling? */
     }
  }

With this approach you need only to worry about the Tcl binary libraries: if the dynamic versions are linked to your application, then distribution of your application should include these binaries. If, on the other hand the static versions are used, your application already contains all of Tcl it needs all by itself.

The limitations of this approach are:

Complete initialisation: the role of init.tcl

The next section outlines the full initialisation procedure that is used in the standard tclsh shell. This section concentrates instead on some practical observations:

The script init.tcl and any it sources (directly or indirectly via auto_load) must be found via the tcl_library variable. On UNIX this variable is initialised via the TCL_LIBRARY environment variable is used, whereas on MS Windows the pathname of the Tcl DLLs is used as well.

As long as these scripts can be found, they can actually reside in a large number of directories with names related to the Tcl library path.

This leads to the following code to create a full-fledged interpreter:

  Tcl_Interp * interp         ;

  /* Initialise the Tcl library thouroughly
  */
  Tcl_FindExecutable( argv[0] ) ;

  /* Create the interp, evaluate "init.tcl" for the script
     level initialisation.
  */
  interp = Tcl_CreateInterp() ;

  if ( Tcl_Init( interp ) != TCL_OK ) {
     ... Report the error
  }

With init.tcl loaded, we have a number of additional commands and global variables:

To create an interpreter that can handle Tk as well, you should be aware of the following:

TODO: how to write the event loop, what choices are available?

Initialisation via the standard shell

The details of the initialisation done in the standard tclsh shell are quite intricate. They involve, in addition to the initialisation via Tcl_FindExecutable() and Tcl_Init() also:

A summary of the steps found in the initialisation code is given below:

Thus, before the shell is ready for processing, a lot of initialisation is done. Much of this process can be customised without the need to change the standard source files.

Overview

This section provides an overview of the resources that an application requires, given the type of usage:

Bare Tcl only interpreter:

Complete initialisation for Tcl only:

Customised Tcl shell (adapted Tcl_AppInit()):

Customised Tk shell (wish; adapted Tk_AppInit()):

Equally important are the limitations:

Bare Tcl only interpreter:

Complete initialisation for Tcl only:

Customised Tcl shell (adapted Tcl_AppInit()):

Customised Tk shell (wish; adapted Tk_AppInit()):

Compiling and linking

Nowadays, it seems the default to use dynamic or shared libraries. So, with many installations, there will exist dynamic versions of the libraries and sometimes there will be no static versions. This has a number of advantages:

However, as the Tcl libraries now reside outside your application, they will have to be shipped with the application and the dynamic loader must somehow be able to find the libraries. The latter certainly has consequences: each system tends to have its own method.

When you have the Tcl/Tk sources, you can decide to create your own libraries. Of special interest are the following two situations:

Stubs were introduced to make binary extensions and applications independent of the specific Tcl version. They are enabled by defining the macro TCL_USE_STUBS during the compilation and linking of the Tcl/Tk library and especially your own extension.

In the initialisation procedure for your pacakge or application you need to initialise the stubs jump table via Tcl_InitStubs():

 #ifdef USE_TCL_STUBS
    if (Tcl_InitStubs(interp, "8.1", 0) == NULL) {
       return TCL_ERROR;
    }
 #endif

(details: [1])

The technique, as Brent Welch explains, is simple in principle:

By enabling stubs, all calls to Tcl routines are turned into function pointers. These pointers are kept in a large table that is filled with the correct pointer values via the Tcl_InitStubs() routine.

Linking your application or extension should then be done against the "stub version" of the Tcl/Tk libraries.

If you do not want dynamic libraries, then perhaps a build with the option STATIC_BUILD is a solution. With this option, static libraries are built. The libraries are then incorporated into the executable itself.

Note: On some platforms, notably Windows, the specific calling convention is then turned to standard C (with dynamic libraries, the calling convention exports the various routines explicitly).

When you do not care about the dynamic libraries having to be present, at least be aware of the way the various systems want to define their position.

The information above is summarised as follows:

Using dynamic libraries:

* Some use the variable SHLIB_PATH instead (notably

HPUX).

Building for general Tcl versions:

Building statically:

Copyright

This document is placed in the public domain.

Powered by Tcl[Index] [History] [HTML Format] [Source Format] [LaTeX Format] [Text Format] [XML Format] [*roff Format (experimental)] [RTF Format (experimental)]

TIP AutoGenerator - written by Donal K. Fellows