Tcl Server Reference

Basic Configuration

The main Tcl script consists of package require commands and module-specific initialization commands. The sample main script is bin/httpd in the distribution. The things you may want to change are inbetween the CONFIGURATION and END CONFIGURATION lines in the script.

Document Root

The document root is the name of the directory on your server machine that corresponds to the root of server's URL hierarchy. This is defined with the Doc_Root command:

Doc_Root /usr/local/htdocs

With no arguments, Doc_Root returns the current root.

If you have additional directories you need to paste into the hierarchy you can use Doc_AddRoot. This creates sort of a symbolic link from a URL to some point in your server's file system:

Doc_AddRoot virtual directory

The term virtual means a URL path like /cgi-bin or /foobar. The directory is a file pathname like /export/disk3/foobar.

Per-User Directories

If you want users to export their own pages under ~user URLs, then use the Doc_PublicHtml command to define what subdirectory of their home is used.

Doc_PublicHtml public_html

Index Files

The Doc_IndexFile specifies a pattern for the base name of the index file. The pattern can be an exact name, like "index.html", or a pattern that allows any suffix (e.g., index.*) If more than one file matches the pattern, the most recently modified file is used.

Doc_IndexFile index.{html,shtml,htm}

Log Files

The server generates log files in a standard format that is understood by various log-processing utilities. (If you have special needs you can implement a new log module.) The log files are automatically rotated every day, except that there is only one error log that is always appended. You specify the base name of the log file and a different suffix is appended for the daily logs and the error file:

Log_SetFile /usr/local/tclhttpd/log:

The logs are flushed periodically instead of on each log entry, except the error log which is flushed immediately. You can set the period with:

Log_FlushMinutes minutes

Server Name and Port

You set the server's host name and port with the Httpd_Server command. This command opens the socket and activates the server:

Httpd_Server 80 tcl.activestate.com

If you need to specify the server's own IP address, in case you have multiple servers running on different interfaces, just append the IP address to this command:

Httpd_Server 80 hostname 152.70.4.123

If you are going to run several servers you'll need to clone the startup script and run a different Tcl process for each server.

URL Domain Handlers

The server partitions the URL space into "domains" that are uniquely identified by a prefix of the URL. The longest matching prefix is used. For example, "/" is the prefix of all URLs, but you can also have a "/device" domain and a "/status" domain that are handled by different Tcl procedures. Basic file system service is provided by the Doc domain (i.e., doc.tcl), which makes a call like this:

Url_PrefixInstall / [list DocDomain $Doc(root)]

but the CGI domain makes a call like

Url_PrefixInstall /cgi-bin [list CgiDomain $directory]

This registers a Tcl procedure that is called each time a request matches the prefix. The longest matching prefix is used to resolve conflicts between prefixes like "/" and "/cgi-bin". The domain handler is invoked like this:

DocDomain $directory $sock $suffix

The $sock and $suffix are added by Url_Dispatch when the handler is invoked. The $directory was defined at the time Url_PrefixInstall was called. A domain handler can be installed with any arguments it needs. The $sock is the socket for the connection, and it is used in calls on the Httpd module to return results. The $suffix is the part of the URL after the prefix is stripped off.

Sample domain handlers include DocDomain, CgiDomain, and DirectDomain, which can be found in doc.tcl, cgisrv.tcl, and direct.tcl, respectively.

Application Direct URLs

The Direct module implements a mapping from URLs to Tcl procedure calls. The name of the URL determines the procedure, and the arguments come from query data. Use Direct_Url to define the command prefix associated with a URL subtree. For example:

Direct_Url /status Status Direct_Url /mail Mail

This declares that URLs beginning with /status are handled by Tcl procedures that begin with Status, and URLS beginning with /mail are handled by Tcl procedures that begin with Mail. The URL must match the Tcl procedure exactly. For example:

proc Status/hello {args} {return hello}

This defines an implementation for the /status/hello URL that returns a page containing the string "hello".

If the procedure has arguments defined, then the Direct module looks for those names in the query data posted (or included) with the URL. The args parameter is used to collect extra form data. If there is no matching form data then the parameter gets its default value or the empty string. For example, the server definesproc Mail/formdata {email subject args} { ... }

If you have an HTML form with email and subject fields, then you can use /mail/formdata as the action URL for your form. The Mail/formdata Tcl procedure is called with email and subject bound to the values from the form. Any additional fields in the form are collected into args as a list that alternates between name and value.

Document Type Handlers

The Doc domain supports handlers for MIME types. The mime.types file establishes a mapping from file suffixes (e.g., html) to MIME types (e.g. text/html). For example, files with suffix .shtml are type application/x-server-include. The Doc module checks to see if a handler exists for a type before returning the file. If it exists, the handler is called like this:

Doc_application/x-server-include $path $suffix $sock

The $path is the absolute pathname of the file. The $suffix is the URL suffix. The $sock is the socket handle for the connection.

Example handlers include Doc_application/x-cgi, Doc_application/x-imagemap, Doc_application/x-tcl-subst, and Doc_application/x-server-include.

Safe-Tcl in an external process

This is currently broken, and safesrv.tcl and safecgio.tcl need to be updated to the new server core. Instead of putting the Safe-Tcl processing in another file, it is done in a the main process using the "Subst" approach described below.

HTML+Tcl Templates

The DocTemplate procedure is a handler for application/x-tcl-template documents. By default, these documents have the suffix .tml. They are a mixture of normal HTML tags and in-line Tcl commands and variable references.

For example, a simple time.tml file might contain:

<h1>Hello World</h1>
The current time is [clock format [clock seconds]]

The DocTemplate procedure replaces the clock Tcl command with its result and returns something like this to the browser:

<h1>Hello World</h1>
The current time is Sat Apr 18 11:01:49 PDT 1998

Caching results in .html files

If your fetch the .tml file directly, the server always does template processing. The return meta-data does not include a modify time, so proxies and browsers will not cache the page. However, if you fetch a .html file and there is a corresponding .tml file, then the server can check to see if the template is newer than the .html file. If it is, then the template is processed, the results are cached in the .html file, and the .html file is returned to the client. In this case a modify time is returned so proxies and browsers can cache the file.

Why cache the results of template processing? We use templates to provide the general look and feel of our site. Most pages are static and just use the templates for standard headers, images, and so on. Caching these pages makes sense because the template will generate the same thing every time.

There are two configuration command that affect template results caching.

Doc_CheckTemplates boolean: This turns on (or off) template checking for regular .html files. If a .html file is requested and there is a corresponding .tml file, then the .html file is regenerated if necessary to keep it up-to-date. By default, this is OFF.
DynamicOnly: This is meant to be called from inside an HTML page. This prevents the results from being cached in the .html file. It means you maintain a .tml file, the browser can fetch the .html file, but the server will always process the template and return the data in a way that prevents caching by proxies and browsers.

What I normally do is run with template checking on and use DynamicOnly in my dynamic pages.

Permissions and the setuid extension

If your web server process is caching template results in .html files it must have permission to write those files. You can run the server as yourself and control permissions that way. However, if you want to run under port 80 you need to start the server as root. For security reasons most web servers do a setuid call to change into a normal user after opening their main socket.

The src directory of the distribution has a setuid.c file that implements a "setuid" Tcl command. This will only work if the server starts as root. The setuid command takes two arguments, a user ID and a group ID. For now they must be numeric, not symbolic IDs.

What I do is have a tclhttpd user account and a "web" group. I have the server setuid to these after it calls Httpd_Server. Then I make the htdocs hierarchy have group ownership of "web" and group write permission. The directories can be owned by the webmaster - they do not have to be owned by tclhttpd and probably should not be. This is what this directory looks like:

drwxrwxr-x   4 welch    web           512 Apr 18 12:38 ./
drwxrwxr-x  13 welch    web          1024 Apr 18 11:30 ../
-rw-r--r--   1 welch    web           201 Apr 18 11:31 .tml
-rw-r--r--   1 tclhttpd web          8184 Apr 18 12:38 index.html
-rw-r--r--   1 welch    web          5566 Apr 17 22:00 index.tml
drwxr-xr-x   2 welch    web           512 Apr 15 23:15 patch2.0.1/
drwxr-xr-x   2 welch    web           512 Jan  2 09:37 patch2.1/
-rw-r--r--   1 welch    web         14493 Apr 18 12:38 reference.html
-rw-r--r--   1 welch    web          1689 Apr 18 11:45 register.html
-rw-r--r--   1 tclhttpd web         14805 Apr 18 11:48 server.html
-rw-r--r--   1 welch    web         12186 Apr 18 11:48 server.tml
-rw-r--r--   1 welch    web          7209 Apr 18 11:37 usenixAbstract.html

You can see where the server has cached a couple .html files and that the current directory is group writable. Note also that I can't edit the cached files because they are owned by the server.

Tcl code for Templates

The server supports a library of Tcl scripts that are to be used with your .tml documents. This is typically in the /libtml directory of your htdocs area. In addition, the server looks for ".tml" files (that's a file beginning with .) in each directory up to the root of the URL tree. The idea is that you put global settings in the /.tml file, and you put per-directory definitions in local .tml files. These files are sourced, so they should contain Tcl commands. The upper-most .tml file is sourced first, and the .tml file in the current directory is sourced last, just before the template file itself.

You can find example .tml files in the sample htdocs directory.

The server checks date stamps on the .tml files and will regenerate a .html page if either its corresponding .tml file or any of the per-directory .tml files are newer. So, touching the root .tml file will cause all your generated .html files to be preprocessed the next time they are fetched.

Configuration Commands

As well as Doc_CheckTemplates described above, your main init script can use these commands to affect template processing:

Doc_TemplateInterp interp: This defines which interpreter in which the templates are processed. You should create this and set up any aliases used in the templates before calling this. By default the main interpreter is used.
Doc_TemplateLibrary dir: This defines where the script library for template support is.
Doc(tmlSuffix): If for some reason you hate the ".tml" suffix, you can redefine this variable to have an alternate suffix.

Old .subst files

The template mechanism above superceeds the originial .subst templates. Those are still supported, but they lack the caching support, the script library, and the per-directory .tml files.

Sessions and Session State

The Session module keeps track of session state in slave interpreters. A session has a 4-character ID that is dynamically generated when a new session is created. Sessions work by passing around the session ID in URL requests. For example, a URL that starts a new session looks like:

/something.special?session=new

The MIME type handler for .special (e.g., Doc_application/x-whizbang) must process the query data and then process the named file. When it sees the session=new it calls Session_Create to create a new slave interpreter for the session and return the ID. If it sees something like

/something.special?session=3f3b

Then it calls Session_Match to hook up with the existing interpreter state.

The Doc_application/x-tcl-snmp is the best example that uses session state. It uses Url_DecodeQuery to parse incoming form data, Session_Match to hook up to session state, and SnmpProcess to handle the form data in an application-specific way. It then falls through to Doc_Subst to process the page in the context of the session.