Tcl: A Platform for Integration Applications

Integration problem | Integration drivers | Traditional approaches | Tcl's advantages | Other scripting languages

If you are writing integration applications then you need Tcl. Tcl provides a dramatically easier way to build integration applications ranging from simple graphical user interfaces to complex financial, Web, and management applications. This page explains what integration applications are, why they are becoming more and more important, and why Tcl provides the best platform for building them.

The business problem: integration

Businesses and engineering teams today are often faced with the problem of making diverse collections of resources work together. We call these programming tasks integration applications. The resources managed by integration applications can take many forms, such as:

Components implemented using frameworks such as ActiveX or JavaBeans.
Devices such as manufacturing equipment or test equipment.
Applications such as Web servers, enterprise applications, and computer-aided design tools.
Data sources such as databases, live news feeds, and Web sites.
Data formats such as HTML and XML.
Protocols such as CORBA, DCOM, and HTTP.

Typically integration applications involve communicating between resources such as these, coordinating their operation, and extending their basic capabilities with additional functions.

Drivers for integration applications

Integration applications have existed for many years but their importance has risen dramatically over the last 5-10 years and will continue to rise in the years ahead. The need for integration applications is being driven by some of the most important trends in the software industry, such as:

Online information management. More and more of the world's information is being stored and accessed online; managing and presenting this information is an integration application. For example, high-end Web sites such as those at AOL's Digital City, CNET, and Travelocity create customized Web pages for their visitors by integrating information from different sources such as databases, live news feeds, and travel reservation systems. Or, Wall Street trading firms must filter and integrate information from a variety of different sources in order to provide traders with the information they need.

Networked control of devices. Over the last decade computers have been embedded in many of the world's devices. These embedded processors typically provide remote control interfaces, and with the rise of the computer networking it has become easy to reach them and manage them remotely. Managing collections of networked devices is an integration application. For example, in factory automation the various pieces of equipment on a factory floor must be integrated with each other and with databases to control and track production. Or, in automated testing the devices are diverse pieces of test equipment, which must be coordinated to exercise a test device and extract performance information.

Network aggregation. The rise of the Internet, combined with enterprise consolidation, is creating large and heterogeneous networks of hardware and software systems. Making these systems work well together is an integration application. For example, different departments of hospitals have historically developed their own (different) computer systems. In the past there was no attempt to communicate between the departments so the differences were not noticed. Now, however, hospitals are integrating all of their systems in order to coordinate and better manage the flow of information. As a result, they must now deal with the differences in the departmental systems. Another example is corporate growth by mergers and acquisitions. A company may initially have computer systems that are uniform, but in a merger it is likely to inherit systems that are different. After the merger, all of these systems must be integrated to work together.

Component frameworks and protocols. Several component frameworks, such as ActiveX and JavaBeans, have become popular in recent years. These frameworks encourage the creation of reusable software components. Making components work together is an integration problem. The rise of the Internet has also led to numerous protocols for communicating between components and applications, such as CORBA and DCOM. Using these protocols is also an integration task. In addition, it is becoming increasingly important to tie together different protocols and component frameworks, which is an even more challenging integration task.

Traditional approaches don't work

Integration applications have characteristics quite different from traditional programming tasks. In traditional programming the main problem is to create data structures and algorithms from scratch. System programming languages such as C, C++, and Java work well for these tasks.

In integration applications, data structures and algorithms are not usually very important: they are encapsulated in the components being integrated. The most difficult challenges in integration applications have to do with linkage and coordination:

How to deal with the variety of things being integrated?
How to make them all work together?
How to evolve the applications rapidly in response to changing needs?

Integration applications often incorporate business rules and processes. As a result, they tend to be ill-structured and evolve rapidly. It is difficult to plan integration applications in advance, and they must often be modified in the field as needs change.

Furthermore, the people who write integration applications are often not sophisticated programmers. Their primary area of expertise is typically in a particular domain such as factory management, financial services, or Web content. They use programming as part of their job but don't consider themselves to be first and foremost programmers.

Because of these characteristics, system programming languages such as C, C++, or even Java are poorly suited to integration tasks. The main problem with system programming languages is inflexibility, which stems from their compilation and strong typing. Compilation freezes the functionality of an application at the time it is compiled, making it difficult to modify or extend the behavior of the application in the field. Strong typing forces programs to be carefully planned in advance and makes it difficult for them to be used in ways that weren't planned. As a result, system programming languages cannot accommodate the variety and rapid evolution that are typical in integration applications. System programming languages are also too complex and difficult to learn for many of the people who build integration applications.

Thus it is expensive to build integration applications with system programming languages, and the resulting applications tend to be brittle and inflexible.

The power of Tcl

In contrast, Tcl provides a superb platform for creating integration applications. Tcl's power comes from two basic features.

First, Tcl makes it easy to connect to any of the things you need to integrate. If you need to connect any X to any Y, it is easy to create one Tcl extension that connects to X, another that connects to Y, and use Tcl as the intermediary between them. Dozens of free extensions are already available for database access, network management, and many other purposes.

Second, with Tcl it is easy to write scripts that manage the connections in powerful ways. In contrast to system programming languages, Tcl is interpreted and typeless. The interpreted nature of Tcl makes it easy to modify and extend applications on the fly and evolve them rapidly. By being typeless and string-oriented, Tcl hides the differences between components and makes it easy to move information between them.

The combination of these two features allows integration applications to be developed 5-10 times more efficiently with Tcl than with system programming languages such as C++ or Java, measured either in development time or in lines of code. Furthermore, the applications created with Tcl are more powerful and flexible. If you are developing an integration application, you should be using Tcl for it.

Other scripting languages

Tcl is a member of the class of languages known as scripting languages. There are many other scripting languages besides Tcl, including JavaScript, Visual Basic, Perl, and others. As a group, all of the scripting languages tend to be used for integration applications, and all offer significant benefits over system programming languages.

Each scripting language has particular strengths. For example, JavaScript is known for its smooth integration with Web browsers, Visual Basic for its easy-to-learn development environment, and Perl for its string-handling capabilities. Tcl's greatest strength is its versatility: it can be embedded in applications or used standalone, it has outstanding GUI capabilities, and it can easily be connected to nearly any other application or protocol. Tcl was designed from the start to be used for many different purposes in many different situations, and the tremendous diversity of Tcl applications demonstrates that it has met this design goal.

In contrast, most other scripting languages were designed for a narrower set of tasks. They perform well for those specific tasks but they aren't used for as many different things as Tcl. For example, JavaScript is the obvious choice to use for simple scripting in a browser, but it is rarely used for anything outside the browser. Visual Basic provides excellent facilities for creating Windows GUIs, but it isn't suitable for integrating Windows desktops with Unix servers. Perl's string handling makes it an excellent choice for system administration tasks, report generation, and Web scripting, but it doesn't have native GUI capabilities and it isn't as easily embeddable as Tcl.