TIP #424: Improving [exec]

Abstract

This extension overcomes day-1 limitations of [exec]'s syntax, allowing for unconstrained arguments to commands, and opening the path to more exotic redirections.

Summary Change

Replace:

   exec foo bar baz > file

With:

   exec | {foo bar baz} > file

Rationale

For decades people have rightfully complained about the stubborn limitation of exec that prevents it from using commands or args resembling a redirection. It's not just Quoting Hell; it is simply impossible to spawn the equivalent of Bourne Shell's "echo \>" from pure Tcl (i.e., without resorting to another shell).

The reason (excuse?) for this is an unfortunate design choice: stick as closely as possible to the Bourne Shell's syntax, which indeed seamlessly intertwines commands, arguments, and redirects. This is unfortunate, because it overlooks a key difference between the two shells:

• In Bourne Shell, since everything is about spawning commands, redirects are expected everywhere; hence their quoting is ubiquitous, and part of the language.

• In Tcl, spawning processes is only a tiny part of the story. Consequently, redirect chars (<>|) are not special, and deserve no core-language quoting rules.

In this situation, it would have been possible to add an exec-specific layer of quoting, just for these characters. But as usual, the quoting char itself (typically "\") would have itself needed quoting ("\\"), which would have overburdened the backslash density of all but the simplest pipelines...

More importantly, the realization that this was Really Wrong came fairly late in Tcl's life; or at least late enough to consider any incompatible fix out of the question.

So exec can be extended, not fixed.

A few such extensions have been suggested over the years, but none reached critical mass. A possible interpretation of this is that they were considered too "disruptive" - while necessary only for a corner case.

The current proposal addresses all the above concerns. Here are its design goals by decreasing importance:

1. Current exec's unescapable warts should disappear

   (Yeah, take care of that corner case.)

2. Current exec's mapping to open | should be carried over

   (This part of exec's design was Good)

3. Simple pipelines should give easy-to-read lines (like current exec)

   (No disruption, Ma'am)

4. Shell-ish advanced redirections like "3>&5" should be supported

   (Not just the corner case

   you get a free lunch too)

Definition

• Extend exec "from its error space", by reserving a single pipe character passed as its first argument:

         exec | ...    ;# activates the new syntax
         open "|| ..." ;# same in [open]
  

• Once the new syntax is unambiguously introduced, parse the rest as follows:

         exec | $cmd1 {*}$redirs1 | ... | $cmdN {*}$redirsN ?&?
         open "|[list | $cmd1 {*}$redirs1 | ... ]"
  

  where:

  • $cmdK and $redirsK are lists

  • $cmdK is a simple command-and-args, no extras

  • $redirsK is a list of current exec redirection operators

Examples:

      exec | {echo >} ;# this returns ">"
      exec | {cmd "<funny>xml</funny>"} 2>@ $ch < /dev/null | {cmd2 arg} >&2

Goals reached:

1. Unescapable warts are gone because the $cmd vs $redir status is positional, not content-based: each command-and args is a separate sublist, with no in-band encoding of redirections.

2. The above mapping is consistent with the existing open |[list foo bar] logic. It respects the invariant saying, for open |, that [string range $openarg 1 end] is always the list that would be passed, expanded, to exec. And it is handy to type open "|| {foo >} > file"

3. Simple pipelines are simple.

   exec | $cmd1 | $cmd2 | $cmd3 > file

4. Advanced redirections are imaginable since the redirection subsyntax now lives on its own. For example, with a putative "NUMBER>@" family of operators, one could define a nonlinear pipe graph:

         lassign [chan pipe] pr pw
         exec | {demuxer ...} 3>@ $pw | {filter ...} | {muxer ...} 3<@ $pr
   

The definition of these advanced operators will be hosted by another TIP.

TL;DR

This very conservative syntax, in addition to preserving the overall style and density of current exec, overcomes all the limitations and reaches Bourne Shell power.

Moreover, it leverages the existing internals, so a nearly free side-effect is that it works with pid and close just like current exec does.

Rejected Alternatives

• Replace the leading "|" in exec | by --extended

• Use a different toplevel command name.

  exec2...

Reference Implementation

Branch "tip-improve-exec" on core.tcl.tk holds the implementation.

Copyright

This document has been placed in the public domain.

[Index] [History] [HTML Format] [Source Format] [LaTeX Format] [Text Format] [XML Format] [*roff Format (experimental)] [RTF Format (experimental)]