TIP: 424 Title: Improving [exec] Version: $Revision: 1.8 $ Author: Alexandre Ferrieux State: Draft Type: Project Vote: Pending Created: 07-Jul-2013 Post-History: Keywords: Tcl,subprocess,execution Tcl-Version: 8.7 ~ Abstract This extension overcomes day-1 limitations of [['''exec''']]'s syntax, allowing for unconstrained arguments to commands, and opening the path to more exotic redirections. ~ Summary Change Replace: | exec foo bar baz > file With: | exec | {foo bar baz} > file ~ Rationale For decades people have rightfully complained about the stubborn limitation of '''exec''' that prevents it from using commands or args resembling a redirection. It's not just Quoting Hell; it is simply impossible to spawn the equivalent of Bourne Shell's "echo \>" from pure Tcl (i.e., without resorting to another shell). The reason (excuse?) for this is an unfortunate design choice: stick as closely as possible to the Bourne Shell's syntax, which indeed seamlessly intertwines commands, arguments, and redirects. This is unfortunate, because it overlooks a key difference between the two shells: * In Bourne Shell, since everything is about spawning commands, redirects are expected everywhere; hence their quoting is ubiquitous, and part of the language. * In Tcl, spawning processes is only a tiny part of the story. Consequently, redirect chars (<>|) are not special, and deserve no core-language quoting rules. In this situation, it would have been possible to add an '''exec'''-specific layer of quoting, just for these characters. But as usual, the quoting char itself (typically "'''\'''") would have itself needed quoting ("'''\\'''"), which would have overburdened the backslash density of all but the simplest pipelines... More importantly, the realization that this was Really Wrong came fairly late in Tcl's life; or at least late enough to consider any incompatible fix out of the question. So '''exec''' can be ''extended'', not ''fixed''. A few such extensions have been suggested over the years, but none reached critical mass. A possible interpretation of this is that they were considered too "disruptive" - while necessary only for a corner case. The current proposal addresses all the above concerns. Here are its design goals by decreasing importance: 1. Current '''exec''''s unescapable warts should disappear > (Yeah, take care of that corner case.) 2. Current '''exec''''s mapping to '''open |''' should be carried over > (This part of '''exec''''s design was Good) 3. Simple pipelines should give easy-to-read lines (like current '''exec''') > (No disruption, Ma'am) 4. Shell-ish advanced redirections like "'''3>&5'''" should be supported > (Not just the corner case: you get a free lunch too) ~ Definition * Extend '''exec''' "from its error space", by reserving a single pipe character passed as its first argument: | exec | ... ;# activates the new syntax | open "|| ..." ;# same in [open] * Once the new syntax is unambiguously introduced, parse the rest as follows: | exec | $cmd1 {*}$redirs1 | ... | $cmdN {*}$redirsN ?&? | open "|[list | $cmd1 {*}$redirs1 | ... ]" > where: > * '''$cmd'''''K'' and '''$redirs'''''K'' are lists > * '''$cmd'''''K'' is a simple command-and-args, no extras > * '''$redirs'''''K'' is a list of current exec redirection operators Examples: | exec | {echo >} ;# this returns ">" | exec | {cmd "xml"} 2>@ $ch < /dev/null | {cmd2 arg} >&2 Goals reached: 1. Unescapable warts are gone because the '''$cmd''' vs '''$redir''' status is positional, not content-based: each command-and args is a separate sublist, with no in-band encoding of redirections. 2. The above mapping is consistent with the existing '''open |[list foo bar]''' logic. It respects the invariant saying, for '''open |''', that '''[string range $openarg 1 end]''' is always the list that would be passed, expanded, to '''exec'''. And it is handy to type '''open "|| {foo >} > file"''' 3. Simple pipelines are simple. > '''exec | $cmd1 | $cmd2 | $cmd3 > file''' 4. Advanced redirections are imaginable since the redirection subsyntax now lives on its own. For example, with a putative "NUMBER>@" family of operators, one could define a nonlinear pipe graph: | lassign [chan pipe] pr pw | exec | {demuxer ...} 3>@ $pw | {filter ...} | {muxer ...} 3<@ $pr The definition of these advanced operators will be hosted by another TIP. ~ TL;DR This very conservative syntax, in addition to preserving the overall style and density of current '''exec''', overcomes all the limitations and reaches Bourne Shell power. Moreover, it leverages the existing internals, so a nearly free side-effect is that it works with '''pid''' and '''close''' just like current '''exec''' does. ~ Rejected Alternatives * Replace the leading "'''|'''" in '''exec |''' by '''--extended''' * Use a different toplevel command name. > '''exec2'''... ~ Reference Implementation Branch "tip-improve-exec" on core.tcl.tk holds the implementation. ~ Copyright This document has been placed in the public domain.