heredoc Shell Quotes

Today we learned a now abandoned perl idiom for bare heredocs. In a recent wiki hangout we were reminded of the challenges of controlling interpolation in bash across ssh commands. We are thus inspired to create this place to collect hard-earned, esoteric programming experience.

Ward wrote some perl to convert Eric Evan's Domain Driven Design (DDD) pattern language into federated wiki. github

The script does not work with perl v5.28.2.

$ perl convert.pl Use of bare << to mean <<"" is forbidden

Perl is a beautiful and terrible language. It turns out an early version of perl permitted bare heredocs and Ward uses them in several places in the convert.pl script. github

1. sub page { 2. my ($title, @story) = @_; 3. my $slug = slug $title; 4. my $story = join ',', @story; 5. `echo $slug >> generated`; 6. open P, ">../pages/$slug"; 7. print P <<; 8. { 9. "title": "$title", 10. "story": [ 11. $story 12. ] 13. } 14. 15. }

Line 7 opens the bare heredoc and line 14 closes it with an empty line. The JSON in-between will be rendered with two variables interpolated: $title and $story and then printed into the file at ../pages/$slug.

The bare closing of the heredoc is still permitted in perl v5.28.2, but the opening must be made more explicit.

7. print P <<"";

See wikipedia's article on Here Documents: wikipedia

Why? Why do we need heredocs? The use of heredocs makes code more readable when there are nested contexts for string interpolation. In the following example $DIR is interpolated in the context of the computer we're currently in and $PWD will be interpolated in the context of the remote computer at some.other.server.example.com.

# remote code execution ssh some.other.server.example.com <<EOF cd $DIR echo \$PWD ls -1 . EOF

.

bash and zsh offer process substitution expressions that allow a subshell to be treated as a file. For example, we can ask our shell which of wiki's plugins do not put the client code inside a folder named "client".

comm -23 \ <(ls -d wiki-plugin-*) <(ls -d wiki-plugin-*/client | sed 's/\/client$//')

The resulting list of plugins:

wiki-plugin-future wiki-plugin-image wiki-plugin-paragraph

See comm man page: man .

See process substitution: doc

POSIX shells can be asked the same sort of question by combining numbered file descriptors, numbered redirection, heredocs, and subshells.stackexchange

This example also shows the surprising feature that multiple heredocs can be used in a single expression.

comm -23 /dev/fd/3 3<<-EOF /dev/fd/4 4<<-EOF $(ls -d wiki-plugin-*) EOF $(ls -d wiki-plugin-*/client | sed 's/\/client$//') EOF

.

See other tricks with POSIX shells at Shell Games.