Add support for per-job output sync.

A new flag to the -O/--output-sync, "job", selects a per-job (that is, per
line of a recipe) output synchronization.  To support this move the close of
the temp file out of the sync_output() function and don't do it until we free
the child, since we may call sync_output() multiple times in a given recipe.
When we set up for a new temp file, if we're in per-job mode we truncate the
file and seek to the beginning to re-use it for every job.

1 files changed, 126 insertions, 51 deletions

diff --git a/doc/make.texi b/doc/make.texi
index ce8e07b..64be3b3 100644
--- a/doc/make.texi
+++ b/doc/make.texi
@@ -219,6 +219,11 @@ Recipe Execution
 * Choosing the Shell::          How @code{make} chooses the shell used
                                   to run recipes.
 
+Parallel Execution
+
+* Parallel Output::             Handling output during parallel execution
+* Parallel Input::              Handling input during parallel execution
+
 Recursive Use of @code{make}
 
 * MAKE Variable::               The special effects of using @samp{$(MAKE)}.
@@ -4057,48 +4062,16 @@ If there is nothing looking like an integer after the @samp{-j} option,
 there is no limit on the number of job slots.  The default number of job
 slots is one, which means serial execution (one thing at a time).
 
-When running several recipes simultaneously the output from each
-recipe appears as soon as it is generated, with the result that
-messages from different recipes may be interspersed.  To avoid this
-you can use the @samp{--output-sync} (@samp{-O}) option; if this
-option is provided then the output from each recipe will be saved
-until the recipe is complete, then written all at once.  This ensures
-that output from different recipes is not mixed together.
-
-Another problem is that two processes cannot both take input from the
-same device; so to make sure that only one recipe tries to take input
-from the terminal at once, @code{make} will invalidate the standard
-input streams of all but one running recipe.  This means that
-attempting to read from standard input will usually be a fatal error (a
-@samp{Broken pipe} signal) for most child processes if there are
-several.
-@cindex broken pipe
-@cindex standard input
-
-It is unpredictable which recipe will have a valid standard input stream
-(which will come from the terminal, or wherever you redirect the standard
-input of @code{make}).  The first recipe run will always get it first, and
-the first recipe started after that one finishes will get it next, and so
-on.
-
-We will change how this aspect of @code{make} works if we find a better
-alternative.  In the mean time, you should not rely on any recipe using
-standard input at all if you are using the parallel execution feature; but
-if you are not using this feature, then standard input works normally in
-all recipes.
-
-Finally, handling recursive @code{make} invocations raises issues.  For
-more information on this, see
-@ref{Options/Recursion, ,Communicating Options to a Sub-@code{make}}.
+Handling recursive @code{make} invocations raises issues for parallel
+execution.  For more information on this, see @ref{Options/Recursion,
+,Communicating Options to a Sub-@code{make}}.
 
 If a recipe fails (is killed by a signal or exits with a nonzero
-status), and errors are not ignored for that recipe
-(@pxref{Errors, ,Errors in Recipes}),
-the remaining recipe lines to remake the same target will not be run.
-If a recipe fails and the @samp{-k} or @samp{--keep-going}
-option was not given
-(@pxref{Options Summary, ,Summary of Options}),
-@code{make} aborts execution.  If make
+status), and errors are not ignored for that recipe (@pxref{Errors,
+,Errors in Recipes}), the remaining recipe lines to remake the same
+target will not be run.  If a recipe fails and the @samp{-k} or
+@samp{--keep-going} option was not given (@pxref{Options Summary,
+,Summary of Options}), @code{make} aborts execution.  If make
 terminates for any reason (including a signal) with child processes
 running, it waits for them to finish before actually exiting.@refill
 
@@ -4131,6 +4104,105 @@ average goes below that limit, or until all the other jobs finish.
 
 By default, there is no load limit.
 
+@menu
+* Parallel Output::             Handling output during parallel execution
+* Parallel Input::              Handling input during parallel execution
+@end menu
+
+@node Parallel Output, Parallel Input, Parallel, Parallel
+@subsection Output During Parallel Execution
+@cindex output during parallel execution
+@cindex parallel execution, output during
+
+When running several recipes in parallel the output from each
+recipe appears as soon as it is generated, with the result that
+messages from different recipes may be interspersed, sometimes even
+appearing on the same line.  This can make reading the output very
+difficult.
+
+@cindex @code{--output-sync}
+@cindex @code{-O}
+To avoid this you can use the @samp{--output-sync} (@samp{-O}) option.
+This option instructs @code{make} to save the output from the commands
+it invokes and print it all once the commands are completed.
+Additionally, if there are multiple recursive @code{make} invocations
+running in parallel, they will communicate so that only one of them is
+generating output at a time.
+
+There are four levels of granularity when synchronizing output,
+specified by giving an argument to the option (e.g.,  @samp{-Ojob} or
+@samp{--output-sync=make}).
+
+@table @code
+@item none
+The is the default: all output is sent directly as it is generated and
+no synchronization is performed.
+
+@item job
+Output from each individual line of the recipe is grouped and printed
+as soon as that line is complete.  If a recipe consists of multiple
+lines, they may be interspersed with lines from other recipes.
+
+@item target
+Output from the entire recipe for each target is grouped and printed
+once the target is complete.  This is the default if the
+@code{--output-sync} or @code{-O} option is given with no argument.
+
+@item make
+Output from each recursive invocation of @code{make} is grouped and
+printed once the recursive invocation is complete.
+
+@end table
+
+Regardless of the mode chosen, the total build time will be the same.
+The only difference is in how the output appears.
+
+The @samp{make} mode provides the most comprehensive grouping,
+allowing output from all targets built by a given makefile to appear
+together.  However, there will be long interludes during the build
+where no output appears while a recursive @code{make} is running,
+followed by a burst of output.  This mode is best for builds being
+performed in the background, where the output will be examined later.
+
+The @samp{job} mode is mainly useful for front-ends that may be
+watching the output of @code{make} and looking for certain generated
+output to determine when recipes are started and completed.
+
+You should be aware that some programs may act differently when they
+determine they're writing output to a terminal versus a file
+(typically described as ``interactive'' vs. ``non-interactive''
+modes).  If your makefile invokes a program like this then using the
+output synchronization options will cause the program to believe it's
+running in ``non-interactive'' mode even when it's writing to the
+terminal.  Of course, invoking @code{make} with output redirected to a
+file will elicit the same behavior.
+
+@node Parallel Input,  , Parallel Output, Parallel
+@subsection Input During Parallel Execution
+@cindex input during parallel execution
+@cindex parallel execution, input during
+@cindex standard input
+
+Two processes cannot both take input from the same device at the same
+time.  To make sure that only one recipe tries to take input from the
+terminal at once, @code{make} will invalidate the standard input
+streams of all but one running recipe.  If another recipe attempts to
+read from standard input it will usually incur a fatal error (a
+@samp{Broken pipe} signal).
+@cindex broken pipe
+
+It is unpredictable which recipe will have a valid standard input stream
+(which will come from the terminal, or wherever you redirect the standard
+input of @code{make}).  The first recipe run will always get it first, and
+the first recipe started after that one finishes will get it next, and so
+on.
+
+We will change how this aspect of @code{make} works if we find a better
+alternative.  In the mean time, you should not rely on any recipe using
+standard input at all if you are using the parallel execution feature; but
+if you are not using this feature, then standard input works normally in
+all recipes.
+
 @node Errors, Interrupts, Parallel, Recipes
 @section Errors in Recipes
 @cindex errors (in recipes)
@@ -8628,16 +8700,19 @@ uninterrupted sequence.  This option is only useful when using the
 (@pxref{Parallel, ,Parallel Execution})  Without this option output
 will be displayed as it is generated by the recipes.@refill
 
-With no type or the type @samp{target}, output from each individual
-target is grouped together.  With the type @samp{make}, the output
-from an entire recursive make is grouped together.  The latter
-achieves better grouping of output from related jobs, but causes
-longer delay since messages do not appear until the entire recursive
-make has completed (this does not increase the total build time,
-though).  In general @samp{target} mode is useful when watching the
-output while make runs, and @samp{make} mode is useful when running a
-complex parallel build in the background and checking its output
-afterwards.
+With no type or the type @samp{target}, output from the entire recipe
+of each target is grouped together.  With the type @samp{job}, output
+from each job in the recipe is grouped together.  With the type
+@samp{make}, the output from an entire recursive make is grouped
+together.  The latter achieves better grouping of output from related
+jobs, but causes longer delay since messages do not appear until the
+entire recursive make has completed (this does not increase the total
+build time, though).  In general @samp{target} mode is useful when
+watching the output while make runs, and @samp{make} mode is useful
+when running a complex parallel build in the background and checking
+its output afterwards.  The @samp{job} mode may be helpful for tools
+which watch the output to determine when recipes have started or
+stopped.
 
 @item -q
 @cindex @code{-q}
@@ -10886,7 +10961,7 @@ will load the dynamic object @file{../mk_funcs.so}.  After the object
 is loaded, @code{make} will invoke the function @code{init_mk_func}.
 
 Regardless of how many times an object file appears in a @code{load}
-directive, it will only be loaded (and it's setup function will only
+directive, it will only be loaded (and its setup function will only
 be invoked) once.
 
 @vindex .LOADED