diff --git a/doc/source/zuul.rst b/doc/source/zuul.rst index 643360d49c..08d2a63c2e 100644 --- a/doc/source/zuul.rst +++ b/doc/source/zuul.rst @@ -228,6 +228,39 @@ YAML, there is no need to recombine split lines):: .. _zuul_github_projects: +Debugging Problems +------------------ + +Occasionally you'll have a job enter an error state or an entire change that +appears to be stuck in a Zuul pipeline. Debugging these problems can be a bit +daunting to start as Zuul's logs are quite verbose. The good news is that once +you learn a few tricks those verbose logs become quite the powerful tool. + +Often the best place to start is grepping the Zuul scheduler debug log for +the pipeline entry identifier (eg change number, tag, or rev sha1):: + + you@zuul02$ grep 123456 /var/log/zuul/debug.log + you@zuul02$ grep c6229660cda0af42ecd5afbe7fefdb51136a0436 /var/log/zuul/debug.log + +In many of these log lines you'll see Zuul event IDs like +`[e: 1718628fe39643e1bd6a88a9a1477b4f]` this ID identifies the event that +triggered Zuul to take action for these changes and is logged through all +the Zuul services. It can be very powerful to do a grep on this event ID and +trace through the actions that the scheduler took for this event:: + + you@zuul02$ grep 1718628fe39643e1bd6a88a9a1477b4f /var/log/zuul/debug.log + +This might lead you to look at executor logs where you can use the same +ID to grep for actions related to this even on the executor:: + + you@ze01$ grep 1718628fe39643e1bd6a88a9a1477b4f /var/log/zuul/executor-debug.log + +As you trace through the logs related to a change or event ID you can look for +`ERROR` or `Traceback` messages to try and identify the underlying source of +the problem. Note that `Traceback` messages are not prefixed with the event +ID which means you'll have to grep with additional context, for example using +`grep -B20 -A20`. + GitHub Projects ===============