Add zuul debugging documentation

Try to provide some hints for debugging unexpected zuul behaviors in our
documentation.

Change-Id: I937df9d8d71ae4984991aa9f327dd1351d539284

doc/source/zuul.rst

@@ -228,6 +228,39 @@ YAML, there is no need to recombine split lines)::
 
 .. _zuul_github_projects:
 
+Debugging Problems
+------------------
+
+Occasionally you'll have a job enter an error state or an entire change that
+appears to be stuck in a Zuul pipeline. Debugging these problems can be a bit
+daunting to start as Zuul's logs are quite verbose. The good news is that once
+you learn a few tricks those verbose logs become quite the powerful tool.
+
+Often the best place to start is grepping the Zuul scheduler debug log for
+the pipeline entry identifier (eg change number, tag, or rev sha1)::
+
+  you@zuul02$ grep 123456 /var/log/zuul/debug.log
+  you@zuul02$ grep c6229660cda0af42ecd5afbe7fefdb51136a0436 /var/log/zuul/debug.log
+
+In many of these log lines you'll see Zuul event IDs like
+`[e: 1718628fe39643e1bd6a88a9a1477b4f]` this ID identifies the event that
+triggered Zuul to take action for these changes and is logged through all
+the Zuul services. It can be very powerful to do a grep on this event ID and
+trace through the actions that the scheduler took for this event::
+
+  you@zuul02$ grep 1718628fe39643e1bd6a88a9a1477b4f /var/log/zuul/debug.log
+
+This might lead you to look at executor logs where you can use the same
+ID to grep for actions related to this even on the executor::
+
+  you@ze01$ grep 1718628fe39643e1bd6a88a9a1477b4f /var/log/zuul/executor-debug.log
+
+As you trace through the logs related to a change or event ID you can look for
+`ERROR` or `Traceback` messages to try and identify the underlying source of
+the problem. Note that `Traceback` messages are not prefixed with the event
+ID which means you'll have to grep with additional context, for example using
+`grep -B20 -A20`.
+
 GitHub Projects
 ===============