Monthly Archives: June 2014

On bug reporting. . .

[This has been sitting my drafts for a while, time to hit send. Long time ago I wrote something like this to a Red Hat internal mailing list, thought I’ll put it out here too.]

A week ago, I was participating in one of the regular upstream OpenStack bug triage sessions (for Nova project) to keep up with the insane flow of bugs. All open source projects are grateful for reported bugs. But more often than not, a good chunk of bugs in such large projects end up being bereft of details.

Writing a good bug report is hard. Writing a coherent, reproducible bug report is much harder (and takes more time). Especially when involved in complex cloud projects like OpentStack where you may have to care about everything from Kernel to user space.

A few things to consider including in a bug report. The following are all dead obvious points (and are usually part of a high-level template in issue trackers like Bugzilla). That said, however many times you repeat, it’s never enough:

  • Summary. A concise summary — is it a bug? an RFE? a tracker-bug?
  • Description. A clear description of the issue and its symptoms in chronological order.
  • Version details. e.g. Havana? IceHouse?, hypervisor details; API versions, anything else that’s relevant.
  • Crystal clear details to reproduce the bug. Bonus points for a reproducer script.
  • Test environment details. (This is crucial.)
    • Most of “cloud” software testing is dearly dependent on test environment. Clearer the details, fewer the round-trips between developers, test engineers, packagers, triagers, etc.
    • Note down if any special hardware is needed for testing, e.g. an exotic NAS, etc.
    • If you altered anything — config files, especially code located in /usr/lib/pythonx.x/site-packages/, or any other dependent lower-layer project configurations (e.g. libvirt’s config file), please say so.
  • Actual results. Post the exact, unedited details of what happens when the bug in question is triggered.
  • Expected results. Describe what is the desired behavior.
  • Additional investigative details (where appropriate).
    • If you’ve done a lot of digging into an issue, writing a detailed summary (even better: a blog post) while its fresh in memory is very useful. Along with addition info like – configuration settings, caveats, relevant log fragment, _stderr_ of a script, or a command being executed, adding trace details — all of which would be useful for archival purposes (and years later, the context would come in very handy)

Why?

  • Useful for new test engineers who do not have all the context of a bug.
  • Useful for documentation writers to help them write correct errata text/release notes.
  • Useful for non-technical folks reading the bugs/RFEs. Clear information saves a heck of a lot of time.
  • Useful for folks like product and program managers who’re always not in the trenches.
  • Useful for downstream support organizations.
  • Should there be a regression years later, having all the info to test/reproduce in the bug, right there makes your day!
  • Reduces needless round-trips of NEEDINFO.
  • Useful for new users referring to these bugs in a different context.

Overall, a very fine bug report reading experience.

You get the drift!

4 Comments

Filed under Uncategorized

Upcoming: OpenStack workshop at Fedora Flock conference, Prague (Aug 6-9, 2014)

UPDATE: I’ll not be able to make it to Prague due to plenty of other things going on. However, Jakub Ružička, an OpenStack developer and RDO project packager kindly agreed (thank you!) to step in to do the talk/workshop.

Fedora Project’s yearly flagship conference Flock is in Prauge (August 6-9, 2014) this time. A proposal I submitted for an OpenStack workshop is accepted. The aim workshop is to do a live setup of a minimal, 2-node OpenStack environment using virtual machines (also involves KVM-based nested virtualization). This workshop also aims to help understand some of the OpenStack Neutron networking components and related debugging techniques.

Slightly more verbose details are in the abstract. An etherpad for capturing notes/planning and remote participation.

Leave a comment

Filed under Uncategorized