In Depth: Give something back: why you should report software bugs

| September 29, 2012 | 0 Comments


In Depth: Give something back: why you should report software bugs

Reporting bugs: The anatomy of a bug

All software has problems. You may have found what you consider to be the most reliable software ever, but I promise you that buried in there are some problems.

We see this in every-day appliances, such as cars being recalled to have their onboard software updated, as well as the slew of updates and improvements we see in computer, laptop, tablet and mobile phone software.

Why does all software have problems? Because it’s written by humans. Software is a complex collection of moving parts, with many of the parts moving at different speeds and with different expectations. Even the best programmers write software with problems, but fortunately the dents in the armour of the programmer’s perfection can be resolved due to another feature of software: problems can be fixed.

In computer lingo, these problems are called bugs, and bugs are part and parcel of the software development process. Fortunately, living in the wonderful open source community that we are part of, every one of you can help find and report these bugs and help our programmers to make the software they share even better.

In the open source world, reporting bugs is a hugely important way in which you can contribute to the community. There is a misconception that reporting bugs is considered to be impolite or disrespectful in highlighting defects, or that it simply causes work for the developers. This is definitely not the case.

As a developer on some projects, I can assure you that our users are our eyes and ears for finding bugs and problems in our software. If we don’t have the bug reports, we don’t know about quality issues.

As such, contributing bug reports is a tremendously valuable exercise, and one that anyone can participate in: it doesn’t require a lot of technical experience, or any programming expertise.

In this article, we won’t be dwelling on a specific community, so everything here can apply to whatever group you’re interested in participating in.

Before we go on, understanding how to report a bug is helped by an appreciation of how software in the Linux community is put together. There are two broad types of community in the Linux world: Upstreams are the independent software projects that create the software you know and love. Examples of these include LibreOffice, OpenShot, Gimp, etc.

Downstreams are the Linux distributions that package and ship the upstream software in an integrated system. Examples include Ubuntu, Fedora and Debian.

The way in which software goes from the programmers’ computers to your Linux distribution works a little like this: The upstream programmers create the features in their upstream codebase (eg, producing a feature in OpenShot). The upstream programmers merge all of their features into a central codebase. Interested users in that upstream will test out the features as they appear in the codebase, and test those features and report bugs in them. The programmers then fix those bugs where they see fit.

When the upstream code is ready for release, they take a snapshot of that code and release it with a release number (eg, OpenShot 1.0). With a new upstream release, the distro developers create packages and release them into the development version of their distribution (eg, Ubuntu, Fedora). Interested users of the distribution test the release and report bugs in the distribution’s bug tracker.

The anatomy of a bug

The distribution’s developers resolve those issues where they see fit and fix them in the distribution (often contributing those fixes back to upstream). When the distribution is ready, it’s released with a release number (eg, Ubuntu 12.04).

Here we have two different types of bugs: Bugs that occur in the upstream project. Bugs that could be introduced in the packaging and integration of the software in the downstream distribution. In a perfect world, bugs discovered in a distribution are reported by its users, and if they are found to be bugs that weren’t introduced by the distribution’s developers, they are reported upstream.

See the Bug linking boxout for more information on how these different bugs can be connected.

Our goal here is to ensure that upstreams know about bugs in which the source is in the upstream software, and that distributions know about bugs that are introduced in the packaging and integration of that software.

Pastebin

Before we can report a bug, we need to know what a bug looks like. In a nutshell, a bug is behaviour in software that is unexpected or inaccurate. Examples of this could include:

You click on a button in a program and the program crashes (this should never happen).

You add two numbers together in a calculator and the wrong result is presented (the calculator should provide accurate results).

A translation is incorrect and shows a culturally incorrect term (the correct definitions should be used for translations).

Each of these examples presumes you know what the correct behaviour should be and you can therefore identify when incorrect behaviour happens. Sometimes, this is more obvious than at others.

You should always remember that bugs are only for unexpected or inaccurate results, not necessarily for design or behaviour decisions that you might not like. For example, if the developers decide to change the icon of an application to a horrible lime green colour, you might hate the look, but it’s not a bug. Don’t clog up their bug lists with disagreements about direction… this is what other communication channels are for.

All bugs have two important attributes: The symptoms for example, if you click a button in an application and it crashes, the symptoms are pretty stark… you click the button and the application crashes. Other bugs may have more subtle symptoms, such as behaviour gradually degrading, or stability issues occurring.

The cause If you click a button in an application and it crashes, what was the button you clicked on, and did it try to load functionality that contained an issue that caused the crash? Most people who find bugs discover the symptoms first and then dig in to the issue to discover the causes.

Reporting bugs: Gathering evidence

When we discover a bug in a piece of software, we want to be able to provide the developer with as much information as possible so that he or she can resolve it. The best form of evidence that we can provide is called reproducing a bug. This is where we provide a set of steps that the developer can follow to reliably trigger the bug on their own system. If they can trigger the bug, it provides a much higher chance of resolving the issue, as the developer can see the issue first-hand and be able to resolve it.

For some bugs, being able to reproduce it is fairly straightforward. All you need to do is try to remember the steps you went through before you saw the bug happen. If you retrace your steps and see the bug happen, you can provide these actions to the developer. If you find that you can retrace your steps to trigger the bug, be sure to try it a few times to ensure they are reliable.

Unfortunately, there are some situations in which it’s difficult to be able to reproduce the actions that caused the bug. In some bugs, it’s not as simple as clicking on a set of things in a certain order to trigger the issue; there may be a delicate balance of data, networking and state that when combined cause the issue to happen. If your computer seems to crash randomly it can be difficult nailing down why it crashes.

In these cases, where we either don’t know or are unable to reproduce the steps that trigger a bug, the next best solution is to provide evidence of when an issue has occurred to give our developers the best chance of identifying the bug. For example, if you’re seeing screen corruption in an application, or the output of a video playing application is blocky, you can provide some visual assets that can help with this, such as:

Screenshots: if an issue occurs that demonstrates a visual problem, take a screenshot and provide this as an example of the problem that you’re seeing. You can also use a variety of other applications to take screenshots, such as Gimp.

Screencasts: if the bug you’re experiencing is either difficult to describe or you want to show it in more detail, you can use a screencasting tool (such as RecordMy Desktop) to record the behaviour in action.

Log files: many applications store information about what they’re doing in logging files. This information is often useful to a developer, but knowing which log files are available requires more knowledge of the application in question. On some systems (eg, Ubuntu), finding the log files is automated when you report the bug.

Source material: if you’re finding that certain source material (eg, a video you want to play in a video player) is causing bugs, it can be useful to provide links to it (possibly with a file-sharing service such as Dropbox or Ubuntu One). Obviously, be sure you have the rights to share this information with others.

Another useful technique is to switch on an application’s debugging mode. Many applications come with a switch that you can turn on to have it spew lots of behind-the-scenes information.

Recently I found a bug in the Rhythmbox media player where it kept crashing. To help try to find the issue, I started it from a command line terminal and used the -d switch to turn on the debugging information. To do so, I typed this into a terminal: rhythmbox -d

The program then started, and I began using it as normal. Oodles of debugging information was listed in the terminal, and a few hours later when the crash happened I noticed that some of the debugging information was referring to the crossfading feature in Rhythmbox. I cut and pasted the debugging information into a text file and saved it, and then switched the cross-fading feature off and started using Rhythmbox as normal.

After about three days of using it with no issues, I concluded that the problem must be with the crossfading feature (as switching it off resulted in a far more stable experience). I then reported the bug, outlining my experience and sharing the debugging information I had gathered.

The process of gathering this evidence is one of the most worthwhile and useful contributions you can make in helping to find not only the symptoms of an issue but also the cause.

Starting conversation

Bugzilla

By now, you should have a collection of evidence, which could include a list of the symptoms, screenshots, screencasts, log files, source material and more. The next step is communicating this content to the developers so they can review it and resolve the issue.

While you will often hear of people reporting a bug in the Linux community, the description sounds as if this is a fire-and-forget process by which to write the bug down, and then the developers go away and fix it. In reality, this is less about writing a report and more about starting a conversation.

When you file a bug, you are initiating the first step in that conversation. It usually works a little bit like this: 1 You report the bug. 2 The developer will respond and ask for further details. 3 You provide the details. 4 Steps 2 and 3 may be repeated, as further information is required to nail down the source of the bug. 5 The developer fixes the bug.

While it’s important that we focus on doing step 1 well, it’s also important to go through with the follow-on discussion and correspondence to help the developer find the source of the issue. With the nature of this process being a conversation, we have a special place to have these kinds of conversations: a bug tracker.

Bug trackers are where we file bugs and hold the follow-up discussions to help the developer identify the causes of the bug. You can think of a bug tracker as a discussion forum infused with a series of different dials and switches for tracking the current status of the bug, its priority, whether it has been fixed, and who is working on it.

The vast majority of bug trackers are based on the web and have these status dials and switches at the top of the bug report, and the discussion of the issue below. In almost all open source projects, the bug trackers are open for the community to view and offer input on.

Examples of bug trackers include Bugzilla, Launchpad, Trac and others. Each of these different bug tracking systems provides the same basic formula for how a bug tracker works, but with various other features and capabilities that differentiate them. Different open source communities have different approaches to how they track their bugs.

Some projects set up their own dedicated bug tracker (often using Bugzilla), whereas other communities use an existing development hosting service (eg, Launchpad). There is no set pattern for how projects run, but fortunately once you have got the hang of one bug tracker, you should be able to use others pretty easily.

For a project that I’m working on at the moment, I’m using Launchpad for bug reporting, and you can see the bugs list here.

I’m not going to talk about a specific bug tracker, but instead discuss the general best practice of contributing useful bug reports. All of this information should be applicable to whatever bug tracker the community you’re interested in is using.

As I mentioned earlier, a bug report in a bug tracker includes some status information about the bug itself (as well as the discussion thread). Let’s now take a look at the most common status fields.

When you file a bug report, you will typically be asked for two important pieces of information:

Component: typically the application that you are filing the bug against (eg, Rhythmbox).

Summary: this is a single-line description of the issue. You can think of this in a similar way to a Subject line in an email: it should describe the issue in a succinct manner. Keep it clear and don’t try to emphasise the importance here (eg, don’t say: "Terrible bug in Rhythmbox, must be fixed!". Instead, say: "cross-fading causes intermittent crashes").

Description: this is a longer definition of the problem, and is similar to the main body of an email. Here, you should go into the detail of the bug and the symptoms, and you should share the thoughts you might have about how to reproduce the issue. The key here is to be detailed, precise and to the point.

Many bug trackers will ask you only for this information, but there are the other status fields I mentioned earlier that developers and other community members will use to provide further information about the bug.

These include: Priority: this is how important it is that the bug should be resolved. Typically, there are a series of options here (eg, Critical, High, Medium, Low) and the developer will choose the severity of the bug and apply a priority. A bug that causes data loss or severe crashes will likely be set to Critical, and one that causes a minor inconvenience will be set to Low.

Status: this is the current status of where the bug is in its life cycle. Common fields here include New (the bug has not had any attention yet), Confirmed (a developer has confirmed the bug exists), Incomplete (a developer has asked for further information that has not yet been provided), In Progress, Fix Committed (the fix has been committed to the code base), Fix Released, and Invalid (the issue reported is not considered a bug by the developer). Assigned To if a developer has volunteered to resolve the issue, they can assign themselves in this field so other developers know that the bug is being worked on.

Some bug reporters tend to get a little over-sensitive about the setting of the Priority and Status fields. Some people can get offended if a bug is reported and the Status is set to Invalid (as the developers do not consider the issue to be a bug).

Another example of this is when a bug reporter considers a bug to be of significant importance, and the Priority is set lower than the reporter desires. In these scenarios, always remember that the developers probably know best here. Priority is entirely relative. If there are 100 bug reports, they can’t all be considered Critical and High, so the developers need to ensure that out of the collection of bugs, the really important ones are set higher. However, this is not a reflection on the quality of your report, but, instead, on the importance of the issue.

More information

With your bug report filed, at some point one of the developers is likely to respond to discuss the bug in the report. The developer will likely ask you to provide more information, and maybe ask you to run some commands and provide the output. You should always try to respond to these requests for information, as it can improve significantly the chances of the bug getting fixed.

The developer may also ask you to run a new version of the software to test whether the bug was resolved. This varies from system to system, and has differing levels of complexity. An upstream developer may ask you to download the new version of the software, compile and run it. Alternatively, you may be asked to install a new package and test and run that.

You should make a judgement call about how comfortable you are running these new versions. If you don’t feel comfortable installing software on your computer (or compiling software), just let the developer know. You might still be able to help by asking someone else to test the version instead of you.

As this discussion progresses, you may see the message that the bug that you reported has been fixed. Congratulations! You just made a wonderful contribution to free software: if you had not taken the time to report the bug and help the developer identify it, it would not have been fixed. Great work!

Filing bugs, getting feedback on those bugs, and learning from that feedback helps you to contribute more effectively and help get more bugs nailed. Good luck!



View full post on Techradar – All the latest technology news

Filed Under: Tech News

Leave a Reply

You must be logged in to post a comment.

Get Adobe Flash player