Testing Open Source http://testingopensource.com Fri, 11 May 2007 18:37:23 +0000 http://wordpress.org/?v=2.5.1 en Bugzilla 3.0 http://testingopensource.com/bugzilla-30/ http://testingopensource.com/bugzilla-30/#comments Fri, 11 May 2007 18:37:23 +0000 Lloyd Budd http://testingopensource.com/bugzilla-30/ Bugzilla 3.0, nine years in the making? Much life left?

Bugzilla 3.0 was recently released. “It was almost 9 years ago when Bugzilla 2.0 was released”, but the reality is that there have been at least 10 releases with major functionality since 2.0.1 Bugzilla and its derivatives2 are the most popular open source bug tracker for large projects3 including GNOME, Linux, Mozilla projects, RedHat Linux, and Novell (SUSE) Linux.

Are you using Bugzilla as well? Why?

My friend Zbigniew Braniecki (gandalf) of my former employer Flock is a passionate Bugzilla developer and shares his enthusiasm about this release, but no details, other that he is working on the Polish translation.

What is the release announcement reporting as interesting about this release?

  • Custom Fields
  • mod_perl support for greatly-improved performance
  • Per-Product Permissions
  • XML-RPC Interface
  • Create and Modify Bugs by Email

Is there anything that excites you particularly about this release?

Dave Miller (justdave) shares what Bugzilla today looks like:

  • A templating system for any output going to end-users be that email or HTML pages, or XML, CSV, RSS.
  • The majority of the back-end code has been refactored into a comprehensive set of Perl modules.
  • Bugzilla works in Apache’s mod_perl.

What makes Bugzilla special to you? Is there people on the team that contribute to you enjoying working with Bugzilla?

I am a little disappointed to see that Mozilla is not running this version4. Eating their own food?

The Bugzilla 3.0 release announcement includes “which is made entirely by volunteers around the world” — isn’t Mozilla and other companies paying people to work on Bugzilla?

Is Bugzilla on a path to a slow death with Malone being the future, or Trac5 which is currently widely used for smaller projects?

Max Kanat-Alexander contemplates a rewrite of Bugzilla. There has been a lot of great feedback to Max’s post including from Bugzilla’s original author Terry Weissman and GNOME Bugzilla gardner Luis Villa (tieguy) who responded:

virtually every bugzilla installation is heavily customized, people have very different expectations about the bugzilla release/upgrade cycle than they do for consumer-facing, uncustomized software. Additionally, lets be frank- bugzilla has not exactly been blazing away with new features (in part because of the perl problem). Yes, it would suck to stop having new features altogether, but it isn’t like bugzilla is currently doing tons of new features and releasing quickly anyway. (Happy first birthday, 2.22. ;)

Personally, I would love to see Bugzilla retired as the user experience for most people in various roles on a software project has and is still awkward. Mozilla Marketer Paul Kim describes his first experience with Bugzilla as overwhelming and was grateful to his colleague (ironically) Mozilla Design Lead Mike Beltzer to guide him through it. “So many form fields. Arcana in spades (”Blocking 1.8.1.4″ - what did that mean?).”

But I am not sure that there is an open source bug tracker ready to take its place. What do you think?

Footnotes

  1. Why did it take Bugzilla 9 years to get from version 2.0 to 3.0?
  2. Almost all Bugzilla installations seem to be heavily customized. Is Collab.net’s Issue Tracker still based on Bugzilla?
  3. PHP and MySQL use PHP bug system, originally written by Rasmus Lerdorf. What are other large projects using?
  4. The version at bugs.mozilla.org listed as 2.23.4+. Maybe that is the same thing?
  5. My friend Jeff Lindsay offers free hosted Trac at DevjaVu
]]> http://testingopensource.com/bugzilla-30/feed/ A Survey on Current Practices in Defect Management in Free/Open Source Software http://testingopensource.com/a-survey-on-current-practices-in-defect-management-in-freeopen-source-software/ http://testingopensource.com/a-survey-on-current-practices-in-defect-management-in-freeopen-source-software/#comments Mon, 02 Apr 2007 17:55:18 +0000 Lloyd Budd http://testingopensource.com/a-survey-on-current-practices-in-defect-management-in-freeopen-source-software/ Anu Gupta and Professor R.K. Singla1 are running a survey on current practices in defect2 management in free/open source software . Participate, help them with this interesting research!

Here are my answers for WordPress. I am very interested in other WordPress participants answers and how it relates to other open source products.

2. What is the Type of Software Project?

Application software (web)

3. What is your Role in the Software Project?

Tester (though I wear many hats)

4. How many years of Experience do you have in the Software Industry?

More than 5 years

5. What is the estimated Lines of Code in the Software Project?

10,000 - 50,000 (29,853)3

6. How many Team Members are actively involved in the Software Project?

10 - 20 (Guessing that would be equivalent to ~full time contributors)

7. What is the estimated Number of Current Users of the Software Product?

More than 50,000 (that is the highest number choice, but I am glad I don’t have to try to further quatify)

8. What means are being used to collect Defect data for the Software Project?

Defect Tracking System

9. Which Defect Tracking System is being used for the Software Project?

Trac

10. When a Defect is recorded in the Defect database?

I check marked all that applied:

  • Change required in the software (not quite sure what this means)
  • Problem/Failure occurred in the software
  • Feature to be enhanced in the software
  • Usability problem faced in the software

11. What kinds of Defects are being handled in the Software Project (in context of Software Artifacts)?

  • Software Requirements Defects
  • Software Design Defects
  • Source Code Defects

(Occasionally there are defects filed verses the Documentation, and the website.)
(I look forward to maturing the testing so “Software Testing Defects” are also included.)

12. What kinds of Defects are being handled in the Software Project (in context of Product Release)?

Both Pre-release and Post-release Defects with clear distinction

13. How frequently the Software Product is being released?

Yearly (though moving to biyearly)

14. What is the average % of Changes in the Software Product from release to release (major releases)?

Less than 10% (Wild guess. Don’t find it that interesting; what does it really mean.)

15. How consistently Defects are being recorded in the Defect database for the Software Project?

Almost consistently

16. A typical Defect Report includes several fields of information. How often the Defect Reports are complete?

Sometimes complete

17. How quickly Defects are being resolved?

Critical Defects 1-2 Days

Major Defects 6-10 Days

Minor Defects More than 30 Days

18. How would you rate the overall contribution of users towards the Project Development, Maintenance and Enhancement?

Defect/Bug Identification Very Good
Patch Submission Good
Feature Request Fair

19. What is the average % of User Reported Defects that remain unresolved?

31% - 50% (Wild guess. Would have like clear definition of “unresolved”)

20. What are the major reasons for unresolved Defects?

(Many combinations of web browsers, web servers, and their versions)
(Broad, diverse user base)

I check marked all that applied (all of them!):

  • Unreproducible / requires more information (Difference between these two?)
  • Resolution Postponed (Many minor bugs; pushed out to later milestones while more severe problems are addressed)
  • Invalid (Problem caused by browser, web server or WordPress plugin)
  • Other: Won’t Fix (Plugin territory; niche enhancement request)

21. What measures are being taken by development team for Defect Prevention in Software Project?

Extensive Testing (Many members of the community run the development trunk and we deploy regularly to WordPress.com)

22. Does Version Control System being updated appropriately to indicate code change corresponding to a particular Defect in the Defect Database?

Yes (I think it is asking if the ticket number is included in Subversion check ins and the ticket is updated with that information. Not quite sure that is really what was asked.)

23. What are Major Challenges in the Defect Management Process?

I check marked all that applied:

  • Lack of sufficient Manpower for Testing and Defect fixing (Sexist language)
  • Other: Lack of Testing and Defect Reporting Expertise in the community (Though continuously improving!)

      • 24. How could Defect Management Process be improved in your opinion?

      I check marked all that applied:

      • Preventing Defects by Reviews and Testing Techniques (Security audits, test automation, basic shared results testing using tools like Mozilla Litmus)
      • Guided bug submission (have to find time to do this)

      Now it is your turn. Go fill out the survey!

      Footnotes

      1. Department of Computer Science and Applications, Panjab University, Chandigarh
      2. Enhancement management? I don’t like separating defects and enhancements when discussing at this high level
      3. aptitude install sloccount
        It looks like David A. Wheeler’s SLOCCount does not know JavaScript, so the estimate is probably low. Results:
        Totals grouped by language (dominant language first):
        php: 29853 (100.00%)
        Total Physical Source Lines of Code (SLOC) = 29,853
        Development Effort Estimate, Person-Years (Person-Months) = 7.08 (84.91)
        (Basic COCOMO model, Person-Months = 2.4 * [KSLOC**1.05] )
        Schedule Estimate, Years (Months) = 1.13 (13.52)
        (Basic COCOMO model, Months = 2.5 * [person-months**0.38])
        Estimated Average Number of Developers (Effort/Schedule) = 6.28
        Total Estimated Cost to Develop = $ 955,826
        (average salary = $56,286/year, overhead = 2.40).
      ]]> http://testingopensource.com/a-survey-on-current-practices-in-defect-management-in-freeopen-source-software/feed/ Continuous Integration and Testing Conference and Tools http://testingopensource.com/continuous-integration-and-testing-conference-and-tools/ http://testingopensource.com/continuous-integration-and-testing-conference-and-tools/#comments Wed, 28 Mar 2007 19:15:50 +0000 Lloyd Budd http://testingopensource.com/continuous-integration-and-testing-conference-and-tools/ Software automation professional and open source testing tool Watir contributor Bret Pettichord writes:

      Continuous Integration and Testing Conference in Dallas Fort Worth on Friday April 27 and Saturday April 28. Continuous Integration is the term of art for the practice of automatically building and testing your software whenever the code is updated. Cruise Control is the commonly used tool that manages continuous integration.

      …

      Continuous integration thus refers both to this technology as well as the evelopment philosophy it supports: encouraging automation, stability,
      frequent commits and collective code ownership.

      Cruise Control is also open source.

      The Continuous Integration and Testing Conference conference is also free! Too bad for me, it is on the other side of the continent

      ]]>       http://testingopensource.com/continuous-integration-and-testing-conference-and-tools/feed/       Mozilla Community Testers Sift Feedback http://testingopensource.com/mozilla-community-testers-sift-feedback/ http://testingopensource.com/mozilla-community-testers-sift-feedback/#comments Tue, 20 Mar 2007 21:20:26 +0000 Lloyd Budd http://testingopensource.com/mozilla-community-testers-sift-feedback/ Today, and next week’s Mozilla Community Bug Day are being spend sifting through feedback (Hendrix). Not support requests nor reported bugs, but “quick feedback on and suggestions for [Mozilla's] products”.

      This activity seems to have been suggested by Mozilla QA member Jay Patel, and is probably necessary because Hendrix sometimes is recommended for submitting bug reports.

      It will be interesting to hear what the results of this activity are. I imagine a lot of bugs get reported there, but will likely be very expensive work — a lot of sifting through incomplete reports and other topics. Hendrix seems like a poor channel to the products and projects.

      Don’t get me wrong, easy submission of all type of feedback is essential, and no-response-expected is appropriate for feature / functionality requests, but problem reporting should almost exclusively come go through two channels:

      • Support
      • Bug Tracking, with support also feeding in.

      These two channels allow follow up and participation. They encourage collaboration!

      I was excited by Basil Hashem’s announcement of the Firefox Community Beta Program, but disappointed when I saw that the suggested method of feedback was “at our community feedback web site [Hendrix]“.

      ]]>       http://testingopensource.com/mozilla-community-testers-sift-feedback/feed/       Jokosher: Call For Testers http://testingopensource.com/jokosher-call-for-testers/ http://testingopensource.com/jokosher-call-for-testers/#comments Fri, 09 Feb 2007 22:38:02 +0000 Lloyd Budd http://testingopensource.com/jokosher-call-for-testers/ Jono Bacon is recruiting test volunteers for Jokosher.

      Jokosher: is a GTK Python desktop audio creation environment.

      This seems like a good volunteer testing opportunity:

      • Product looks cool and useful.
      • Enthusiastic about volunteer testers: “really, really” and “serious here”.
      • Early on in product’s life cycle so opportunity for novice to expert testers. In fact, novices are invited: “is a pretty simple process and requires no programming knowledge”.
      • The team is focused on a high qaulity product: “feature freeze” and “Without lots of testing, Jokosher will simply not be stable enough.”
      • Get to work with Jono Bacon.

      “Now is the time, test it, report it, discuss it!“

      ]]>       http://testingopensource.com/jokosher-call-for-testers/feed/       First Post http://testingopensource.com/first-post/ http://testingopensource.com/first-post/#comments Wed, 07 Feb 2007 22:44:15 +0000 Lloyd Budd http://testingopensource.com/?p=3 I hope Testing Open Source will develop into a collaborative, volunteer run resource for testers of open source software.

      If you would like to contribute please leave a comment here. Some ideas on articles to contribute:

      • Announcements of new OSS testing tools or major releases
      • Discussion of OSS testing tools and processes
      • Comparison of testing tools
      • Announcements of OSS testing events: social, training, or significant volunteer opportunities
      • Announcement of OSS Quality Assurance (QA) jobs
      ]]>       http://testingopensource.com/first-post/feed/