A Survey on Current Practices in Defect Management in Free/Open Source Software
Anu Gupta and Professor R.K. Singla1 are running a survey on current practices in defect2 management in free/open source software . Participate, help them with this interesting research!
Here are my answers for WordPress. I am very interested in other WordPress participants answers and how it relates to other open source products.
2. What is the Type of Software Project?
Application software (web)
3. What is your Role in the Software Project?
Tester (though I wear many hats)
4. How many years of Experience do you have in the Software Industry?
More than 5 years
5. What is the estimated Lines of Code in the Software Project?
10,000 - 50,000 (29,853)3
6. How many Team Members are actively involved in the Software Project?
10 - 20 (Guessing that would be equivalent to ~full time contributors)
7. What is the estimated Number of Current Users of the Software Product?
More than 50,000 (that is the highest number choice, but I am glad I don’t have to try to further quatify)
8. What means are being used to collect Defect data for the Software Project?
Defect Tracking System
9. Which Defect Tracking System is being used for the Software Project?
10. When a Defect is recorded in the Defect database?
I check marked all that applied:
- Change required in the software (not quite sure what this means)
- Problem/Failure occurred in the software
- Feature to be enhanced in the software
- Usability problem faced in the software
11. What kinds of Defects are being handled in the Software Project (in context of Software Artifacts)?
- Software Requirements Defects
- Software Design Defects
- Source Code Defects
(Occasionally there are defects filed verses the Documentation, and the website.)
(I look forward to maturing the testing so “Software Testing Defects” are also included.)
12. What kinds of Defects are being handled in the Software Project (in context of Product Release)?
Both Pre-release and Post-release Defects with clear distinction
13. How frequently the Software Product is being released?
Yearly (though moving to biyearly)
14. What is the average % of Changes in the Software Product from release to release (major releases)?
Less than 10% (Wild guess. Don’t find it that interesting; what does it really mean.)
15. How consistently Defects are being recorded in the Defect database for the Software Project?
Almost consistently
16. A typical Defect Report includes several fields of information. How often the Defect Reports are complete?
Sometimes complete
17. How quickly Defects are being resolved?
Critical Defects 1-2 Days
Major Defects 6-10 Days
Minor Defects More than 30 Days
18. How would you rate the overall contribution of users towards the Project Development, Maintenance and Enhancement?
Defect/Bug Identification Very Good
Patch Submission Good
Feature Request Fair
19. What is the average % of User Reported Defects that remain unresolved?
31% - 50% (Wild guess. Would have like clear definition of “unresolved”)
20. What are the major reasons for unresolved Defects?
(Many combinations of web browsers, web servers, and their versions)
(Broad, diverse user base)
I check marked all that applied (all of them!):
- Unreproducible / requires more information (Difference between these two?)
- Resolution Postponed (Many minor bugs; pushed out to later milestones while more severe problems are addressed)
- Invalid (Problem caused by browser, web server or WordPress plugin)
- Other: Won’t Fix (Plugin territory; niche enhancement request)
21. What measures are being taken by development team for Defect Prevention in Software Project?
Extensive Testing (Many members of the community run the development trunk and we deploy regularly to WordPress.com)
22. Does Version Control System being updated appropriately to indicate code change corresponding to a particular Defect in the Defect Database?
Yes (I think it is asking if the ticket number is included in Subversion check ins and the ticket is updated with that information. Not quite sure that is really what was asked.)
23. What are Major Challenges in the Defect Management Process?
I check marked all that applied:
- Lack of sufficient Manpower for Testing and Defect fixing (Sexist language)
- Other: Lack of Testing and Defect Reporting Expertise in the community (Though continuously improving!)
- Preventing Defects by Reviews and Testing Techniques (Security audits, test automation, basic shared results testing using tools like Mozilla Litmus)
- Guided bug submission (have to find time to do this)
- Department of Computer Science and Applications, Panjab University, Chandigarh [↩]
- Enhancement management? I don’t like separating defects and enhancements when discussing at this high level [↩]
aptitude install sloccount
It looks like David A. Wheeler’s SLOCCount does not know JavaScript, so the estimate is probably low. Results:
Totals grouped by language (dominant language first):
php: 29853 (100.00%)
Total Physical Source Lines of Code (SLOC) = 29,853
Development Effort Estimate, Person-Years (Person-Months) = 7.08 (84.91)
(Basic COCOMO model, Person-Months = 2.4 * [KSLOC**1.05] )
Schedule Estimate, Years (Months) = 1.13 (13.52)
(Basic COCOMO model, Months = 2.5 * [person-months**0.38])
Estimated Average Number of Developers (Effort/Schedule) = 6.28
Total Estimated Cost to Develop = $ 955,826
(average salary = $56,286/year, overhead = 2.40). [↩]
24. How could Defect Management Process be improved in your opinion?
I check marked all that applied:
Now it is your turn. Go fill out the survey!
Footnotes
I think putting the number of team members so low is a slap to the other people who have either contributed in the past, contribute in other ways besides pure code, or contribute by offering plugins that futher the popularity of the project. Perhaps I’ve misread the survey, and core functionality is the only thing being queried, but nonetheless, WP wouldn’t be what it is if only 10-20 people were “team members”.