The week three readings in Managing System Development course were articles on different software engineering methodologies.
Brooks Jr. F. P. (1987) No Silver Bullet Essence and Accidents of Software Engineering, Computer, 20, 10-19
An interesting analogy between Software Projects and Werewolves and based on personal experience I can vouch that projects can turn overnight into monsters difficult to control and manage. A software project like Werewolves can destroy individuals, team’s even organisations. Once the things start going bad then heads start rolling starting from Project Managers, Configuration Managers & development leads in that order.
But to expect a Silver Bullet to kill the uncertainty & complexity is expecting moon. As Brooks rightly mentions this ambiguity & density is inherent in software engineering realm. “No other technology since civilization began has seen six orders of magnitude in performance-price gain in 30 years”
Brooks went philosophical when he invoked Aristotle to divide difficulties inherent into software realm into essences and production difficulties into accidents. But I don’t believe philosophy is going to help us here. The difficulties inherent in the nature of software as well as during production of software are varied in nature and collating these into any single philosophical term is to over simplify the problem. In software development we are trying to describe a bit of our own intelligence, personality traits & inter human dynamics which is not easy.
I liked Brook’s division of essential difficulties in nature of software into
1) Complexity
2) Conformity
3) Changeability
4) Invisibility
And agree with him that the past breakthroughs have partially solved some glaring issues. For example High-Level Languages, Time Sharing & Unified Programming Environments have to an extent given us perhaps a Silver Pellet if not a Bullet per se.
As the article is quite old so the future hopes on the horizon at that time have since come and gone.
1) Ada
2) Object-Oriented Languages
3) Artificial Intelligence
4) Expert Systems
5) Automatic Programming
6) Graphical programming
Most of these programming constructs have since been used extensively and retrospectively most of these constructs have helped in the advancement of Software Engineering.
Most effective statement in the article comes in the Incremental Development section which is grow, don’t build software. I agree the industry has evolved from writing software to building software and the next step in evolution should be to grow software.
Racoon, L. B. A. (1995) The chaos model and the chaos cycle. ACM Sigsoft Software Engineering Notes, 20, 12
Racoon_TheChaosModelAndChaosLifecycle_1995
This is the most insightful of the articles I’ve read on software engineering and I totally concur that the developer’s viewpoint has been overlooked in traditional models.
The software engineering is complex in nature and Chaos model “which combines a simple, people oriented, problem-solving loop with fractals” is a good fit in that. Racoon’s analysis on how Chaos model “improves or understanding of the contribution and limitations of users, developers, and technologies” is enlightening. And it does makes sense when Chaos Model is used to “define the phases of the life cycle in terms of fractals and show that all phases occur throughout the life cycle”.
Now when I look back and analyse poorly executed projects using linear Problem-Solving loop, the failings can be derived from the Problem Definition phase. Most of the time all resources & time is spent in solving wrong problems. The acceptance of the fact that project life cycles are not linear but fractal in nature can definitely allow us to approach the problems from different viewpoint. The traditional software techniques do not agree on the fact that all levels of software development have the same value to the project as a whole. And it is part of the problem a uniform focus instead of focus on few large issues by top management and running down from that order results in skewed projects.
I totally agree that software development is a Human activity and enough checkpoints should be put in place to ensure
1) Developers don’t mistakenly identify the problem or ignore the problem definition
2) Wrong technology is not used or right technology is not misused
3) Miscommunication is avoided in the project team
Fractal phase definitions can be used as an alternative method during project management to get a different viewpoint.
1) Requirement Analysis
2) Design
3) Implementation
4) Maintenance
5) Prototyping
But it is to be used with a caveat that it is long winding method and needs strong PMO to ensure that the project does not get trapped in a fractal mess.
Laurie Williams, Gabe Brown, Adam Meltzer, Nachiappan Nagappan, (2010) Scrum + Engineering Practices: Experiences of Three Microsoft Teams
WilliamsEtAl_SCRUMExperience_2010
It is a good read on Scrum Methodology which is an agile software development process that works as a project management wrapper around existing engineering practices to iteratively and incrementally develop software.
The authors have tried to summarised their findings with three different Microsoft teams which implemented Scrum Methodology to execute projects. The teams are varied in size and nature and the projects also have a a number of different variables. To the uninitiated Scrum is composed of the following project management practices:
• The Product Owner creates the requirements, prioritizes them, and documents them in the Product Backlog during Release Planning. In Scrum, requirements are called features.
• Scrum teams work in short iterations. When Scrum was first defined [16, 29], iterations were 30-days long. More recently Scrum teams often use even shorter iterations, such as two-week iterations. In Scrum, the current iteration is called the Sprint.
• A Sprint Planning Meeting is held with the development team, testers, management, the project manager, and the Product Owner. In the Sprint Planning Meeting, this group chooses which features (which are most often user-visible, user valued, and able to be implemented within one iteration) from the product backlog are to be included in the next iteration, driven by highest business value and risk and the capacity of the team.
• Once the Sprint begins, features cannot be added to the Sprint.
• Short, 10-15 minute Daily Scrum meetings are held. While others (such as managers) may attend these meetings, only the developers and testers and the Scrum Master (the name given to the project manager in Scrum) can speak. Each team member answers the
following questions:
o What have you done since the last Daily Scrum?
o What will you do between now and the next Daily Scrum?
o What is getting in your way of doing work?
• At the end of a Sprint, a Sprint Review takes place to review progress and to demonstrate completed features to the Product Owner, management, users, and the team
members.
• After the Sprint Review, the team conducts a Retrospective Meeting. In the retrospective Meeting, the team discusses what went well in the last Sprint and how they might improve their processes for the next Sprint.
The Scrum based process used by teams was as following
1) Basic Scrum: All teams began with four week iteration of the Scrum Process as described above. The teams performed “just-in-time” design of features before or during the iteration in which feature was to be developed.
2) Planning Poker: The teams used Planning Poker to estimate the person hours required to complete functionality within an iteration.
3) Continuous Integration: The teams utilised the continuous integration practice where members of a team integrate their work into the main build frequently.
4) Unit Test-Driven Development: With this practice, a software engineer cycles on a minute-by-minute basis between writing failing automated unit tests and writing implementation code to pass those tests.
5) Quality Gates: The Microsoft teams call their done criteria “quality gates”. The quality gates established for these teams included the following:
• All unit tests must pass
• Unit test code coverage must be at least 80% (for
all teams except Team B)
• All public methods must have documentation
• All non-unit test code must not have any static
analysis errors or warnings (see Sub-Section 9 of
this section)
• Build must compile with no errors or warnings on
the highest level
6) Source Control: Source control is management of changes to documents, programs, and other information stored as computer files through a source control system. The Microsoft teams used the Visual Studio Team Foundation Server Version Control tool.
7) Code Coverage: Engineers were required to manage their automated unit test coverage and monitored this coverage with each build. Two of the teams (A and C) followed the Microsoft Engineering Excellence recommendation of having 80% unit test coverage.
8) Peer Review: In each iteration, the teams conducted design reviews of architecture diagrams and of code when adding new features.
9) Static Analysis Tool: The use of static analysis tools can identify common
coding problems [17] or unusual code [1] early in the development process [9]. The teams utilized the FxCop static analysis tool built into Visual Studio.
10) XML Documentation: The team used .NET-style inline XML generated documentation on all public classes, properties, and methods. As a result the code was self-documenting.
The findings of the study was that the productivity of the teams as they transitioned to
agile temporarily dropped for three iterations. The team attributed this drop to their unfamiliarity of Scrum and required a “gelling” period to start delivering value based on
the development process. From their fourth sprint on they experienced a significant improvement in productivity without an increase in defects. Teams transitioning to the
use of an agile software development should plan for a similar temporary productivity decrease.
Teams that used Scrum and sound engineering practices showed better quality in terms of defect density compared with similar non-Scrum teams including data benchmarked across 40 projects from nine companies. These results indicates that Scrum combined with sound engineering practices have the potential to yield a higher quality product. Team B that followed Scrum but the engineering practices to a lesser degree than Teams A and C had the highest defect density.
Tarun's Blog 