Why you might need to design your own Hybrid Project Management Method.
Table of contents
The importance of long term planning
Phased scrum — a possible solution for most Software Projects
Preface
Nowadays, it seems that projects can only distinguish between two methodologies. Either you work “agile” or you do not. If questions are asked about the process model, it is frequently argued that “agile” or “not agile” is used and therefore things are done the way they are done. In short, it appears that there are only two exclusive types of project management in software development— “agile” (often Scrum) and the contrast “not agile” (often waterfall or V-model). Let’s see why this is not true and why it is more important to understand the methodologies in order to set up project-specific management.
At the end, I will outline a proposed solution that should fit most software projects: Phased Scrum.
Agile Software Development
Even before the COVID-19 pandemic, agile software development was a hot topic. Numerous concepts based on agility have emerged. The most famous are extreme programming, Scrum, Scrumban, and (Biz)DevOps.
But, what does agile software development really mean?
What is Agile Software Development?
Agile was popularized by the 2001 Manifesto for Agile Software Development. If we read the manifesto carefully and categorize the content, we can identify the following main points:
- Find out what to solve or serve.
- Find out what to build.
- Build the right solution by delivering value all the time.
- Organize yourself to apply points one, two, and three continuously.
- Some actual development content.
It is noticeable that these principles do not focus on the actual development activity. These principles rather describe the steps before coding. It is about constantly finding out what to solve or what to serve and, consequently, what needs to be developed. Therefore, it can be deduced that agile development is mainly about discovery as it is about the actual development.
Why is the Focus on Discovery?
The reason is tangible if we look at the waterfall model. The waterfall model is a linear (not iterative) process model. Its structure is based on predefined phases that have to be passed through in a binding order. In practice, the model is extended to allow some jumps back into previous phases. However, those jumps should be minimized and are not part of the actual concept. The basic idea behind this concept is that if everything is planned precisely, nothing should go wrong. The waterfall model looks like this:
As Mohandas K. Gandhi would say: “I think it would be a good idea”. He means that theory and practice can be worlds apart. Assuming optimal planning capability, this method would work. Yet, in life this condition is often not given. Complexity, misunderstandings, or unplanned events are likely to destroy this linear and binding development process. Well, and those events frequently occur in software projects. Furthermore, mistakes propagate for too long and lead to sunk costs or project failure. In addition, silo-thinking is strongly encouraged by the encapsulation of different areas of responsibility (departments), which results from the strict phase separation. Let’s visualize the waterfall model in a different and more practice-oriented way.
It can be seen that this model lacks interactions and collaborations between the different parties that work together to develop a solution or product during the different phases. This is exactly where agility comes into play. To counteract misunderstandings, complexity, and unplanned events, small task packages are needed that run through all relevant areas and parties at short intervals. All relevant stakeholders, therefore, participate constantly and actively in the project and should see themselves as one team with direct feedback loops. With this approach, it should be possible to react quickly to obstacles of any kind. Like this, you build the right solution and deliver value all the time in a complex world where misunderstandings and misconceptions are constantly present. Now we have reasons why the focus of pure agile software development is on discovery.
Extremistan
As the criticism of the waterfall model has shown, agility is often needed in software projects. It should also not be concealed that companies and customers often do not know in detail, or mistakenly believe they know, what they actually want. Customer opinions may be inaccurate or incomplete. Furthermore, there may also be a lack of cognitive or methodical competence in the project, where agility provides a clear remedy.
These points lead us to pure (extreme) agility. If no one knows at the beginning what is needed or how to get to the finished product (solution), there can be no or little planning. In this case, the requirements or development activities have to be discovered during development. In other words, pure agile software development is what we do when we do not know what we are doing.
The following video illustrates the pure agility very well by focusing on discovery:
Each model, whether agile or classic, has different use cases where they shine. Waterfall works for projects that are more like a repetitive process than a project. The repetitive construction of family homes is a good use case for waterfall project management. Pure agile is the solution to realize something which is extremely complex, or difficult to define at the beginning. Basically, these two use cases are two extreme points. However, most projects fall between these extreme points.
As mentioned in the preface, I have the feeling that some people think that there is only “agile” and the contrast “not agile” — as if there were only two possible choices, like having a Boolean attribute. The problem with this pigeonhole thinking is that, depending on the method, either everything or very little is planned. I have seen software projects that follow the Scrum model and refuse to do any long-term planning. It is often argued that long-term planning does not fit into the agile way of working and they do not want to develop classically. In this case, often the next sprint is planned in detail and the further next 2–3 sprints are roughly drafted by the order of the product backlog items. Nevertheless, most companies that have argued this way would have benefited from long-term planning.
The Importance of long-term Planning
Dwight Eisenhower said: “Planning is everything, the plan is nothing.” He means that plans are often out of date by the moment they are created. If a plan loses its validity so quickly, why plan at all? Eisenhower means that the value in planning is in the understanding and the thought processes that arise among people that form the plan. Exactly that deep understanding allows them to react quickly and flexibly to changes while focusing on a specific goal. At this point, it must also be mentioned that planning is not a one-time event. It is a process that starts all over again when enough new information is available. Basically, this planning process enables participants to respond in an agile way.
The more you work according to plan, the more you get what was planned, but not what is needed. It becomes clear that this criticism of classic project management does not mean that long-term planning is not helpful or unnecessary for the project’s success. To forego long-term planning to be able to act agilely is simply nonsense. In short, if you can plan something, you should do it, but you do not have to stick to the plan when new information leads to a better solution.
If the solution really cannot be estimated or planned, it requires pure agile software development. Nevertheless, in practice, very few projects are like this. Therefore, the question must be asked why most not pure agile IT projects do not have detailed planning at the beginning. Of course, there is the pigeonhole thinking and the blind following of guidelines like Scrum etc., which hinders detailed planning at the start of a project. Nevertheless, the advantage of thoughtful planning is so obvious that this cannot actually be the reason for its neglect. Sometimes I wonder if pure agile software development is just a workaround for other problems at most companies. Since I have already mentioned Mr. Eisenhauer, I would like to give a short example of such a workaround using the Eisenhower Decision Matrix.
The Eisenhower Decision Matrix differentiates between important and urgent tasks. What is urgent is seldom important, vice versa. Furthermore, overtime is found in most companies. This indicates that there is more to be done than can be achieved with regular working hours. Therefore, the implication can be made that the focus for most companies is on urgent tasks, like daily activities, deadlines, or unplanned work, which leads to overtime. In addition, it is normal for people to care less about the future than about the present. Consequently, little time can be found in most companies for important and non-urgent tasks, like real planning.
When there is a lack of time for thoughtful planning, pure agile software development is a suitable workaround, but not the best solution.
However, this is only one example of how pure agile software development can be used as a workaround for a very different problem. The misunderstanding of the actual purpose of planning by the management is another possible and often encountered reason. If the plan should always be followed and changes to the plan are difficult to enforce, it is better not to create a plan in order to avoid exhausting arguing. In short, if the real added value of planning cannot be lived, there is no need to plan.
Phased Scrum — A possible Solution for most Software Projects
To create a practical and appropriate methodologyfor the software development process, we need to combine the advantages of different approaches. The more process models we know and understand, the better a suitable project management method can be found for the individual and unique project. If this theoretical knowledge is combined with sufficient practical experience, real project management can result. The final ingredient is the understanding of the framework against which the project management must be aligned. Just as each person has to be dealt with individually, each corporate culture has to be dealt with differently.
Let’s assume we have an environment where the values of the servant leadership are lived, the people involved in the project have sufficient experience, and the goals and functionalities of the desired software solution are largely known. How should this project be managed?
To make this project a success, I would use a method which I call Phased Scrum, combined with management by objectives techniques. In this method, there are large phases in which a slightly adapted Scrum is used. Basically, Scrum is used as a sub-method inside planned phases. Except for the initial planning phase, the phases have nothing to do with the waterfall model. This is the case because the phases are created by a logical requirement aggregation and therefore have nothing to do with the abstract phases of the waterfall model. Since the phases are created from aggregated requirements, they can have different durations and are always project specific. Unlike the waterfall model, the planned phases are not set in stone. It is not the goal to change them constantly. Nevertheless, to take advantage of the real benefit of planning, they must be able to change easily according to new information.
Initial Planning Phase
To take advantage of the planning process we must plan. I mean we have to not only plan a little bit, but everything also needs to be thought through. We will certainly make mistakes, which we discover later, but that is not what matters. It comes down to thinking broadly and deeply about what we want to create. There is no shortcut if we want the benefits of the planning process. Otherwise, we greatly increase the risk of succumbing to the Dunning–Kruger effect. If we have thought everything through, not only does this effect not occur, we can also deal with new findings much better and adapt our current state of planning a lot easier.
In order to complete this initial planning, we can use the work-breakdown structure. The Project Management Body of Knowledge (PMBOK 5) defines the work-breakdown structure as a “hierarchical decomposition of the total scope of work to be carried out by the project team to accomplish the project objectives and create the required deliverables.” The basic concept consists of a series of recursive passes in which parts of an idea are broken down into smaller pieces until the task area or manageable sections are reached. The result is a tree structure, which shows a subdivision of effort required to achieve an objective. The hierarchical decomposition could be shaped by a functional or object-oriented view.
As each element of the structure is created, the following questions should be asked, among others:
- Does the component solve a problem that needs to be solved?
- Are there dependencies to other components?
- Is the arrangement of the component correct?
- What are the pieces it is made from?
- Do I have enough knowledge to assess the feasibility of the component?
During this initial planning process, the planning team may encounter areas that they cannot fully evaluate. In this case, it is detected early where necessary knowledge or experience is lacking for the project implementation. Depending on the relevance of the planning gap and the resource availability to close it, an early review of the (economic) feasibility should be made. Basically, every new idea should be tested in this way, except for some core R&D projects.
Based on the (initial) tree structure, the phases can be created in which work is done according to Scrum. For these phases to be created, the required skills, resources, and time must be considered for every manageable item at the bottom of the work-breakdown structure, as well as the dependencies among these items.
Scrum-Based Phase Execution
After the initial planning phase has been completed and the project plan is in place, implementation begins. It is possible to implement only one phase at a time. If the dependencies and the encapsulation of the phase contents allow it without taking too much risk, several phases can also be realized simultaneously.
Scrum is used during phase realization. I will not explain how Scrum works, it has already been explained countless times, ad nauseam, elsewhere. Nonetheless, the focus should be on working together across all relevant departments and uncovering challenges of all kinds while delivering validated added value continuously. However, it is not pure agile software development since all requirements and approaches are elaborated at all times at a manageable level. Only new information can destroy its raison d’être.
At the end of each phase, the original plan should be adapted to new information or findings (rolling-wave planning).
Conclusion
Software development is mostly not about writing source code. It is essentially about figuring out what to do, and how to do it, or how it can be implemented in an expandable and efficient way. For this reason, it makes sense to think about this very carefully before writing the first line of source code. Therefore, I think the initial planning phase is very important for most projects. Certainly, adherence to the plan will not lead to the best solution so that the plan will have to be changed in the light of new findings. This is why the proposed rolling-wave planning is necessary.
During the realization of logically bundled task packages (phases), development must be very close to Scrum or comparable methodologies in order to master the complexity. Basically, many sub-projects are created and planned out of one project. The results of each sub-project influence each other, while being encapsulated from each other, so that the best solution can be developed flexibly as a whole. It is an agility-driven divide and conquer methodology. In some way, it is applying the scaled agile framework (SAFe) to only one project while focusing on deep and broad constant planning.
Thanks for reading!
Project Management Consultant
Originally written on Medium by Philip Bamberger
Philip Bamberger is a Project Management Consultant at SteepConsult.
Highly adept at linking the dots between different fields of expertise, he defines himself as a Strategic Thinker, Problem Solver, Business Analyst, Project Manager and Developer.
