Risk Simulation Problems Missed by Most Project Risk Management Software Users

In this article, we will talk about risk simulation problems missed by most project risk management software users.

Reliable project delivery is impossible without proper risk management.

Projects always include something unique, estimates of future durations and costs are always uncertain, risk events can happen and lead to changes in project scope, etc. That is why it is necessary to estimate existing uncertainties and create project contingency reserves that are necessary for meeting project targets with sufficient probabilities.

Contingency reserves are determined by risk simulation usually done using Monte Carlo method. Some project management sotware tools like Spider Project and Safran include risk simulation capabilities, users of others tools rely on external software for risk simulation.

Monte Carlo method can be used for simulation of both uncertainties of initial data and risk events.

Monte Carlo Simulation works the following way:

• People enter three estimates of initial data that are uncertain (optimistic, most probable and pessimistic) and define what probability distribution each uncertain parameter has.
• Risk events are included in the project risk model with their probabilities and impacts.
• The software calculates the project schedule and accompanying parameters (cost, material requirements, etc.) over and over, each time using different set of initial data in accordance with their probability distributions. The number of iterations usually is defined by the risk management software user, usually this number is measured in thousands.
• As the result we get the distributions of possible outcome values.

Risk simulation is becoming popular but most risk simulation and management tools and the ways how they are used miss some important functionalities that make the results of this simulation unreliable. Most common of these problems we will discuss in this article.

To be reliable, Risk Simulation must:

• Take into account all existing constraints and restrictions, including resource constraints, material supply constraints, funding restrictions, etc.
• Simulate possible risk events and risk responses to be applied by the management in case these events do occur
• Simulate corrective actions to be taken by the management in case project performance gets worse (or better) than expected as a result of risk events

To meet risk simulation requirements, one has to be able to use the GERT network that includes the probabilistic and conditional branches, and simulates potential loops in the project execution.

The risk model must simulate original uncertainty, existing dependencies between project parameters, and how the project will be managed in the real life.

Major Risk Simulation Problems that will be discussed in this article include:

• Simulating the result of uncertainty rather than its source
• Missing correlations between probability distributions of project parameters
• Missing corrective actions to be taken if a risk event occurs and threatens project objectives
• Using leveling heuristics other than those employed for managing projects with resource constraints

Using Risk Management Software for Simulating the Result of Uncertainty Rather Than Its Source

Let’s consider a simple project consisting of a single activity; the volume of work for this activity is 80 pieces.

Activity duration depends on the assigned resource’s productivity with the optimistic estimate of 1.25 pc/hour, the most likely estimate being 1 pc/hour, and the pessimistic estimate being 0.8 pc/hour. The workday is 8 hours. Thus, the optimistic duration of the activity/project is 8 days, the most likely duration is 10 days, and the pessimistic duration is 12.5 days.

The source of uncertainty is the assigned resource’s productivity – however, most risk simulation tools do not work with resource productivity and only simulate duration uncertainty.

If to simulate productivity uncertainty the probability to finish activity execution earlier than in 10 days is 55.39% but if to simulate duration uncertainty the probability of the same is only 44.95% (look at Exhibit 1 and 2).

Duration uncertainty is the result of productivity uncertainty. Simulating not the source of uncertainty but its result leads to wrong estimates.

Missing Correlations Between Probability Distributions of Project Parameters

Let’s assume that we decided to create a more detailed model of our project and, instead of one activity with the most likely duration of 10 days, created a path of ten activities with the most likely duration of 1 day each and the same distribution of activity duration (-20%, +25%).

If to consider these ten activities independent the project duration probability distribution will change and the probability of finishing it earlier than in 10 days becomes 28.84% instead of initial 44.95%. It does not look logical – probability distribution should not depend on the way an activity was presented in the project schedule.

Activity duration depends on the productivity of assigned resources: the higher the productivity, the faster all ten activities will be performed. Therefore, the durations of all 10 activities are strongly correlated with each other. With this condition initial probability will not change. If the source of uncertainty (resource productivity) was simulated instead of the result (activity duration) this problem would not exist.

It is not easy to set correlations between durations for all activities executed by the same resources. Therefore, the probabilities calculated using the Monte Carlo method can be incorrect if some dependencies were missed.

Missing Corrective Actions to Take in Case a Risk Event Occurs and Threatens Project Objectives

When projects are late and over budget, corrective actions are taken. The Monte Carlo model should simulate these corrective actions in the respective iterations, in other case the results will not be reliable because the model does not reflect real life actions. It means that the project model must include conditional branches “if … then” and risk simulation software must automatically select the right branch, based on the predefined conditions.

Besides, risk simulation software must be able to work with risk events and probabilistic branches activated by certain probabilities in case of occurrence of risk events.

If risk responses and corrective actions are not simulated, the probability distributions shown by MC method will be incorrect.

Using Leveling Heuristics Other Than Those Used for Managing Projects with Resource Constraints

Project constraints can include resource, supply, and funding restrictions to be taken into account in all MC iterations. It means that in each iteration resources must be leveled using the same leveling algorithms as in the project risk management software that is used for project resource management. Otherwise, MC probability distributions will be incorrect.

But different risk management software programs, however, use their proprietary leveling algorithms and create different resource-constrained schedules for the same projects. This is due to the absence of mathematical solution for finding the best (shortest) schedule when project resources are constrained.

Let’s have a look at the resource-constrained schedules calculated for the same project by Microsoft Project (left) and Spider Project (right):

We have entered the same activity duration ranges (-25%, +50%) and triangular distributions of activity durations, and obtained different probability distributions for project duration calculated with MS Project heuristic and Spider Project heuristic.

The probability of finishing the project faster than in 25 workdays is 81.25% for Spider Project users and only 1% for MS Project users. Now imagine that MS Project user used Spider Project for MC simulation and setting reliable target for his project with resource constraints! So, if to set project targets based on one heuristic and to use another one for project resources allocation and management MC simulation results can be misleading.

Any external risk simulation tool is not reliable when project schedule has resource constraints.

Conclusions

Using Monte Carlo simulation for quantitative risk analysis:

• Try to set ranges to those parameters that cause uncertainty and not to the uncertainty results
• Set correlations between uncertain parameters that depend on each other
• Include in the risk model corrective actions people apply when the reaching of project goals is endangered
• Do not rely on external risk simulation tools if your projects have resource constraints

Tags: Monte Carlo Simulation MS Project Risk Management Software Safran Spider Project