How to make sure that Lessons Learned stay that way

I often wonder just how much project management organisations really learn from project successes and mistakes. I think we could all definitely learn better than we currently do.

How to make sure lessons learned stay that way

I will start by saying that we should not wait until the end of the project to learn lessons (both good and bad) – these will be happening all the way throughout the project and should be logged as they happen. Completion of a Stage (including the final one at the end of the project) present a convenient time to review lessons and take action (including communicating the lessons to the wider project community), but waiting until the end of the project to even identify the lessons represents a missed opportunity for continuous improvement. Having said that, the end of the project is where projects usually consider lessons learned (if they do this at all), in the form of a post-implementation review.

In my portfolio management office (PfMO) experience I have seen post-implementation reviews come up with the same lessons again and again (e.g. build contingency into plans; take more time and care to gather requirements; don’t Go Live without full sign off / back-out plans / user training, etc.), but I have seen even more examples where post-implementation reviews weren’t carried out at all because: the project was too small / too simple; the project finished too long ago / hasn’t finished yet; I don’t have the time; the project team is now working on something else / has all left the company; nothing happens with Lessons Learned anyway; I don’t think it’s worth the effort (delete as applicable!).

I think the PMO can help with Lessons Learned, and that the problems lie not only with identifying the lessons, but also with what to do with the lessons that have been identified to embed the learning into the organisation’s experience and memory. Here are some ways that the PMO can help, to make sure the Lessons are not only learned but also acted on:

  1. The PMO can add the lessons to the corporate PM methodology.
    Done well, robust PM processes and methodologies form part of an organisation’s corporate memory, and are built making intelligent use of the lessons the organisation has learned from previous projects. They are aimed at preventing the repetition of past mistakes without unnecessarily impeding delivery. But the more you add to the methodology, the more rigid and prescriptive it becomes, and the more likely it is to either be cheerfully ignored or blindly implemented in full (and you can bet it will be ignored on the big risky projects and implemented in full on the little simple ones!). The PMO should also avoid becoming the methodology police.
  2. The PMO could collect the lessons identified in a Big Database
    …but to be useful this relies on someone (PMs? the PMO?) reviewing new projects against all the lessons in the database, and identifying relevant points to note.
  3. The PMO could add the lessons to a checklist
    …but that relies on people (PMs)using the checklist and giving some serious consideration to what the items on the checklist represent (rather than just paying lip service and ticking the box). And there will always be the temptation for PMs to come up with a good reason why a checklist item doesn’t apply to the current project…
  4. The PMO can run Lessons Learned sessions
    where the PM of a completed project presents their Lessons to the rest of the Project Management community (but this relies on an open and honest communications culture, and/or bribing the attendees with cakes), or…
  5. The PMO could pre-load the Risk Register template
    with the risk of repeating mistakes made on previous projects – the mitigation for which would be to review the Lessons Learned Big Database for things to consider (and the PMO could add value by offering this as a service).

I think the last two approaches are the best, because one encourages the development of a learning community (this is the subject of a separate post) and the other places responsibility for reviewing past lessons (or not) with Project Managers, but supports them with a PMO service.

So that’s my approach to making project lessons “stickier”, learned through experience. What do you think? How do you ensure lessons from past projects are not just identified and logged, but learned and applied to new projects? Let me know in the comments.

If you would like to have the benefits of an approach like this but prefer not to have to worry about this sort of techy stuff, why not talk to us about taking care of it all for you on your next big project or programme?

Project reporting shouldn’t be “green side up”

Following on from my post on watermelon reporting, I wanted to share another, related, phenomenon with you – “green side up” reporting. This describes the phenomenon where the health of programmes and portfolios is reported more favourably the higher up the organisation the reports are circulated; that is to say that in the project world everything looks green when viewed from above.

As with watermelon reporting, it’s not my expression but I can’t remember where I heard it.

So what are the factors that might lead to this progressive greening of status?

  • Programme Managers want to be perceived as being in control of their programmes, so although the programme may contain one or two amber or even red projects, they want to avoid the unwelcome attention of the CxOs, and so they report the programme as green overall;
  • The amber or red status of one project is diluted by the green status of other projects in the programme: if a programme comprises say ten projects, how many of these need to be amber or red before the programme overall is considered amber or red?
  • As with watermelon reporting, the problem probably really should have been reported (perhaps as amber) some time ago, and to report it as red now would expose that it wasn’t as amber reported before;
  • Also as with watermelon reporting, there’s still an opportunity to turn it around before the next report and due and then no-one will need to know anything was ever red, will they?

Leaving aside factors shared with watermelon reporting, which I have covered in a separate post, Green Side Up reporting is a Bad Idea because:

  • The organisation’s executives (CxOs) will think that everything is always green (in the same way that VIPs and Royals think that the whole world smells of fresh paint all the time!). This could mean that CxOs only become aware of problems when it is already too late to do anything about them;
  • The organisation’s executives (CxOs) will not believe the reports they are receiving.

So what are the alternatives? Here are a few possibilities:

  • The RAG status of programmes could be calculated based on some sort of average of the contents (the calculation of which only the PMO will understand, and which in any case will result in all programmes and projects being amber all the time instead of green or red, which doesn’t really help the CxOs);
  • The status of programmes and portfolios could be reported in the form of a “RAG pie”, with slices showing the proportions of red, amber and green (more accurate, but not exactly simple!);

    Pie chart showing the percentage of projects that are Red, Amber and Green
    Pie chart showing the percentage of projects that are Red, Amber and Green
  • The health status of programmes and portfolios could still be based on the assessment of the programme managers (because they are paid to manage programme-level problems), but the Portfolio PMO could add an exception reporting mechanism that exposes any red projects and provides more detail on these separately so that the CxOs can dig into this if need be.

I think the last one is probably the most practical and pragmatic. What do you think? Have you seen Green Side Up reporting? How is the health of your programmes related to the health of their constituent projects? Do you have any options to add to my list? How do your CxOs stay informed about the real health of their programmes? Let me know in the comments.

Watch out for watermelons in project reports

How do watermelons get into project reports? and why should you watch out for them?

Watch out for watermelons in project reports

“Watermelon reporting” describes the phenomenon where according to a project status report things appear to be green on the outside (i.e. the project’s RAG status is reported as green, with no issues), but if you delve a little deeper and look inside, it’s actually red right through (i.e. there are serious issues). I can’t take the credit for coining this expression but I can’t remember where I heard it either [14Jul 16 update – I’ve found a 2011 article on DZone, so it would seem that the credit for coining the phrase is due to Marc Loffler – kudos to him!].

So what are the factors that might lead a PM to report a project as green when it’s really red?

  • It’s embarrassing for a PM to admit that “their” project is not going as well as it could/should be;
  • It increases the risk that the project will be cancelled (and we all know what happens to a PM whose projects get cancelled…);
  • The problem really should have been reported (perhaps as amber) some time ago, and to report it as red now would expose that it wasn’t reported amber before;
  • There’s still an opportunity to turn it around before the next report is due and then no-one will need to know it was ever red, will they?
  • “We don’t have red projects in this organisation” (oh yes you do!)

Looking at these reasons, you can see that most of them come from the assumption that red is a Bad Thing, and that this is the PM’s Fault – i.e. the organisation has a “blame” culture. Unfortunately, concealing red status inside a green status report is seldom a good idea, for the following reasons:

  • The sooner a problem is identified, the cheaper it is to rectify;
  • The sooner the PM escalates problems to the Sponsor, the more likely it is that the Sponsor will hear about it from the PM (rather than from another senior stakeholder with an axe to grind, for example!);
  • The sooner the Sponsor hears about the problem, the more time they have to perhaps mobilise resources to help, pave the way for a resolution, or manage the message;
  • The consequences of the concealment being discovered get worse the longer the concealment persists (and this is more a case of when rather than if, a bit like when children lie to their parents)
  • If the project really is red enough to be cancelled, then doing this would be in the interests of the organisation, and no PM will win friends by keeping a bad project going longer than it should be.

Of course the PM should not be flagging every little issue and running to the sponsor every half hour (they are paid to sort most things out) but they should be flagging the project as red when any of the constraints (i.e. budget, schedule, quality, benefits) are seriously threatened, and feeling no shame in doing so.

Red should not be seen as an indicator that there is (or should be!) blood on the corporate carpet; instead red should be seen as the colour of the distress flare that the PM is effectively launching to alert stakeholders to the fact that there is trouble, and some tough decisions may need to be taken.

As a PMO, wherever I see watermelon reporting, I encourage the PM to be brave and honest enough to ask for help. Have you seen watermelon reporting? Do your project managers “keep calm and report it as green”? If so, how have you dealt with it? Let me know in the comments.

If you would like to have the benefits of an approach like this but prefer not to have to worry about this sort of stuff, why not talk to us about taking care of it all for you on your next big project or programme?

Project data in, portfolio insight out

How do you give senior management a “helicopter view” of all the change that is going on in an organisation, so that they can make informed decisions on how to spend their change budget? Here’s how I tackled it…

Data in insight out

What I found…

I came to a new Portfolio Management Office (PfMO) in which status was being reported in various formats. Reports were consolidated but this was based on low-value-add copying and pasting of information into lengthy reports. Pretty much every project proposal was being approved, with no view of the organisation’s capacity to implement them. This resulted in projects getting in each other’s way, competing for resources and overloading Subject Matter Experts (SMEs*).

The Goal

The COO tasked me with developing an integrated view of the portfolio to enable sensible prioritisation decisions to be made, and with developing a resource view looking up to one year ahead.

There was neither the appetite nor the budget to invest in a software tool, so the solution needed to be home-grown.

What I did

Standardise and integrate

I ensured that project data was being collected in a standardised way to enable automated collation into a portfolio view by developing an integrated “Project Status Workbook”. This is effectively a multi-page spreadsheet combining RAG Status reports, Risk, Issue, Dependency and Change Registers into a single recording and reporting document for each project, with a stakeholder dashboard showing key pieces of information in a simple graphical format.

Then I developed a consolidating spreadsheet, using VBA macros to collate project data in various categories.

Analyse and interpret

I analysed the data to provide management information to the Portfolio Board (e.g. % of Projects Red/Amber/Green; % of projects Delivered on Time/to Budget; Portfolio focus on Run/Grow the Business, etc.).

I charted trends to provide insight into project management effectiveness.

I introduced simple Demand Management, using a simple spreadsheet to compare resource demand (from active and proposed projects, on a FTEs per month basis) with planned supply in various categories (PMs, BAs, Developers, BAU SMEs, etc.) to enable the Portfolio Board (C-Level executives) to assess the viability of the planned project portfolio within the prevailing (tight!) resource constraints.

Whenever a new project request was received, I added in the forecast resource demand to show the effect on the “deliverability” of the portfolio as a whole.

I used a derivative of this approach when carrying out the annual financial planning exercise. This started with the annual portfolio budget figure, and showed how the projects (prioritised by the portfolio board using a weighted scoring approach covering factors such as complexity, benefits, risk, strategic alignment, etc.) chip away at the portfolio budget until it runs out at “The Cut”, beyond which lower priority projects would need to be put on hold, delayed, rejected or stopped.

I overlaid the resource view onto the financial view to see how the scheduling of the chosen projects affected the resource demand.

…and the result?

The Project Portfolio Board received a concise, consistent and informative view of Portfolio Status, and were able to consider project proposals against the context of the organisation’s capacity to implement them. Projects were better prioritised and scheduled so that difficulties related to resource bottlenecks were reduced.

So that’s my approach to generating portfolio insights, learned through the experience of getting it wrong and coming up with a better way. It this approach useful to you? Let me know in the comments.

If you would like to have the benefits of an approach like this but prefer not to have to worry about this sort of techy stuff, why not talk to us about taking care of it all for you?


*I use the term SMEs to mean people from the operational business who are subject matter experts in the processes or business areas that are to be changed. They do not necessarily contribute to the change itself or form part of the core project team, but their contribution is invaluable in shaping the changes that are made and how they are to be made.

How to create a streamlined project management methodology

How do you develop a project management methodology that isn’t too heavy or too light, that respects the experience of project managers, and that is accessible? Here’s how I approached it…

How to create a streamlined PM methodology

I was made the PMO Manager of a department of about 20 project managers. Although I didn’t have responsibility for the project managers or the portfolio of projects, I did have responsibility for the methodology they were working to and for the reporting of project information to the project portfolio board (comprising most of the organisation’s senior management).

Review the “As Is” methodology

At the time I was appointed, I assumed responsibility for managing the methodology – this was a comprehensive rulebook, spanning over 125 pages and including over 30 document templates, that was updated roughly quarterly. This rulebook was generally either blindly implemented in full (by the less experienced project managers, potentially impeding project delivery) or cheerfully ignored (by the more experienced project managers, potentially increasing the risk to delivery).

Seeing that this was not an ideal situation, I reviewed the methodology to identify the key control points and templates that delivered the most value, either for the project managers or for project and portfolio stakeholders.

I involved the project managers, who helped me to identify where things were inefficient and could be improved. I asked some of them to help me develop and test-drive new improved multi-purpose templates and more efficient ways of reporting, that removed the old cut-and-paste consolidation.

Develop the “To Be” methodology in consultation with stakeholders

To improve ease of use (and uptake) I represented the entire project management approach as a process flow diagram or “framework” on a single A4 page, with hyperlinks to the latest templates for each section. I identified the steps and documents that would always be required for the type of projects generally undertaken, and made these items part of what I called the mandatory “backbone” of the new approach. Everything else I made either conditionally mandatory (depending on project dimensions such as size, risk, complexity, etc.) or fully optional elements. These optional elements could be selected from the “toolkit” or not, as the result of a discussion held early on in the project between the project manager and their line manager (who typically was responsible for the project management of a section of the project portfolio). I designed a document to record the choices made and the reasons why any optional element was though not to be relevant or useful to the project; this empowered the project manager to make choices, but also made the project manager accountable for those choices.

Implement the solution and train users

I trained the project managers in the new approach and published it on the organisation’s intranet. I encouraged project managers not to print paper copies of the guidance or templates or to re-use old completed documents, instead pointing them to the intranet site so that they would always be using the latest versions of the PM approach and the templates.

Review performance, and tune the methodology

I carried out regular post-implementation surveys and facilitated review meetings, and integrated the lessons learned into the framework. I abandoned quarterly releases of a weighty manual, instead making changes to templates and guidelines as often as these were appropriate (and informing stakeholders of the changes and why they had been made).

The new framework gave increased flexibility to project managers whilst ensuring their accountability for their project management approach. The project management community bought in to the framework as they had been closely involved in its development.

So that’s my approach to refreshing a project management methodology, learned through experience. It this approach useful to you? Let me know in the comments.

If you would like to have the benefits of an approach like this but prefer not to have to worry about this sort of techy stuff, why not talk to us about taking care of it all for you?

How to take charge of your project portfolio’s annual budget

Project and Programme Managers (PMs) are used to managing projects within a budget for the project stage or the whole project, but they sometimes overlook the context of the organisational annual budget; this is especially important if the project spans multiple financial years.

How to take charge of your project portfolio's annual budget

Project funding approval for a whole project or project stage can be thought of as the Portfolio Board deciding that the project’s Business Case is viable, and that the project should go ahead.

This approval can be given once for the whole project (usually if the project is relatively small, short in duration, low in complexity and risk), or gatewise at the beginning of each stage (usually if the project is relatively large, long in duration, high in complexity and risk). This approval can also be separate and distinct from the financial year budgeting process (figure 1).

Figure 1: Financial year vs. project budgeting
Figure 1: Financial year vs. project budgeting

Inclusion of project funding in the financial year budget can be thought of as (usually) the Finance department deciding that the organisation can afford to spend £x on projects as a whole, with the projects making up the budget being selected based on some prioritisation rationale.

So a project managers’ forecasted project costs need to stay not only within the budget for the project (current stage and overall), but also the financial year budget (this is especially important if the project covers more than one financial year).

So how to apply this to portfolios? I was brought into a PMO facing an annual project budget considerably smaller than the previous year’s, with no way to forecast whether the proposed portfolio could be delivered within budget. The organisation had also mandated moving labour spend from Consultants, via Staff Augmentation (comprising flexible resources and independent Contractors), to permanent Internal staff. This is how I approached it:

  1. I devised a standardised way of classifying labour costs to enable comparison across projects (projects had previously been accounting for labour costs in a variety of ways).
  2. I introduced an approach requiring PMs to validate their remaining project cost forecasts every reporting cycle when updating actual costs.
  3. I created an automated report to extract financial data from the project portfolio system and present a monthly breakdown of costs over the financial year (with actuals in the past and forecasts in the future). Labour was broken down further (Consultancy / Staff Augmentation / Internal) to reveal trends (figure 2).
Figure 2: Breakdown of monthly costs by standard cost category
Figure 2: Breakdown of monthly costs by standard cost category

Thus, stakeholders could see every month whether the portfolio was on track to deliver within the financial year budget and track the movement of labour from high towards lower cost labour sources as mandated.

After a few months monitoring, I noticed that there was a “bow wave” phenomenon, with projects seeming to “push” cost forecasts in front of the current month (figure 3). I suspected this was because the projects were forecasting to use up the remainder of the funds allocated to them, so I challenged the PMs to ensure that the forecast to the end of the year represents the forecast cost of the work remaining.

Figure 3: Project cost forecast "bow wave"
Figure 3: Project cost forecast “bow wave”

 

After some revisions to the forecasts, and delays to a number of projects (some deliberate, and some arising from outside the organisation), the outcome was that all the projects in the portfolio could be delivered well within the new lower budget, with a surplus being returned to the organisation.

Does your organisation monitor against annual financial budgets as well as end-to-end or gatewise project budgets? How does your PMO ensure that projects stay within all of these? Let me know in the comments.

How PMOs improve project governance

One of the benefits of having a Project Management Office (PMO) is that good governance reduces the risks of project delivery and increases its predictability. I will explain how this works in practice using as a case study a Portfolio PMO that I managed.

How PMOs improve Project Governance

The organisation ran a fairly large number of relatively short term projects to set up software systems for new distribution clients. These Client Onboarding projects were run by project managers, but without reference to a documented methodology or a central co-ordinating body. The timescales were often unrealistically tight, and projects would often either miss the due date, or would overspend on additional resources in order to meet the due date.

The tight timescales were usually imposed as a result of sales representatives making ambitious promises to clients without reference to the organisation’s capacity to deliver. Overspend to achieve the deadlines was authorised by the same sales representatives that set the deadlines, but the money to fund the overspend did not come from the Sales budget, and no assessment was made of the effect of the increased effort on the other projects under way.

We were tasked with turning this situation around.

We made the following changes:

  • We set up a “pipeline” list of all the projects likely to be started in the near future, and put in place an approval mechanism so that no new projects (including client onboarding) would be started without the approval of a Portfolio Board.
  • We developed a monitoring approach to enable the Portfolio Management Office (PfMO) to detect situations where a project Change Request may be required. A discussion between the PM and the PfMO would be prompted by the triggering of either of two indicators:
    • Finances: Actuals to Date and Estimates At Completion (EAC) were monitored against approved Budgets At Completion (BAC), with indicators to show when EAC exceeded BAC by more than a tolerance amount (expressed in both absolute and relative terms).
    • Schedule:Key project milestones were monitored, with indicators to show if these were forecast to be late against the baselined dates.
  • We measured changes from baseline on both a cumulative and incremental basis, to prevent large changes being concealed as a series of small changes.
  • We escalated the authorisation of Project Change Requests to an organisational level dependent on the size of the change (in terms of costs, time, and business value)
  • We devised templates to support PMs in requesting changes and determining the approval level required, and monitored numbers of approved and rejected changes (to identify “noisy” projects).

As a result, project changes were reviewed and approved at an appropriate organisational level, the highest of which included a full review of the change impact on the rest of the portfolio. In the space of a year, this (along with other initiatives) moved the Client project portfolio from a situation in which all Client projects missed either time or budget constraints (and around 15% missed both!) to a situation in which just 20% of Client projects missed time or budget constraints, and none missed both.

Even if you are cynical and take the view that implementing such a Change Request process merely sanctions the breaking of initial Time and Cost constraints, then at the very least it ensures this is done with eyes open and in the full knowledge of the impact on the project portfolio as a whole.

This approach effectively reduces the execution risk to the portfolio, and enables the Portfolio Board to consciously decide where to focus the organisation’s limited resources.

 

Brilliant Baselines is now Pragmatic PMO

BBL 2 PPMO

What was Brilliant Baselines Limited is now Pragmatic PMO Limited. This is the same company, just with a new name (take a look here to discover the rationale behind the name).

For technical reasons, it was not possible to copy historical content from BrilliantBaselines.com to PragmaticPMO.com.

If you are looking for an article that was previously on brilliantbaselines.com, please get in touch with us and we will either send you a copy of the article or re-publish it on this new site.