Question 669 - CISA discussion

A post-implementation review (PIR) is a process to evaluate whether the objectives of the project were met, determine how effectively this was achieved, learn lessons for the future, and ensure that the organisation gets the most benefit from the implementation of projects1. A PIR is an important tool for assessing the success and value of a project, as well as identifying the areas for improvement and best practices for future projects.

One of the key elements of a PIR is to measure the benefits of the project against the expected outcomes and benefits that were defined at the beginning of the project.Measurable benefits are the quantifiable and verifiable results or outcomes that the project delivers to the organisation or its stakeholders, such as increased revenue, reduced costs, improved quality, enhanced customer satisfaction, or compliance with regulations2. Measurable benefits should be aligned with the organisation's strategy, vision, and goals, and should be SMART (specific, measurable, achievable, relevant, and time-bound).

The finding that measurable benefits were not defined is of greatest significance among the four findings, because it implies that:

The project did not have a clear and agreed-upon purpose, scope, objectives, and deliverables

The project did not have a valid and realistic business case or justification for its initiation and implementation

The project did not have a robust and effective monitoring and evaluation mechanism to track its progress, performance, and impact

The project did not have a reliable and transparent way to demonstrate its value proposition and return on investment to the organisation or its stakeholders

The project did not have a meaningful and actionable way to learn from its achievements and challenges, and to improve its processes and practices

Therefore, an IS auditor should recommend that measurable benefits are defined for any project before its implementation, and that they are reviewed and reported regularly during and after the project's completion.

The other possible findings are:

A lessons-learned session was never conducted: This is a significant finding, but not as significant as the lack of measurable benefits. A lessons-learned session is a process of capturing and documenting the knowledge, experience, and feedback gained from a project, both positive and negative. A lessons-learned session helps to identify the strengths and weaknesses of the project management process, as well as the best practices and lessons for future projects. A lessons-learned session should be conducted at the end of each project phase or milestone, as well as at the end of the project. However, even without a formal lessons-learned session, some learning may still occur informally or implicitly among the project team members or stakeholders.

The projects 10% budget overrun was not reported to senior management: This is a significant finding, but not as significant as the lack of measurable benefits. A budget overrun is a situation where the actual cost of a project exceeds its planned or estimated cost. A budget overrun may indicate poor planning, estimation, or control of the project resources, or unexpected changes or risks that occurred during the project implementation. A budget overrun should be reported to senior management as soon as possible, along with the reasons for it and the corrective actions taken or proposed. However, a budget overrun may not necessarily affect the quality or value of the project deliverables or outcomes if they are still within acceptable standards or expectations.

Monthly dashboards did not always contain deliverables: This is a significant finding, but not as significant as the lack of measurable benefits. A dashboard is a visual tool that displays key performance indicators (KPIs) or metrics related to a project's progress, status, or results. A dashboard helps to monitor and communicate the performance of a project to various stakeholders in a concise and clear manner. A dashboard should include deliverables as one of its components, along with other elements such as schedule, budget, quality, risks, issues, or benefits. However, even without deliverables in monthly dashboards, some information about them may still be available from other sources such as reports or documents.