Software Project Management: Project Evaluation

PROJECT EVALUATION

1 Strategic Assessment

2 Technical Assessment

3 Cost Benefit Analysis

4 Cash Flow Forecasting

5 Cost Benefit Evaluation Techniques

6 Risk Evaluation

 Project Evaluation:

A high level assessment of the project

to see whether it is worthwhile to proceed with the project

to see whether the project will fit in the strategic planning of the whole organization

            Project Evaluation

Why

Want to decide whether a project can proceed before it is too late

Want to decide which of the several alternative projects has a better success rate, a higher turnover, a higher ...

Is it desirable to carry out the development and operation of the software system

Who

Senior management

Project manager/coordinator

Team leader

When

• Usually at the beginning of the project e.g. Step 0 of Step Wise Framework

What

Strategic assessment

Technical assessment

Economic assessment

How

Cost-benefit analysis

Cash flow forecasting

Cost-benefit evaluation techniques

•   Risk analysis

        1 Strategic Assessment

Used to assess whether a project fits in the long-term goal of the organization

Usually carried out by senior management

Needs a strategic plan that clearly defines the objectives of the organization

Evaluates individual projects against the strategic plan or the overall business objectives Programme management

suitable for projects developed for use in the organization Portfolio management

suitable for project developed for other companies by software houses

SA – Programme Management

Individual projects as components of a programme within the organization

Programme as “a group of projects that are managed in a coordinated way to gain benefits that would not be possible were the projects to be managed independently

SA – Programme Management Issues

   Objectives

How does the project contribute to the long-term goal of the organization?

Will the product increase the market share? By how much?

   IS plan

Does the product fit into the overall IS plan?

How does the product relate to other existing systems?

Organization structure

How does the product affect the existing organizational structure? the existing workflow? the overall business model?

MIS

What information does the product provide?

To whom is the information provided?

How does the product relate to other existing MISs?

Personnel

What are the staff implications?

What are the impacts on the overall policy on staff development?

Image

How does the product affect the image of the organization?

            SA – Portfolio Management

suitable for product developed by a software company for an organization

may need to assess the product for the client organization

Programme management issues apply

need to carry out strategic assessment for the providing software company

Long-term goal of the software company

The effects of the project on the portfolio of the company (synergies and conflicts)

Any added-value to the overall portfolio of the company

2 Technical Assessment

Functionality against hardware and software

Thestrategic IS plan of the organization

any constraints imposed by the IS plan

1 Economic Assessment

Why?

Consider whether the project is the best among other options

Prioritise the projects so that the resources can be allocated effectively if several projects are underway

How?

Cost-benefit analysis

Cash flow forecasting

Various cost-benefit evaluation techniques

NPV and IRR

2 EA – Cost-benefit Analysis

A standard way to assess the economic benefits

Two steps

Identify and estimate all the costs and benefits of carrying out the project

Express the costs and benefits in a common unit for easy comparison (e.g. $)

Costs

Development costs

Setup costs

Operational costs Benefits

Direct benefits

Assessable indirect benefits

Intangible benefits

            EA – Cash Flow Forecasting

What?

Estimation of the cash flow over time

Why?

An excess of estimated benefits over the estimated costs is not sufficient

Need detailed estimation of benefits and costs versus time

What?

Estimation of the cash flow over time

Why?

An excess of estimated benefits over the estimated costs is not sufficient

Need detailed estimation of benefits and costs versus time

Need to forecast the expenditure and the income

Accurate forecast is not e asy

Need to revise the forecast from time to time

4 Cost-benefit Evaluation Techniques

•   Net profit

= Total income – Total costs

•   Payback period

= Time taken to break even

•   Return on Investment ( ROI)

5 Cost-benefit Evaluation Techniques – NPV Net present value (NPV)

It is the sum of the presennt values of all future amounts.

Present value is the value which a future amount is worth at present

It takes into account the p rofitability of a project and the timing of the cash flows

Let n be the number of yea r and r be the discount rate, the present value (PV) is given

by

Issues in NPV

Choosingan appro priate discount rate is difficult

Ensuring that the rankings of projects are not sensitive to small changes in discount rate

Guidelines:

Use the standard rate prescribed by the organization

Use interest rate + premium rate

Use a target rate of return

Rank the projects using various discount rates

Disadvantage

May not be directly comparable with earnings from other investments or the costs of borrowing capital

Internal Rate of Return (IRR)

The percentage discount rate that would produce a NPV of zero

A relative measure

Advantages

Convenient

Directly comparable with rate of return on other projects and with interest rates

Useful

Dismiss a project due to its small IRR value

Indicate further precise evaluation of a project

Supported by MS Excel and Lotus 1-2-3

Estimation

Why? – to define the project budget and to ‘refine’ the product to realize the budget

Who? – the manager

What? – size and cost

When? – always

How? – techniques and models

Issues related to Estimation

Difficult to make accurate estimation

Better to have previous data and analyze the actual values against their estimates so that you know how accurate you are

Even better to have previous data of the whole organization so that you know how accurate the estimation method, if any, used within the organization

Positive Attitude Towards Estimation

Use your estimation as a guide to manage your project

From time to time, you need to revise your estimation based on the current status of the project

Estimation Approaches

Expert judgement

Ask the knowledgeable experts

Estimation by analogy

Use the data of a similar and completed project

Pricing to win

Use the price that is low enough to win the contract

Top-down

An overall estimate is determined and then broken down into each component task

Bottom-up

The estimates of each component task are aggregated to form the overall estimate

Algorithmic model

Estimation is  based  on  the characteristics  of the product and  the development

environment

Size Estimation

Problems related to size estimation

Size Estimation Model

Function Point Analysis (FPA)

Problems related to size estimation

Nature of software

Novel application of software

Fast changing technology

Lack of homogeneity of project experience

Subjective nature of estimation

Political implications within the organization

3 Cost-Benefit Analysis

Cost/benefit analysis, comparing

– Expected costs

– Expected benefits

Issues

– Estimating costs

– Estimating benefits

Use of financial models to evaluate

 Cost-Benefit Analysis-Two Steps

Identifying and estimating all of the costs and benefits of carrying out the project and operating the delivered application

Expressing the costs and benefits in common units

 Cost-Benefit Analysis-Cost Estimation

Estimate costs to compare with benefits/other investment options

Overall estimation based on

– Estimation of required activities (structure)

– Estimation for each activity

– Estimation of installation/setup cost

– Estimation of operational cost

Difficult, as a lot of these are`estimates’;

estimation errors cascade

Cost-Benefit Analysis-Cost Category

Development costs

Setup costs

Operational costs

Cost-Benefit Analysis-Development Costs

Salaries (base, incentives, and bonuses)

Equipment for development

– Hardware

– Software

Cost-Benefit Analysis-Setup Cost

Hardware and software infrastructure

Recruitment/staff training

Installation and conversion costs

Cost-Benefit Analysis-Operational Costs

Costs of operating the system once it has been installed

– Support costs

– Hosting costs

– Licensing costs

– Maintenance costs

– Backup costs

Cost-Benefit Analysis-Benefit Estimation

Estimate benefits of new system based on– Estimation of cost savings and money generation when deployed– Value of information obtained for objective driven project

– Value of intangibles

Cost Benefits Analysis-Benefits Types

Direct benefits

Indirect benefits

Intangible benefits

Cost Benefits Analysis-Direct Benefits

Directly accountable to new system

– Cost savings (e.g., less staff, less paper, quicker turnaround)

– Money generation (e.g., new revenue stream, new markets) Measurable after system is operational

Have to be estimated for cost/benefit analysis

Cost Benefits Analysis -Intangible Benefits

Positive side effects of new system

External system (e.g., increase branding, entry to new markets)

Internal system (increased interest in job for users, enabler for other systems) Often very specific to a project; not measurable even after a system is operational Part of strategic decision rather than cost/benefit analysis

 4 Cash Flow Forecasting

Indicates when expenditure and income will take place

1Cash Flow Analysis

Typically there are outgoing payments initially and then incoming payments There might be additional costs at the end of the project life

Cash flow considerations

– Is initial funding for the project available?

– Is timing of incoming/outgoing cash flow in line with financial plans?

– If cash flow is critical, forecasting should be done quarterly or monthly

Risky/expensive projects might be funded using venture capital

5 Cost-Benefit Evaluation-Techniques

Costs and benefits have to be expressed using the same scale to be comparable Usually expressed in payments at certain times (cash flow table)

Payments at different points in time are not comparable based only on the amount Time of payment should be considered

Techniques

– Net profit

– Payback period

– Return on investment

– Net present value

– Internal rate of return

Cost-Benefit Evaluation Techniques -Net Profit

Difference between total cost and total income Pros: Easy to calculate

Cons

– Does not show profit relative to size investment (e.g., consider Project 2)

– Does not consider timing of payments (e.g., compare Projects 1 and 3) Not very useful other than for "back of envelope" evaluations

            Cost-Benefit Evaluation Techniques -Payback Period

Time taken to break even

Pros

– Easy to calculate

– Gives some idea of cash flow impact Cons: Ignores overall profitability

Not very useful by itself, but a good measure for cash flow impact

            Costs-Benefit Evaluation Techniques-Return On Investment

Also known as the accounting rate of return (ARR)

Provides a way of comparing the net profitability to the investment required The common formula– ROI = (average annual profit/total investment) X 100

            Cost-Benefit Evaluation Techniques -Return On Investment

Pros: Easy to calculate

Cons

– Does not consider the timing of payments

– Misleading: does not consider bank interest rates Not very useful other than for "back of envelope" evaluations

            Cost-Benefit Evaluation Techniques-Net Present Value

A project evaluation technique that takes into account the profitability of a project and the timing of the cash flows that are produced

Sum of all incoming and outgoing payments, discounted using an interest rate, to a fixed point in time (the present)

Cost-Benefit Evaluation Techniques-Net Present Value

Present value = (value in year t)/(1+r)^t

– r is the discount rate

– t is the number of years into the future that the cash flow occurs

– (1+r)^t is known as discount factor

In the case of 10% rate and one year

– Discount factor = 1/(1+0.10) = 0.9091 In the case of 10% rate and two years

– Discount factor = 1/(1.10 x 1.10) = 0.8294

Pros

– Takes into account profitability

– Considers timing of payments

– Considers economic situation through discount rate Cons: Discount rate can be difficult to choose

Standard measure to compare different options

            Cost-Benefit Evaluation Techniques -Internal Rate of Return

Internal rate of return (IRR) is the discount rate that would produce an NPV of 0 for the

project

Can be used to compare different investment opportunities

There is a Microsoft Excel function to calculate IRR

Pros: Calculates figure which is easily comparable to interest rates

Cons: Difficult to calculate (iterative)

Standard way to compare projects

            Definition of Risk

A risk is a potential problem – it might happen and it might not Conceptual definition of risk

–  Risk concerns future happenings

–  Risk involves change in mind, opinion, actions, places, etc.

– Risk involves choice and the uncertainty that choice entails Two characteristics of risk

– Uncertainty – the risk may or may not happen, that is, there are no 100% risks (those, instead, are called constraints)

–  Loss – the risk becomes a reality and unwanted consequences or losses occur

1 Risk Categorization – Approach

Project risks

They threaten the project plan

If they become real, it is likely that the project schedule will slip and that costs will increase

Technical risks

They threaten the quality and timeliness of the software to be produced If they become real, implementation may become difficult or impossible

Business risks

They threaten the viability of the software to be built

If they become real, they jeopardize the project or the product Sub-categories of Business risks

Market risk – building an excellent product or system that no one really wants

Strategic risk – building a product that no longer fits into the overall business strategy for the company

Sales risk – building a product that the sales force doesn't understand how to sell

Management risk – losing the support of senior management due to a change in focus or a change in people

Budget risk – losing budgetary or personnel commitment

Known risks

Those risks that can be uncovered after careful evaluation of the project plan, the business and technical environment in which the project is being developed, and other reliable information sources (e.g., unrealistic delivery date)

Predictable risks

Those risks that are extrapolated from past project experience (e.g., past turnover)

Unpredictable risks

Those risks that can and do occur, but are extremely difficult to identify in advance

.2Reactive vs. Proactive Risk Strategies

Reactive risk strategies

–  "Don't worry, I'll think of something"

–  The majority of software teams and managers rely on this approach

–  Nothing is done about risks until something goes wrong

            The team then flies into action in an attempt to correct the problem rapidly (fire fighting)

–  Crisis management is the choice of management techniques

Proactive risk strategies

–  Steps for risk management are followed (see next slide)

– Primary objective is to avoid risk and to have a contingency plan in place to handle unavoidable risks in a controlled and effective manner

            Steps for Risk Management

            Identify possible risks; recognize what can go wrong

            Analyze each risk to estimate the probability that it will occur and the impact (i.e., damage) that it will do if it does occur

3) Rank the risks  by probability and impact

- Impact may be negligible, marginal, critical, and catastrophic

            Develop a contingency plan to manage those risks having high probability and high impact

            Risk Identification

            Risk identification is a systematic attempt to specify threats to the project plan

            By identifying known and predictable risks, the project manager takes a first step toward avoiding them when possible and controlling them when necessary

            Generic risks

–  Risks that are a potential threat to every software project

            Product-specific risks

– Risks that can be identified only by those a with a clear understanding of the technology, the people, and the environment that is specific to the software that is to be built

–  This requires examination of the project plan and the statement of scope

–  "What special characteristics of this product may threaten our project plan?"

            Risk Item Checklist

            Used as one way to identify risks

            Focuses on known and predictable risks in specific subcategories (see next slide)

            Can be organized in several ways

–  A list of characteristics relevant to each risk subcategory

–  Questionnaire that leads to an estimate on the impact of each risk

– A list containing a set of risk component and drivers and their probability of occurrence

            Known and Predictable Risk Categories

            Product size – risks associated with overall size of the software to be built

            Business impact – risks associated with constraints imposed by management or the marketplace

            Customer characteristics – risks associated with sophistication of the customer and the developer's ability to communicate with the customer in a timely manner

            Process definition – risks associated with the degree to which the software process has been defined and is followed

            Development environment – risks associated with availability and quality of the tools to be used to build the project

            Technology to be built – risks associated with complexity of the system to be built and the "newness" of the technology in the system

            Staff size and experience – risks associated with overall technical and project experience of the software engineers who will do the work

            Questionnaire on Project Risk

Have top software and customer managers formally committed to support the project?

Are end-users enthusiastically committed to the project and the system/product to be built?

Are requirements fully understood by the software engineering team and its customers?

Have customers been involved fully in the definition of requirements?

Do end-users have realistic expectations?

Is the project scope stable?

Does the software engineering team have the right mix of skills?

Are project requirements stable?

Does the project team have experience with the technology to be implemented?

Is the number of people on the project team adequate to do the job?

Do all customer/user constituencies agree on the importance of the project and on the requirements for the system/product to be built?

            Risk Components and Drivers

            The project manager identifies the risk drivers that affect the following risk components

– Performance risk - the degree of uncertainty that the product will meet its requirements and be fit for its intended use

–  Cost risk - the degree of uncertainty that the project budget will be maintained

– Support risk - the degree of uncertainty that the resultant software will be easy to correct, adapt, and enhance

– Schedule risk - the degree of uncertainty that the project schedule will be maintained and that the product will be delivered on time

The impact of each risk driver on the risk component is divided into one of four impact levels

–  Negligible, marginal, critical, and catastrophic

Risk drivers can be assessed as impossible, improbable, probable, and frequent

6 Risk Projection (Estimation)

Risk projection (or estimation) attempts to rate each risk in two ways

–  The probability that the risk is real

–  The consequence of the problems associated with the risk, should it occur

The project planner, managers, and technical staff perform four risk projection steps (see next slide)

The intent of these steps is to consider risks in a manner that leads to prioritization

Be prioritizing risks, the software team can allocate limited resources where they will have the most impact

1 Risk Projection/Estimation Steps

Establish a scale that reflects the perceived likelihood of a risk (e.g., 1-low, 10-high)

Delineate the consequences of the risk

Estimate the impact of the risk on the project and product

Note the overall accuracy of the risk projection so that there will be no misunderstandings

2 Contents of a Risk Table

A risk table provides a project manager with a simple technique for risk projection

It consists of five columns

–  Risk Summary – short description of the risk

–  Risk Category – one of seven risk categories (slide 12)

–  Probability – estimation of risk occurrence based on group input

–  Impact – (1) catastrophic (2) critical (3) marginal (4) negligible

– RMMM – Pointer to a paragraph in the Risk Mitigation, Monitoring, and Management Plan

3 Developing a Risk Table

List all risks in the first column (by way of the help of the risk item checklists)

Mark the category of each risk

Estimate the probability of each risk occurring

Assess the impact of each risk based on an averaging of the four risk components to determine an overall impact value (See next slide)

Sort the rows by probability and impact in descending order

Draw a horizontal cutoff line in the table that indicates the risks that will be given further attention

.4 Assessing Risk Impact

Three factors affect the consequences that are likely if a risk does occur

–  Its nature – This indicates the problems that are likely if the risk occurs

– Its scope – This combines the severity of the risk (how serious was it) with its overall distribution (how much was affected)

–  Its timing – This considers when and for how long the impact will be felt

The overall risk exposure formula is RE = P x C

–  P = the probability of occurrence for a risk

–  C = the cost to the project should the risk actually occur

Example

–  P = 80% probability that 18 of 60 software components will have to be developed

–  C = Total cost of developing 18 components is $25,000

–  RE = .80 x $25,000 = $20,000

Risk Mitigation, Monitoring, and Management

• An effective strategy for dealing with risk must consider three issues (Note: these are not mutually exclusive)

–  Risk mitigation (i.e., avoidance)

–  Risk monitoring

–  Risk management and contingency planning

Risk mitigation (avoidance) is the primary strategy and is achieved through a plan Example: Risk of high staff turnover

Seven Principles of Risk ManagementMaintain a global perspective

– View software risks within the context of a system and the business problem that is is intended to solve

Take a forward-looking view

–  Think about risks that may arise in the future; establish contingency plans

Encourage open communication

–  Encourage all stakeholders and users to point out risks at any time

Integrate risk management

–  Integrate the consideration of risk into the software process

Emphasize a continuous process of risk management

– Modify identified risks as more becomes known and add new risks as better insight is achieved

Develop a shared product vision

– A shared vision by all stakeholders facilitates better risk identification and assessment

Encourage teamwork when managing risk

– Pool the skills and experience of all stakeholders when conducting risk management activities