Monitoring and Evaluation of Small Business

Monitoring and Evaluation of Business

Monitoring is the systematic collection and analysis of information as a project progresses.

It is aimed at improving the efficiency and effectiveness of a project or organization. It is based on targets set and activities planned during the planning phases of work. It helps to keep the work on track, and can let management know when things are going wrong.

What monitoring and evaluation have in common is that they are geared towards learning from what you are doing and how you are doing it, by focusing on:

Efficiency

Effectiveness Impact

Efficiency tells you that the input into the work is appropriate in terms of the output. This could be input in terms of money, time, staff, equipment and so on. When you run a project and are concerned about its reliability or about going to scale, then it is very important to get the efficiency element right.

Effectiveness is a measure of the extent to which a development programmes or project achieves the specific objectives it set.

Why do monitoring and evaluation?

In many organizations, “monitoring and evaluation” is something that that is seen as a donor requirement rather than a management Tool. Donors are certainly entitled to know whether their money is being properly spent, and whether it is being well spent. But the primary (most important) use of monitoring and evaluation should be for the organization or project itself to see how it is doing against objectives, whether it is having an impact, whether it is working efficiently, and to learn how to do it better. Monitoring and evaluation are both tools which help a project or organisation know when plans are not working, and when circumstances have changed.

Monitoring and evaluation can:

1.     Help you identify problems and their causes;

2.     Suggest possible solutions to problems;

3.     Raise questions about assumptions and strategy;

4.     Push you to reflect on where you are going and how you are getting there;

5.     Provide you with information and insight;

6.     Encourage you to act on the information and insight;

7.     Increase the likelihood that you will make a positive development difference.

Monitoring involves:

1.     Establishing indicators (See Glossary of Terms) of efficiency, effectiveness and impact;

2.     Setting up systems to collect information relating to these indicators;

3.     Collecting and recording the information;

4.     Analysing the information;

5.     Using the information to inform day-to-day management.

6.     Monitoring is an internal function in any project or organisation.

Evaluation involves:

Looking at what the project or organisation intended to achieve – what difference did it want to make? What impact did it want to make?

Assessing its progress towards what it wanted to achieve, its impact targets.

Looking at the strategy of the project or organisation. Did it have a strategy? Was it effective in following its strategy? Did the strategy work? If not, why not?

Looking at how it worked. Was there an efficient use of resources? What were the opportunity costs (see Glossary of Terms) of the way it chose to work? How sustainable is the way in which the project or organisation works? What are the implications for the various stakeholders in the way the organisation works. In an evaluation, we look at efficiency, effectiveness and impact.

There are many different ways of doing an evaluation. Some of the more common terms you may have come across are:

Self-evaluation: This involves an organisation or project holding up a mirror to itself and assessing how it is doing, as a way of learning and improving practice. It takes a very self-reflective and honest organisation to do this effectively, but it can be an important learning experience.

Participatory evaluation: This is a form of internal evaluation. The intention is to involve as many people with a direct stake in the work as possible. This may mean project staff and beneficiaries working together on the evaluation. If an outsider is called in, it is to act as a facilitator of the process, not an evaluator.

Rapid Participatory Appraisal: Originally used in rural areas, the same methodology can, in fact, be applied in most communities. This is a qualitative (see Glossary of Terms) way of doing evaluations. It is semi-structured and carried out by an interdisciplinary team over a short time. It is used as a starting point for understanding a local situation and is a quick, cheap, useful way to gather information. It involves the use of secondary (see Glossary of Terms) data review, direct observation, semi-structured interviews, key informants, group interviews, games, diagrams, maps and calendars. In an evaluation context, it allows one to get valuable input from those who are supposed to be benefiting from the development work. It is flexible and interactive.

External evaluation: This is an evaluation done by a carefully chosen outsider or outsider team.

Interactive evaluation: This involves a very active interaction between an outside evaluator or evaluation team and the organisation or project being evaluated. Sometimes an insider may be included in the evaluation team.

Advantages and Disadvantages of Internal And External Evaluations

Advantages of Internal Evaluations:

The evaluators are very familiar with the work, the organisational culture and the aims and objectives.

Sometimes people are more willing to speak to insiders than to outsiders.

An internal evaluation is very clearly a management tool, a way of self-correcting, and much less threatening than an external evaluation. This may make it easier for those involved to

accept findings and criticisms.

An internal evaluation will cost less than an external evaluation.

Disadvantage

The evaluation team may have a vested interest in reaching positive conclusions about the work or organisation. For this reason, other stakeholders, such as donors, may prefer an

external evaluation.

The team may not be specifically skilled or trained in evaluation.

The evaluation will take up a considerable amount of organisational time – while it may cost less than an external evaluation, the opportunity costs may be high.

Advantages of External Evaluation

External evaluation (done by a team or person with no vested interest in the project)

The evaluation is likely to be more objective as the evaluators will have some distance from the work.

The evaluators should have a range of evaluation skills and experience. Sometimes people are more willing to speak to outsiders than to insiders.

Using an outside evaluator gives greater credibility to findings, particularly positive findings.

Disadvantage

Someone from outside the organization or project may not understand the culture or even what the work is trying to achieve.

Those directly involved may feel threatened by outsiders and be less likely to talk openly and cooperate in the process.

External evaluation can be very costly.

An external evaluator may misunderstand what you want from the evaluation and not give you what you need.

Selecting an External Evaluator or Evaluation Team

Qualities to look for in an external evaluator or evaluation team:

(1) An understanding of development issues.

(2) An understanding of organisational issues.

(3) Experience in evaluating development projects, programmes or organisations.

(4) A good track record with previous clients.

(5) Research skills.

(6) A commitment to quality.

(7) A commitment to deadlines.

(8) Objectivity, honesty and fairness.

(9) Logic and the ability to operate systematically.

(10)                    Ability to communicate verbally and in writing.

(11)                    A style and approach that fits with your organisation.

(12)                    Values that are compatible with those of the organisation.

(13)                    Reasonable rates (fees), measured against the going rates.

Different Approaches to Evaluation

Approach Major purpose Typical focus questions Likely methodology

Goal-based Assessing achievement of goals and objectives.

Were the goals achieved? Efficiently? Were they the right goals?

Comparing baseline (see Glossary of Terms) and progress data (see Glossary of Terms); finding ways to measure indicators.

Decision-making Providing information. Is the project effective? Should it continue? How might it be modified?

Assessing range of options related to the project context, inputs, process, and product. Establishing some kind of decision-making consensus.

Goal-free Assessing the full range of project effects, intended and unintended. What are all the outcomes? What value do they have?

Independent determination of needs and standards to judge project worth. Qualitative and quantitative techniques to uncover any possible results.

Expert judgement Use of expertise. How does an outside professional rate this project? Critical review based on experience, informal surveying, and subjective insights.

Our feeling is that the best evaluators use a combination of all these approaches, and that an organisation can ask for a particular emphasis but should not exclude findings that make use of a different approach.