Six problems in Operationalising Monitoring, Evaluation and Learning (and their solutions)

pksddn Operational MEL, Posts

Does your job description include the words – Monitoring, Evaluation and Learning (MEL)? Are you supposed to design and/or implement MEL in a non-profit, Development organisation? And you have no problems? Then this post is not for you! 🙂

For the vast majority of those who do face problems in MEL, you may be surprised to know that you are not alone. A LOT of persons face similar problems, as I found out when I moderated an e-discussion on this subject.

Here is what happened. I was working for a massive livelihood project in the Indian Himalayas. The project supported women’s credit groups to save money and use these savings to run value chains. These ranged from fruits, vegetables, backyard poultry, spices, bee-keeping …. you get the idea. We worked with 47,000 women in a huge geographical area. And I was the ONLY MEL officer for the whole project.

A heavy workload was only ONE of my problems. I faced many other problems, which I shall describe shortly. Looking for help, I decided to run an e-discussion with fellow MEL officers in other similar projects in Asia to ask them how they were faring. After long and passionate discussions, we found that ALL of us faced similar problems, and were all suffering alone! We also came up with a few solutions.

Here is what we learnt:

Problem 1. MEL is seen more as a reporting and policing system than as a learning system

This is probably THE core problem faced by most MEL staff. In a vast majority of development organisations, MEL is rarely seen as something that supports project planning and implementation. An information retrieval system? Yes. Source for data when the donor report is due? Yes. A learning tool to improve project performance? REALLY?

In some cases, the MEL system (and officer) is also perceived as the internal policeman, keeping tabs on non-performing staff. Mostly, the MEL system is deliberately kept only in a minimal state of activation. I found this out in one of my assignments, when my Director refused to give me permission to carry out planned monitoring visits, talk to project staff or hold review meetings. The reason? Fear. Fear that I would “uncover something” that would be detrimental to the project reputation. This is just one example of the many fears and misconceptions that surround MEL in most projects.

Problem 2. Lack of clarity about when MEL work really starts

A common (mis) perception among project staff (and even in the minds of some MEL officers) is that “MEL comes after we have started implementing a project. There is nothing for the MEL officer to do at the early stages of the project”. However, setting up robust and sensitive MEL systems needs to be done right at the beginning of the project. This ensures clarity on the objectives that are to be achieved, and the metrics to track the same.

Problem 3. Lack of clarity on why a Baseline is needed and when to do it

A corollary to the above apathy about integrating MEL from the beginning of the project is the all-pervasive confusion around the word “Baseline”. Most people consider “Baseline” as an all-encompassing survey of target households covering ALL aspects of their lives. Often, this baseline report is too general to be used or misses information on key project objectives.

In one project, I was setting up a system to monitor the Sustainable Development Goals at Gram Panchayat (local government) level. When I looked at the Baseline, I found that data had been collected on everything except on the relevant SDG indicators which the project was supposed to impact! In such cases, where baseline data is missing, the only solution is to “construct” a baseline (more on that in a subsequent post).

Good MEL practice is to ensure that BEFORE the project starts, data on key parameters and indicators that the project wishes to change is collected. Often, this requires targeted data collection only on a few parameters and can be infinitely more useful than voluminous data on non-essential points. As the old adage says, “Measure what you treasure”.

Problem 4. Lack of articulation on what the MEL system needs to measure

Good MEL practice recommends that the project leadership and teams be clear about the changes (Impact, Outcomes and Outputs) the project will bring in the lives of target communities. The MEL system needs to be built to capture these changes.

Don’t be surprised, however, if you find that your project teams have not taken the time to articulate the changes they want to focus on. In most cases, conceptual clarity about targeted changes and the Theory of Change is a rarity. Most teams are too busy implementing the project to think about what they are trying to achieve. The Logframe or the Project Document is seen as the Bible – and is followed literally, rather than as a general guide. This cascades into a consequent lack of clarity on what the MEL system needs to track. As Yogi Berra said – “If you don’t know where you are going, you’ll end up someplace else.”

Problem 5. Seeing MEL Trainings as something that only MEL officers need to take

MEL trainings, on the rare occasions when they do happen, are generally targeted only at MEL staff. This is actually a manifestation of a lack of clarity on whose role is doing MEL. Good MEL practice recommends that MEL is the responsibility of EVERYONE, and the MEL officer is a facilitator and a resource person for technical support. This skewed understanding about MEL roles further reinforces the notion that MEL is only a donor requirement and not a learning tool.

Problem 6. Lack of hand-holding support for designing and implementing MEL systems 

For a new MEL officer, some (not all) organisations offer a one-time “orientation” in MEL. This is often not sufficient to equip them for designing and implementing a good MEL system. In most organisations where I worked as a MEL officer, I was just thrown into the deep sea of MEL and expected to “figure it out”.

Instead, what a MEL team requires is a continuous hand-holding support to design and implement a light but effective MEL system. The ability to design such a system comes from experience, which a junior MEL officer normally does not have. She/ he needs handholding support and mentoring, which is very rarely provided.

So what can you do to resolve the above issues? 

Solution 1. Ensure that the “L” in MEL is bigger than the M & E 

In most projects which have MEL systems, time and resources are spent on Monitoring, Evaluation and Learning – in that order. Instead, the ideal emphasis should be primarily on Learning, then Evaluation and least of all on Monitoring. (figure below)

But for learning to happen, a few simple tricks seem essential:

  1. Emphasise the Learning role of MEL at all times: Use every forum available for this – e.g. during trainings, strategic discussions, or while designing MEL tools. Most of the times, learning is lost because MEL officers themselves do not ask the question – “So what can we learn from this?”.
  2. b. Incorporate regular learning events into project schedules: Ideally, an Annual Learning Workshop (as different from an Annual Progress Review Workshop) is the MINIMUM that must be done. Just like an evaluation calendar, an annual learning calendar is a great tool to ensure that time is set aside for learning events/ processes.
    c. Use learning events/ meetings to resolve urgent problems: Identify a handful of urgent project issues troubling project staff and conduct thematic research, meeting or workshops on these issues. If the learnings that emerge resolve these problems, it will convince project staff of the day-to-day usefulness of MEL.
    d. Support cross-project learning processes: Learning events that foster learnings across projects (or geographical units) provide the most rapid learnings. This is because the normal tendency for projects is to learn from scratch what other projects have already tried in a different space-time. (e.g. One of the projects I was monitoring was trying to learn how to design social institutions for irrigation management from scratch – not benefitting from already available massive data (and Ph D dissertations) available on similar successful experiments in the same country or other countries).
    e. Peer reviews for rapid learning: We also found that peer reviews (e.g. a team from a different province reviewing project successes and constraints) can be a powerful cross-learning process. Teams usually come away with Aha Conversations like – “Oh, this is how to resolve X issue!” or “Hey, in our area, we did Y to resolve this problem, and it worked! Can you do the same?”. Such processes must be carefully designed to facilitate close interaction with target communities (e.g. women farmers) but should also create space for reflection within project teams.
    e. Knowledge Products for visibility and motivation: MEL teams need to be prolific in creating Knowledge Products from learning events and processes. Sometimes, these products are the most visible signs of individual and team performance. Also, everyone loves to see their project, their farmers, their ideas being documented in colourful, interesting to read documents. Moreover, in the digital age, these products can be circulated easily in soft forms. And if these products can also be in the local language, local communities can also benefit.

Solution 2. Build the constituency for MEL through regular FUN trainings

One of the most useful tricks that I used for advocating the importance of MEL was to always carry with me a module for a three-hour fun session on MEL basics. This session included team games, videos and group work. I called it the “MEL is Fun” module. Often, I would use this as a refresher for all staff or as an orientation for newcomers. Most people did not mind spending three hours. Then, when everyone got excited about MEL, I would offer a three-day training. In most countries, it worked very well and I saw a distinct shift in attitude towards MEL! One of my colleagues said as much:

“Making MEL fun is not an easy task, but you did it! My Team and I have a lot of respect and we highly value your contribution in introducing practical (operational) MEL where our program staff turned your MEL training into action”. ~ Mandefro Nigussie, Deputy Regional Director, Oxfam America – Horn of Africa Regional Office, Addis Ababa

Solution 3. Standardise MEL processes and formats – and don’t change them too often

In one meeting, a project staff told me – “I hate MEL!”. Curious about what could evoke such a harsh emotional response, I asked her what was wrong. It turned out that her MEL officer kept changing the MEL formats almost on a monthly basis, and she had to put in huge efforts and time to re-collect data and fill in tables. Understandably, she hated MEL!

To avoid such hatred against MEL, the MEL Officers need to put adequate thought into designing formats for data collection and test these rigorously before rolling them out on an organisation-wide scale. After this, formats should be changed only once a year, if at all.

Mobile technologies have made things much easier now. For example, in the MIS systems that we design these days, we use Mobile-based systems to track data with one clear ground rule – no data that is entered once should ever need to be entered again. The MIS system should be intelligent enough to “remember” what was already collected. This includes basic information such as the names of target beneficiaries, the interventions made in the last reporting period, and so on. Additional data – on the rare occasions when required at all – is collected in short 2-minute data collection modules, which can be quickly collected during the next community visit. And always, the purpose any data being collected needs to be clear to project teams. This vastly reduces the psychological burden of project staff.

Solution 4. Make support for MEL tasks easily available 

In addition to all the confusing things that are loaded on to a new-comer to the organisation, a hastily-put-together “MEL orientation” is often part of a package. Mostly, this confuses rather than clarifies things to the scared new-comer.

I found that instead of this approach, taking people through a MEL wiki which put all the processes, formats and (what I called) “Minimum Things to do in MEL” became REALLY useful and liked. Most people were relieved that they could access prioritised and useful MEL stuff whenever they wanted. In this Wiki, I usually maintained templates, formats and guidance notes for all mandatory MEL processes (e.g. Evaluation Terms of Reference, a two-page Field Visit Report with fun elements added in, etc.). See the Wiki I designed for Oxfam GB Asia Region here.

Another very useful slide to make permanently available is the one outlining very clearly mandatory MEL tasks and – importantly – where the budget for these tasks would come from. Some organisations ensure that a 5 to 10 per cent budget for MEL is mandatorily built into project budgets. But not all organisations do this, and information on where to get money for MEL often ensures faster implementation of MEL systems.

Project teams (and MEL teams) also normally find useful another slide on their expected roles in MEL, and what kind of support they could (and could not) expect from the MEL team.

Solution 5. Ensure timely Baselines

Finally, MEL officers must ensure (or push for or cajole) all projects to collect Baseline data within the first quarter of the project launch. This data must, at the minimum, cover information related to the key change areas targeted by the project. This change is often visualised in the Logframe or the Theory of Change.

For livelihood projects, I was very impressed by the Baseline Advisory developed by the Government of India’s National Rural Livelihood Mission to all State Livelihood Missions. Take a look here.

In a nutshell, the above have been some of my learnings in trying to operationalise MEL.

Do you face similar problems? Are there other issues you would like to flag? I look forward to hearing from you, fellow MEL practitioners and travellers!


Image source: