Join us!
Basic- Yellow 120px

Our Programs Module 8

Module 8: Top Tips

macaw-edited-for-article

  • Before designing your M&E system ensure that you are clear about the changes you are trying to achieve - and how social change can be brought about in an enduring way.
  • Develop your own, locally appropriate and useful system of M&E, and build this into your advocacy plan – whilst still following general M&E guidance.
  • The purpose of M&E is to enable animal welfare organizations to review their advocacy experiences and to use this learning to improve their advocacy work – this means that honesty and transparency should be valued.
  • Design a clear and simple monitoring system to ensure that your activities are contributing to your agreed objectives, and to ensure you have a record of any outcomes – intended or unintended.
  • The Logframe can be a very valuable tool for summarizing your objectives (see Tool 27).
  • When carrying out an evaluation, review outcomes & achievements (or changes), your processes, and lessons learned.
  • Develop your indicators carefully to reflect the real value of your advocacy work, including aspects that are important to your organization or coalition/alliance (such as capacity building and relationship development).
  • Record and discuss all of your outcomes – both the intended and unintended.
  • Assess any relevant changes in the external or policy environment, and think about what your response to these changes should be.

Further Resources

macaw-edited-for-article

Websites

World Animal Net (WAN) Animal Protection Society Resources: Continuous Improvement

Intrac: Tracking Progress in Advocacy: Why and How to Monitor and Evaluate Advocacy Projects and Programmes

ODI: A Guide to Monitoring and Evaluating Policy Influence

World Bank: Making Monitoring & Evaluation Systems Work

Aus Aid: Guidance for M&E on Civil Society Programmes

The Learning Organisation

Continuous Improvement

Kaizen Institute

Kaizen

Kaizen Training and Research Page - Creative Safety Supply Research and Training Center

Books

The Fifth Discipline: The Art and Practice of the Learning Organisation
By: Peter M. Senge
Publisher: Bantam Doubleday Dell Publishing Group
ISBN: 0385260954

Ten Steps to a Learning Organisation
By: Peter Kline, Bernhard Saunders
Publisher: Great Ocean Publishers
ISBN: 0915556324

50 Ways Towards a Learning Organisation
By: Andrew Forrest
Publisher: Spiro Press
ISBN: 1858355990

The Goal: A Process of Ongoing Improvement
By: Eliyahu M. Goldratt, Jeff Cox
Publisher: North River Press
ISBN: 0884271781

The Kaizen Blitz
By: Anthony C. Laraia
Publisher: Jon Wiley
ISBN: 0471246484

Kick Down the Door of Complacency: Seize the Power of Continuous Improvement
By: Charles C. Harwood
Publisher: St Lucie Press
ISBN: 157444168X

Kaizen and You: Personal Success Through Continuous Improvement
By: Igor Popovich
Publisher: Management Books 2000
ISBN: 1852522615

The Ever-changing Organisation: Creating the Capacity for Continuous Change, Learning and Improvement
By: G.R. Pieters, D.W. Young
Publisher: St Lucie Press
ISBN: 1574442627

Creating a Learning Organisation (50-minute Series)
By: Barbara J. Braham
Publisher: Crisp Publications Inc
ISBN: 1560523514

How to Design Your M&E System

macaw-edited-for-article

Each advocacy program is different, with its own specific context, objectives and methodology. So it is necessary to develop your own, locally appropriate and useful system of M&E for these – whilst still following general M&E guidance. This requires careful planning.

Before thinking about designing and setting up a monitoring system it is important to be clear about what changes you are trying to achieve and how you believe social change can be brought about in your context, and how it can be sustained – see Module 1.

Once you have identified the change you want to see, and how this can be brought about (your ‘Theory of Change’), then the sort of information that you need to monitor and evaluate (and at what points these M&E activities will occur) will become more evident. You need to remind yourself that what you are trying to do is to monitor whether the progress you want is being achieved and – then finally, in your evaluation, to show how that success was achieved (or why it did not happen as planned).

This should be done at the action planning stage, and the results built into your plan. You need to establish prior to implementation what information is necessary for tracking progress, and how you will obtain this information. If you use a Log Frame, the indicators you develop for your objectives/outcomes should be designed in a way that can measure whether activities have achieved the desired results. You will also need to develop some baseline information, against which your indicators will be assessed.

When developing indicators, it is often stressed that your objectives should be ‘SMART’ (specific, measurable, achievable, relevant and time-bound). But in the case of advocacy work, you will usually need to include some more subjective measures – and a mixture of qualitative and quantitative indicators in order to capture the real value of your work.

Ensure that you develop indicators that assess your progress as well as outcomes. Establish clear milestones. For example, possible indicators could be: active and constructive engagement in a policy or advocacy network; raised policy and implementation questions at key partner meetings; raised awareness about your issue amongst key stakeholders, raised the issue in partnership or coalition/alliance discussions; and included an assessment of the policy environment and key players in your strategy review etc. Whilst effective policy change appears to be a useful success criterion at face value, the full story is somewhat more complex.

Think carefully about how activities should feed into outcomes and the achievement of objectives when establishing indicators. Indicators should be linked in to roles and responsibilities, and planned timeframes, where possible.

Advocacy Tools

Tool 41. Advocacy Outcomes and Achievements
This tool can be used as a guide to the evaluation of your advocacy outcomes and achievements.

Tool 42. Advocacy Evaluation Case Study
This tool is an example of an advocacy evaluation case study.

If you are in receipt of external funding for the advocacy activities, you may be required to report in a specific way to the donor providing the funds. This is often a problem because the policy change you are working for may come well after the period of funding. If you have an understanding donor it is worth asking for them to fund a separate review or evaluation a year after the project finished, to assess if the change has been made and if progress is sustained.

The following tool for evaluating advocacy has been developed specifically for funders, and gives a helpful insight into the funders’ perspective:

Advocacy Tool

Tool 43. Tips for Evaluating Advocacy
A reference tool that builds understanding of the M&E requirements of grantors.

Animal welfare organizations should always strive to avoid over-stating our role, influence and contribution to change. Social change is usually due to a number of factors and influences. We should only attribute a change to our work if we are certain that we did play a meaningful role, or use proxy indicators if we feel that we made a real contribution to change.

Proxy indicators are indicators that we believe are plausible indicators of outcomes or progress towards an outcome. They can be helpful in advocacy work because of the difficulty of attributing results, and the often intangible outcomes. In particular, they can be a legitimate way to measure an individual organization’s roles in coalition work.

Proxy Indicators

An animal welfare organization is involved with a number of partners in a coalition (or alliance) that has been working on the lack of implementation of the OIE animal welfare standards in their country. They have undertaken a number of awareness-raising activities. The government (the key target) has suddenly introduced a new strategy for capacity building and monitoring for official enforcement activities. The organization’s review shows that a) they engaged in a number of activities focused on achieving this change, through the coalition/alliance and b) it has achieved the change they wanted as an outcome. Although it is impossible to directly attribute the government’s revised strategy to the organization’s work – it is a) plausible to believe there is a relationship between the advocacy activities and the change – attribute the change to the objective and b) plausible to believe that their involvement in the coalition/alliance was a contributor to the coalition/alliance’s success.

Here we could use the number of activities undertaken by the coalition/alliance (including the organization) as the proxy indicator.

Continuous Improvement

macaw-edited-for-article

Introduction
What is a 'Learning Organization'?
The Importance of Learning
The Steady Process
The Learning Cycle
The Process
What Type of Learning?
Characteristics of a 'Learning Organization'
Kaizen


Introduction

The animal welfare environment is a fast-changing one, where we are faced with an ever-complex range of problems and opportunities. Our opponents are better resourced in so many ways. Our ‘competitors’ are always trying to steal advantage from us. This makes it vital that we continue to make the most of our scarce resources, and our most valuable asset – our staff. Continuous improvement (or ‘learning’) and Kaizen are ways of achieving this, without a damaging ‘revolution’.

What is 'Learning Organization'?

A ‘Learning Organization’ is an organization that learns and encourages learning among its people. It promotes exchange of information between employees hence creating a more knowledgeable workforce. This produces a very flexible organization where people will accept and adapt to new ideas and changes through a shared vision.

"A Learning Organisation is one in which people at all levels, individuals and collectively, are continually increasing their capacity to produce results they really care about." Senge

Key aspects of a ‘Learning Organisation’ are that it established procedures to:

  • Apply techniques to measure the organization's strengths and weaknesses, successes and failures
  • Identity areas for improvement within the organization
  • Set organizational policies and approaches to all aspects of management
  • Implement techniques to improve organizational effectiveness.

The Importance of Learning

The importance of learning was first put forward by the Chinese philosopher Confucius (551 - 479 BC). He believed that everyone should benefit from learning:

"Without learning, the wise become foolish; by learning, the foolish become wise."
"Learn as if you could never have enough of learning, as if you might miss something."

Reason for the growing emphasis on organizational learning is because of the increased pace of change. Classically, work has been thought of as being conservative and difficult to change. Now, there is such a fast-changing environment that ‘business as usual’ is no longer an option.

As various management writers put it:

"Organisations must develop a capacity for fast-paced innovation... learn to love change" Peters
"As the competitive environment becomes more complex and variegate, the need for greater genetic variety - a broader range of managerial beliefs, and a greater repertoire of managerial actions - grows apace" Hamel and Prahaled
"Top companies seem to organise around people - honoring these needs - feeling of control, something to believe in, challenge, lifelong learning, recognition" Waterman

With the pace of change ever quickening, the need to develop mechanisms for continuous learning and innovation is greater than ever.

The Steady Process

If the changeover to a ‘Learning Organisation’ happened overnight, the environment around the workers would be complex and dynamic. This would cause fear, uncertainty and confusion, which would hamper learning and openness to change. So it can only be introduced into an organization that is prepared to reach a balance between change and stability, i.e. a balance between the old and the new. Thus, part of the senior management’s job involves innovation, but more is process review and some is maintenance. For middle managers, jobs consist largely of process review and maintenance, and for workers, mainly maintenance, with some process review. The core stability is maintained through a steady review and introduction process, supported by staff and management development.

The Learning Cycle

Evaluation is necessary for an organization to learn from its mistakes and also to appreciate its successes. Discussion and contribution in a team framework is vital, followed by assessment and planning. Each team member should be encouraged to assess his/her own performance. This requires continuous feedback and assessment, which is commonly depicted using the ‘Learning Cycle’ model below:

Animal welfare organizations can be very poor at evaluation, instead dashing from advocacy campaign to advocacy campaign. This may give the feeling of constant activity, but it completely misses the important chance to learn and improve upon experiences.

Your M&E is an important part of your organizational learning and continuous improvement. If your organization encourages learning this will create more knowledgeable staff and a flexible organization where people will accept and adapt to new ideas and changes through a shared vision.

The Process

An organization that learns and wants its people to learn, should try to follow certain concepts in learning techniques and mold itself to accommodate for a number of specific attributes. These include:

  • Thrive on Change
  • Encourage Experimentation
  • Facilitate Learning from the Surrounding Environment
  • Facilitate Learning from Employees
  • Reward Learning
  • A Sense of Caring and Mutual Support
  • Communicate Success and Failure

What Type of Learning?

A ‘Learning Organisation’ is not simply about 'more training'. While training does help develop certain types of skill, a ‘Learning Organisation’ involves the development of higher levels of knowledge and skill. This includes four levels of learning:

  • Learning facts, knowledge, processes and procedures. Applies to known situations where changes are minor.
  • Learning new job skills which are transferable to other situations. Applies to new situations where existing responses need to be changed. Bringing in outside expertise is a useful tool here.
  • Learning to adapt. Applies to more dynamic situations where the solutions need developing. Experimentation; and deriving lessons from success and failure is important here.
  • Learning to learn. Is about innovation and creativity; designing the future rather than merely adapting to it. This is where assumptions are challenged and knowledge is reframed.

Characteristics of a 'Learning Organization'

Some of the key characteristics of a ‘Learning Organisation’ are given below

A Learning Culture

An organizational climate that nurtures learning.

  • Future, external orientation - these organizations develop understanding of their environment; senior teams take time out to think about the future. Widespread use of external sources and advisors e.g. consultants
  • Free exchange and flow of information - systems are in place to ensure that expertise is available where it is needed; individuals network extensively, crossing organizational boundaries to develop their knowledge and expertise.
  • Commitment to learning, personal development - support from top management; people at all levels encouraged to learn and learning is rewarded.
  • Valuing people – ideas and creativity are stimulated, made use of and developed. Diversity is recognized as a strength. Views can be challenged.
  • Climate of openness and trust - individuals are encouraged to develop ideas, to speak out, to challenge actions.
  • Learning from experience - learning from mistakes is often more powerful than learning from success. Failure is tolerated, provided lessons are learnt

Key Management Processes

Management processes that encourage interaction across boundaries. These are infrastructure, development and management processes, for example:

  • Strategic and Scenario Planning - approaches to planning that go beyond the numbers, encourage challenging assumptions, thinking 'outside of the box'. They also allocate a proportion of resources for new challenges.
  • Competitor Analysis - as part of a process of continuous monitoring and analysis of all key factors in the external environment, including political factors.
  • Information and Knowledge Management - using techniques to identify, audit, value (cost/benefit), develop and exploit information as a resource.
  • Capability Planning - profiling both qualitatively and quantitatively the competencies of the organization.
  • Team and Organisation development - the use of facilitators to help groups with work, job and organization design and team development - reinforcing values, developing vision, cohesiveness and a climate of stretching goals, sharing and support
  • Performance Measurement - finding appropriate measures and indicators of performance; ones that encourage investment in learning.
  • Reward and Recognition Systems - processes and systems that recognise acquisition of new skills, team-work as well as individual effort, celebrate successes and accomplishments, and encourage continuous personal development.

Kaizen

Another very similar concept to a ‘Learning Organisation’ is Kaizen. This is a key Japanese management philosophy that means ‘improvement’. Kaizen strategy calls for never-ending efforts for improvement involving everyone in the organization - managers and workers alike.

In practice, Kaizen can be implemented in organizations by improving every aspect of the work process in a step-by-step approach, while gradually developing employee skills through training education and increased involvement. The principles in Kaizen implementation are:

  • Human resources are the most important company asset
  • Processes must evolve by gradual improvement rather than radical changes
  • Improvement must be based on statistical/quantitative evaluation of performance (quite difficult to apply in the animal protection environment)

Support throughout the entire structure is necessary to become successful at developing a strong Kaizen approach. Management as well as workers need to believe in the Kaizen idea and strive toward obtaining the small goals in order to reach overall success. Therefore, all members of an organization need to be trained in a manner to support this. Resources, measurements, rewards and incentives all need to be aligned to and working with the Kaizen structure of ideas.

The Kaizen Mindset includes the following:

  • Not a day should go by without some kind of improvement being made somewhere in the company
  • Mission-driven strategy for improvement - any management activity should eventually lead to increased mission achievement
  • Quality first: professionalism and quality as goals
  • Recognition that any organization has problems and establishing culture where everyone can freely admit these problems and suggest improvement
  • Problem solving is seen as cross-functional systemic and collaborative approach
  • Emphasis on process - establishing a way of thinking orientated at improving processes, and a management system that supports and acknowledges people's process-orientated efforts for improvement
  • A positive, win-win attitude, not a blame culture

When Should You Monitor/Evaluate?

macaw-edited-for-article

It is recommended that advocacy activities are always monitored to ensure that they are kept on track. But this is particularly important where new advocacy activities are carried out. For example, if you develop a new communications material (say a poster, or a newsletter) you should monitor to see if you get more phone or mail responses from new supporters.

The frequency of monitoring will depend on the urgency and importance of your advocacy project, and how this fits in with your usual reporting procedures. If action is urgent and important, there should be frequent monitoring (at least monthly). As a general guideline, some organizations require a six-monthly activity review, in the form of a written report.

Where you are working in an advocacy coalition or alliance, or with an advocacy partner, there should be at least one formal review every year. You might want to hold more frequent reviews with some key partners (e.g. for a funded project or a major advocacy campaign).

It should be remembered that monitoring can also include verbal reports (e.g. reporting progress against plan to your manager or the coalition/alliance) and/or written reports (e.g. monthly reports). You will need to agree: lines of reporting (e.g. you can report to your boss or the coalition/alliance), the regularity of reports, and whether these are verbal or written.

Evaluation should always be carried out on completion of advocacy activities. It should also be carried out periodically on longer-duration activities.

Donors may have their own requirements on the frequency of monitoring and evaluation needed.

It is also recommended to bring colleagues (and volunteers) involved in each advocacy program area together on a regular basis, to share experiences and good practice, identify lessons and areas to build on, and assess progress towards advocacy objectives.

For any of this to work effectively, organizations/partners and volunteers need to understand the purpose and process of the review, and what their roles are within it.

It is also recommended to close the gap between daily activities and the ‘big picture’, by bringing planning and review schedules at different levels into line with each other. Furthermore, concentrating on outcomes enables us to move away from lists of activities (which show we are occupied, but not necessarily productive…) towards concrete achievements.

Log in to update your WAN Directory listing