Management Effectiveness

Requirement in the Programme of Work: Goal 4.2 - To evaluate and improve the effectiveness of protected areas management

Target: By 2010, frameworks for monitoring, evaluating and reporting protected areas management effectiveness at sites, national and regional systems, and transboundary protected area levels adopted and implemented by Parties.

The following account on management effectiveness is mostly taken from (1) Evaluating effectiveness: A framework for assessing management effectiveness of protected areas 2nd edition, IUCN WCPA and University of Queensland, Gland Switzerland and Brisane Australia, by Hockings, M, S Stolton, F Leverington, N Dudley and J Corrau (2006) and (ii) from the presentations of Jamison Ervin of Nature Conservancy for the Caribbean workshops.

What is Management effectiveness evaluation?

The evaluation of management effectiveness is generally achieved by the assessment of series of criteria (represented by carefully selected indicators) against agreed objectives or standards. Management effectiveness evaluation is defined as the assessment of how well protected areas are being managed – primarily the extent to which management is protecting values and achieving goals and objectives. The term management effectiveness reflects three main ‘themes’ in protected area management:

• design issues relating to both individual sites and protected area systems;
• adequacy and appropriateness of management systems and processes;
• and delivery of protected area objectives including conservation of values.

IUCN-WCPA has developed a management effectiveness evaluation framework which provides a consistent basis for designing evaluation systems for protected areas. Components of the IUCN framework include design of systems and individual protected areas (context and planning), appropriateness of management systems and processes (inputs and processes), and delivery of protected area objectives (outputs and outcomes). These components are divided into six elements, elaborated below, each comprising a number of evaluation indicators to assess management effectiveness.

Assessment needs to be made in the context of the protected area, so first assessments need to gather data on issues relating to the areas values, threats and opportunities, stakeholders, and the management and political context. Management starts with planning of strategies needed to fulfil the vision, goals and objectives of protection and to reduce threats. To put these plans in place and meet management objectives, managers need inputs (resources) of staff, money and equipment. Management activities are implemented according to accepted processes (i.e. best practices); which produce outputs by completing activities outlined in work plans. The end result of management is the achievement of outcomes, i.e. reaching the goals and objectives set for the biological conservation, economic development, social sustainability or cultural heritage of the protected area.

Design and issues

Context: Assessment of importance, threats and policy/cultural environment

The context review, although not an analysis of management, helps managers put their decisions in context and helps prioritise action based on biological, cultural and political information. The context assessment can help managers answer the following questions:

• Why is the protected area important?
• What are the threats facing the protected area?
• Is the government supportive of the protected area?
• What is the role and effect of stakeholders on protected area management?

Planning: Assessment of protected area design and planning

The assessment of management planning draws from the findings of the context assessment, Planning involves understanding the direction and objectives of management and deciding on the strategies that are required to achieve these, within the context of the protected area’s status and characteristics. The planning assessment can therefore help managers to answer three questions:

• Is the legal status and tenure of the protected area is clear?
• How do the protected area’s characteristics (e.g. size and shape) influence management?
• Is there an adequate management planning process?

Appropriateness of management systems and processes

Inputs: Assessment of resources needed to carry out protected area management

Allocating funds and staff time and developing infrastructure to fulfil management needs should be linked to and, in large part, directed by planning decisions. Although protected area management plans rarely provide specific commitments of funds and staff, they establish the basis for short-term or annual operational planning in which decisions about allocation of resources (budgets) are made. An assessment of resources allows managers to identify shortfalls in staff, funds and equipment in relation to planned activities. Assessment considers the resources required for effective management, measures these against what is available and looks at the effectiveness of resource use. The assessment thus has to address two important questions:

• Has the protected area got the resources needed to meet its management objectives?
• Are resources used in the best way?

Processes: Assessment of the way in which management is conducted

The use of the best possible management practices is essential for effective protected area site or system management. Assessment can show if best practice is being applied in day-to-day management and if management practices can be or need to be improved. The process assessment asks:

• Are agreed policies and procedures for management in place?
• Are the best systems and standards of management being followed?
• How can the management practices be improved?

Delivery of protected area objectives

Outputs: Assessment of the implementation of management programmes, actions and services

The results of management activity can be considered in two ways – the outputs and outcomes. First, there are the direct outputs produced by management, which usually consist of a set of products or services (e.g. an area of controlled burn, number of invasive species eradicated, length of track maintained, numbers of guided walks conducted or numbers of anti-poaching patrols). The output assessment looks at what managers and their staff have been doing and whether the targets set in management plans or annual work programmes have been met. An output assessment therefore asks questions such as:

• Has the the number number or level of products and services been delivered?
• Have planned actions, tasks and strategies been implemented?

Outcomes: Assessment of the extent to which objectives have been achieved

The outcome assessment considers impacts of management on the broad objectives set for the protected area. It looks beyond the implementation of immediate management activities (i.e. the outputs) to the longer term success of the protected area by attempting to answer the question:

• Has management resulted in the achievement of the objectives of, and desired outcomes for, the protected area?

Approaches to outcome evaluation ideally involve long-term monitoring of the condition of the biological and cultural resources of the system/site, socio-economic aspects of use and the impacts of the management on local communities. In the final analysis, outcome evaluation is the true test of management effectiveness. Even if other aspects of management are highly effective, a protected area will clearly not be effective if it loses its core values.

Summary of the IUCN-WCPA Framework for assessing management effectiveness of protected areas and protected area systems

Approaches

Since the development of WCPA framework in 2000, technical experience increased rapidly resulting in a range of assessment systems based upon the framework. There are now three basic approaches ;

1. In-depth, evidence based assessments aimed at building monitoring systems and long-term understanding of management in an individual protected area, such as the Enhancing our Heritage system being developed for World Heritage sites.
2. System-wide peer–based assessment developed specifically for use on a system-wide scale such as the WWF RAPPAM system and the systems developed in Finland, Catalonia (Spain) and New South Wales (Australia).
3. Scorecard expert–based assessments

There are four major steps in assessing protected area management effectiveness (i) getting started (ii) gathering data (iii) analyzing results and (iv) integrating into capacity assessments

Getting started

Important considerations for getting started include (i) who – setting up an organizing committee consisting of key agency staff, NGOs, donors and local people, who have detailed knowledge about policies, field level conditions and interagency dynamics as well as organizational, communication, facilitation and analytical skills (ii) Where – the scope of assessment, either an entire system or an individual protected area. (iii) how - gathering information through workshops, direct interviews using questionnaires based on WCPA framework. For example WWF- RAPPM questionnaire contain more than 100 questions and relies on qualitative scoring by workshop participants.

Gathering Data

This includes four parameters of information collection: (i) significance and vulnerability gathering

(ii) Threats and Pressures include : Any human activity that impairs biodiversity; Applies to existing and future threats; scored by extent, severity and permanence (1-64); and can incorporate multiple data sources

(iii) Management elements

Assessment Methodologies: General

(iv) System – wide enabling environment

Analyzing data

This includes (i) management elements (ii) threats and pressures, and (iii) cross–cutting analyses. Management elements in turn consist (a) identifying broad trends, (b) identifying specific weaknesses, (c) identifying themes. Threats and pressures includes (i) analyzing single threat, (ii) comparing threats, (iii) comparing threats across protected areas, and (iv) comparing vulnerability. Cross cutting analysis includes understanding threat and significance, and (ii) understanding the relationship between effectiveness, threat and significance.

Integrating into capacity assessments

Tools and Rresources

1. RAPPAM Ervin, J. (2003). WWF: Rapid Assessment and Prioritization of Protected Area Management (RAPPAM) Methodology. WWF, Gland, Switzerland.

The WWF Rapid Assessment and Prioritization of Protected Area Management (RAPPAM) methodology provides a country-wide assessment of the effectiveness of protected area management, threats, vulnerabilities and degradation. The RAPPAM methodology is already available in the following languages: English, French, Spanish, Portuguese, Russian, Mongolian, Bulgarian, Georgian, Bahasa Indonesia, Khmer,. For a download of the English version, please visit http://www.panda.org/about_wwf/what_we_do/forests/our_solutions/protection/rappam/index.cfm

2. World Bank/WWF Tracking Tool

Stolton, S., Hockings, M., Dudley, N., MacKinnon, K. and Whitten, T. (2003). Reporting Progress in Protected Areas: A Site-Level Management Effectiveness Tracking Tool. World Bank/WWF Alliance for Forest Conservation and Sustainable Use.

Commonly referred to as the Tracking Tool, this rapid assessment is being used in all World Bank/WWF Alliance protected area project sites to track changes in effectiveness of management. The system has also been adopted by the Global Environment Facility as the basis for tracking changes in management effectiveness in all GEF protected area project sites. A version of the Tracking Tool has been developed for Marine Protected Areas by the World Bank (see listing under Marine Protected Areas. The Tracking Tool is available in the following languages: Bahasa Indonesia, Chinese, English, French, Khmer, Lao, Mongolian, Portuguese, Romanian, Russian, Spanish and Vietnamese. For a download of the English version, please refer to http://www.panda.org/about_wwf/what_we_do/forests/our_solutions/protection/rappam/tracking_tool/index.cfm

3. WWF/CATIE methodology

Cifuentes, M. and Izurieta Valery, A.A. (1999). Evaluation of Protected Area Management Effectiveness: Analysis of Procedures and Outline for a Manual.

The WWF/CATIE evaluation methodology was developed as a structured, sequential and simple-to-use evaluation methodology, based on a scoring system which was developed to address the special needs of protected areas in Latin America.. Together with the PROARCA-CAPAS methodology, the WWF-CATIE system has been widely applied across Central America. Available in English and Spanish versions. http://www.iucn.org/themes/WCPA/pubs/mgteffectpdfs/PARKSfin_esp.pdf http://www.iucn.org/themes/WCPA/pubs/mgteffectpdfs/Art_Eng.pdfCont.

4. PROARCA-CAPAS scorecard Courrau, J. (1999). Strategy for monitoring and management of protected areas in Central America. USA, PROARCA-CAPAS Program, The Nature Conservancy.

The PROARCA/CAPAS system is based on the ‘scoring model’ to evaluate protected area management developed by TNC in the early 1990’s. The PROARCA/CAPAS methodology includes assessment of 43 indicators in five fields; natural and cultural resources, social, administrative, political/legal, and economic/ financial. Available online at: http://www.iucn.org/themes/wcpa/pubs/mgteffectpdfs/c.america-eng.pdf

5. National Parks and Conservation Association State of the Parks The National Parks Conservation Association’s State of the Parks program aims to provide accurate and timely information on natural and cultural resource conditions and stewardship capacity for selected national parks in the USA. Available online at: http://www.npca.org/across_the_nation/park_pulse/

6. The Nature Conservancy –Conservation Action Planning Low, G. (2003). Landscape-scale Conservation: A Practitioner’s Guide. The Nature Conservancy, USA. TNC has developed an integrated process for planning, implementing and measuring conservation success for its conservation projects. This process is called the “Conservation Action Planning (CAP)” process. The CAP Toolkit and supporting material is available at: http://conserveonline.org/workspaces/cap/CAP_Toolkit.zip/file_view

7. World Heritage Areas Enhancing our Heritage: monitoring and managing for success in natural World Heritage sites. Hockings, M., Stolton, S., Courrau, J.,Dudley, N. and Parrish, J. (2004).The World Heritage Management Effectiveness Workbook: How tobuild monitoring, assessment andreporting systems to improve the management effectiveness of naturalWorld Heritage sites. Revised Edition. University of Queensland, Australia.

Evaluation methodology developed for detailed site level assessment. The Workbook provides guidelines and assessment tools for each element of the WCPA Framework. These tools have been designed to allow specific needs and circumstances of the site to be taken into account and to provide a means for integration of existing monitoring data into the evaluation system. While designed specifically to meet the needs of natural World Heritage sites, the methodology is applicable to any protected area. Available online at: http://www.enhancingheritage.net

8. Marine Protected Areas IUCN/NOAA/WWF Guidebook Pomeroy, R.S., Parks, J.E. and Watson, L.M. (2004). How is your MPA doing? A Guidebook of Natural and Social Indicators for Evaluating Marine Protected Area Management Effectiveness. IUCN, Gland, Switzerland and Cambridge, UK.

The guidebook provides a step-by-step process for planning and evaluating the management effectiveness of MPAs. It lists 42 MPA-specific indicators that MPA managers can choose to use for evaluating their site. The book draws on the work of the MPA Management Effectiveness Initiative, shaped by IUCN’s World Commission on Protected Areas (WCPA) - Marine and World Wild Fund for Nature (WWF). Available online at: http://www.effectivempa.noaa.gov/guidebook/guidebook.htmlCont.

9. Western Indian Ocean Guidebook Wells, S. and Mangubhai, S. (2004).Assessing Management Effectiveness of Marine Protected Areas: A Workbook for the Western Indian Ocean. IUCN Eastern AfricanRegional Programme, Nairobi, Kenya. Available online at: http://www.wiomsa.org/data/content/DOCUMENTS/2005112212511831IUCN%20BOOK%20part%201.pdf

10. World Bank MPA Scorecard Staub, F. and Hatziolos, M.E. (2003).Score Card to Assess Progress inAchieving Management ffectiveness Goals for Marine Protected Areas.The World Bank, Washington, DC,USA.

This marine version of the World Bank/WWF Alliance Tracking Tool was prepared by the World Bank for use in Marine Protected Areas. It is available for download in English, French and Spanish versions from: http://www.icriforum.org/mpa/MPAeffectiveness.html

11. Foundations of Success Foundations of Success (FOS) is a not-for-profit organization committed to working with practitioners to learn how to do conservation better through the process of adaptive management. The FOS website provides information and documentation on adaptive management and evaluation including the results of a comprehensive review of approaches to monitoring and evaluation in a range of fields including conservation. Website: http://fosonline.org/

12. Conservation Measures Partnership The Conservation Measures Partnership (CMP) is a partnership of conservation NGOs that seek better ways to design, manage and measure the impacts of their conservation actions. Two products from the CMP relevant to evaluation of management effectiveness are a Taxonomy of Direct Threats and Conservation Actions and a set of Open Standards for the Practice of Conservation. Both products are available from the CMP website at: www.conservationmeasures.org/CMP/

13. Selected Evaluation Studies Global studies WWF report on management of forest protected areas.Dudley, N., Belukurov, A., Borodin,O., Higgins-Zogib, L., Hockings, M.,Lacerda, L. and Stolton, S. (2004). Are protected areas working: Ananalysis of forest protected areas by WWF. WWF, Gland, Switzerland.

Analysis and report on the results of application of the World Bank/WWF Alliance Tracking Tool in over 200 forest protected areas in 37 countries. http://assets.panda.org/downloads/areprotectedareasworking.pdf

14. Management effectiveness evaluation of Finland’s protected areas Gilligan, B., Dudley, N., Fernandez de Tejada, A. and Toivonen, H. (2005). Management Effectiveness Evaluation of Finland’s Protected Areas. Nature Protection Publications of Metsähallitus. Series A 147. Study used an external team of evaluators who visited many of the protected areas and completed an assessment based around the elements in the IUCN-WCPA Framework combined with a RAPPAM based assessment completed by Agency staff. The report is available in electronic format at http://www.metsa.fi/mee.

15. Evaluation of management effectiveness of protected areas in Catalonia Mallarach, J.M. and Varga, J.V. (Eds) (2004). EI PEIN deu anys després: balanç I perspectives. Diversitas: 50, Universitat de Girona, Girona.

The entire methodology, including the description of all 85 indicators, and a 40 page summary of the findings can be found at the web site of Institució Catalana d’Història Natural at http://www.iec.es/institucio/societats/ICHistoriaNatural/Avaluacioespais.htm

16. ParksWatch ParksWatch is a watchdog and monitoring organization that works through partnerships with in-country NGOs and individuals to conduct on the- ground evaluations of national parks and other protected areas. Results from a series of evaluation studies of protected areas in Latin America are available online on the ParksWatch website at: http://www.parkswatch.org/main.php

17. NSW State of the Parks 2004 Department of Environment and Conservation (NSW). (2005). State of the Parks 2004. Department of Environment and Conservation, Sydney, Australia. http://www.epa.nsw.gov.au/sop04/index.htm

18. Marine protected areas in Western Indian Ocean Wells, S.M. (2004). Assessment of management effectiveness in selected marine protected areas in the Western Indian Ocean. IUCN Eastern Africa Regional Programme, Nairobi, Kenya.

A Workbook for assessing management effectiveness in MPAs in the WIOhas been developed, based on the workbook and methodology developed for World Heritage sites and using the WCPA/METF Framework. This report provides the results of testing the Workbook at eight pilot sites in Kenya, Tanzania and the Seychelles. Available for download from: http://www.icran.org/pdf/ICRAN_IUCN_ME_study_Eastern_Africa.pdf

19. Tasmanian Wilderness World Heritage Area.Parks and Wildlife Service. (2004).State of the Tasmanian Wilderness World Heritage Area – an evaluation of management effectiveness. Report No. 1, Department of Tourism Parks Heritage and the Arts, Hobart, Tasmania.

This report is the result of a long-term process of monitoring and evaluation established for the Tasmanian Wilderness World Heritage Area using an outcomes-based evaluation approach integrated into the management cycle for the site. The report is available on CD or can be downloaded from: http://www.parks.tas.gov.au

20. Enhancing our Heritage site reports Reports from project sites (Ecuador: Sangay National Park; Honduras: Río Plátano Biosphere Reserve; India: Kaziranga National Park; India: Keoladeo National Park; Nepal: Royal Chitwan National Park; Seychelles: Aldabra Atoll; South Africa: Greater St Lucia Wetland Park; Uganda: Bwindi Impenetrable National Park; United Republic of Tanzania: Serengeti National Park; Venezuela: Canaima National Park) included in the Enhancing our Heritage project are available from: http://www.enhancingheritage.net