Performance evaluation of project expenditures in fiscal budget unitsOperation GuidecatalogueTabl

Performance evaluation of project expenditures in fiscal budget units
Operation Guide
catalogue
Table of Contents I
Table III
Figure III
Column IV
1 Introduction 1
1.1 Background 1
1.2 Purpose 1
1.3 Content 1
1.4 Use 4
2 Basic Concepts of Project Performance Evaluation 5
2.1 Definition of Performance Evaluation 5
2.2 Principles of Performance Evaluation 5
2.4 Criteria for Performance Evaluation 6
Step 7 of 2.5 Performance Evaluation
3. Organization of evaluation tasks 9
3.1 Organizational Model of Evaluation Tasks 9
3.2 Responsibilities of Evaluation Managers 9
3.3 Performance Evaluation Team 9
4. Develop an evaluation task outline 11
4.1 Function 11
4.2 Writing Steps 11
5 Performance Evaluation Design 15
5.1 Preliminary Preparation 15
5.2 Development Performance Evaluation Framework 16
5.3 Completion of Performance Evaluation Implementation Plan 27
6 Implementation of Performance Evaluation 30
6.1 Evidence Collection 30
6.2 Evidence organization 35
6.3 Analysis 36
6.4 Performance Rating 37
6.5 Forming Evaluation Conclusions, Lessons Learned, and Recommendations 39
Report on 7 Evaluation Results 41
7.1 Writing of Executive Summary 42
7.2 Main Report 42
7.3 Attachment 47
7.4 Communication with Stakeholders 47
7.5 Final draft of report formation 47
8 Use and diffusion of evaluation results 48
8.1 Use of Evaluation Results 48
8.2 Diffusion of Evaluation Results 49
9 Quality Control of Performance Evaluation 50
9.1 Characteristics of High Quality Performance Evaluation 50
9.2 Quality control at different stages 50
Attachment 53
Attachment 1 "Evaluation Task Outline" Format 53
Attachment 2 "Performance Evaluation Implementation Plan" Format 54
Attachment 3 Format of Performance Evaluation Report for Fiscal Expenditure Projects 57
surface
Table 4-1 Evaluation Task Outline Evaluation Task Implementation Plan Template 14
Table 5.1 Basic Project Information Table 16
Table 5.2 Performance Evaluation Framework Structure 17
Table 53 Applicability of Key Evaluation Questions 20
Table 5.4 Development of Correlation Evaluation Indicators - Taking Liaoning Urban Transportation Project as an Example 22
Table 5 Development of Efficiency Evaluation Indicators - Taking Liaoning Urban Transportation Project as an Example 23
Table 56 Development of Performance Evaluation Indicators - Taking Liaoning Urban Transportation Project as an Example 24
Table 57 Development of Sustainability Evaluation Indicators - Taking Liaoning Urban Transportation Project as an Example 25
Table 58 Advantages and Disadvantages of Evidence Collection Methods 27
Table 6.1 Case Study Methods for Evidence Collection - Using Liaoning Urban Transportation Project as an Example 31
Table 6 List of Interviewees 32
Table 6 List of Participants in the Symposium 34
Table 7 1 Key Project Information 43
chart
Figure 1 Performance Evaluation Operation Guidelines Framework 3
Figure 21 Project Logic Model 6
Figure 5.1 Performance Evaluation Design Process 15
Figure 5.2 Development process of performance evaluation framework during the design phase 18
Figure 71 Performance Evaluation Report Writing Process 42
column
Column 4 1 Checklist of Verification Issues for High Quality Evaluation Task Outline 14
Column 5: Principles for Developing Evaluation Indicators 21
Column 5 2 Introduction to Liaoning Urban Transportation Projects 21
Column 5 3 Performance Evaluation Evidence Sources 26
Column 6 1 General Rules for Evidence Collection 30
Column 6 2 Interview Record Format 32
Column 6 3 Schedule of Provincial Symposium on Urban Transportation Projects in Liaoning Province 33
Column 6 4 Forum Record Format 34
Column 6 5 Field Investigation Record Format 35
Column 6 6 Example of Evaluation Conclusion - Performance Evaluation of Hebei Highway Development Project 39
Column 6 7 Example of Lessons Learned - Performance Evaluation of Hebei Highway Development Projects 39
Column 6 8 Suggestions Example - Hebei Highway Development Project Performance Evaluation 40
Column 7 1 Key Points for Writing Performance Evaluation Reports 41
Introduction
1.1 Background
The Ministry of Finance has issued the Interim Measures for the Evaluation and Management of Financial Expenditure Performance (hereinafter referred to as the "Measures"). Since then, there have been unified departmental regulations for evaluating the performance of fiscal expenditures. The Measures have put forward clear requirements for the implementation and management of performance evaluation, and have made principled provisions on the responsibilities of all parties involved in performance evaluation, evaluation content, determination of evaluation indicators, evaluation methods, implementation procedures, report writing, and application of evaluation results. The Measures also point out that various regions can formulate management measures and implementation rules that are suitable for local conditions based on the Measures.
To this end, the Finance Department of XX Province referred to the "Guidelines for Performance Evaluation of International Financial Organization Loan Projects" issued by the Ministry of Finance and, based on the pilot performance evaluation, organized the compilation of the "Guidelines for Performance Evaluation of Financial Expenditure Projects" (hereinafter referred to as the "Guidelines") to guide government departments at all levels in the province to carry out performance evaluation work.
1.2 Purpose
This guide aims to provide specific operational guidance for government departments and evaluators at all levels in the province to conduct performance evaluation of fiscal expenditure projects (referred to as performance evaluation), in order to help them better understand, manage, and implement evaluation activities, and improve the standardization and quality of evaluation. This guide has three main objectives:
Assist government department managers, project implementers, and evaluators at all levels to better understand and master the basic concepts, general guidelines, methods, and other related knowledge of performance evaluation for fiscal expenditure projects;
Assist government departments at all levels to effectively carry out performance evaluation management work and control the quality of performance evaluation;
Guide evaluators to implement performance evaluation tasks in a standardized manner, thereby improving the quality of performance evaluation.
In addition, this Guide can also serve as an important reference for various levels of government departments to establish performance target management systems that are suitable for local and departmental realities.
1.3 Content
This guide includes 9 chapters of content. Chapter 1 is the introduction, which mainly explains the background, purpose, content, and use of the Guide. Chapter 2 is the basic concept of performance evaluation, mainly introducing the definitions, evaluation principles, criteria, and evaluation steps involved in this Guide. Chapters 3 and 4 are about performance evaluation management, mainly elaborating on the organization of evaluation tasks and the design and writing of evaluation task outlines. Chapters 5-7 discuss the design and implementation of performance evaluation, with a focus on the design, implementation, and report writing of performance evaluation. Chapter 8 is about the use and diffusion of evaluation results, introducing how to use evaluation results and how to spread evaluation results. Chapter 9 is the quality control of performance evaluation. Quality control is crucial for performance evaluation and is one of the important tasks for evaluation managers. Quality control runs through the entire process of performance evaluation, from design, implementation to report writing.
Figure 1-1 Framework for Performance Evaluation of Fiscal Expenditure Projects
1.4 Use
This Guide applies to the expenditure items of fiscal fund arrangements involved in the Measures (hereinafter referred to as projects). The users of this guide include personnel responsible for performance evaluation management in various levels of financial departments and other government departments, evaluators involved in performance evaluation of fiscal expenditure projects, consulting experts, and other relevant personnel. The content of this guide is tailored to the needs of different users. Chapters 1, 2, and 8 are suitable for all users to read and use, Chapters 3, 4, and 9 are mainly for performance evaluation managers to read and use, and Chapters 5, 6, and 7 are mainly for evaluators and consulting experts to read and use. Of course, we encourage users to read the entire content of the Guide in order to better understand and carry out performance evaluation work.
Basic concepts of project performance evaluation
2.1 Definition of Performance Evaluation
The definition of performance evaluation is a foundation for communication and understanding among all parties involved in project performance evaluation. The performance rating referred to in this guideline refers to the objective and fair evaluation of fiscal expenditure output and effectiveness by the financial and budget departments (units) based on the set performance goals, using scientific and reasonable evaluation methods, indicator systems, and evaluation standards.
2.2 Principles of Performance Evaluation
The principles of objectivity and fairness. Evaluation activities should be carried out based on performance evaluation criteria, key evaluation issues, etc., to ensure the authenticity of evaluation results.
Scientific and standardized principles. The design of performance evaluation should be scientific and operable, and the implementation of evaluation should follow standardized procedures and methods.
Reliability principle. Performance evaluation should report both aspects of project success and aspects of project failure.
Goal oriented principle. Performance evaluation is guided by project goals, focusing on evaluating the relevance of project goals, the degree of goal achievement, efficiency relative to goal achievement, and the sustainability of project effectiveness.
The principle of participation. All stakeholders of the project, including project management, project implementation units, project beneficiaries, relevant experts, etc., should participate in the evaluation process in an appropriate manner as much as possible.
2.3 Logical Model of the Project
The design of fiscal expenditure projects should use logical models. The project logic model is an effective tool for understanding the causal relationship between input and output. The project logic model includes five key elements: input, activity, output, effectiveness, and impact. The logic behind it is to invest certain resources in a project in order to carry out necessary activities; At least some activities will generate some output, such as products and services; The use of these outputs should bring about some kind of change.
Inputs: Refers to various resources used for project implementation, including human, financial, physical, time, and professional skills resources.
Activities: What we do. Refers to the actions and work of mobilizing and utilizing various resources to achieve project goals, which can be expressed using verbs (such as development, construction, training, research, etc.).
Outputs: What we produce. The products and services generated through the implementation of various activities are tangible and computable. For example, medical services were provided to 50 people, 100 primary school teachers were trained, and 100 kilometers of roads were built.
Outcomes: Why do we do it. Refers to the direct short-term or medium-term effects that can be increased, reduced, enhanced, improved, or maintained through the use of project outputs. For example, the construction of new roads has shortened travel time and improved transportation efficiency. Setting clear results is crucial for designing and constructing a results oriented monitoring and evaluation system. Effectiveness cannot be directly assessed, but must be transformed into a set of key indicators that can provide information on whether effectiveness has been achieved.
Impact: refers to long-term or comprehensive changes caused by the accumulation of results, similar to strategic goals. For example, improving transportation efficiency reduces transportation costs (effectiveness), thereby promoting local economic development (impact).
Figure 2-1 illustrates the logical model of a training project as an example.
Figure 2-1 Project Logic Model - Taking a Training Project as an Example
2.4 Criteria for performance evaluation
This guide adopts the following four evaluation criteria, namely relevance, efficiency, effectiveness, and sustainability.
2.4.1 Correlation
Relevance refers to the degree to which project goals align with government development policies and priorities, departmental/industry development plans, and the needs of beneficiary groups. By evaluating the relevance, it is possible to check whether the project design and implementation are tailored to local actual needs and address local issues.
2.4.2 Efficiency
Efficiency refers to the economic efficiency of converting resources/inputs into results. The content of efficiency evaluation includes the time efficiency of project implementation, fund utilization, output under the use of established resources, and the economic internal rate of return (EIRR) of the project.
The time efficiency of project implementation is evaluated to determine whether the project has started as planned and completed within the planned implementation period. The use of funds examines whether the use of project funds is carried out in accordance with the budget. The output situation under the use of established resources refers to whether the project has achieved all expected outputs. The economic internal rate of return of a project refers to the discount rate at which the cumulative net present value of the economy during the project cycle is equal to zero. It is a relative indicator reflecting the project's contribution to the national economy and society.
2.4.3 Effect
Effect refers to the degree of achievement of project goals, which is also reflected in the benefits of the target group. The effectiveness is evaluated from two aspects: the degree of achievement of project goals and the degree of project targeting. The project goal is the expected outcome during project design, which refers to the results to be achieved through a series of project activities, usually manifested as positive effects on society, economy, community, and target audience. Project targeting refers to examining whether the project results are truly enjoyed by the target group. That is to say, the evaluation of project effectiveness depends not only on the degree to which the project achieves its expected goals, but also on whether its results (usually manifested as the products or services provided by the project) benefit the target audience.
2.4.4 Sustainability
Sustainability refers to the possibility of continuous operation and effectiveness of a project after completion. It evaluates various factors that ensure the continuous operation and effectiveness of the project, including the continuity of project management or operating institutions, maintenance and utilization of project outputs, and institutional guarantees. In addition, due to the strong focus of the financial department on project repayment, sustainability also evaluates the repayment ability of projects.
2.5 Steps for Performance Evaluation
Performance evaluation generally includes the following steps, and the detailed content and methods of each step will be explained in subsequent chapters.
Step 1: Preliminary preparation
Preliminary preparation includes determining performance evaluation tasks and evaluation organization implementation models (see Chapter 3), preparing evaluation task outlines (see Chapter 4), and so on.
Step 2: Evaluate the design
Design a detailed evaluation implementation plan based on the evaluation task outline, including developing a performance evaluation framework, selecting evidence collection methods, designing a list of questions for face-to-face interviews, symposiums, and field investigations, and arranging personnel, time, and funding (see Chapter 5).
Step 3: Evaluate implementation
Conduct performance evaluation according to the designed plan, including evidence collection and organization, evidence analysis, formation of evaluation conclusions, lessons learned, and suggestions (see Chapter 6).
Step 4: Report the evaluation results
Write a performance evaluation report in the format of Annex 3 of this Guide, and communicate with stakeholders during this process, incorporating their perspectives and opinions into the evaluation report (see Chapter 7).
Step 5: Use and diffusion of evaluation results
The project management department utilizes evaluation results to solve problems, improve management, or provide a basis for decision-making. The financial department will link the evaluation results with the budget arrangements for future years (see Chapter 8).
3. Organization of evaluation tasks
The specific evaluation tasks are determined by the annual evaluation plan issued by the financial department in the form of documents.
3.1 Organizational Model of Evaluation Tasks
The evaluation task can be organized and implemented by the management department (finance department and budget department at all levels), or it can be entrusted to external professional institutions or experts as needed, and a commission contract can be signed with them. Regardless of the approach taken, the management department should assign an evaluation manager and form a qualified evaluation team to conduct performance evaluations.
3.2 Responsibilities of Evaluation Managers
The evaluation manager should be a management personnel familiar with the evaluated project and should fulfill the following responsibilities:
Write a performance evaluation task outline;