A complete and adequate description of the components of a program is essential to evaluate its implementation.

The components are:

– the strategies,

– activities,

– behaviors,

– forms of communication and

– technologies for the implementation of the program and the specification of the beneficiaries and the place where the implementation takes place.

Proper and accurate identification of the program components will allow an assessment of which aspects of the program were implemented as planned and which factors might influence implementation differences.

The correct specification of the components was observed to assess concerns regarding the scope of the program (intended beneficiaries). In addition, it is conjectured about the possible links between the results of the implementation and the results of the program itself (in terms of production, intermediate results, impacts, etc.)…

At the same time the specification (or detail) of the contents of the program is a requirement of the evaluation process.

The very initiative to plan and carry out the evaluation process contributes to a more adequate and realistic specification of the content of the program. This is an important condition to ensure that the program is more effective (because the internal consistency of the program has gone through a preliminary review) and, secondly, that the evaluation of results and impact is more effective since the performance of the program is compare with more consistent and realistic goals and expectations.

To allow the evaluation that the process can improve the design and specification of a public program, some techniques can be used.

1 – Formative evaluation: based on data collected from pilot projects and beneficiaries on the realization of a particular intervention and providing information on the feasibility of certain activities and instruments and to what extent they are appropriate for the design plan and the beneficiaries provided;

2 – Verification of “evaluability” systematic set of procedures for the correct development of the theory behind a public program, detailing and clarifying the intended uses for the data in the evaluation process, before the start of a full-scale evaluation.

Its most important steps include (Scheirer, 1994: 49-50):

a) Engage key policy makers, managers and staff through a series of meetings to clarify their expectations for the program and the evaluation itself;

b) Using a model called a matrix logic diagram, detailing the expected causal relationships between three aspects of the program: resources assigned to the program, execution of specific activities planned by the program, and expected results;

c) Refinement of the theory behind the program through an interactive process, using visits to project sites and available information, to examine the reality of field operations and the extent to which the proposed theory is plausible;

d) Clarify the intended uses for information obtained from the evaluation, through discussions with policy makers and program managers, including program changes;

e) Use of theory to help in the specification of the program. application of theories relevant to the substantive issue from which the program arises, and the use of data to elucidate underlying processes.

This type of evaluation process is important not only for specifying program content, but also for linking program activities to income measures (indicators) that will be used in subsequent impact evaluations.

The term theory here refers to the interrelated principles that explain and assume the behavior of a person, group, or organization.

Chen (1990) distinguishes two types of theories:

– The regulations, which define what a program should be and

– the causal, which empirically describes the causal relationships between the proposed solutions (including contextual factors) and the result.

The central problem in this case is to investigate the effectiveness of the program and to achieve this purpose, it uses the mechanisms to establish causal relationships between the actions of a program and the final result.

The purpose of such an evaluation can be defined as the identification of the net effects of a social intervention. Like the goal assessment, this approach is carried out after the end of the program or the same steps.

Evaluation processes – This type of evaluation systematically investigates the development of social programs with the purpose of measuring the coverage of the social program, establishing the degree to which the goal is being achieved and, especially, monitoring its internal processes. The objective is to detect possible defects in the development of procedures to identify barriers and obstacles to their implementation and generate important data for their reprogramming, through the registration of events and activities.

Thus, the appropriate use of the information produced during the development of the program allows changes in its content during execution. Unlike previous approaches, therefore, this method of evaluation is carried out simultaneously with the development of the program, also called formative evaluation. Its implementation requires, however, that we be able to design the flows and processes of the program.

In addition, it presupposes the existence of an adequate management information system, which serves as a basis for the work of managers and evaluators when appropriate.

An application of the evaluation methodology of social programs:

A comprehensive evaluation system using methodologies that provide for the evaluation of results and evaluation processes. In addition, the scenarios and forms of operation used in the proposed model.

Evaluation of results:

Here, outcomes are defined as immediate outcomes, long-term outcomes (impacts), and medium-term outcomes (impacts).

For the evaluation, the use of impact indicators is suggested to measure long-term results, related to the objectives of the program, and output indicators to measure immediate and medium-term results. Output indicators measure the effects of the program: on the target population as a whole and between u

serve the program. In the first case, two types of output indicators should be proposed, with field research or with the help of existing databases and/or inputs:

– Degree of global coverage:

Measures the coverage rate of the program’s target population. Both the deficit and the surplus of beneficiaries are the reasons for the changes in the route. The first demonstrates the need for expansion, and the second, that there is a waste of resources (not eligible since the target population is benefiting);

– The degree of coverage varies according to the program:

It measures the participation of different subgroups of the proposed target population. This rate may represent discrimination (or bias) in the selection of program clients based on region, age, gender, etc. As for the second point, that is, the evaluation of the results for the users of the program, it can be used to measure indicators. of benefits, which take into account the specific objectives of each program or project.

Rob Vos (1993) gives some examples of the most used indicators between the users of the program and the target population:

1 – for nutrition programs – malnutrition rates by age, mortality and morbidity;

2 – for education programs – illiteracy, repetition, evasion rates; schooling coefficients and degrees of education;

3 – health programs – mortality rates in general, infant mortality, maternal mortality and birth rates, fertility and life expectancy at birth;

4 – for housing programs – quantitative housing deficit, quality of housing construction and availability of basic services. Indicators show the means of input or the resources available to achieve the objectives. Scarce and inadequate resources (in financial terms, manpower, equipment, etc.). It almost always tends to undermine the expected results.

Vos (1993) mentions some examples of more common input indicators such as:

a) – to nutrition programs – availability of food per person;

b) – to education programs – student/teacher, student/school ratio, number of series offered by the school and availability of teaching materials for students;

c) – for health programs – number of doctors per capita, for health posts per capita; of beds per inhabitant and vaccines available per capita.

But the access indicators make it possible to identify the determinants that make effective use of the resources available in the programs to achieve the planned goals. The most common are:

a) – to health programs – the number of medical consultations per equivalent adult; distance to the nearest health service, disposable income per family (so useful to facilitate the purchase of medicines, for example) and cultural factors;

b) – for education programs – outside of school, curricular adequacy and available income for the family (to enable, for example, the purchase of school supplies).

In addition, the use of questionnaires allows knowing customer satisfaction, being a good quality indicator but not the only one or the most complete. In this sense, it is still possible to establish composite indicators through the construction of indices formed by a set of attributes defined from the characteristics of the service.

Evaluation process The evaluation process can be defined as a way to identify the actual content of a public program, where it is proceeding as planned, is it reaching its intended audience, and whether the benefits are being distributed across the board. planned intensity (Scheirer, 1994: 40).