To determine exactly what is the performance of a program we need to do an evaluation. The evaluation discussed here is participatory, that is it involves the people in the community in the process and decisions. Evaluation finds out the answers to specific questions with regards to the program. It helps to build up mutual responsibility and promotes dialogue among the people involved: program leaders, workers, people's organizations, and the people whom the program benefits. The evaluation should examine the total socio-political context and the social relations within which the situation which the people are addressing is embedded.

An evaluation system needs to be developed which not only measures the performance of the program and determines program effectiveness, but also one which has the people participating and learning in the evaluation process.

Evaluation of programs occurs for various reasons. Primarily the reasons can be summarized as follows (Ecumenical Center for Development, 1986):

a) To assess effectivity as well as efficiency of program delivery in responding to particular needs of various groups which benefit from it.

b) To serve as a venue for reflection: to gain insights and understanding from program experiences, so as to be able to conceptualize experiences within the context of one's specific direction or work experience.

c) To acquire a realistic basis for inferences and decisions

necessary for the programming of future action.

d) To keep funder up to date on the status of the program.

e) To provide information which will assist the funder and other local coordinating groups in formulating policies and guidelines relevant to the program's area of operations.

Evaluation differs from monitoring in that it answers more than the questions of what, where and when. Evaluation requires the answers to why and how. Monitoring also differs from program evaluation as it is an on-going periodic review of target indicators for purposes of control of the implementation process and to provide quick information feedback. Monitoring is carried out during the implementation of the program.

It is essential for the community to periodically look at what has been accomplished and what has not been accomplished by monitoring. This feedback on program execution requires the systematic collection and analysis of data on program performance. Through this assessment future program implementation can be improved. Through monitoring, deviations from plans are clearly identified and quickly corrected.

Monitoring involves a system of observation, reporting and control (Khan, 1978). It includes the following elements:

a) Relies partially on the information produced by internal program reports.

b) Requires the generation of additional program-related information, focused on critical implementation issues.

c) Involves recording of actual inputs, activities and outputs.

d) Involves comparative analysis of actual and planned estimates.

e) Involves diagnosis of implementation problems.

f) Gathers additional information through field visits and discussions with program staff to avoid implementation problems.

g) Warns program management of the need for corrective action.

h) Recommends corrective actions.

i) Incorporates feed-forward activities--identification and replication of successes.

Program monitoring involves measuring physical progress against plans and activity schedules, measuring financial progress against programed budget allocation and identifying problem areas.

The key to monitoring implementation is reviewing the schedules. The planned activity schedule and estimated costs for each activity are periodically compared with the actual time and cost used. This way trends and any future problems can be identified. For example, if the program activity is only one- quarter complete but one-half of the money has been used this should be reviewed. Also, activities which were identified as critical should be carefully monitored to ensure that they are being completed on schedule. Significant deviations are identified so that adjustments can be made without a serious loss of time and resources.

Finally, a report should be written to present the information on program progress. The report should provide the basic information necessary for designing future plans of action which are fitted to changing conditions. The report should be clear, concise and presented to the community. For this reason the report must be written in the local language or dialect.

Evaluation is carried out for the community itself that wants to assess progress. It is not done for the government or funding agencies. Before evaluation begins, decide exactly who will benefit and what the community sees as the thrusts of the evaluation.

Before beginning the evaluation, the community should decide on the thrusts of the evaluation. This can be accomplished in a pre-evaluation community dialogue. In general, the community may decide to concentrate on one thrust, or a combination of the following thrusts:

1) is the project/program producing the results expected of it - in other words, is it effective?

2) is the project/program providing value-for-money? Is it managed economically and efficiently?

3) is the project/program financially and organizationally accountable to the community?

If the answer to all three questions is yes, the community can report that all is well for program performance. If the answer to any one question is no, then the community will have to conduct whatever further examinations are necessary to allow them to make recommendations that, when implemented, will result in satisfactory arrangements and performance.

The evaluation should give priority to community level needs. In participatory evaluation all those in the program are involved to some degree in the decision-making process concerning the progress and future direction of the program.

The evaluation should be carried out by community people directly affected by the program, program staff and other people in the community. Evaluation is for the people. Evaluation is not done for the experts. The people should therefore carry out the evaluation. Those involved in the program on a day-to-day basis usually know a great deal about it. They should be the the center or beginning point int the evaluation. Sometimes outsiders can be used in the evaluation to provide the people in the program with a fresh look because they are not directly and personnaly involved. To maximize self-reliance it is best to involve all those involved in a program and to keep outside involvement to a minimum. Guide the evaluation with the motto, `everyone has something to contribute to the evaluation'.

Participatory evaluation is a process which is meant to raise the consciousness of the people. It is a search for solutions. It seeks to transform reality in the process of defining it. The purpose of the evaluation is to make decisions and change policies, not to strengthen decisions and policy makers (Fernandes and Tandon, 1983).

A comprehensive and action-oriented participatory evaluation is an examination that provides a constructive assessment of the extent to which the objectives and aspirations of the people are being achieved effectively, efficiently and with due regard to economy. It is important to understand the three E's (economy, efficincy and effectiveness) when beginning the evaluation process. They can be put into questions that the community may ask:

economy: Are we getting the right inputs at the best cost?

efficiency: Are we getting the most output from our inputs?

effectiveness: Are the outputs getting the results we want?

An evaluation assessment is essentially an evaluation plan done for a program which has been operating for a long period of time, usually more than six months. The evaluation assessment should identify the potential evaluation questions to be answered in a future evaluation. The assessment will identify the indicators that will answer the evaluation questions and also the data sources. This is important because frequently data will need to be collected on an on-going basis. Therefore, we must identify data needs in advance.

The evaluation assessment should include the following sections:

1) Program background and history

2) Program objectives (if stated)

3) Program logic model (Appendix 1)

4) Evaluation plan (Appendix 2)

The program background and history is simply a description of how the program started, who started it and the historical development of the program up to the present.

The program objectives are also stated if they have been formally developed. The objectives should be stated in concrete and measurable terms as much as possible.

The program logic model, as the example in Appendix 1 illustrates, is a model of the program flow. It is useful in drawing connections between activities, objectives and actual impacts.

The evaluation plan is a basic outline of how the specific evaluation questions will be answered. The evaluation questions should be formulated according to the parameters discussed in the next section. The development of evaluation indicators, data sources and methodologies is discussed in part 5.

The evaluation plan includes the following sections:

- Background of the Evaluation

- Objectives of the Evaluation

- The Evaluation Design

- Schedule of Activities

- Budget

The evaluation plan also outlines how the evaluation questions which were chosen by the community, will be answered. The sample evaluation plan (Appendix 2) gives a guide which can be used in the plan.

An evaluation framework is the same as an evaluation assessment except it is completed for new programs, usually before program implementation or operation. Much of the information needed in the evaluation framework can be obtained by referring to the social investigation notes of the community organizer. It is important that the framework answers questions which the people in the community want answered.

The evaluation study is the actual evaluation of the program. The evaluation study will be easier now because it is simply based on the plan developed during the evaluation framework or assessment. At this stage the community wants to know the progress of the program. Also, the community usually rethinks the basic philosophy and goals of the program and asks themselves for the reasons for existing as a program. Alternative programs may be suggested or found to better address the situation the community is trying to change. To do this, time must be taken to investigate conditions in the community and to analyze causes. The community should also evaluate the extent to which the program involves the participation of the people in the community and the education or awareness building value of the program. These aspects of the evaluation will be discussed further in the next section on evaluation questions.

The evaluation questions can be grouped under five headings. Grouping the questions which need to be answered under these headings will help simplify the evaluation.

. Rationale

. Community Development

. Objectives Achievement

. Impacts and Effects

. Alternatives

Is the program needed. What are the reasons for the programs continued existence. These are essential questions which must be addressed in the evaluation.

When we started the program we had certain ideas and reasons for doing so. We must now review or rethink these views which may included the following:

. Are the activities primarily to meet the needs of the

poorest sections of society?

. What is the situation or problem the program addresses?

. How did we look at the causes of the situation, then

and now?

. Change toward what?

. How does our program fit into this process of change?

These questions are important because they will shape our future plans, direction, the process of implementation and the people we involve. We should look at the problems in a socio- political context. If we view our problem as mainly a technical problem we tend to give technical solutions. For example, we have too many malnourished children. In a purely technical analysis it would probably be recommended that we have a feeding program, select the third degree malnourished and feed them in special centers until they are well enough to go home. But the situation at home remains the same and the parents still cannot afford to eat adequately. If this malnourishment problem is viewed in a socio-political context we may determine that while a large segment of the community lacks adequate food, another powerful segment has more than enough. It then becomes a question of power and control over the production and distribution of the food. With this viewpoint programs can be developed to empower the disadvantaged in the community and gain control over production, marketing, credit and distribution facilities.

The basic question in this regard is: did our program serve the truly needy, the disadvantaged in the community?

We have frequently discussed the importance of people's participation in the organizing, planning, implementation and now in the evaluation of the program. The following list of strategies will hopefully help to encourage and mobilize people to participate (Tan, 1983):

1. Raise the consciousness of the community.

2. Make use of small sectoral groupings.

3. Give importance to attitudinal and value formation.

4. Involve the program workers and the community in the planning, implementation and evaluation of the program.

5. Have more practical sessions and learning activities in

trainings.

6. Share knowledge with the community.

7. Respect the culture and way of life of the people.

8. Eliminate the gap between the staff and the community.

9. Emphasize self-reliance.

10. Form organizations of community workers.

In participatory program evaluation, participation is at each stage of the evaluation is central, including the initiation of the study, the selection of objectives and methods, and collecting and analyzing specific data. This kind of evaluation implies genuine participaton in realistic and effective decision-making processes. The most crucial question then becomes, are we ready and willing to do it? Remember, the best evaluations are done by the beneficiaries: the people.

The evaluation should examine the level of people's participation and the level of people's awareness. The following list may help the community decide what to look for when evaluating these important processes (Araullo, 1984):

1. the level of people's participation

- Has the program been proposed out of a process of discussion

by the people themselves?

- Have the people participated freely, with the major voice in planning and implementing the program?

- Do the people have a role in the evaluation process itself?

- Does the program seek to win the logistical as well as moral support of the people, and how does it use and administer the contributed resources?

2. the level of people's awareness

- Have the people grown in the understanding of their situation and why it is so, and in their will to do something to change their situation?

- Has the program brought deeper insights into how internal and external power factors work in the community? Into how decisions are made and by whom?

- Has the solidarity of the people grown through the program?

- Is there any attempt to relate local problems and needs to the national situation?

- What cultural values have been strengthened during the operation of the program?

We should also ask questions concerning the program staff and associated organizations and agency's performance.

- Is there a significant increase in awareness regarding the

people's situation, root causes, authentic solutions?

-Is there growth in respect for and trust in the capacity of people to take their destiny into their own hands?

-Is there an increase in the commitment to serve the poor, deprived and oppressed?

-Is there growth in appropriate skills and capacities; for example, the capacity to teach complex subject matter simply and effectively?

If the program has stated objectives we merely assess whether or not we met these objectives. Which objectives were achieved--which were not achieved? If we answer these questions adequately we can measure success or failure in terms of doing what we thought we intended to do.

This task is made much easier if the original program objectives were stated in specific and measurable terms. However, if the community participated in the original selection of objectives then a participatory evaluation of the objectives should be simple. We will discuss the methods of measuring objectives achievement in the section on evaluation methods.

The evaluation must also identify and measure the unintended impacts and effects of the program. Are the program's activities having a positive impact? Things we did not expect to happen as a result of the program will likely occur. Whether these unintended impacts and effects are positive or negative, good or bad, we must identify and measure them. If the program is contributing to a worsening situation for other needy or disadvantaged sectors in the community, this must be seriously examined. If the wealthy or advantaged sectors of the community complain because of the results of the program this should not change the direction of the program, but should be recorded as a possible threat to the program. The powerful people in the community who find themselves losing control of the community resources to the majority may try to undermine or destroy the program. It is important that we identify all the impacts the program has on the community or other communities. Indicators to identify the impacts and effects, and methods to measure them are discussed in the section on evaluation methods.

Are there better alternative programs which would meet the needs of the community? The organization should be adaptable to changing conditions. Perhaps the reasons for starting the program are no longer of concern. Perhaps the community wants to move on to a more long-term solution to their problems. It may be necessary to confront the powerful members of the community to adequately address the causes of the situation or problem identified in the community.

This section deals with the technical aspects of conducting an evaluation. Both the Program Logic Model (Appendix 3) and the Evaluation Plan (Appendix 4) are essential before beginning the evaluation study.

In order to answer the evaluation questions adequately we need data. This data should be readily available if our assessment or framework was done properly. From the data we can use indicators to measure our progress or achievements. Various methods will be discussed which are useful for collecting and analyzing data. It is important that the evaluation methods be simplified in order to meet the people's needs.

The evaluation process consists of (Chan and Villarosa):

1. Designing the contents of the evaluation

2. Analysis

3. Examination of progress

4. The evaluation review

5. Evaluation reporting

6. Follow-up

Indicators are variables that help us to measure changes. Indicators are needed when we cannot measure things directly. When we are interested in changes in values, attitudes and beliefs we must use indicators. Indicators are only reflections of the "real thing" therefore they are only partial measures or progress markers. Indicators are required to measure the success or failure in achieving program objectives. They are also required to measure imapcts and effects of the program, community development, and communication and coordination.

Here is a sample listing of possible indicators. You should add your own based on the particularities of your program:

1. Objectives achievement indicators

- can the benefits of the program be increased to cover a larger number of people

2. Impacts and effects indicators

3. Community development indicators

4. Communication and coordination indicators

- regular reports to the community and other organizations

This section tells how we answer all our evalution questions. It is important that the methods we choose encourage and facilitate maximum participation of the people. The real challenge here is to choose methods and techniques that suit the capabilities of the participants. Some of the methods which have been used in participatory evaluation include:

- observation, social investigation

- community dialogue

- surveys and questionnaires

- interviews

- analysis of records, reports, plans and other written materials

- the use of sampling

- simulations, games and exercises

- comparative study i.e. a study of contrasting an area where the program was implemented with an area where the program was not tried out

- self-evaluation/staff reflection sessions

- process observation

This is a long list of possible methods and one study will probably use two or three of the methods listed. There are also other methods that are not listed here and the community people may come up with new ones. For example, native communities I worked beside frequently used different methods with stories and pictures. The more frequently used methods include community dialogue, observation and analysis of written materials. Since most participatory evaluations make use of the community dialogue method we will discuss it in some detail.

A community dialogue (CD) is a systematic discussion on program and community concerns between program staff and members, beneficiaries and other community organizatons and members. It is a useful method not only in evaluation but also in planning and implementation. All those affected by the program should be involved or represented. CD is intended to be a shared educational experience for all the participants.

Before we conduct a CD we must prepare. We must define the purpose of the CD. Next, we must develop guidelines or questions for the discussions. The discussion questions must be formulated in the language or dialect of the community. It is also necessary to pre-test or try out the questions on a few people to see if they understand what is being asked.

A CD team should be organized to define the roles and responsibilities of the participants. They must come from the community and be given orientation as to the tasks of a CD team member. They will identify the prospective participants of the CD, work out the CD schedule (day, time and place), send letters of invitation and telephone, if necessary. It is also important that they follow-up and confirm the attendance of the CD participants.

The CD team must identify and train a CD facilitator and a recorder/observer. The minimum criteria for selection of the positions are as follows:

- must possess basic platform or presentation skills

- must know how to probe

- must be dedicated and committed to social change

- must have basic facilitating skills

- must be familiar with the language or dialect of the target area

- must possess recording skills

- must be knowledgeable about observation and/or recording

techniques

- must be able to adopt/adapt prescribed observation/

recording techniques

- must be dedicated and committed to social change

The CD team must prepare materials and equipment needed for the CD.

- flip charts

- posters

- transparencies

- tape recorder and batteries

- sound slides

- ballpen/pencil

- notebook/pad paper

- stapler

- flashlight

- calculator

- newsprint

- markers

- masking tape

- craft paper

a) Set an atmosphere conducive to CD

b) Provide a brief background of the Community Dialogue

- state the objective of the community dialogue

- explain why the community will benefit from the results of the community dialogue

- explain the topics, mechanics, and methodology of the

community dialogue

- solicit the participants' expectations from the community dialogue. Match their expectations with the objective of the community dialogue and clarify which among the expectations could be possibly accomplished for the duration of the community dialogue.

c) Route an attendance sheet for the participants to accomplish.

a) Restate the objective of the community dialogue and its usefulness for the community.

b) Synthesize the ideas raised by the group.

c) Ask the participants for unsurfaced questions or ideas that would further contribute to the attainment of the objective of the community dialogue.

d) Determine the specific expectations that have been met.

e) Thank participants for their attention, time, etc.

f) Explain the next activities to be undertaken (consolidation and validation of data) and future involvement of the community.

We should process and analyze the data obtained from the CD. A draft report should be written by the CD team. The report should be again discussed with community participants for verification and reflection. This can be done in the form of a summary report. This should be done immediately after the CD.

From the draft report and community validation a plan of action is formulated. The feedback session may be the appropriate venue for formulating the plan of action. Additional strategizing sessions may also be necessary. The community must receive a complete copy of the final CD report.

Sometimes a community may want to use statistics to analysis their program. Statistics are not something that only university experts can do. Frequently the kind of basic statistics that community people want can be done by the people themselves. Different statistics can be used for a variety of reasons, but one I have found useful for participatory evaluation is the chi-square.

In this section we are concerned with the extent to which the sample (or portion of the population or group being studied) is representative of the population. For example, if you wish to determine the needs of a particular community, you need not ask everyone in the community, for this would be too time consuming. A portion or sample of the community can be surveyed or interviewed to answer the questions. When using this technique we must however use certain statistical tests to determine whether or not the sample (eg. a portion of the community) is an accurate estimation of the actual population (eg. the entire community). In addition, we may wish to determine whether or not differences of the sample observations are significant or if they could be due to sampling variation alone. For this, we use certain statistical tests. In this section we will outline a test for comparing two samples.

There are many tests available for this kind of statistical inference. This brief section will only introduce and provide step-wise procedures for one of the available tests. The choice of the appropriate statistical test is very important and it may be necessary to consult further text to ensure this. This section refers to what is known as a nonparametric test and involves easy computation.

Nonparametric tests, for cases where you have one sample, are basically used for determining "goodness of fit". The term "goodness of fit" refers to the comparison of some observed distribution with a theoretical frequency distribution. This means that we can compare the number of observations (frequency) that fall in specified classes or categories; for example: the number of people that belong to different income categories or the number of women that stated they are faced by certain situations.

The chi-square test can also determine whether or not two groups (tenant farmer and owner-cultivator) differ with respect to some characteristic being studied (views on land reform). This means comparing the relative frequencies of the two samples or groups. The following provides the steps to be followed in using the chi-square test in this fashion. We will use an illustration to assist in outlining the steps.

The example will investigate whether or not there exists a difference between tenant farmers' and owner-cultivators' responses (Yes or No) to the statement, "All croplands should be redistributed at no cost to the actual tillers". The researcher randomly selects 50 tenant farmers and 50 owner-cultivators from the study area.

1. First the number of Yes and No responses in each group are summarized into a table:

Responses to the Statement by Land Status

_________________________________________________________________

Response Tenant Owner-Cultivator Total Number

_________________________________________________________________

Yes 32 20 52

No 18 30 48

_________________________________________________________________

Total 50 50 100

_________________________________________________________________

2. We must now state our hypothesis (Ha). The hypothesis is a statement which we will prove or disprove using the chi-square test. In the example the hypothesis is as follows; Ha: there is a significant difference between tenant farmers and owner- cultivators in the proportion of their Yes and No responses to the statement. The opposite of the hypothesis is the null hypothesis Ho. It is generally a hypothesis of no difference. If it is rejected by the statistical test, then we are actually accepting the alternative hypothesis Ha. In this case the null hypothesis is; Ho: there is no difference between tenant farmers and owner-cultivators is the proportion of their Yes and No response to the statement. We now assume that the null hypothesis is true until we prove that it is incorrect using our chi-square test.

3. We must specify the significance level. The level of significance of a statistical test is the "probability level that is considered too low to warrant support of the null hypothesis being tested". This level should be selected before any data is collected. The commonly used values for level of significance are 5 percent and 1 percent. If 5 percent is chosen, the probability of rejecting a true hypothesis is at most 5 percent. Similarly, if a significance of 1 percent is selected, the probability of rejecting a true hypothesis is at most 1 percent.

4. Next, we selected the statistical test to be used. In this case we will be using the chi-square test, but this test cannot be used in all cases.

5. Beginning here, we summarize the procedures for using the chi-square test.

i) Using the table we earlier constructed, let A,B,C, and D be the frequencies in the table's four cells.

_________________________________________________________________

Response Tenant Owner-cultivator Total Number

_________________________________________________________________

Yes A = 32 B = 20 A+B = 52

No C = 18 D = 30 C+D = 48

_________________________________________________________________

Total A+C = 50 B+D = 50 N = 100

_________________________________________________________________

ii) Now we can compute for the value of the chi-square (X²). It is a simple process of substituting the numbers from the table of A,B,C, and D in the formula.

chi-square =N ([AD - BC] - N/2)²

(X²) (A + B)(C + D)(A + C)(B + D)

=100([(32)(30) - (20)(18)] - 100/2)²

(52)(45)(50)(50)

=100([96960 - 360] - 50)²

(52)(45)(50)(50)

=100(600 - 50)²

(52)(45)(50)(50)

=30,250,000 = 5.1709

5,800,000

iii) Next we calculate the `degree of freedom' which simply refers to the number of independent cell frequencies. In our example we had 4 cells (A,B,C, and D), two in each row. The formula to calculate `degree of freedom' (df) is: df = (# cells in row one - 1) (# cells in row 2 - 1). In this case we have 2 cells in each row, resulting in the following: df = (2-1)(2-1) = 1

iv) Using 5 percent as our significance leval and 1 as our calculated `degree of freedom' we ascertain from the chi-square table a value of 3.841. The chi-square table is a special table for finding values of the chi-square. This table can be found in most statistics textbooks.

v) Using the value from the table (3.841) and the computed chi- square (5.1709) from our example we can determine whether to accept or reject our null hypothesis. If the computed chi-square is greater than the table value, then we reject the null hypothesis. In our example we reject the null hypothesis because the computed value 5.1709 is greater than 3.841. If the computed value was less than the table value, then we would accept the null hypothesis.

vi) We have concluded that there is a significant difference in the attitudes of tenants and owner-cultivators concerning the free distribution of all croplands to the actual tillers. At this step of the procedure we can draw conclusions and interpret our decision.

Activities

-----------------------------------------------------------------

Immediate

Objectives

-----------------------------------------------------------------

Long Term

Objectives

-----------------------------------------------------------------

Unintended

Impacts

I. BACKGROUND OF THE EVALUATION

The Marina Clinic in Dauin, Negros Oriental, Philippines has been operating as a primary health care unit of the Silliman University Medical Center (SUMC) since 1972. It serves as a clinic base of nine (9) mountain barangays (villages) whose work includes providing health care program; training of community health workers (also known as barangay health workers), traditional beith attendants, and local leaders; income generating programs through consumers' cooperative stores; soil conservation; pre-school program; and other community development programs. It uses health as a point of entry for community development. Ever since the program's inception in 1972, there has been no thoroughgoing evaluation and appraisal of the clinic's performance.

In April, 1985, the Head of the SUMC Extension Service Department requested the Ecumenical Center for Development (ECD) to undertake a third party evaluation study of the Marina Clinic. ECD's Programs, research and services Department conducts, as one of its services, program evaluation studies for interested program holders. The department has completed seven (7) major evaluation studies on health, socio-economic, housing, and integrated community development programs.

Thus, after several exchanges of communication between ECD and Marina Clinic, an agreement was reached that the research activity be conducted and thus this evaluation plan was formulated.

II. OBJECTIVES OF THE EVALUATION

The evaluation study is generally intended to: (1) Determine the effectiveness of the Marina Clinic; (2) Define and describe the process by which the clinic implements its programs and delivers its services; and (3) Determine factors (socio-economic, political and cultural) which either facilitate or hinder the attainment of the clinic's goals and objectives. Moreover, the study shall also look into the viability of the health extension program as an "arm" of the Silliman Hospital.

III. THE EVALUATION DESIGN

Like most of the evalution studies conducted and participated in by ECD, the evaluation of the Marina Clinic uses the process (formative) evaluation research design. This type of evaluation is held by ECD as an appropriate method for evaluating development programs, where the evaluators, community people and pragram staff define the program as it appears in action and describe how it works, and come up with recommendations on how to improve future work.

A. Scope of the Evaluation

The study shall examine four specific areas relative to the program, namely: (1) Needs Assessment; (2) Process of Implementation; (3) Effectiveness of the Clinic; and (4) Intervening Factors in the Course of Operations.

1. Needs Assessment. This refers to how the implementors of the Marina Clinic assess the needs of the target beneficiaries (the people it serves). Specifically, the following aspects are to be examined:

a. Needs of the target beneficiaries

b. Program Conceptualization

c. Program Concept and Orientation

d. Mechanisms and strategies employed by the clinic to

meet the needs of the beneficiaries

2. Process of Implementation. This area of evaluation pertains to the internal dynamics of the marina Clinic as an organization. This also refers to the activities undertaken in order to achieve the clinic's goals and objectives. The study shall look into the following:

a. Administrative structure

b. Management of the Clinic

c. Resources committed compatible with the Clinic's

concept and orientation

d. Changes and innovation in design, concept and

orientation

e. People's participation

f. Linkage and coordination with other agencies and

programs

g. Feedback mechanisms and monitoring system

3. Effectiveness of the Program. This refers to the effects of the clinic in terms of the services it has offered to the beneficiaries. Specific areas of examination are:

a. Effectivity and relevance of the program to target beneficiaries and communities

b. Achievement of goals

c. Efficiency and relevance of operational strategies

and activities

4. Intervening Factors. This refers to factors and events which affect the operation of the program.

a. Favorable factors

b. Major constraints

B. Implementation Scheme

The evaluation study is a cooperative endeavor between ECD and Marina Clinic. This activity is a participatory one wherein the participation of clinic personnel and administrators, beneficiaries and board members are to be elicited at different stages of the research.

An evaluation report will be submitted by ECD to Marina Clinic and its funding partners at the end of the study. The report shall contain analysis of tends, strengths and weaknesses of the Clinic as well as recommendations.

C. Data-Gathering Techniques

1. Methods of Data Collection. The main instrument to be used in data-gathering is community/group dialogue. Existing program documents such as program proposals, reports, assessment papers and other relevant materials shall be reviewed. Moreover, interviews shall be conducted among the implementors and beneficiaries. The evaluators too, shall keep a record of relevant observations in the course of their field work in the program areas.

a. Community/Group Dialogue. Dialogues shall be conducted among identified population of the study (see below: Population of the Study). These dialogues, composed of five (5) to fifteen (15) participants shall be a free-flowing process of interaction and information sharing. Every individual is free to share his knowledge/opinion and to react to the opinions and statements of other participants. Documentors of the dialogues shall record the consensus reached by the group. At the end of the session, validation shall be attempted by summarizing the major findings and information recorded.

A dialogue guide shall be used to facilitate the discussions. Several sets of guides shall be used, depending on who the participants are. The dialogue guides contain similar general topics; they vary only in depth and treatment of specific topics.

The facilitator sets the mood for the dialogue and starts it off by making the participants understand the reasons and purposes of the dialogue. With the use of the dialogue guide, he sees to it that all information which the participants may possibly share are elicited and expressed.

The proceeding of the dialogues shall be recorded in notebooks as well as by the use of a tape recorder.

b. Review of Relevant Documents and Materials. All documents and literature relevant to the program and the evaluation activity shall be reviewed and their contents analyzed. Information from such sources will clarigy certain aspects of the program aside from providing helpful background information. Documented experiences of similar institutions will also be of big help.

c. Interviews. Structured and unstructured interviews shall be conducted with key program personnel, community health workers, members of the board and program beneficiaries to shed light on matters such as: the needs and problems of the beneficiaries, personal experiences with the program, comments, etc.

d. Unstructured Observations. During the course of the fieldwork and area visitation, observations shall be recorded by the evaluators in field notes. Useful data to supplement research findings shall be provided by such observations.

2. Population of the Study: Sources of Data. The participants to the dialogues and interviews (also referred to as informants) shall be drawn from two sets of population: the implementors (personnel and members of the Board) and the beneficiaries. The beneficiaries gorup will further be divided into sub-groups--the community health workers and the community residents at large. Such dialogues for the beneficiaries shall be held in selected communities.

Community dialogues with the beneficiaries will provide data and information on the community-level operation of the Clinic. Dialogues with the personnel and board members will provide information on the wider perspective of operations.

IV. SCHEDULE OF ACTIVITIES

V. BUDGET