Assignment #2

Evaluation Focus

In preparing this assignment I had two points of focus: to determine both the effectiveness of the program in relation to its goal in promoting regular physical activity in order to lower the incidence of diabetes in First Nations women as well as to assess the effectiveness of the implementation of the program. The evaluation would target these two areas in an effort to align with the goals and details as provided by the program description.  The program overview explicitly states that the program is both a means and an end to address these health concerns within the First Nations population in Saskatchewan.   In this vain, I believe that a mix between the Scriven, Naturalistic and the CIPP models would be appropriate. 

Justification

In order to assess and evaluate the effectiveness of the program in reducing the rates of diabetes in First Nations women, in an effort to optimize the program, it is important then to gather information and evaluate the program by determining its end result as well as its implementation process.  The CIPP model focusses on providing feedback and assessing the context, input, process and product of a program.  In this way, the CIPP model meets the need to assess the effectiveness of the program in meeting its goals as both a means and an end to the health concerns the program wishes to target.  The Scriven model also focusses on really understanding the program goals, and as such, serves as a good theoretical base for this evaluation.  The Naturalistic model fits specifically with one focus area of this evaluation: providing feedback from the participants themselves.  The following plan details the incorporation of these models in more detail and further justification is provided throughout.    

Focus Questions

1.      What is the primary goal of the program?

a.       Does the primary goal of the program match with the implementation process?

b.      Are there any pitfalls in the implementation in providing the needs of the participants and goals?

2.      Is targeting the GDM is First Nations women an effective way to attain lowered rates of diabetes among First Nations populations?

3.      Did the physical/recreational activity lower the incidence of GDM among participants?

4.      Did the implementation process of the program optimize the probability of success in achieving the end (lowering the incidence of GDM among participants)?

a.       Was participant attendance consistent enough to base conclusions concerning reductions rates for GDM upon?

b.      Was the drop in format optimal for encouraging participants to attend?

c.       Did providing services free of charge (child care, snacks, bus passes) provide an incentive for participants?  Was it something that worked to raise/sustain participation?

d.      Did the choice of venue help to encourage participation?

e.       Did the nature of the activities (flexible, participant input format) encourage and foster participation?

f.       Did the inclusion of an Elder in the decision making process aide in providing the desired input and perspective?

Information Gathering

Existing data would need to be gathered in an attempt to connect the incidence of diabetes among First \nations population and GDM in pregnant women.  This may prove to be a limitation of the evaluation because such information may be difficult to acquire.  Never the less, it is still an important piece of the context piece of the evaluation, as prescribed by the CIPP model. 

In order to evaluate program success in relation to meeting the end goal, follow up information would need to be provided on the participants’ health status.  The number of women who developed GDM would need to be determined.  In addition to acquiring data on the women within the program, existing data, on rates of GDM in First Nations populations, would need to be acquired in order to determine whether the program had an impact on lowering the rates as compared to women who did not participate in the program.

Evaluation of the implementation process of the program would be determined through participant, program organizer and facilitator feedback.  The feedback would be provided through an interview and survey process.  This information would serve to provide the input component of the CIPP model.  Staffing and approaches to the implementation of the program would be examined and assessed.  Surveys would be completed by the program organizers and facilitators and would take the form of scaled response questions.  The aim would be to assess the effectiveness of the program elements as opposed to attempting to gather human interest information, and as such, the scale response style survey is appropriate to glean such information.  Application of a Naturalistic approach would be most conducive in acquiring the feedback of participants.  The feedback would be best relayed through open ended style responses provided within a focus group setting.   A focus group setting would be appropriate within the context because the women might be more likely to provide in detailed and honest responses where they are able to interact with their peers as opposed to a one on one interview with a stranger.  Both formats would include questions focusing on the structure and delivery model of the program itself. 

Data Analysis

The data collected in order to determine the occurrence of GDM in participants would be compared to a sample set of women who developed GDM who did not participate in the program.  This data analysis would provide information on the effect the program had on actually lowering the rate of GDM.  In order for this comparison to take place, allocation of a sample set to base the comparison on would be required.  This comparison of data of women within the program and those who did not participate would provide a solid base for determining the effectiveness of the program in meeting its end goal.  A limitation of the analysis would be that the reliability of the results data from the program correlates directly to the consistency of participant participation.  The data provided and conclusions drawn from the comparison would aid in determining the end result (Product) of the program. 

The information collected from the focus group and surveys would be analyzed in order to determine any common threads that appear.  The key points from the focus groups would indicate the effectiveness of the different components of program implementation (Input, Process)

Using the Information

The information and conclusions from the program evaluation would take the form of a final report in order to satisfy the needs of the program developers and facilitators.  In addition to a final report, it would also be useful and appropriate to create a less formal report in order to present to program participants.