GOAL	OBJECTIVES	ACTIVITIES	OUTPUTS	OUTCOMES	IMPACTS
Statement of the overall purpose of the project	Specific statements of what the project sets out to accomplish	Specific tasks to complete through implementation of the project	Immediate results (direct products of project activities)	Intermediate results (1 to 3 years after project starts)	Long-term results (3 to 10 years after project starts)
To meet the needs of students with social/emotional and/or behavioral disorders and provide those students with the necessary skills in order to become a successful student in the regular classroom.	Help students: -Develop appropriate student-like behaviours -Learn and use appropriate social skills -Work on academic skills -Learn and practice self-control. -Re-integrate into a regular classroom setting.	Once placed within the program students: -Follow an individualized and highly structured schedule that includes:  - chunking tasks into 20min blocks. - work completion = reward card system.  -large amount of one on one Instruction from S2 teacher. -academic tasks that align with regular classroom work.  -social skills and anger   management strategies and techniques. -support from and constant communication with home.  -gradual re-integration into regular classes when S2 expectations are consistently met.	Have students use/model the techniques and strategies they have learned within the S2 classroom setting. Have students set and achieve task and academic related goals by employing the strategies they have learned.	Have students gradually re-integrate into the regular classroom setting. See students transfer their skills/strategies and techniques to the regular classroom setting.	See students continue to exhibit their strategies/techniques and skills in the regular classroom. See students successfully transition into the high school environment. See students functioning by using their skills/strategies and techniques in the appropriate academic climate (i.e. transition program at high school, modified programming, regular programming, alternate programming)

Assumptions

•  That all of the stakeholders are committed to the S2 Program goal to meet specified student needs in relation to behaviour related issues  
• All stakeholders believe that S2 is the best and most appropriate place for the student to get their specific needs met so that they can be successful in the regular classroom setting.
• That the structure and environment provided by S2 is that which will provide the best opportunities for success for students with behavioural, emotional or social disorders.

External Factors

•   One of the most significant factors that effects the success of the program in terms of providing students with  behavioural, emotional or social disorders the best opportunity for success is simply whether or not the program has space or not.  The individualized attention and instructional plan that students receive within the S2 Program makes it such that only a small number of students are able to enroll within the program.  An additional factor related to availability is that the Middle Years level of the program is only offered through Bishop Lloyd Middle School ( one of two middle schools within the city)
• Another key external factor which is paramount in the success of each student, and so the program, is the support and participation of parents and guardians of students enrolled in the program .  The program has a strict ‘send home’ policy which without parental support would simply be impossible.  Parental support and contact is key throughout the program it is essential that the structured environment which is provided at school is also mimicked in the home.

Inputs

• The central component to the S2 program is most definitely the S2 teacher.  The teacher is key participant responsible for the development and implementation of the
• inividualized programs that cater to each student’s needs.    They will influence every aspect of the students’ academic roles while they are in the S2 program.
• Additional inputs for the program would obviously include the students themselves, parents, administration, behavioural specialist and coordinator (Division position) as well as the regular classroom teachers in the building.

Activities

• At the core of activities is the individualized nature of the program.  Each student will have a specific and individual academic and behavioural plan which is catered to their needs and abilities. 
• Once progress is documented and consistent, the re-integration process can begin.  Gradual re-integration included a trial and error system that may include sending the student back to S2.  If the student’s behaviour remains consistent and positive, then the student in integrated into more regular classes, until the student is completely integrated into regular classes.
• If the student displays regression in behaviour or does not implement the tools they have acquired through the S2 program, then the regular classroom teacher is responsible for sending the student back to S2.
• Once reintegration begins, regular contact between S2 teacher and the regular classroom teacher is essential is determining the ongoing progress of that student.
•  The activities of the program require that regular meetings occur concerning students with administration, parents and program coordinator.

Outcomes/Impacts

• The overarching goal of the program is to equip students with the skills and techniques they need in order to cope with and function appropriately in a regular classroom environment.
• The short term goals of the program are closely related firstly to success and consistency within the S2 environment and then eventually the same progress and consistency in the regular classroom environment.
• The long term goals of the program are related to student progress once they have moved out of our building.  Although this is the long term goal of the program, there is currently no formal means or structures in place that would provide teachers and staff with the information to judge the attainment of the goal.

Assignment #5 Survey

General Findings/Thoughts
In the initial stages of developing the survey, I concluded that the best approach would be to attempt to narrow the focus of the survey to some key components of the program.  As part of the final evaluation, this survey would be only a portion of a larger survey targeted at several other components of the S2 Program.  I decided to break the program down into three components: referral and admittance procedures, structure of the process that the students experience and re-integration.  In developing this short survey I discovered that it would be most effective to focus only on one component.

In terms of revising and editing the first draft of the survey I discovered that it is vital to keep wording and sentence structure as clear and simple as possible.  I found that when I went back to my questions, I often thought that they could be worded more clearly and simply in order to ensure the most accurate results from participants.  I also augmented the survey considerably from the original by deleting and replacing several questions.  These more significant changes were made in an effort to align the survey with the goal, as mentioned above, of focusing on one key area of the S2 Program.

Follow this link to the original survey.

Revisions with Justifications/Explanations 

Title
The title of the survey was changed in order to better reflect the focus of the survey.


Part One
The first two questions of the survey were added in order to classify the results when analyzing the data.  It is important to have information regarding the position of the participant and also how long they have occupied this position.  This effects the interpretation of the data in that length of time in the position as well as the position itself will have an impact on how the participant answers the questions.  The perspective of an administrator may differ greatly from that of the S2 teacher, and is an important factor to consider when analyzing the data.  The question relating to the description of the participant's role in the S2 Program was included because this is an area which is explicitly outlined in the guiding documents of the program.  It would be important to find out whether the role outlined in the guiding documents matches what is outlined by the participant.  The implications of any differences that arise would be an important element in evaluating overall effectiveness of the program because program guiding documents should match what is currently happening in the program.

Part Two
Questions 1 &2
These questions were simply reworded in an effort to simplify and clarify the questions.  The wording of the revised questions is much easier to read and understand.  Designing questions that are easy to read and understand quickly is important - participants will become discouraged or disgruntled if they have to constantly reread questions in order to ensure that have understood what is being asked.

Question 3
The word "intended" was removed from this questions because it simply doesn't need to be there.  The goals of the program are obviously those that are intended so the word simply serves no purpose in the question.

Question 4
"The referral process" was changed to "the current referral process" in an effort to make sure that the participants are referring to the current processes when responding to the question.

Question 5
No changes were made.

Questions 6, 7 & 8
The original questions were deleted entirely and changed to the present questions because all three of these questions (in the original survey) did not fit the focus of the survey.  The questions were focused on other components of the program that did not relate to the referral and admission processes.  Specifically, question 6 is an appropriate follow up to question 5 and so fits a logical organizational pattern.  Question 8 was included because parental participation and involvement in the program is something that is highlighted throughout the guiding documents for the S2 Program.

Part Three:
Question 9
No changes made.

Question 10
This question was deleted and replaced with the current question in an effort to keep the survey focused on the referral and admittance procedures.  I chose to include this question because it is relevant to collect data on what supports and information are provided to regular classroom teachers as these are the people that make the initial referrals.  It seems important to consider whether teachers are provided with adequate knowledge and support in order to make appropriate decisions concerning the referral of their students.

Opinion/Feedback Section 
This section was not changed.  I think it is important to gather general feedback on the survey from participants to ensure that the the survey is designed in a manner that will ensure the most accurate results.

Follow this link to the modified version of the survey.

Evaluation Assessment

Focusing the Evaluation

Chosen Program: Senior Level (Grades 7-9) Structured Success Program

Purpose of Evaluation

The purpose of the evaluation is goal oriented.  The evaluation will focus on examining and assessing the goals of the program.  The program goals need to align with the anticipated outcomes (both short and long term) as set out by the program.  The process of the program as well as the will be evaluate in an effort to ensure that the goals of the program are being met and the outcomes are successful.

Audience for Evaluation

The evaluation is intended to provide the following stakeholders with information on the effectiveness of the program:

<!ü Administration

<!ü  S2 Staff – S2 teacher and Educational Assistant

<!ü  Parents of students enrolled as well as parents of potential S2 students

<!ü  Other teaching staff

Information presented by the evaluation is intended to provide teaching staff, administration and program coordinator with the data and analysis required to determine whether changes are required in order to make the program more effective.  The evaluation would provide parents of potential S2 students with information on the success of the program so they can then make an informed decision concerning whether the program is the best fit for their child.  Parents of students currently enrolled would gain additional insight to the effectiveness of the program their child is participating in.  The evaluation would provide other teaching staff with the information necessary in order to make an informed decision in relation to referring students to the S2 program.  As an overarching statement, the purpose is to provide each of these stakeholders with a better understanding of the program in particular reference to program goals and effectiveness.

Questions the Evaluation Seeks to Answer

The evaluation will be focused on seeking out answers to the following questions:

<!ü Do the goals and objectives of the program meet the desired outcomes?

<!ü Do the activities and the structure of the program (process) work to meet the desired outcomes?

<!ü Is there anything about the process that needs to change in order for the program to become more successful in attaining desired outcomes?

Information Needed to Answer Questions

In order to assess the goals and objective in relation to whether they meet desired outcomes, analysis and discussion will need to take place.  Discussion between S2 staff, program coordinator and administration concerning whether the connection between outcomes and goals is clear and logical would aide in attempting to answer this first question.  Additional information may be gathered from similar programs in order to provide some perspective on goals associated with re-integrating behavioral/emotional/social students back into the classroom. 

Information regarding student success in exhibiting learned skills and behaviours is required in order to determine process effectiveness.  Any issues that arise in answering the above question will lend information into answering whether and what changes/improvements can be made in terms of the process of the program.

Collecting the Information

Resources

<!ü Access to S2 student records (progress reports from within S2)

<!ü Access to information regarding students who have transitioned to the high school

<!ü Access to input/feedback from:

<!ü S2 staff

<!ü Administration

<!ü Division Program Coordinator/Behavioural Specialist

<!ü Other teaching staff (those who have had S2 students integrated into their classrooms and/or removed from their classes to be placed into the S2 program)

Sources of Information

<!ü Progress reports of students currently enrolled in S2

<!ü Records of past S2 students

<!ü Information on students transitioned to high school

<!ü Staff/administration and division feedback/input

Data Methods

<!ü Survey

<!ü  Interview

<!ü  Document review

<!ü  Observation (of S2 students in the S2 environment and integrated into the regular classroom)

** Since the program is well established, integrated and ongoing program all of the data collection will take place during the program.

Sample will consist of those students currently enrolled in the program and previously enrolled students who transitioned to the local public high school.

Using the Information

Data Analysis

• ü  Analysis of student records for indicators of success/improvements in behaviour
• ü  Analysis of interview responses to determine staff feedback
• ü  Analysis of survey results to determine trends in success/improvements

Interpretation of Information

Once compiled, information will be interpreted by S2 staff, administration, program coordinator and evaluation coordinator (me).  While interpretation of data and analyses will be a combined effort, summary of the interpretation and data will be completed by evaluation coordinator.

Communication of Information

Information will be reported back to staff, administration and program coordinator through a formal evaluation report.  A follow up meeting in order to discuss and expound upon the findings of the evaluation would also be preferable.  Although presented in a different format, the information would also be shared with parents of students currently enrolled in the program.   A concise version of the report would be made available to these parents as well as parents of potential students.  In this way the evaluation would provide parents and other stakeholders with an enhanced understanding of the program in its current state and perhaps where it needs to go in the future.  

Assignment #3 Logic Model

S2 Senior Level (Grades 7-9) Program Logic Model 

GOAL	OBJECTIVES	ACTIVITIES	OUTPUTS	OUTCOMES	IMPACTS
Statement of the overall purpose of the project	Specific statements of what the project sets out to accomplish	Specific tasks to complete through implementation of the project	Immediate results (direct products of project activities)	Intermediate results (1 to 3 years after project starts)	Long-term results (3 to 10 years after project starts)
To meet the needs of students with social/emotional and/or behavioral disorders and provide those students with the necessary skills in order to become a successful student in the regular classroom.	Help students: -Develop appropriate student-like behaviours -Learn and use appropriate social skills -Work on academic skills -Learn and practice self-control. -Re-integrate into a regular classroom setting.	Once placed within the program students: -Follow an individualized and highly structured schedule that includes:  - chunking tasks into 20min blocks. - work completion = reward card   system. -large amount of one on one Instruction from S2 teacher. -academic tasks that align with regular classroom work. -social skills and anger management strategies and techniques. -support from and constant communication with home. -gradual re-integration into regular classes when S2 expectations are consistently met.	Have students use/model the techniques and strategies they have learned within the S2 classroom setting. Have students set and achieve task and academic related goals by employing the strategies they have learned.	Have students gradually re-integrate into the regular classroom setting. See students transfer their skills/strategies and techniques to the regular classroom setting.	See students continue to exhibit their strategies/techniques and skills in the regular classroom. See students successfully transition into the high school environment. See students functioning by using their skills/strategies and techniques in the appropriate academic climate (i.e. transition program at high school, modified programming, regular programming, alternate programming)

 

Assignment #2

Evaluation Focus

In preparing this assignment I had two points of focus: to determine both the effectiveness of the program in relation to its goal in promoting regular physical activity in order to lower the incidence of diabetes in First Nations women as well as to assess the effectiveness of the implementation of the program. The evaluation would target these two areas in an effort to align with the goals and details as provided by the program description.  The program overview explicitly states that the program is both a means and an end to address these health concerns within the First Nations population in Saskatchewan.   In this vain, I believe that a mix between the Scriven, Naturalistic and the CIPP models would be appropriate. 

Justification

In order to assess and evaluate the effectiveness of the program in reducing the rates of diabetes in First Nations women, in an effort to optimize the program, it is important then to gather information and evaluate the program by determining its end result as well as its implementation process.  The CIPP model focusses on providing feedback and assessing the context, input, process and product of a program.  In this way, the CIPP model meets the need to assess the effectiveness of the program in meeting its goals as both a means and an end to the health concerns the program wishes to target.  The Scriven model also focusses on really understanding the program goals, and as such, serves as a good theoretical base for this evaluation.  The Naturalistic model fits specifically with one focus area of this evaluation: providing feedback from the participants themselves.  The following plan details the incorporation of these models in more detail and further justification is provided throughout.    

Focus Questions

1.      What is the primary goal of the program?

a.       Does the primary goal of the program match with the implementation process?

b.      Are there any pitfalls in the implementation in providing the needs of the participants and goals?

2.      Is targeting the GDM is First Nations women an effective way to attain lowered rates of diabetes among First Nations populations?

3.      Did the physical/recreational activity lower the incidence of GDM among participants?

4.      Did the implementation process of the program optimize the probability of success in achieving the end (lowering the incidence of GDM among participants)?

a.       Was participant attendance consistent enough to base conclusions concerning reductions rates for GDM upon?

b.      Was the drop in format optimal for encouraging participants to attend?

c.       Did providing services free of charge (child care, snacks, bus passes) provide an incentive for participants?  Was it something that worked to raise/sustain participation?

d.      Did the choice of venue help to encourage participation?

e.       Did the nature of the activities (flexible, participant input format) encourage and foster participation?

f.       Did the inclusion of an Elder in the decision making process aide in providing the desired input and perspective?

Information Gathering

Existing data would need to be gathered in an attempt to connect the incidence of diabetes among First \nations population and GDM in pregnant women.  This may prove to be a limitation of the evaluation because such information may be difficult to acquire.  Never the less, it is still an important piece of the context piece of the evaluation, as prescribed by the CIPP model. 

In order to evaluate program success in relation to meeting the end goal, follow up information would need to be provided on the participants’ health status.  The number of women who developed GDM would need to be determined.  In addition to acquiring data on the women within the program, existing data, on rates of GDM in First Nations populations, would need to be acquired in order to determine whether the program had an impact on lowering the rates as compared to women who did not participate in the program.

Evaluation of the implementation process of the program would be determined through participant, program organizer and facilitator feedback.  The feedback would be provided through an interview and survey process.  This information would serve to provide the input component of the CIPP model.  Staffing and approaches to the implementation of the program would be examined and assessed.  Surveys would be completed by the program organizers and facilitators and would take the form of scaled response questions.  The aim would be to assess the effectiveness of the program elements as opposed to attempting to gather human interest information, and as such, the scale response style survey is appropriate to glean such information.  Application of a Naturalistic approach would be most conducive in acquiring the feedback of participants.  The feedback would be best relayed through open ended style responses provided within a focus group setting.   A focus group setting would be appropriate within the context because the women might be more likely to provide in detailed and honest responses where they are able to interact with their peers as opposed to a one on one interview with a stranger.  Both formats would include questions focusing on the structure and delivery model of the program itself. 

Data Analysis

The data collected in order to determine the occurrence of GDM in participants would be compared to a sample set of women who developed GDM who did not participate in the program.  This data analysis would provide information on the effect the program had on actually lowering the rate of GDM.  In order for this comparison to take place, allocation of a sample set to base the comparison on would be required.  This comparison of data of women within the program and those who did not participate would provide a solid base for determining the effectiveness of the program in meeting its end goal.  A limitation of the analysis would be that the reliability of the results data from the program correlates directly to the consistency of participant participation.  The data provided and conclusions drawn from the comparison would aid in determining the end result (Product) of the program. 

The information collected from the focus group and surveys would be analyzed in order to determine any common threads that appear.  The key points from the focus groups would indicate the effectiveness of the different components of program implementation (Input, Process)

Using the Information

The information and conclusions from the program evaluation would take the form of a final report in order to satisfy the needs of the program developers and facilitators.  In addition to a final report, it would also be useful and appropriate to create a less formal report in order to present to program participants.  

ECUR809.3 Assignment #1

After some struggles selecting a completed evaluation to review/analyze, I finally decided upon an evaluation that concerns a particular special education program.  The evaluation that I decided upon, “An Evaluation of the Structured Success Program: From the Students Perspective,” centers upon an evaluation of an alternate education program offered and implemented in Saskatoon.  Once I stumbled upon the article, it peaked my interest because the Structured Success Program is also offered within the school division I am currently working for and is run out of the building where I teach.  The program is an alternate program for those students who have serious behavioral and/or emotional disorders.  Students who are unable to operate within the regular classroom environment, and whose behaviors are a significant disruption to the learning environment for classmates, are placed in the Structured Success Program.  It is the goal of the program to teach and model appropriate behavior and social skills so that the students will eventually be integrated back into the regular classroom setting.

Although the evaluation does focus somewhat on the overall student perception of the Structured Success Program, the title is a little misleading.  A considerable amount of the evaluation is focused on the students’ perception of their classroom environment and its relationship with their self-concept.  The evaluation aimed to examine these two perceptions and then compare these findings with results from the regular classroom.  The aim of the comparison was to fill a gap in research that exists in studying the relationship between classroom environment and self-concept for students with B/ED.  This aim does still link with the overall purpose of the evaluation, but the connection between the two is not explicitly stated.  The comparison is made because, it seems, the goal is to get an overall understanding of students’ perceptions of their classroom environment, their self-concept, as well as students’ general impressions/perceptions concerning the program. 

The methods that were used in order to evaluate students’ perceptions and impressions of the Structured Success Program were all student completed surveys and questionnaires.  In order to determine the students’ perception of their classroom environment and self-concept, the CES (Classroom Environment Scale) and MSCS (Multidimensional Self-Concept Scale) were used.  More specifically, 17 students currently enrolled in the S2P from five schools in Saskatoon participated in the study.  The subjects were all males ranging from 11-14 years, with eleven being enrolled full time in the program and six only part time.  The length of time that these students were enrolled in the program also varied from 3-27 months.  The data collection for the evaluation was completed in two phases.  During phase one of the data collection participants completed the two self-report questionnaires (CES and MSCS).  The results from these two tests were then compared with results completed by students in regular classroom environments.  In the second phase of data collection students participated in semi-structured interviews with the author of the evaluation.  The questions from the interview included some limited response style questions as well as more open ended questions that required more detailed responses from the participants.  The aim of the second phase was to ascertain the general perceptions and sentiments of students in relation to their enrollment in and experiences within the program.

Perhaps the most evident strength of the methods employed is simply that they match with the intended purpose to provide information on students’ perceptions of their classroom environment, self-concept and the program in general.  The goal was to provide some information in terms of how regular classroom students’ perceptions of environment and self-concept compares with students with B/ED, and the administering of the CES and MSCS serves that purpose.  The evaluation also included an overview of the similarities and differences found as a result of administering and comparing the two sets of results.  The comparison is presented in a clear, concise manner and the reasons for the variances in responses are also explained.  The nature of the interview process also aligns with the goal to provide insight into the general perceptions of the program.  The open ended nature of the interview questions allows students to provide detailed, personal responses in relation to their experiences in the program.  In considering alternative formats for the open ended style responses, the evaluator also notes, rightly so, that a focus group format would have likely proved ineffective with this group of students.          

Although the overarching purpose of the evaluation seems to be achieved at first glance, there are some weakness and limitations of the evaluation.  The nature of the S2P student lends itself to general inaccuracies in attempting to establish empirical data to base conclusions upon.  The fact that these students usually harbor a general resentment for testing and mandatory work means that there is a possibility that students are not really thinking about the questions as they answer them, but simply trying to “get it over with.”  The evaluation itself provides evidence to suggest that these assumptions may very well be the case.  The interview phase of the data collection only included 5 out of the 17 participants.  The evaluator could not get the other students to participate in this section of the evaluation and also had to offer a $10 incentive just to get those 5 to complete the interview.  The conclusion here is an obvious one: how can you base conclusions about a program based on the responses of 5 out of your 17 participants.  For that matter, 17 participants isn’t exactly a large number to base conclusions upon either.  The small number of participants in the data collection process is an obvious weakness, one that the evaluator chooses to mention as well. 

The number of the participants is a concern, but in addition to that, the participants were also all males.  This seems to cause some concern as it certainly seems plausible to assume that there may be variances in the perceptions of females in the program compared with their male counterparts.  In addition to the problems with numbers in the interview portion of the evaluation, there is also little information/insight offered into the purpose of the interviews in relation to what the information will be used for.  The evaluator states that the interview process provided students with the opportunity to offer their opinions and experiences concerning the program, but to what end?   What will this information be used for?  These questions are not addressed.  The conclusion notes that understanding students’ perceptions within special education programs is needed because it may have implications for the future development and redevelopment of those programs.  It seems then that, despite some limitations, this evaluation still provides some research and conclusions in an attempt to shed light on an underrepresented group of students.  


Source:
Da Silva, Trudy A. "An Evaluation of the Structured Sucess Program: From the Students' Point of View." Canadian Journal of School Psychology. 2003 18: 129.  http://cjs.sagepub.com/content/18/1-2/129.