ECUR 809
Program Evaluation Module 1:
Assignment 1
Lauren Willey
P.A.R.T.Y.
Program Evaluation 2006-2008
Program
Overview
The Saskatoon Health
Region P.A.R.T.Y. (Prevent Alcohol and Risk-Related Trauma in Youth) Program is
a prevention and awareness program that is directed at youth to reduce
preventable injuries and death. With a
goal of promoting smart choices and safe risks in high school students, this
one day, in-hospital program exposes teens to the dangers associated with
drinking and driving, riding without use of a seat belt, and not wearing a bike
helmet. Put on by the Acquired Brain Outreach Team, the P.A.R.T.Y. Program runs
in approximately 30 communities across Saskatchewan.
Evaluation Model
Following
Stufflebeam’s CIPP model of evaluation, the purpose of this formative
assessment was to determine the program’s effectiveness in relation to their
short-term goals. While the CIPP model
defines evaluations as providing information regarding context, input, process,
and products, this particular evaluation focused solely on the program’s
product. Stufflebeam and McKee (2003)
noted that product evaluations could be divided into subcategories. In the case
of the P.A.R.T.Y Program, evaluators used an effectiveness model to judge
the success of the program. Motivated to assess outcomes and gage the program’s
effectiveness in meeting their needs, the evaluators sought to decipher whether
students’ attitudes and knowledge around risk taking behavior changed after
attending the one day session. It was determined that the P.A.R.T.Y. Program is
beneficial in improving students’ attitudes and risk taking behavior in the
short term, but that further evaluations would have to be conducted in order to
effectively assess the intermediate and long-term outcomes of the program.
Method
In
order to assess the short-term outcomes of the program, the evaluators chose a
quasi-experimental, pre-post design, with no control group. 408 grade 10 students from both urban and
rural high schools were given a questionnaire one week prior to attending the
program and were provided the same survey one week following their
participation. The questions were a mix of attitudinal and knowledge-based
questions surrounding injury and risk, with both qualitative and quantitative
data being collected.
Strengths
of Evaluation
Although
there are few strengths apparent in this program assessment, evaluators
attempted to meet certain objectives suggested for an effectiveness evaluation.
The first of these is that the results of the questionnaires were analyzed
using statistically sound methods, helping contribute to the validity of the
results. Secondly, the evaluators compared assessments of other P.A.R.T.Y.
programs in an attempt to judge the effectiveness of the Saskatoon Health
Region’s. A last strength is that the
evaluators acknowledged the weakness of their design and included recommendations
for future modifications.
Weaknesses of
Evaluation
There are many changes
that would need to be made to the evaluation and approach in order to make it
reliable and beneficial to stakeholders. The primary weakness of the evaluation
is that there were no suggestions made for strengthening and improving the
program, a vital component of a formative assessment. Instead, the conclusion
of the evaluation described the modifications that would need to be made to the
assessment’s approach in order to increase the validity of the findings and
provide a more accurate appraisal of the program’s outcomes. These suggestions included (a) using a control
school to compare results to; (b) using a one year follow up time frame instead
of two weeks, and (c) going beyond the questionnaire by assessing behavioral
change using the driving records of those who participated in the program. The latter recommendation would be beneficial as
changes to students’ attitudes and knowledge surrounding risk-taking behaviors
does not necessarily translate into a reduction in risk taking behavior. A final weakness to the report is that there
was no mention of the program’s stakeholders or who the audience of the
assessment would be.
In conclusion, the
evaluation was not well planned or executed. To provide a more reliable and
valuable assessment in the future, the evaluation should adhere to the recommendations
provided, as well as offer stakeholders feedback for program improvement.
References
Kershaw
T., Marko J & Gerwing, J. (2010). P.A.R.T.Y. Program Evaluation 2006-2008.
Saskatoon: Saskatoon Health Region.
Stufflebeam, D. (2002). The CIPP model for
evaluation. Evaluation models, 279-317.
Excellent analysis Lauren. You dissect the program evaluation with great skill. You identify a fact we will discuss further that you can use one aspect of a model so if the don't want to use all of the CIPP model they do not have to. I agree that this if a fairly weak evaluation and it is a stretch to find something good about it. The suggestions you make for improvement are accurate and well founded.
ReplyDeleteWell done.
Jay