Use the Evaluation Toolkit to plan, analyze, report, and apply evaluation data. The Toolkit includes planning steps that can also be used as a checklist, and links to evaluation instruments.

Evaluation Steps/Checklist
How do we evaluate program effects?

STEP 1: Define Program and Evaluation Goals
What do we want to find out?
• Define measurable and attainable long- and short-term program goals (attitudinal, behavioral, academic).
• See Evaluation Plan Matrix Sample, Column 1 for examples.
• Use program goals to define measurable and attainable evaluation goals.
• See Evaluation Plan Matrix Sample, Column 2 for examples.

• Examine school data:
• Academic: GPAs, failure rates, retention, graduation, standardized test scores
• Behavioral: attendance, "tardies," discipline incidents (rate and type)
  

• Explore self-report survey options:
• School-Connect Teacher Satisfaction Survey
• School-Connect Students Satisfaction Survey
• Student and Staff School Climate Surveys
  

• Conduct focus groups and/or interviews with students, staff, and/or parents:

• See School-Connect Satisfaction Surveys above for sample questions.
• Design interview questions based on program and evaluation goals.
• Ask for anecdotal examples of students using School-Connect skills (e.g., effective communication, emotional management, study strategies, conflict resolution).
• See STEP 5 for implementation assessment options. Monitoring implementation is essential to deriving outcome findings and understanding cause and effects.

STEP 3: Chart Evaluation Procedures
How do we collect the data?

• Create an Evaluation Plan Matrix to align program and evaluation goals with data collection methods. (see Evaluation Plan Matrix Sample)
• Use program and evaluation goals from STEP 1 for Columns 1 and 2.
• Choose data collection methods for Column 3.

STEP 4: Plan Schedule for Collection and Analysis
When do we collect and analyze the data?

• Plan data analysis feasibility. Consider:
• Data entry or Scantron fees
• Data analysis services (in-house, at the district level, or through a consulting firm)
  

• Conduct pretests before implementing curriculum. Pretest options:
• End of the previous school year
• Beginning of the semester or school year

• Conduct teacher satisfaction surveys a minimum of four weeks after steady implementation.
• Conduct posttests at the end of the semester or school year.

• For more reliable results, implement self-report surveys:
• Mid-week (Tuesday, Wednesday, or Thursday)
• Away from holidays, school breaks, and exam periods

STEP 5 (ongoing): Assess Implementation Quality and Consistency
How do we know we are implementing the program as intended?
See the Implementation Toolkit for implementation guidelines.

• Check in with School-Connect teachers on a regular basis.
• Consider using the School-Connect Lesson Observation Form to provide feedback to teachers on lesson facilitation.
• Ask teachers about frequency of use and amount of preparation. (See School-Connect Teacher Satisfaction Survey, questions 1-4.)
• Assess what percent of the lessons and lesson outlines are being implemented relative to the initial implementation plan.
• Identify high-implementation and low-implementation instructors; factor implementation levels into outcome results. (Do students with higher implementation teachers show more significant results? And vice versa?)

STEP 6:  Monitor Progress and Report Findings
How do we report our findings?

• Chart data collection and implementation feedback over time.
• Examine the correlation between implementation fidelity and outcome data.
• Share evaluation findings with staff and administration.

STEP 7: Discuss and Apply Evaluation Findings
What do we do as a result of our findings?

• Use data results to inform and improve implementation plan and quality.
• Consider additional training in social and emotional skill building.