The evaluation of the Initiative will be conducted using a national evaluator in collaboration with local evaluators. The success of this evaluation depends largely on the relationship between the local evaluators and the national evaluator. While the national evaluator will be solely responsiblefor certain evaluation activities, many activities will be executed jointly. The national evaluator will be expected to:

• Document and analyze the prevalence and incidence of a set of core indicators across all program sites (surveillance).

• Provide technical assistance to each site as needed to assist the site in describing how the Initiative is operating. These data will also be used for ongoing initiative improvement and for preparation of a report summarizing cross-cutting themes.

• Design and conduct an intensive evaluation with a subset of the approximately 50 Safe School sites, including collection and analysis of relevant process data.

• Conduct an economic evaluation -- a cost benefit/cost offset analysis in the subset of sites.

• Prepare reports on evaluation results for publication.

The national evaluator will have primary responsibility for the following evaluation activities within and across sites:

• Develop a set of core indicators and outcomes in collaboration with the IEMT for which data will be collected from all sites.

• Develop a common protocol to be used at each site for collection of core data.

• Create and implement the data collection and management center for cross-site core data.

• Provide timely consultation, training, and materials to sites, as appropriate, to help each site achieve its specific objectives.

• In collaboration with each site, develop and collect data on site-specific variables that can be used by sites as part of their local evaluation.

• Provide information on site-specific core data to each site in a timely manner.

• Design, coordinate, and oversee detailed, intensive analyses, including economic analyses, at selected sites. Findings from these analyses will be used to inform policy, science, and programs at the national level.

The primary responsibility of the national evaluator is to provide the IEMT with new knowledge to inform science, policy, and programs, with an emphasis on outcomes in the educational, mental health, and juvenile justice arenas. The primary responsibility of the local evaluator is to provide the Safe Schools/Healthy Students education/mental health/law enforcement partnership with consultation and information on access, fidelity, and intensity of program delivery, with an emphasis on process, to help the site achieve its specific objectives. The national and local evaluators will collaborate and coordinate their activities as they work to meet their responsibilities to their primary constituents.

The relationship between the national evaluator and the local evaluators should be collaborative and supportive. The national evaluator will provide guidance and leadership in key areas of the overall evaluation effort and the local evaluator will comply with the requirements of the IEMT and the national evaluator. Specifically, local evaluators must participate in surveillance and other activities in accordance with the procedures established by the national evaluator. When appropriate, local evaluators will be compensated for data collection work relating to the national evaluation. The table below outlines the relative responsibilities of the national and local evaluators within each major area of evaluation activity.

Surveillance of Core Indicators (Conducted in All Sites)

National Evaluator Responsibilities

Local Evaluator Responsibilities

Assist the IEMT in identifying core indicators for each element. Develop common protocol for collection of core indicator data across sites, including standard demographic data. Develop Office of Management and Budget (OMB) clearance package. Work with sites as needed to develop methodology and protocol for collection of site-specific, noncore data. Collect common core indicator data. Analyze core indicator data and write reports for IEMT. Provide timely information to each site on site-specific core data. Prepare public use data sets and code books.

Coordinate local data collection activities with core indicator data collection. Assist national evaluator in obtaining local clearances and collecting data. Work with national evaluator as needed to develop methodology and protocol for collection of site-specific, noncore data. Provide local-level consultation and clarification to national evaluator as needed for core data analyses.

Process Evaluation (Conducted in All Sites)

National Evaluator Responsibilities

Local Evaluator Responsibilities

As requested and as needed by the sites, the national evaluator will: Provide sites with sample research protocols. Provide sites with sample questionnaires, forms, and other materials. Provide local staff with information on collecting, analyzing, and using data. Convene local evaluators for technical assistance. Assist sites in effectively and efficiently using information (e.g., records, data). As needed, develop OMB clearance package. Aggregate process data across sites. Write a report summarizing cross-cutting themes.

As needed, collaborate with national evaluator to develop and collect process data. Develop a process evaluation plan. Develop data collection methodology. Collect all data. Analyze data. Use data to continually improve and focus programs. Provide process data to national evaluator for cross-site summary report.

Intensive Evaluation (Conducted in Selected Sites)

National Evaluator Responsibilities

Local Evaluator Responsibilities

In coordination with the IEMT, select sites to be included in the intensive evaluation. Collaborate with selected sites and the IEMT in developing specific research questions. With the IEMT, determine strongest and most feasible controlled design to address research questions. Develop common protocol to collect process and outcome data, including economic data in selected sites. Develop OMB clearance package. Collect data in a standardized manner. Coordinate and monitor all collection and management (e.g., cleaning, aggregating) of intensive data. Analyze data and write report(s). Prepare public use data sets and code books.

Collaborate with the IEMT and national evaluator in developing research questions and other materials. Coordinate local data collection activities with national evaluator. Assist national evaluator in obtaining local clearances and collecting data in a timely fashion. Provide local-level consultation and clarification to national evaluator as needed for data analyses. Collaborate in analyzing data and writing reports as needed.

National Evaluation of the Safe Schools/ Healthy Students Initiative	Due Date: July 21, 1999