The national evaluator will be responsible for developing the products listed below. All products will be submitted to the IEMT for review and approval prior to the development of the final product(s).

Evaluation Plans/Designs

The grantee will submit an updated version of the list of research questions that it submitted for review in its application. The list will indicate the category or categories of evaluation activities (surveillance of indicators, process evaluation, intensive outcome evaluation) that will be used to address each of the questions. The list will include questions that relate to each of the six critical elements and questions that address issues that cut across these areas.

The grantee will submit a list of proposed expert panel members to serve in an advisory capacity for the evaluation. The grantee will convene meetings of the panel, with participation by the IEMT, at appropriate points throughout the study and will also arrange for review of draft study products by panel members.

The grantee will submit a literature review for each of the six critical elements. At the same time, the grantee will also submit a conceptual framework depicting the theoretical basis for the three major categories of evaluation activities.

The grantee will submit a list of indicators for the surveillance portion of the evaluation. The list will indicate the research question(s) that each indicator addresses and will specify how each indicator will be operationally defined. The grantee will also submit plans specifying how the grantee intends to collect, clean, manage, and analyze the data on the indicators; coordinate with local evaluators at each site; obtain all necessary local clearances; and maintain confidentiality, particularly with regard to student data.

The grantee will submit a data collection plan for the process study. The plan will specify what information is to be collected at each site and which data collection methods the grantee intends to use. At the same time, the grantee will also submit a data analysis plan for the process study, indicating how the grantee intends to clean, manage, analyze, and report on the data. The grantee will also specify how it will coordinate with local evaluators at each site, obtain all necessary local clearances, and maintain confidentiality, particularly with regard to student data.

The grantee will submit a site selection plan for the intensive outcome evaluation that will specify how the grantee intends to select the sites, including comparison sites, for the study; recruit the comparison sites; and ensure their continued participation throughout the study. The grantee will then submit a proposed list of sites for the study.

The grantee will convene at least one meeting of local evaluators from intensive study sites, with participation by the IEMT, to discuss issues related to process and outcome (including economic) evaluation at these sites.

The grantee will submit a data collection plan for the intensive study. The plan will specify what information is to be collected at each site (that is, outcome data, including economic data; sufficient process data to provide context for the outcome data; and any other data the grantee considers relevant). The plan will also specify what data collection methods the grantee intends to use, how the grantee will coordinate with each site (that is, with the local evaluators at grantee sites and with appropriate personnel at comparison sites), how the grantee will obtain all necessary local clearances, and how the grantee will maintain confidentiality, particularly with regard to student data. At the same time, the grantee will also submit a data analysis plan for the intensive study, indicating how the grantee intends to clean, manage, analyze, and report on the data.

Instruments and OMB Clearance Packages

The grantee will submit all data collection instruments called for in the data collection plans for the indicator surveillance, the process study, and the intensive outcome study, including the common protocol(s).

The grantee also will submit OMB clearance packages for the indicator surveillance, the process study, and the intensive outcome study and will make changes to the clearance packages as appropriate to respond to OMB comments.

Reports

Reports of surveillance, process, and outcome measures will be used to satisfy the requirements of the Government Performance and Results Act.

Surveillance Reports. The grantee will submit reports for the indicator surveillance as follows: an annual interim report providing data for all sites for each year of the study and a final report summarizing all the data collected throughout the study.

Process Reports. The grantee will submit reports on aggregated process data for all sites included in the study as follows: an annual interim report providing process data for each year of the study and a final report summarizing all the process data collected for all sites throughout the study, with a focus on cross-cutting themes.

Intensive Study Reports. The grantee will submit reports for the intensive study as follows: an annual interim report providing data for each year of the study from the selected sites and a final report summarizing all the data collected throughout the study.

Other Reports. The grantee will use data from the process and intensive studies to prepare a nontechnical report intended for a general audience of practitioners and the public, with a theme of "What We've Learned." The report should inform practitioners and other concerned individuals about lessons derived from the evaluation (by the grantee and by various sites) about effective implementation; obstacles encountered and overcome; and the challenges, impact, and benefits of a coordinated and integrated communitywide approach to addressing issues of school safety and healthy development.

The grantee will cooperate with the U.S. Departments of Education and Justice in the production of the Annual Report on School Safety by providing information profiles, updated on an annual basis, on the progress of selected grantee sites in terms of surveillance indicators, implementation, and outcomes.

The grantee will cooperate with U.S. Departments of Education, Health and Human Services, and Justice in the production of Research in Brief reports or similar short reports summarizing major findings from the study or specific topics of interest.

Public Use Data Sets

The grantee will submit a plan for making core indicator data for all sites, process data for all sites, and intensive data for selected sites available for public use in a timely fashion (no more than 3 years after data have been collected and cleaned). The plan will specify measures to be taken to protect the privacy and confidentiality of all individual participants in the Initiative target population and, to the extent feasible, all other participants (e.g., school administrators, faculty, mental health and law enforcement officials). The grantee will submit appropriate technical data documentation for the public user, including code books. The grantee will submit a plan for dissemination of public use data sets and documentation, including how inquiries for technical assistance from potential public users will be addressed.

National Evaluation of the Safe Schools/ Healthy Students Initiative	Due Date: July 21, 1999