The IEMT will competitively award one cooperative agreement under this solicitation. Given the purpose of the evaluation, the overall evaluation design is intended to carefully document both the process and the outcome aspects of the initiative. The design should encompass the formation of the community collaboration, the impact of these collaborations on school safety and healthy student development, economic analyses, and surveillance of core indicators.

Goals and Objectives

The goals of the Initiative are as follows: (1) to help students develop the skills and emotional resilience necessary to promote positive mental health, engage in prosocial behavior, and prevent violent behavior and drug use; (2) to ensure that all students that attend the targeted schools are able to learn in a safe, disciplined, and drug-free environment; and (3) to help develop an infrastructure that will institutionalize and sustain integrated services after Federal funding has ended.

Given this mandate, the evaluation design must address the process and development of community collaborations to increase school safety and healthy child and youth development. Further, the evaluation must explore each of the six individual components of the collaboration: school safety, alcohol and other drug and violence prevention and intervention programs, school and community mental health preventive and treatment intervention services, early childhood psychosocial and emotional development programs, education reform, and safe school policies. The collaborative and each of the six facets must be examined in terms of surveillance (of core indicators), process, and intensive outcome analyses.

Collaboration: Surveillance, Process, and Outcome Measures

The overarching goal of the evaluation is to document the effectiveness of collaborative community efforts to promote safe schools and provide opportunities for healthy childhood development. Measures that address this question might include surveillance to determine the prevalence of various indicators of collaboration/integration of services (e.g., interagency agreements, joint planning activities, and jointly funded positions) and other mechanisms to encourage collaboration as well as a description of the frequency, nature, and intensity of cross-agency communications. Process measures might include an examination of how new and existing services have been linked and integrated; what changes have occurred in communication patterns; what provisions have been made for integrated, cross-agency professional development; and what kinds of modifications have been made to management information systems (MIS), management structure, agency policies, and/or philosophies. Outcome measures might include analyses to ascertain if the integrated collaboration is producing its intended effects and if enhanced collaboration results in changes in outcomes over and above the contributions of individual services, whether there were added costs or savings associated with achieving integrated collaboration, and whether agencies were able to share costs/resources.

Safe Schools/Healthy Students' Six Elements: Surveillance, Process, and Outcome Measures

In addition to examining the collaborative's impact, the evaluation design must include plans to explore the contribution of the six components of the collaborative. The following are examples of possible research questions for each of the six individual elements. This list is not meant to be exhaustive.

Examples of surveillance measures include:

• School Safety. Existence of partnerships with law enforcement, such as the number of personnel devoted to working with the target population; physical plan characteristics; presence or absence of random inspections; use of identification cards; and/or use of security devices.

• Alcohol and Other Drug and Violence Prevention and Intervention. Incidence and prevalence of alcohol and drug use; rates of interpersonal injury, weapon carrying, and gang-related crime in elementary, junior high, and senior high schools and the community; number of truant students; number of suspensions; and existence of programs to prevent or intervene in substance use/abuse and violent behaviors.

• School and Community Mental Health Preventive and Treatment Intervention. Incidence and prevalence of mental disorders (e.g., conduct and related problems, depression, anxiety disorders) among elementary, junior high, and senior high school children and youth; presence of screening, assessment, and referral mechanisms for mental disorders in the school setting; and presence of school-based mental health services for high-risk children and families.

• Early Childhood Pyschosocial and Emotional Development. Incidence of adverse mental health outcomes (e.g., conduct problems and other antisocial behaviors, depression, anxiety disorders) among young children; and number and types of services for early childhood psychosocial and emotional development (e.g., prenatal care, nurse home visitation programs, accredited infant and early childhood education programs).

• Education Reform. Levels of academic achievement in core subjects for selected grades (e.g., standardized test scores); average class size (ratio of students to classroom teachers); and number, type, and capacity of extended-day and summer programs.

• Safe School Policies. Presence of discipline codes, penalties for infractions, zero-tolerance for drugs and weapons on school premises, and policies and programs that address truancy.

Examples of process measures include:

• School Safety. Nature of the relationships among teachers, students, school personnel, parents, and school resource officers.

• Alcohol and Other Drug and Violence Prevention and Intervention. Implementation of new programs and services that are integrated into other new or existing programs and services; and changes in roles and responsibilities, communication patterns, referrals for treatment, and data sharing.

• School and Community Mental Health Preventive and Treatment Intervention. Implementation of preventive and treatment intervention programs and/or services -- access, fidelity, dosage.

• Early Childhood Pyschosocial and Emotional Development. Implementation of preventive intervention programs and/or services -- access, fidelity, dosage; processes used to coordinate services and communicate relevant information across key transition points between early childhood settings (e.g., between preschool and school entry); and nature of professional development programs related to early childhood psychosocial and emotional development that are available for early childhood teachers and other practitioners.

• Education Reform. Implementation of education reform efforts (e.g., professional development for teachers, high standards for all students); use of research-based approaches; and fidelity to research/models.

• Safe School Policies. Development and implementation of policy elements addressing truancy, discipline, prevention education/intervention, violence, alcohol and other drug use, sexual harassment, hate crime, and bullying. (Note: Participants in the development of policies should be identified.)

Examples of outcomes, including economic measures, include:

• School Safety. Impact of changes to school's physical design on crime, fear, and quality of life; cost of security measures versus cost of crime and violence; and indirect economic costs (e.g., maintenance of security measures, waiting in line at metal detectors).

• Alcohol and Other Drug and Violence Prevention and Intervention. Impacts of new programs or new systems for prevention, treatment, and referral for substance use/abuse or violent behavior; reductions in substance use/abuse and violence, such as rates of interpersonal injury, weapon carrying, gang-related crime, truancy, suspensions, and juvenile arrest rates; and cost of implementing and maintaining programs and services to prevent and reduce these problem behaviors in elementary, junior high, and senior high schools and in neighborhoods.

• School and Community Mental Health Preventive and Treatment Intervention. Impact of preventive services -- reduced incidence of adverse mental health outcomes in students, reduction in need for Individual Education Plans (IEP's), and reduction in mental health services referrals; impact of treatment services -- reduced prevalence of adverse mental health outcomes in students, attenuation of severity of disorder, and increased functioning; relative costs of risk and prevalence reduction through appropriate mental health screening, assessment, referral, and treatment services compared with failure to implement preventive and treatment services (e.g., IEP's, no mental health screening or referral programs).

• Early Childhood Pyschosocial and Emotional Development. Impact of preventive services -- prevalence of family conflict, abuse, and neglect; incidence of adverse mental health outcomes in children (e.g., conduct problems, antisocial behaviors, affective/anxiety disorders); reduction in IEP's; reduction in mental health services referrals; impact of early childhood services on children's successful academic and social transition to kindergarten and first grade; relative costs or risk reduction through prenatal and early childhood preventive services compared with costs associated with failure to reduce early risk (e.g., costs of poor prenatal care, IEP's, grade retentions, mental health service referrals).

• Education Reform. Impact of education reform efforts on school climate, orderliness, and safety; impact of other components of the initiative (e.g., early childhood education, safe schools policies) on students' academic achievement; and costs of implementing education reform efforts.

• Safe School Policies. Development and/or implementation of more effective safe school policies; reductions in truancy, discipline problems, and violent incidents; and costs, benefits, and utility associated with each safe schools policy element.

National Evaluation of the Safe Schools/ Healthy Students Initiative	Due Date: July 21, 1999