Some students also indicated that they enjoyed learning about drugs, what is and isn’t a drug, and how to identify harmful substance use behaviors. One student noted the information they learned through RAYS was “valuable” and that they could “take [it] with [them] in life” citing the program as a “second chance” to change their health behaviors. Multiple students also noted the core components of peer involvement and tailored support. One student specifically mentioned how it “wasn’t just adults involved” but also other “kids going through things just like [them], with the same experiences that [they] have”. Enrollees also cited the people involved as important contributors to making the program helpful and enjoyable. Students described the individuals involved in administering and implementing RAYS as welcoming and open-minded, fostering a non-judgmental environment. One student noted how RAYS advocates and staff are “open and listen to what you have to say” and provide valuable support in helping the students obtain the resources they may need.The self-reflective nature of RAYS components was also brought up by multiple students. They noted how RAYS helped them to reflect on the mistakes that they had made in an educational rather than a punitive environment. These alternative to punitive approaches were highly regarded by students as something valuable and engaging. One enrollee said that they “liked that it gave [them] an opportunity to understand what [they] did wrong without just being taken out of school” while also giving them “the chance to take constructive criticism about [their] substance use”. Students also noted the community engagement piece in positive regards as something that allowed them to maintain a relationship with their peers and school community. When asked what they disliked about the program, most students noted that there wasn’t anything they felt that they strongly disliked or would like to see done differently.

Nonetheless, a few enrollees cited aspects related to the logistics of program delivery and knowledge of peer advocates. Some students believed that the program was time-consuming and interfered with their academics. For instance, indoor grow rack two students noted that they were pulled out of class to attend RAYS-related activities as required by their Restorative Plans. A few students also said that the program felt more prolonged compared to just “being suspended and then going back”. Multiple students also felt that much of the program was not a choice, despite RAYS being a voluntary program. One student noted that they felt like they were “forced” to “give a formal apology” while others did not appreciate that some of their peers acted like they did not want to be there. However, a few students did feel that more education and training for the peer advocates was needed. For instance, one student noted that they believed peer advocates needed more training on how to approach sensitive topics that may arise during discussions between advocates and enrollees. When it came to the program timeline, students who cited these aspects had mixed responses. Some believed that more time was needed to be able to complete the required components of their Restorative Plans, while others felt that the program was overextended and time-consuming. Total suspension counts by academic year for RAYS schools and non-RAYS schools are reported in Figure 3. “RAYS schools” represent the four target school sites in Nevada County implementing the RAYS program while “non-RAYS” schools are the selected comparable sites not implementing RAYS. Figure 3 presents the total number of suspensions reported to the CDE for the 2017-18 through 2021-22 academic years at these sites. Prior to the launch of RAYS in August of 2021, Nevada schools reported a higher number of suspensions overall when compared to the non-RAYS sites. Examination of these counts from pre- to post-launch of RAYS reveals a notable difference in the number of suspensions between RAYS and non-RAYS schools, with RAYS schools reporting less than half of those at non-RAYS sites after this time point.

The total number of drug-related suspensions at both RAYS and non-RAYS sites from the 2017-18 to 2021-22 academic year are presented in Figure 4. The CDE considers possession, use, sale, or furnishing of any opiates, opium derivatives, hallucinogenic substances , depressants, stimulants, alcohol, tobacco products and any other controlled substance listed in chapter 2 of the California Health and Safety Code as a drug-related incident warranting suspension. 11,14 In the 2018-19 academic year, Nevada County sites reported almost three times the number of drug-related suspensions in relation to comparable schools . Examination of these suspension counts revealed that after RAYS was implemented in August of 2021 , RAYS schools reported approximately half the amount of drug-related suspensions as the non-RAYS schools.Table 10 provides a breakdown of the exit reasons and completion statuses for all students who exited RAYS during this period . Roughly 4 out of 5 students who exited the program were marked as ‘successful completions’, meaning they completed most if not all the components outlined in their Restorative Plans. Of these successful completions, 17.94% returned and joined RAYS as peer advocates. For the remainder of the sample, RAYS services were found to be inappropriate or unsuitable for 6.25% of students, 4.17% left school or were lost to follow-up, and 8.33% left for another reason. Out of all students, 70.83% completed all components outlined in their Restorative Plans while 29.17% did not. These completion rates are in line with the program objective to reach a 75% successful completion rate among RAYS enrollees by April 2024 .The increasing utilization of restorative practices to address disciplinary incidents in school settings has highlighted the need for evaluations to assess the effectiveness of such programs in lieu of traditional punitive measures. This study reported findings from an intermediate evaluation of an RJP program with AOD use education and treatment counseling components. To properly execute this evaluation, student pretest and posttest survey data was analyzed to assess individual-level impacts of the RAYS program on AOD use behaviors and perceptions of substances, self-responsibility, and resource awareness. Process measures including all activity-level and enrollee case management data were tabulated and examined to assess implementation fidelity in meeting program goals and objectives outlined in the original grant proposal.

Discipline data was used to inform evaluation questions pertaining to changes in overall and drug-related suspensions from pre- to post-implementation of RAYS in Nevada County sites. Of the sample of students who submitted pretest and posttest data , most reported favorable behavioral changes from pre- to post-exposure to RAYS. The majority of RAYS enrollees reported a decrease in past 30-day use of alcohol, marijuana, and vapes with nicotine or just flavoring, indicating a potentially favorable impact of the program in addressing student AOD use behaviors. Additionally, exposure to the RAYS program also seemed to have an impact on student awareness of substance use support and services at their sites, with the majority of students saying they would be able to identify such entities at their schools. In general, students also had positive experiences with RAYS and believed the resources provided were helpful. Examination of the trends in disciplinary incidents from the 2017-18 to 2021-22 academic year revealed a decrease in the total number of suspensions within RAYS schools over time. In relation to comparable sites with regional and demographic similarities, schools at which RAYS was being implemented reported a decreasing trend in suspensions while non-RAYS schools reported an increase. Results from this intermediate evaluation of RAYS also allow for an assessment of implementation fidelity in meeting the original goals and objectives outlined in the project proposal and local evaluation plan . The three main goals of the program were to reduce overall suspension rates at the four target sites, reduce youth marijuana and other substance use, and increase access to SUD treatment services. Relevant process and outcome measures used to inform implementation fidelity assessment and to measure goal and objective attainment are outlined in Tables B and C in the Appendix. Process evaluation measures encompass acquisition of services, RAYS completion rates and statuses , and recidivism rate calculations. The outcome evaluation measures include variables such as the suspension rates, suspension counts, AOD use rates, indoor farming equipment and perceptions of AOD use. Assessment of success in meeting some of these goals and objectives is further discussed in the sections below as they pertain to the relevant process and outcome measure variables. One of the objectives was to reduce marijuana and other substance use among youth who participated in the RAYS program. Based on findings from pretest and posttest data, the majority of students who exited RAYS during the evaluation period reported a decrease in past 30-day use across all products including marijuana, vapes with nicotine or just flavoring, and alcohol. Students who indicated that they used marijuana or vapes reported the highest percent decreases from pretest to posttest. Given that one of the main reasons for establishing RAYS in NCSOS was to address high marijuana, vape, and other drug use rates among students, the decreases seen from pre- to post-exposure of the RAYS program is indicative of the potentially positive impacts on student substance use behaviors. Program components addressing substance use such as the harm reduction classes and SUD counseling may be playing an important role in educating students on the harms of using and providing necessary emotional and mental health support, respectively. However, pretest and posttest data for the harm reduction classes, although outlined in the project proposal, has not been collected due to logistical challenges with survey administration.

Therefore, it is recommended that harm reduction pretests and posttests be administered to students participating in this component in order to assess the specific impacts of these educational sessions on AOD use behaviors and perceptions. Nonetheless, when it comes to harm perceptions, the proportion of students who believed marijuana use was harmful was relatively low compared to other substances. These low harm perception rates for marijuana use reported among the RAYS sample are similar to national trends. For instance, in the most recent report of the MTF survey, about 22% of adolescents in the US perceived regular use of marijuana as risky.2 In comparison, 33.3% of RAYS students thought it was very or extremely harmful to use marijuana every day at pretest. This proportion decreased to 28.6% at posttest, following similar percent decreases reported in recent MTF survey cycles. Nonetheless, perceived harm of everyday vape and alcohol use was higher, but no favorable changes were noted for either product. Low perceived harm may be attributed to a variety of social and environmental factors. For marijuana specifically, researchers have highlighted increased legalization of marijuana for medicinal and recreational purposes as a factor for low harm perceptions among youth. Furthermore, growing public acceptance of marijuana has also been noted as a reason for the decrease in perceived harm among adolescents. This absence in perceived harmfulness of marijuana, despite notable decreases in use rates among RAYS students, sheds light on the potentially significant impacts that social environment and awareness of substances have on adolescent AOD perceptions. Although RAYS may play an important role in addressing use behaviors, these intermediate findings reveal the effects of the program on AOD harm perceptions was minimal. Nonetheless, more research is needed to specifically assess whether harm reduction class components may play a role in this and if not, how course components may be adapted to further target AOD use harm perceptions and awareness. It is crucial to continue monitoring AOD use rates and perceived harm over time in order to inform public health messaging and education efforts. Within the context of RAYS, it is also important that the formal evaluation consider assessing these same AOD use and perception variables at the school-wide level to compare data from the RAYS sample to the broader student body in Nevada County. This comparison will allow for a higher-level examination in order to detect any shifts in school-wide culture with regards to AOD use and harm perceptions. Previous literature has emphasized the importance of gathering data on student experiences in restorative justice programs in order to inform adaptations and enhancements.28 Furthermore, authors also highlight a gap in current restorative program evaluation research with regards to limited or the lack of student feedback in the form of quantitative and qualitative data.