Some students believed that the program was time-consuming and interfered with their academics. For instance, two students noted that they were pulled out of class to attend RAYS-related activities as required by their Restorative Plans. A few students also said that the program felt more prolonged compared to just “being suspended and then going back”. Multiple students also felt that much of the program was not a choice, despite RAYS being a voluntary program. One student noted that they felt like they were “forced” to “give a formal apology” while others did not appreciate that some of their peers acted like they did not want to be there. However, a few students did feel that more education and training for the peer advocates was needed. For instance, one student noted that they believed peer advocates needed more training on how to approach sensitive topics that may arise during discussions between advocates and enrollees. When it came to the program timeline, students who cited these aspects had mixed responses. Some believed that more time was needed to be able to complete the required components of their Restorative Plans, while others felt that the program was overextended and time-consuming.Total suspension counts by academic year for RAYS schools and non-RAYS schools are reported in Figure 3. “RAYS schools” represent the four target school sites in Nevada County implementing the RAYS program while “non-RAYS” schools are the selected comparable sites not implementing RAYS. Figure 3 presents the total number of suspensions reported to the CDE for the 2017-18 through 2021-22 academic years at these sites. Prior to the launch of RAYS in August of 2021, Nevada schools reported a higher number of suspensions overall when compared to the non-RAYS sites.

Examination of these counts from pre- to post-launch of RAYS reveals a notable difference in the number of suspensions between RAYS and non-RAYS schools, equipment for growing weed with RAYS schools reporting less than half of those at non-RAYS sites after this time point.The total number of drug-related suspensions at both RAYS and non-RAYS sites from the 2017-18 to 2021-22 academic year are presented in Figure 4. The CDE considers possession, use, sale, or furnishing of any opiates, opium derivatives, hallucinogenic substances , depressants, stimulants, alcohol, tobacco products and any other controlled substance listed in chapter 2 of the California Health and Safety Code as a drug-related incident warranting suspension. 11,14 In the 2018-19 academic year, Nevada County sites reported almost three times the number of drug-related suspensions in relation to comparable schools . Examination of these suspension counts revealed that after RAYS was implemented in August of 2021 , RAYS schools reported approximately half the amount of drug-related suspensions as the non-RAYS schools.Table 10 provides a breakdown of the exit reasons and completion statuses for all students who exited RAYS during this period . Roughly 4 out of 5 students who exited the program were marked as ‘successful completions’, meaning they completed most if not all the components outlined in their Restorative Plans. Of these successful completions, 17.94% returned and joined RAYS as peer advocates. For the remainder of the sample, RAYS services were found to be inappropriate or unsuitable for 6.25% of students, 4.17% left school or were lost to follow-up, and 8.33% left for another reason. Out of all students, 70.83% completed all components outlined in their Restorative Plans while 29.17% did not. These completion rates are in line with the program objective to reach a 75% successful completion rate among RAYS enrollees by April 2024 .

The increasing utilization of restorative practices to address disciplinary incidents in school settings has highlighted the need for evaluations to assess the effectiveness of such programs in lieu of traditional punitive measures. This study reported findings from an intermediate evaluation of an RJP program with AOD use education and treatment counseling components. To properly execute this evaluation, student pretest and posttest survey data was analyzed to assess individual-level impacts of the RAYS program on AOD use behaviors and perceptions of substances, self-responsibility, and resource awareness. Process measures including all activity-level and enrollee case management data were tabulated and examined to assess implementation fidelity in meeting program goals and objectives outlined in the original grant proposal. Discipline data was used to inform evaluation questions pertaining to changes in overall and drug-related suspensions from pre- to post-implementation of RAYS in Nevada County sites. Of the sample of students who submitted pretest and posttest data , most reported favorable behavioral changes from pre- to post-exposure to RAYS. The majority of RAYS enrollees reported a decrease in past 30-day use of alcohol, marijuana, and vapes with nicotine or just flavoring, indicating a potentially favorable impact of the program in addressing student AOD use behaviors. Additionally, exposure to the RAYS program also seemed to have an impact on student awareness of substance use support and services at their sites, with the majority of students saying they would be able to identify such entities at their schools. In general, students also had positive experiences with RAYS and believed the resources provided were helpful. Examination of the trends in disciplinary incidents from the 2017-18 to 2021-22 academic year revealed a decrease in the total number of suspensions within RAYS schools over time. In relation to comparable sites with regional and demographic similarities, schools at which RAYS was being implemented reported a decreasing trend in suspensions while non-RAYS schools reported an increase.

Results from this intermediate evaluation of RAYS also allow for an assessment of implementation fidelity in meeting the original goals and objectives outlined in the project proposal and local evaluation plan . The three main goals of the program were to reduce overall suspension rates at the four target sites, reduce youth marijuana and other substance use, and increase access to SUD treatment services. Relevant process and outcome measures used to inform implementation fidelity assessment and to measure goal and objective attainment are outlined in Tables B and C in the Appendix. Process evaluation measures encompass acquisition of services, RAYS completion rates and statuses , and recidivism rate calculations. The outcome evaluation measures include variables such as the suspension rates, suspension counts, AOD use rates, and perceptions of AOD use. Assessment of success in meeting some of these goals and objectives is further discussed in the sections below as they pertain to the relevant process and outcome measure variables.One of the objectives was to reduce marijuana and other substance use among youth who participated in the RAYS program. Based on findings from pretest and posttest data, the majority of students who exited RAYS during the evaluation period reported a decrease in past 30-day use across all products including marijuana, vapes with nicotine or just flavoring, and alcohol. Students who indicated that they used marijuana or vapes reported the highest percent decreases from pretest to posttest. Given that one of the main reasons for establishing RAYS in NCSOS was to address high marijuana, vape, and other drug use rates among students, the decreases seen from pre- to post-exposure of the RAYS program is indicative of the potentially positive impacts on student substance use behaviors. Program components addressing substance use such as the harm reduction classes and SUD counseling may be playing an important role in educating students on the harms of using and providing necessary emotional and mental health support, respectively. However, pretest and posttest data for the harm reduction classes, although outlined in the project proposal, has not been collected due to logistical challenges with survey administration. Therefore, it is recommended that harm reduction pretests and posttests be administered to students participating in this component in order to assess the specific impacts of these educational sessions on AOD use behaviors and perceptions. Nonetheless, when it comes to harm perceptions, the proportion of students who believed marijuana use was harmful was relatively low compared to other substances. These low harm perception rates for marijuana use reported among the RAYS sample are similar to national trends. For instance, in the most recent report of the MTF survey, grow tables 4×8 about 22% of adolescents in the US perceived regular use of marijuana as risky. In comparison, 33.3% of RAYS students thought it was very or extremely harmful to use marijuana every day at pretest. This proportion decreased to 28.6% at posttest, following similar percent decreases reported in recent MTF survey cycles. Nonetheless, perceived harm of everyday vape and alcohol use was higher, but no favorable changes were noted for either product. Low perceived harm may be attributed to a variety of social and environmental factors. For marijuana specifically, researchers have highlighted increased legalization of marijuana for medicinal and recreational purposes as a factor for low harm perceptions among youth. Furthermore, growing public acceptance of marijuana has also been noted as a reason for the decrease in perceived harm among adolescents. This absence in perceived harmfulness of marijuana, despite notable decreases in use rates among RAYS students, sheds light on the potentially significant impacts that social environment and awareness of substances have on adolescent AOD perceptions.

Although RAYS may play an important role in addressing use behaviors, these intermediate findings reveal the effects of the program on AOD harm perceptions was minimal. Nonetheless, more research is needed to specifically assess whether harm reduction class components may play a role in this and if not, how course components may be adapted to further target AOD use harm perceptions and awareness. It is crucial to continue monitoring AOD use rates and perceived harm over time in order to inform public health messaging and education efforts. Within the context of RAYS, it is also important that the formal evaluation consider assessing these same AOD use and perception variables at the school-wide level to compare data from the RAYS sample to the broader student body in Nevada County. This comparison will allow for a higher-level examination in order to detect any shifts in school-wide culture with regards to AOD use and harm perceptions. Previous literature has emphasized the importance of gathering data on student experiences in restorative justice programs in order to inform adaptations and enhancements.28 Furthermore, authors also highlight a gap in current restorative program evaluation research with regards to limited or the lack of student feedback in the form of quantitative and qualitative data. This study sought to address this gap in data collection through the inclusion of open text and quantitative items in the posttest instrument to gather data on student perceptions of the RAYS program and their overall experiences. Items included in the survey assessed resource awareness, self-reflection of AOD use, and the likelihood of recommending the RAYS program to peers. Overall, student resource awareness increased for substance use treatment and support services. More students believed they could find a resource or individual for themselves or a peer after going through RAYS. This puts RAYS on track to meet the program objective of increasing access to and awareness of SUD treatment services . Another quantifiable objective was to provide SUD treatment services to at least 100 students by April 2024. Based on case data, of the 63 students who enrolled in RAYS during this evaluation period, 40 were referred and participated in at least one substance use counseling session . Therefore, at this state, the program is just under 50% of the target goal. It is important to note that not all students who go through RAYS are diverted from drug-related disciplinary incidents, thus may not require SUD treatment services.Prior to the launch of the RAYS program, Nevada County sites reported higher overall and drug-related suspensions in relation to selected comparable sites in a neighboring county. Overall suspension counts were notably higher in Nevada County schools, reporting over double the number of suspensions in relation to comparable sites during the 2017-18 and 2018-19 school years. Nonetheless, a sharp decline in the 2021-22 academic year, after the launch of RAYS, implies some effect of the program on the disciplinary landscape in Nevada County sites. This notable reduction in the number disciplinary incidents after implementation of an RJP program has been reported in previous studies. Hashim and colleagues noted that at the 1-year mark after implementation of a restorative justice program, there was a significant reduction in the number of suspensions. This same trend is seen in the initial decline in the number of disciplinary incidents in Nevada County sites implementing RAYS.