Only 32% of the businesses listed on Yelp were verified to be active brick-and-mortar dispensaries. This is not surprising because Yelp, which provides a general business listing service not specifically designed for marijuana industry, had more records irrelevant to marijuana dispensary than Weed maps and Leafly. Taken together, no single secondary data source could provide a reasonably complete and accurate list of active brick-and-mortar dispensaries in a large state like California. We recommend surveillance and research to consider their unique strengths and weaknesses when a single data source is used to minimize required resources. When resources are available, we recommend the integration of multiple secondary data sources, preferably including a licensing directory and multiple online crowd sourcing platforms, as well as verification through phone calls such as what has been done in this study or through even better approaches such as a field census. The verification could considerably improve the accuracy of the data compiled from secondary data sources. Our findings were overall consistent with the two smaller-scale studies conducted in California, both in Los Angeles County. One was conducted in 2016-2017, before recreational marijuana dispensaries were allowed to open . This study obtained medical marijuana dispensary information from five online crowd sourcing platforms. Weed maps was suggested to be the most accurate and up-to-date platform, contributing to 95% of the final records. Call verification was conducted in 10% of the dispensaries and found to generally align with the information posted on online crowd sourcing platforms. The other study was conducted in 2018-2019, after recreational marijuana dispensaries were allowed to open . It extracted data from Weed maps and Yelp and verified dispensary information through site visits. About 80% dispensaries that were determined to be active through online data cleaning were confirmed to be active in site visits,cannabis grow equipment and licensed dispensaries accounted for roughly 40% of the active dispensaries.

Neither study reported validity statistics for each specific data source. Our study expanded on the prior research by covering a much larger geographic region, computing detailed validity statistics for each data source by dispensary category and county population size, and by using two gold standards and two tests to demonstrate validities in different scenarios and for different purposes. This study has limitations. First, due to the lack of feasibility of conducting a field census in such a large geographic region, phone calls were made to verify information obtained from secondary data sources. While this approach was cost effective, businesses not listed in these secondary data sources were excluded from the analysis, potentially the smaller, unlicensed dispensaries that did not intend to promote themselves on online crowd sourcing platforms because of cost and law enforcement concerns. Future research using field census approach is warranted to assess to what extent unlicensed dispensaries were underrepresented in our study. We could also have mis-classified dispensaries as inactive if they provided incorrect contacts or could not be reached after multiple call attempts. Search terms in Yelp may not successfully capture all marijuana-related businesses. As a result of these caveats, our call-verified, combined database would be an underestimation instead of the true “universe” of the active dispensaries in California. Second, validity measures were not all applicable in some scenarios where “true negative” or “false positive” could not be identified with the current study design. Third, regulations on online crowd sourcing platforms have been rapidly evolving. Before our data collection, Weed maps served as the major platform to advertise and promote dispensaries including the unlicensed ones in California. Right after our data collection, California regulators required Weed maps to remove unlicensed businesses from its website. By January 2020, Weed maps had removed over 2,000 businesses . Weed maps may no longer be a good data resource for identifying unlicensed dispensaries, particularly in California, even though it had satisfactory validity statistics in our study. Future studies should consider alternative crowd sourcing platforms that post unlicensed dispensary information. Fourth, we evaluated the three most commonly used online crowd sourcing platforms.The findings were not applicable to commercial providers of business listings, either, such as Info USA and Dun & Bradstreet that recently incorporated marijuana businesses into their databases. Finally, findings may not be generalizable to the identification of delivery-only services or dispensaries in other states.

Notwithstanding the limitations, the findings of this study provide empirical evidence regarding the validity of using secondary data sources to identify brick-and-mortar marijuana dispensaries in a large geographic region. The data collection and verification protocol and validity statistics could be used by local governments and communities to best strategize regular surveillance on the availability and accessibility of marijuana dispensaries and their compliance to laws. Future research could also use these findings to replicate dispensary identification in other states where marijuana has been commercialized. We hope a comprehensive and accurate enumeration of marijuana dispensaries could facilitate future research evaluating marijuana dispensaries and their impacts on public health. One of the objectives of the NIH-initiative, the Adolescent Brain Cognitive Development Study, is to establish a national, multi-site, longitudinal cohort study to prospectively examine the youth from childhood through adolescence to examine the risk and protective factors influencing the trajectories of substance use and its consequences, examine the impact of detailed patterns of substance use on neurocognitive development, health and psychosocial outcomes, and to study the interactive relationship between substance use and psychopathology in youth . The goal of this article is to provide an overview of the ABCD Study Substance Use Work group goals, rationale for the substance use battery, and detailed methods of the battery in order for the scientific community to achieve improved harmonization in substance use assessment, which have varied widely, especially in measurement of frequency/quantity patterns of use . The Substance Use module was developed for the ABCD Study by the Substance Use Work group, comprised of experts on assessment of substance use quantity and frequency patterns, SUD diagnostic interviews, influences on substance use risk, and dimensional assessment of substance use problems and consequences in adolescents. The Substance Use Work group Co-Chairs are Drs. Mary Heitzeg and Krista Lisdahl . The Substance Use Work group members include Drs. Kevin Conway , Sarah Feld stein Ewing , Raul Gonzalez , Sara Jo Nixon , Devin Prouty , Kenneth Sher, , Susan Tapert , and Gordon Willis . In determining methods and constructs to measure, the work group considered the ABCD Study aims and requested methodology outlined by the ABCD Study NIH funding opportunity announcement .

The work group met weekly or biweekly and identified three primary areas to be measured: 1) factors impacting risk of substance use; 2) assessment of detailed substance use patterns; and 3) consequences of substance use. Constructs within these domains were identified by the work group utilizing member input, literature review, and consultation with instrument authors and external experts. During the process of finalizing the battery the work group prioritized instruments that demonstrated sound psychometric properties, fit the longitudinal design, were developmentally appropriate, reduced participant burden, were open-access, and could be administered by computer. In order to improve cross-study harmonization, if an instrument fit these criteria, priority was given to instruments provided by PhenX Patterns of Substance Use module for adolescents , the Monitoring the Future Study , the Population Assessment of Tobacco and Health Study , and the National Consortium on Alcohol and Neuro development in Adolescence Study . Further, great care and consideration was put into organizing a gating structure to avoid exposing non- or low-using children to novel substances . After developing the draft protocol, the work group received and integrated feedback from the NIH advisors and the ABCD Study Coordinating Center, and piloted the protocol at multiple sites with 9–10 year olds to ensure youth comprehension, confirm data quality and timing. Consistent with the goals of the ABCD Study, curated data and detailed data dictionaries, including all the substance use measures, will be released yearly to the NIMH Data Archive .One of the goals of the ABCD Study is to characterize youth prior to the initiation of significant substance use. Adolescence is a period of ongoing neuro development that is linked with an increase in risk-taking behaviors, including the onset of substance use . Initiation of drinking alcohol and use of most illicit substances typically begins in the early teen years, although high-risk demographic communities report initiating use during the elementary and early middle school years . In the U.S., among 8th graders,cannabis grow table lifetime use of alcohol , electronic cigarettes , cannabis , tobacco cigarettes inhalant , prescription amphetamines and prescription tranquilizers are the most commonly used substances . Data is unavailable for 8th graders, but an alarming 18% of 12th graders have used any prescription drug and 7.8% of 12th graders report non-medical use of prescription pain relievers . The latter is a particularly important area, given increase risk of developing an opiate use disorder associated with adolescent exposure, significant barriers to treatment, and alarming rate of overdose deaths in adolescents . Caffeine use is very common in youth, with 73.9% of 6–11 year olds consuming caffeinated food or beverage on any given day within the past week and adolescents consuming an average of 50 mg per day . It is notable that detailed data on substance use patterns in 9- and 10- year olds is less frequently reported, as the youngest age US national surveys assess is 12 or 13 years old [e.g., the MTF begins the assessment in 8th grade while the National Survey on Drug Use and Health begins at age 12]. Data that are available for youth younger than 12 comes from state assessments , such as the Texas School Survey of Drug and Alcohol Use, which measures substance use in youth attending grades 4–6 .

This survey reports lifetime use for the following drug categories in 4th graders: alcohol , nicotine , cannabis , and inhalants − other drug categories were not assessed. This survey also revealed that a significant portion of 4th graders report that they never heard of cannabis , inhalants , nicotine , and alcohol . Taken together, data suggests that youth may initiate first sipping or trying substances in late childhood , and incidents of substance use initiation increase from late childhood into early adolescence. Notably, although some youth may be sipping alcohol or trying tobacco, the vast majority of 9 and 10 year olds are substance-naïve and indeed may not have heard of several drug categories. Thus, studies assessing this age group need to avoid exposing substance-naïve youth to new substance use concepts.As stated above, one area identified by the Substance Use Work group to measure is factors that influence risk of substance use initiation, substance use trajectories, and substance use consequences, such as early sipping alcohol or puffing tobacco, acute initial subjective response, drug curiosity and intentions to use, peer substance use, parental rules, and availability of substances. Community samples have shown that up to a third of 8- and 9-year olds report sipping alcohol , demonstrating that very early substance experimentation begins in late childhood. Studies have found that early sipping predicted drinking onset by age 14 . Similarly, Jackson and colleagues found that sipping alcohol prior to 6th grade predicted drinking a full drink, getting drunk, and drinking heavily by 9th grade, even after controlling for a range of etiologically relevant environmental and individual difference covariates. In contrast to the literature on alcohol, which sometimes operationalizes determinants of a sip and having a full drink as distinct, the field of nicotine has not tended to make this distinction. With few exceptions , studies rarely distinguish between having had a puff and having had one or more cigarettes and data are generally missing on interim levels of progression from a puffs to first cigarette or to regular smoking. Even less is known about the progression of trying a puff or taste of cannabis to more regular experimentation. Closely assessing initial tobacco or cannabis use could help to both characterize the progression of substance use across substances as well as help to determine variables key to such progression. Another important factor to measure is individual acute subjective response to early substance experimentation, such as level of response to alcohol, which has been found to predict risk of developing alcohol related consequences in teens and alcohol use disorders in adulthood .