A Review of Instruments Measuring the Negative Consequences of Drinking in College Students

Studies indicate that of the nine million college students, about 45% engage in heavy or problem drinking [1]. There has been a growing research literature [2-6] aimed toward understanding the negative physical, social, and behavioral consequences that apply specifically to college students who misuse alcohol; these consequences include death, unintended physical injuries, assaults, property damage, arrests, fines, unwanted sexual experiences, decreased cognitive functioning, decreased academic functioning, and relationship difficulties [7]. In response to the growing importance of this topic especially from a research perspective, several instruments have been developed which provide measures of the types and extent of negative consequences experienced by college students who misuse alcohol (see below). Some of these instruments incorporate scales of negative consequences as part of a more comprehensive measure of alcohol abuse [8] while others primarily measure negative consequences of alcohol use [9].

The purpose of this article is to review the major most commonly used available instruments to assess the negative consequences of alcohol abuse in college students although it is hoped that this review will also be applicable to assessment in related areas. In order to limit the review to the specific topic at hand, we only considered instruments which were specifically developed to measure negative consequences of drinking in general college populations. We do not review instruments which are primarily used for clinical assessment which tend to primarily measure indications of, or the severity of, alcohol addiction. We do not consider instruments or scales which address the frequency and quantity of alcohol consumed [10]. There are many instruments designed to identify alcohol use disorders, alcohol dependence, and problem drinking for diagnostic and treatment purposes which are not particularly targeted for college students including the DSM- 5, Composite International Diagnostic Interview [11], Michigan Alcoholism Screening Test [12] and CAGE Questionnaire [13] etc. These instruments have been reviewed elsewhere [14-16] and will not be addressed in this article. In addition, there is some recent interest in the positive consequences of drinking [17] which is also beyond the scope of this review. Rather this review is limited to popularly used and psychometrically sound instruments which can be administered in survey format to better understand the negative consequences of drinking in college populations among casual as well as binge drinkers.

There is a confusing array of definitions as to what constitutes negative consequences of drinking both in terms of the categories of consequences which are measured and the type of items which are considered comprising a category of consequences. Perkins HW [18] comprehensive classification of negative consequences of risky drinking behavior divides negative consequences into the broad categories of damage to self, damage to other people, and institutional costs. Damage to self, includes academic impairment, blackouts, personal injuries and death, short and long-term physical illnesses, unintended and unprotected sexual activity, suicide, sexual coercion/rape victimization, impaired driving, legal repercussions, and impaired athletic performance. Damage to other people includes property damage and vandalism, fights and interpersonal violence, sexual violence, hate-related incidents, and noise disturbances. Institutional consequences refer to deleterious effects on the institutional well-being of colleges and universities; we will exclude this domain from further consideration in this article because it is not assessed by any of the measures considered in this article. However, in operationalizing the category definitions by delineating specific subcategories, Perkins HW [18] omits or neglects some potentially important characteristics: Regarding damage to self, Perkins HW [18] doesn’t mention doing something you later regret and forgetting where you were or what you did, characteristics assessed in several instruments noted below. Perkins includes the subcategory of “academic impairment”; but this is not clearly delineated and could include missing classes, getting behind in schoolwork, inability to study or concentrate, among others. Perhaps more importantly, entire categories of negative consequences are omitted, for example, referral for help (e.g., being referred to an alcohol treatment program or educational program), symptoms of alcohol dependence (e.g., craving a drink upon waking), or emotional problems which could result from alcohol misuse like anxiety or depression.

Instruments which contain measures of different dimensions of negative consequences differ markedly on the dimensions which are contained. For example, the Rutgers Alcohol Problem Index [9] includes three dimensions- dependent symptoms, social consequences, and personal consequences- while the Young Adult Alcohol Consequences Questionnaire [19] assesses seven dimensions – social interpersonal consequences, impaired control, diminished self-, poor self-care, risky behavior, academic/ occupational, physiological dependence, and black-out drinking. The lack of agreement as to what dimensions of negative consequences should be measured and what items should be included to measure each is problematic.

The three major psychometric issues relevant to this review are reliability, validity, and differential item functioning (the following discussion is a brief summary of relevant psychometric concepts; for an extended discussion see [20]). A measure’s reliability refers to its consistency over time: will the same results be reached if a person is measured at time 2, that were reached when they were measured at some earlier time 1. The most common methods of calculating reliability are coefficient alpha for internal consistency and the Pearson correlation coefficient for test-retest reliability. Validity is designed to address the extent to which the measure assesses what it is supposed to measure. There are a variety of dimensions of validity including content validity, convergent or concurrent validity, and construct validity. Content validity refers to whether the measure adequately covers the content of the domain which it is measuring. It is generally assessed by expert judgement using a variety of non-parametric statistics such as the percentage of experts who agree on the appropriateness of the content of the measure as a whole or of individual items. Convergent or concurrent validity refers to the extent to which the measure produces high correlations with other similar measures. For example, college students who score relatively high for negative consequences on the negative consequences scale of the SAQ [8] should be expected to score relatively high on the RAPI [9]. Construct validity is defined as the extent to which a measure exhibits relationship among its items as well as between scores on the measure and external variables which are theoretically justified. While there are a variety of ways of looking at and statistically testing for construct validity, one of the most common is factor analysis. Factor analysis looks at the structure of a scale to determine the number of dimensions within the scale. Confirmatory factor analysis (CFA) is used to determine if the number of dimensions specified and the items included for each dimension meet reasonable statistical criteria. More sophisticated item response theory (IRT) models are used to confirm specific theoretical formulations concerning the dimensions of a measure or to revise scales accordingly. Rasch models are often used to reduce multidimensional measures to unidimensional measures often by eliminating items resulting in shortened versions of the original instrument. Many of the measures discussed below have alternative versions with multi or unidimensional scales, respectively. Another important psychometric issue is differential item functioning testing whether scale items produce essentially similar results for different populations by race, gender, or other demographics. There is a continuing issue in the literature regarding how items concerning the negative effects of alcohol abuse differ by gender. Specifically, women might be more vulnerable to consequences such as regretted sexual experiences or sexual victimization than men, while men might be more likely to have committed property damage than women [18].

All the instruments presented below are available in questionnaire or survey format, could be self-administered, and have been used extensively for research on general college populations. An instrument developed as early as the late 1970’s to assess the severity of alcohol problems in college students is the Student Alcohol Questionnaire [8]. This instrument contains a problems resulting from drinking subscale with 18 items covering physiological symptoms such as vomiting or having a hangover, drunk or unsafe driving, trouble with the law, losing a job, academic problems, getting into fights, damaging property, inappropriate sexual activity, criticism by a dating partner, participating in drinking games, and a generic question of realizing one has a drinking problem; the subscale is considered to be unidimensional. The items are rated on 4-point Likert scales reflecting the occurrence and frequency of each consequence. This subscale has high levels of reliability [21]. This researcher was unable to find any psychometric studies of the validity of this instrument. The instrument has been translated into several languages [22].

Another instrument developed specifically to measure negative consequences of alcohol abuse in college students is the College Alcohol Problems Scale [23]. CAPS is composed of 20 items representing negative consequences of drinking. For each consequence, students are asked to indicate how often they had encountered the problems during the last year as a result of drinking alcoholic beverages on a five-point Likert scale from (1=never/almost never to 5=very often). A factor analysis finds two dimensions: socio-emotional problems and community problems with six and four items, respectively. The socio-emotional problems subscale includes items on depression, anxiety, troubles with family, friends, and intimate relationships. The community problems subscale includes acute physiological effects (e.g., vomiting, hangovers, memory loss) along with illegal, antisocial, and potentially dangerous activities (e.g., driving under the influence, problems with the law). Internal consistency analysis substantiated reliability for the two subscales [23]. Concurrent validity was established by correlations with the two subscales with both the Quantity-Frequency Index (QFI) and a modified version of the MAST. Construct validity was established with CFA validating the two- factor structure [24].

A brief version, CAPS-r, was developed by Maddock et al. [24] which was composed of only 8 items. This version contains two subscales, social problems (including items for unplanned sexual activity, driving under the influence, unprotected sex, and illegal activities) and a personal problems subscale (including items for feeling depressed, nervous, bad, or having appetite/sleeping difficulties). A CFA confirmed the two-factor model [24] and both criterion and construct validity was established by its reasonably high correlations with the YAAPST and a variety of other variables; these results held up across gender [24]. The CAPS-r was translated into Spanish and validated with a Spanish-speaking college student population [25].

The Rutgers Alcohol Problem Index [9] was developed to measure negative consequences attributable to drinking in adolescents and young adults and is used extensively in research with college students. The RAPI includes a list of 23 problems including getting into fights, causing shame or embarrassment, neglecting one’s responsibilities, noticing a change in one’s personality, and having a bad time. Students respond to the items indicating how often they have experienced these problems “ever” (T1) and “within the last three years” (T2) on 5-point Likert scales ranging from 1 (never) to 5 (more than 10 times). It includes two items related to academics- missed a day of school or work; and not able to do homework or study for a test. The test displays good internal consistency reliability at both T1 and T2 time periods. Construct validity was established by high correlations between the measure and alcohol use intensity. [9,26] presented a threefactor model comprised of subscales for dependent symptoms, social consequences, and personal consequences. However, [27] advocated for a one-dimensional model comprised of 16 items which demonstrated good psychometric properties using an IRT model. Differential Item Functioning (DIF) analysis by gender was conducted, and criterion related validity was established between total scores and polysubstance abuse. RAPI has also performed well psychometrically in an online format [28].

The College Alcohol Survey was originally developed to assess the health and behavioral consequences of binge drinking in college students [29] and was administered four times as part of the Harvard School of Public Health College Alcohol Study during the 1993- 2001 period [30]. It is a comprehensive survey with one subscale comprised of 12 items to measure negative consequences including missing class, getting behind in schoolwork, forgetting where you were, doing something you regret, requiring medical treatment, receiving a warning, being fined, having to do community service, getting a referral, and disciplinary action . The survey focuses on binge drinking and assesses secondhand effects of binge drinkers on non-binge drinkers with an additional 8 items covering disruption of sleep or studies, property damage, and verbal, physical, or sexual violence [30]. For each negative consequence, students are asked to indicate whether they experienced that consequence. This author was unable to locate psychometric studies on this instrument.

The Core Alcohol and Drug Survey [31] was developed in 1990 by the US Department of Education’s Fund for the Improvement of Postsecondary Education (FIPSE) as a self-report instrument to assess the nature, scope, and consequences of alcohol and other drug use among college students. Relative to the consequences, there is a subscale of 19 items which includes physiological effects (hangover, hangover more than 5 times, vomiting, memory loss), academics (poor academic performance, missing class), dangerous or illegal behavior (trouble with authorities, damaging property, driving under influence, arrest, suicidal behavior, sexual misconduct, been hurt or injured), as well as additional items including tried to stop using, doing something regrettable, and interpersonal problems (been in a fight, been criticized). Students are asked to respond on a 5-point Likert scale as to how often they experienced each consequence in the last year. A factor analysis conducted by De Bruyn et al. [32] yielded a revised instrument containing 7 items with two dimensions: “personal negative consequences” and “social negative consequences”. The personal negative consequences dimension refers to consequences experienced by the drinker themselves and contains three items: had a hangover, became nauseated or vomited, and missed a class. The social negative consequences dimension refers to consequences that affects other people as well as the drinker. This factor is measured by four items: got into an argument or fight, been criticized by someone I know, done something I later regretted, and been hurt or injured [32]. The two-factor model was reduced to produce a shortened version with a uni-dimensional scale comprised of these 7 items. The shortened version had good model fit in a confirmatory factor analysis as well as composite reliability, construct and concurrent validity [33].

The Young Adult Alcohol Problems Screening Test [34] assesses age-relevant consequences of alcohol abuse including getting into sexual situations that were later regretted, missing classes, and receiving lower grades than usual, and includes questions that assess more serious consequences common in clinical populations. Th YAAPST assesses the frequency of consequences during the past year as well as lifetime problems. The YAAPST is a unidimensional scale with good reliability (internal consistency and test-retest), criterion validity (with interview-based alcohol abuse/dependence diagnoses as the criterion), concurrent validity (comparing the YAAPST with other drinking measures), and construct validity (demonstrating substantial correlations with relevant personality, motivational, and peer influence variables) [34]. An IRT analysis supported the dichotomous scoring of responses and it was recommended to eliminate a few items based on differential item functioning (DIF) conducted by gender and race. A shortened 20- item scale is available along with norm tables to compare degree of symptom severity [35].

The Young Adult Alcohol Consequences Questionnaire [35] was developed to capture a broader spectrum of consequences than the YAAPST discussed above, especially those items indicative of less severe consequences which might be more prevalent in college students rather than clinical populations. In addition to an overall score, the measure assesses seven dimensions of alcohol consequences using a 48 item scale: social interpersonal consequences (6 items, e.g., “ I have become very rude, obnoxious or insulting after drinking”), impaired control (6 items, “I often drank more than I originally planned”), diminished self-perception (4 items, e.g., “I have badly about myself because of my drinking”), poor self-care (8 items, e.g., “I have been less physically active because of my drinking”), risky behavior (8 items, e.g., “I have taken foolish risks when I have been drinking”), academic/occupational (5 items, e.g., “I have neglected by obligations to family, work, or school because of my drinking”), physiological dependence (4 items, e.g., “I have felt anxious, agitated, or restless after stopping or cutting down on drinking”), black-out drinking (7 items, e.g., “I have awakened the day after drinking and found I could not remember a part of the evening before”). The response options are dichotomous (yes/no) to indicate whether that consequence had been experienced in the last 6 months as well as over the past 1 month. Confirmatory factor analysis supports the eight-factor solution with all factors loading on a single higher-order factor. Gender differences on total and subscale scores are reasonable and concurrent validity is established by correlating the total and subscale scores to the RAPI [19]. A briefer 24 item unidimensional version has been constructed using the Rasch model and it demonstrates good construct validity [35].

Considering the prevalence and severity of harmful consequences of student drinking among college students, and the increasing research interest in this area, it is incumbent upon researchers to be aware of the measurement instruments available to assess the nature and extent of negative consequences attributable to alcohol misuse in this population. The instruments reviewed represent psychometrically sound tools to further this investigation of negative consequences. Nevertheless, there are important limitations of these instruments and areas where further research is needed to validate existing instruments or develop additional ones. The lack of consistency in the definitions and conceptualization of negative consequences suggests the need for using content analysis for the development and evaluation of relevant measures. Major questions which are subsumed under this analysis would include: what are the dimensions of negative consequences to be assessed and what items should be included as part of each of these dimensions?

While most of the instruments have substantial psychometric validation, there is no consensus on the criterion which are most appropriate for criterion related validity studies. Instruments differ markedly on the criterion measures used with some using frequency of drinking as a criterion and others using alternate instruments such as RAPI scores. Regarding construct validity, the often multiple and conflicting factor analyses make it unclear as to how many dimensions are needed to represent the construct of negative drinking consequences. The uncertainty as to whether this is a unidimensional or a multidimensional construct is particularly problematic. It is also not clear how differential item functioning should be addressed. Clearly, there are consequences which are more common in one gender group than another; it is unclear as to whether items with DIF should be eliminated or whether norming tables should be developed by gender to assess the degree of seriousness of consequences to an individual student. In addition, there has been little attention to DIF among racial groups which is particularly troubling especially considering that alcohol-related behaviors differ significantly among Black Americans [36].

It is hoped that this review has alerted researchers interested in the negative consequences of alcohol misuse in college students to a variety of sound instruments to measure these consequences. It is also expected that the limitations noted above and others will be addressed by efforts to improve the above measures as well as to develop additional measures of this critically important construct.

None.

No conflict of interest.

1. Buscemi J, Murphy JG, Martens MP, McDevitt Murphy ME, Dennhardt AA, et al. (2010) Help-seeking for alcohol-related problems in college students: correlates and preferred resources. Psychol Addict Behav 24(4): 571-580.
2. Geisner IM, Lewis MA, Rhew IC, Mittmann AJ, Larimer ME, et al. (2016) Does one day of drinking matter? 21st birthday drinking predicts subsequent drinking and consequences. Addict Behav 64: 57-61.
3. Goodhines PA, Zaso MJ, Gellis LA, Park A (2019) Sleep-related functional impairment as a moderator of risky drinking and subsequent negative drinking consequences in college students. Addictive Behaviors 93: 146-153.
4. LaBrie JW, Earle AM, Hummer JF, Boyle SC (2016) Is prepartying a cause of heavy drinking and consequences rather than just a correlate? A Longitudinal Look at the Relationship Between Prepartying, Alcohol Approval, and Subsequent Drinking and Consequences. Subst Use Misuse 51(8): 1013-1023.
5. Pedersen ER, Skidmore JR, Aresi G (2014). Demographic and predeparture factors associated with drinking and alcohol-related consequences for college students completing study abroad experiences. J Am Coll Health 62(4); 244-254.
6. Reavy R, Cleveland MJ, Mallett KA, Scaglione NM, Sell NM, et al. (2016) An examination of the relationship between consequence-specific normative belief patterns and alcohol-related consequences among college students. Alcohol Clin Exp Res 40(12): 2631-2638.
7. Martinez JA, Sher KJ, Wood PK (2014) Drinking consequences and subsequent drinking in college students over 4 years. Psychol Addict Behav 28(4): 1240-1245.
8. Engs RC (1975) The Student Alcohol Questionnaire. Indiana University, Bloomington, Indiana, US.
9. White HR, Labouvie EW (1989) Towards the assessment of adolescent problem drinking. J Stud Alcohol 50(1): 30-37.
10. Collins RL, Parks GA, Marlatt GA (1985) Social determinants of alcohol consumption: The effects of social interaction and model status on the self-administration of alcohol. J Consult Clin Psychol 53(2): 189-200.
11. Robins LN, Wing J, Wittchen HU, Helzer JE, Babor TF, et al. (1989) The Composite International Diagnostic Interview: An epidemiologic instrument suitable for use in conjunction with different diagnostic systems and in different cultures. Arch Gen Psychiatry 45(12): 1069-1077.
12. Selzer ML (1971) The Michigan Alcoholism Screening Test: The quest for a new diagnostic instrument. Am J Psychiatry 127(12), 1653-1658.
13. Ewing JA (1984) Detecting alcoholism: the CAGE questionnaire. JAMA 252(14): 1905-1907.
14. Allen JP, Mattson ME (1993) Psychometric instruments to assist in alcoholism treatment planning. J Subst Abuse Treat 10(3): 289-296.
15. Dodgen CE, Shea WM (2000) Substance use disorders: Assessment and treatment. Academic Press, San Diego, CA, US.
16. Samet S, Waxman R, Hatzenbuehler M, Hasin DS (2007) Assessing addiction: concepts and instruments. Addict Sci Clin Pract 4(1): 19-31.
17. Zaso MJ, Park A, Kim J, Gellis LA, Kwon H, et al. (2016) The associations among prior drinking consequences, subjective evaluations, and subsequent alcohol outcomes. Psychol Addict Behav 30(3): 367-376.
18. Perkins HW (2002) Surveying the damage: A review of research on consequences of alcohol misuse in college populations. J Stud Alcohol Suppl (14): 91-100.
19. Read JP, Kahler CW, Strong DR, Colder CR (2006) Development and preliminary validation of the young adult alcohol consequences questionnaire. J Stud Alcohol 67(1): 169-177.
20. Crocker L, Algina J (1986) Introduction to classical and modern test theory. Harcourt Brace Jovanovich, New York, USA.
21. Engs RC, Hanson DJ (1994) The Student Alcohol Questionnaire: An updated reliability of the drinking patterns, problems, knowledge, and attitude subscales. Psychol Rep 74(1): 12-14.
22. Engs Ruth (1970) The Health Concerns Questionnaire. Unpublished Master's Thesis, IU Scholar Works, University of Oregon, Eugene, US.
23. O Hare T (1997) Measuring problem drinking in first time offenders: Development and validation of the College Alcohol Problem Scale (CAPS). J Subst Abuse Treat 14(4): 383-387.
24. Maddock JE, Laforge RG, Rossi JS, O Hare T (2001) The College Alcohol Problems Scale. Addict Behav 26(3): 385-398.
25. McCabe B, Escoto ER, Cruz Ortega LG, Munoz Rojas D, Gattamorta K (2017) Translation and psychometric properties of a Spanish version of the College Alcohol Problems Scale. Addict Behav 75: 36-39.
26. Martens MP, Neighbors C, Dams O Connor K, Lee CM, Larimer ME (2007) The factor structure of a dichotomously scored Rutgers Alcohol Problems Index. J Stud Alcohol Drugs 68(4): 597-606.
27. Shono Y, Ames SL, Edwards MC, Stacy AW (2018) The Rutgers Alcohol Problem Index for Adolescent Alcohol and Drug Problems: A comprehensive modern psychometric study. J Stud Alcohol Drugs 79(4): 658-663.
28. Thomas BA, McCambridge J (2008) Comparative psychometric study of a range of hazardous drinking measures administered online in a youth population. Drug Alcohol Depend 96(1-2): 121-127.
29. Wechsler H, Davenport A, Dowdall G, Moeykens B, Castillo S (1994) Health and behavioral consequences of binge drinking in college: A national survey of students at 140 campuses. JAMA 272(21): 1672-1677.
30. Wechsler H1, Lee JE, Kuo M, Seibring M, Nelson TF, et al. (2002). Trends in college binge drinking during a period of increased prevention efforts. Findings from 4 Harvard School of Public Health College Alcohol Study surveys: 1993-2001. J Am Coll Health 50(5): 203-217.
31. Presley CA, Harrold R, Scouten E, Lyerla R, Meilman PW (1993) Core Alcohol and Drug Survey User’s Manual. 4^th edn, Carbondale: Southern Illinois University, US.
32. De Bruyn S, Wouters E, Ponnet K, Van Damme J, Maes L, et al. (2018) Problem drinking among Flemish students: beverage type, early drinking onset and negative personal & social consequences. BMC Public Health 18(1): 234-242.
33. De Bruyn S, Wouters E, Ponnet K, Van Damme J, Van Hal G (2017) The psychometric properties of a shortened Dutch version of the consequences scale used in the Core Alcohol and Drug Survey. PLoS One 12(12): e0187876.
34. Hurlbut SC, Sher KJ (1992) Assessing alcohol problems in college students. J Am Coll Health 41(2): 49-58.
35. Kahler C W, Strong DR, Read JP (2005) Toward efficient and comprehensive measurement of alcohol problem continuum in college students: The brief young adult alcohol consequences questionnaire. Alcohol Clin Exp Res 29(7): 1180-1189.
36. Desalu JM, Goodhines PA, Park A (2019) Racial discrimination and alcohol use and negative drinking consequences among Black Americans: a meta-analytical review. Addiction 114(6): 957-967.