Visual Cues Can Interrupt Low-Return Decisions: An Experimental Analysis of Melioration in Behavioral Addiction

Suboptimal behavior, including addiction, can be conceptualized as the consequence of a decision strategy called melioration [1,2] that is utilized in choice situations in which the value of an alternative is affected by the rate of its availability. It is a function of the economic principle of supply and demand [3,4]. The higher the rate of availability of an alternative (usually associated with lower cost), the lower the overall value [5]. Melioration can result in negative consequences that are not recognized by the individual until their cumulative negative effect becomes unavoidable [6,2,7]. This negative effect may go unrecognized because individual decisions in the series of repeated choices are not perceived as adding much weight to the overall consequences of the series [8]. This failure to perceive the overall outcome has been referred to as the “primrose path to addiction” [7] (p. 74).

Melioration as Part of the Addictive Process

The economic principle of melioration depends on the individual’s ability to detect the rate (or value) of each of the competing alternatives (sources of reward) that are available in a choice situation. If the rate is undetectable, behavior is likely to be a random switching between alternatives. If detectable, then as long as the value of one alternative is higher than that of the others that are available, the probability that a person will continue to select that alternative will remain high [9,7]. The individual is likely to switch to an alternative when the rate of the current reward is no longer higher than the reward of the alternatives [10,5].

Some behaviors produce negative overall effects when they are selected. As smaller, less valuable overall choices are repeatedly selected, the overall value of all the alternatives diminishes because the subjective value of the larger reward is repeatedly devalued when compared in individual decisions. In the long run, the overall value of both the larger and the smaller alternatives equalize. This pattern of choosing represents the addictive process at work. Small choices are repeatedly made that eventually strip away long-term value of all the alternatives [11,2]. In the end, little reinforcement is enjoyed [10,12]. Melioration is, therefore, a value-reducing strategy and stands in contrast to the traditional economic principle of maximization. Reversal of this melioration process is difficult for the individual in recovery from behavioral addiction without a method of changing the process of valuing decisions [13]. It requires the repeated selection of an alternative that has long-term reinforcement value but that is typically delayed. Additionally, the value of this type of reinforcer is often perceived to be low compared to immediately available [14,15].

Melioration vs Maximization

Melioration seems to be inconsistent with traditional economic theory, which suggests that humans tend to maximize overall utility, that is, to prefer behaviors that optimize or produce the highest rate of overall reinforcement [7,9,10]. One reason people might engage in a melioration choice strategy is a limitation on the amount of information available to them when they assign value to the options that are present. A person with a limited perspective (that is, with a myopic or shortsighted view of the situation) may weigh the options more heavily in favor of the option that currently yields higher utility [6,7], regardless of any consequences that selecting that option may have on future returns from the entire set of options.

Current subjective value could be assigned by perception of supply and demand, availability that is present or delayed [16], magnitude of reward [9], or the context of choice [10]. If the person has a broader perspective of the situation (that is, considers future as well as present rewards), value may be assigned to each option based on both positive and negative consequences for future valuation. The ability to evaluate the overall outcomes of the options in context (that is, over time) may allow their present values to be differentiated more accurately [2,7,17]. This evaluation could include the effects of recent decisions on the present situation, leading to a decision that will yield the greatest value overall (maximization).

Self-Control and Precommitment

Self-control is defined by Logue as the choice of a larger, delayed outcome over a smaller but immediately available alternative. Impulsiveness refers to the choice of the latter. According to Logue, to be able to choose a larger, delayed outcome, one is required to be sensitive to the rates of return both now and in the future. With such sensitivity intact, it is possible to engage in a precommitment strategy that can be implemented while both the smaller, less valuable alternatives and the larger, deferred alternatives are still in the future. That is, one can make a binding decision for the richer alternative before one arrives at the point where the poorer alternative is immediately available [16].

Time Discounting and Restructuring

Herrnstein, et al. [9] hypothesized that people might be impulsive because they (1) cannot retain and process enough information about the rates of response and the rates of return, (2) are unaware of the relationship between the two categories of rates, or (3) follow a principle called time discounting. The principle of time discounting suggests that the individual discounts the value of the larger reward in favor of the smaller reward because of the greater proximity (in time) of the latter [14,18]. As the delay to the choice point becomes smaller, the value of the immediate reward will increase to the point that it has a greater current value than that of the larger, more delayed reward. The ability to abstain from choosing the currently better option and instead to wait to get the larger one has been demonstrated experimentally with humans and animals in various situations [10,19]. Nevertheless, both humans and animals can fail to demonstrate self-control by making choices that result in a smaller overall reward [7,20,21].

In connection with pre-commitment strategies mentioned previously, another key to maximizing overall return over a series of decisions is called restructuring. This is the act of incorporating more information (or context) into the current utility-yield prediction [22,7]. In other words, considering more information before choosing among alternatives allows the person to see the current choice as part of a broader set of choices [2,17,23] and base their decision on the overall outcome rather than on the immediate choice outcome.

Experimental Design Validating Melioration and Restructuring

Prominent in behavioral economics is the principle of matching. Herrnstein [22] explained that when several contingencies of reinforcement are in operation concurrently, subjects will match the relative rates of reinforcement from those contingencies. In other words, subjects’ behavior will “match” the reinforcement contingencies provided in the situation, maximizing their overall reinforcement. Numerous experiments with animal subjects have been published in support of melioration as an explanation of suboptimal choice [24]. Different methods were used in these experiments, but generally the designs provided a choice between two schedules of reinforcement that were presented to the subject simultaneously (concurrent schedules). For instance, research has evaluated manipulations of increasing/decreasing value of stimuli using rats [25].

Reports of melioration with human subjects appear less often in the literature. Additional research with humans is warranted [4]. One of the major difficulties in these designs has been establishing one that provides an immediately available reinforcement that, when chosen, increases the negative consequences of the available options as melioration predicts. Herrnstein et al.’s study [9], which involved human subjects, asked whether humans meliorate or maximize and sought to identify the conditions in which each type of decision making is likely to occur. This foundational study provided a method that demonstrated the process of melioration with humans in a context that may be used to experimentally document the addictive process described earlier [26,27].

Definitions

As an introduction to Herrnstein et al.’s [9] experimental design, a few definitions may be helpful. The term internality is defined by Herrnstein et al. [9] as the effect of a persons’ recent allocation of behavior on future returns. This allocation of choices may impact the value the participant assigns to the alternatives in each situation. In the context of the experimental design, the averaging window is closely related to internality. The averaging window is simply the number of past decisions that are included in the calculation of the consequences of the current decision. Increasing the size of the window (that is, using a greater number of past responses in the calculation of the current choice consequence) decreases the detection of the internality. If too many past choices are considered, the impact of each individual response on the overall payoff is likely to be negligible and not recognized by the participant. If the averaging window is small (only a few past choices are involved in the calculation), the internality is high, and each choice dramatically changes the consequence for each subsequent choice.

Rationale for Replication and Detailed Description of Herrnstein, et al.

A primary purpose of the present study was to revisit a foundational research study on melioration [9] in an entirely different setting. This would establish the reliability of the experimental design to accurately detect melioration and also introduce real-time cues to influence decision making. Such replication addresses the gap in psychological research left by isolated studies and narrow scholarly networks [28]. Additionally, it provides the mechanism to ‘close the loop’ on study reports of limitations to address and next steps to explore. Therefore, the methods of the original study on melioration are described in detail here.

In the original experiments reported by Herrnstein, et al. [9], the reward was an animated coin that appeared on a computer monitor and was later exchanged for money. In the third experiment, however, reward delay rather than amount was varied. Two mutually exclusive alternatives were presented to the subject. Each produced an equal amount (one cent). The delay in delivering the coin (and thus the delay to the next trial) was a function of the proportion of past responses for the shorter delay. However, as the proportion of choices for the shorter delay increased over the previous 10 trials (the averaging window), the length of the delay for both coins also grew on each trial. Melioration would predict that the longer-term effects (the increasing delay for both alternatives) would be ignored in favor of local preference, that is, the coin providing the shorter delay would always be chosen. But by continually choosing the shorter delay, future reward (the number of trials the subject could complete within the fixedduration session) would be consistently reduced. Thus, the subject would earn significantly less money during the session than might have been earned.

Given the payoff functions utilized, maximization would predict exclusive selection of the choice with the larger delay. This would have caused the overall delay to be minimized (never increasing the delay). More trials would have been completed within the session and more money earned as a result. In this way, the time delay became an immediately consumable reinforcer that the participant was able to experience. In most types of choice experiments, reinforcements (money, points, etc.) are received after the experiment has concluded. This reduces the likelihood that humans will respond to experimental procedures in a manner consistent with melioration because all reinforcements are delayed until the conclusion of the experimental session. By making this factor immediately available, this design better approximates the immediate payoff (and consequences) made in the human decision process.

In the original study, subjects were placed in an experimental session in which they could choose one of two mutually exclusive alternatives to press the right or the left arrow key on a computer keyboard that was placed in front of a monitor. After a key was pressed, a coin fell from a dispenser on the right or left side of the monitor into a coin holder below. While the coin was falling from the dispenser into a collector, no further choices could be made. The left coin would always take 2 seconds longer to fall than the right coin. The time required for the coin to reach the collector constituted the delay before the next trial could begin. The length of the delay was an increasing linear function of the number of right-key choices during the most recent 10 trials (the averaging window).

Research Questions

• Does an experimental design intended to produce a pattern characteristic of melioration result in behaviors confirming melioration?
• When introduced following the onset of a melioration behavior pattern, do different visual cues significantly reduce the pattern of melioration in favor of maximization?
• Do the effects of visual cues on response patterns endure after the visual cues are withdrawn?

Thus, the present research was an attempt to correct limitations to previous research and provide validation of the ability to interrupt melioration in humans and encourage self-control and maximization in an experimental setting. This study used simulations from a foundational study [9] to represent a situation in which addictive processes and melioration are relevant and then extended that study by providing visual cue aids in promoting maximization.

General Method

A within-subjects design was used to determine whether an individual’s behavior would shift toward maximization and away from melioration within the experimental session. A traditional ABA design recorded initial melioration in the first portion (baseline) followed by exposure to visual cues exposing the internality (treatment). A final period resembled the first in which no cue was provided.

Participants and Recruitment

Participants consisted of 76 students of Brigham Young University who were recruited by multiple Psychology 111 class instructors during winter semester 2004. Potential participants were instructed to contact the principal researcher via e-mail to schedule a session. When participants arrived at their scheduled time, they were required to read and sign a consent form stating the risks and benefits for participation in the study. Participation was voluntary and participants were informed they could leave the experimental session at any time they wished. Participants were instructed that in leaving, they would receive the amount of money earned to that point in the session but would not receive a $2.50 session completion fee. Participants were also informed that upon completion of Experiment 1, some participants might be invited to participate in a follow-up session. Participation for the follow-up session was also to be voluntary, and no penalty would be imposed should they choose not to return.

This study was conducted with the approval of the Institutional Review Board Brigham Young University. Only students 18 years of age and older were included in the study. The mean age of all participants was 20 years old, with ages ranging from 18 to 32 years old. Participants consisted of 38 females and 38 males. Thirty participants (40%) were freshman, twenty (26%) were sophomore, twenty (26%) were juniors, five (7%) were seniors, and one (1%) identified as “other.”

Apparatus

A “money machine” computer program modeled after the descriptions provided by Herrnstein, et al. [9] was written in Python programming language, with output delivered to Microsoft Excel. During the experiments, the program was run on Windows PC operating system, with each computer console operated as an independent experiment station in separate private rooms. In each of the rooms, there was only a desk (upon which sat the computer, monitor, keyboard and mouse) and a seat for the person. No clock or other timing device was available in the experimental session.

Procedure

Research assistants were kept blind as to the experimental condition that each of the participants would receive. Participants were assigned to the experimental conditions sequentially based on numbers and letters representing the experiment, cue, and laterality variables. Once the participant had signed the consent form, a research assistant would escort the participant to an experimental room and enter information from the tracking sheet into the computer and thereby select the appropriate program to run. Once the program had begun, the research assistant would remain in the room while the participant entered her or his demographic information into the computer. Each participant was asked to enter her or his age, class status, and gender and a unique, non-identifying participant code into the program, used only to match the participant to their session earnings within that experiment. The research assistant exited the room when an instruction screen was displayed, and the experiment concluded with open-ended questions assessing participants’ perceptions of the experiments (Appendix).

Experiment 1

All 76 participants completed Experiment 1. Twenty-six individuals completed the experiment with a condition of viewing corresponding visual graphs (13 males and 13 females), 26 showing a timer (13 males and 13 females), and 24 with both cues present (12 males and 12 females). In this experiment, each participant completed a 20-minute session, with all participants receiving the same instructions. Next, each participant was given a practice period of 1 minute in which to become familiar with the experimental procedure. A 5-min baseline condition then began in which the participant earned money in the absence of visual cues. At the conclusion of that time, a 1-min break was taken, followed by a 5-min period in which the participant continued earning money and in which one of the three visual cue options was presented (the graph, timer, or a combination). Following that interval, another 1-min break occurred. The final 5-min period was a return to the baseline condition in which money was earned but no visual cues were present. A “thank you” screen indicating that the experiment was concluded appeared when the final 5-min period expired.

Once a participant completed the experimental session, a research assistant gave the participant a post session survey with her or his corresponding identification number. During the time that the participant filled out the survey, a research assistant would access the Microsoft Excel file for that session and identify the total number of coins collected within the session by the participant. The program recorded the number of coins from each time period as well as the total number of coins collected over the entire session. After the participant had completely filled out the post session survey, the research assistant would pay her or him 1 cent for each coin collected during the experiment plus $2.50 as an experiment completion fee.

Additionally, the software program recorded the ratios of choices corresponding to maximization for each time period as well as for the entire session. The program also recorded an indicator informing the research assistant that she or he was to invite the participant to complete a follow-up session. The indicator represented whether or not the participant had satisfied the selection criteria to be included in Experiment 2. If this indicator appeared on the participant’s Excel file, the research assistant would invite the participant to complete the second experiment.

Experiment 2

Participants in Experiment 1 who responded in the second time period with at least 55% of their choice selections representing maximization were candidates for Experiment 2. Using a slightly higher criterion than chance responding (50%) was intended to include those participants who at least partially responded in a pattern representing maximization. The second criterion for inclusion in Experiment 2 was a return to a lower level of responding during the final time period by those who initially responded with more than 55% maximization in the second time period.

A return to a lower level of responding was defined as an overall drop of 25% or more maximization responses in the final time period compared with responses in the second time period. In this way, participants were selected on a relative decrease in performance based on their own responding pattern. Based on these criteria, 37 participants in Experiment 1 qualified to complete Experiment 2, and all consented to participate. Fifteen were male (40.5%) and 22 (59.5%) were female. Participants ranged in age from 18 to 23 years old, with a mean age of 20. Participants consisted of 38% freshman, 38% sophomores, 19% juniors, and 5% seniors. Nine participants received the timer cue (6 females, 3 males), 13 received the graph cue (6 female, 7 male), and 15 received the combination cue (9 female, 6 male).

The second experiment assessed whether a fading procedure would aid individuals in maintaining maximizing choice allocation following. The procedure was almost identical to that described in Experiment 1 and each participant was matched to the cue she or he had previously received. The only variation occurred in the second time period. After the second money earning portion of the experiment began, the visual cue was present for the first 10 trials made. Once those 10 trials were made, only the first 8 of the next 10 trials had the visual cue present. After 8 subsequent choices were made, the cue was removed for 2 choices. The next 10 trials consisted of the first 6 with the cue present and the remaining 4 without the cue present. Ten trial blocks with the visual cue present for 4 and 2 choices respectively concluded the time period. A 1-min break followed, and a final 5-min money-earning time period concluded the session, as with Experiment 1.

Behavior choice allocation that more closely approximated maximization in the second time period compared with the first would represent exposure to the internality and resultant behavior modification. Choices more similar to maximization in the third time period compared with the first would suggest that the fading procedure was a more successful way of removing the cue and allowing the participant to maintain awareness of the internality in the absence of the visual cue.

Analyses

In each of the time periods (baseline, treatment, baseline) the total number of responses made by each participant as well as the ratio of responses consistent with maximization within the time period were recorded. The ratio scores were used in all statistical analyses. Preliminary analyses were conducted to determine if differences in scores were evident across participant gender or across the laterality of the presentation key (left or right). An alpha level of .05 was used for all statistical tests. An a priori analysis of group differences based on gender or laterality were not significant. Therefore, all subsequent statistical analyses were based on the full group of participants. Treatment effects were analyzed using one-way repeated measures ANCOVA comparing participants’ ratio responses from the three time periods. Differences across the type of cue presented (graph, timer, and combination) were analyzed using one-way between subjects ANCOVA of the ratio scores during the treatment period, controlling for the initial ratio score in the first baseline condition.

Experiment 1 Results

Treatment Effect

Evaluations of participants’ ratio responses across the three time periods yielded a statistically significant effect, F(2, 150) = 4.947, p = .008, partial η² = .06, observed power = .80. Follow-up protected t tests revealed that ratios representing maximization increased significantly from the first baseline period (m = .34, sd = .25) to the treatment period (m = .42, sd = .24), t(75) = 2.92, p = .005, dz = .34, 95% CI of the mean difference [.03, .15]. Ratio scores decreased significantly from the treatment period (m = .42, sd = .24) to the second baseline period (m = .34, sd = .30), t(75) = 2.72, p = .008, dz = .31, 95% CI [.02, .14]. No significant difference existed between the two baseline condition ratio scores, t(75) = 0.20, p = .845.

Cue Presented

We next evaluated differences across cue presentation (graph, timer, and combination) of the ratio score during the treatment period. The baseline scores were significantly related to treatment period scores, F(1, 72) = 16.432, p < .001, partial η² = .19, observed power = .98. The main effect of cue type was not significant, F(2, 72) = 0.58, p = .563, partial η² = .02, observed power = .14, with scores from groups receiving the graph (m = .45, sd = .30), the timer (m = .37, sd = .20), and the combination (m = .45, sd = .18) cue not significantly different from each other after controlling for the initial ratio score in the first baseline period.

Experiment 2 Results

Treatment Effect

A one-way repeated measures ANCOVA was calculated comparing participants’ ratio responses from the three time periods. A significant effect was found, F(2, 72) = 9.67, p < .001, partial η² = .21, observed power = .98. Follow-up protected t tests revealed that ratios representing maximization increased significantly from the first baseline period (m = .17, sd = .22) to the treatment period (m = .27, sd = .23), t(36) = 3.15, p = .003, dz = .52, 95% CI of the mean difference [.04, .17]. Ratio scores decreased significantly from the treatment period (m = .27, sd = .23) to the second baseline period (m = .15, sd = .20), t(36) = 3.46, p = .001, dz = .57, 95% CI [.05, .19]. No significant difference existed between the ratio scores of the two baseline conditions, t(36) = 1.06, p = .295.

Cue Presented

A one-way between subjects ANCOVA was calculated to examine the effect of cue presented (graph, timer, and combination) on ratio score during the treatment period, covarying out the effect of initial ratio score in the first baseline condition. First baseline scores were significantly related to treatment period scores, F(1, 33) = 16.264, p < .001, partial η² = .33, observed power = .98. The main effect of cue type on treatment-period responding, controlling for initial baseline ratio score, was not significant, F(2, 33) = 3.06, p = .060, partial η² = .16, observed power = .55. Scores from groups receiving the graph (m = .07, sd = .10), the timer (m = .31, sd = .27), and the combination cue (m = .35, sd = .23) were not significantly different from each other after controlling for the initial ratio score in the first baseline period.

Experiment 1 Discussion

The Addictive Process Represented

During the initial period of Experiment 1, participants responded in a manner consistent with melioration. This result helps support the validity of the present research model in replicating previously cited methods. Strong preference for a melioration strategy of choice selection seems to be quickly established and maintained. Paralleling a pattern indicative of a melioration strategy suggested by the statistical data is the finding that 30 of the 76 participants (29%) indicated on their post session surveys that they would recommend a strategy of determining which key resulted in the shortest initial delay and then to select only that key. One participant wrote, “I compared the time it took for each dispenser to drop a coin and get the coin in the cup. Once I figured out which dispenser went faster, I weighed down the arrow key to the corresponding dispenser and just watched the money come.” This participant earned much less money than those not using a choice strategy exclusively based on melioration.

Treatment Effect

Having established a baseline condition from which comparisons could be made, the second period of the experiment was intended to aid participants in recovery from the addictive nature of the choice situation. Participants in Experiment 1 demonstrated choice allocation significantly less consistent with melioration during the second time period of the session in which the visual cue was present than they did during the initial phase (baseline). This change of responding was notable, particularly when considering that most of the participants had already experienced a decrease in the overall value of their choices as a consequence of their previous selection of the key representing melioration. Participants were much more likely to go through a process of selecting the key representing maximization and waiting longer than they had previously.

In the post session written responses, more than 50% of the participants indicated that during the treatment periods, they experimented with their choice allocation based on the cues presented. Interestingly, when this overall result was broken down by the type of cue received, it was found that only 5% of those receiving the timer cue reported a strategy of experimenting with their choice selection. Of those participants receiving the graph and combination cues, post session survey responses indicated experimentation with choice selection was a strategy used by 80% and 53% respectively. Participants receiving the two cue presentations including the graph appear to have been more likely to experiment with choosing the longer delay key (maximization) than were the participants receiving the timer cue only. Fourteen of the participants (18%) indicated that they experimented with their choices and eventually decided to strike a balance between the two choices.

A participant receiving the graph wrote, “When the pie chart came up, I found that the right side slowed the more I clicked it, so I experimented with the left and saw it sped up. Then I tried the right and saw it dropped faster than before, but slowed again, so out of curiosity I clicked the left till the whole chart was red and saw the rate stayed at a constant 5 seconds.” That same participant also indicated that the graph was helpful in finding the trends of the coins that influenced her strategy for choosing.

The third time period of the experiment represented a return to the baseline condition. With the removal of the cues presented, the mean ratio of responses representing maximization dropped significantly to the same ratio as in the first baseline condition. Further examination of the trends from these time periods revealed that although the mean ratio was identical, the distribution of scores was very different across the two baseline conditions. In the first baseline condition, 25 of the participants (33%) allocated approximately half or more of their responses to the key representing maximization. That number increased to 34 (44%) in the treatment condition with the cue present. During the second baseline condition, 30 participants (39%) allocated approximately 50% or more of their responses to the maximization key. That is a change of only 5% fewer than the treatment condition, and a notable increase from the first baseline period, in which only 7 participants (9%) had this type of score.

Overall, the results indicated that, for this group, a treatment effect occurred and there was a significant decrease in meliorating behavior when the cues were presented. There was also a significant increase in meliorating behavior when the cues were removed, although the total number of people maximizing after the cue was removed was higher than the baseline period.

Differences Between Cue Presentations

There were no statistically significant differences in maximization ratios across the three types of cues presented. Participants receiving the graph and combination cues consistently allocated more of their choices to the key representing maximization than did those receiving the timer cue; however, the differences were nominal. Another indication of the compatibility of cues comes from the responses on the post session surveys. Across the three groups, 10 participants (40%) receiving the timer cue specifically stated that there was a relationship between the keys. Nine participants (36%) receiving the graph cue indicated a relationship between the keys. Ten participants (38%) stated they noticed a relationship between the keys and the delay. These responses suggest that a similar number of participants across all three cue presentations indicated awareness of part of the internality, or delay function, leading to maximizing behavior.

Identification of the Internality

Another interesting indicator of cue presentation efficacy is a tally of participants who were able to clearly describe the internality in their qualitative responses after completing the experiment (Appendix A). Correct identification of the internality in place would suggest choice selection could be based on the overall value of the choice and the process of melioration would be less likely to be used. A total of 8 participants (approximately 10%) specifically identified the internality on their post session surveys. Five of these participants had received the timer cue and 3 of them had received the combination cue. One participant who received the timer cue wrote, “Left [maximization]-wait a few trials, it will speed up and be constant. Right [melioration]-the first few are fast, then it slows down.” Several participants, not just those able to verbalize the internality in place described restructuring and selfcontrol strategies such as these.

Experiment 2

Experiment 2 was intended to provide an alternate means of fading a cue in an attempt to promote learning. During the initial baseline condition in which no cue was present, the mean ratio of choices corresponding to maximization was low, with participants allocating more than 80% of their choices to melioration. This group of participants likely responded this way because there was little learning effect demonstrated within the previous session.

Treatment Effect

During the treatment period with intermittent cues, participants from the graph and combination groups experimented much more than those receiving the timer cue. The overall result was that participants made many more attempts to change their patterns of responding when the visual cues were present compared to the baseline time periods. All participants who wrote that they recognized a relationship between the choices received the graph or the timer cue. Nine of the eleven specifically indicated that they made their choice selections in an attempt to maximize overall reward. These participants seemed to gain a greater perspective of the value of patterns of choices rather than focusing on any particular choice. Six participants indicated that they actively experimented with both choices because they noticed the cues and used them to try to increase their reward. Three participants indicated that they used a strategy of keeping a balance between their choices.

Three participants correctly stated the internality in operation. One participant who received the graph explained, “The right [maximization] continues to reward people the more you stick with it. The left [melioration] has instant gratification but slows down quickly and ruins your prospects for the investment of time with the right key.” The original hypothesis was that participants receiving a fading procedure would maintain a higher level of responding following the treatment period of the experiment. The mean ratio of choice allocation to the key representing maximization during the second baseline phase was .15. This indicates that as a group, participants did not maintain their level of responding and returned to a lower level of choice allocation to maximization. In fact, whereas 6 participants allocated half or more of their choices to maximization in the first baseline time period, only 4 participants did so in the second baseline time period.

Many of the participants indicated on their surveys that they did notice the cues presented but that they only experimented with their choosing while the cues were visible. For example, one participant said, “Use only the left [melioration] arrow unless the pie chart is present. If it is, alternate between the arrows.” The intermittent presentation of the timer likely drew more attention to them than in previous experiments. Those participants that noticed and attended to the timers were more accurate in describing the internality. Other participants did notice the timers but chose to ignore them because they did not know what they represented.

Differences Between Cue Presentations

Of the 37 participants completing Experiment 2, all 9 of the participants receiving the timer cue followed a melioration strategy and 7 of them specifically indicated as much on their post session surveys. Only 2 of them reached or exceeded the 50th percentile in overall earnings for Experiment 2. Many participants wrote that they did notice the timers, but that they did not pay much attention to them because they were not sure how to interpret the information the timers were presenting. More than half of the participants receiving the graph cue indicated there was a relationship between the two choices and indicated the need for consistency when choosing the key with the longer initial delay (one indicated that 10 choices was the criteria for delay changing). Though several participants indicated there was a relationship between the two choices available, many of them reported that they only used that information to influence their decision when the cue was present.

Fifteen participants receiving the combination cue completed Experiment 2. These participants seemed to represent more equality between maximization, melioration, and balancing strategies for decision making. Four participants stated there was a relationship between the two choices and indicated that they used the graph to determine which key to push, and one participant correctly identified the internality. Only one of them indicated that she used the timer at all. Six participants indicated a strategy of exclusive melioration with one stating, “I wasn’t willing to keep using the left [maximizing] dispenser.” Three participants stated that they did experiment with their choice allocation, and two said that they kept their choices balanced. In Experiment 2, the hypothesis that a learning effect would be recorded following the fading procedure was not confirmed. There was support for a treatment effect, though the effect was more modest than expected.

General Discussion

Within-Subjects Design

The within-subjects design maintained the same amount of actual time for the experiment as Herrnstein, et al. [9] and the pilot study, while decreasing the exposure to the cue during the treatment time period. Participants in the current study had a maximum exposure time of 5 minutes. Because most participants indicated (in their written reports and with their patterns of responding) that a strategy of melioration was quickly set up within the first baseline condition, a 5-minute period seemed to be an adequate allotment of time for them to establish a pattern of behavior. However, a 5-minute period of time may not have been sufficient for experimentation with the choice selections to result in exposing the internality. Future experiments may attempt to establish an appropriate level of exposure to the visual cue that would produce an optimal illumination of the internality.

Verbal Identification of a Relationship Between Choice Consequences

In the present study, participants provided written descriptions of their experiences in the experiments (Appendix). In those responses, 40% of the participants receiving the timer indicated a relationship with their choice selections. In the present study, during the treatment time period, the timer was visible. This may have increased the novelty of its presentation and elicited more attention. Participants in the present study did not have the additional instructions and no option for activation of the cues was provided. Requiring participant activation of the cues may be another improvement on the present design that could increase detection of the internality. Future experiments may require participants to activate and deactivate the cues, increasing the possibility of the cue receiving more attention during the experimental session.

Potential Applications of the Findings

In these experiments, visual cues representing aspects of the internality were presented to participants to assist them in allocating their choices based on overall value, not just local value. The presence of the visual cue may have promoted more behavioral experimentation, allowing the participants to notice the effects of their individual choices on the overall outcome of the series of choices. This type of restructuring may benefit people in choice situations representing specific addiction-prone behaviors. Promoting variability in choice may reduce the exclusive use of melioration and may help interrupt the addictive process. One implication of these findings is that although the internality present in the choice situation might not have been completely exposed, the presence of the visual cues seemed to have fostered more experimentation with choice allocation. Many participants indicated that they did not maintain a choice strategy exclusively based on melioration in the presence of the visual cue. In these cases, the visual cues may have served as a sign to change behavior. Some participants explained that they were willing to experiment with different choice allocations when the graph was present, sometimes allowing them to become aware of the relationship between the keys. Individuals who use a choice strategy exclusively based on melioration may never be exposed to the effects of choices more consistent with maximization. By experimenting with choice selection, individuals are more likely to gain a better perspective of the consequences of their choices.

The visual cues were intended to decrease the cognitive demands by easing the requirements in assessing the values of choices both immediately and in the future. Making that process overt may further assist in increasing maximizing. The internality of this experiment included an increasingly larger delay to the next coin with each selection of the key representing melioration. Though several of the participants recognized there was a relationship between the choices (keys), only a few verbally described the internality specifically. Participants may benefit from an explanation of the internality either prior to or within the session. With this information, participants may experience the effects of the internality and become more sensitive to the rates of availability of the choices encountered. In the experimental situation, participants may be more sensitive to the key representing maximization.

Limitations of the Study

The participants for this study were recruited exclusively from undergraduate introductory psychology courses in 2004. Therefore, this sample does not represent a current sample of individuals. Future studies may be helpful in expanding the applicability of these findings to include populations representing a more broad range of ages, levels of education, cultural backgrounds, socioeconomic statuses, etc. Post hoc power analyses conducted using the recovered original dataset indicate the primary treatment effects were well powered (observed power = .80 and .98 for the repeated-measures treatment effect in Experiments 1 and 2, respectively; observed power = .98 for the baseline covariate in both experiments), while the non-significant cue-type main effects were underpowered to detect a small-to-moderate effect (observed power = .14 in Experiment 1; .55 in Experiment 2). The marginal cue-type effect in Experiment 2 (p = .06) may reflect insufficient power at n = 37 rather than a true absence of an effect, and this should be read as an open question for replication rather than a settled null result.

The task of choosing between two choices (keys) was intended to represent a choice between two mutually exclusive alternatives. Often, human decisions are much more complex than a twochoice model. Outside this experimental setting, the choice of one alternative over another will likely include a variety of other factors in addition to the internality that may contribute to the overall value assigned to that choice (e.g., time available to make the choice, clarity of the choice alternatives, physical and environmental factors, history of selection, etc.). A possible next step for research would be to specify a choice that could be isolated and measured to extend the applicability of this research to other choice situations.

The present research study was intended to provide experimental support to previously stated theories of melioration and behavioral addictive processes with human participants. The results of this research study replicated a historically foundational study [9] suggesting that melioration and the addictive process can be recorded in an experimental setting. As an improvement on previous methods, this study suggests that once established, melioration may be interrupted temporarily in favor of maximization by providing information about the internality present in a choice situation. Notably, that effect did not endure once visual cues were withdrawn. Overall, participants receiving more information about the effect of their choice allocations altered their responding from a strategy of melioration. Some participants selected a strategy of balancing their choices, while others identified the internality present and adopted a strategy consistent with maximization. Future research designs may further clarify conditions in which human melioration may be interrupted and the internality of choice situations may be exposed, leading to more optimal behavior choices.

The authors affirm that they have no conflicts of interest.

1. Herrnstein RJ, Vaughan W Jr (1980) Melioration and behavioral allocation. In: Staddon JER (Ed.), Limits to action: The allocation of individual behavior. Academic Press, New York, USA.
2. Heyman GM (2021) How individuals make choices explains addiction’s distinctive, non-eliminable features. Behav Brain Res 397: 112899.
3. Bickel WK, Marsch LA (2000) The tyranny of small decisions: Origins, outcomes and proposed solutions. In: Bickel WK, Vuchinich RE (Eds.), Reframing health behavior change with behavioral economics. Lawrence Erlbaum Associates, Mahwah, NJ, USA.
4. Pierce-Messick ZJ, Acuff SF, Johansen AN, Strickland JC (2026) Brief measurement of non-drug reinforcement: association with alcohol demand and substance use. Addict Res Theory: 1-9.
5. Acuff SF, Oddo LE, Johansen AN, Strickland JC (2024) Contextual and psychosocial factors influencing drug reward in humans: The importance of non-drug reinforcement. Pharmacol Biochem Behav 241: 173802.
6. Bickel WK, Johnson MW, Koffarnus MN, MacKillop J, Murphy JG (2014) The behavioral economics of substance use disorders: Reinforcement pathologies and their repair. Annu Rev Clin Psychol 10: 641–677.
7. Rachlin H (2000) The science of self-control. Harvard University Press, Cambridge, MA, USA.
8. Ruiz SG, Paskewitz S, Baskin-Sommers A (2026) The relationship between regular substance use and cost comparisons in stable and volatile learning contexts. Transl Psychiatry 16: 103.
9. Herrnstein RJ, Loewenstein GF, Prelec D, Vaughan W Jr (1993) Utility maximization and melioration: Internalities in individual choice. J Behav Decis Mak 6(3): 149–185.
10. Acuff SF, MacKillop J, Murphy JG (2023) A contextualized reinforcer pathology approach to addiction. Nat Rev Psychol 2(5): 309–323.
11. Heyman GM (1996) Resolving the contradictions of addiction. Behav Brain Sci 19: 561–574.
12. Herrnstein RJ, Prelec D (1991) Melioration: A theory of distributed choice. J Econ Perspect 5(3): 137–156.
13. Bickel WK, Witkiewitz K, Athamneh LN, Kuhlemeier A (2024) Recovery from alcohol use disorder: Reinforcer pathology theory, measurement, and methods. Alcohol Clin Exp Res 48: 1626–1636.
14. Bickel WK, Athamneh LN, Basso JC, Mellis AM, DeHart WB, et al. (2019) Excessive discounting of delayed reinforcers as a trans-disease process: Update on the state of the science. Curr Opin Psychol 30: 59–64.
15. Weinsztok S, Brassard S, Balodis I, Martin LE, Amlung M (2021) Delay discounting in established and proposed behavioral addictions: A systematic review and meta-analysis. Front Behav Neurosci 15: 786358.
16. Logue AW (1995) Self-control: Waiting until tomorrow for what you want today. Prentice Hall, Englewood Cliffs, NJ, USA.
17. Pick-Alony R, Liberman N, Trope Y (2014) High level of construal and psychological distance reduce melioration. J Behav Decis Mak 27(4): 291–300.
18. Odum AL (2020) Delay discounting of different outcomes: Review and theory. J Exp Anal Behav 113(1): 86–101.
19. Rachlin H (1997) Four teleological theories of addiction. Psychon Bull Rev 4: 462–473.
20. Miller P, Kable JW, Lempert KM (2024) Willingness to wait outperforms delay discounting in predicting drinking severity. J Exp Anal Behav 122(3): 247–258.
21. Smith TR, Panfil K, Kirkpatrick K (2022) Generalizability of time-based interventions: Effects of choice procedure and smaller-sooner delay. Behav Processes 196: 104584.
22. Herrnstein RJ (1997) The matching law: Papers in psychology and economics. Russell Sage Foundation, New York, NY, USA; Harvard University Press, Cambridge, MA, USA.
23. Ainslie G, Haslam N (1992) Self-control. In: Loewenstein G, Elster J (Eds.), Choice over time. Russell Sage Foundation, New York, NY, USA.
24. Bland VJ, Cowie S, Elliffe D, Podlesnik CA (2018) Melioration revisited: A systematic replication of Vaughan (1981). J Exp Anal Behav 109(3): 551–563.
25. Kearns DN, Bird T, Pilz EM, Chavez-Lopez K, Rego F, et al. (2025) Demand for heroin in rats: effects of non-drug alternative substitutes and complements. Neuropsychopharmacology 50(12): 1896–1903.
26. Heyman GM, Dunn B (2002) Decision biases and persistent illicit drug use: An experimental study of distributed choice and addiction. Drug Alcohol Depend 67(2): 193–203.
27. Stillwell DJ, Tunney RJ (2009) Melioration behaviour in the Harvard game is reduced by simplifying decision outcomes. Q J Exp Psychol 62(11): 2252–2261.
28. Open Science Collaboration (2015) Estimating the reproducibility of psychological science. Science 349: aac4716.