Effectiveness of a Serious Game for Teaching and Increasing Awareness about the German Energy Transition

In the current scenario, where sustainable changes are needed to fight climate change and mitigate CO₂ emissions, gamification has been applied in order to achieve attitude change and motivational pull. Despite many serious games that have proven successful, they often lack the structure expected from games intended to change attitudes. The literature shows that a more practical approach is needed, and that it is hard to find scientific support when developing a serious game. In this situation, where more empirical research is needed, we set and tested serious game mechanics and their impact on sustainability outcomes. To evaluate our game’s effects, we used interviews and pre- and post-survey questionnaires to compare attitudes changing regarding energy sustainability. We also discuss the most significant challenges, which can help new authors in their future work. We can conclude that the game performs well, providing a comprehensive view of the sustainable energy transition. After playing the game, the users reported being more aware of their own impact on society and having a better understanding of how climate policies are made in real life. Lastly, we conclude that serious games have the potential for a larger diffusion in society, especially for supplemental teaching on the topic of sustainable energy transition in schools or in training courses of companies.

JEL Classification:A20, Q20, Q22, Q42

Keywords:Serious game; Gamification; Games; Climate change education; Environmental awareness

A reduction in energy sector emissions is crucial to addressing some of our most pressing societal issues, such as greenhouse gas (GHG) emissions that contribute to climate change and the negative impact of air pollution on human health. Since the world increasingly recognizes climate change problems and their negative effects on the world as a real problem and threat, individuals have become more aware of their own possible contributions. As a result, many people have started to change their behavior and adopt solutions that could contribute to mitigating this problem. However, addressing climate concerns requires a combination of different sectors, such as energy efficiency, renewable energy generation, and mobility options, which can be particularly challenging. In addition, given the massive flow of information we are exposed to, it is unclear whether most individuals understand the most efficient measures to successfully cut energy usage or achieve energy-related policy goals [1,2]. Empirically, it is important to raise climate change awareness, since this is an important determinant of public support for climate policies [3]. In addition, the issue of climate science education is not properly taught in public high schools and is frequently disconnected, incoherent, and tagged on to other science topics [4-7].

In a scenario where information might cause citizens to be lost due to over-information, new forms of instruction are emerging for educating users also in the energy field. Over the last years, gamification has been strongly applied for marketing, attitude change, and motivational pull [8].

Teaching sustainability content to the general public by using traditional educational methods, such as lectures, assignments, and projects, can be difficult. One main reason is that the subject matter is complex and vast, and another is that current sustainability education methods do not always engage students [9,10]. As a result, academics have emphasized the importance of future research on interactive ways of communication in order to develop a meaningful knowledge of the climate system [11].

However, serious games also have their limitations, requiring theoretical knowledge of how to explore the game and achieve a better impact among the players. Although many games have proven successful, they lack the structure and mechanisms of what is expected from a truly “serious” game. Besides that, the literature shows that a more practical approach is needed, and it is hard to find scientific evidence from literature for trivial decisions when developing a serious game.

Our literature review also shows that just a few studies meet the requirement of evaluating how serious games actually affect energy efficiency. The majority of the studies, in fact, only tested usability or described the game structure without any evaluation. For a better development of the scientific field, studies on serious games must be methodologically and numerically stronger. The other challenge involves costs, implementation speed, and user preferences for a serious game. Users – mostly young audiences– are accustomed to commercial games, so low-budget games may appear too simple and unappealing.

Considering this framework, this research aims to better understand serious game mechanics and their impact on sustainability outcomes. For this, we used interviews and pre- and post-survey questionnaires to evaluate the game’s effect on users’ attitudes towards energy sustainability. We also intend to extend the serious game design into a reported incipient topic of the literature [12-14]. We do so by empirically discussing and reporting the most significant challenges when building a game and providing helpful tips that can help newcomers to this field of research in their future work.

We review all the steps of building a serious game while building one ourselves. We based our actions on the literature and used that for a comprehensive review of the most important features in gamification, i.e., what works and what the additional guidelines ignored by the literature are. In addition, we show the results of our game regarding the learning process and how effective it was when teaching the sustainable energy transition to young people.

Our research hence contributes to analyzing the potential of games for learning about energy transition and climate change, while improving serious game designing and sharing our experiences.

This paper is organized as follows. First, we discuss the gamification’s design and theoretical base in more detail. Then, in section 3, we describe the game building and data used. Next, section 4 reports and discusses our findings, presenting the results. Finally, section 5 concludes.

Gamification elements and mechanics

While traditional games are defined as “structured play” with rules, goals, and challenges for the sole purpose of entertainment [15], serious games have goals other than enjoyment, such as education and skill gain or behavior modification. In contrast to games, gamification is characterized by its serious purpose [16]. Serious games use a safe setting to experiment with and explore various decisions and behaviors. Despite similarities with simulation-based learning, the factors directed towards player involvement brought about by competitive and entertainmentrelated elements distinguish serious games [17].

Gamification definitions differ and typically focus on either game features and mechanics or the process of gaming and gameful experiences in serious contexts [16]. Deterding et al. [18] define gamification as the “application of game components in nongaming scenarios”.

Reeves and Read [19] identified 10 ingredients of effective games: (1) Self representation with avatars; (2) threedimensional environments; (3) narrative context; (4) feedback and reputations;(5) ranks and levels; (6) marketplaces and economies; (7) competition under rules that are explicit and enforced; (8) teams; (9) parallel communication systems that can be easily configured; and (10) time pressure. Naturally, one can find each of these elements outside of games, and alone none of them is easily identifiable as a game feature only [18]. The division between “game” and “artifact containing game aspects” is sometimes blurred—for instance, is Duolingo a game or a “gamified” app? Furthermore, how game elements are perceived can vary depending on the designer or the user’s role.

Recently, many publications have focused on serious games for a different range of areas [20-24].

The “core” of serious game design is the ideal overlap of educational philosophy, subject matter content, and game design [24]. The author’s design, play, and experience framework (DPE) presents a vocabulary for discussing methods, a technique for analyzing structure, and a process for designing a serious learning game. The framework focuses primarily on serious game design, but it also breaks down the pieces of the game’s design. There are four priority design elements: learning (content and pedagogy), storytelling (character, setting, and narrative), gameplay (mechanics), and user experience (user interface).

Annetta and Bronack [25] suggest a quantitative method for measuring the relevance of tester evaluations in addition to a complex set of theory-based criteria. Prologue, Tutorial/Practice Level, Interactive Feedback, Identity, Immersion, Pleasurable Frustration, Manipulation, Increasing Complexity, Rules, Informed Learning, Pedagogical Effectiveness, Reading Effectiveness, and Communication are the thirteen elements identified by them. In contrast to Winn [24], the authors present an empirical method for evaluating their test results. Sanchez [26] focuses on many other elements: motivation competition, motivation-autonomy, motivation-relatedness, content, freedom, rules and feedback, mistakes, failure and emotional aspects, and game integration.

Besides elements, game mechanics serve as tools to reward players by acting as conditioned reinforcers, encouraging desirable behavior during gameplay [27]. As a personal and emotional connection with this type of behavior is often lacking, rewardbased game mechanics could help “to create external motivation for users to perform a sustainable behavior” [28].

Effects of a gamification application

Gamification, here defined as the use of game design elements in non-game contexts, commonly with the end goal of affecting user behavior [8,18], has been presented as a way to demonstrate knowledge concepts, increase learning, and motivate behavioral change [29]. Besides, simulation-based education can improve competencies such as teamwork, problem-solving, decisionmaking, and critical thinking [30].

A nascent and growing body of literature supports the outcomes of gamification, typically distinguishing between behavioral outcomes, (cognitive) learning outcomes, and either affective outcomes, motivational outcomes, or both [16].

Behavioral changes are required to protect the ecological environment, and thus environmental behaviors should be encouraged. There are various methods for supporting behavioral change, one of which is the use of serious games.

Serious games can be divided into three categories, each with its own set of characteristics based on Pro-Environmental Behaviour (PEB) [31]:
• Environmental education: applied to educating individuals about certain environmental topics, mostly related to climate, and providing knowledge about specific related behaviors.
• Consumption awareness: to raise awareness about energy consumption, both personal and related to a specific environment, such as household or office, also applied using competition.
• Energy efficiency behaviors: to encourage and monitor the execution of selected behaviors that have an effective impact on energy consumption.

For instance, the game Enercities [32], has the challenge of building a sustainable city. Gamification and design elements are used to engage the player while managing a city and its decisions regarding sustainability. The authors evaluated the game via online questionnaires from various European countries. The authors then divided respondents into an experimental and a control group in order to determine whether attitudes toward energy-consuming behaviors had changed after playing the game. The results showed that participants in the experimental group knew more about the consequences of specific behaviors (e.g., turning off the TV and lights) than participants in the control group did.

Chappin et al. [9] used an already well-known board game, Settlers of Catan, to simulate the effects of petroleum extraction, including economic benefits and environmental pollution. As a result, public understanding of sustainability issues, such as externalities, resource dependence, and the tragedy of the commons, was enhanced. The goal was to influence public attitudes and behavior toward sustainable development. The researchers confirmed through qualitative analysis that learners’ attitudes towards sustainability shifted when following the acquisition of sustainability knowledge, which eventually also influenced their sustainability-related behavior.

The Stop Disasters! game [33] uses simulation to teach learners about the potential risks of disasters, such as tsunamis, hurricanes, wildfires, earthquakes, and floods, as well as disaster prevention, monitoring, and mitigation strategies. According to the authors, focusing on wildfire scenarios aided in the development of knowledge about wildfire prevention. Furthermore, the majority of the school students found the game enjoyable.

Another game that describes the building of a community and its energy system is Changing the Game—Neighborhood Edition [34]. The players attempt to accomplish the German government’s energy transition targets, such as reducing CO2 emissions and maximizing energy savings. Various energy technologies and energy-saving techniques that can be integrated are described. The creators concentrated on the neighborhood level in order to create a stronger connection to ordinary life and to develop the game in a specific place. Because the purpose of the paper is to co-design a game, subsequent studies will evaluate the game’s effects on transmitting information about the energy transition.

As a result, there is evidence that gamification tactics have a good impact on audience education and may result in lower energy use and the adoption of sustainable practices. Gamification also improves the user experience and promotes engagement strategy.

The need to inform, educate, and inspire young people about sustainability, energy consumption, and energy conservation, is in line with national and European policies. These policies prescribe a reduction of greenhouse gas emissions, most importantly of CO2, a reduction in energy consumption, stimulation of the development and implementation of renewable energy sources, and a decreased dependency on fossil fuels [35]. We also believe that the serious game described in this paper contributes to the general and specific Sustainable Development Goals (SDGs) adopted by the United Nations General Assembly in 2015 [36]. Furthermore, we believe that our game matches the fourth slot, “Qualitative Education,” and helps to contribute to many other of the 17 main global SDGs.

To stimulate household energy conservation, the most costeffective way of achieving energy savings is through the education of young people [37]. In addition, by increasing awareness of the important role of energy in society and energy consumption at home, new habits will shape more conscious energy-related behavior.

Our game aims to teach players about energy transition technology and energy-saving strategies. Besides that, it should provide information about the sustainable use of resources, promote a sustainable lifestyle, and encourage people to think more about energy transition issues. Therefore, the target group consists of all those aged 16 and upwards who enjoy games and are interested in the fields of energy transition and sustainability, as specifically envisioned by the Fridays for Future protests [38].

The serious game is intended to help the audience to better understand the complexities of the interactions between the climate and energy systems, related policies, and assumptions, and some of the social dynamics of climate and energy decision-making, while at the same time meeting the requirements for policy-supporting games [39]. Players assume the role of a president from a fictitious country and are responsible for making the energy transition and leading their country along a CO₂-neutral path. The game simulation reveals persistent climate change problems, normal sustainability issues, electric grid challenges, budget allocation, and the magnitude and effects of immediate actions to tackle the issues.

More specifically, our game is designed to explore and share knowledge about different topics, such as:
• The accumulation of CO₂ in the atmosphere and the resulting need for early and strong action to reduce emissions.
• The necessary dynamics of energy-related budgets and their limitations on affecting the energy transition.
• The importance of regulating CO₂.
• The repercussions of mitigation approaches (e.g., rebound and/or problem-shifting effects).
• The game also displays social dynamics of climate and energy decision-making, such as the pressure on policymakers and various other stakeholder groups to take action.

Participants in the game always play the role of a president while interacting with other representatives from private industry, government, and non-governmental organizations (NGOs) and focusing on energy, land use, and climate policy. The player in this role is empowered to make decisions while attempting to balance the interests of other groups.

A typical gameplay: Before beginning the game, it is important to offer the player a brief introduction to the game. We started the game by introducing the mechanics that control the whole game with a question offering help. By swiping left on the screen, the player chooses “yes” and receives a confirmation message on top of each card that they asked for help and/or want to know more about the games’ rules. Swiping right means “no,” and the game starts (Figure 1).

The introduction to the game is not vital but provides an interesting input for the player. In the introduction, the player is aware that she/he is playing as the president, and her/his election promise was to lead the country to a carbon-neutral future in the next 30 years.

The other non-player characters (NPCs), such as ministers, industry representatives, NGO leaders, and politicians, are also presented during the introduction while giving tips and explaining how the game works. The game is over when the energy transition bar has reached 100%, meaning that the country has achieved carbon neutrality. At the start of the play, several decision proposals will be presented to the player by many interest groups to achieve carbon neutrality. Each decision proposal will impact the four key performance indicators (KPIs): Budget, Security of Supply, Sustainability, and Public Support. Each KPI ranges from 0 to 100 points. Arriving at 0 with any one of the KPIs ends the game (defeat). However, when any KPI falls below 20, a message of help appears, where we explain on the screen what is happening, and the player has the chance to review and improve the action taken. We also inserted other NPCs that do not have a defined role but bring a different dynamic to the game.

Playing the game: After being introduced to the game and to the characters, the player is free to decide what actions to choose concerning the cards that appear (by a simulation of drawing cards from a deck). Initially, those cards have a random appearance order. However, as the game evolves and the player makes choices, those choices influence which cards will be added to the deck or not. For instance, there are four layers until the full completion of a nuclear power plant. To build it virtually, the player must accept the four cards, completing the whole process. When the players have fully developed a technology (solar, wind, hydro, biomass, nuclear, or gas), they receive a badge of completion and are awarded extra points on each KPI. In this way, we expect the players to follow their progress on the respective bar and to provide feedback regarding the development of each technology. Besides that, the players can track the impact of each decision on the general progress bar (Figure 2).

We also designed the cards in such a way that they provide knowledge to the gamer, while also having a fun aspect. The main message that appears usually provides some context and makes a suggestion involving an action to be taken. When clicked on in the top right corner, an icon suggesting ideas/studying provides additional information (Figure 3). It is worth mentioning that we do not reveal to the player the back end of their actions. Therefore, the consequence of activities is not completely known. One way for the player to minimize uncertainty due to these unpredictable consequences is to click on the icon and read the content.

The decision to include the optional ideas/studying icon was based on many aspects. First, providing the additional information as optional arouses a player’s interest in performing with a better idea of the consequences. It also keeps the layout of the game “clean” by avoiding too much textual content that could be annoying for the player. Second, by not revealing how the points or the game’s progress work, we keep the players engaged while avoiding “cheaters,” i.e. players who would avoid the learning content and perform according to the rules.

The game can also present “reaction cards” regardless of a player’s performance. We used such strong game elements to provide good feedback and content simultaneously. Initially, none of these cards were present in the deck. However, following a player’s decision to build a wind power plant, regular NPCs representing citizens would appear and comment on the decision. The text on the cards describes what an average politician would face but also provides different public perceptions of the mentioned technology to the player.

Workshops and playability tests

After the first developments and when we had a viable version of the game, we ran a series of workshops and tests within our university and in a company environment. The goal was to share the developments with other persons of interest, receive suggestions, collect user data, and make further improvements. Before playing, we explained the game’s main ideas and gave the context concerning the roles in the final objective.

Choosing a card by swiping right or left was widely accepted, since we took it from already established apps, such as Tinder. However, the participants suggested that we could explore other options, such as “yes” or “no” buttons on the sides of the cards. We implemented this suggestion and moved the buttons to the inner parts of the card.

The introduction part also received many suggestions. The user recommended elaborating on the introduction cards and explaining some features still unclear to the players, such as what the KPIs are and what they represent. We improved the badge icons on the toolbar and better explained the abbreviations of the game, such as NGO leader and KPI. Because the introduction might become longer and affect the players’ engagement, we added a skip option so that players could save time in a second turn. We also started highlighting abbreviations and other terms that were not easily understandable by a broad audience.

Regarding the graphic design and what should be visually presented, the users reported that it would be better to show the general score of the game on the main screen instead of making the player scroll down to see the general progress. We also discussed the percentage of each KPI and their designations on the main screen. For each decision proposal, the test users also mentioned that the fonts could be highlighted or made even bigger in order to make it easier for the players to read them.

Concerning the game dynamics, we also received several suggestions. The area where we received the most requests was the creation of a mechanism to enhance competition among players, such as a score based on the final year or score and KPI weights. In addition, a message at the end of the game could display the score and invite the player to play another round, facilitating re-engaging and retaining the player longer. The workshop participants also suggested gamification features to improve the player’s experience.

Among them was a “joker” card that could give the player some activism in the game, since the current situation still makes it rather passive, with the player’s role being to choose alternatives between the given cards. In this matter, one could develop a card to boost one of the KPIs at the time selected by the player or create a game mechanism to have the same results, such as tax increases or government propaganda.

The remaining suggestions were: to show the player the remaining time, to warn them about a possible shortage of cards, and to change the reaction cards to news or an advisor, since this design and dialogue are quite general. Making the game with different scenarios where you could play using other countries with additional natural resources was also one of the ideas brought up.

Evaluating the game simulation

To better evaluate the game, we conducted a focus group [40] with ten trainees who had participated in the match as volunteers in the winter of 2021. The focus group was held days after the game simulation. For about an hour, the players met with the project evaluator and discussed key aspects of the game. In addition, we recorded and made written notes that summarized the players’ discussions and comments. Although these participants were interns from an energy company, participation was voluntary and had no bearing on professional performance. Among the participants, 7 were female and 3 male. They had previous knowledge in the energy sector due to the company’s activities but reported a growing interest in recent protests for better energy policies and reported to be socially involved in some sort of groups outside of school and work that also gathered to discuss sustainable ways of living.

Besides the focus group, a pre- and a post-simulation survey were used to evaluate learning outcomes from the game. We conducted a questionnaire to assess the game’s impact on influencing sustainable behaviors. The Environmental Identity Scale (EIS), a 12-item scale adapted from Clayton and Opotow [41], was used to better understand whether a game focusing on sustainable measures can change or develop a specific human behavior in terms of environmental awareness. Understanding a participant’s interpretation of sustainable transition is important because it demonstrates that sustainable identity is linked to supporting values and environmental behaviors.

A questionnaire was developed to assess the participants’ knowledge and beliefs about the reality and causes of climate change [42]. Participants’ effective responses to climate change were elicited using semantic differential scales, which asked them for their impressions about climate change on a scale from a “big problem” to a “hoax”. We also created additional items to test learning outcomes specific to the game simulation. These questions addressed many diverse topics, such as the accumulation of CO2 in the atmosphere, alternative and renewable energy sources and their consequences, sustainable policies, and general energy economics, as well as the potential impact of policies and actions on climate change mitigation. Building a questionnaire to evaluate the knowledge from the game was not easy, given that the literature deeply and controversially discusses the polarizing impact of science literacy [43,44]

In this section, we intend to divide our results into two parts. Section one focuses on the results of the tests and the information collected from the workshops. The second part focuses on learning from our experience while developing the game. We include this as one of our major findings since the serious game state-of-the-art is often lacking.

Survey and focus group results Focus group responses

The literature indicates that the serious game is an effective tool for engaging participants in the climate-energy challenge and improving the understanding of the scale, urgency, policies, and actions that have the potential to meet international climate policy goals. We linked our plans concerning the game’s effect on the players to (i) increased knowledge of climate change causes, dynamics, and impacts; (ii) climate engagement, including increased feelings of urgency and sympathy towards climate change; and (iii) the creation of an immersive, social learning experience.

Participants’ open-ended responses show the game’s potential for learning and affective engagement with climate change and energy transition. In addition, participants especially directed their comments to yet unknown effects and consequences of renewable energy and to how different energy sources, apart from renewables, can also help the energy transition. Among the comments made by participants that reflect their learning are:
• “Playing this game brought me into real-life situations that I did not pay attention to before. It made me more aware of all the factors contributing to climate change and the actions I could take to contribute positively.”
• “I already knew that CO2 emissions contribute to climate change and that the situation required urgent actions, but I didn’t realize that new technologies, like Carbon Capture Storage (CCS), were so important for reaching this goal.”
• “I was surprised to learn more about technologies. For instance, I didn’t know that solar panels or wind farms had an environmental impact. I also learned that biogas or natural gas is a good choice for keeping the security of supply and replacing coal and oil.”
• “I felt that you have to “pick your poison” both in the game and in real life. I faced many situations where I wanted to invest but didn’t have the money. So, I had to find sources of money that I disliked, like exploring oil and coal reserves.”
Other comments indicated that players felt an increased sense of urgency about climate change, and frequently a desire to act immediately. As an example:
• “This activity helped to show me on my screen how urgent the need for action is. I felt worried, knowing I was coming towards the end of the game and that my policies weren’t working. When we hear “25 years in the future”, it might sound like a lot, but I realized when I was playing what a short time that really is.”
• “I enjoy imagining whether it would be possible to make all the changes we’ve been proposing in the real world.”
• “I felt empowered as president. Of course, making decisions and having to listen to your citizens’ opinions is difficult.”

The game received a considerably positive response from the focus group. Although many concepts had to be learned during the game, and some information was new for the participants, overall, the players acted pragmatically and faced the problem directly.

The energy transition was a big challenge that was possible to solve. By the end, all the players had had at least one successful round in which they had won the game. However, despite already being trainees at an energy company, many participants indicated that they have gained additional knowledge, such as about the energy system, climate problems, and current policies. In addition, the players identified many new competencies:
• A better understanding of the energy sources available to a country, especially how renewable energy sources affect the security of supply.
• The current political situation demands action, but it is also not easy to make choices when you are in a leading political role.
• A sense of how each policy decision is linked to a reaction and that even established technologies can trigger negative reactions from certain groups of interest.
• That there is no single policy or a single correct way to tackle climate change; all the possibilities are on the table, and each country can use its specific strengths.

Other important results to discuss come from the questionnaire that the participants filled out after the second survey. They shed light on one of our goals of the game: to enable players to get a feeling for the complex scenarios that policymakers find themselves in. For example, players reported a changed perception regarding how energy policy is conducted and that assuming the role of a leader is harder than they initially thought. These results support the workshop results and reinforce the ability of the game to simulate the reality of climate policies.

Survey results

The data collection process began during the COVID-19 period, in the winter of 2021/22. The initial idea was to introduce the game in high schools in Germany. However, we used a different public (but still inside the game’s scope) and online channel without classes and any potential contact with education personnel (i.e., any type of staff dealing with training, teaching or other educational activities). Participants were invited via an e-mail group and introduced to the idea of the game. We then waited for the responses of those interested in participating. We sent a link containing a pre-test questionnaire and the game link to the interested respondents; a post-test evaluation questionnaire was sent later on. These two surveys were adapted from previous studies and included items to measure environmental beliefs and attitudes and proenvironmental behavior [45-47].

Participants played the game after completing the pretest evaluation survey, and when they had finished, they took a screenshot of the game and reported the results so that we could keep track of their progress and check whether the players had completed the game. In the end, the participants completed the post-test evaluation survey after four weeks and indicated their interest in participating in the focus group discussion.

As a result, when assessing our sample of 21 players, the statistical t-test and the p-value revealed that our serious game did indeed elicit a statistically significant shift in climate behaviors and literacy at the 5% significance level (Table 1).

Table 1:Comparison of pre-and post-serious game survey responses and t-test results (SD = stand. dev.).

The serious game survey shows evidence of participants learning analytical skills more effectively. However, we can also infer that participants did not completely understand some survey questions. Furthermore, for some questions, especially when approaching the topic of climate change, ceiling effects—where the majority of values come from the upper limit of the scale— might have made it difficult in certain instances for us to identify significant changes pre- and post-game. For example, before playing the game, most participants already thought climate change was a real and serious threat (Question 2). Besides that, the game showed no change in recycling habits, mostly because waste separation is already mandatory in Germany (Question 30).

We can also see a pattern in the series of questions that tried to evaluate how the participants perceived the government’s actions. Even after playing the game and experiencing the challenges that politicians face in real life, the results show no significance in these questions (Questions 5, 16, 18, 20, and 23). Despite the game not affecting government evaluation, we can conjecture that this reflects how society generally evaluates the government and, more specifically, the perception that public spending on the energy transition is poorly managed and that actions undertaken so far are still distant from those necessary to stop climate change.

On the other side, participants demonstrated a better awareness of the different clean technologies and practices most likely to result in successful climate change mitigation. In addition, they succeeded after playing the game in identifying other problems when exploring coal, oil, water, and land use. They also improved their knowledge of energy terminologies, such as “security of supply” and “energy trilemma”. In the end, the results also showed learning on the impacts of carbon capture and storage (CCS) and clean technologies adoption (e.g., solar, wind, and CCS).

Additionally, the survey indicates the participants’ attitude change towards engaging in behavior that might persuade others— such as employing time to discuss climate change. They also showed changes in their habits concerning energy consumption patterns and sustainability. They generally showed a better disposition to deploy time and money for a more sustainable future. This disposition translates into positive results indicating a change in attitude, for instance, regarding using more public transportation, looking for better contracts from energy providers, and eating less meat (Questions 24-29, 31, 32).

As a result, for this group, we can conclude that participation in the serious game experience had a twofold effect. First, it helped them shape a better and more realistic idea of effective energy and climate change problems and solutions. Second, it indicates that the serious game experience made the players more aware of their role in the real world and how their behavioral change can impact climate change, i.e. the behavioral change of looking for better solutions (regarding the sustainable energy transition in Germany).

One important point to discuss is how a player perceives the current climate change policies. Although the survey indicates no significant changes, many participants stated that the game was important for giving them an idea of how politics work and the many variables that are involved in a decision.

It is also important to discuss that the players in the end answered a small set of questions concerning the evaluation of the game. These questions, only present in the post-game survey, show that for most of the players (around 80%) the game was “easy” or “neither difficult nor easy”. Furthermore, the players also positively evaluated game elements, such as helping buttons, scores, and badges (90% found them useful or extremely useful for achieving the game’s goal). Besides that, they also reported the game’s content as useful or extremely useful. However, the reaction cards, where NPC would assess their actions so far, were said to be useful or extremely useful for just 50% of the participants. Finally, we can infer that the game was successful because all the participants affirmed, that they had fun (100%) and learned something they did not know before (76%).

Learning and recommendations

In this subsection, we share our main barriers and challenges and how we solved them, and suggest what future research can do in the future to improve the designs and the applications of serious games.

-Define your public policy target as much as possible. Defining your public policy target is the first step towards a good serious game. Understanding who your main players are and what motivates them is a good basis to support further development. However, when we approached the topic of “increasing awareness,” we had the misperception that sustainability and renewable energy are important topics for everybody, making the mistake of addressing an audience as broad as possible.

In our case, we defined the target audience as teenagers and young adults at high school or university. For as much as targeted, this cut was optimal. First, because these groups generally do not socialize with each other, i.e. a 15-year-old adolescent understands the game topics differently than a 23-year-old college student. Second, despite having a 3-6 years difference, the intellectual level is quite diverse, and older individuals typically already have a deeper understanding of climate change problems. As a result, they can understand the complexities much better, demanding a game with this complexity and explanations about the decision’s outcome.

-Engage with the technical team from the beginning. Our experience demonstrates that not including the technical teams right from the beginning can cause the final project to be delayed or to undergo significant modifications. When initially dividing the project into smaller work packages, we first looked at the literature and analyzed other games. After that, in the second working package, we used the findings from previous research and compared them to build the game and think about its structure, logic, and interface design. Later on, in the third working package, we met with software developers and technical team members to communicate our ideas and discuss how these could best be implemented.

In reality, this approach proved to be inefficient. While doing the initial research on the first two working packages, we naturally considered the possible financial and technical barriers to the game, thinking about small steps and simple ideas. However, when the third working package began, and the technical discussions appeared, the game had to be completely modified. First, there was no sufficient budget to build a visualization tool in the way that we needed and, more specifically, a map that displayed the country’s progress and the landscape’s modifications. Second, it turned out to be impossible to develop a multi-player platform where players could compete with each other or visualize their peers’ performance. These two restrictions deeply changed the initial idea of the game. Instead of a map that the player would use to make the decisions based on a city visualization, we brought in dialogue cards. This decision made the player more passive, but the core idea—making the player make decisions—did not change. Second, during the test phase, we encouraged competition by incentivizing the players to share their results, creating a healthy environment where competition was helping the game spread.

To conclude, if we had first involved technical parts in the game building earlier, we would have had time to discuss new approaches and would not have had to adapt the game quickly so as not to delay it. Furthermore, although the developers had no experience with serious games, they provided many insights based on previous experiences with alternative software development projects.

-Loose, tight approach, discuss the ideas openly. Another aspect we should mention to improve the game’s development is having an open system until the game is finished. Once we established the structure of the dialogue cards and set up the dialogues, mechanics, and how points and KPIs would work, no further changes could be made.

During the pre-test and post-test, we wanted to use the feedback and to change minor things in the game. However, since the structure and how we had already connected the game platform, even a small change would interfere or not be feasible at all. Therefore, our insights show that it is much better to keep the structure loose and to finish its implementation only when feedback is positive, and no further change is needed.

-Think about the future of the game. One of the big barriers that we found when looking for serious games was that even though several papers had published their results, the game was not available online to play, or the server had been discontinued. This problem leads to important learning when designing games: what will happen when the project is over?

From our experience, most serious games are funded for a short and specific time, and the results are in danger of being lost after this period. This problem may happen mainly for two reasons: The first one is that the future migration of the game is not possible because it is complex to move it between two different platforms, especially if the earlier version had budget restrictions. This beta version (an earlier version mainly built for testing) is too simple and cannot be used in a more complex language.

For instance, BBC [48] was a game where the player takes the role of president of the European Nations and must tackle climate change while staying popular enough with the voters to remain in office. It has a similar storyline to the game that we present here, but the website is no longer available for playing. The main reason is that the application required a Flash player plug-in, which became unavailable at the end of 2020.

The second reason is a lack of planning after the project is finished. In this case, all stakeholders generally do not consider the costs and how responsibilities will be shared after the game is developed and the project ends. This barrier includes the costs of maintaining the game online, updating it, and offering support for suggestions and improvements.

This stage, often neglected, is crucial to help serious games grow. The currently available examples show that new developers will base themselves on previous works, companies will see what has been done in the past, and players will get engaged with a recent sort of game.

-Solve any bugs before testing. Before the official testing phase, we had internal workshops with academia and private sector colleagues. Our goal was to test the game for and with a narrow target audience, where we could hear opinions about their feelings about playing the game and other general suggestions that might indicate whether the game was on the right track.

However, since the game was not completely functional, it was normal to have some bugs. When working with software development methods, it is clear to the stakeholders that the developments will occur in incremental steps. Nevertheless, we perceived that neither the academic nor the industry team for the scenario and the first tests were unsuccessful. The requested feedback from our peers was often not directed to the important aspects of the game, such as mechanics, designs, fun, or general elements. Instead, the workshop participants mainly provided feedback about the features that were not working properly and that were the most problematic. Therefore, during the workshop, we spent most of the time talking about the already known bugs.

The lessons learned show two possible approaches to overcoming this challenge between the users and the bugs on the test phase. The first one is to get rid of all the bugs and to test the game later. Although ideal, this solution might not be optimal since until the full completion of the game, some bugs will appear, and bugs will generally be in the later phases after the game is completely defined and ready for the test phase to play with. The second solution is to explain the current work state and to familiarize your testers with the recent work and the fact that bugs exist. However, once bugs do not interfere any more with the playing of the game, we should disregard them, and the discussion should focus on the game itself.

-For small games, think small. Before developing the game, we did an extensive literature review of serious games. We found that many authors would indicate several gamifications features to be used in a game. However, unlike what the literature suggests, our experience shows that at least for small games, incorporating all gamification elements does not necessarily help fostering serious competitive play.

This problem happens because when designing smaller games, which occur most of the time, the availability of resources is not abundant, and the game will have a limited scope. When working with a small game, adding many features might result in confusion and a lack of focus.

We suggest keeping the game as close as possible to a very well-defined target. For example, both literature and other market games may indicate that a good game is a game with many game features (action, construction, and/or stories) and uses many features (leader board, badges, and/or points). We find that short stories in serious games do not support many such interactions.

-The bar is high. As mentioned before, the availability of serious online games to play and use as a benchmark is limited. As a result, most results are not properly maintained after some time and then disappear. In addition, serious games are not particularly popular among console players. The public has already adapted to more traditional games. In this sense, when engaging with any audience with experience in playing previous games, the requirements from the players show that they have very high expectations.

Managing players’ expectations coming from commercially developed games and still having a degree of excitement to engage this same player is a challenge. In our experience, we faced criticism and comparisons between our serious games and games costing millions of dollars. To overcome this barrier and better establish a gamer’s expectation, it is vital to explain the role of serious games in society, the purpose of these games, and how they should not and why they cannot be compared to commercial games. It is a similar approach to the Indie games that players know, which focus on innovation and digital distribution, and which are generally developed by individuals or smaller companies without significant financial support.

With this paper, we provide a better understanding of serious game mechanics and their impact on sustainability outcomes. To achieve this, we used interviews and pre- and post-survey questionnaires to evaluate the game’s effect on users’ attitudes and knowledge regarding energy sustainability. We also contribute to the literature concerning serious game design by empirically discussing and reporting the biggest challenges when building a game. Further, we share helpful guidelines for authors of future serious games, since there is not much-written work on this incipient topic.

We can conclude that our serious game presents a comprehensive view of the sustainable energy transition. Besides that, the game has the potential for a larger dissemination, being used for supplemental teaching of the topic in schools or training courses offered by companies. In a scenario where people seek more information about how to contribute to mitigating climate change, the game proved to be a useful source of fun and knowledge sharing.

The results from our workshop show that the serious game reached its goal by presenting important knowledge in a fun and dynamic environment. The players reported how important it was to learn about a diverse number of topics related to energy transitions and how curious it was to “choose your poison,” but that having to choose between alternatives that were not the best would still be helpful to achieving the final goal. The players also gave a positive feedback concerning the game design, and they approved the game’s structure, the explanation of new concepts, and the helpful cards.

In addition to the workshop, we applied a survey to compare the game’s effectiveness regarding changes in knowledge, attitudes, and behavior. The results reveal that the game was effective in changes of command, behavior, and partly of attitudes. More specifically, the players learned about renewable energy and its impact. They were also aware of the different effects of other energy sources, such as oil and coal. The other positive results are related to new attitudes and behaviors. First, they mentioned their will to discuss climate change with their peers and expend time and money on new alternatives, such as green energy providers, changing to different lighting, eating less meat, and using public transport. However, despite the players citing how “unusual” it was to be the president and take the decisions, we noticed no change in this area. Players generally did not change their beliefs and evaluated their government as inefficient, slow, and not investing enough to prevent climate change.

Developing a serious game is not a simple project. Therefore, we are sharing our experiences with the many obstacles arising and are providing useful guidance for others. For example, we state that a loose approach is better initially and that defining a small target while discussing it with developers from the beginning onwards is extremely important. Other valuable actions are to think about possible future migration of the game and how to scale the game for commercial projects, or even keep it available for other researchers to use as a basis.

Our study has its limitations. First, the game was designed and tested among young people. In the future, a game with more “adult” content can be designed and tested to check how the knowledge and behavior change with a more mature audience. This more mature audience would bring two additional challenges. First, it demands a different game, with easier mechanics for a public not so adapted to serious games; second, we should adjust the game content for this new audience due to the maturity of knowledge and possibly more years of formal education. Another improvement point is the number of participants. Due to the COVID-19 pandemic, we did not have responses from schools around our region for implementing testing of the game, so we used trainees from an energy company. In the future, involving more users with different backgrounds would enable testing the game even better.

In conclusion, we find evidence that a new serious game where players act as president and try to lead the energy transition in their country made players more environmentally conscious and aware of the climate problems that the world faces today. The findings indicate that serious games positively impact players’ environmental knowledge and instigate them to discuss ideas and change habits towards ecological needs. We contribute to the serious game literature by providing another example of a successful case while sharing valuable insights for developing a serious game. We expect this will help serious game implementation to gain more traction among society and to expand and embrace different audiences and new subjects used in companies and schools in order to better understand what is needed to achieve a sustainable future.

Funding received from the E.ON - E.ON ERC industrial research cooperation (project D20) is gratefully acknowledged.

No conflict of interest.

1. Dean AJ, Gulliver RE, Wilson KA (2021) “Taking action for the Reef?"-Australians do not connect Reef conservation with individual climate-related actions. Conservation Letters 14(2): e12765.
2. Gardner GT, Stern PC (2008) The short list: The most effective actions US households can take to curb climate change. Environment: Science and Policy for Sustainable Development 50(5): 12-25.
3. Rhodes E, Axsen J, Jaccard M (2017) Exploring citizen support for different types of climate policy. Ecological Economics 137: 56-69.
4. Church W, Skelton L (2010) Sustainability education in k-12 classrooms. Journal of Sustainability Education 1: 1-13.
5. Corner A, Roberts O, Chiari S, Völler S, Mayrhuber ES, et al. (2015) How do young people engage with climate change? The role of knowledge, values, message framing, and trusted communicators. Wiley Interdisciplinary Reviews: Climate Change 6(5): 523-534.
6. Harker Schuch I, Mills FP, Lade SJ, Colvin RM (2020) CO2peration - structuring a 3D interactive digital game to improve climate literacy in the 12-13-year-old age group. Computers & Education 144(2): 103705.
7. Harker Schuch I, Watson M (2019) Developing a climate literacy framework for upper secondary students. In W Leal Filho, SL Hemstock (Eds.), Climate change and the role of education. Springer International Publishing pp.291-318.
8. Hamari J, Koivisto J (2013) Social motivations to use gamification: An empirical study of gamifying exercise. Proceedings of the 21st European Conference on Information Systems, June 6-8, 2013, Utrecht, The Netherlands.
9. Chappin EJ, Bijvoet X, Oei A (2017) Teaching sustainability to a broad audience through an entertainment game - the effect of Catan: Oil springs. Journal of Cleaner Production 156: 556-568.
10. Prensky M (2006) Don’t bother me, mom, I’m learning! how computer and video games are preparing your kids for 21st century success and how. Paragon House Publishers.
11. Ballantyne AG, Wibeck V, Neset TS (2016) Images of climate change-a pilot study of young people’s perceptions of ICT - based climate visualization. Climatic Change 134(1-2): 73-85.
12. Ahmad M, Rahim LA, Arshad NI (2015) An analysis of educational games design frameworks from a software engineering perspective. Journal of Information and Communication Technology 14: 123-151.
13. Morganti L, Pallavicini F, Cadel E, Candelieri A, Archetti F, et al. (2017) Gaming for earth: Serious games and gamification to engage consumers in pro-environmental behaviours for energy efficiency. Energy Research & Social Science 29: 95-102.
14. Rapp A, Hopfgartner F, Hamari J, Linehan C, Cena F (2019) Strengthening gamification studies: Current trends and future opportunities of gamification research. International Journal of Human-Computer Studies 127: 1-6.
15. Cheng MT, Chen JH, Chu SJ, Chen SY (2015) The use of serious games in science education: A review of selected empirical research from 2002 to 2013. Journal of Computers in Education 2(3): 353-375.
16. Krath J, Schürmann L, Von Korflesch HF (2021) Revealing the theoretical basis of gamification: A systematic review and analysis of theory in research on gamification, serious games, and game-based learning. Computers in Human Behavior 125: 106963.
17. De la Torre R, Onggo BS, Corlu CG, Nogal M, Juan AA (2021) The role of simulation and serious games in teaching concepts on circular economy and sustainable energy. Energies 14(4): 1138.
18. Deterding S, Dixon D, Khaled R, Nacke L (2011) From game design elements to gamefulness: Defining "gamification". Proceedings of the 15th international academic MindTrek conference: Envisioning future media environments pp.9-15.
19. Reeves B, Read L (2009) Total engagement: Using games and virtual worlds to change the way people work and businesses compete. Harvard Business Review Press.
20. AlSkaif T, Lampropoulos I, Van Den Broek M, Van Sark W (2018) Gamification based framework for engagement of residential customers in energy applications. Energy Research & Social Science 44: 187-195.
21. Brom C, Stárková T, Bromová E, Dechterenko F (2019) Gamifying a simulation: Do a game goal, choice, points, and praise enhance learning? Journal of Educational Computing Research 57(6): 1575-1613.
22. Charrouf Y, Taha Janan M (2019) The use of a serious game in entrepreneurship teaching. Education and Information Technologies 24(6): 3841-3854.
23. Kankaanranta MH, Neittaanmäki P (2008) Design and use of serious games. Springer Dordrecht 37.
24. Winn BM (2009) The design, play, and experience framework. Handbook of research on effective electronic gaming in education. IGI Global pp.1010-1024.
25. Annetta LA, Bronack S (2011) Serious educational game assessment: Practical methods and models for educational games, simulations, and virtual worlds. SensePublishers Rotterdam.
26. Sanchez E (2011) Key criteria for game design. A framework. The Business Game Ltd Hampshire, UK.
27. Landers RN (2014) Developing a theory of gamified learning: Linking serious games and gamification of learning. Simulation & Gaming 45(6): 752-768.
28. Wemyss D, Cellina F, Lobsiger Kägi E, De Luca V, Castri R (2019) Does it last? long-term impacts of an app-based behavior change intervention on household electricity savings in Switzerland. Energy Research & Social Science 47: 16-27.
29. Beck AL, Chitalia S, Rai V (2019) Not so gameful: A critical review of gamification in mobile energy applications. Energy Research & Social Science 51: 32-39.
30. Campos N, Nogal M, Caliz C, Juan AA (2020) Simulation-based education involving online and on-campus models in different European universities. International Journal of Educational Technology in Higher Education 17(1): 1-15.
31. Steg L, Vlek C (2009) Encouraging pro-environmental behavior: An integrative review and research agenda. Journal of Environmental Psychology 29(3): 309-317.
32. Knol E, De Vries PW (2011) Enercities - a serious game to stimulate sustainability and energy conservation: Preliminary results. eLearning p.25.
33. Pereira G, Prada R, Paiva A (2014) Disaster prevention social awareness: The stop disasters! case study. 6th International Conference on Games and Virtual Worlds for Serious Applications (VS-GAMES 2014) pp.1-8.
34. Lanezki M, Siemer C, Wehkamp S (2020) Changing the game-neighborhood: An energy transition board game, developed in a co-design process: A case study. Sustainability 12(24): 10509.
35. European Commission (2015) Directive 2010/31/ue, Energy Performance of Building Directive Recast (EPBD Recast). Brussels.
36. United Nations (2015) United Nations general assembly. Sustainable Development Goals. URL: https://sdgs.un.org/goals
37. Bertoldi P, Economidou M, Palermo V, Boza-Kiss B, Todeschi V (2021) How to finance energy renovation of residential buildings: Review of current and emerging financing instruments in the EU. WIREs Energy and Environment 10(1): e384.
38. Fridays for Future (2022) Fridays for future. Our demands. URL: https://fridaysforfuture.org/what-we-do/our-demands/
39. Mayer I, Bekebrede G, Harteveld C, Warmelink H, Zhou Q, et al. (2014) The research and evaluation of serious games: Toward a comprehensive methodology. British Journal of Educational Technology 45(3): 502-527.
40. Krueger RA (2014) Focus groups: A practical guide for applied research. Sage Publications.
41. Clayton S, Opotow S (2003) Identity and the natural environment: The psychological significance of nature. MIT Press.
42. Leiserowitz A (2006) Climate change risk perception and policy preferences: The role of affect, imagery, and values. Climatic Change 77(1): 45-72.
43. Kahan DM, Peters E, Wittlin M, Slovic P, Ouellette LL, et al. (2012) The polarizing impact of science literacy and numeracy on perceived climate change risks. Nature Climate Change 2(10): 732-735.
44. Rooney Varga JN, Sterman JD, Fracassi E, Franck T, Kapmeier F, et al. (2018) Combining role-play with interactive simulation to motivate informed climate action: Evidence from the world climate simulation. PloS one 13(8): e0202877.
45. Gadenne D, Sharma B, Kerr D, Smith T (2011) The influence of consumers’ environmental beliefs and attitudes on energy saving behaviors. Energy Policy 39(12): 7684-7694.
46. Gatersleben B, Steg L, Vlek C (2002) Measurement and determinants of environmentally significant consumer behavior. Environment and Behavior 34(3): 335-362.
47. Karp DG (1996) Values and their effect on pro-environmental behavior. Environment and Behavior 28(1): 111-133.
48. BBC (2014) Serious games. URL: http://games4sustainability.org/gamepedia/bbc-climate-challenge