In previous work, Gamrat, Zimmerman, Dudek, and Peck defined badges as “online representations of learning experiences and activities that tell a story about the learner’s education and skills.”1 While at first glance this definition seems straightforward and badges appear easy to adopt, we offer that such may not be the case. In this chapter, we identify three internal challenges faced by badge creators and issuers and three external challenges faced by the larger education community. We argue that these six challenges present some of the largest barriers to the adoption of digital badges in education.
Digital badges require a significant degree of internal reflection about what they represent for an organization. Digital badges offer additional value and complexity because of the associated metadata and the transparent representation of the skills, abilities, and experiences valued by the badge issuer. For these reasons, the badge creators and issuers may invest significant time before they can reach a consensus on the badges that positively represent them and their institutions. Variations in badge design, assessment practices, and attention to the many details surrounding badge creation all contribute to this time factor.
Badges can range in size and scope. Badge design affords a great deal of flexibility regarding the scope of the task required to earn the badge. An organization could create badges that are awarded for attending a one-hour workshop or that completely align to an undergraduate or graduate degree. Digital badges’ advantage is the capture of detailed information explaining the learning experience, what is required of the learner, and documented evidence of the completion of these requirements. Conventional wisdom might suggest that mapping a badge or suite of badges to something as large as a degree might be too complex, and also unnecessary because a series of data explaining the student’s learning experience already exists—otherwise known as a transcript.
Since badges have value in different forms and represent different types of experiences, badge issuers may want to consider why they are issuing the badge, and badge earners may want to consider the number and types of badges they might share with a potential employer. Badge issuers should evaluate their goals for issuing badges, as this will help to determine the size and scope of the badges. For example, the badges may act as an additional layer of information that might supplement recognition that is already awarded, such as those associated with the completion of a class, degree, or certification program. Other badges could document skills and accomplishments beyond what an organization traditionally recognizes—for example, soft skills such as teamwork or problem solving.
Badges offer richer detail about achievements and experiences, which makes a badge a potentially valuable supplement to a résumé. However, in research conducted by Raish and Rimland, employers reported interest in reviewing badges within a limited scope, suggesting that the rules of résumés (brevity and conciseness) still apply.2 Badge scope might require the consideration for how many criterion points to include in a badge—that is, how many steps or submitted artifacts are required to complete the badge. Digital badges may help learners to better elaborate on their learning experiences, but a badge author may want to consider this function of a badge and how it can help to translate the badge beyond its original context.
As with any credential, various forms of badge assessments exist. Quizzes and written papers are often used for competency-based badges. Project-based badges may require more complex evidence of success, including portfolios or other tangible artifacts. Participation-based badges usually require the completion of a workshop, project, or course, where the assessment is not necessarily about the quality of a produced artifact but rather on social interaction and completion.3 When used in concert, these assessment techniques may provide a rich perspective on what students know and can do.
Educators, administrators, and employers all want to ensure the claims students make about their earned credentials, certificates, or degrees are authentic. Rigorous assessments help to validate the claims about student achievement made by certificates and degrees in most educational environments, but often the assessment is hidden from the public. If badges are to be generally accepted as an alternative credential, then the assessment provided by the badge issuer must be visible to all and valid. As a badge ideally makes public the criteria and assessments for earning the badge and thus can expose weak assessment practices, badge creators and organizations are opened up to scrutiny and criticism.
The badge issuer ideally uses reliable and valid assessment instruments built into the badge criteria. Several statistical methods exist to ascertain the reliability and validity of multiple-choice quizzes. Badges that require portfolios or written materials may rely on a grading rubric that should be examined for intra-rater and inter-rater reliability.4 If possible, badge issuers should include evidence of the reliability and validity of assessment instruments used in the badge to increase the badge’s veracity. The quality of the assessment instruments used in badges is critical.
The authors recently participated in an advisory committee helping interested faculty and staff through think-aloud exercises focused on the creation and implementation of digital badges. From this advisory role and our own experiences authoring badges, we can confidently say that the details of badge design can quickly become complicated. The concept of badging is easy to grasp. The processes needed to implement a badging system are complex. For example, just the instructional design considerations for badges are multifaceted, including aspects of content, assessment, and program scalability.5 At first, implementing digital badges can seem easy, but addressing these complexities can be overwhelming. This may be especially true when the badge issuer is an organization with multiple stakeholders, inputs, and concerns.
As described above, the scope of a badge can be difficult to determine, but goal articulation will help to narrow what the badge means and what experience it is intended to recognize. To avoid becoming overwhelmed by the detailed questions presented by Gamrat, Bixler, and Raish, we recommend badge issuers start by considering primary reasoning for using digital badges.6 Ahn, Pellicone, and Butler found that most researchers have approached digital badges with the goal of exploring student motivation, pedagogical considerations, or impacts to credentialing.7 After determining the primary goal for creating and issuing badges, badge creators may be better able to consider content, assessment, relationships between digital badges, and program scalability.
Probably the most challenging detail for implementing a digital badging system is to determine how to scale the initiative. Badge ecosystems—the mechanisms needed to create, store, and issue badges—require a great deal of thought to conceptualize and significant resources (personnel and computer systems) to implement. Among the many questions to address are, Who handles the computing infrastructure? Who handles the uploading and storing of digital badges? Who updates the information?8 Infrastructure questions add to the details of pedagogical and administrative questions, resulting in a balance among the three. Implementation and pedagogical strategies that work at the scale of a few dozen learners might be impossible with hundreds or thousands. Issuing digital badges in large numbers requires careful reflection, balancing high-quality assessment and grading timeliness. Decisions regarding implementing badges in large numbers likely cannot be made unilaterally, and achieving consensus can be a time-intensive process.
Badges present a value proposition at different levels of education: elementary and secondary education, higher education, and continuing education or professional development. Within informal and formal education, badges might be helpful in providing structure or motivation to students. However, when learners want to make use of their earned badges within a different context, such as a job application, potential barriers external to where the badges were earned arise.
Open badges were initially created by Mozilla as a way of establishing a set of standard metadata associated with digital badges. IMS Global picked up the open badges efforts from Mozilla and established the Open Badges 2.0 (OBv2) specifications in June 2018.9 The specifications support the ability to transport digital badging information from system to system. The specifications on stored data are broad, so actual data from badge to badge can vary greatly while still following the Open Badges standard. This variability presents a challenge similar to that of comparing résumés or portfolios of work. While some large organizations have automated methods for streamlining résumé processing, manual review is still required to make decisions about what is valuable and how to compare across multiple and differing claims. To use an example, consider two applicants for a job, each claiming to have experience with multimedia production and both pointing to educational experiences to support the claim. Using digital badges or a course transcript presents similar comparison challenges. However, the digital badge for the experience has an advantage in that it fully describes the experience and provides evidence of the student’s claim, whereas the transcript provides only the name of the course, the weight in course credits, and a letter grade received. The badge evidence is superior to the transcript but requires more effort to interpret. In an era where expert systems are used for initial comparison of job applicant credentials, digital badges will continue to require human appraisal.
Badges form representations or claims of educational experiences that may or may not be valued by others. Similar to other educational currencies such as degrees or professional certifications, badges can offer a way of representing academic achievement through digital metadata. That is, for learners and employers, digital badges function as a mechanism to represent learning. We argue that digital badges, like other educational recognition, act as a currency to varying degrees in the three largest areas for learning: elementary and secondary learners preparing for postsecondary school, postsecondary learners preparing for jobs, and on-the-job learning for professional growth and advancement. In this chapter, we offer these three generalized scenarios for the adoption of badges as educational currency.
As indicated in the three generalized scenarios above, badge authors might view badges as metaphorical currency because the badges represent something they value. The currency metaphor must also pass the value test for both the primary and secondary audiences. That is, does the learner (primary audience) value the badge enough to put in the work to earn it, and do college admission offices and employers (secondary audience) see value in the badges that are earned? This is a significant external barrier to adoption as it requires significant numbers of people to agree on the value of a badge.
If badges are not deemed acceptable currency by the majority who interact with them, they will never be widely accepted. Badge earners need at least one badge in a given area for badges to be effective, but is there a true minimal number needed for badges to be accepted by earners, teachers, administrators, and employers? Although all these groups must be considered, this chapter examines buy-in at the individual level.
How are individual goals related to the acceptance of badges? At the individual level, goal setting is critical, and it is generally believed that grades and personal satisfaction are generally tied to academic goals. Although research is sparse here, the findings of one relevant study by Fanfarelli and McDaniel indicated the number of badges earned seems to correlate with a higher final grade for undergraduate males but not females. However, the authors infer that women may derive greater satisfaction from a badging system and earning badges than men.18
Research by Denny discovered a correlation between the number of times individuals viewed information about badges (possible goal setting and confirmation) and the number of badges collected.19 The mechanisms that prompted individuals to view badge information seemed to play a role in the number of views. It must be noted that this was a study limited to badges collected on a single platform, not about badges collected over multiple platforms or offerings, making generalizations difficult. While these findings are preliminary and more research is needed in this area, it may be that the importance of the number of badges earned varies from individual to individual, and also from group to group. Mechanisms prompting students to view badges may contribute to the perception of the importance of badges at the individual level.
The structure of the goals within a suite of badges may also contribute to the acceptance and adoption of badges by the individual. Conceptualizing badge creation with proximal and distal goals may assist in the development of a suite of badges. The learner sees an immediate need to complete proximal goals—also called subgoals—and their value is clear to the learner. Distal goals are less urgent, and their value may be less clear.20 A suite of badges, each with a small focus that contributes to a larger goal, is used in some programs, such as the Penn State Information Literacy Badge, and may be one method of creating the critical mass needed for a suite of badges.21 The Penn State Information Literacy Badge consists of ten sub-badges that can be used to earn three “meta badges.” If students earn the three meta badges, they can earn the overall “über badge,” indicating they have successfully earned all the badges in the suite. The sub-badges here relate to proximal goals, such as “Refining Your Search,” and the “Savvy Searcher” meta badge provides the distal goal.
Many educational institutions have not broadly implemented digital badges. Variations in badge design, evidence provided, the potential exposure of substandard assessments, complex support structures for badges, the variable value of badges, and buy-in by potential stakeholders all contribute to a slow pace of adoption. While it is possible from a technical standpoint to author and award digital badges, it is likely that the reasons outlined above are some of the major hurdles delaying implementing large-scale efforts with digital badges. We hope that the sections above offer the reader perspective on the challenges for adoption and some ideas on how they may be mitigated.
* Chris Gamrat (PhD, Instructional Systems) is an Instructional Designer for the College of Information Sciences and Technology at Penn State University. He develops and supports the college’s Master of Professional Studies degree programs. Gamrat completed his Ph.D. in 2018 in Learning, Design, and Technology from Penn State. He is currently focusing his research on personalized learning.
Brett Bixler (PhD, Instructional Systems) is an Instructional Designer in the e-Learning Design Innovation Group (eLDIG), part of Penn State’s Smeal College of Business. Bixler believes that engaging, motivating activities are at the heart of stellar learning experiences. In his current position, he works with the latest education technologies and learning theories to design engaging courses. Bixler is actively investigating the use of badging, educational games, and gamification for educational purposes, and works with various committees throughout the university to provide teaching and learning opportunities to Penn State students, faculty, and staff.