BiblioBouts Project Activities
The BiblioBouts Project featured 13 activities over a 4-year (October 1, 2008 to September 30, 2012) period.
1. Design and Develop BiblioBouts #1 (Year 1)
The BiblioBouts Project team designs and develops the BiblioBouts information literacy game. BiblioBouts is an online tournament made up of a series of mini-games or bouts, each of which introduces students to a specific subset of information literacy skills within the overall research process. The game gives players feedback abut how well they perform vis-a-vis their opponents. It also gives them repeated opportunities for practice and reinforcement so they retain and add what they learn to their daily work habits.
The alpha version of BiblioBouts is limited to basic functionality for signing onto the game, finding sources, saving sources in Zotero which transfers them to BiblioBouts, reviewing and choosing one's own sources to submit to the game, evaluating opponents' sources, categorizing them, and choosing the best sources from the pool of all sources submitted to the game for one's best bibliography on a specific topic. Instructors use the game's administrative interface to create a game but most charge BiblioBouts Project team members or their library's liaison with game creation.
2. Learn About the Research Needs of Incoming Students (Year 1)
To learn about the research needs of incoming students, the BiblioBouts Project team interview faculty at participating institutions who teach the classes in which BiblioBouts is deployed. Open-ended questions ask instructors about the learning objectives of their courses, the written assignment that students complete while they play BiblioBouts, the criteria they use to grade assignments including the citations students include in their papers, and the expectations they have of students who will be playing the game.
3. Conduct Baseline Study #1 (Year 2)
To rate the citations students choose for BiblioBouts and their research papers, team members develop the Five-Faceted Taxonomy for Classifying Digital Sources that features these facets: (1) information format, (2) literary content, (3) author identity, (4) editorial process, and (5) publication purpose; see Chris Leeder's "Developing a faceted taxonomy" article in College & Research Libraries 73, 2 (2012): 115–133, for taxonomy details. Within each facet are listed 6 to 24 categories that describe the source in hand. There are 24 categories in the Information Format facet, and examples are blog, policy statement, consumer magazine, encyclopedia, monograph, and scholarly journal. BiblioBouts team members consult Ulrich’s Periodicals Periodicals Directory to develop some categories. Each category is applied a score that ranks citations with respect to the extent to which they are likely to be vetted by an objective review and/or editorial process. Category scores are determined by a panel of judges, i.e., three instructors whose classes played BiblioBouts. Team members conduct a pilot test to determine whether the taxonomy is satisfactory for assessing the quality of the citations in student research papers.
4. Test BiblioBouts #1 (Year 2)
The BiblioBouts Project team deploys the alpha version of BiblioBouts in selected classes at 4 participating institutions: (1) Saginaw Valley State University (fall 2009), (2) Troy University Montgomery Campus (winter 2010), (3) University of Baltimore (winter 2010), and (4) University of Michigan (fall 2009 and winter 2010). Faculty are in charge of integrating game play into their courses including leading class discussions that transition students from one game phase to the next and monitoring the intensity and nature of student game play. They call on our project liaisons who are librarians to show their students how to use Zotero, demonstrate BiblioBouts, and brief students on library-portal-based databases that are promising sources of information for the broad-based topics instructors choose for the game.
5. Evaluate Game Play #1 (Year 2)
BiblioBouts Project team members and liaisons at participating institutions conduct an evaluation of the game. We use a multi-methodological approach to collect data to answer 10 research questions. See the Evaluation Plan for details.
6. Analyze Evaluation Data and Report Findings #1 (Year 2)
The BiblioBouts Project team analyzes evaluation data focusing especially on findings that they use to redesign and enhance BiblioBouts in its second design and development stage (activity 7) and to facilitate integration into courses for additional testing (activity 9). Emerging from the analysis are these major themes and subthemes:
- Before game play
- Playing BiblioBouts
- Playing the [Donor, Closer, Rating & Tagging, Sorter, Best Bibliography] mini-game
- What do students learn as a result of playing BiblioBouts?
- Students’ preferences for learning about information literacy
- What other games do students want to play?
7. Make Design and Development Improvements to BiblioBouts #2 (Year 2)
The project team draws on evaluation findings to build on its initial design and development of BiblioBouts. The most pressing development issue facing the BiblioBouts team is a simplification of the game’s initial sign-on procedure. We consult the Zotero programming team and follow through on their suggestion to build a system that uses Zotero file storage and the Zotero user name and password to authenticate users to BiblioBouts.
Second is the Sorter bout. Players detect redundancy between the Tagging & Rating bout's tagging task and the Sorter's categorization task. Some players sort hundreds of sources, running up a huge lead that discourages other students from playing.
Third are the design problems that plague with the Best Bibliography bout. Players want to choose their own research topics and best sources for the papers they write.
Fourth is the need for a game-play update that instructors can monitor to determine their students' participation in the game. Instructors also need access to game-play data so they can grade students on the extent to which they play the game and the quality of their game play.
Fifth, students want access to everyone's sources after the game ends so they can find sources to support the arguments they make while writing their papers.
The BiblioBouts team's response to these five issues results in these major redesign efforts and enhancements:
- Eliminating the Sorter bout.
- In the Rating & Tagging bout, prompting students to enter the 3 “big ideas” discussed in the source at hand into 3 separate dialogue boxes using a type-ahead capability to facilitate data entry and minimize spelling and typographical errors.
- In the Best Bibliography bout, prompting students to: (1) enter their own research topics, (2) choose 3 big ideas that they would discuss in their papers on this topic using a type-ahead capability that prompts them to choose the big ideas that they and fellow students contributed to the Rating & Tagging bout, and (3) choose the best 10 sources for their own topic.
- Adding a visually-oriented game-play update that instructors can scan to determine who is and is not playing BiblioBouts and a final report that they can use to grade their students' game play.
- Adding a Post-Game Library that contains all sources students submitted to the game including citations, abstracts, big ideas, ratings, and digital full-texts.
Other improvements that the BiblioBouts Project team makes to the game are:
- Adding scoring feedback to BiblioBouts that tells players why and when they earn points.
- Streamlining interface of the Rating & Tagging bout to be less time-consuming, less repetitive, and more useful for game players.
- Adding a feedback interface to Rating & Tagging so players can see their opponents' ratings and tags.
- Also in Rating & Tagging, eliminating "discipline" and "audience" tags due to their limited usefulness in subsequent bouts and replacing them with format tags that ask players what the donation is (such as scholarly journal, newspaper, blog) and who published it (such as higher education, government, commercial business).
- In the Best Bibliography bout, improving its sorting capabilities, e.g., sort by relevance scores, sort by credibility scores, and sort by big ideas.
- Building a database of all closed, tagged, and rated resources that game players and administrators can search during the Best Bibliography bout and for a limited time period after the BiblioBouts game ends.
- Allowing instructors and/or game administrators to set game parameters for quotas and caps.
- Adding more scoring opportunities to BiblioBouts that reward players for positive actions such as opening citations and full texts, matching other players' tags, agreeing with other players' relevance and credibility assessments and reducing bonus points to players who exceed Donor and Tagging & Rating quotas to discourage spamming and insurmountable point spreads between leaders and the pack.
8. Conduct Baseline Study #2 (Year 2)
Originally, it was the team's intent to collect research papers from students did not play the game and compare the quality of their citations with papers written by students who played the game. Instead, we assess the quality of citations that players (1) submit in the Closer bout, (2) choose in the Best Bibliography bout, and (3) cite in the research papers their instructors grade. We use the taxonomy (see activity #3 above) to assess citations. We hypothesize that citation quality will improve as a result of playing BiblioBouts.
9. Test BiblioBouts #2 (Year 3)
Beta version 1.0 of BiblioBouts game play debuts in select courses at our participating institutions and at institutions that have heard about BiblioBouts from our publicity, the Chronicle article, and the blogging that followed. This is a redesigned and enhanced BiblioBouts that reflects improvements to the game (see activity #7 above) and that are the result of the first evaluation of the game (see activity #5 above).
10. Evaluate Game Play #2 (Year 3)
BiblioBouts Project team members and liaisons at participating institutions conduct an evaluation of beta versions 1.0 of BiblioBouts. Mostly we rely on pre- and post-game questionnaires, focus groups, and personal interviews, methods we found especially useful in the first evaluation (see Activity #5 above).
11. Analyze Evaluation Data and Report Findings #2 (Year 3)
The project team analyzes evaluation data focusing on findings that they can use to put the finishing touches on BiblioBouts and that reveal the game's ability to improve student research, research skills and knowledge, and student performance in the classroom. See Publications & Reports for a list of published papers and manuscripts in process.
12. Make Design and Development Improvements to BiblioBouts #3 (Year 3)
The project team draws on evaluation findings to build on the design and development of BiblioBouts. The most pressing need is to separate BiblioBouts from source finding, saving, and managing tasks. To accomplish this, we eliminate the Donor bout from BiblioBouts. In the future, BiblioBouts will expect players who sign onto the game to have "in hand" a minimum number of sources (prescribed by their instructors) and submit these sources using the citation manager of their choice (e.g., EndNote, RefWorks, Papers). For the time being, players will use Zotero to submit sources from BiblioBouts.
Improvements that enhance BiblioBouts' game-like qualities are badges that players earn for meritorious and less-than-meritorious game play and a scoring log that details every point players earn.
Functionality improvements include adding a big-ideas search to the Best Bibliography bout's Source Library and Post-Game Library and emailing news items to players.
Examples of interface improvements are enhancing library sources with the most frequently co-occurring tags from the Tagging & Rating bout, adding improved navigation controls to Tagging & Rating feedback, and improving the display of sources in the game's two libraries.
Improvements to the game's scoring algorithm ensure that players who meet instructor-assigned caps and exceed quotas, agree with their opponents’ credibility and relevance ratings and content tags, and are the first to close the relevant sources that their opponents choose for their best bibliographies win BiblioBouts or place high on its leader board. See Game play for our vision of BiblioBouts beta version 2.0.
13. Support Widespread Distribution and Adoption of BiblioBouts (Year 3)
BiblioBouts beta version 2.0 was available for deployment from September 2011 to December 2012. Contact us at info@bibliobouts.org with questions about our progress finding future funding for BiblioBouts, identifying a permanent home that provide user support and ensure its future development, maintenance, and enhancement, or about BiblioBouts generally.
Future funding for BiblioBouts would enable us to add LMS-system support so that instructors can use their institution's Learning Management System (e.g., Sakai, Blackboard, Moodle, etc.) to invite students to the game and grade their performance. Profiling BiblioBouts to ingest sources from other citation managers and adding cross-platform support to Chrome, Internet Explorer 7 and 8, and Safari browsers would increase BiblioBouts' versatility. Implementing a BiblioBouts curriculum might be the tipping point to convince instructors to adopt BiblioBouts in their courses. Establishing user forums would enable experienced users to help newbies resolve a host of issues, from trouble-shooting technical problems to identifying winning strategies.
• Participants & Contacts • Project Reports & Publications • Instructor FAQ •
• Participants & Contacts • Project Reports & Publications • Instructor FAQ •
