Date of Award
3-2024
Document Type
Thesis
Degree Name
Master of Science in Computer Science
Department
Department of Electrical and Computer Engineering
First Advisor
Mark G. Reith, PhD
Abstract
The Department of Defense (DoD) has identified the need for a technically proficient workforce in the areas of science, technology, engineering, and mathematics. To meet this need, the DoD is actively seeking innovative technology capable of creating workforce development opportunities that are both accessible and effective. Educational research indicates serious games provide a potential avenue to achieve this goal. Unfortunately, a limited number of tools that simplify the game development process and leverage artificial intelligence are available. Content-generating artificial intelligence might help reduce instructor and game-based assessment designers' workloads while promoting individualized learning in students. This research presents a novel tool that utilizes ChatGPT's content generation ability to create narrative game-based assessments and evaluate student impacts. Four instructors used the tool and generated game-based assessments for class use. Data were collected via an instructor survey, student pre/post-assessment surveys, a traditional assessment, and a game-based assessment. Although findings show that ChatGPT narratives result in less engaging and fun experiences, evidence suggests the generated game-based assessments do not produce significantly different scores than traditional assessments. The lack of a significant score difference suggests that game-based assessments are no worse than traditional tests in terms of assessment effectiveness. The generated game-based assessments may also reduce student test anxiety when taking an assessment of appropriate difficulty. The GBA Creation Tool, data collected during this research, and ChatGPT Integration Framework appear beneficial to the game-based assessment research community and the DoD.
AFIT Designator
AFIT-ENG-MS-24-M-024
DTIC Accession Number
AD1318967
Recommended Citation
Patterson, Kevin B., "Opportunities and Limitations: Integrating Narrative AI into Game-Based Assessment Creation and Evaluating Student Impacts" (2024). Theses and Dissertations. 7685.
https://scholar.afit.edu/etd/7685
Comments
A 12-month embargo was observed for posting this work on AFIT Scholar.
Distribution Statement A, Approved for Public Release. PA case number on file.