Date of Award

3-2024

Document Type

Thesis

Degree Name

Master of Science in Computer Science

Department

Department of Electrical and Computer Engineering

First Advisor

Mark G. Reith, PhD

Abstract

The Department of Defense is adopting Digital Engineering practices for its workforce. Simultaneously, the larger Systems Engineering community strives to modernize and define those Digital Engineering practices. These efforts to move from traditionally document-based approaches to pure digital implementations will require enhanced capabilities to manage and digitally track the lifecycle of a program or product. However, this growth must address tool design through an iterative process focusing on usability for many user types. Many tools and technologies exist but often lack an assessment of usability when engineers design tools for other engineers. Including usability when developing solutions for Digital Engineering tasks may simplify tools and increase further adoption, garnering a larger community of users. This research evaluates twelve participants using Stitch, a custom web-based tool for creating Systems Modeling Language Version 2 (SysMLv2) models and interacting with Virtual Agents. The study uses a mixed-methods approach that combines telemetry metrics and survey responses to evaluate the application’s usability. Through a series of four vignette pairs, Students and Professionals build Systems Modeling Language Version 2 (SysMLv2) models from source documents and investigate model elements. This process balances a traditional file-browser approach and support from Virtual Agents with a pair of visual styles. In this process, the telemetry and survey responses provide insight into the usability of Stitch and the effectiveness of the Virtual Agents. Trends in the study uncover variations in performance and usability within and between both groups. In some cases, Stitch was usable, and the Virtual Agents effectively supported building models. However, the study also found that participants expect additional capabilities and features to support their model-building efforts. This thesis contributes Stitch, built with a user-centered design methodology and evaluated by a use case study as foundational work to understand building SysMLv2 models with support from Virtual Agents. Furthermore, this thesis contributes four pairs of vignettes that guide users through building SysMLv2 models in an instructional fashion. Additionally, two custom model styles and a method for extensibility provide visual queues to aid model interpretation. The vignettes also guide users through interacting with Virtual Agents to access documents while learning to develop SysMLv2 models. Finally, this thesis contributes a set of recommendations for future work to improve the usability of Stitch and other Digital Engineering tools.

AFIT Designator

AFIT-ENG-MS-24-M-006

Comments

A 12-month embargo was observed for posting this work on AFIT Scholar.

Distribution Statement A, Approved for Public Release. PA case number on file.

Download

Share

COinS