Ann Li

Despite the prevalence of welding in construction, manufacturing, and creative arts, training novice welders is a time-intensive and challenging process.

Metal welding is a craft manufacturing skill that can be unusually difficult to externalize and represent to novices. Building competency requires an apprentice to iteratively practice embodied skills and sensitize themselves to a sensorially complex practice. Learners must build deep embodied knowledge such as hand-eye coordination, muscle memory, and sound identification.

To explore these challenges, a series of co-design workshops was conducted with a youth program in welding and fabrication. Opportunities for mixed reality, sensing, and tinyML processes to augment welding training and practice were identified, leading to the design of an extended reality welding helmet and torch.

We apply machine learning to a lightly-modified off-the-shelf XR and welding setup to enable in-situ welding training, enhancing the embodied learning of welding in three key ways.

An XR system assists with limitations of traditional welding education, by providing realtime, in-helmet, and in-situ feedback and response to augment the educational experience

By enabling users to encounter real behaviors, forces, sounds, and overall experience of an active weld, we provide a novel in-situ experience with spatial and biometric analysis

The designed system allows us to explore other aspects of embodied learning in welding practice, including the analysis of biometric data, performance analysis, and the inclusion of meditation to further augment the educational experience.

Visual XR Guides and Integrated Motion Sensing

Combined motion-sensing and visual XR feedback helps improve proprioceptive and embodied learning

Sensing Sonic Cues During Welding Practice

Acoustic sensing focuses learner attention on non-visual cues of weld performance

Mediated Meditation and Regulation

Biometric sensing enhances mindfulness and stress management in sensorially challenging environments

Prior work in the areas of extended reality and welding create isolated, fully immersive simulations of welding, rather than an augmentation of actual welding in context.

Existing products and research include full VR welding experiences (training without a functional helmet or torch, but a visual augmentation with hand tracking) and AR/computer vision-based simulations using tracking markers.

The novelty of our work includes the ability for it to be used as an in-situ, functioning augmentation (implemented to work with a live spark) as well as our holistic augmentation (beyond training muscle memory, our system also uses additional sensors and the Quest's onboard sensors to promote good practice through feedback on mindful breathing, weld performance, and other attributes).

Ann Li

is a multidisciplinary designer
driven by storytelling, sense-making, and curiosity, exploring experiential narratives and creative tech

Best viewed on desktop

Ann Li

Augmenting Embodied Learning, Manufacturing Futures Institute

Integrating XR and tinyML to train skilled workers for future opportunities in creative arts and manufacturing

Roles

Context

Team

Tools

Despite the prevalence of welding in construction, manufacturing, and creative arts, training novice welders is a time-intensive and challenging process.

We apply machine learning to a lightly-modified off-the-shelf XR and welding setup to enable in-situ welding training, enhancing the embodied learning of welding in three key ways.

An XR system assists with limitations of traditional welding education, by providing realtime, in-helmet, and in-situ feedback and response to augment the educational experience

By enabling users to encounter real behaviors, forces, sounds, and overall experience of an active weld, we provide a novel in-situ experience with spatial and biometric analysis

Visual XR Guides and Integrated Motion Sensing

Sensing Sonic Cues During Welding Practice

Mediated Meditation and Regulation

Prior work in the areas of extended reality and welding create isolated, fully immersive simulations of welding, rather than an augmentation of actual welding in context.

Publications and press for more details on our co-design process and hardware development, system setup and training specifics, related works, and citations

Ann Li

Ann Li

is a multidisciplinary designerdriven by storytelling, sense-making, and curiosity, exploring experiential narratives and creative tech

Best viewed on desktop

Ann Li

Augmenting Embodied Learning, Manufacturing Futures Institute

Integrating XR and tinyML to train skilled workers for future opportunities in creative arts and manufacturing

Roles

Context

Team

Tools

Despite the prevalence of welding in construction, manufacturing, and creative arts, training novice welders is a time-intensive and challenging process.

We apply machine learning to a lightly-modified off-the-shelf XR and welding setup to enable in-situ welding training, enhancing the embodied learning of welding in three key ways.

An XR system assists with limitations of traditional welding education, by providing realtime, in-helmet, and in-situ feedback and response to augment the educational experience

By enabling users to encounter real behaviors, forces, sounds, and overall experience of an active weld, we provide a novel in-situ experience with spatial and biometric analysis

Visual XR Guides and Integrated Motion Sensing

Sensing Sonic Cues During Welding Practice

Mediated Meditation and Regulation

Prior work in the areas of extended reality and welding create isolated, fully immersive simulations of welding, rather than an augmentation of actual welding in context.

Publications and press for more details on our co-design process and hardware development, system setup and training specifics, related works, and citations

is a multidisciplinary designer
driven by storytelling, sense-making, and curiosity, exploring experiential narratives and creative tech