XR Stories welcomes University of York researchers in residence

XR Stories has awarded R&D residency places to three University of York researchers, Matt Barnard, Steven Xiaotian Dai and Sam Crane.

The residents are working alongside the XR Stories team on a part-time basis, accessing a range of extended reality technologies in the project’s immersive R&D lab to enhance their ongoing research activities.

Over the next year, the residents will also have access to add hoc-support from the XR Stories team.

About the residents

Matt Barnard is a Lecturer in Sound Recording and Music Production in the Department of Theatre, Film, Television and Interactive Media (TFTI). Matt’s residency project will build on his existing research on deep learning processes to create animated visuals from audio signals. Matt aims to create a gestural control system that allows users to manipulate the audio, and in turn the animations, through movement. The R&D residency outputs will include a technical framework, an interactive control system and an interactive installation for implementation across industry activities including game sound design, live performance, and composition and music production. 

Steven Xiaotian Dai is a Lecturer in Cyber-Physical Systems in the Department of Computer Science. Steven’s residency project aims to integrate robotic technology and digital twins into the virtual production workflow to enhance precision, efficiency, and creative flexibility. Steven will undertake research into real-time synchronisation between the digital twin and physical robots, AI-assisted camera movement optimisation, and user-friendly interfaces to enable seamless director-robot interaction. The R&D residency outputs will include a fully operational robotic camera control system, a digital twin of the robotic system, and a user-friendly software interface for directors and operators.

Sam Crane is a postgraduate researcher in the Department of TFTI, undertaking a PhD by creative practice. In addition to his postgraduate role, Sam is an award-winning filmmaker, video artist and actor and his R&D residency sits at the convergence of these creative processes. Sam’s residency project aims to create a hybrid real/virtual live performance of Hamlet, ‘Be The Players Ready’, that will take place simultaneously in a digital video game and an irl (in real life) theatre space. Sam will explore merging in-game and irl worlds with LED screens, projections and stage lighting; camera-based gesture recognition; and audience participation as ‘The Players’, both in person on PS5s and online. The residency will also explore mechanisms for allowing in-game and online audiences to impact irl stage activity and the placing of live captioning/use of a BSL interpreter. The residency outputs will include a cost-effective, accessible model for immersive performance, repurposing established game environments for live performance and challenging the industry’s reliance on high-end VR and motion capture. 

Published on 25 June 2025