Developing a real-time animation workflow

Peter Heron, Founder of Valentine Parsley Ltd partnered with XR Stories on a research and development (R&D) project to develop an Unreal Engine-based workflow that allows filmmakers to direct animated characters inside Unreal Engine (UE).

Funded by an R&D grant from Animation Ireland, the project explored options to develop a low-cost, low-carbon, real-time animation workflow for a 24-hour live feed of a digital sun bear reserve.

Over a three month period, the project team worked in the XR Stories immersive R&D lab in The Guildhall, York to develop:

A prototype of a fully responsive real-time 3D environment inside Unreal Engine 5: To maximise the realism and creative possibilities of the workflow, all aspects of the digital reserve (the floor, foliage, lighting, and weather states) remain fully interactive, dynamic, and synchronised to a 24-hour cycle, offering multiple camera angles and directable interactions.

Animation workflows: The project established that working with actors to motion capture bear animations would exceed the budget and scope of the R&D project. Instead, the team worked with stock bear animations and shifted focus to a prototype behavioural system that features a user interface that enables the director to select from different times of day and different weather conditions with associated different bear behaviours. The initial prototype  system has the potential to be expanded significantly.

A virtual environment: The project team made use of a high quality pre-made/stock bear from Unreal Engine’s Fab Store and worked with UE’s landscape and foliage tools to build a small reserve for the bear character to inhabit, set time states linked with sun trajectory, and introduced an environmental variable – rain.

Virtual cameras: The team established three camera positions, switchable via the user interface, that correlate to where the bear is at different times of the day and weather conditions. CineCamera parameters were chosen to simulate the type of cameras used in wildlife filmmaking (16:9 digital cine camera film backs and longer lenses, with shallow depth of field).

A green screen live broadcast workflow: The project created a greenscreen broadcast  workflow and resources (including a user interface) for filmmakers and directors to control the broadcast at a basic level. Intended for use by Irish studios, the workflow uses free industry standard software (OBS) which can be refined and amended for future phases of the wider project, which is still in development.

Visit the XR Stories immersive R&D lab hire page for information about how to access the project’s in-house team and state-of-the-art research and development facility. 

Categories: R&D