CLIPE: Creating realistic and interactive virtual characters for VR/AR environments

At a time when virtual reality (VR) and augmented reality (AR) are rapidly growing, it is increasingly important to create virtual worlds with characters that not only look realistic, but can also interact with other users, react in a believable manner and adapt to complex situations. Until now, developing this type of character required large teams of designers and programmers, increasing costs and production times. This makes it difficult to scale projects and limits opportunities in areas such as video games, entertainment, training, simulations for industrial environments or the healthcare sector.

To address this challenge, the CLIPE project (Creating Lively Interactive Populated Environments) was launched with the aim of training a new generation of researchers and innovators capable of creating realistic virtual characters, interacting with users and responding credibly, while at the same time developing advanced technologies for the creation of more intuitive, autonomous and realistic virtual characters, adapted to new complex scenarios.

To achieve this, innovative techniques in machine learning, procedural animation, simulation and perception studies have been combined, enabling characters to be endowed with new example-based skills, the generation of large-scale virtual populations, and the rigorous evaluation of the quality of their behaviours and communication capabilities.

In particular, CLIPE improves three fundamental aspects:

• The quality of user–character interaction, ensuring that characters do not pass through the user but instead respond through collisions and realistic behaviours.
• Embodiment (the ability to see and feel the virtual body as one’s own body).
• Users’ psychomotor abilities, and specifically the improvement of precision when interacting with characters and objects.

Among the capabilities developed within the framework of the CLIPE project are also included:

• Machine learning techniques to provide characters with behaviours and skills derived from real examples.
• Procedural animation and simulation (an animation technique in which movements, behaviours or transformations of an object are generated automatically through algorithms, rather than being created frame by frame by an animator), to generate coherent virtual characters adaptable to different contexts.
• Perception studies and interaction analysis, aimed at evaluating the credibility, communication and responsiveness of characters.
• Specialised training, combining advanced research with placements in companies in the sector to ensure the industrial applicability of the results.
  
  

CLIPE will contribute to positioning Europe at the forefront of virtual character design for VR and AR, strengthening the competitiveness of animation, video game and immersive technology companies. In addition, its training programme will equip young researchers with key skills to succeed in highly competitive sectors, both in industry and in the entrepreneurial ecosystem.

Consortium, budget and funding

In addition to UPC, academic and research partners include: the University of Cyprus (coordinator), INRIA, University College London, Trinity College Dublin, the Max Planck Institute for Intelligent Systems, KTH Royal Institute of Technology, École Polytechnique and Silversky3d Virtual Reality Technologies Ltd (a spin-off from the University of Cyprus). These are complemented by industrial partners: Treedy’s SPRL, Amazon Development Center Germany GmbH, the British Broadcasting Corporation (BBC), Golaem and Ubisoft.

The project had a total budget of € 4,009,882.68, funded by the Horizon 2020 programme (2014–2020), under the Marie Skłodowska-Curie Actions – Innovative Training Networks (European Training Networks) sub-programme of the European Commission (H2020-860768-CLIPE). CLIPE had a duration of 4 years and 6 months (March 2020 – August 2024).