Creating a 3D object still requires significant manual input. Professionals open their software, perform various modeling operations, adjust textures and materials, verify the result, and then adapt the model to the constraints of the environment where it will be used.
With the advent of generative models and multi-agent systems, this workflow is beginning to evolve. The operational goal is not to have an AI autonomously produce a 3D model; rather, it is about rethinking part of the creation process. Certain tasks can be delegated to specialized agents, allowing professionals to focus more on defining requirements, overseeing the output, and making necessary adjustments.
This is one of the avenues being explored by Immersion, a consortium member as part of MOSAICO project, which aims to develop solutions based on communities of AI agents for software engineering tasks. Immersion’s mission is to explore applications of artificial intelligence in immersive technologies, particularly for the development of XR applications.
A specific need for XR applications
As a partner in our European research project MOSAICO, Immersion develops among other things 3D and extended reality (XR) applications and experiences. This expertise entails specific constraints: models must be adapted to the hardware used, available computing power, and the performance requirements of each headset or XR environment. These constraints also apply to the applications themselves, particularly regarding resource and energy consumption. With the advent of artificial intelligence, Immersion, European expert in immersive and collaborative technologies is exploring the most effective process for the reasoned integration of AI into its value chain. This is one of the key objectives of the MOSAICO project.
In this context, practical experience shows that relying on a single general-purpose AI tool does not necessarily meet a developer’s needs. Producing 3D content may require a range of skills and, consequently, multiple complementary agents.
The use case studied in MOSAICO therefore relies on several specialized models. Trellis can be used for 3D model generation, iMotion for 3D animation, and UniRig for automating character rigging the process of creating the underlying structure that enables animation. The goal is to have these various components work together within an architecture capable of orchestrating the different stages. A multi-agent approach makes it possible to distribute tasks among several specialized agents, rather than requiring a single model to handle the entire process.
Moving from using software to describing an expected result
This evolution primarily changes the way work is approached.
In a traditional process, the professional performs the various operations directly within their 3D creation software. They know which tools to use and execute the steps themselves to achieve the desired result.
With multiple specialized agents, some of these operations can be defined upfront. The professional describes what they aim to achieve; agents then handle specific steps, iteratively using the necessary tools, and can consult the user when information or a decision is required.
Early experiments with the Model Context Protocol (MCP) illustrate this shift. This protocol enables an AI model to interact with external tools and resources. In a 3D creation environment, this makes it possible for the AI to communicate with software and tools installed on the machine and to execute specific operations.
In a 3D creation scenario, the professional could thus describe the expected outcome rather than manually performing every step of modeling a vehicle. The various necessary tools would then be engaged to handle parts of the creation process.
The change, therefore, lies not only in the tool being used but also in how the professional defines the objective and interacts with their work environment.
Specialized agents to meet the demands of XR
Creating a 3D model or environment is only part of the challenge. In XR applications, the resulting 3D content must then run on specific hardware.
However, headsets do not all share the same hardware and graphics capabilities. Factors such as the number of displayable polygons, texture resolution, scene complexity, and expected performance levels vary significantly from one device to another. Adapting content to these varying constraints still relies, to some extent, on repetitive manual intervention.
Beyond optimizing visual performance, this work also aims to improve the energy efficiency of applications. The limited battery life of headsets poses a significant obstacle to certain professional use cases. A key objective is therefore to reduce application energy consumption, thereby extending battery-powered usage times and facilitating integration into prolonged operational workflows.
Specialized agents could step in earlier in the process. They could identify the target hardware’s specifications, analyze the generated content, and either propose or implement the necessary optimizations.
For Immersion, this aspect is of strategic importance. One of the goals of the MOSAICO use case is to explore how these various agents can help produce 3D content more efficiently while accounting for the performance constraints of XR hardware.
Save time without eliminating human oversight
The primary, tangible benefit expected from these new workflows enhanced by agentic AI is a reduction in the time spent on repetitive tasks and various preparation stages. Currently, a single piece of content may require multiple adaptations for use across different devices. Part of the workflow involves identifying platform-specific constraints and making the necessary adjustments.
If this knowledge could be integrated into the process, certain operations could be performed more systematically. Work done once could also serve as a foundation for future adaptations, guided by checks and rules tailored to specific needs.
However, the goal is not merely faster execution; process reliability is also a key factor. Automating certain operations and using specialized agents for verification could minimize rework and ensure greater consistency across projects.
Nevertheless, this approach requires maintaining human oversight. The professional remains responsible for the final outcome and must be able to understand the system’s output, the rationale behind specific operations, and any corrections that may be required.
A new place for 3D expertise
This development raises a broader question: as some technical tasks become increasingly automated, how is the role of the 3D artist evolving?
The underlying know-how does not disappear, but the nature of the expertise may shift. Whereas professionals once spent much of their time directly executing tasks within the software, they may now find themselves devoting more time to defining requirements, overseeing the process, and evaluating the results.
This requires new skills. Professionals must be able to precisely describe what they expect from an AI system, while also understanding its inner workings well enough to analyze its choices and identify errors.
Trials conducted as part of the MOSAICO project demonstrate that AI does more than just evolve production tools; it can also transform the relationship between professionals and the tools they use.
Experiment before generalizing
Within MOSAICO, Immersion plays a dual role: contributing to ongoing research on these transformations and testing them against the real-world needs of 3D and XR development.
There are technical challenges involved as well. Indeed, 3D generation tools are even less standardized and documented than tools based on large language models. It is necessary to determine how to enable communication between the various components, select the right protocols, and build an architecture capable of orchestrating them.
Agent-to-Agent (A2A) interaction is particularly important for managing exchanges between agents. In the MOSAICO approach, this communication must allow agents developed using different technologies to collaborate within the same environment.
Immersion’s use case thus provides an opportunity to test the multi-agent approach in a scenario with numerous constraints: 3D content generation, animation, rigging, optimization for various devices, and consideration of available resources.
Naturally, the goal is not to replace existing practices overnight. Instead, the aim is to identify tasks that can be delegated to agents, measure the actual value they provide, and determine how to integrate them into the workflows of 3D teams.
Ultimately, this type of experimentation could transform the way immersive content is created: fewer repetitive operations performed directly within software, greater orchestration between various tools and agents, and always a professional on hand to define objectives, review results, and contribute their expertise.
The MOSAICO demonstrator is now operational on-site at Immersion. Teams are currently working on integrating agents tailored to the project’s use cases specifically, a 3D asset generation agent covering the entire creation process from text to animation, and an XR development assistance agent. The latter utilizes a Retrieval-Augmented Generation (RAG) system, enabling the AI to draw upon a knowledge base comprising Immersion’s expertise and collected data on the subject to generate relevant responses. Once these agents are integrated, the teams will be able to begin evaluating the use cases.
The study and implementation of this use case were made possible thanks to the combined work of Immersion, Institut Mines-Télécom Atlantique, the University of York, and Net Company Eclipse.
You can download the full article here.