Augmented Reality Based on Navratri Motion Capture Project | Maria & Saanvi

Idea: We wanted to recreate the cultural environment for the mannequin of our motion capture that we created in the last project. This mannequin replicated a dance from the Hindu festival Navratri which spans 9 nights and was going on from October 15-24. The dance, formally known as Garba, includes a step known as Dodhiyu, which we chose to animate. Since it is performed in a circle around multiple people, we duplicated our mannequin to perform the dance in a circle. Below is an image of the dance in its main environment.

We wanted to recreate an environment replicating the abundant lights, arches, and colors that are typical to the festival. As a complete animation, we hoped to set the stage for this environment and then trigger the mannequins to start dancing within the environment as they would in a typical Navratri setting.

Creating the Mannequin (Process): For the last project, we used Rokoko.com to create our animation. Using our phones we filmed the dance in a well lit room against a white back drop. We also experimented with filming multiple bodies at once, but were unsuccessful as Rokoko uses just one mannequin.

IMG_5622

 

After we were happy with our animation, we exported the .fbx file and imported it into Blender to duplicate and arrange our dance.

In Blender we were tasked with duplicating our animated mannequin. We learned that it was not possible to simply select the object duplicate by pressing Shift D, but rather we grouped the mannequin with the armature and used Shift A > Collection Instance > Collection > Move. Using these key commands we arranged the animated mannequins in a circle.

Final Animation:
However, we faced some challenges exporting this file in Blender as a .fbx file. Since the software kept crashing, we had to use the initial file from Rokoko.com to import into Aero. In Aero, we then duplicated the one mannequin (doing Dodhiyu) and created 6 total mannequins to provide the circular performance effect.

Building the Environment (Process): To create the surrounding lights and arches in the environment, we used varying processes. Maria created the surrounding string lights in Blender and added in an animation so that these lights would flicker when the animation was played. However, in translating this to Aero, the light flicker animation did not replicate on Aero; this was a challenge that we faced and, instead, ended up simply exporting a solid color set of string lights into Aero.

We also created a backdrop or a set entering the festival of Navratri. At the front entrance of the festival, there is typically a sign welcoming everyone into the venue; thus, we developed a 2D set of three arches. Saanvi helped create and add-on the colorful aspects of the arches on Illustrator. Once this Illustrator file was exported as a .psd in Photoshop, we utilized the layers tool on Aero to pull apart the layers of the background. The arches and the ‘happy navratri’ text on the sign became two separate layers in the environment.

After importing this Illustrator file, the string lights, and the .fbx file of the mannequin doing Dodhiiyu into Aero, we were able to align all of the assets on top of one another and produce the screen recording below. Below you can see a video of Maria walking through the space in Aero using the Aero app.

Another animation that we also tried was a firework animation. This animation also didn’t translate onto Aero from Blender. A screen recording of the firework animation on Blender is attached below.
Firework Animation:
Although the firework animation and the lights animation did not work as expected, we were able to translate the basic files into Aero and work with them as simple assets. The final composition was able to translate the environment of Navratri. Our trigger for the animated behavior (mannequins) was set on the sign of the entrance and signals for the mannequins to start dancing. In the future, we would be happy to improve this aspect of the project and make our trigger and behavior sequence for the animation more complex.
Improvements: In the next project, we want to animate the lights in the Navratri environment and provide a better augmented reality experience. We would also set our triggers and behavior better for the videos so that we can truly create an augmented reality experience.