Project 4: Under the Sea by Esther and Katie
Concept: For our fourth AR project “Under the Sea,” we expanded on our initial mannequin models from Project 3. The previous project included ballet dancing movements. When Esther and I were thinking about an environment where we could place these dancers, we wanted to imagine a rather unexpected scene. The fluid ballet moves reminded us of enchanted figures or mermaid-like people which then led us to the creation of an underwater scene, “submerging” the users into the experience.
Process: First we took videos of the ballet dancing and swimming motions and converted them into moving models on Rokoko. After acquiring the FBX files of the moving mannequins, we imported them into Adobe Aero. For our additional swimming figure that the user can interact with, we acquired the mannequin from Mixamo. Mixamo provided a cleaner movement file as to the video files converted on Rokoko.
We wanted to incorporate features of the ocean like plant life and the background. With the underwater background, Esther wanted the surroundings to be able to surround the user while they are walking through the AR space. An image of the ocean surrounds all sides of the camera you can see as you move your device around. The added FBX files that represent the details of the environment such as the seaweed were found online. We transformed and scaled the files appropriately to match the scale of the background as well as the mannequins. The mannequins fitting the size of the seaweed and swimming out of the plant were intentional for us to create a fantasy-like scene.
To further create an interactive and controllable AR experience for the users, we added triggers and actions in Aero. The interactive element of the space is when the user presses the button, the mannequins begin dancing and swimming out of the seaweed and toward the direction of the button.
. 
Reflection: Esther was experienced with the Adobe Aero software, making the overall process much easier for us. While we attempted to navigate through the program in class, we went through trial and error as we had issues with having the online-imported FBX files as well as having the FBX files move properly. Esther also ran into issues with the triggers where the mannequins began teleporting when the buttons were pressed. In order to fix this issue, she used pins manually to have the mannequins move in the desired direction. As for myself, I had close to no experience with Aero and assisted Esther in gathering mannequin movement files and converting them for Adobe Aero. Overall, creating an immersive experience for users on various devices is a process that we hope to see more.
Link to Aero: https://adobeaero.app.link/menF9L55zEb