October – December 2022 (7 weeks)
Mixed Reality, Augmented Reality, UX/UI Design, Brand Identity, UX Research
Kristen Cai, Angela Nam, Shafiqah Zulfikar, Andrew Sim
Figma, Adobe After Effects, Adobe Aero, Cinema 4D, Unity
I led UX research, product strategy, and concept video development for this project.
.png)
In order to bridge the gap between art museum visitors’ understanding of art and the actual stories and techniques behind the art pieces, Artium uses approachable storytelling as well as multi-sensory and two-directional experiences to transform the museum as a learning resource, and turn curious visitors into collectors. Through this approach, Artium aims to foster a deeper sense of understanding and connection for those who visit Carnegie Museum of Art (CMOA) by putting art back into its original context.
Artwork-specific content will shown after visitor activates the experience. Visitors can select the stories they're interested in to learn more about the art and the artist's back story.
Using hand gestures, visitors can manipulate and interact with 3D objects unique to each piece.
Visitors can collect the 3D object they interact with into their virtual wristband and view back even after their museum visit.
Visitors can reveal original inscriptions on ancient sculpture pieces, to learn the meaning and its relevance to the culture.
Using the raise arms gesture, visitors can restore the colors of ancient architectural masterpieces and see the original interior back in 432 BCE.
Art museums are commonly considered institutions for providing education and cultural enrichment, where visitors can learn about different art forms, techniques, and styles. However, it is commonly known that typical museum settings put a barrier between visitors and artworks. Inaccessible language on placards and traditional audio guides prevent common museum visitors from understanding and appreciating the context behind the artworks they might otherwise have a deeper connection to.
During the exploratory phase of the project, our team tested our different types of emerging technologies to visualize how our concepts and initial discoveries will be transitioned in action. Hololens 2 was eventually chosen due to its ability to allow visitors to still interact with the physical environment in addition to the virtual reality.
In order to create an engaging experience for an emerging technology that we were unfamiliar with, we delved deeply into the technical aspects of the device to gain a thorough understanding of its functions. We dedicated substantial time to experimenting with diverse gestures and interaction models to determine the most effective approach.
To keep our project at a manageable scope, we decided to focus on three exhibition sections at the CMOA for the Artium experience: the Impressionism collection at the Scaife Gallery, the Religious paintings, and the Hall of Architecture.
We crafted the content that was specific to each artwork we would highlight based on thorough research on the history, context, and artist stories. To enhance visitor experience, we even designed gestures that were not originally included in the Hololens 2 guesture guide but were feasible from the technical perspective. We hoped our work could inspire the future development of the AR technology.
Since visitors yearned for a more affordable way to preserve their museum experience other than buying souvenirs at the museum shop, we added a feature to allow them to collect “objects” from artworks they have interacted with during the experience. To extend visitors’ experience beyond the museum, visitors are able to access the collected objects in their personal museum – in a sense, they’re their own curators.
Designing UI for the Hololens was challenging as it was our first time designing interfaces for a non-2D surface. We had to consider the depth of the space, which would affect how users interact with the elements in the interface. Secondly, how the elements in the interface would move and interact with each other in a 3D space was also crucial. Lastly, we had to be aware of the limitations of the 3D medium, such as the fact that it could be more difficult to see details in a 3D space than in a 2D space.
We experimented with multiple iterations of the UI before settling down on the final version, which encompassed all the considerations above, making the interaction experience straightforward and intuitive.
My foray into using augmented reality to enhance museum experiences was both novel and gratifying. This new venture required me to consider a multitude of factors, such as visitors' positioning, crowdedness, and acoustic ambiance, that can influence the overall encounter. This experience also challenged me to transcend my comfort zone of designing in two dimensions and compelled me to take a more comprehensive view of the visitors' journey.
It's essential to respect visitors' preferences and highlight the importance of experiencing art through the most natural medium possible. Additionally, we recognize that not all visitors may have access to augmented reality technology, so it's vital to provide information in the traditional format as well.
The experience model we developed for CMOA has potential to expand beyond the city of Pittsburgh – we hope the future development of technologies in museums can bring such experience to other corners of the world.
During the project, I recognized the importance of allowing visitors to record their feelings when they see artworks they especially resonate with. Visitors can record their emotions and reactions to the artworks they see, which could be shared with other visitors. This would allow visitors to connect with each other over their shared love of art and create a sense of community.
