Prototyping of Google Glass app for mood analysis and wellness tracking

What is It

Upon the release of Google Glass, a workshop was conducted at the University of California, Berkeley to explore new possibilities for interaction in Google Glass. Our team of two crafted a physical prototype for a Google Glass app using low-cost materials. The purpose of the app was to allow the user to analyze mood and wellness of the interactee while conversing or partaking in social interaction.

How It Works

The app was designed to upload a real-time video every 10 seconds using the headset-mounted world camera and then have Google AI framework to analyze the uploaded frames based on a model created from a large collection of quantified computer vision data based on recognition of facial expressions.

A 5-point scale system and animated emojis were used as primary mechanisms to convey a range of positive, neutral and negative moods. The heart rate of the interactee was also measured and accounted for using the captured frames based on a rate of redness fluctuation on one's face, which occurs as a part of normal blood circulation in human.

Tasks

Based on these functions, we selected the most appropriate UI patterns based on typical social contexts. The exercise also involved combining a number of modals of interactions and feedback mechanisms (e.g. head-mounted display, voice commands, touch gestures [tap, swiping and multi-touch], haptic and audio feedback, head gestures [tilt, nod and shake] and gaze) to introduce new UI patterns not available in the Google Glass API. During a mock session, we wore the prototype and role-played several key scenarios. We also discussed ethical implication of AI and camera on wearable devices as well as privacy concerns thereof.

Contribution

Tortoise: biomimicry dress for sexual assault prevention

What is It

In response to the public's growing concerns towards women's safety, I designed an armor-like dress using a combination of Grasshopper, a generative and algorithmic 3D modeling tool, and 3D production software such as as Grasshopper, Rhino, Maya and Zbrush. The dress was designed in collaboration with wearable designer Francis Bitonti. The components of the dress were later 3D printed at Autodesk's San Francisco Technology Center.

How It Works

The dress is comprised of an array of ridges on its surface made from a liquid metal called rubidium. When a danger is sensed by the dress through a detection of rapid heart rate increase, the hardened rubidium surface is heated with high voltage to transform its surface into a liquid state and then morphed into spikes with embedded magnets.

Inspiration

The design was inspired by a 15th century Korean battleship in a shape of tortoise called, Geobukseon developed by an admiral and military general Yi Sun-Shin. The functional ornaments on the battleship including a dragon figurehead, armor in a shape of tortoise shell and metal spikes were used to invoke fear towards enemy combatants and protect the the ship from boarding tactics. Thus such design elements were incorporated into the dress.

Contribution

Building metaveillance prosthetics

What is It

With a guidance from Steve Mann, the father of wearable computing, I prototyped a prosthetic device that can be worn like a scarf or belt to allow capturing and tracking of human body locomotion including movement, distance, velocity and angle.

How It Works

The tracking is enabled through embedded array of infrared LEDs that are worn on a particular body part or appdenage. Once the wearer's sequential IR light field over a specified duration is captured through a camera, a field of "metaveillance" created by the wearer's movement can be visualized; thus revealing the invisible data.

Contribution

Tableware optimized for mixed reality dining

What is It

For Project Nourished, I developed a series of tableware to aid eating and drinking in VR and AR as well as delivering scents in various methods such as heating and diffusion.

How It Works

The specialized fork and chopsticks utilize tracking markers that reflect IR lights emitted by IR LED source and then their movements were translated into absolute XYZ coordinates using the video footage. An alternative method of tracking relative movement using accelerometer was also explored to compare the accuracy of captured motion data. Furthermore, micro interactions such as picking up of the food and releasing it into user's mouth were also detected by using conductive sensors as a set of binary data.

Contribution

Interspecies digital play interface

What is It

I created cat-friendly digital video game that can be played with pet owner in real-time.

Abstract
Despite owners' desires to include pets in their everyday activities, pets were not included in the digital experience prior to 2010. The Cat Cat Revolution is a digital game that allows both cats and humans to participate in play through a species-appropriate interface. The game applies Human-Computer Interaction principles to pets and casts pets as participants in the gaming experience.

Background

I originally proposed and developed a prototype of the game at a hackathon. After winning a third place, the game was developed and tested further for the purpose of academic research.

Outcome

During the pilot study, pet owners characterized the game as a mutually positive experience, describing the game as a fun way to play. CCR explores the effects of including pets in the digital experience. The research helped to create a whole new genre of pet products that offer digital mediation between owner and their pet. The paper was later accepted at ACM CHI '11: SIGCHI Conference on Human Factors in Computing Systems and has been cited in over 40 research publications around the globe to further our understanding of human and animal behaviors in relation to technology.

Contribution

Stimulation of orbitofrontal cortex with tDCS to increase desirability to consume inedible objects

What is It

The experiment was conducted based on a hypothesis that any forms including that of inedible objects may have the potential to satiate hunger or trigger desire to eat as long as an object in question carries specific patterns or attributes such as shape, color and motion which affect human's disire to consume. The purpose of the experiment was to question the definition of so-called food and draw attention to underexplored classifications of edible foods and non-edible foods.

How It Works

Using transcranial direct current stimulation (tDCS) and the 10-20 electrode placement method, I applied 2mA current onto C4 region to stimulate insular cortex while observing a series of moving images of non-edible objects that have been previously identified by 20 participants as "appetizing" or "wanting to eat."

References

Contribution

Reconfigurable edible sheet: an alternative form factor for ingestible

What is It

An experiment was conducted to explore dynamic forms and low-caloric configuration of food.

Process

Using a technique used in molecular gastronomy, I polymerized carrot juice into gelatinized sheets that can be reconfigured into new forms like origami. This was accomplished by first mixing the juice with calcium lactate gluconate and then pouring the mixture into a sheet of paper. The paper was previously soaked with a solution containing water and sodium alginate to allow the sheet's surface to act as an activator. Once the edible sheets gelatinized, they were folded into complex shapes and objects.

Outcome

By leveraging hydrocolloids, the process transformed a carrot with its original physical characteristics and textural qualities into complex three-dimensional forms—all while reducing the amount of calories per volume.

Contribution

Designing sensory care package for individuals suffering from homeless

What is It

Parachute is a care package for people who are experiencing homeless cope with difficult living conditions with sensory stimulation. The products contained in the package are designed to improve emotional wellbeing and promote activation of a variety of human senses.

Background

The project began after realizing that there is a disparity between the types of products given to the homeless through charity verses the objects being sold by the homeless in the streets. While many products are needed for survival, many of the products offered to the homeless by charitable organizations and individual doners do not necessarily address the emotional needs of the homeless people.

Process

The products were initially selected based on five qualities: therapy, sensory stimulation, delightfulness, price and utility. I also wanted to ensure that products purchased in the package are accessible and maximum amount of sensory stimulation is provided per dollar amount spent.

In order to select best-suited products from the initial list that address emotional needs, five participants were given a task to rate each product by its attributes including amount of sensory stimulation, level of therapeutic experience, level of immersion, entertainment value and level of usefulness. Once the survey was complete, a score was calculated for each product based on weighted value assigned to each attribute.

Using the survey result, a final selection of products was purchased from eBay and Aliexpress. Upon receiving the products, products were taken out of the original packaging (e.g. plastic blister packs and clamshell packs) to avoid requirement of additional tools and to deter people from reselling them. Once the products were placed into five cardboard boxes, they was handed out to people in Skid Row of Los Angeles. Photos were not taken publicly to respect the privacy of people experiencing homeless.

Contribution