The SPLIT Experience


When a person gaze towards something or someone, naturally the eye’s focus would constantly jump back and forth, scanning and following contours while identifying various information of the subject. These quick eye movements are call Saccades. Although our brain interprets these saccades to be seamless, but biologically we are blind between these saccade points.

This experiment is the next iteration of my previous project “Extended Sight”, it takes on the metaphors of the saccade blindness in conjunction with jump cut techniques to display different angles of one scene in films. This device consists of two parts, a glasses mount eye tracking camera which controls two semi-circular acrylic armature driven by two servos that rotate in x and y axis. The main purpose of this structure is to mount a viewing camera, so the camera can rotate 180 degrees around a object that can be placed on a central platform. Due to the scale constraints, he maximum size that this specific prototype will allow to revolve around is 6in x 8.5in. The display screen of this viewing camera is mounted on the right eye of the glasses. Therefore, when the user is wearing this glasses, the left eye is used for controlling the viewing camera, and the right eye is used for looking at the display of that camera. Due to my technological constraints of the type of viewing screen, the only way to be able to foveate on the lcd display screen while the left eye is being tracked, is by rotating the screen in the same direction the pupil is moving. This requires an additional two axis bracket driven by two feather weight servos to turn the display towards the same direction that the eyes is pointing at.

Through this device, the eyes are no longer used only for navigation and identification, but it adds a new exaptative function of control. It takes the viewer out of their normal perspective and see things in a whole new way.


Progressively reveal layers of capabilities

If physical objects can provide abilities to control and navigate through algorithmic systems, what are ways to hide and reveal features to help isolate it’s functions and capabilities? This experiment investigates how complex system can be hidden within a simple form and how it can progressively reveal layers of capabilities through user interaction. When certain parts are initiated, it physically animates and transforms to display or hide these features. This system provide constrains oppose to layout out all the features and controls on a single display.

Key Traits:

- The goal is to design a system ithat s complex, but initially presents itself simple. so different functions are revealed as needed.

- Initial interface presented in it’s raw form

- Progressively revealing layers of capabilities

- Isolate functions and introduce new feature

- If with other objects, it will change it’s form to interact with other object.

- Ideally when every side of this object can open, it can provide a perpetual sense of discovery during use.

Left object is in it’s initial activated state and the right object in it’s natural raw state

Another image of the object in it’s initial activated state

object of this experiment in it’s full display mode after all switches have been initiated

During the making phase, I’ve tested on several raw geometric forms and ended up settling on a polyhedron. This is the first prototyping phase, so the quick mockup is made with cardboards.

Placing hall effect sensors inside the prototype

Because for this experiment I’m using magnet sensors, I had to rig this glove with magnets on all fingertips to get a sense of it’s interaction. This image shows the glove flipped inside out to hotglue the magnet in for future removal.

This is a initial sketch of how this object may interact with other objects to form a larger entity.

These are some initial sketches about this idea and how a polygon may change forms.


Tactile Organic Mutational Interface

This is the next iteration of my original mutation interface except it isn’t driven by accelerometer or servos. The form of this tactile interface is purely organic. Which makes it look blooby so it fits nicely in your hand. It starts out neutral shape like a sphere or ellipse, and as the user squeeze, push, twist and grip the device, it changes it’s form to provide new features. These features are yet to be defined. The form also protrude itself with bumps at various points as indicators to notify new affordances to the user. Since the goal of this device is to operate purely through touch without the need to see it, it is perfect for controls or perform tasks when the user’s sights or even other senses are occupied. For example in a driving scenario, it maybe detrimental to take your eyes off the road, so the user can use this device without having to look at it.


Physical Interface Changing Over Time

Temporal Interaction: This experiment investigates how physical interface changes over time can introduce new ways of interaction. Short maybe a couple of hours, long maybe as much as a weeks. I envision it’s form changes on it’s own and also through user interaction. This metaphore for this type of object and user relationship is very similar to crafting and caring for a Japanese Banzai tree. Through the user’s tentative attention to care for and craft this object, it begins to develop personal value through each individual user. Although the interaction part is still very much on the surface level, but this one is very interesting to me. What causes the form to change over time is something I still have to work out. It maybe a set of data from the internet, or even by personal encounters or experiences.

Key Characteristics:

- changes through human interactions

- changes over time on it’s own

- introduce new ways of interaction as the form changes

- slower interface change. over time, in the scale of days.

- changes base on how it’s used, or data coming from somewhere else

- builds a closer relationship with the user through investing time and efforts over a long period of time.

Other Possible Form Iterations

Mutation Interface: Physical Interactive Form Follows Function

Physical Interactive Form Follows Function

Keywords: Organic Mutation, Morphing, Interactive Physical Interface, Cognitive Visual Angle, Form Follows Function Physically

Question: What if the form, function and interface can be changed by a simple gesture of wrist rotation?

Summary: Mutation Interface is an interactive morphing interface inspired by the ideas of organic mutation, morphing, and Cognitive visual angle. It is a speculative experiment that maybe suggestive for a interactive physical interface that can be implemented for consumer electronics. This stand-alone device is driven by a 3-axis accelerometer, controlled by an Arduino board, which triggers three servo motors, one for each axis of rotation. The servo arms are attached with strong skeletal armatures that rotates in all directions without getting in the way of each other. This device is then wrapped in cloth to hide the mechanics within to produce a morphing look and feel from one shape to another, creating multiple and interchangeable surfaces.

this video demonstrates the way this device moves with the 3 axis acceleronmeter

begin prototyping and making it work

cleaned up the wires





Extended Sight + PAMGIS

Extended Sight + PAMGIS
(Physical Animated Motion Graphics Immersive Space)

Keywords: Immersive Physical Animation Data Visualizer, Saccade, Jump cut editing, viewing from different angles, Eye tracking camera control.

Question: What if the human eye sight can be relocated to another part of the body or another part of the world?

There is two parts to this project. Extended Sight explores the idea of moving a person’s eye sight to another location. This can be another location on the body as part of another limb or any other body part, or remotely at another location. This provokes the question if our eye sight is removed from our head and placed somewhere else, can we see from different perspectives? This project can also inform ideas for story telling. If there are multiple amounts of these eye controlled cameras are placed in different countries to tell different narratives happening at the same time, how can the footage and data collected inform a new way of story telling?

PAMGIS (Physical Animated Motion Graphics Immersive Space)
By placing this camera inside an enclosed box with physical graphic and informational elements strategically positioned, this immersive physical data visualization installation designed to recreate the experience of watching a motion graphics animation in a physical space. From a first person point of view in conjunction with eye tracking controlled camera placed in this space, the viewer is given the affordance to navigate through the text and graphical information freely. The space is designed to reverse engineer the way animation is produced in a scripted environment. As the viewer pans around the camera and bringing graphic elements into frame, in the viewfinder the look of the animation will be inspired by the way motion graphics feel as if they are animated on screen.










TRAILS: Super Hero Power Telekinesis

Synchronization, Transformation, Visual Search Selection, Visuospatial function

Summary: This experiment explores the idea of Visuospatial function, Object Transformation, Multiplicity Synchronization, and Visual Scanning. Utilizing the eyetracking technology as a control to analyze eye movement, which the scanning movement triggers the opening of a box. The box is inspired by transforming mechanical parts. This single function box is then duplicated in a mass amount organized as a grid format. As the result, the boxes open in synchronized motion as the eye pan across and looks around.

Question Raised: When a physical transformation in mass quantities is beautifully synchronized to create a patterned trail, what can of new functions and feelings can it inform other than a mesmerizing effect?



This experiment began from using eye tracking and stage use spot light as a metaphore to create an eye-controlled car headlight. At this early stage, I was experimenting while creating something practical rather than speculative and provocative. Here’s a video to demo how the car headlight works to help amplify the clarity of the driver’s view.




Super Hero Power Telekinesis
Taking the idea further and add multiplication of the transformation box idea and eye tracking controlled spot light. The visual effects metaphore this idea is inspired by comes from super heros from comics or movies often have some special power that allows them to manipulate an object with their mind, eyes, or other sensory input. Put that in conjunction with the visual effects language of multiplicity and synchronization, I put together this after effects animation to demo how the installation would work as your eyes move across the peripheral view.



MDP Science Fair Oct 31, 2011

Great successful day at Art Center MDP’s annual science fair. A great variety of topics ranging from sound, touch, urban, language, memory, gaming, gesture, and mine on visual perception. My current working title is “Perceptual Exaptations”.

The three experiments I displayed starting from the right consists of:

Trails – Transformation of the physical world with your gaze.

The middle is Mutational Physical Interface, an accelerometer driven form changing physical interface.

On the very left is Extended Sight + Physical Animated Motion Graphic Immersive Space. The glasses consists of a eyetracker (hacked PS3 IR web cam) and a LCD driven by two servos for the right eye. The camera controlled by the glasses is placed inside a box decorated with dimensional graphics, scripted to allow the information look as if they animate onto screen while the camera pans around.

Week 5: Hacking the PlayStation 3 EyeToy for Eye Tracking

The first time I hacked a PS3 Eye Toy to use as an eye tracker was a year ago, during my concept year here in MDP for a class call New Ecology of Things (NET) taught by Phil Van Allen. The warmup project at the time was called “Useless Networks” where I created a clever but useless helmet called “The 11th Finger“. I learned how to create this device by following the tutorials for creating the Eyewriter. Thanks to the smart people over at the Graffiti Research Lab for their detailed tutorial on their website as well as on Instructables.




To take apart the case, you need to dig in with a flat head screwdriver and pry it open while twisting








Week 4: Prototyping / Learning through making

These are the first iterations of my experimentation and implimenting sketches from the previous week
Tangible Motion Graphics – Idea around Physical Animated Spacial Experience


Through these scale models and experiments, this investigation begins to speculate the distance and perceivable range around the viewer. When this is experienced in the dark, the manipulation to play with fading is very useful. If I continue this investigation, these graphic elements that takes up space will be motorized.


There are countless examples of visual effects in film and even in our daily lives that revolves around the idea of Transformation. Yes, that includes Hollywood hit The Transformers franchise, along with many other science fiction films. This is the first testing of a self reveal opening. Imagine if this container is duplicated on all 6 sides of a cube. Where this cube opens can be independent interfaces that are utilized for different tasks.


Mutation – Cube Duplication

This rough initial prototype investigates the possibility of hidden compartments or flaps to revel additional mass to allow for physical duplication or multiplicity. Visual effects often allow almost the impossible of virtual mass expansion. For example the transportaion (flying cars) in the cartoon The Jetsons, cars would fold up and become a suitcase. Or even in the japanese comic/anime Dragonball Z, the characters can click on a pill shaped device and throw it onto the ground…then poof! it turns into a motorcycle, private hotel, perserved food, or other large devices.


Synchronization – Mechanical Iris study

The idea of synchronization in visual effects or animation always has an mesmerizing effect on the viewer. In this initial experiment as part of my study of simply moving many pieces of shapes at the same time to morph or expose. This can be embed into container openings or duplication of many of these containers to create a physical interface for access and various applications.