• home
• about
• blog
• portfolio
• contact

What are Haptic Interfaces?

This paper provides an account of the technical and biological requirements to create and sense force feedback for haptic technologies. I also cover how the research has created multi-modal interfaces for consumer products. Looking into the future of haptics I will report about the current advances for force feedback in technologies used to remote medical surgeries.

Introduction

The paper which follows covers the reading I have conducted to report on the nature of haptic interfaces for human machine interaction and the current state of this technology for consumer centric products. So what are haptic interfaces and how do the work?

The word haptic is derived from the Greek word ‘haphe’, which refers to the sense of touch. In terms of technology haptic interfaces are devices implemented to in a manner which uses an organisms tactile sense. This method of force feedback evokes a nerve impulse through receptors under the skin sending messages back to the brain about the current or changing state of a system. This is enabled through the engineering of devices to produce vibrations, motions or forces which the user will sense.

This is all made possible by the complexity of the our largest organ, the skin.

Skin Anatomy

The modern man Homo sapiens, is the product of complex evolution producing organs and systems required to support life and sense the surrounding environment. The species commonly accepted origin is 200,000 years ago identified through African fossil records. The area interest for interfaces with haptic feedback is our species skin.

The skin is the largest organ of any human. Its primary task is to provide us protection from microbial organisms and general detritus located around us. It also coincidentally delivers a convenient container for all the liquids and organs required to support our lives. However the important aspect of the skin required to sense our world around is located with the layers of cells which build up the skin.

On the surface of the skin is a layer of old dead skin cells which creates a protective layer above the layer below of living cells which forms the Epidermis. The Epidermis which has a depth of 0.4 to 1.6 mm is an important layer containing among many the Langerhans cells responsible for immune responses and the production of melanin which gives colouring in response to genetic traits or tanning resulting from exposure to Ultra Violet light.

Below is the Dermis which is typically five to seven times the thickness of the Epidermis. The Dermis the most active region of the skin contains connective membranes filled with blood vessels, lymphatic systems, nerve fibres and nerve receptors. The Dermis creates the elastic support system for the nerve endings required by humans to sense the haptic feedback by transmitting electrical impulses to the brain along the nerve fibres. The bundling of nerves for sending messages around an organisms body is analogous to fibre optical cables laid around the world for information interchange required by the Internet. Finally below the Dermis is the Hypodermis which bonds the skins to the muscles below.

Touch User Interfaces

Touch User Interfaces or TUIs are Graphical User Interfaces using a computer screen with touch screen functionality as a combined input and output devices to capture and report changes to the system. A real world example of such as device would be a point of sale devices used by cashiers in large chains of shops such as super markets. The technology is now being applied to more and more consumer products and includes mobile phones and small hand held computers.

Touch User Interfaces endeavour to produce the most natural interface between person and machine by recognising one or more different gestures produced by the computer user. The most simple of a gesture would be a single click then a double click to build in complexity and expressing a different command. Gestures for a non touch interface could be captured using a mouse, tap device using buttons or pressure sensing to record focus and movement of the pointer of the Graphical User Interface. In the point of sale example above the screen will record the location as the user presses an area of the screen and relate this pressure to the position of the display targets.

Tactile Interfaces

Tactile interfaces are uniquely different to your typical graphically based interfaces because they are intended to reproduce the physical attributes of an objects. The information represented can include the objects shape, texture and temperature. Chouvardas et al report in their paper ‘Tactile displays: Overview and recent advances’ stated that the following:

“Tactile displays have been proposed as an interface in Virtual Environment (VE)–Virtual Reality (VR) applications, as feedback in teleoperation, as a complement or substitution of the visual presentation of information, and as tactile communication in mobile environments.” [1].

Chouvardas et al continue to report that the advances into tactile displays is critical to aid computer users with disabilities either vision or hearing the opportunity for transmission of information through an alternative channel. Information about the object of interest to the users is transferred by:

“Exploiting the modalities of the skin’s sensors” [1].

This uses the receptors of the nerve fibres to transfer information to the brain delivering the haptic response to the changes in the system presented through the following stimuli:

• Static pressures or vibration (mechanical energy),
• Electric field,
• Thermal flow created using temperature gradients.

A notable area to employ tactile interfaces is in the field of virtual reality and telepresence to interact with a machine in a remote location.

Virtual Reality

Virtual Reality (VR) is a special technology which allows a person to interact with a computer simulated environment. Virtual environments are simulations of factual or fiction places such as the beaches of Normandy or an alien world.

The practice of modelling environments have given rise to the immersive story driving worlds seen in modern computer games for computer and games consoles. This sort of virtual reality uses very simple input and output devices to present the world to the user with computer monitors, keyboards and mouse or games pads.

More advanced virtual reality experiences may include multi-modal interfaces such as wire-gloves which feel the movement of the users fingers responding with force feedback, omnidirectional treadmills and head mounted displays. Wire-gloves are also know as cyber-gloves or data-gloves. The most commonly recognised implementation of a wire-gloves for human computer interface was for the futuristic interface used by Tom Cruise in Steve Spielberg’s science fiction film Minority Report. However the more common application of wire-gloves is for providing haptic feedback to computer users running virtual reality simulations to sensing the virtual artefacts in the simulation by applying forces when the users is closing their hand around the object such as a cup.

Consumer Applications of Haptic Interfaces

Applications for haptic technologies are far and wide. Haptics can be applied to computer games, virtual reality, medicine, robotics and much more. To briefly recap, haptic interfaces are used to provide force feedback about the current state of systems.

Haptic technology for force feedback in computer games has been available for over 10 years since the launch of the iconic Nintendo 64 console system. The Nintendo 64 console, named after its 64 bit architecture for the graphics processing unit used to render the graphics. Launched first in 29th September 1996 for the Japanese market the console was highly successful.

The newly designed console controller provided connectivity for memory cards and Rumble Pak in its base. The rumble pack was a post console launch extension to the controller which in selected situations by the game programmer such as when firing a weapon or receiving damage, would provide force feedback vibrations to immerse the player in the game. The application of force feedback for computer games had previously been limited to the arcade machines. To date the legacy of the Rumble Pak as forced other console manufactures to controllers with haptic technologies to provide owners of later generations of consoles the immersive games experience at home pioneered by Nintendo.

Currently there are in excess 3.3 Billion mobile phones available to a 6.6 Billion people on our planet. Considering that there is now roughly one mobile phone for every two people on planet Earth the opportunity to deliver a haptic interface to consumers is much greater than the number of owners of games console, as mobile phones are increasingly considered a necessity rather than luxury items. Dong-Soo Kwon has described in his paper[3] recent changes in mobile phones to include haptic technologies as the following:

“Recently, the mobile device industry has embraced haptic feedback in the form of simple vibrotactile devices based on eccentric motors. These are now embedded into the majority of PDAs and cellular phones, and many game controllers. But the feedback provided by these devices is crude..” [3].

It is true even with advances in haptic technology developments and the application of this research has been slow to permeate the mobile phone market. The current theme of the most high order handsets include touch screens such as the Apple iPhone handset which has a 3.5 inch (diagonal) screen with four buttons. This includes two volume controls, home button and combination handset lock / power button.

A large number of criticisms aimed at the iPhone and similar devices with large screens and few buttons opting instead for onscreen keyboards provide little feedback with shinny or slippery screens. This is no longer as true with the launch of the newest Blackberry mobile phone device from Research In Motion also known as RIM.

The new Blackberry device called the Storm has haptic characteristics. The Blackberry Storm again like the iPhone and other devices has a large screen, however the Storm has a large button suspended above the screen.

“The Storm, however, has a touchscreen that effectively floats a fraction of a millimetre above a whole bank of sensors so when you scroll down to an icon you want you just press down on the screen in the right place. It gives the sort of physical click that any BlackBerry user will recognise from the trackball on previous devices. Its simple but remarkably effective”. [4].

According to what I have read and my personal experience when I used the Blackberry Storm during a demonstration in the Loughborough Vodaphone store on its launch weekend it seemed initially strange. But after a minute of use it became apparent that the interface was error resistant and delivered and experience similar to using an device with real buttons. In fact I forgot that it was a touch screen device unlike using my iPhone which uses the touch screen in the same fashion as the screen of a PDA device.

Further Applications of Haptic Interfaces for Medicine

Medicine I my mind seems an extremely important application of haptic and robotic technologies to augment the surgeons skills. Major advantages of robotic surgery are precision, miniaturisation, smaller incisions, decreased blood loss, less pain, and quicker healing time. Further advantages are articulation beyond normal manipulation and three-dimensional magnification.

The use of robotics in surgery with multiple cameras for three dimensional imager is referred to as stereotaxy. According to my reading stereotaxy or stereotactic surgery is a minimally-invasive form of surgical intervention which makes use of a three-dimensional co-ordinates system to locate small targets inside the body. This technology is used to perform action such as biopsy, lesion, injection, stimulation, implantation and radio-surgery for cancer treatments.

The most notable implementation of such a technology has been reported by Alexander D. Greer, Perry M. Newhook, and Garnette R. Sutherland in 2008 for the work on the neuroArm a Medical Robot (MR) and the hardware for force feedback during surgical operations. The photo across to the right shows the workstation used by surgeons to perform operations. The apparatus could be better described as a telesurgical device which allows the user to view the subject on a series of monitors with a several views and three dimensional imagery to deliver a clear picture of the patients state. With an internet connection the surgeon could in theory perform the surgery and great distance or from the next room in this example.

The use of haptics in this example is to provide force feedback to enable to operator to feel remotely the changes in tissue density as they are operating. The excerpt below from the paper explains the conclusions for using haptic feedback.

“The haptic feedback has proven invaluable, demonstrated by an operator’s significant difficulty with soft tissue manipulation when haptic forces are deactivated.” [5].

I found the paper at times to be extremely in depth. There has been much time and effort spent on this project as the neuroArm robotic device was first developed in 1999. I do feel that this is a fantastic development to medicine and application of haptic technologies. Though I am not sure I am prepared to go under the robotic knife myself.

Summary

It is my personal belief that if science fiction is by any measure the perfect way to see the future of computer developments and devices for human interaction. The continued implementation of haptic and tactile devices to aid people with disabilities we continue to advance and I feel all will benefit as we increasing design products with Universal Design central to the development process. Much of haptic technology is currently limited to consumers however I believe that future generations of mobile phone devices and games console accessories will continue to implement more haptic feedback into these product ranges. Perhaps also into desktop computer and laptop with the increasing application of touch user interfaces for user input.

It is my conclusion that however impressive haptic technology is for consumers it is still embryonic when compared to full fledged VR simulations.

Bibliography

1. V.G. Chouvardas, A.N. Miliou, M.K. Hatalis, 2007, Tactile display: Overview and recent advances.
2. Alexander D. Greer, M. Newhook, and Garnette R. Sutherland, Human-Machine Interface for Robotic Surgery and Stereotaxy.
3. Dong-Soo Kwon, 2007, Will Haptics Technology Be Used in Mobile Devices?: A Historical Review of Haptics Technology and Its Potential Applications in Multi-modal Interfaces.
4. Richard Wray, 2008, Guardian, Hands on with the BlackBerry Storm. 
http://www.guardian.co.uk/technology/blog/2008/oct/08/blackberrystormreview
5. Alexander D. Greer, Perry M. Newhook, and Garnette R. Sutherland, 2008, Human-Machine Interface for Robotic Surgery and Stereotaxy.

This work is licenced under a Creative Commons Licence