Behind the mask

While undoubtedly good fun, the ‘smile’ calibrator carries a serious technological message: people are surrounded by all types of machines and devices that provide many kinds of information and services. Omron believes that it is important to create machines that are capable of adapting themselves to individual users, thus providing information services and interfaces that are optimised for each person. To make this possible, machines must be able to recognise each person and make judgments about what that particular individual needs.

When you meet someone for the first time, your first impression is generally from looking at the person's face. By looking at the person's face, you can immediately identify whether the person is male or female, the person's approximate age, and so on. If you have met that person before, you would associate the person's face with a name, and would probably be able to guess the person's overall physical health or even the person's mood or feelings.

Face vision

In other words, the human face is a rich source of information that machines can use to understand people visually in much the same way that humans understand each other. Omron was quick to focus on the immense potential of face recognition, and as early as 1995 began the development of its revolutionary face sensing technology called "OKAO Vision" (which means "face vision" in Japanese).

The latest enhancement is capable of estimating seven different facial expressions – happiness, surprise, fear, disgust, anger, sadness, and neutral - all in real time. The technology offers interesting possibilities in a variety of applications. If incorporated into a communication robot, for instance, the robot could react to a user's facial expressions. The technology is also ideal for image search engines, or interactive games that respond to the facial expressions of players.

Omron's Facial Expression Estimation technology combines proprietary 3D model-fitting technology and a statistical classification method, which are based on a massive database of facial images accumulated over many years. The technology has until now been used primarily to recognize individual faces, but by enhancing these technologies it has become possible to also estimate facial expressions.

OKAO Vision can recognize a face even if the image is inclined or upside down, monochromatic or color, or blurred. It can even accurately detect a face of someone wearing a hat or eyeglasses, regardless of age, or even the face of someone laughing.

Facial image sensing technology can detect and extract a wide range of information from facial images, and is now one of Omron's core technologies. The developments to date have enabled such functions as Face Detection, Face Recognition, Smile Degree Estimation, Gaze and Blink Estimation, Eye Open-Close Estimation, and Age and Gender Estimation. "OKAO Vision" technology is also used in the auto-focus function of digital cameras and smartphones, as well as the skin beautification function of printers, and image management for personal computers.

Tamper prevention

Developments like OKAO are bringing new ways for users to interact with systems. OKAO could boost safety and prevent tampering for example, if the person approaching a system can be identified from a distance and verified as someone allowed to work on the machine. OKAO could go further still: today an HMI must be touchable, which means an engineer or operator must be close by and passwords must be entered manually.

Remote maintenance has been a theme for many years, starting with the sending of SMS text messages or emails and today allowing direct internet access. But on the ground, maintenance staff must still walk to the fault location, collect and fit a replacement part and finally go back to the HMI in order to test and restart the machine. One way OKAO in the HMI might make this much easier is by recognising the person doing the work, allowing them access and responding to their remote hand gestures from some distance away to avoid the necessity of walking back and forth.

In addition, by significantly reducing the amount of information required for detection, OKAO Vision can be implemented in a single-chip configuration. This has made it possible to integrate face sensing technology into compact mobile terminals, whereas conventional technologies require a high-performance computer.