基於自我組織特徵映射圖之人臉表情辨識
摘 要
表情在日常生活中扮演著很重要的角色，是一種非語言的溝通方式，
因此表情辨識成為許多專家學者在研究發展的議題。本論文主要是發展一
套自動化的表情辨識系統，可透過擷取數位攝影機的影像，自動化地偵測
人臉、擷取特徵到表情辨識。藉由人臉偵測、雙眼偵測、特徵區域的概念、
特徵點的選取與光流追蹤的方法，再加入有限狀態機的機制，可以有效率
且快速地組成一套自動化的表情辨識系統。
本系統採用偵測雙眼來精確的定位人臉特徵區域，提出改良式的自我
組織特徵映射圖演算法，能自動化且即時地做臉部特徵點的追蹤與選取。
並採用兩階段鄰域機制及相關係數光流追蹤法，提供快速地人臉特徵點的
追蹤方式，藉由這些特徵點在臉部的移動判別人臉的表情。根據以上的方
法，建構出本論文的表情辨識系統，在各個表情資料庫都呈現相當好的成
效。最後，再結合有限狀態機的機制，將連續的序列影像自動化分割出許
多表情序列影像段，讓系統可以透過數位攝影機即時判別使用者的表情動
作，以達到自動化即時表情辨識系統。
關鍵字：表情辨識、特徵區域、特徵擷取、眼睛偵測、自我組織特徵映射
圖、光流追蹤。

 

 

A SOM-based Facial Expression Recognition System
ABSTRACT
Visual communication is very important in daily lives for humans as social
beings. Especially, facial expressions can reveal lots of information without the
use of a word. Automatic facial expression recognition systems can be applied to
many practical applications such as human-computer interaction,
stress-monitoring systems, low-bandwidth videoconferencing, human behavior
analysis, etc. Thus in recent years, the research of developing automatic facial
expression recognition systems has attracted a lot of attention from varied fields.
The goal of this thesis is to develop an automatic facial expression recognition
system which can automatically detect human faces, extract features, and
recognize facial expressions. The inputs to the proposed automatic facial
expression recognition algorithm are a sequence of images since dynamic
images can provide more information about facial expressions than a single
static image.
After a human face is detected, the system first detects eyes and then
accurately locates the regions of face features. The movements of facial points
(eyebrows, eyes, and mouth) have a strong relation to the information about the
shown facial expression; however, the extraction of facial features sometimes is
a very challenging task. A modified self-organizing feature map algorithm is
developed to automatically and effectively extract feature points. Then we adopt
a two-stage neighborhood-correlation optical flow tracking algorithm to track
the human feature points. The optical flow information of the feature points is
used for the facial expression recognition. Most importantly, a segmentation
method based on a finite state machine is proposed to automatically segment a
video stream into units of facial expressions. Each segmented unit is then input
to the recognition module to decide which facial expression exists in the
corresponding unit. Experiments were conducted to test the performance of the
proposed facial recognition system.
Keywords: facial expression recognition, facial feature region, facial feature
extraction, eye detection, self-organizing feature map, optical flow tracking.