即時手型辨識系統及其於家電控制之應用
摘要
近年來，手型辨識在人機互動的相關研究中，吸引了各個領域的學者投入，而常見的應用有遊戲控制、機械手臂操作、機器人控制及家電控制等等，其簡單既直覺化的操作方式取代了傳統遙控器的使用。 本論文提出一種基於影像方式之即時手型辨識系統，並將系統應用於遠端控制家電。實作的方法先對攝影機讀到的影像做前處理和膚色擷取後，留下手臂資訊，再用切割手臂的演算法將手掌擷取出來，並利用兩種特徵來描述手型，一是手掌輪廓上每一點到手掌質心的距離曲線；二是利用快速傅立葉將該距離曲線轉換所得到的頻率域特徵。最後，將此兩種特徵搭配KNN和決策樹的辨識方法，能夠準確地辨識十一種手型，而利用這些手型和左右揮動的動作組合，轉換成控制家電的指令，藉由無線傳輸傳到我們自製的環境控制器，發射紅外線訊號以達到控制家電之目的。我們的系統目前成功測詴的家電有電視、收音機和電扇等。 我們請了八位受詴者使用手型辨識系統，在實驗中，並不限制使用左右手，且不需要穿戴任何辨識物或手套，也沒有穿著長短袖的限制。第一個實驗僅分析對十一種手型的辨識率，而結果有91%正確率。第二個實驗，我們針對三種家電設計了一些操作情境，並記錄受詴者控制一連串家電指令所需花費的時間，而實驗的結果顯示其時間都在使用者可接受的範圍。 除了在電腦上實作我們的系統外，為了使整套系統縮小化和具有可攜性，方便置放於每個家庭中，我們也將系統實現於嵌入式系統數位訊號處理器上，其在光源充足的室內環境下可與在電腦上的版本有極相似的表現。

關鍵詞：人機互動、手型辨識、控制家電

A Real-Time Hand Gesture Recognition System and its Application in Manipulating Household Appliances
Abstract
Recently, in the field of human computer interaction, hand gesture recognition researches have attracted many researchers from many different kinds of domains. The applications of hand gesture recognition vary from game control, robot arm operation, robot control, household appliance control, etc. Due to its convenience and intuitiveness, the hand-gesture-based controller has the potential of replacing a traditional remote control. This thesis presents a real-time image-based hand gesture recognition system. The proposed hand gesture recognition system is applied in household appliances control. The implementation of the proposed hand gesture recognition system is as follows. First of all, we locate the arm region from an image captured by a web camera via several image preprocessing operators and the use of skin information. We then use an arm-cutting algorithm to cut out the hand from the arm region. The hand shape is then represented by two kinds of features. The first kind of features is the so-called “distance curve” or “signature” which is a 1-D functional representation of the hand shape. It is formed by plotting the distance from every point on the hand boundary to its center of mass. The second kind of features is the frequency domain parameters of the distance curve transformed by FFT. Finally, the KNN incorporated with a decision tree is adopted to recognize 11 hand gestures based on these two kinds of features. A combination of 11 hand gestures and 2 waving motions will be used as commands to control several household appliances. The recognized commands are wirelessly transmitted to an environment control unit which can translate the received commands to infrared signals and the signals are then pointed to the household appliance to be controlled. Our system has been successfully tested with a TV, a radio and a fan. Eight subjects were invited to test the proposed hand gesture recognition VI system. In the experiments, users can either use their right hands or left hands without the need of wearing any accessory such as a hand glove. In addition, there is no restriction on wearing long or short sleeves. The first experiment analyzed the successful recognition rate of the 11 hand gestures and the recognition result came out with 91% successful rate. In second experiment, we test the time performance of the proposed system by testing some scenarios for controlling three household appliances. The performance was tested based on the time interval which a subject took to finish a sequence of commands according to each designed scenario. The results showed that the time performance was acceptable. To make our system small and portable, we implement the proposed system on a DSP platform. The DSP-based system could achieve as similar performance as the PC-based system under an appropriate lighting condition.

Keywords: Human Computer Interaction, hand gesture recognition, household appliance control.