以Kinect深度影像攝影機為基礎之嬰幼兒音樂互動系統
摘要
隨著資訊化時代來臨,社會競爭力越來越大,早期嬰幼兒教育日益受到重視。許多研究指出嬰兒天生並非「白板」一塊,而是有我們未知的各種心理及學習能力,而且嬰幼兒時期心理發展的水平和質量對之後的兒童期、青年期乃至一生都俱有重要的影響。然而嬰兒之聽覺系統在出生時相較於聽覺系統就已發展成熟,所以音樂對於嬰幼兒時期被視為重要的學習教材。研究指出,孩童若越早接觸音樂,能夠幫助孩童提升其語言發展、增強創新能力,並且改善身體的協調性。因此本論文就以工程角度及音樂為基礎,結合 Kinect 深度影像攝影機,提出一個以往不同的互動方式,並應用在嬰兒育樂教育上。 而科技的進步,市面上出現許多跟嬰幼兒互動之電子玩具,這些電子玩具,由於設計上的限制和成本的考量,不免單調且缺乏創造性。因此,本論文提出了一個為嬰幼兒設計的音樂互動系統,結合空間概念、物體運動捕捉及粒子濾波器為基礎的物體追蹤,建構出五種根據不同嬰幼兒發展時期所設計的互動模式:音樂鈴模式(Music Ring Mode)、鋼琴彈奏模式(Piano Mode)、聲音組合模式(Sound Combination Mode)、環境探索模式(Environmental Exploration Mode)及音樂變化模式(Music variation Mode)。藉由以上互動模式,本論文建構出一個有探索、學習、育樂環境之音樂互動環境,讓嬰兒能夠伴隨音樂的旋律,探索世界、認識自我,奠定往後心理及生理發展的基礎。最後,實驗將針對系統準確率作驗證及使用經驗進行相關探討與提供改進的方法。

關鍵字:音樂學習、嬰兒教育、粒子濾波器、人機界面、Kinect

 

 

A Kinect-based Interactive System for Children
Abstract
With the era of information technology, social competitiveness is higher and higher, early childhood education is getting more attention gradually. Many studies have pointed out that the infant is not a "whiteboard", but has a variety of psychological and learning ability that we don’t know, and the level and quality of infant mental development has great impact on childhood, adolescence, even the future of life. However, the auditory system of the infant at birth compared to other sensory system had been developed, so the music is regarded as an important learning material for children. Many researches indicate that to receive music education early can help children enhance their language development and enhance its ability to innovate or improve the coordination of the body. Therefore, this thesis proposed a different interact compared to the past from engineering and music perspective by using Kinect and apply in infant education. With advances in technology, there appear many electronic toys to interact with infants and young children, these electronic toys are inevitably monotonous and lack of creativity due to design constraints and cost considerations. Therefore, this thesis proposed a music interactive system for children, combined with the concept of space, motion capture and particle-filter based object tracking, that consists of five different interactive modes: Music ring mode, Piano mode, Sound combination mode, Environmental exploration mode and Music variation mode. By these interactive modes, this system construct a music interactive entertainment with exploratory, learning, interactive that the baby can able to accompany the melody, to explore the world, self-understanding, to lay the foundation of psychophysiology development. Finally, the experiment will verify the system accuracy and using experience of user to discuss and provide improved methods.

Keywords : music learning, infant education, particle filter, human-computer interface, kinect