When Computer Vision Technology Meets Pupils’ Learning - Developing Marine Interactive Multimedia Game Materials

Ying-Chieh Liu, Su-Ju Lu, Ying-Yu Chen, and Ting-Shiang Shiu


Game-based learning, Physically interactive game, Augmented Reality (AR), image detection technology, marine education in primary school, story-telling activities


This research investigates the development of story-telling curriculums with the use of video camera-based motion capture technology. Also, an initial evaluation is conducted. For those that are less attractive but fundamentally important learning materials, the challenges are to enhance learners’ motivation and to improve learning outcomes. Thereby, to construct an innovative learning approach, an attempt was made to integrate story-telling curriculums with computer vision technology. A minimal amount of equipments are required to develop the learning materials, including a personal computer, a webcam, and a projector. Elements of learning are grasped during the curriculum activities with two common applications using camera-based motion capture technology. The two applications are: (1) physically interactive game that utilize body movements of users to interact with the objects on the screen, and (2) main characters of the story are augmented with a 3D video objects (e.g., fishes) through a signal captured from frame-based images (called marker) to present an enhanced visualization on the screen.

An experimental teaching was conducted with 69 pupils aged 7-8 years old. The results showed that the majority are positively motivated by this type of learning activities. Moreover, a comparison of pre-and-post test was made, where a difference (p<0.001) was found. This paper explores innovative teachings that incorporate storytelling activities with the use of computer vision technology. Further enhancement includes investigating pupils’ learning attitude, motivation, and performance, as well as investigating related technology in effective use.

Important Links:

Go Back