PROJECTS

[Reference /引用はこちら]
Xiang Zhang, Kaori Ikematsu, Kunihiro Kato, and Yuta Sugiura. 2022. ReflecTouch: Detecting Grasp Posture of Smartphone Using Corneal Reflection Images. In CHI Conference on Human Factors in Computing Systems (CHI ’22). Association for Computing Machinery, New York, NY, USA, Article 289, 1–8. [DOI][PDF][30s Video Preview] [Video Figure][Presentation Video]

スマートフォンをユーザがどのように把持しているのかを測定することで，把持姿勢に合わせた画面表示や操作ボタン位置の自動切り替えが可能となり，ユーザに合わせたインタラクションが可能になる．本研究では，スマートフォンを操作する際に角膜にスマートフォンの画面の光が反射して写る像を内蔵インカメラで撮影することで把持姿勢を推定する手法を提案する．角膜に反射したスマートフォンの像ではユーザが画面に指を置いている部分が影となって欠けるため，把持姿勢の推定が可能となる．本稿では，提案手法の精度を検証する実験を行った．

By sensing how a user is holding a smartphone, adaptive user interfaces are possible such as those that automatically switch the displayed content and position of graphical user interface (GUI) components following how the phone is being held. We propose ReflecTouch, a novel method for detecting how a smartphone is being held by capturing images of the smartphone screen reflected on the cornea with a built-in front camera. In these images, the areas where the user places their fingers on the screen appear as shadows, which makes it possible to estimate the grasp posture. Since most smartphones have a front camera, this method can be used regardless of the device model; in addition, no additional sensor or hardware is required. We conducted data collection experiments to verify the classification accuracy of the proposed method for six different grasp postures, and the accuracy was 85%.