Title: Modelling and Applying the Visuo-Haptic Illusion in Grasping

When and where: Thursday, May 21; ISIR; AGATE Room,

Abstract: Tangible interactions involve multiple sensory cues, enabling the accurate perception of object properties, such as size. Research has shown, however, that if we decouple these cues (for example, by altering the visual cue), then the resulting discrepancies present new opportunities for interactions. Perception over time though, not only relies on momentary sensory cues, but also on a priori beliefs about the object, implying a continuing update cycle. This cycle is poorly understood and its impact on interaction remains unknown. We study visuo-haptic perception of size over time and (a) reveal how perception drifts, (b) examine the effects of visual priming and dead-reckoning, and (c) present a model of visuo-haptic perception as a cyclical, self-adjusting system. Besides, the biomechanical features of grasping also create opportunities for illusory approaches. Inspired by the observation that humans naturally adjust finger configurations based on object size and weight, we present ThumbShift, a novel haptic controller that physically moves and rotates the user’s thumb to render subtle shifts in finger collaboration and affect whole-hand grasp perception. Results of our user studies show perceived weight shifts significantly—by approximately 19%—in magnitude estimation tasks. We also report on the influence of mass centre position which extends the weight perception changing ability to about 56%, and how finger force distribution works in altering users’ perception.

Bio: Jian Zhang is a PhD candidate of Human-Computer Interaction at the University of Melbourne (one year before graduation), supervised by Dr. Wafa Johal and Dr. Jarrod Knibbe. He received the master’s degree of artificial intelligence from Tokyo Institute of Technology, Japan and Bachelor’s degree of Mechanical Engineering from Shanghai Jiao Tong University, China. His current research interests include the computational modelling of visuo-haptic illusions, biomechanics in hand manipulation, motor adaptation and designing haptic feedback devices. He’s open to collaboration opportunities on theoretical and modelling studies, as well as prototyping of novel devices. He’s also actively seeking potential postdoctoral positions.