Esen K. Tütüncü
With more and more intelligent systems surrounding us, many design decisions and system architectures that facilitate the flow of information and the scope of interaction have become crucial for the sustainability and the longevity of the Human Computer Interaction.
Although it is clear that the interaction can be done via direct input(touch screen, camera, microphone) making use of the built-in sensors to extract implicit data from the user is a promising step in HCI that could potentially change the way we use our phones. The purpose of this study is to investigate possible ways in which sensors embedded into mobile devices can be used to assess cognitive load or emotional state that dynamically modulate the device interface in order to allow the user to better engage and interact.
The hypothesis is that we can use the built-smartphone camera to measure Pulse Rate Variability (PRV), and derive the users cognitive load based on it. Furthermore, this would allow us to not only measure it real-time but to dynamically modulate the interface based on it to control the induced cognitive load.
You can download the thesis here
Figure: The flash illuminates the skin that is placed over the camera, which allows the detection of blood flow and its intervals. Overview of the setup: After placing the index finger on the smartphone camera, the changes of the blood flow to the fingertip resulting into different light reflections of the skin is recorded by the video frames.
originally from: Bánhalmi, A. et al. Analysis of a pulse rate variability measurement using a smartphone camera. Journal of healthcare engineering 2018.