We report a diagnostic tool to distinguish erratic from other eye movements. We explain its properties and successfully apply it to the illustrative experiment by Melnik et al. . The tool is based upon entropy measurement of a stochastic model, as developed in complexity theory.
Entropy eye, Memory, Model, Saccades, Stress, Visual perception, Vision training
By deduction from complexity of (behavioral) models, we develop an entropic computational tool to distinguish erroneous/redundant eye movements from task relevant eye movements.
Our first subject in this matter is: Why?
Eye movements give a continuous readout of internal neural decision-making processes and reflect decisiontask requirements  of human observers in traffic and at home. The societal impact for – say visually heavy loaded professions such as racing cyclists – is clarified by Fracasso et al. . Eye blinks darken our view about 10% of our lifetime and attending to our own thoughts also ignores the visual world [4-7]. Neurological diseases may affect patients’ eye movements. Erratic eye movements appear with growing age [8,9] while pursuing slows down . Also, stress causes erroneous eye movements, even in healthy athletes as well as in professionally calm officials like judges in court [11,12].
The importance of evaluating eye movements is such that eye tracking is currently used as a health biomarker. In this regard, a USA patent number 20190239790 to monitor eye movements for health assessment is pending .
Segers et al. successfully designed a vision training to bring overacting eyes in quiet mode, to enhance perception and reaction times of athletes  and elderly (report in progress ). Vision training intermittently shuts off free viewing, similar to eye blinks. We noticed a variety of set-points with varying effects upon enhancing visual stamina. In this paper, we report a new method – a computational tool – to decide upon what the eyes are doing. It is an instrument to discern whether eye movements are for help because of loss of memory, or for other causes such as seeking information. The success of the training method is enabled by brains’ shared cognitive control mechanism between reading and other voluntary saccadic tasks  and affects early saccade averaging by top-down processes [17-19] or even stronger , or ‘unseen’ stimuli because of a very fast shutter  which speeds up reaction times.
The eye’s retina is an outgrowth of the fetus’ neural plate , so the retina still is brain, but in an exogenous place. Some vision processes are partly performed in the retina itself. The optic nerve transmits the signal from the retina to visual centers in the brain and these pass them on to the nerves that control the eye muscles. To get a detailed perspective on this architectural issue new results are available . Neurons have a quantized energy nature, this necessitates saccades, the rapid eye movements. The fastest possible reactions of the human body are these saccades, ranging from 10 to 300 degrees per second (deg/sec) between places where the eye rests (called fixation points). When scanning a scene, we do not get a continuous, smooth stream; rather, we unconsciously quantize our view as a series of separate images (because the eye pauses at fixation points). The separation comes from a small period of blindness when the eyes move (during the saccade between 20 to 100 milliseconds (ms)), but we perceive clear images by mental reconstruction.
To remember information, people unconsciously move the eyes in the same pattern over and over again, even when looking ‘inside’ in memory with eyes ‘in the void’, as if they are rehearsing: moving their eyes in the same pattern as when they first saw the objects. And we do this more often when we’re older [24,25], although older adults perform this strategy almost as well as younger adults. Vision training optimizes this by reducing the oversampling , because oversampling hampers perception . The brain then looses ‘sight’, as is well known from familiar stressful situations such as in athletic games, or traffic.
In the absence of stress, eye movements naturally tend to reduce uncertainty in perception [28-31]. This recurrence reduces with vision training and is almost absent in experts, see for instance (table 2 in ). Though there are individual differences in tolerance of uncertainty by aiming at reducing such redundant updates via saccades . The neural system is so versatile  that it enables to adopt different representations of memories of falls and accidents, if they are currently still feared . van Moorselaar et al. , experimentally found that memory may change(!), depending on whether deemed relevant now, in the future, or not at all. McSorly discussed the search to diminish uncertainty views [30,31] because -as said- reducing visual uncertainty causes recurrent saccades. For instance, Gotardi et al.  discovered a difference between eye movements of drivers with and without anxiety. This concludes the motivation and application of this research.
But, how are futile eye movements distinguished from purposeful eye movements?
Melnik et al.  conducted a ground-breaking study to quantify the trade-off between the use of eye movements for working-memory or for using the outside world as a working memory (during purposeful actions). This trade-off is a basic trait of vision training with shutter glasses: the eyes become intermittently blinded, so they are forced to switch between internal memory and ‘the world’, i.e. external memory. In Melnik et al. study , was the cost for a new sample of visual information that participants had to “pay” a short visual delay. The idea of ‘payment’ by delay or a time penalty is also used in . Participants’ use of internal working memory increased with the waiting time for saccades. This result specifically supports the use of shutter glasses to decrease training erratic movements. For example, eyes of non-expert batters behave wildly compared to experienced batters . There is no visual attention at endpoints of saccades, so they should be optimized . Experts’ reduction of saccades is also observed in Kim et al. study .
Older adults use this update strategy when remembering becomes difficult, or when the task becomes too difficult on its own. As if older adults are using their eyes to create a ‘motor trace’ to compensate for memory declines during aging . An early study of eye movements dependent on age is in Bono et al. . Insight in the storage mechanism which partly is hierarchical and partly is flat sequential is from Yokoi et al. .
What happens when information to remember becomes too much for the brain? Apparently, we turn to our eyes for help to use the brain’s ability to see the world as an external memory . This “Embodied Cognition” postulate by Clark and Chalmers  says that instead of storing visual information in working memory: it may be equally retrieved by appropriate eye movements . Aagten-Murphy [41,42] distinguishes ego- and allocentric memories for independent working memory resource. In our approach, we follow Melnik et al.  by counting iterations to and from states of the subject’s activity. To model this we use a behavioral automaton, as explained below.
Shutter glass views give - because of its fast intermittence - a brief glimpse of the environment. An object, say a ball, if moving in front of the viewer is perceived as if fast blinking. The eyes move simultaneously. We want to know the purpose of the eyes saccades. Is the subject’s trying to remind (unconsciously) or is it updating its view because of lack of memory? Is the sensorimotor system reducing uncertainty by updating views? Early postulates [43,44] are that memory is at stake here, but a neural model proved that speed up of reaction times is a bistable sensorimotor learning process . Melnik et al.  take the view that saccades are behavioral events. This differs from the physics view [33,45,46] to include energy in the model. This energy approach is neglected here for gaining sharper insight in the main issue of this paper: to find when does an eye wander too much? This build upon von Neumann’s work on behavioral modelling, supported by regular expressions from Kleene which became Automata theory for computer science  and Ring theory in mathematics . We endow a stochastic version of such a finite automaton with an entropy measure to model functions from states of eyes resulting from saccades. Markov processes often are used in describing finite automata. We refrain from these because of the experimental set-up by Melnik et al.  via state spaces with mean behavior of saccades. This grouping or ensemble-based  approach enables to build an entropy instrument upon states of systems.
Entropic Computational Tool
We design an entropy index to distinguish eye movements by endogenous and exogenous saccades. The entropy of a behavior is at maximum if every saccade has equal probability. The behavior is then maximally disorganized . If saccades are employed strategically for updating memory the entropy lowers. The idea in this research is to use Melnik et al. state space  and compute its entropic measure : if movements from memory to the outer world have equal probability there is no difference (different transitions becomes equally likely, or: the saccades are a mess). Melnik et al. construct two state spaces one with constrained states and one without constraints, i.e. with unconstrained states.
An exhaustive analysis of entropies of automata is in Cortes et al. . Comparison of graphs of states spaces can be done in the geometrical sense  but could also be done by comparing entropy models between those spaces . The simplest of comparisons is by taking a difference between models . Melnik’s approach is surprisingly natural because entropy of a combination of systems is an additive operation . We construct it without the physics of light and energy as was suggested in multiple studies [33,45,46]. Briefly said, we take an event-based model of entropy (complexity), in the manner of treatment in automata theory [47,49] and linguistics .
Melnik’s research is 3 for ‘Model Area’, ‘Work Area’ and ‘Resource Area’. The number of incoming transitions is li,j from state i to state j, while mi is the total number of transitions incoming to state j. In total there are b = 9 types of saccades, or ‘transitions’ (see Table 1 in ). The entropy is maximum m = 2 78 m = 3 32 m =263, with saccades to the Workspace state and saccades to the Resource Area in Melnik’s experiment. These values are reconstructed from the data in Melink et al. . The dwelling of the eyes within Melik’s Model Area is explicitly taken into account here, because our goal is to discern whether the eyes look if the brain needs help, by lack of memory. The incoming transition probabilities in the unconstrained experiment for the lack of memory are [153/263, 78/263, 32/263]. The incoming transition probabilities in the constrained experiment for the lack of memory are [209/273, 47/273, 17/273].
Computation of the entropies reveals the difference between the states. It is also seen as a complexity measure, from the perspective of noise. The unconstrained (free viewing) state has probability p = 0.58, with entropy E = 0.14. The constrained viewing, however, has probability p = 0.77, with entropy E = 0.09. The gain achieved by the constraining of the view is evident (and found significant by Melnik et al. ), it is clearly expressed by the arrow in Figure 1.
Prior neural work paved the way to distinguish neural fields competing in visual processing and perception. The complexity of these findings has been reduced severely by us to explain, via a mathematical model measuring the complexity of such models by an Entropy measure. In detail, we found a diagnostic curve, if saccades are performed for update of vision or update of internal memory (of the image).
If entropy of saccades is high, then the eyes try to help the brain to help: the world is its external memory. If the entropy is low: the brain does not use the world as its external memory but relies on memory.
Availability of data and materials
The first, second and third author declare that for him/her no competing interests exists. The fourth author declares interest in ever bringing the results to commercial profit in his company.
HK designed the study, SK wrote the stochastic parts. PK-M conducted the literature search and analysis. HK designed the entropy model, reconciled it with PK-M’s analysis, and wrote the software; HK, PK-M wrote the manuscript. SK corrected the manuscript. GS designed the vision training, its reconciliation with the current model and proposed a measurement approach to gauge such type of models. All authors read and approved the final manuscript.
We thank Nedra Church for improvement of readability of this text.
2. Fooken J, Spering M. Decoding go/no-go decisions from eye movements. J Vis. 2019;19(2):1-13.
3. Fracasso A, Kaunitz L, Melcher D. Saccade kinematics modulate perisaccadic perception. J Vis. 2015;15:4, 1-12.
4. Walcher S, Körner C, Benedek M. Looking for ideas: Eye behavior during goal-directed internally focused cognition. Conscious Cogn. 2017;53:165-75.
5. Walcher S, Körner C, Benedek M. Data on eye behavior during idea generation and letter-by-letter reading. Data Br. 2017;15:18-24.
6. Sen S, Daimi SN, Watanabe K, Takahashi K, Bhattacharya J, Saha G. Switch or stay? Automatic classification of internal mental states in bistable perception. Cogn Neurodyn. 2019.
7. Zhang Y, Kumada T. Automatic detection of mind wandering in a simulated driving task with behavioral measures. PLoS One. 2018;13:e0207092.
8. Flanagan JR, Terao Y, Johansson RS. Gaze Behavior When Reaching to Remembered Targets. J Neurophysiol. 2008.
9. Warren PA, Graf EW, Champion RA, Maloney LT. Visual extrapolation under risk: human observers estimate and compensate for exogenous uncertainty. Proc R Soc B Biol Sci. 2012.
10. Bono F, Oliveri RL, Zappia M, Aguglia U, Puccio G, Quattrone A. Computerized analysis of eye movements as a function of age. Arch Gerontol Geriatr. 1996;22:261-9.
11. Du Toit P, Krüger P, Mahomed A, Kleynhans M,Jay-Du Preez T, Govender C, et al. The effects of sports vision exercises on the visual skills of university students. African J Phys Heal Educ Recreat Danc. 2011;17:429-40.
12. Gotardi GC, Polastri PF, Schor P, Oudejans RRD, Van Der Kamp J, Savelsbergh GJP, et al. Adverse effects of anxiety on attentional control differ as a function of experience: A simulated driving study. Appl Ergon. 2019;:41-7.
13. Gross AT, Hunfalvay M. Systems and Methods for Assessing User Physiology based on Eye Tracking Data. 2019;1-3.
14. Koppelaar H, Kordestani-Moghadan P, Khan K, Kouhkani S, Segers G, Warmerdam M. Reaction Time Improvements by Neural Bistability. Behav Sci (Basel). 2019;9:1-14.
15. Koppelaar H, Kordestani-Moghadan P, Irandoust F, Segers G, de Haas L, Bantjer T. Vision training against falling. (unpublished data).
16. Feng G. Is there a common control mechanism for anti-saccades and reading eye movements? Evidence from distributional analyses. Vision Res. 2012;57:35-50.
17. Heeman J, Theeuwes J, Van der Stigchel S. The time course of top-down control on saccade averaging. Vision Res. 2014;100:29-37.
18. Heeman J, Nijboer TCW, Van der Stoep N, Theeuwes J, Van der Stigchel S. Oculomotor interference of bimodal distractors. Vision Res. 2016;123.
19. Heeman J, Van der Stigchel S, Theeuwes J. The influence of distractors on express saccades. J Vis. 2017;17:1-17.
20. Coe BC, Munoz DP. Mechanisms of saccade suppression revealed in the anti-saccade task. Philosophical Transactions of the Royal Society B: Biological Sciences. 2017;372.
21. Savazzi S, Marzi CA. Speeding Up Reaction Time with Invisible Stimuli. Curr Biol. 2002;12:403-7.
22. Hubel DH, Wiesel TN. Receptive fields of optic nerve fibres in the spider monkey. Neurophysiology. 1960;154:572-80.
23. Gomez JI, Zhen ZI, Weiner KS. Human visual cortex is organized along two genetically opposed hierarchical gradients with unique developmental and evolutionary origins. pLos Biol. 2019;17:e3000362 1-29.
24. Liu Z-X, Shen K, Olsen RK, Ryan JD. Age-related changes in the relationship between visual exploration and hippocampal activity. Neuropsychologia. 2018.
25. Wynn JS, Olsen RK, Binns MA, Buchsbaum BR, Ryan JD. Fixation reinstatement supports visuospatial memory in older adults. J Exp Psychol Hum Percept Perform. 2018;44:1119-27.
26. Koppelaar H, Khan K, Segers G, van Warmerdam M, Kouhkani S. Sensorimotor abilities explained by bistability of FitzHugh-Nagumo model (unpublished).
27. Vandormael H, Herce Castañón S, Balaguer J, Li V, Summerfield C. Robust sampling of decision information during perceptual choice. Proc Natl Acad Sci. 2017;114:2771-2776.
28. McSorley E, Morriss J. What you see is what you want to see: Motivationally relevant stimuli can interrupt current resource allocation. Cogn Emot. 2017;31:168-74.
29. McSorley E, Morriss J, van Reekum CM. Eye spy with my little eye: Motivational relevance of visual stimuli guide eye-movements at different processing stages. Biol Psychol. 2017;123:8-14.
30. Morriss J, Mcsorley E, Van Reekum CM. I don’t know where to look: the impact of intolerance of uncertainty on saccades towards non-predictive emotional face distractors. Cogn Emot. 2017;32:1-11.
31. Morriss J, McSorley E. Intolerance of uncertainty is associated with reduced attentional inhibition in the absence of direct threat. Behav Res Ther. 2019;118:1-6.
32. Kim JH, Zhao X, Du W. Assessing the performance of visual identification tasks using time window-based eye inter-fixation duration. Int J Ind Ergon. 2018;64:15-22.
33. Mirza MB, Adams RA, Mathys C, Friston KJ. Human visual exploration reduces uncertainty about the sensed world. PLoS One. 2018;:1-20.
34. Van Moorselaar D, Olivers CNL, Theeuwes J, Lamme, Victor AF, Sligte IG. Forgotten But Not Gone: Retro-Cue Costs and Benefits in a Double-Cueing Paradigm Suggest Multiple States in Visual Short-Term Memory. J Exp Psychol Learn Mem Cogn. 2015;41:1755-63.
35. Reed-Jones RJ, Dorgo S, Hitchings MK, Bader JO.Vision and agility training in community dwelling older adults: Incorporating visual training into programs for fall prevention. Gait Posture. 2012;35:585-9.
36. Fogt N, Kuntsch E, Zimmerman A. Horizontal Head and Eye Rotations of Non-Expert Baseball Batters. Optom Vis Perform. 2019;7:29-46.
37. Wollenberg L, Deubel H, Szinte M. Visual attention is not deployed at the endpoint of averaging saccades. PLoS Biol. 2018;16:1-23.
38. Yokoi A, Diedrichsen J. Neural Organization of Hierarchical Motor Sequence Representations in the Human Neocortex. Neuron. 2019.
39. Clark A, Chalmers D. The extended mind. Analysis. 1998;58:10-23.
40. Tatler BW, Kuhn G. Don’t look now: the Magic of Misdirection. In: Gompel RPG van, Fischer MH, Murray WS, Hill RL, editors. Eye movements: A window on mind and brain. First. Amsterdam, The Netherlands: Elsevier Ltd.; 2007. p. 697-714.
41. Aagten-Murphy D, Bays PM. Automatic and intentional influences on saccade landing. J Neurophysiol. 2017;118:1105-1122.
42. Aagten-Murphy D, Bays PM. Independent working memory resources for egocentric and allocentric spatial information. PLoS Comput Biol. 2019;15:1-20.
43. Appelbaum LG, Schroeder JE, Cain MS, Mitroff SR. Improved visual cognition through stroboscopic training. Front Psychol. 2011;2 Oct:1-13.
44. Appelbaum LG, Cain MS, Schroeder JE, Darling EF, Mitroff SR. Stroboscopic visual training improves information encoding in short-term memory. Attention, Perception, Psychophys. 2012;74:1681-91.
45. Wong-Riley M. Energy metabolism of the visual system. Eye Brain. 2010;2:99-116.
46. Friston K, Adams RA, Perrinet L, Breakspear M.Perceptions as Hypotheses: Saccades as Experiments. Front Psychol. 2012;3:1-20.
47. Arbib M. Theories of Automata. 1956.
48. Nasehpour P, Parvardi AH. Finitely Additive, Modular and Probability Functions on Pre-semirings. 2016;1-24.
49. Cortes C, Mohri M, Rastogi A, Riley M. On the computation of the relative entropy of probabilistic automata. Int J Found Comput Sci. 2008;19:219-42.
50. Khinchin AI. Mathematical Foundations of Information Theory. Dover Publications Inc.; 1957.
51. Abdulmunem AA, Lai Y-K, Hassan AK, Sun X. Human Action Recognition Using Graph Matching. In: AIP Conference Proceedings. 2019. p. 50003 1-7.
52. Khouzani M, Malacaria P. Optimal Channel Design: A Game Theoretical Analysis. Entropy. 2018;20:e20090675 1-20.