Emotional Computing in Education

Psychology has long played a role in education by providing the surveys and questionnaires required to monitor students’ attitudes, dispositions and habits of mind. Today, psychology is coming to play an increasingly prevalent role in schools through intertwined developments in digital technology and education policy. New technologies of emotional computing and big data-driven “psycho-informatics” are being developed to conduct new forms of mood-monitoring and psychological experimentation within the classroom, supported by policy agendas that emphasize the social and emotional aspects of schooling.

Psycho-policy

A significant emerging area of education policy development focuses on the measurement and management of students’ “social-emotional learning.” A number of related terms have been used to describe social-emotional learning, such as non-cognitive learning, non-academic learning, character development, personal qualities, self-control, resilience, growth mindsets, mindfulness and grit. The influential report “Promoting Grit, Tenacity and Perseverance” was published in 2013 by the U.S. Department of Education, with some arguing that the 2015 Every Student Succeeds Act — the federal law requiring all states to collect information on at least one “non-academic” aspect of learning — is now providing the context for “testing for joy and grit.”

Major international organizations have begun to promote the development and measurement of social and emotional skills, particularly through technological means. The World Economic Forum published its report “New Vision for Education: Fostering Social and Emotional Learning through Technology” in 2016. Likewise, the Organization of Economic Cooperation and Development (OECD) has established its Education and Social Progress project to develop specific measurement instruments for social and emotional skills. The project is intended to generate evidence about children’s emotional lives “for policy-makers, school administrators, practitioners and parents to help children achieve their full potential, improve their life prospects and contribute to societal progress.”

These developments are part of a long shift in educational language to emphasize the “affective” aspects of education as well as its “effectiveness,” twinned with new data scientific methods of emotional measurement that have been proposed for use in educational settings.

Psycho-surveillance

In a new article, I have argued that the social-emotional policy agenda is being introduced into schools indirectly through popular classroom apps such as ClassDojo. ClassDojo is a free mobile app that allows teachers to award “positive points” for individual children’s behaviour and participation in the classroom. By 2016, it was being used by over 3 million teachers and 35 million children in 180 countries, primarily in elementary schools, with the stated aim to create happier students and develop qualities such as character, perseverance and grit.

ClassDojo reinforces ambitions to measure children’s social and emotional learning in schools. It enacts these ambitions by facilitating psychological surveillance, requiring teachers to monitor and collect student data that relate to new measurable psychological concepts such as character development, growth mindsets and other personal qualities.

In a study of  governmental adoptions of psychology and behavioural economics in other aspects of public “psycho-policy,” Lynne Friedli and Robert Stearn have documented the emergence of state strategies of “psycho-compulsion, defined as the imposition of psychological explanations … together with mandatory activities intended to modify beliefs, attitude, disposition or personality.”

ClassDojo exemplifies the rise of psycho-policies in schools that focus on both the surveillance of psychological characteristics and on the design of psycho-compulsion interventions intended to modify behaviours and emotions to meet specific measurable goals, particularly through the imposition of positive emotions and behavioural qualities.

Affective computing

ClassDojo is just one early sign of much more intensive psychological surveillance in schools that will be enabled by the development of “mood-monitoring” apps and even sophisticated forms of “affective computing.”

The World Economic Forum report on using technologies to foster social-emotional skills is indicative of future directions. It advocates the use of wearable biometric sensor devices to track physical responses to learning situations, such as fluctuations in stress and emotion, and to “provide a minute-by-minute record  of someone’s emotional state, potentially helping to build self-awareness and even empathy, both of which are critical components of social and emotional skills.”

Even more recently, the educational technology site EdSurge has published a piece on “emotive computing” in the classroom. Its claim is that emotive computing involves teaching computer-based robots “to recognize human emotions, based on signals, and then react appropriately based on an evaluation of how the person is feeling. Robots may actually be more useful than humans in this role, as they are not clouded by emotion, instead using intelligent technology to detect hidden responses.”

Some of the technologies profiled include facial recognition systems that use cameras to capture student responses, algorithms to identify their attention levels, and measuring smiles, frowns and audio to classify student engagement, “each using a combination of psychology and data-mining to detect micro expressions and classify human reactions.”

In addition, EdSurge profiles a number of innovations in “affective computing” that are being applied to education:

  • Transdermal Optical Imaging, with a camera that is able to measure facial blood flow information and determine student emotions where visual cues are not obvious
  • Electroencephalogram (EEG) electrical brain activity tests to measure students’ emotional arousal, task performance and provide computer mediation to individuals
  • Wearable affective technology such as a social-emotional intelligence prosthetic to detect human affects in children in real-time, which uses a small camera and analyzes facial expressions and head movements to infer the cognitive-affective state of the child
  • A glove-like device that maps students’ physiological arousal and measures the wearer’s skin conductivity, to deduce how excited a person it
  • Emotionally intelligent computing systems that can analyze sentiment and respond with appropriate expressions, enabling educators to deliver highly-personalized content that motivates children

Many of these innovations in emotive or affective computing originate in academic R&D settings, though commercial companies are taking them increasingly seriously and seeking to develop new tools to promote in the education technology market.

Real-time emotional feedback

All of these developments are part of ongoing attempts to make “real-time mood-monitoring” and “emotional feedback” devices into key technologies for knowing, measuring, and managing human emotions in contemporary societies, as the sociologist William Davies has argued. Davies argues that positive emotions have attained a particularly privileged position in recent years, making the science of happiness, well-being indicators and ‘mood-tracking’ critical to contemporary forms of management and public policy.

“The spread of ‘smart,’ mobile, wearable technologies,” Davies argues, “potentially allows humans to dwell in a purely ‘real-time’ cognitive state (feeling, experiencing, responding and liking) and allowing machines to perform acts of judgment, evaluation and decision-making, at least as an ideal.” Such forms of “affective capture,” he suggests, represent new ways of “valuing” the emotions, where the emotions become the object of assessment and judgment, and from there the targeted object of modification. Real-time mood-tracking devices are intended “to achieve a form of emotional augmentation,” to “transform it” and “render that emotion preferable in some way (be it more positive, more acceptable, simpler etc.), turning it into a different emotion.”

As Davies notes, psychological scales and categories pertaining to the measurement of emotions are integral to how many mood-monitoring devices operate. When wearable mood-monitoring devices are used for political purposes, such as in education and other areas of public policy, psychological scales can be used to define politically preferable forms of emotional conduct, and to impose interventions if deemed appropriate. In the process, individuals’ emotions would be changed to become more preferable, positive and acceptable.

Psycho-informatics in school

The combination of real-time data mining with psychological experimentation is currently inspiring a new psychological sub-discipline of psycho-informatics, “the single biggest methodological shift since the beginning of psychology or psychiatry.” Based on “the vision of a transparent human,” psycho-informatics makes use of wearable sensors that can track movements and smartphones to trace online activities, and then deploys data mining and machine learning in order to detect, characterize and classify behavioural patterns and trends as they change over time.

These psycho-informatic forms of mood-tracking and emotional augmentation and modification are indicative of future digital directions in social-emotional learning. The alleged potential of new forms of affective computing in education is to data mine students according to psychological signals detected from the skin, face and brain. The capture and assessment of affective data could then be used to inform interventions intended to impose socially desirable forms of positive affect on students. As the examples provided by WEF and EdSurge indicate, new psycho-informatic techniques are being designed to enforce new social-emotional learning policies and practices of affective measurement that value certain politically preferred forms of emotional conduct in classrooms.

ClassDojo is prototypical of how wearable mood monitoring devices and psycho-informatics might roll out to schools. Technically advanced forms of psycho-informatic mood-tracking and affective computing devices driven by psychological insights into the emotional aspects of learning — and how to detect affect from the body — could become increasingly in-demand as social-emotional learning policy makes positive affect and psychological compulsion into key policy priorities. Psycho-informatics could even potentially be linked into the accountability mechanisms by which schools are measured, assessed and judged. Apps and devices that can compel students to be happier, more resilient, and more “gritty” would be essential to schools if social-emotional learning becomes an accountability measure.

Banner image credit: Scott Brown