Bioaffect:Current events
IAT881: BioAffective Computing & Interactive Media
Spring 2011
Dr. Diane Gromala
Structure of class meetings
In each class meeting, unless otherwise noted, we will have:
topics for discussion & readings, assignments, reports, examples (artifacts) and discussion of methods.
Topics
Topics for discussion will be informed by the assigned readings.
I will assume that everyone has a thorough understanding of the readings (or quizzes may result).
We will discuss each topic by traditional means: discussion and textual responses.
For each reading, one of you will be assigned to address the topic by non-traditional, non-textual means.
This may be a video, performance, prank, interactive artwork, computer application, game, sound,
or any other legal means you come up with.
Assignments
Some assignments will simply be turned in, but most will be addressed in class.
Reports
Reports are presentations of issues assigned to you. Examples are a sensory mode,
a technology, an example (artifact, such as an application or interactive artwork)
and ideas for your project. Most should be no longer than 5 to 7 minutes, with 10-15 minutes of discussion or demos.
Basic suggestions for these reports will be provided, but you are encouraged to be creative and experimental.
For instance, perhaps the person who is assigned to report on the sense of hearing may bring in
things to listen to, things that might modify the way we hear, or design a sonic demo.
Similarly, whomever is assigned, say, the biofeedback technology of GSR might set
up a scenario using the technology, and show videoclips from the Millgram experiment.
Week One January 12: Introductions to affective computing, course requirements and each other.
Readings for next week:
The Emotional Self, Deborah Lupton: Introduction, Chapter 1 & Chapter 3 Media:LuptonReading1.pdf
Affective Computing, Rosalind Picard, Introduction & Chapter 1 Media:LuptonChapter1.pdf
Assignments:
1. Add information about yourself and your interests + ideas you’d like to explore to this wiki, in the Community Portal section.
2. Reports regarding “the senses” assigned.
Reporting requirements: Each of you has chosen a sensory mode. For our next class meeting,
bring a 7 page PowerPoint or KeyNote slide set. Be concise -- the words we see should only
be touchstones that help you recall your recently acquired knowledge.
Slide 1: (large) the name of the sensory mode (small) your name, IAT881: Bioaffective Computing, Spring 2011.
Slide 2: An articulation of how the sensory mode works in physiological terms.
Think about how the sensory mode you are covering might relate to other sensory modes. For instance,
touch requires proximity and presence, but vision does not.
Go beyond general assumptions.
For example, many may assume that vision just requires a set of eyes, but the brain is also crucial.
Viscera is very complex, so whomever has this one should probably focus on the enteric system, or refer to Wilson's reading, below.
We will know if whomever covers taste has done their homework if they tell us that we have taste buds
that recognize 4 (not 5) basic kinds of taste. Images, sound, animations, diagrams and videoclips are
okay, but be sure you include proper citations/attributions in small text.
Slide 3: Historical considerations. For vision, for example, some ancient Greeks believed that our eyes
emitted rays. (Note: this slide may likely have some overlap with the subsequent slide.)
Slide 4: Cultural considerations. How was or is your sensory mode considered in diverse cultures
(evil eye, eyes are the windows to our souls)? What role does your sensory mode have in one or
more cultures? (It is argued that some centuries in Western cultures were dominated by
visual or so-called scopic regimes -- what does that mean?)
Slide 5: Find an exemplary artifact regarding your sensory mode. This might be a painting,
videogame, machine, interactive artwork, musical composition, foley effect, CGI,
piece of grafitti, or so on. Choose your artifact to make a point that interests you.
Slide 6: What existing technologies relate to, simulate or address your sensory mode? Tell us
about them generally and specifically. Are there any sci-fi technologies that are interesting? If so, why?
Slide 7: Brainstorm about one project you'd love to create that directly relates to your sensory mode.
Think outside of the box. Don't be shy.
Suggestions:
a) Conduct Ypres-like research about that sensory mode. For exteroceptive senses, a good place to start is to
refer to Diane Ackerman's book A Natural History of the Senses. For interoceptive senses, a good place to
start is to refer to Drew Leder's The Absent Body. You will be confronted with a seemingly insurmountable
task -- to find and plow through an immense amount of information, and to make sense of it in a concise
but not dumbed-down manner. My advice is to maintain a hard-core, laser-like focus. Find the information
you need in the burning building, grab it, and get the heck out. One aspect of this assignment is for you to
figure out ways to quickly locate the best, most up-to-date knowledge from diverse domains, to determine
who the major players (theorists, scientists) are, what the main issues are, where exemplary works of
art, design, media or computing are, how to digest it all, and how to brainstorm ideas about that sense modality.
These are incredibly important skills.
b) Assume that you will have 30-45 seconds per slide, on average. Your report (slide presentation) will be
followed by a discussion, so use your reporting time in a concise manner. The rest of us will likely
expand upon what you have presented.
c) You can't possibly cover everything, so don't try. You DO want to make an incredibly well-informed
presentation, and you DO want to communicate the most important information. But at many points in
this process, you will have to make important decisions about WHAT is important and WHY it is important.
That's part of the assignment.
d) Often, it is easier to gain an understanding of how something works by looking at examples
of dysfunction, or so-called outliers. An example of this is the work of someone like Oliver Sacks.
e) You are strongly encouraged to enhance our understanding by helping us get our hands dirty as it were.
That means you are encouraged to bring stuff in, have us do things, or otherwise find ways to make
your information come out of the textual box. Remember that you don't have a lot of time, however.
One idea is to augment just ONE slide with something in the material world. For touch, for example,
you might consider bring in things to touch, or a technology that affects the way we experience touch.
f) Have fun. This stuff is really interesting. Seriously.
g) Remember, strategic risk-taking will be rewarded, in heaven, on earth and in class.
Week Two January 19: The Substrate of Sensory Experience: Intero- and Exteroception
Topics for discussion:
• Affective Computing, according to Picard
• Exteroception (Seeing, Tasting, Touching, Hearing, Smelling) and
Interoception (Proprioception, Temperature, Pain, Orientation, Kinesthetic Sense, Visceral Sense)
• Technologies related to affective computing: biofeedback, (and possibly: GPS, accelerometers)
Readings for next week:
1. (re-read Lupton)
2. CHI Paper: Humanities: Feminist HCI: Taking Stock and Outlining an Agenda for Design; doi: 10.1145/1753326.1753521
3. CHI Paper: Scientific: Predicting tie strength with social media; doi: 10.1145/1518701.1518736 (sent via email)
4. Barnes: "The Simulation of Smiles (SIMS) model: Embodied simulation and the meaning of facial expression" (sent via email) http://journals.cambridge.org/action/displayIssue?iid=7947586
Assignment:
Conduct an “autopsy” of the "Feminist HCI" and the "Predicting tie strength" papers.
For this first assignment, we will examine 2 papers that received best paper awards.
(For the next assignment, papers will come from more obviously different knowledge bases and fields.)
Bring the results of your autopsy in a form that can be easily shared with the rest of your colleagues in class.
The term autopsy: "to see for oneself" (Ancient Greek)
Goal: to uncover reasons why each paper was deemed to be a good paper, and to formulate reasons
for why each paper was recognized as a "best paper" by CHI decision-makers. It is assumed that few
of you are privvy to the internal politics or extenuating circumstances for best paper decision making.
This is loosely based on criteria used for judging academic papers generally, and much of it pertains to non-academic papers as well, so you will doubtless find pragmatic use for this exercise.
Directions for conducting a forensic autopsy (assume you are a SIAT-CSI):
1. Assess overall context: you know that each paper received the best paper distinction from the CHI conference.
Each of you have varying degrees of knowledge about that conference.
1.1 Articulate what you know and what you have discovered about that conference. (Given time constraints, it may be useful to ask
your colleagues and faculty members, to visit CHI's site, and to visit the URLs that lists best paper awards
for the past few years. http://jeffhuang.com/best_paper_awards.html,
http://www.chi2006.org/bestofchi.php, www.chi2007.org/pressroom/0425-best-papers.pdf
1.2 Make an initial assessment about factors that may have played roles in the decision to recognize each paper as "best paper."
2. Identification.
2.1 Who are the authors?
2.2 What factors may be associated with assessing their credibility?
2.3 What is the subject of the paper?
3. The Y-incision:
3.1 Why may this subject be relevant to the CHI community?
3.2 Is the subject innovative, original or one which may have only been rarely addressed before? Does it provide new insights? If yes, articulate your reasons; if no, articulate your reasons.
4. Structural integrity:
4.1 Is each paper technically sound? While you may not be familiar with this field, look for clues anyway.
Is the technology dated (look at the citations)?
Do you see the terminology elsewhere on the CHI site or in a CHI search? If the technology is experimental, does it seem realistic and doable?
(If the authors describe, say, a computer based on DNA soup or on jet-pack technology, or if such technologies are not visible in the citations, the answer is probably no.)
4.2 Is the article structurally sound in terms of readability, structure of argument, grammar and use of images & graphics? Taken together, these should promote comprehension, even if you are not an expert.
5. Assessing major organs:
Does each component make sense or cohere?
Are the claims backed up by appropriate citations? Every claim needs to be supported unless it is "common knowledge."
Is there a component that leaps out at you because it is exceptionally confusing, not backed up, not articulated well, too broad or too specific? Or does it just not seem to fit? Often, even good papers have problems (or "issues") -- usually one glaring issue that needs to be addressed by rewriting and resubmitting. Is there an identifiable weakness? If so, what is it?
If possible, it may be helpful to structure your answers in parallel columns, or on two parallel pages, one per article.
You may use words, phrases or bulleted lists for some of your answers, but use sentences when you need to really explain something in detail.
Remember, you are to read (and re-read) and assess the "Feminism HCI" and the "Predicting Social Tie Strength" articles, NOT the "Simulation of Smiles" article. THe more effort you put into this, the better your results will be now and later. Like a new CSI (crime scene investigator), you are expected to use specialized tools and methods -- in this case, to search for information elsewhere. Over time, this becomes a habit, but you will almost never judge a paper simply by reading it, especially when you are in interdisciplinary realms. Knowledge is constantly updated, so even the most prominent specialists habitually make sure that they keep current with knowledge in their field (in some fields like medicine, one needs to regularly accrue continuing education credits and certification), scours journals and databases and generally seeks knowledge for pain and pleasure.
GRADING: I will take your background and current knowledge into account, but I expect everyone to demonstrate that they have made serious efforts. Thus, answers such as "this is crap," and "I disagree with the basic premise," and "I have no idea how to even begin to figure this out" will doom you to the depths of very bad grades.
Like the grading schema you witnessed during the slide presentations, grades in the realm of an A mean that you not only do the assignment, but that you do it exceptionally well and go beyond the instructions. Some of that is sheer insight and being able to make profound connections, while some of it means you've spent hours slogging through information, and/or you demonstrate superior investigative techniques -- like finding experts for advice or sending the authors or editors a friendly email. (Note: busy people will generally not reply to emails unless you provide them with a reason to do so, and make it easy for them to do so. So asking them open-ended questions usually won't elicit a response.) Grades in the B range mean that you have done a good, solid job.
Note: As witnessed in the slide presentations, you may choose a relatively "objective" stance, or you may choose to integrate knowledge you are expert in, or you may choose a position (point of view) that may not be obvious or usual. That is encouraged -- it just has to be relevant and really good. Contrary to popular belief, you CAN follow the assignment specifications AND be creative. And stay within presentation time limits.
Week Three January 26: Biopsychology of Emotion
Guest lecturer: Dr. Steven Barnes, neuroscientist & artist, UBC/SFU (sjb@nervouscreation.com)
Discussions:
1. Lupton: Discussion leaders: Veronica & Jay
2. Biopsychological Theories of Emotion: Dr. Steven Barnes Reading: Simulation of Smiles: http://journals.cambridge.org/action/displayIssue?iid=7947586
Please send your autopsy assignments to Dr. Gromala via email. Due: Wed. Jan 26th, 5:30pm. Don't forget to title the email, beginning with BIOAFFECT.
(NOTE: slide presentations from Daniel & Andrea will be forthcoming.)
Week Four: February 2 biofeedback basics & review
Discussions: Review: Lupton & Picard: What counts as "emotion"? Mood? Affect? Temperment? + Barnes' reading
Biofeedback demos: GSR
Readings for this week:
Picard, Affective Computing, Chapter 2
NOTE: We will be reading most of Picard's book, so it would be a good idea to plan to read 2 chapters per week (say, during the reading break).
Week Five: February 9
Class Trip: Vancouver Aquarium, SEE THE 4D Show: www.visitvanaqua.org/news/4D
Note: the show runs every 20 minutes or so in the afternoon. www.visitvanaqua.org/visitors/show-descriptions
THE DOORS CLOSE AT 4:45pm, & the LAST SHOW TIME IS at 6:00pm -- be sure to bring your student ID.
Assignment 1: Write a one page paper about the so-called "4D" aspect of the show, roughly in 500 words or so.
Your paper should include a concise description (1 paragraph) of the show, and should be an analytical. Bonus points: add a paragraph of what you WOULD do to create a more affective experience (define), with any technology, real or imagined.
Assignment 2 (self-monitoring prep):
a) Find 3 examples of art or art+science projects that demonstrate an area of self-monitoring (see: quantified self.com, Dunne & Raby or Scofidio & Diller for starters). Write a list and provide URLs or book, article or movie titles where possible. For each, provide a one sentence description of what was monitored, how/what it was mapped to, and what it resulted in. Add a sentence about what you thought was interesting about each one.
b) Explore (20) self-monitoring technologies. At least half MUST be outside of the box of "norm" (i.e., commonly considered self-monitoring technologies like commonly available pedometers, biofeedback). Make a list of at least 20 (you'll hit a wall at 10, but it will open up when you hcange how you think about them). Describe, in a phrase, what the technology measures so that any of your colleagues would understand it if they read it with your verbal explanation. To jog your habits of thinking, refer to MIT's Media Lab, any scientific lab that deals with animals (your list is about self-monitoring technologies, but many analyses of animals apply to humans; don't forget that we are a host and have close links to other species), and Theo Gray's book Mad Science (available at Dr. Shaw's desk).
c) Think about (10) connections between self & world, and how when you self-monitor, you MAY be consciously or not consciously affected by moment-to-moment changes in your not-self context (world). Make a list of 10 aspects of world, from phenomena to places, states, effects or zones. What aspects of your "external" world are you interested in? Things you can't see, like radio, gamma or EMF waves? Bacteria that live on your eyelashes (is that itch from something that is on your skin?). Or, why are some places so seemingly "creepy" that your fine body hairs stand up, while others are not -- is it all in your head? Electromagnetic vibes from earthquake faults? Concentrated pressure systems? Ectoplasm residue from ghosts or jellyfish? Static electricity? Subtle oddities in light and atmosphere? Petro deposits? Insects? Subsonic noise? This part of the assignment is more speculative -- a technology to measure phenomena may not exist. It might be useful to think about specific places and ask why you may respond to them, and think about technologies that may measure phenomena that are not (easily) perceptible.
BOTH ASSIGNMENTS ARE DUE IN EMAIL FORM: February 16 5:00 p.m. (via email), include BIOAFFECT FEB.16 in title of email.
Readings for next week:
Picard's Affective Computing, Part I
Week Six: February 16 READING BREAK
Self-monitoring project: investigate a monitoring technology and design a self-monitoring project.
Readings for next week:
Finish reading Picard's Affective Computing book.
Assignment for next week:
Develop and roughly flesh out ideas for your class project. Each of you will be required to discuss them in class next week.
The more developed the idea, the more useful feedback will be for you. (You can work with a colleague or two. Group size limit is 3 people, unlimited other organisms.)
Week Seven: Feb. 22: Catch-up
Initial project ideas articulated.
Week Eight: March 1:Catch-up continued
First half of class: Assignments discussed; examples of bioaffective art, self-monitoring projects & technologies.
Picard reading part 2
Second half of class: testing technologies & discussing projects as they evolve.
Readings for next week:
Farquhar's, Beyond the Body Proper, Introduction (available online at Google Books - with some pages missing: [1]) Also, PDF: Media:Lock_Farquhar_Intro.pdf: Gromala
Springgay: Fantastical BodyMedia:Fatastical_body_small.pdf Lonnie
Noe: Out of our HeadsMedia:Out_of_our_heads2.pdf D'Arcy
The Body: Orlan: D'Arcy
The Inner Touch: Of Flying Animals (Ch 21) D'Arcy and Jay
The Inner Touch: The Anaesthetic Animal (CH 24) Tyler
(note: the last 4 essays are very short but meaty)
Week Nine: March 8: Beyond the Normative:Disabilities, Synesthesia & (co-)adaption
First half of class: Discussions: readings, bioaffective art & technology examples continued
The Inner Touch: Of Flying Animals (Ch 21) D'Arcy and Jay
Discussions: Humans and their technologies.
Second half of class: Terry will bring in different film/video clips to discuss the rhetoric of his project
Readings for next week:
Psychosomatic: Stress & Neurobiological Kindling: Terry
Week Ten: March 16
Discussions: Psychosomatic: Stress & Neurobiological Kindling: Terry
The Body: Bodies, Disabilities: Terry
Heller-Roazen: The Inner Touch: The Anaesthetic Animal: Tyler
Picard: Part 1: Mark
Picard Part 2: (Ch.5 Affective Signals & Systems) Jay, Veronica, Lonnie, Daniel
Readings for next week:
Picard Part 2: (Ch. 6 Rec. & Expressing Affect) Jay, Veronica, Lonnie, Daniel
Week Eleven: March 23: Body Image/Body Schema; Habituation, Healing: Shamanism, CAM
Discussions: Picard Part 2: (Ch. 6 Rec. & Expressing Affect) Jay, Veronica, Lonnie, Daniel
Readings for next week:
Picard Part 2: (Ch. 7) Jay, Veronica, Lonnie, Daniel (Ch.8 Wearables+) Suk Kyoung
Haraway, Cyborg Manifesto: Terry
The Taboo of Subjectivity Scientific Paradigms: Intro & Chapter 1: Suk Kyoung & Lonnie
(note on this reading: identify the 4 basic approaches)
Week Twelve: Medical Technologies & Affect
Discussions: Haraway, Cyborg Manifesto: Terry
The Taboo of Subjectivity Scientific Paradigms: Intro & Chapter 1: Suk Kyoung & Lonnie
1. Medical Frontiers: Research re: brain-computer interfaces, drug delivery technologies, genetics, nanotechnology.
2. CAM (Complementary & Alternative Medicine)
4 CLASS PRESENTATIONS
Week Thirteen: April 6 Review & Archiving, Wet & Dry
Course Review
5 CLASS PRESENTATIONS
FINAL PROJECT ARCHIVAL INFORMATION TURNED IN, IN CLASS.
Be sure everything is labelled and dated. No label, no grade.
Required Readings for the course
Picard Affective Computing, http://troy.lib.sfu.ca/record=b1876254 (Full text available online) The basis for all discussion, updated research and extensions of her model.
Lupton: The Emotional self : a sociocultural exploration,
http://troy.lib.sfu.ca/record=b1904667
Farquhar: Beyond the Body Proper, Introduction (available online at Google Books - with some pages missing: [2]) Also, PDF: Media:Lock_Farquhar_Intro.pdf: Gromala
Noe: Out of our Heads Media:Out_of_our_heads2.pdf D'Arcy
Massumi: Parables for the Virtual: Movement, Affect, Sensation http://troy.lib.sfu.ca/record=b2483710 (Full text available online)
Niedenthal: "The Simulation of Smiles (SIMS) model: Embodied simulation and the meaning of facial expression" http://journals.cambridge.org/action/displayIssue?iid=7947586
Springgay: Fantastical Body Media:Fatastical_body_small.pdf Lonnie
Wilson: Psychosomatic: Feminism and the Neurological Body, Chapter 2: The Brain in the Gut http://troy.lib.sfu.ca/record=b3617961 (Full text available online)
Suggested Readings for the course: Intero- and Exteroception
The Absent Body, Drew Leder http://troy.lib.sfu.ca/record=b1465856
A Natural History of the Senses, Diane Ackerman http://troy.lib.sfu.ca/record=b2461966
Suggested Readings for the course: Beyond the Norm
The Mind's Eye, Oliver Sacks http://troy.lib.sfu.ca/record=b5664168
Suggested Readings for the course: technologies
Physical Computing: Sensing and Controlling . . . , Tom Igoe and Dan O'Sullivan http://troy.lib.sfu.ca/record=b3872653 (Full text available online)
The Affect Theory Reader, Melissa Gregg & Gregory J. Seigworth, eds. http://troy.lib.sfu.ca/record=b5532632
The Taboo of Subjectivity: Toward . . . B. ALan Wallace http://troy.lib.sfu.ca/record=b2137665 (Full text available online)
"'Thought Technologies Manual"'
http://www.thoughttechnology.com/pdf/manuals/SA7560%20Rev%203.pdf
Interesting Readings?
10 Most Bizarre Scientific Papers
http://www.oddee.com/item_90683.aspx
ASSIGNED READINGS: BRIEF ANNOTATIONS
Everyone should write a very concise paragraph covering the reading -- somewhere between a dictionary and encyclopedia entry.
If you share a reading with another student, you should collaborate on the entry.
Goal 1: to be a resource for all students in class
Goal 2: for you to inculcate the reading
Goal 3: for you to speak in every class "as" the author(s) assigned to you. I.e., "Author X would say . . . "
example: (Veronika & Jay)
Lupton, The Emotional Self: Jay and Veronica
Lupton asserts that emotions have been historically been viewed as embodied sensations, so they were (and still are) are considered to be the antithesis of reason and rationality (p.3).
Lupton's main aim is to discuss how emotions contribute to our sense of self. In doing so, she covers many divergent theories of emotions.
In the first chapter, she divides knowledge about emotions into 2 major areas:
1. emotions as inherent: humans have a set of basic emotions, even though the expression of them may vary according to culture & history
• emotions are generally viewed as:
genetically inscribed, inherited rather than learned (positivist, essentialist, organismic)
linked to physiological changes, the result of stimuli
instinctive reflexes ranging from uncontrollable to moderately controllable through willpower/self-control
• evolutionary psychologists: the function of emotions for survival & continuation of the human species
• inherent perspectives connect to this idea of emotions being "animalistic" tendencies, implying that emotions should be suppressed for a progressive society.
• Cognitive theorists: the interest is on the interrelationship among bodily response, context & individual recognition/awareness; emotions are viewed as coping mechanisms
• William James: physical responses precede the emotion, interpreted based on judgement of a situation
The citation of Mesquita and Frijda suggests theories which connect an emotion to "preparation for action". They boil emotion down to an evaluation of events, their valence, their controllability, etc. which is a very functional approach to the concept.
- This area ties directly to the idea of "behavioural ecology", completely lost on this author. - Evolutionary adaptations which start for one purpose can become appropriated - discussion is missing here!
The philosophical approach to truth about emotion is practical, but the power of cognitive models to predict is omitted completely from this article.
- The authors should mention the practicality of cognitive science, for technology, therapy, diagnosis, etc.
2. emotions as sociocultural constructions:
Emotions are phenomena that are shaped, experienced and interpreted through social and cultural practices (p.2).
"Weak thesis" (Kemper): some emotions are universal, some acquired by socialization, & universal emotions can be altered through socializing "agents"
- When we discussed this in class, most of us agreed that the weak sociocultural approach seemed to be most intuitive. - We also discussed how the "weak" vs. "strong" nomenclature is a bit biased; the author is having a hard time presenting a neutral case.
"Strong thesis": emotions are irreducibly sociocultural products, entirely learned and acculturated; purely contextual; intersubjective rather than individual experience
Harre: there is no such thing as an emotion, only multiple ways of acting & feeling in a bodily way; one "does" rather than "has" an emotion; self-assessment plays a role
Cultural anthropologists (Heelas, Lutz): strengthen the sociocultural direction by doing cross-cultural analyses & demonstrating major emotional differences in diff. groups of people
Social historians (Stearns): strengthen the sociocultural direction too, but in this case, longitudinally: demonstrating the dynamics of emotion through different eras (i.e. Victorian domination of grief)
Structuralists (such as Durkheim): examine the bonding nature of rituals "collective effervescence"; social order is not just rational, but underpinned by affective/social ties
Phenomenological Approach: a person's interpretation of bodily sensations are the emotion, NOT behavior or awareness of physiological changes
Picard Affective Computing: Part 1: Applications (+extend): Mark
Picard defines affective computing as: artificial intelligent systems that can detect, evaluate, and process emotions. It is a human-computer interaction that includes 'sensing and reasoning context' that is derived from emotional influences.
In the book, Picard provides examples of how we can implement affective computing in various fields such as learning, social development, entertainment, preventative medicine, and consumer products. A few examples are: embedding emotions in learning, gaming that measures performance using biofeedback, healthcare specifically helping people with autism by providing feedback using their emotional responses, and software applications such as incorporating social human emotions into digital communication. Picard also points out that no application can solely be based on affective computing. It has to be associated with other tools such as the user interface or computer vision.
Issues related to emotion recognition: -how we express emotions has a broad range and the meanings or modalities vary across cultures, gender, and age. Restrictions on accessing emotions is a concern as well for example it depends on the level of neural brain activity and biological configuration. According to Picard, being able to accurately interpret and define this level of complexity is highly unlikely.
-the other difficulty is how we interpret our emotions and the variability that exists amongst individuals. Also our emotional circumstance tends to rapidly change from one state to another and this minimal latency between the shifts will be difficult for a system to interpolate.
Picard Part 1: Potential Concerns (+ Turkle handout): Terry
Picard Part 1: Emotions are Physical and Cognitive: Daniel
Picard Part 2: Affective Signals & Systems: Veronica, Jay & Daniel
Picard is going to describe the mechanics of detecting and expressing emotions on three levels. Low-level, mid-level and high-level... this dimension pertains to the level of analysis. Whereas low level are direct sensors, mid-level are abstract combinations of sensors, and high level are concepts which power connectionist networks.
Another dimension is the emotional state of the perciever, this can affect which emotion is interpreted. Bottomup and topdown influence on perception of emotion.
Context is also important, one may have the same emotional physiology in different contexts for different reasons. Emotion is not only bottom-up (from signals to symbols) but also top down.
Bell is a physical metaphor for emotion. Curved response. Successive strikes are cumulative. Different bells respond differently (based on "temperament").
Linear, time-invariant system are easy to understand. Picard suggests that computationally modeling people would depict them as neither. Instead, parameters which change over time and person would be defined to control the basic response curve (sigmoid).
Picard models emotional response as a sigmoid. As events happen the response strengthens but it hits a ceiling. The threshold, gain and ceiling are all dependent on context, personality, mood and other factors. So the sigmoid is the proximal emotional response, while the contextual properties and dimensions of the sigmoid are controlled my mood, among other things.
Context is plausible in the proximal, but long-term moods still remain fairly nebulous. Picard suggests some constant monitor of every event and input may be able to simulate it and affect the sigmoid.
Picard acknowledges that valence and intensity is a simplisitic model. She suggests that multiple parallel sigmoids could represent a wide range of emotional dimensions which may or may not affect eachother.
The sigmoid based system represents response decay, repeated strikes, temperatment, nonlinearity, time-invariance, activation, saturdation, feedback, mood.
Picard then discusses physiological signals and how they correlate to emotion. She is defining a gap between these correlations and models for emotion, and depicts them in a way which implies that it could eventually be filled.
Picard Part 2: Emotional Synthesis: Veronica, Jay & Daniel
This chapters moves up from low-level signal processing and considers the role of pattern analysis in emotion detection.
Again, as in Chapter 4, Picard suggests we can't directly observe emotions from the outside... Green would argue (part-whole)
Picard looks through a wide range of emotional theories which suggest a few basic emotions, and some theories which suggest "dimensions" of emotion (Arousal (calm vs. excited), Valence (negative vs. positive)).
Picard acknowledges different theories of emotion, but suggests that these models will be useful for representation : even if they turn out to be an oversimplification.
Patternmatching lies in the chasm between individual person-specific emotions and universal patterns.
A key question is whether emotions are mixed or pure. How is a state of multiple emotions represented? This is important as we are not likely to get pure emotions from users of affective computers.
Facial recogntion is a prototypical example of pattern analysis being used to model emotions (ekman, etc.). Another example is the synthesis and recognition of affective vocal intonation. Picard reviews work in both these domains in detail.
Roy and Pentland are studying whether vocal utterances are generally approving vs. disapproving using audio measure (mean frequency, harmonic ratios, etc.) - high accuracy was found.
Given the promise in all this work, a future direction is the combination of face/voice for more conclusive recognition. The McGurk effect shows why this is a powerful process and similar to how humans perform it.
Picard used a series of physio sensors to read emotions over time and found significant predictable patterns for emotions
Hidden Markov Model estimates probability of emotional transition based on observation... what are the probabilities of moving from joy to sadness vs. interest to sadness? Here we're goin up a level in analysis: you could better identify ambiguous physiological state by knowing previous states.
Interesting that Werbos came up with backpropogataion in connectionist networks based on an emotional idea from freud: freud saw emotions flowing back through cause-effect relationships (if waiting in line leads to a packed club which is happiness, then waiting in line gets associated with happiness too... the emotion back-propagates).
Picard Part 2: Affective Wearables (+extend): Suk Kyoung, Mark
Niedenthal: The Simulation of Smiles (Barnes' facial expression): Veronica
Springgay: Fantastical Body: D'Arcy
Noe: Out of Our Heads: Lonnie
Farquhar: Beyond the Body Proper, Introduction: Gromala
In this introduction to a collection of essays, Farquhar notes the immense increase in academic research concerning what some academics call "the materialist turn" (now) which follows "the linguistic turn" (roughly, from WWII to 1990s) in the Humanities. Thus, terms like "the lived body" are used to emphasize the dynamic aspect of "living" bodies that are inescapably interconnected to and continuously affected by physical, material and biological forces (was "nature") as well as social and cultural forces (was "nurture"). Although clearly not a scientist, Farquhar draws upon the "fascinating empirical studies" that enlarge our "vision of human capacities" (p.1).
Of particular interest: Farquar articulates how theorists have challenged the commonsense, "proper," naturalized and normative ideas of a body (as a fixed, skin-bounded, biomechanical container [science] & unit of individuality [humanities/social science]). The focus is on trying to perceive our bodies as dynamic, intersubjective and plural (context matters). To do this, it is often necessary to re-examine habits and "common sense," because what we often take for granted is really the ways particular social/cultural ideas become "naturalized." To more fully understand how social & cultural contexts affect dynamic bodies, refer to the examples of genetics (genetic code vs. genetic predisposition: habits & behavior can "switch on") and especially (historical & cultural comparisons of) medicine.
Check out important ideas of how technologies have changed our operational ideas of bodies: "cyborg" (Haraway) and "posthuman" (humans aren't at the centre of the universe; Hayles & esp. Wolfe), p.3-4.
Massumi: Parables...Movement, Affect, Sensation: Tyler
Like Farquhar, Massumi is not a scientist, but isn't allergic to science. Tho neither goes as far as many in SIAT might hope for, it is important to keep in mind the multiplicities of what it means to be interdisciplinary: from experts who "quote" or use ideas from another field to dual-experts (academics who regularly publish & are recognized in 2 fields, either closely aligned or divergent).
Haraway: Cyborg Manifesto: Terry
This was a standard reading since its publication.
Hayles: How We Became Postmodern, Ch.3 (Contesting for the Body of Information: the Macy Conferences on Cybernetics)
Wolfe:
Wolfe combines animal studies with systems theory.
Fraser & Greco: The Body: A Reader: Intro: Tyler
Fraser & Greco: The Body: A Reader: Bodies, Disabilities: Terry
Heller-Roazen: The Inner Touch: Murratia... (Ch. 1) Jay
Heller-Roazen: The Inner Touch: Awakening... (Ch 7) Suk Kyoung
Heller-Roazen: The Inner Touch: The Unnamed King (CH 14) D'Arcy & Mark
Heller-Roazen: The Inner Touch: Thorns (Ch 19) Suk Kyoung
Heller-Roazen: The Inner Touch: The Anaesthetic Animal (CH 24) Tyler
Wilson: Psychosomatic: Brain in the Gut: D'Arcy
Wallace: The Taboo of Subjectivity: Scientific Paradigms: Intro & Chapter 1: Suk Kyoung & Lonnie
