Greetings all! So I have part of my literature review written as a rough draft, but before I submit it here I would like to review some challenges that I faced in writing it and describe how I spent my time. Last week I spent a few hours combing over the videos of our test subjects being taught by our prototype virtual tutors, and took notes on behaviors that were commonly appearing. Part of this method can be though of as grounded theory. Personally, I’m using an ethnographic approach to reviewing this data, called interpretive description. This method is a relatively new method in qualitative data analysis, that allows one to extract (interpret) the big concepts from various descriptions (in my case videos), to find the most important questions to ask about the data.
My only challenge so far has been to try to structure my literature review in a way that will generate questions. It is far easier to read many articles and write a summary than to think of a gap in the field, in my opinion. So without further ado, here is my first pass at the literature review, using my outline as a guide.
Comfort with virtual agents can depend on factors like eye gaze of the virtual agent and ability of the virtual agent to respond to social cues. One notable social cue that is important when one is trying to form a bond, whether with human or virtual agent, is the idea of self disclosure. Self disclosure is the sharing of personal information about oneself. It could be something as simple as “I’m not good at math” (cite Madaio annotation manual here) or as profound as “ I never liked math in childhood and felt pressured by my parents to do it.” When people are comfortable, they disclose information about themselves and while they do that, there’s always a question appearing in their minds of whether it is worth self disclosing to the partner (Farber 2006). Some people tend to self disclose less, and this is may be due to shyness. Although historically, shyness is a vague term, lately the literature has proposed it be conceptualized in three components: affective, cognitive, and social. Shyness is believed to be a trait of personality that causes behaviors somewhere between neuroticism and extrovertedness. Those may sound like very opposite terms, but somewhere in between is shyness (Crozier 2005). In fact, the cognitive aspect of shyness can be considered similar to anxiety, in that one part of shyness is the fear of socializing and meeting new people (this symptoms is present in cases that are not clinically severe, just everyday shy people) (Crozier 2005). Shyness is important to discuss in terms of self disclosure because it can inhibit self disclosure because anxiety reduces self disclosure (Madaio 2016), and it can be due to a feeling of inferiority or lack of comfort. In order to make virtual agents that humans are comfortable dealing with, we must consider shyness and self disclosure as important factors. It remains a question in the field as to whether human shyness (measured by the shyness scale of Cheek and Buss) (Crozier 2005) can be minimized by virtual agents and what strategies are useful for this.
Self disclosure is increased in patients who are feeling anxious by virtual agents disclosing human back stories (“I was born in Italy, my parents never loved me”), as opposed to machine back stories (“My parts are made from Dell and my mother is a CPU”) (Kang 2012). Kang et al studied the effects of human-story telling virtual agents on self disclosure on psychotherapy sessions but they did not consider how human story-telling can improve self disclosure in regular, non-patient people, like kids studying math in middle school. Also, Yu (2013) demonstrated that depressed patients (similar to severe shyness that extends into neuroticism ,where people are not just unmotivated to socialize, but afraid to socialize) have a tendency to begin talking very early on after a rapport building question is posed, particualrly a self disclosure question. In other words, patients who are clinically shy tend to want self disclosure. Now we have to extend this into the dimension of regular shy people, who are not patients in psychotherapy. As stated before, regular shy people feel the same elements of shyness in clinically depressed shy people, but to a lesser degree, so efforts should be made to see how the theories used to help clinically shy people can carry over to regular shy people.
Yu, Z., Scherer, S., Devault, D., Gratch, J., Stratou, G., Morency, L-P., Cassell, J. (2013). Multimodal Prediction of Psychological Disorders: Learning Verbal and Nonverbal Commonalities in Adjacency Pairs. Presented at SenDial. http://www.cs.cmu.edu/afs/cs/user/zhouyu/www/semdial_2013_zhou.pdf
Subjects. N = 130 dyads in semi-structured interviews. Each subject was paired with 1 virtual human interviewer.
Results. Depressed people speak for less time responding to questions to the rapport building phase of the session (e.g. Do you travel a lot?) (last page, 2 paragraphs above conclusion). Speech onset time and speech rate correlate negatively with depression.
Methods. The idea is to use the HCRF (hidden conditional random fields), which is a mathematical/computational method that learns commonalities (called hidden states) between variables, makes these “hidden states” into categories by which variables are grouped, and considers the time dimension — important to this study because it parses behavior with respect to each question posed over time. This study uses the HCRF together with multimodal data (verbal and nonverbal, i.e. acoustics, visual data) to categorize verbal and nonverbal behaviors into categories of psychological disorders. Previous studies did not consider as many modes of data, but have shown correlations between certain modes, e.g. speech rate, and psychological disorders, e.g. depression. Importantly, this study makes categories at the adjacency pair (question-response pair) level, whereas previous pairs used a whole-session level that was too general.
Ozyesil, Z. (2012). The Prediction Level of Self-Esteem on Humor Style and Positive-Negative Affect. Psychology, Vol. 3 (8), 638-641. http://dx.doi.org/10.4236/psych.2012.38098
Subjects. N = 440 undergraduate students (77% female), chosen by simple random sampling. Mean age = 20.33, with std dev = 1.73.
Results. Most interesting is that self esteem is negatively correlated (p < 0.001) with self defeating humor and negative affect. Moreover, self-esteem explained 4.2% of the total variance in negative affect. Self esteem is a personal judgement of worthiness, and is a personality trait. Humor is also a personality trait, and affiliative humor is believed to increase interpersonal bonds (rapport), and thus is used by people with high self esteem; the presence of one predicts the other reliably (high positive correlation). Low self esteem is believed to lead to anxiety and depression, so these people tend to have negative affect. Higher levels of self defeating humor thus predict lower levels of self esteem and higher levels of depression. If these people develop higher self esteem, it improves their positive affect, and thus improves their interpersonal relationships.
Methods. The study used the Rosenberg self esteem scale (questionnaire that rates participants on a 10 item scale, like a Likert scale), the Humor Styles Questionnaire, and the Positive and Negative Affects Schedule. The study tried to demonstrate humor style can be predicted by self esteem. Self esteem is a personality trait, not just a simple positive evaluation of the self; personality traits are dynamic characteristic patterns of thoughts, emotion, and actions that make a person similar or different from others. Humor comes in four types for the purposes of this study: self enhancing, affiliative, aggressive, and self defeating. The first type helps reduce conflict while protecting the self, and is a coping strategy. The second promotes interpersonal bonding, the third is used as a way to criticize others, and the last occurs when people allows themselves to be the butt of jokes for the sake of forming interpersonal bonds, but they are in fact hindering positive feelings about themselves.
Crozier, W. Ray. “Measuring Shyness: Analysis of the Revised Cheek and Buss Shyness Scale.” Personality and Individual Differences, vol. 38, no. 8, 2005, pp. 1947–1956., doi:10.1016/j.paid.2004.12.002.
Farber, Barry A. Self-Disclosure in Psychotherapy. Guilford Press, 2006.
Kang, Sin-Hwa, et al. “Understanding the Nonverbal Behavior of Socially Anxious People during Intimate Self-Disclosure.” Intelligent Virtual Agents Lecture Notes in Computer Science, 2012, pp. 212–217., doi:10.1007/978-3-642-33197-8_22.
Kang et al. (2008). “Agreeable People Like Agreeable Virtual Tutors”.