My team and I have decided to put our annotation manuals into practice. My manual that dictates how to code for meta-cognitive behaviors can be found at this link: https://docs.google.com/document/d/1t1ShtJwWvxF1ADDGEWjsZxGJEts5y2W0Zs8YL8b5iio/edit?usp=sharing
To recap, my meta-cognitive behavior manual was written based on prior research that suggested that students who are able to recognize that they are in a Stuck state are able to avoid negative emotions (negative affect), such as frustration, and are able to develop more intrinsic motivation due to the fact they develop a strategy to break out of the Stuck state. Evidence of this is that there is a correlation between students who realize they are stuck and those who are exerting more effort into the problem at hand.
For that past week, I used a few hours to continue revising my manual and I made it neater, by adding a table and more sources. I also spent another couple hours using my colleague’s well-written manual, to code for the behaviors she saw as interesting. The behaviors she was focusing on were question-answer pairs, because she is studying reciprocity. I recommend reading her blog about the topic of reciprocity, created for the CREU program, because it is expertly crafted (https://michelinaastlecreu.wordpress.com). Anyway, after coding for both my behaviors and Mimi’s behaviors of interest, my team met for another two hours this week to discuss how we can improve our manuals.
Two major areas that were challenging to improve were that Mimi’s definitions of question-answer pairs was a little vague. For example, though she wanted pairs of questions and answers to be related to the problem being tutored pedagogically, some questions were in a grey zone where they didn’t have enough content or were ambiguous. These edge cases are important to discuss in any application of computer science, and I’ve noticed how important it is to review definitions that I’ve made in order to ensure they are actually applicable to the real world. I think this experience with CREU is helping me understand the difference between a theory created in my head and the real world, which is namely that the real world contains many exceptions and edge cases that even the best theories sometimes cannot account for.