Data - 1

As in every machine learning project, have good data and features is the most sensible part of my project. There are several steps to comply:

Define which data we need:

The first thing is obviously to try to find out which kind of data we will need. That was already defined when I joined the project and they are listed in the previous article: Sensors - 1

Data acquisition:

Here again you can refer to Sensors -1 and -2 to see how the data will be collected.

An other thing we need to define while deciding how to do the data acquisition (other than the tools we will use) is the frequency with which we will collect the data. In my case, each time that the value of one sensor changes, the new value is collected. So, I will be able to have a continuous curve for the value of each sensor (in fact I am limited by the frequency of the sensor).

Features identification and extraction:

The features identification is one of the hardest part of my project. Even once you have collected good data, you need to identify the good features in order to have a successful machine learning analysis. Hopefully, the literature review provided me some examples of similar work and so some trails to follow. I already described some of them  in Literature review - 2, but the ones described there only deal with static data and according to D'Mello et al. [1] or Mota et al. [2] body movement is more useful to detect person affect rather than static body posture. So, I will try to adapt their work (basically they use difference between pressure detected at different time) in order to recognize driver's movement.

Labeling data

As I use supervised machine learning algorithms, I need to obtain labeled data. This step can be tricky while dealing with cognitive load because it is hard to know the ground-truth. Even for us it is hard to evaluate our emotion and usually an other person will not evaluate his emotion in the same way that we do. But still we need to label our data so here are some possibilities:

• self-reported: the user evaluates himself his emotions. This is certainly the most common way to obtain labeled data. A scale can be defined in order to improve the consistency of the data. Here again two methods are possible:
• Using self-report on real time: While the user is doing the experiment we provide him a way to evaluate his frustration. Choosing this method we have to provide very easy way for the user to report his frustration because we must ensure to not distract him.
• Using self-report a posteriori: Once the experiment is over, we review a video of the experiment with the user and ask him to evaluate his emotion. The problem is that the user can forget or misinterpret some of his emotions.
• external judge: We ask a person who is used to deal with human frustration to see the video of the experiments and judge the level of frustration of the user. The problem is that even for the expert we do not know if they are better than the user to evaluate user's frustration.
• mixed sources: Finally we can use several sources and select the data where the different sources agree on. The Cohen's kappa coefficient [3] can be used to evaluate the agreement of the different sources.

References
[1] Automatic detection of learner's affect from gross body language, S.D'Mello, A. Graesser, 2009
[2] Automated Posture Analysis for Detecting Learner’s Interest Level, S. Mota, R. W. Picard
[3] A coefficient of agreement for nominal scales, J. Cohen, 1960