Literature Review - 3

 Here I just wrap-up, in a table, the different machine learning algorithms used in some papers I read. I selected only the experiments relatively closed to ours and where the machine learning analysis was described. 

Table

Notation

ANN: Artificial Neural Network

BN: Bayesian Network

FP: False Positive (rate)

GMM: Gaussian Mixture Models

GP: Gaussian Process
KNN: K Nearest Neighbors
HMM: Hidden Markov Model
SVM: Support Vector Machine
TP: True Positive (rate)

References

[1] Analysis and detection of cognitive load and frustration in drivers’ speech, H. Bořil, S. O. Sadjadi, T. Kleinschmidt, J. H. L. Hansen

[2] Analysis of real-world driver’s frustration, L. Malta, C. Miyajima, N. Kitaoka, K. Takeda

[3] Automatic prediction of frustration, A. Kapoor, W. Burleson, R. W. Picard, 2006

[4] Affective Intelligent Car Interfaces with Emotion Recognition, C. L. Lisetti, F.Nasoz, 2005

[5] Sitting posture analysis by pressure sensors, K. Kamiya, M. Kudo, H. Nonaka, J. Toyama

[6] Automated Posture Analysis for Detecting Learner’s Interest Level, S. Mota, R. W. Picard

Data - 1

As in every machine learning project, have good data and features is the most sensible part of my project. There are several steps to comply:

Define which data we need:

The first thing is obviously to try to find out which kind of data we will need. That was already defined when I joined the project and they are listed in the previous article: Sensors - 1

Data acquisition:

Here again you can refer to Sensors -1 and -2 to see how the data will be collected.

An other thing we need to define while deciding how to do the data acquisition (other than the tools we will use) is the frequency with which we will collect the data. In my case, each time that the value of one sensor changes, the new value is collected. So, I will be able to have a continuous curve for the value of each sensor (in fact I am limited by the frequency of the sensor).

Features identification and extraction:

The features identification is one of the hardest part of my project. Even once you have collected good data, you need to identify the good features in order to have a successful machine learning analysis. Hopefully, the literature review provided me some examples of similar work and so some trails to follow. I already described some of them  in Literature review - 2, but the ones described there only deal with static data and according to D'Mello et al. [1] or Mota et al. [2] body movement is more useful to detect person affect rather than static body posture. So, I will try to adapt their work (basically they use difference between pressure detected at different time) in order to recognize driver's movement.

Labeling data

As I use supervised machine learning algorithms, I need to obtain labeled data. This step can be tricky while dealing with cognitive load because it is hard to know the ground-truth. Even for us it is hard to evaluate our emotion and usually an other person will not evaluate his emotion in the same way that we do. But still we need to label our data so here are some possibilities:

• self-reported: the user evaluates himself his emotions. This is certainly the most common way to obtain labeled data. A scale can be defined in order to improve the consistency of the data. Here again two methods are possible:
• Using self-report on real time: While the user is doing the experiment we provide him a way to evaluate his frustration. Choosing this method we have to provide very easy way for the user to report his frustration because we must ensure to not distract him.
• Using self-report a posteriori: Once the experiment is over, we review a video of the experiment with the user and ask him to evaluate his emotion. The problem is that the user can forget or misinterpret some of his emotions.
• external judge: We ask a person who is used to deal with human frustration to see the video of the experiments and judge the level of frustration of the user. The problem is that even for the expert we do not know if they are better than the user to evaluate user's frustration.
• mixed sources: Finally we can use several sources and select the data where the different sources agree on. The Cohen's kappa coefficient [3] can be used to evaluate the agreement of the different sources.

References
[1] Automatic detection of learner's affect from gross body language, S.D'Mello, A. Graesser, 2009
[2] Automated Posture Analysis for Detecting Learner’s Interest Level, S. Mota, R. W. Picard
[3] A coefficient of agreement for nominal scales, J. Cohen, 1960

Literature review - 2

As I planned last week, I tried to focus my research on papers dealing with seated posture and emotion and I did not find any new paper which linked the both. But I found some very interesting papers dealing with seated posture and how to analyze it.

For example, Andreoni et al. [1] are analyzing the posture of car drivers using the pressure and the ELITE System for 3D kinematic measurement of posture in order to improve the comfort of the driver's seat.  The final goal and the sensors are not the same as mine, but this paper give me the idea to extract some posture angle for the machine learning analysis. I think that I will be able to do that using some distance sensors well positioned.

In their paper, Kamiya et al. [2] use a pressure sensor to detect the posture of the person (not a driver), they achieve very good results, but my main interest in this paper was about the data preprocessing. They explain very well how they proceeded and it sounds very good. In particular, they use the total pressure exerted at a reference position to divide all the data and so to take the weight of the person into account.

Tan et al. [3] use the body pressure on driver's seat to know which action the driver is is doing. Turning, accelerating ... This study is quite promising for me because it shows that if my sensors are well positioned and I find the rights features (the features extracted are not described in this paper) I should be able to detect what the driver is doing and so it provides me a way to validate the good position of my sensors. Then I guess that if I am able to monitor what a driver is doing on his seat, it could be possible to detect some movements reflecting his mental state.

The last paper is written by Lee [4], in this paper Lee tries to see how two distinct groups (old and experimented drivers/young and novice drivers) will react when some frustration events occurs while they are driving on a simulator. This study is useful for me because the context of the experiment is quite similar to mine and then it provides some interesting ideas in order to design the user experiment.

References

[1] Method for the analysis of posture and interface pressure of car drivers, G. Andreoni,G. C. Santambrogio, Marco Rabuffetti, A. Pedotti, 2001

[2] Sitting posture analysis by pressure sensors, K. Kamiya, M. Kudo, H. Nonaka, J. Toyama

[3] The chair as a novel haptic user interface, H. Z. Tang, I. Lu, A. Pentland, 1997

[4] Measuring drivers’ frustration in a driving simulator, Y. Lee, 2010

Sensors - 2

I was mainly working with the sensors this week. I try to find out where we should position the weight sensors on the seat. One of the problem is that I have only two sensors yet, so I had to run several tests to know where they are useful or not. The difficulty is that in order to obtain comparable results, I have to act exactly in the same way while testing the sensors. I quickly designed a protocol with some classic drivers' actions and I try to run the tests.  The results for the different positions were hardly comparable but I still managed to choose some potentials interesting positions for the sensors on the backrest.

After discussion with Ronnie, he suggested me to do a timed Powerpoint presentation in order to know exactly how long each action has to go on. So, I did that for the tests on the cushion and it works much better. The results were much more similar so I could easily select some potential interesting positions for the sensors.

In an other hand, I improved the acquisition program, the new version is not totally working yet but  it should not take me to much time next Monday or Tuesday.

Test protocol

1. Define some classical drivers' actions 
2. Find a position for the sensor
3. Run the experiment and collect the data
4. While all potentials interesting positions are not tested go back to 2.
5. Normalize the data
6. Select the most promising data (the ones which provide different result and if several positions give the same patterns choose the one with the biggest amplitude after normalization)

Further work

• Do the same with the distance sensors 
• Run the experiment with other people with different morphology and see if we still have interesting results. I will certainly rune the experiments with few people and see if I can detect which movement they are doing based on the sensors' results. So, I will run some machine learning algorithms for a classification problem, if I get a good percentage of recognition we will consider that the sensors are well positioned and that we find some interesting features to analyze.