- Korean Emotion Multimodal Database
in 2019 (KEMDy19)
KEMDy19
is a multimodal emotion data set that collects speech data, text data
transcribed from the speech, and bio-data such as the electrocardiogram (ECG),
electrodermal activity (EDA), and wrist skin temperature during the interactive
conversation process between two speakers.
The
collecting procedure of KEMDy19 is approved by the institutional review board
of the Korea national institute for bioethics policy (P01-201907-22-010).
Related emotion databases
·
KESDy18 : Korean Emotional Speech Dataset in
2018
https://nanum.etri.re.kr/share/kjnoh2/KESDy18?lang=En_us
·
KEMDy20 : Korean Emotion Multi-modal Database in 2020
https://nanum.etri.re.kr/share/kjnoh2/KEMDy20?lang=En_us
Collecting procedure
In
KEMDy19, 2 voice actors a pair one male and one female participated in one
session (20 sessions in total) for a total of 40 Korean voice actors (20
male/female each).
The
10 external taggers evaluated the speech segments while listening to the
recorded utterances. The annotators tagged one of seven categorical emotion
labels (“angry”, “sad”, “happy”, “disgust”, “fear”, “surprise”, “neutral”).
And, they tagged labels of arousal (low-high: 1-5) and valence-level
(negative-positive: 1-5) on a 5-point scale for each segment.
The
final categorical emotion label was determined by a majority vote. The label of
arousal and the valence-level were determined from the average value of the
levels tagged by the evaluators.
|
Directory
|
File
extension
|
Explanation
|
|
./annotation
|
.csv
|
Emotion label file tagged by
the external annotators by session/participant/utterance data
|
|
./wav
|
.wav /
.txt
|
Wav file of speech data in
session/emotional situation/(.wav)
Transcription text file of
speech data in session/emotional situation/(.txt)
|
|
./ECG
|
.csv
|
ECG data collected through
Refit device by session/participant
|
|
./EDA
|
.csv
|
EDA data collected through E4
device by session/participant
|
|
./TEMP
|
.csv
|
Wrist temperature data through
E4 device by session/participant
|
- ·
./wav/~/.txt : usage of special characters
|
Special
character
|
Usage
|
|
c/
|
Continuous vocalization
without a speechless period(less than 0.3 seconds)
|
|
n/
|
Episodic noises Included in
speech data
|
|
N/
|
Speech data contains more
than 50% noise
|
|
u/
|
Speech data that cannot be
understood verbally
|
|
l/ (small 'L')
|
Speech data includes
‘Um/Uh/mm’ sound
|
|
b/
|
Speech data includes breath
or cough sounds
|
|
*
|
Recognizing only some of the
words in speech data
|
|
+
|
Repetitive stuttering during vocalization
|
|
/
|
Interjection included in
speech data
|
- ·
./annotation/.csv
· ./ECG/session1~20/original/.csv
- col. A : 250Hz sampling period sequence
- col. B : Refit ECG value
- col. C : Time order of measurements in
the session
- col. D : Segment ID to which the
corresponding ECG value belongs
· ./EDA/session1~20/original/.csv
- col. A:
E4 EDA change value of consecutive samples
- col. B :
4Hz sampling period sequence
- col. C :
Time order of measurements in the session
- col. D :
Segment ID to which the corresponding EDA value belongs
· ./TEMP/session1~20/original/.csv
- col. A :
Wrist temperature of E4
- col. B :
4Hz sampling period sequence
- col. C :
Time order of measurements in the session
- col. D
: Segment ID to which the corresponding EDA value belongs
Publication
[1] K. J. Noh and H. Jeong, “KEMDy19,” https://nanum.etri.re.kr/share/kjnoh2/KEMDy19/update?lang=En_us
[2] Noh, K.J.; Jeong, C.Y.; Lim, J.; Chung, S.; Kim, G.; Lim, J.M.; Jeong, H. Multi-Path and Group-Loss-Based Network for Speech Emotion Recognition in Multi-Domain Datasets. Sensors 2021, 21, 1579. https://doi.org/10.3390/s21051579
[3] NOH, Kyoungju; JEONG, Hyuntae. Emotion-Aware Speaker Identification with Transfer Learning. IEEE Access, 2023.
