Skip to Main content Skip to Navigation
Conference papers

A Gaussian mixture representation of gesture kinematics for on-line Sign Language video annotation

Fabio Martínez 1, 2 Antoine Manzanera 1 Michèle Gouiffès 2 Annelies Braffort 3
2 AMI - Architectures et Modèles pour l'Interaction
LIMSI - Laboratoire d'Informatique pour la Mécanique et les Sciences de l'Ingénieur : 247329
3 ILES - Information, Langue Ecrite et Signée
LIMSI - Laboratoire d'Informatique pour la Mécanique et les Sciences de l'Ingénieur
Abstract : Sign languages (SLs) are visuo-gestural representations used by deaf communities. Recognition of SLs usually requires manual annotations, which are expert dependent, prone to errors and time consuming. This work introduces a method to support SL annotations based on a motion descriptor that characterizes dynamic gestures in videos. The proposed approach starts by computing local kinematic cues, represented as mixtures of Gaussians which together correspond to gestures with a semantic equivalence in the sign language corpora. At each frame, a spatial pyramid partition allows a fine-to-coarse sub-regional description of motion-cues distribution. Then for each sub-region, a histogram of motion-cues occurrence is built, forming a frame-gesture descriptor which can be used for on-line annotation. The proposed approach is evaluated using a bag-of-features framework, in which every frame-level histogram is mapped to an SVM. Experimental results show competitive results in terms of accuracy and time computation for a signing dataset.
Complete list of metadata

Cited literature [20 references]  Display  Hide  Download

https://hal.archives-ouvertes.fr/hal-01245123
Contributor : Antoine Manzanera Connect in order to contact the contributor
Submitted on : Wednesday, December 16, 2015 - 5:16:40 PM
Last modification on : Saturday, December 4, 2021 - 4:03:55 AM
Long-term archiving on: : Saturday, April 29, 2017 - 5:14:08 PM

File

isvc15.pdf
Files produced by the author(s)

Identifiers

  • HAL Id : hal-01245123, version 1

Citation

Fabio Martínez, Antoine Manzanera, Michèle Gouiffès, Annelies Braffort. A Gaussian mixture representation of gesture kinematics for on-line Sign Language video annotation. International Symposium on Visual Computing ISVC'15, Dec 2015, Las Vegas, United States. ⟨hal-01245123⟩

Share

Metrics

Record views

287

Files downloads

416