Attestation committee
Accreditation committee
Expert committee
Dispositions, instructions
Normative acts
MANPOWER TRAINING
Nomenclature
Institutions
Scientific councils
Seminars
Theses		 Scientific advisers
Scientists
Doctoral students
Postdoctoral students
CNAA logo

 română | русский | english


Human actions recognition with modular neural networks


Author: Albu Veaceslav
Degree:doctor of informatics
Speciality: 01.05.04 - Mathematical modelling, mathematical methods, software
Year:2016
Scientific adviser: Svetlana Cojocaru
doctor habilitat, professor, Institute of Mathematics and Computer Science
Institution: Institute of Mathematics and Computer Science

Status

The thesis was presented on the 9 November, 2016
Approved by NCAA on the 28 December, 2016

Abstract

Adobe PDF document – 0.86 Mb / in english
Adobe PDF document – 1.01 Mb / in romanian

Thesis

CZU 519. 95

Adobe PDF document 3.98 Mb / in english
168 pages

Keywords

Deep Neural Networks, Computer Vision, Emotion Classification, Gesture Classification

Summary

The thesis was elaborated at the Institute of Mathematics and Computer Science of the Academy of Sciences of Moldova, Chisinau, in 2016. The thesis is written in English and contains Introduction, 3 chapters, general conclusions and recommendations, bibliography of 109 titles. The main text amounts to 121 pages. This work includes: 37 figures, 2 tables, 44 formulas, and 5 annexes. The results are published in 8 scientific papers.

The area of the present studies is the field of emotion and action recognition using modular neural networks.

The aim and objectives of this research is to develop a tool for classification of human reactions (including facial features and body movements) into typical and non-typical in a certain environment. This tool provides statistical observations and measurements of human emotional states during an interaction session with a software product (or, optionally, with a hardware plus software complex).

Scientific novelty is a novel modular neural network architecture, constituted from two separate parts and combine the results to introduce the classification of the infrared sensor inputs, which is the first system of this kind, being applied both to emotion and human action recognition.

The important solved scientific problem is elaboration of a multimodal method for classification of human reactions (joining emotions and actions) into typical and non-typical in a certain environment, that ensures an effective functioning of systems destined to human actions monitoring in real time.

Theoretical significance. Our research solutions provide ground for solving of following problems: formulation of the tool’s architecture for robust classification of emotions and gestures of a human subject into typical vs. non-typical; the substantiation of the possibility and efficiency of using deep learning in an integrated approach for the detection of expression of the whole body in real time.

Practical value: this kind of classification task is very useful in different applications, where the number of gestures of the human is limited, such as: customers at the various types of automated machines, drivers, assembly line workers, hospital patients etc.

Summary


1. THEORY AND MODELS OF EMOTION AND ACTION RECOGNITION
  • 1.1. PSYCHOLOGICAL BASIS FOR EMOTION RECOGNITION
  • 1.2. MODELS OF EMOTIONS AND ACTIONS
  • 1.3. COMPUTER VISION MODELS FOR OBJECT RECOGNITION
  • 1.4. CONCLUSIONS

2. NEURAL NETWORK ARCHITECTURE AND LEARNING ALGORITHMS
  • 2.1. BASIC NOTIONS AND DEFINITIONS OF THE ANN THEORY
  • 2.2. MODULAR NEURAL NETWORKS
  • 2.3. DEEP NEURAL NETWORK
  • 2.4. CONCLUSIONS

3. RESEARCH APPLICATIONS AND PSYCHOLOGICAL EXPERIMENTS
  • 3.1. EXPERIMENTAL SETUP AND EQUIPMENT
  • 3.2. ATM EMULATION PROGRAM
  • 3.3. PSYCHOLOGICAL EXPERIMENTS: GROUP ONE
  • 3.4. PSYCHOLOGICAL EXPERIMENTS: GROUP TWO
  • 3.5. COMPUTATIONAL ENVIRONMENT AND IMPLEMENTATION
  • 3.6. COMPARISON TO RELATED WORK
  • 3.7. CONCLUSIONS

CONCLUSIONS AND RECCOMENDATIONS

GENERAL CONCLUSIONS

FUTURE WORK

BIBLIOGRAPHY