ABSTRACT

In this paper we have proposed an automatic speech recognition framework using agents. In this we have included both audio recognition and visual recognition. The audio and visual modalities are complementary to each other and the combination of the two can improve the accuracy in affective user models. The audio features extracted are processed by audition agent. The visual processing agent takes care of the lip and face detection. Finally both these agents assist audio visual fusion agent in fusion of these modalities for automatic speech recognition.