Context-aware Conversational Management System
This interactive summary explores research by Hoashalarajh Rajendran et al. presented at IEEE R10-HTC 2023. The study addresses the challenge of robots providing repetitive or context-blind answers by introducing a Response Cloud integrated with a machine learning-based context classifier.
This research was conducted during a pivotal transitional period in conversational AI, which was the beginning of the public emergence of Large Language Models (LLMs) . While LLMs demonstrated unprecedented generative capabilities, their black-box nature made it difficult to isolate exactly what users valued in a contextual interaction.
To systematically investigate this, we engineered a pilot system that decoupled context-awareness from text generation. The system successfully classified user context but deliberately utilized similarity-based response mapping rather than generative outputs.
Beyond Hard-coded Dialogue
Traditional HRI systems often rely on static query-response pairs. If a user asks the same question twice, the robot responds identically, lacking "social memory." This research proposes a Response Cloud that structures dialogue into Domains and Intents, allowing the robot to select different responses while remaining contextually relevant.
Context Labeling
Classifies user queries into domains like 'University', 'Health', or 'General'.
Response Cloud
Hierarchical tree structure: Domain → Intent → Response Set.
Dynamic Updating
Enables users to update the cloud during live interaction.
Research Highlights
-
✓
92.5% Accuracy
Achieved using Random Forest Classifier for text-based context classification across multiple domains.
-
✓
Improved context classification with friendly responsive interactions
Evaluated via user study focusing on 'Intelligence' and 'Interaction Comfort'.
-
✓
Cosine Similarity Matching
Utilized Cosine Similarity for efficient intent mapping within the Response Cloud.
The CMS Pipeline
The Conversational Management System (CMS) bridges raw speech recognition and response generation through context filtering. Click a module to see how it handles human queries.
Input Layer
User Query
Preprocessing
Context Engine
Context Classifier
Method: Random Forest
Feature: TF-IDF Vector
Response Layer
Response Cloud
Traverses the specific domain tree to find matching intents and varied responses.
Interaction: Click a pipeline component above to view technical specs.
The Response Cloud Logic
How does the system select the "best" response within a domain? It uses similarity indices and a hierarchical tree traversal.
1 Intent Matching
User queries are compared against predefined 'Key Queries' in the cloud using the Cosine Similarity Index in the paper. Here is another Index called Jaccard Similarity Index that worth knowing about how it functions. For our puropose Cosine Similarity Index suited much better than Jaccard Similarity Index.
Example:
Query: "What is your name?"
Target: "Tell me your name"
Intersection: {your, name} → Match Found!
2 Response Variation
To avoid the "I already said that" problem, each intent node contains a set of responses. The CMS tracks the previous response index.
// Response Set for Intent "Greet"
R1: "Hello! How can I help?"
R2: "Hi there! What's on your mind?"
R3: "Greetings! Ready to assist."
By implementing a modulo-based selection algorithm within the Response Cloud, the system rotates through a set of unique responses for each intent, maintaining dialogue freshness and minimizing interaction fatigue.
Experimental Results
Performance was measured both quantitatively (Classification Accuracy) and qualitatively (User Study on Interaction Quality).
User feedbacks on the scope for further improvement
User Study Results (1-5 Scale)
28 participants evaluated the system across four key metrics. A score > 4 indicates high user satisfaction.
According to user responses, the pilot system is performing well in predicting and classifying appropriate contexts; on the other hand, it needs significant improvement in smooth dialogue transitions.