Challenge

Fly Me is a travel agency offering flight booking services for private or professional customers.

In order to make the booking process easier, we want to automate it with a chatbot.
To be efficient, the bot must be able to :

• interact with the user in a natural way
• recognize the user's intent (book a flight, ...)
• recognize the desired caracteristics of the trip (origin and destination cities, ...)
• ask for more information if necessary
• allow the user to cancel the booking process
• raise an alert if the booking process fails too often

Goals

In this project, we are going to :

• deploy a chatbot in the cloud
• train a language understanding model
  • in order to infer the user's intent and trip caracteristics from the user's input
• implement unit and integration tests
  • to ensure that the chatbot is working as expected
• monitor the chatbot's performance
  • raise an alert in case of recurring issues

Exploratory data analysis

For this MVP, we used the Azure Frames Dataset :

• 1369 dialogues between a human and a bot, composed of ~15 turns on average
• the bot can :
  • ask the user for information
  • suggest a trip to the user
  • ask the user to confirm
• the user can :
  • ask the bot for suggestions
  • inform the bot about relevant information
  • cancel or confirm the trip

Dialog example

Step 1 : LUIS Model training and deployment

The first step in this project was to train and deploy a language understanding model using the LUIS natural language service :

• create an Azure LUIS resource
  • add the intents and entities to the LUIS model
• format the raw data to be compatible with Azure LUIS
  • add the examples to the LUIS model
• run the LUIS model training
• deploy the model to the cloud

Step 2 : chatbot development and deployment

The second step in this project was to develop and deploy a Chatbot :

• create a Bot Service resource
• implement the bot with the Bot Framework SDK
  • implement dialogs : welcome, gather intention, ask informations, confirm, cancel, ...
  • integrate the LUIS service
  • add unit and integration tests
    • test the LUIS service
    • test the LUIS service integration
    • test a standard dialog
• deploy the bot to the Azure Web App

Step 3 : chatbot monitoring

The final step in this project was to monitor the chatbot's performance :

• create a Application Insights resource
• log a Telemetry trace every time the bot is used (booking_accepted or booking_refused)
• add an Alert if the bot fails too often :
  • if More than 5 Booking Refused last 5 minutes
  • then Send Email and SMS to admins

Curent MVP System Architecture

Resources inventory

Demo

Demo	Monitoring

Target performance management policy

In order to achieve the target performance, the following policy must be implemented :

• store each dialogs and inferred intents and entities in a database
• transform and load (ETL) the raw data in a datawarehouse in a format compatible with the LUIS service
• daily re-train the language understanding model with the new intents and entities of the successful bookings

Target production architecture

Next steps

• integrate the bot with multiple Channels (Website, Discord, Teams, Slack, ...)
• improve the bot capacity to handle more Intentions and Entities
• connect the bot to an actual Flight booking system
• monitor more precisely the bot's performance : errors, performance, availability, ...
• implement the model continuous training and deployment