• AI-Based Voice-Enabled Chatbot Solution for Training Restaurant Staff

AI-Based Voice-Enabled Chatbot Solution for Training Restaurant Staff

CUSTOMER: A pioneer of the limited-service food industry
INDUSTRY: Retail – Food Industry TECHNOLOGY USED: Amazon Textract, Amazon Translate, Amazon Lex, Amazon SES, AI, NLP, Python

About the Client

The client is a bona fide pioneer of the limited-service food industry in the United States. They have been serving customers throughout the country since 1961 and are one of the largest hot dog chains. The company operates and franchises about 350 quick-service restaurants (QSR) in California and a dozen others mostly in the western states. About 90% of the company’s locations are operated by franchisees, serving some of the world’s most popular restaurant brands.

Business Need

The client had various PDF manuals on a central WordPress server. All these PDF documents were only available in the English language and constantly referred to by the client’s vendors/franchises for standard operating procedures, policies, hamburger preparation guidelines, and more. However, it was difficult for the users to search for specific information through these various PDF documents.

Additionally, the QSR chain had a floating staff, and hence training them on the various operations of the restaurant was a challenging task for the L&D team.

The following were the significant challenges faced by the L&D team:

  • The information was scattered across several documents
  • The documents did not have specific formats and contained information as images, in tables and text.
  • The staff came from multiple linguistic backgrounds
  • The staff used email to ask questions or seek responses to their queries, and the process was person dependent

Harbinger’s Solution

Harbinger proposed an AI-based chatbot solution so that users can ask any question or text the chatbot for the information required. The chatbot searches through all the PDF documents using natural language processing (NLP) to understand the context of the question/text and provide the user with the correct answer(s) along with the document PDF link and the page number.

Harbinger formulated the solution for the design and development of the chatbot based on three parameters:

  • 1) PDF document processing
    • Keep a track of all newly uploaded and modified documents
    • Parse the documents for better searchability
    • Map indexed document content with original documents, pages, and sections
  • 2) Datastore
    • Use datastore capable of text data search
    • Search and filter using dynamic attribute and search criteria
  • 3) Conversational interface
    • Integrate the chatbot with the existing WordPress setup
    • Enable users to interact with the chatbot through WebUI
However, the PDF documents did not have any standard format to read and extract text from those. The solution therefore included developing different parsing logic in Node JS to read the text and store it in the database.

To overcome the other challenges, Harbinger eLearning experts implemented the AI-based chatbot solution with the following unique differentiators:

  • The chatbot was voice-enabled and could take queries and respond in multiple languages
  • The chatbot was able to search through documents from many sources and deliver precise information about the query
The following technologies were used for this solution:

AWS Textract

This technology was used for extracting the PDF data to JSON. This extracted text from images and tables embedded in the PDF document.

Python APIs

This technology was used for fetching the accurate answer from the Elasticsearch response. AWS Elasticsearch helps users find relevant data with a fast, personalized search experience within the chatbot.

AWS Translate

This technology was used for translating answers. For instance, in case of Spanish input, Amazon Translate service was used to get the English translated text for search operation and response.


This technology was used for conversational interfaces using voice and text. The audio input was received by the bot by using the device microphone and MediaDevices API.


This technology was used for sending emails for the reports generated.

With this solution, the backend engine of the bot searched through the PDFs and provided the best matching answer. In case of more than one matching document or answer found, the chatbot listed the top few matching records.

Business Outcomes

Harbinger successfully designed, developed, and implemented the AI-based chatbot to help search information from the client’s PDF documents by the way of a search phrase or query sent to the chatbot.

  • The chatbot was voice-enabled and could take queries and respond in multiple languages. This not only helped minimize individual dependency but also overcome linguistic barriers.
  • The client is very happy with the bot. As the next step, they are in discussion with Harbinger for voice AI-based device integration

Do you have a similar case to discuss? Let’s talk!