Welcome To The World Of Walt 2.0

By Author: D Edward Levy
Total Articles: 49
Comment this article

TTS is excited to announce the launch of the second generation of WALT, the lead training agent for the hospit-AI-lity travel training platform. Still in the evaluation phase, WALT received an array of back-end and user facing improvements that has improved his speed and response effectiveness. As the technology moves towards scaling and eventual deployment, accuracy and efficiency can be tracked and managed with more detail while WALT’s overall performance improves with more detailed user facing responses. WALT 2.0 is available for evaluation and your feedback is greatly appreciated. Read on for details in the upgrades and scroll to the bottom for the news release to access WALT and take him for spin.

Improved Data Management

At the heart of WALT is a RAG pipeline that pulls resources from a static set of documents within a knowledge base. The components that make up the framework can each be optimized for better performance and accuracy. Beginning with the resources themselves, improved document structuring will ensure more efficient loading and processing. Chunking documents is a core RAG process and optimizing ...
... chunk size and overlap improves chunk structure and delivers more accuracy when vectorizing. Upgrading the embedding model vastly improves vectorization and response targeting with varying user language. These upgrades are the springboard for eventual scaling and field deployment as more adjustments and optimizations will further increase WALT’s response accuracy.

WALT:
Worker
Augmented
Localized
Trainer

Automated and Visualized Evaluations

Evaluations are a critical process in the development of RAG systems and WALT 2.0 sees across the board improvement and upgrades in the evaluation sphere. An automated Evaluation Console has been engineered to check and validate current setups and optimizations and provides scores and graphical representations of important metrics such as Mean Reciprocal Rank (MMR), Normalized DCG (nDCG) and Keyword Coverage. WALT’s question and response performance can be monitored and validated in real time and results actioned much more quickly during the evaluation phase. Evaluations after script changes and knowledge base updates can be performed more effectively and practically with the establishment of the evaluation console. Please reach out for WALT’s current MMR, nDCG and KC statistics. The user facing facet of WALT will now embed a ‘Document Source Window’ that displays which document WALT chose to answer the user’s inquiry. This will display in real time and as the inquiry is processed and displays RAG action in real time. This will provide valuable feedback as WALT’s knowledge base grows and becomes more detailed.

Better User Experiences at the Evaluation

WALT 2.0 will also get a new front end for a much better user and evaluation experience. The addition of the Document Source Window will keep users informed on the resources WALT picks to answer their inquiries. A large screen, scroll type chat layout will keep older responses on screen for user reference. Additional graphical and branding elements will keep the environment clean and easy to navigate when interacting with WALT. An updated Gradio U/I will ensure the evaluation will be available across any device or browser type. Users will have the option to have their conversation transcript with WALT emailed to them with just 2 clicks. Access to Walt and the hospit-AI-lity Evaluation Console are available upon request.

hospit-AI-lity Chugs Towards Scale and Deployment

Ongoing evaluation of Walt will increase his accuracy and will eventually segue into user facing questions being placed and ‘mini-tests’ given. Major checkpoints ahead include migrating to a local LLM model such as Ollama with continued evaluation and testing. WALT and hospit-AI-lity will be deployed in the field as a mobile device app, ideally on a tablet sized device, leading to the eventual minification of the platform and full functionality on handheld devices. A full basic guest services training map is currently under development and will be assigned as the primary resource set for WALT, tracking and managing performance with a larger volume of documents with much more detail and information. WALT 2.0 evaluation is live and feedback is requested. Access WALT here: https://traveltechnologysolutions.net/EvalPortal/#haiy