This project involved the development of a Smart Query System for Pubs Advisory Service, a tool designed to effectively process and analyse complex legal documents. We specifically employed several strategies including Retrieval Augmented Generation (RAG) to ensure both accuracy and contextual relevance in the information extracted from the documents.

Users can effortlessly send large volumes of documents to an email address and receive the extracted information in the desired format. This AI service integrates external services like Google Sheets to allow users to customise their queries and toggle parameters. Overall, our AI solution has dramatically streamlined our client's administrative tasks, enabling them to refocus their time and energy on their legal tasks instead.

Pubs Advisory Service specialises in providing expert guidance, information, and representation services to a diverse range of clients within the pub sector. With a solid reputation as the lead advisory and guidance organisation for pubs in the UK, Pubs Advisory Service proudly holds a prominent market position and is the go-to choice for pubs seeking comprehensive legal support.

People entering the Licensed Trade sector need detailed independent advice on a wide range of issues. Access to such pre-entry advice can prove invaluable for conducting thorough due diligence and making informed decisions, ultimately setting the stage for successful pub ownership and prosperous growth. However, because each commercial lease agreement is unique and requires individual attention, this detailed advice can be costly and time-consuming to prepare.

Previous methods to perform document searches on commercial lease agreements often involved scanning them – using Adobe's inbuilt Optical Character Recognition (OCR) to transcribe handwriting – and performing traditional keyword searches (e.g. ctrl+F) throughout the PDF. Unfortunately, this approach had several limitations:

• OCR technology often struggled with handwritten text and poor-quality scans
• Keyword searches failed to understand context or find relevant information expressed in different terminology
• Manual review was time-intensive and prone to human error
• Information was often scattered across multiple sections, requiring extensive cross-referencing

The Smart Query System we developed enables document analysis through a straightforward process:

1. Users email their lease documents to a designated address
2. The AI system processes the documents using advanced RAG techniques
3. Users receive extracted information with direct citations and references
4. Results can be customised through Google Sheets integration

This solution provides a scalable, cost-effective approach to lease analysis that integrates seamlessly with existing workflows.

Consultancy

We began by conducting in-depth consultations with Pubs Advisory Service to gain a comprehensive understanding of their industry and the obstacles they encounter. OpenKit's main goal was to explore how the current landscape of generative AI could be harnessed to enhance productivity and overall business performance for Pubs Advisory Service, and provide them with a substantial competitive edge.

This unique solution required thorough exploration and prototyping. As such, a portion of the project was allocated to experimenting with various approaches, tools and methodologies.

First, OpenKit analysed the business landscape and gained a deep and nuanced understanding of the client's day-to-day business activities. We maintained an open dialogue with the client, discussing the different tools and services that we could offer, ranging from Smart Query Systems to advanced autonomous agents. Because the tool had to be user-friendly, integrating easily into the client and their customers' daily workflow, we settled on using an email interface through which users could effortlessly send questions and receive answers. We determined that avoiding the need to build a new interface from scratch would significantly reduce development time and costs for our client.

This collaborative process enabled us to establish a clear project goal and scope, which guided our research and development efforts, resulting in a refined final product.

After the consultancy period, we developed a demo system in order to showcase the potential of the proposed solution and validate our approach. During this development phase, OpenKit carefully considered various strategies to optimise both functionality and performance. We employed several techniques including Retrieval Augmented Generation (RAG) systems to ensure that the AI responses are accurate and grounded in the document content.

The RAG approach was crucial in preventing AI hallucinations and ensuring that all extracted information could be traced back to specific sections of the original documents. This was particularly important for legal applications where accuracy and verifiability are paramount.

The Smart Query System incorporates several key features designed to meet the specific needs of legal document analysis:

We developed this Smart Query System iteratively, meaning we alternated between testing and development phases. This open exchange allowed us to make the system much more robust and optimise it for live use. After Pubs Advisory Service first began testing the system's performance, they provided detailed feedback. Following their report, we refined and improved the system, ensuring their maximum satisfaction.

Notable improvements included allowing users to customise their prompts without the need for any technical expertise. We did this by integrating a signposted Google Sheets interface through which they could update their questions and toggle certain parameters:

We faced a range of challenges during the completion of this project, which allowed us to demonstrate our expert problem-solving abilities.

1. Challenge: Creating a scalable, low-cost administrative tool.

   Solution: We designed a solution that utilised cloud-based services and serverless architecture to ensure cost-effectiveness and scalability.

2. Challenge: Ensuring accuracy and preventing AI hallucinations.

   Solution: Implementation of RAG (Retrieval Augmented Generation) techniques that ground responses in actual document content, providing verifiable citations for all extracted information.

3. Challenge: The system needed to accommodate high user volumes whilst maintaining performance.

   Solution: OpenKit designed and developed a serverless scheduling system which queued all requests upon reception, allowing it to process them individually. This ensured a reliable, failsafe performance even when handling a dynamic amount of concurrent documents.

Our client tested our AI solution against similar existing market solutions. The results were extremely positive, as our solution outperformed competitors in terms of accuracy, reliability, and cost-effectiveness. This evaluation solidified our solution's position as the preferred choice for Pubs Advisory Service.

• Overcoming complex challenges to build a bespoke, innovative AI solution
• Outperforming existing services on key factors, including performance and price
• Establishing a standard for high scalability and reliability
• Optimising the ease of use by relying on well-known email and spreadsheet interfaces

Pubs Advisory Service was highly impressed by the results of our collaboration and has expressed an eagerness to continue working with OpenKit on several forthcoming projects.

• OpenAI LLM models
• Supabase
• Typescript
• Node.js