Open source tools for generative AI: KILN and vLLM

The shift to open source in generative AI

Generative AI started out dominated by proprietary, closed models from a handful of large tech companies. That has changed. Open-source models and tooling now match or beat the closed options for many jobs, and the shift is about more than cost: open tools give you control over how a model behaves, room to fit it to your own data, and a community that moves faster than any single vendor.

At OpenKit we build client systems on open-source tools, and we have tested enough of them to be opinionated about which ones hold up in production ²¹.

The main reason we lean this way is transparency, though it is worth being precise about what “open” actually buys you. Most models described as open publish their weights and their inference code rather than their training data or the process that produced them, so you can inspect and test how a model behaves without being able to audit how it came to behave that way. What open weights and open tooling do give you is the ability to probe a model for bias and safety problems on your own data, in your own environment, before anything reaches production. That matters more as AI moves into decisions that affect real people.

This guide covers the open-source tools behind modern generative AI development. It gives most attention to the two we rely on hardest: KILN AI for model evaluation and vLLM for efficient inference, which is central to how we run private AI deployments that keep data and models fully under a client’s control.

Categories of open-source generative AI tools

The open-source generative AI stack breaks into several categories, each covering a different part of the development lifecycle:

1. Model hubs and development frameworks

The Hugging Face ecosystem

The Hugging Face ecosystem sits at the centre of open-source AI. It grew from a model repository into a full platform and community for machine learning, particularly natural language processing (NLP) and generative AI ¹.

Key components of the Hugging Face ecosystem:

• Hugging Face Hub: Often dubbed the “GitHub of machine learning,” the Hub hosts pre-trained models (e.g., Llama, Mistral, Gemma) covering a wide range of tasks in NLP, computer vision, audio, and multimodal AI, alongside the datasets used to train new models or fine-tune existing ones. The counts move too fast to quote; check the Hub itself if the scale matters to your decision.
• Spaces: This feature allows users to create and share interactive ML applications and demos, typically built with Gradio or Streamlit. Spaces are invaluable for showcasing model capabilities, gathering user feedback, and facilitating rapid prototyping.
• Libraries:
  • Transformers: This flagship library provides access to thousands of pre-trained models (like BERT, GPT, T5) with utilities to easily download, configure, and use them for inference or fine-tuning ².
  • Tokenizers: A specialised library offering high-performance tokenisation, a critical preprocessing step for most NLP models.
  • Datasets: This library simplifies accessing, loading, and processing the vast array of datasets available on the Hub and elsewhere.
  • Accelerate: Designed to simplify running PyTorch training scripts across various distributed computing setups and enables Big Model Inference for models too large to fit on a single GPU.
• Open LLM Leaderboard: This community resource tracks, ranks, and evaluates open-source LLMs and chatbots across benchmarks like ARC, HellaSwag, MMLU, TruthfulQA, and Winogrande, providing an objective comparison of open LLMs ^{3, 4}.

OpenKit insight: At OpenKit, the Hugging Face Hub is our main source for foundation models, which we then adapt to specific client needs. We regularly use the Transformers library for model fine-tuning and inference in the early stages of development.

Application frameworks for building LLM-powered systems

Several frameworks have emerged to simplify the process of building applications around LLMs:

LangChain

LangChain has rapidly emerged as a prominent open-source framework for developing applications powered by LLMs. Its core philosophy revolves around composability, allowing developers to chain together various components to create sophisticated workflows ⁵.

Core components of LangChain:

• Models: Provides a standardised interface to interact with a multitude of LLMs.
• Prompts: Includes utilities for constructing, managing, and optimising prompts.
• Chains: Sequences of calls to LLMs, utilities, or other chains, allowing for multi-step processes.
• Indexes: Facilitate the structuring of data so that LLMs can effectively use it (especially for RAG).
• Memory: Enables LLMs to retain information from previous interactions.
• Agents: Lets an LLM decide what to do next and act on it, using tools like search engines or APIs.
• Callbacks: A system for logging, monitoring, and streaming intermediate steps.

LlamaIndex

LlamaIndex is another powerful open-source framework, but with a more specific focus: it is a data framework designed explicitly for building context-augmented LLM applications, with a strong emphasis on Retrieval Augmented Generation (RAG) ⁶.

Core components of LlamaIndex:

• Data Connectors (LlamaHub): A rich collection of connectors to ingest data from various sources (APIs, PDFs, SQL databases).
• Data Indexes: Tools to structure ingested data (e.g., vector stores) into representations optimised for LLM consumption.
• Engines (Query & Chat): Interfaces for question-answering and conversational interactions with your data.
• Agents: LLM-powered knowledge workers that can use tools, including RAG pipelines, to perform tasks.
• Workflows: Multi-step, event-driven processes combining various components for complex applications.

Feature Focus	LangChain	LlamaIndex
Primary Goal	General LLM application development	Building RAG & data-connected LLM apps
Key Strength	Versatility, complex agentic workflows	Data ingestion, indexing, retrieval for RAG
Data Handling	Supports data connection, more general	Specialised for connecting LLMs to custom data
Common Use Cases	Chatbots, summarisation, complex agents	Q&A over documents, knowledge bases, RAG
Table 1: LangChain vs. LlamaIndex at a Glance 22

How we pick a framework: While many developers rely heavily on LangChain, at OpenKit we primarily build custom architectures tailored to each client’s specific requirements. We occasionally use LlamaIndex for certain RAG applications where its data connectors and indexing capabilities offer efficiency advantages, but we often prefer purpose-built solutions that give us complete control over every aspect of the application flow for enterprise needs.

2. Evaluation suites

Generative AI is powerful, but accuracy, safety and reliability do not come for free. Without systematic evaluation you cannot measure performance objectively or know where to improve ⁷.

KILN AI eval framework, our primary evaluation tool

KILN AI is an accessible tool designed to simplify various stages of the LLM lifecycle, including fine-tuning, synthetic data generation, and, most importantly, model and task evaluation ²⁰.

Core evaluation functionality in KILN AI:

• Quality Assessment: Evaluates models and tasks using a suite of evaluators.
• Team Collaboration: Facilitates collaboration among team members on datasets, ratings, and results.
• Integrated Workflow: Integrates evaluation with fine-tuning and synthetic data generation.

Key evaluation metrics and methodologies in KILN AI ⁸:

Metric/Methodology	Description
Correlation Scores	Kendall Tau, Spearman, Pearson: Measure alignment of automated evals with human ratings.
Error Metrics	MAE, MSE (and normalised versions): Quantify deviation from ground truth.
Task-Specific Scores	Custom metrics (e.g., 1-5 stars, pass/fail) tailored to specific tasks.
LLM as Judge	Uses another LLM to evaluate outputs based on a defined rubric.
G-Eval	An advanced “LLM as Judge” using token probabilities for more nuanced scoring.
Golden Dataset Comparison	Benchmarks automated methods against expert-rated examples.
Table 2: Key Evaluation Metrics & Methodologies in KILN AI

OpenKit insight: KILN AI Evals is our primary evaluation framework at OpenKit. We particularly value its ability to correlate automated evaluations with human judgment, which is what keeps output at the standard our clients expect. For critical projects, we create “golden datasets” meticulously rated by domain experts, which serve as the gold standard for our automated evaluation pipelines.

Other evaluation tools worth knowing

The open-source ecosystem offers various other frameworks for LLM evaluation, each with unique strengths ^{9, 10}:

• DeepEval: A Python-native framework (“Pytest for LLMs”) with 14+ metrics for summarisation, hallucination, etc.
• RAGAs: Specialised for RAG pipelines, with metrics like Faithfulness and Contextual Precision.
• Promptfoo: CLI tool for systematic prompt testing, evaluation, and comparison.
• LangSmith (by LangChain): Observability and evaluation platform, good for bias and safety testing.
• Arize Phoenix: An open-source LLM observability tool with evaluation for Q&A accuracy and hallucination.
• Langfuse: Full-stack open-source LLM engineering platform (tracing, evaluation, prompt management).
• OpenAI Evals: Primarily for evaluating OpenAI models, supporting dataset-driven testing.

3. Inference and serving engines

Deploying models efficiently is where a lot of value is won or lost. These tools handle high-performance serving and local LLM deployment.

vLLM for high-performance serving

vLLM is an open-source library engineered for fast and memory-efficient Large Language Model (LLM) inference and serving. Its primary design goal is to maximise throughput and minimise latency when serving LLMs, particularly in scenarios with high concurrency ^{11, 12}.

Core concepts and innovations:

• PagedAttention: vLLM’s flagship innovation, inspired by OS virtual memory paging:
  • Divides the KV cache (attention keys and values) into non-contiguous blocks (“pages”) ¹³.
  • Virtually eliminates memory fragmentation, cutting waste from as much as 80% in traditional systems to under 4% ¹⁴.
  • Enables larger batch sizes, longer context windows, and efficient memory sharing (e.g., for parallel sampling or shared prefixes) ¹⁴.
• Continuous Batching: Instead of waiting for a full batch, vLLM processes requests dynamically as they arrive, adding them to the current batch. This maximises GPU utilisation and reduces average latency ¹².
• Optimised CUDA Kernels: Uses hand-tuned CUDA kernels for critical operations, further boosting performance on NVIDIA GPUs ¹².
• Broad Model & Feature Support: Compatible with many Hugging Face models (Llama, Gemma, Phi, Qwen, Mistral, etc.), supports tensor parallelism, various quantisation methods (GPTQ, AWQ, FP8), speculative decoding, and an OpenAI-compatible API server ¹¹.

OpenKit insight: vLLM is our preferred inference engine for production deployments. Its performance characteristics and memory efficiency let us serve multiple clients while keeping the systems responsive. We typically deploy vLLM within containerised environments hosted on client infrastructure or on secure cloud instances, so the data stays inside the environment the client controls.

For our legal document analysis platform BAiSICS, vLLM’s ability to handle long contexts efficiently has proven invaluable when processing complex legal documents that often exceed 50 pages.

llama.cpp for efficient local inference

llama.cpp is one of the foundations of the local LLM movement. It’s a C/C++ library for running LLMs with minimal setup, few dependencies, and strong performance across a wide range of hardware ¹⁵.

Key features:

• GGUF Format: A binary format designed for rapid loading and efficient storage of models, supporting various quantisation schemes.
• Quantisation: Extensive support for model quantisation (e.g., 2-bit to 8-bit integer quantisation, “k-quants”), dramatically reducing model size and memory requirements, enabling large models to run on consumer hardware.
• Broad Hardware Support: Runs on diverse hardware including Apple Silicon (Metal), NVIDIA GPUs (CUDA), AMD GPUs (HIP), and offers highly optimised CPU execution (AVX, AVX2, AVX512).
• CPU+GPU Hybrid Inference: Can offload parts of models to a GPU and run the remaining layers on the CPU, useful for models exceeding VRAM.
• Minimal Dependencies: Plain C/C++ implementation avoids complex dependency webs.
• OpenAI-Compatible Server: Includes llama-server which provides an HTTP server with an API compatible with OpenAI specifications.

NVIDIA TensorRT-LLM: While not fully open source in the same vein as llama.cpp, TensorRT-LLM is an NVIDIA-optimised open-source library for defining, optimising, and executing LLMs on NVIDIA GPUs with extreme performance. It includes advanced optimisations like INT4/INT8 quantisation, in-flight batching, and custom CUDA kernels ²³.

4. Local deployment UIs and toolkits

Command-line tools are powerful, but interfaces make running and managing local LLMs easier. Most build on backends like llama.cpp or Ollama, and the active community around local LLMs has produced a great many of them. The ones below are the ones we have actually used or evaluated; each name links to its own source.

Tool	Key Features	Primary Backend(s)	Target User
LM Studio	Desktop app (Win/Mac/Lin), easy model discovery (GGUF, MLX), chat UI, local server, RAG 17.	llama.cpp (for GGUF)	Beginners, quick experimenters
Oobabooga Text Generation WebUI	Gradio web UI, multi-backend (llama.cpp, Transformers), advanced parameters, chat modes, extensions 18.	llama.cpp, Transformers, ExLlamaV2	Experimenters, advanced users
Ollama	Lightweight server & CLI, Modelfile for customisation, OpenAI API 16.	llama.cpp	Developers, local serving
Open WebUI	Popular frontend, often paired with Ollama for a user-friendly chat experience.	Ollama, other API-compatible engines	Users wanting a good chat interface
KoboldCPP	llama.cpp-based UI focused on creative writing, roleplay, and story generation.	llama.cpp	Creative writers, roleplayers
SillyTavern	Character-focused chat UI, connects to various backends (KoboldCPP, Oobabooga API, OpenAI API).	Various (via API)	Character interaction, roleplay
Table 3: Local LLM interfaces and toolkits

OpenKit insight: While our production systems typically use vLLM, we find tools like llama.cpp (often via LM Studio or Oobabooga’s Text Generation WebUI) invaluable during development and testing. This approach allows us to quickly validate concepts and model performance on local hardware before scaling up to production-grade deployments with vLLM.

5. MLOps for generative AI

Running generative AI in production takes more than individual tools. It needs systematic MLOps (Machine Learning Operations) practices to manage the whole lifecycle, often called LLMOps when the focus is Large Language Models ¹⁹.

Key MLOps components for generative AI:

• Data Management & Versioning: Essential for training datasets, fine-tuning data, prompt engineering assets, and RAG knowledge bases. Tools like DVC (Data Version Control) help manage and version large datasets alongside code.
• Experiment Tracking: Logging prompts, model configurations, hyperparameters, evaluation metrics, and outputs is essential for reproducibility and comparative analysis. Platforms like MLflow and Weights & Biases offer components that integrate well with open-source workflows.
• Model Registries: Storing, versioning, and managing trained or fine-tuned models, along with their metadata and lineage. The Hugging Face Hub serves as a de facto public model registry, while tools like MLflow provide private registry capabilities.
• Automated Pipelines: Creating reproducible workflows for data preprocessing, model training/fine-tuning, evaluation, and deployment. Orchestration tools like Kubeflow and Apache Airflow can manage these complex pipelines.
• Monitoring: Continuously tracking model performance (accuracy, drift), data drift, output quality (e.g., toxicity, relevance), and operational health (latency, throughput, cost) in production environments.

How OpenKit uses KILN AI for model evaluation

At OpenKit, ensuring the quality, reliability, and ethical alignment of our AI solutions is paramount. Our evaluation process using KILN AI typically follows these steps:

1. Creating evaluation datasets

We develop test sets that cover the full range of expected inputs and edge cases for each client’s specific use case. For legal document analysis, this includes:

• Different document types (leases, contracts, legal opinions)
• Various document qualities (clean PDFs, scanned documents)
• Range of complexity levels and potential ambiguities

2. Establishing golden datasets with expert ratings

For critical applications, we create “golden datasets” with examples meticulously rated by human subject matter experts (SMEs):

• Legal professionals rate document summaries for accuracy and completeness.
• Domain experts evaluate factual correctness in specialised fields.
• Client stakeholders assess alignment with business requirements. By comparing how different automated evaluators in KILN score this golden set, we determine which automated method best correlates with human judgment ⁸.

3. Designing custom evaluation criteria

KILN AI allows us to create our own evaluation configurations with custom goals, rubrics, and scoring mechanisms. For our BAiSICS legal platform, we developed specialised evaluations for:

• Legal Accuracy: Assessing correctness of extracted legal information.
• Comprehensive Coverage: Ensuring all relevant sections are analysed.
• Citation Quality: Verifying accurate references to source material.
• Contextual Understanding: Evaluating comprehension of legal context.

4. Implementing continuous improvement cycles

Evaluation results feed directly into our development workflow:

• Failures and edge cases inform new training or fine-tuning examples.
• Successful patterns are reinforced.
• Confidence thresholds for AI outputs are calibrated based on evaluation results.
• Regular re-evaluation ensures continued performance as data and requirements evolve.

Case study: Pubs Advisory Service For Pubs Advisory Service, we applied KILN AI to evaluate the lease agreement analysis solution:

• Built a golden dataset of lease agreements annotated by people who read these contracts for a living.
• Compared the automated evaluators against those human ratings and kept the one that tracked them most closely ⁸.
• Monitored for edge cases and factual accuracy using automated KILN evals.
• Re-validated as new document types arrived, so performance did not quietly drift.

vLLM in OpenKit’s production stack

For production deployment, we prioritise both performance and privacy. vLLM serves as the backbone of our inference infrastructure.

Deployment architecture

Our typical vLLM implementation includes:

1. Infrastructure Setup:
   • Containerised deployment (e.g., Docker, Kubernetes) for consistency and scalability.
   • GPU resource allocation optimised for specific model sizes and expected load.
   • Scalable architecture supporting both vertical (more powerful instances) and horizontal (more instances) scaling.
2. Security-First Design:
   • Deployment on client-controlled infrastructure or secure private cloud instances (e.g., AWS, Azure, GCP).
   • End-to-end encryption for data in transit (TLS/SSL) and at rest.
   • Role-based access controls (RBAC) and full audit logging.
   • Designed to help meet the data protection obligations the client operates under, including UK GDPR.
3. Integration Layer:
   • Custom API gateway tailored to client workflows and existing enterprise systems.
   • Authentication (e.g., OAuth 2.0, API keys) and authorisation mechanisms.
   • Input validation, sanitisation, and preprocessing pipelines.
   • Response post-processing, formatting, and caching strategies.
4. Operational Excellence:
   • Comprehensive monitoring of key performance indicators (KPIs): latency, throughput, error rates, GPU utilisation.
   • Automated scaling policies based on real-time demand patterns.
   • Health checks and automated fallback mechanisms for system resilience.
   • Regular performance benchmarking and optimisation of deployed models and infrastructure.

Client spotlight: bespoke document intelligence (Pubs Advisory Service) “Their deep understanding of our business needs, coupled with their expertise in GPT and Cloud (AWS) software-services, enabled them to navigate complexities and deliver a bespoke AI solution tailored to our operations. The team demonstrated a high level of professionalism and adaptability, ensuring a smooth project delivery.”

— Chris Wright, Pubs Advisory Service

Performance optimisations for legal document processing

For our legal document analysis solutions, such as the BAiSICS platform, we’ve implemented several vLLM-specific optimisations:

1. Context Length Management:
   • Advanced chunking strategies tailored for long and dense legal documents, respecting semantic boundaries.
   • Dynamic batch sizing based on token count and document complexity to maximise GPU utilisation.
   • Intelligent token budgeting per request to handle variable document lengths efficiently.
2. Memory Optimisation:
   • Full use of PagedAttention for KV cache management with long legal texts ^{13, 14}.
   • Continuous batching to ensure high throughput even with varying request loads from multiple analysts ¹².
   • Careful selection of model quantisation (e.g., AWQ, GPTQ where applicable and supported by vLLM ¹¹) to balance document comprehension quality with inference speed and memory footprint.
3. Specialised Processing Pipelines:
   • Pre-processing workflows optimised for cleaning and structuring text from scanned legal documents (PDFs, OCR outputs).
   • Custom prompt templates engineered for different legal clauses and document types (e.g., contracts, case law, statutes).
   • Response formatting tailored to legal information extraction, summarisation, and comparison tasks.

These optimisations using vLLM have changed what the work looks like for the people doing it:

• An analyst can ask a question of a document rather than reading the whole file to find the answer.
• Efficient handling of complex legal documents often exceeding 50-100 pages.
• Capability to support concurrent analysis of multiple documents by teams of legal professionals.
• Consistently low-latency responses for interactive querying and analysis of document contents.

Best practices for implementing open-source generative AI tools

Based on our experience implementing these tools for enterprise clients, we’ve developed several best practices:

1. Establish clear evaluation metrics early

Define success metrics at the project outset. These should align with business objectives rather than just technical benchmarks. For legal document analysis, for example, metrics might include:

• Accuracy of extracted terms (e.g., F1 score for named entity recognition).
• Completeness and factual consistency of summaries (e.g., ROUGE, BERTScore, human review).
• Speed improvement over manual review (e.g., time saved per document).
• Correlation with expert human assessment (using KILN AI’s methodologies ⁸).

2. Design for data privacy and security from the start

When working with sensitive information, especially in sectors like legal, finance, or healthcare:

• Deploy inference engines (like vLLM) on private infrastructure (on-premise or secure private cloud).
• Ensure data does not leave client-controlled environments during any stage, including evaluation.
• Implement fine-grained access controls, end-to-end encryption, and regular security audits.
• Establish clear data governance, retention, and deletion policies compliant with regulations.

3. Combine multiple tools

No single tool addresses all needs. Our most successful implementations often create a tailored stack:

• Hugging Face Hub: For sourcing and initially experimenting with foundation models.
• KILN AI Eval Framework: For rigorous, ongoing evaluation of model outputs and prompt effectiveness.
• Custom Architecture or LlamaIndex (selectively): For efficient data ingestion and retrieval in RAG applications.
• vLLM or llama.cpp: For optimised inference, choosing vLLM for scalable production and llama.cpp for local/edge or CPU-bound scenarios.
• MLOps tools (MLflow, DVC): For experiment tracking, data versioning, and pipeline orchestration.

4. Evaluate continuously across the lifecycle

Evaluation isn’t a one-time activity. We integrate KILN AI evaluations (or similar frameworks) throughout the development process ^{7, 8}:

• During initial model selection and benchmarking.
• Throughout prompt engineering and fine-tuning iterations.
• In pre-production validation against “golden datasets”.
• As part of ongoing monitoring in production to detect drift or degradation.

5. Stay close to the community

The open-source AI space evolves rapidly:

• Actively monitor repositories and communities (e.g., Hugging Face forums, GitHub discussions for key tools) for updates, best practices, and emerging techniques.
• Engage with research papers that often introduce or validate new tools and methodologies (e.g., those found on arXiv).
• Encourage internal knowledge sharing and experimentation within your team.

Where this leaves enterprise AI

The open-source generative AI ecosystem has matured quickly. Its tools now rival, and in some cases beat, their proprietary counterparts on performance, flexibility, and transparency. At OpenKit we have settled on a small set we trust, chiefly KILN AI for evaluation and vLLM for inference, and built our delivery around them.

The advantages of this open-source-centric approach for enterprises are clear:

• Greater control over data privacy, model behaviour, and infrastructure choices.
• Enhanced customisation to tailor AI solutions precisely to specific business domains and workflows.
• Reduced vendor lock-in risk and greater long-term strategic flexibility.
• Improved transparency and auditability, which matters in regulated industries and for building trust.
• Lower cost in many cases, without giving up quality or sophistication in the AI solution.

Open-source tools are only part of the job. The rest is the domain expertise, rigorous evaluation, security, and integration that make a system fit for business-critical work, and that is where most of our effort goes. Used well, these tools let organisations get real value from generative AI without handing over control of their data or their models.

If you want to work out how these tools fit your own processes, the AI services overview sets out what we build with them.

Start with an audit

Most engagements start with an AI Audit and Transformation, a fixed-scope, fixed-fee piece of work that finds where AI earns its place in your business and where it does not, then puts the first of it into service. You leave with a written report and a prioritised 12 month roadmap.

Start the conversation ↗ How the audit works

References

1. Hugging Face Transformers Introduction - GeeksforGeeks. Accessed May 2025.
2. Hugging Face Transformers Documentation - Hugging Face. Accessed May 2025.
3. Open LLM Leaderboard Collection - Hugging Face. Accessed May 2025.
4. Open LLM Leaderboard Main Space - Hugging Face. Accessed May 2025.
5. What Is LangChain and How to Use It - Edureka. Accessed May 2025.
6. LlamaIndex Documentation - LlamaIndex Team. Accessed May 2025.
7. Building an LLM evaluation framework: best practices - Datadog. Accessed May 2025.
8. Kiln AI Evaluations Documentation - Kiln AI. Accessed May 2025.
9. Top 6 Open-Source Frameworks for Evaluating Large Language Models - Athina AI Hub. Accessed May 2025.
10. LLM Evaluation Frameworks: Head-to-Head Comparison - Comet ML. Accessed May 2025.
11. vLLM Official Documentation - vLLM Project. Accessed May 2025.
12. What is vLLM? How to Install and Use vLLM, Explained - Apidog Blog. Accessed May 2025.
13. vLLM Paged Attention Kernel Design - vLLM Project. Accessed May 2025.
14. Introduction to vLLM and PagedAttention - RunPod Blog. Accessed May 2025.
15. llama.cpp GitHub Repository - Georgi Gerganov. Accessed May 2025.
16. LLM Serving Frameworks Overview (Ollama, vLLM, SGLang, LLaMA.cpp Server) - Hyperbolic Blog. Accessed May 2025.
17. LM Studio Official Website - LM Studio. Accessed May 2025.
18. Oobabooga Text Generation WebUI GitHub - oobabooga. Accessed May 2025.
19. LLMOps workflows on Databricks - Databricks Documentation. Accessed May 2025.
20. Kiln AI GitHub Repository - Kiln AI. Accessed May 2025.
21. OpenKit AI Development Services - OpenKit Ltd. Accessed May 2025.
22. Llamaindex vs Langchain: What’s the difference? - IBM Blog. Accessed May 2025.
23. NVIDIA TensorRT-LLM - NVIDIA Developer. Accessed May 2025.