Open Source LLM Observability Tools and Platforms

• LLM Observability in the Context of LLMOps for Generative AI
• What is LLM Observability?
• Expected Functionalities of an LLM Observability Solution
• Open Source LLM Observability Tools and Platforms
• Non-open source
• Other - related
• References

LLM Observability in the Context of LLMOps for Generative AI

AI is transforming the world, and one area where it has made significant strides is in generative models, particularly in the field of Large Language Models (LLMs) like GPT-3 and transformer models. However, as impressive as these models are, managing, monitoring, and understanding their behavior and output remains a challenge. Enter LLMOps, a new field focusing on the management and deployment of LLMs, and a key aspect of this is LLM Observability.

• LLM Observability in the Context of LLMOps for Generative AI
• What is LLM Observability?
• Expected Functionalities of an LLM Observability Solution
• Open Source LLM Observability Tools and Platforms
• Other - related
• References

What is LLM Observability?

LLM Observability is the ability to understand, monitor, and infer the internal state of an LLM from its external outputs. It encompasses several areas including model health monitoring, performance tracking, debugging, and evaluating model fairness and safety.

In the context of LLMOps, LLM Observability is critical. LLMs are complex and can be unpredictable, producing outputs that range from harmless to potentially harmful or biased. It's therefore essential to have the right tools and methodologies for observing and understanding these models' behaviors in real-time, during training, testing, and after deployment.

Expected Functionalities of an LLM Observability Solution

1. Model Performance Monitoring: An observability solution should be able to track and monitor the performance of an LLM in real-time. This includes tracking metrics like accuracy, precision, recall, and F1 score, as well as more specific metrics like perplexity or token costs in the case of language models.

2. Model Health Monitoring: The solution should be capable of monitoring the overall health of the model, identifying and alerting on anomalies or potentially problematic patterns in the model's behavior.

3. Debugging and Error Tracking: If something does go wrong, the solution should provide debugging and error tracking functionalities, helping developers identify, trace, and fix issues.

4. Fairness, Bias, and Safety Evaluation: Given the potential for bias and ethical issues in AI, any observability solution should include features for evaluating fairness and safety, helping ensure that the model's outputs are unbiased and ethically sound.

5. Interpretability: LLMs can often be "black boxes", producing outputs without clear reasoning. A good observability solution should help make the model's decision-making process more transparent, providing insights into why a particular output was produced.

6. Integration with Existing LLMOps Tools: Finally, the solution should be capable of integrating with existing LLMOps tools and workflows, from model development and training to deployment and maintenance.

LLM Observability is a crucial aspect of LLMOps for generative AI. It provides the visibility and control needed to effectively manage, deploy, and maintain Large Language Models, ensuring they perform as expected, are free from bias, and are safe to use.

Open Source LLM Observability Tools and Platforms

1. Azure OpenAI Logger - - "Batteries included" logging solution for your Azure OpenAI instance.

1. Deepchecks - - Tests for Continuous Validation of ML Models & Data. Deepchecks is a Python package for comprehensively validating your machine learning models and data with minimal effort.
2. Evidently - - Evaluate and monitor ML models from validation to production.
3. Giskard - - Testing framework dedicated to ML models, from tabular to LLMs. Detect risks of biases, performance issues and errors in 4 lines of code.
4. whylogs - - The open standard for data logging
5. lunary - - The production toolkit for LLMs. observability, prompt management, and evaluations.
6. openllmetry - - Open-source observability for your LLM application, based on OpenTelemetry
7. phoenix (Arize Ai) - - AI Observability & Evaluation - Evaluate, troubleshoot, and fine-tune your LLM, CV, and NLP models in a notebook.
8. langfuse - - Open source LLM engineering platform. observability, metrics, evals, prompt management SDKs + integrations for Typescript, Python
9. LangKit - - An open-source toolkit for monitoring Large Language Models (LLMs). Extracts signals from prompts & responses, ensuring safety & security. Features include text quality, relevance metrics, & sentiment analysis. Comprehensive tool for LLM observability.
10. agentops - - Python SDK for agent evals and observability
11. pezzo - - Open-source, developer-first LLMOps platform designed to streamline prompt design, version management, instant delivery, collaboration, troubleshooting, observability and more.
12. Fiddler AI - - Evaluate, monitor, analyse, and improve machine learning and generative models from pre-production to production. Ship more ML and LLMs into production, and monitor ML and LLM metrics like hallucination, PII, and toxicity.
13. OmniLog - - Observability tool for your LLM prompts.

Non-open source

• Generative AI Studio - Galileo

Other - related

• Great Expectations - Always know what to expect from your data.
• AgentOps-AI/tokencost - Easy token price estimates for LLMs
• observability prompts - LLM observability related prompts
• LLM Observability
• baml - A programming language to build strongly-typed LLM functions. Testing and observability included
• aperture - Rate limiting, caching, and request prioritization for modern workloads

References

• LLM Monitoring and Observability — A Summary of Techniques and Approaches for Responsible AI | by Josh Poduska | Towards Data Science
• Monitoring LLMs: Metrics, challenges, & hallucinations
• mattcvincent/intro-llm-observability - Intro to LLM Observability
• Demystifying Perplexity: An AI Expert‘s Comprehensive Guide - 33rd Square
• Perplexity - a Hugging Face Space by evaluate-metric
• List of top LLM Observability Tools - good intro

Edits:

• 2024-12-19: Added reference to list of top observability tools
• 2024-06-26: Added summary

@article{Saf2024Open,
    author  = {Krystian Safjan},
    title   = {Open Source LLM Observability Tools and Platforms},
    journal = {Krystian's Safjan Blog},
    year    = {2024},
}