Understanding the Differences in Language Models - Transformers vs. Markov Models


This article explores distinguishing details of Markov Models and Transformer-based models like GPT, focusing on how they predict the next character in a sequence. It explores the fundamental differences between these models, with a particular emphasis on how the self-attention mechanism in Transformer models makes a difference compared to the fixed context length in Markov models.

Continue reading

From Fixed-Size to NLP Chunking - A Deep Dive into Text Chunking Techniques


Discover text chunking - the secret sauce behind accurate search results and smarter language models! By understanding how to effectively chunk text, we can improve the way we index documents, handle user queries, and utilize search results. Ready to uncover the secrets of text chunking?

Continue reading

Understanding Retrieval-Augmented Generation (RAG) empowering LLMs


Understand innovative artificial intelligence framework that empower large language models (LLMs) by anchoring them to external knowledge sources with accurate, current information.

Continue reading

Easy Text Vectorization With VectorHub and Sentence Transformers


Learn how to use Sentence Transformers for text vectorization with different models using consistent API.

Continue reading

Leveraging Language Models in Corporate Environments - The Future of Knowledge Management


Explore the benefits and challenges of using Large Language Models (LLMs) in corporate environments for improved knowledge management. Learn how to implement LLMs and overcome potential obstacles.

Continue reading

How to Detect ChatGPT-Generated Text?


Discover the latest methods for distinguishing machine-generated text from the human-written text. Learn about statistical, syntactic, semantic, and neural network-based approaches. Stay up-to-date with the latest research in NLP and AI.

Continue reading

Visual Text Exploration as Part of Preprocessing Before Classification


This post discusses importance of visual text exploration in preprocessing for classification, covers techniques (wordcloud, Sentiment Analysis, topic modeling, data cleaning) & how to use them with popular libraries. Encourages readers to try for own projects.

Continue reading

Top 10 Python Libraries for Document Classification


Unlock the power of document classification with these top Python libraries! Discover the best tools for effortless text analysis and more.

Continue reading

What's Cooking


Exploratory Data Analysis of the Kaggle's "What's cooking" competition dataset to get understanding what kind of data we are dealing with and get intuition of existing dependencies.

Continue reading