Understanding the Differences in Language Models - Transformers vs. Markov Models

2023-10-07

This article explores distinguishing details of Markov Models and Transformer-based models like GPT, focusing on how they predict the next character in a sequence. It explores the fundamental differences between these models, with a particular emphasis on how the self-attention mechanism in Transformer models makes a difference compared to the fixed context length in Markov models.


Continue reading

From Fixed-Size to NLP Chunking - A Deep Dive into Text Chunking Techniques

2023-09-11

Discover text chunking - the secret sauce behind accurate search results and smarter language models! By understanding how to effectively chunk text, we can improve the way we index documents, handle user queries, and utilize search results. Ready to uncover the secrets of text chunking?


Continue reading

Understanding Retrieval-Augmented Generation (RAG) empowering LLMs

2023-08-24

Understand innovative artificial intelligence framework that empower large language models (LLMs) by anchoring them to external knowledge sources with accurate, current information.


Continue reading

Easy Text Vectorization With VectorHub and Sentence Transformers

2023-07-04

Learn how to use Sentence Transformers for text vectorization with different models using consistent API.


Continue reading

Leveraging Language Models in Corporate Environments - The Future of Knowledge Management

2023-02-01

Explore the benefits and challenges of using Large Language Models (LLMs) in corporate environments for improved knowledge management. Learn how to implement LLMs and overcome potential obstacles.


Continue reading

How to Detect ChatGPT-Generated Text?

2023-01-11

Discover the latest methods for distinguishing machine-generated text from the human-written text. Learn about statistical, syntactic, semantic, and neural network-based approaches. Stay up-to-date with the latest research in NLP and AI.


Continue reading

Visual Text Exploration as Part of Preprocessing Before Classification

2022-10-11

This post discusses importance of visual text exploration in preprocessing for classification, covers techniques (wordcloud, Sentiment Analysis, topic modeling, data cleaning) & how to use them with popular libraries. Encourages readers to try for own projects.


Continue reading

Top 10 Python Libraries for Document Classification

2022-05-01

Unlock the power of document classification with these top Python libraries! Discover the best tools for effortless text analysis and more.


Continue reading

What's Cooking

2018-04-05

Exploratory Data Analysis of the Kaggle's "What's cooking" competition dataset to get understanding what kind of data we are dealing with and get intuition of existing dependencies.


Continue reading