Introducing chat-splitter

I have recently released the initial version of the chat-splitter crate. The purpose of this blog post is to offer an overview of what chat-splitter is, the issue it addresses, and how it operates. The context length problem Large language models face challenges when processing lengthy sequences of text. State-of-the-art models, like OpenAI’s GPT-4, have a maximum token limit that restricts their ability to handle contexts beyond a certain threshold. The maximum context window typically ranges from 2k tokens for legacy GPT-3 models to as high as 32k tokens for the most recent GPT-4 model, and potentially more1....

July 16, 2023 · 4 min · Felipe S. S. Schneider