Meet PTChatterly: a new AI benchmarking and sizing framework from PT that can quantify the performance and user experience your customers can expect from an underlying solution running their in-house LLMs.

In-house large language models (LLMs) allow organizations to build AI-assisted chatbots trained on their own private data, opening up new opportunities for innovation and efficiency. They’re a hot topic in AI today, but for customers trying to find the right underlying cloud or on-prem hardware for their in-house LLMs, it’s difficult to know what to choose. Similarly, organizations selling computing solutions face challenges quantifying their solutions’ performance for LLM use cases.

To answer this need, we built PTChatterly.

As a new LLM benchmark and sizing framework, PTChatterly can tell your customers exactly what performance they can expect from the solution running their private LLMs. Utilizing an existing LLM and the retrieval augmented generation (RAG) method, it searches a local corpus of data and constructs responses in AI-assisted chatbot conversations with multiple simulated users. It generates meaningful, real-world metrics, e.g., “32 people can have simultaneous conversations with at worst XX response time.”

This tool is exclusively available from PT—and we’re ready to run it on your solution. Contact us to get started.