VERSES AI says Genius outperforms OpenAI model in code-breaking challenge

VERSES AI revealed performance highlights of its flagship product Genius winning the code-breaking game Mastermind in a side by side comparison with a leading generative AI model, OpenAI’s o1 Preview, which is positioned as an industry-leading reasoning model. Over one hundred test runs, Genius consistently outperformed OpenAI’s o1-preview model one hundred and forty times faster and more than five thousand times cheaper .”Today we’re showcasing Genius’ advanced reasoning performance against state-of-the-art deep learning-based methods that LLMs are based on,” said Hari Thiruvengada, VERSES CTO. “Mastermind was the perfect choice for this test because it requires reasoning through each step logically, predicting the cause-and-effect outcomes of its decisions, and dynamically adapting to crack the code. This exercise demonstrates how Genius outperforms tasks requiring logical and cause-effect reasoning, while exposing the inherent limitations of correlational language-based approaches in today’s leading reasoning models. This is just a preview of what’s to come. We’re excited to show how additional reasoning capabilities, available in Genius today and demonstrated with Mastermind, will be further showcased in our upcoming Atari 10k benchmark results.” Performance Highlights: Accuracy and Reliability. Genius solved the code every time in a consistent number of steps. Speed. Genius consistently solved games in 1.1-4.5 seconds, while ChatGPT’s solve times ranged from 7.9 to 889 seconds Efficiency. Genius’ total compute time for 100 games was just over 5 minutes, compared to ChatGPT’s 12.5 hours. Cost. Genius’ compute cost was estimated at $0.05 for all 100 games, compared to ChatGPT’s o1 model at $263. In summary, Genius solved Mastermind 100% of the time, was 140 times faster and 5260 times cheaper than o1-preview.