Today’s LLMs Are Just Parrots Not Capable Of True Reasoning, These Researchers Say

A new research paper suggests that current LLMs’ ability to reason is ‘inherently flawed,’ and that there is a better framework to build reasoning AI.

The current major large language models from OpenAI to Anthropic employ a prompting technique known as ‘chain-of-thought’ [CoT], a mechanism to improve the LLM’s reasoning capabilities.

However, a group of researchers says that chain-of-thought prompting is “inherently flawed” based on their testing, and that instead, neuro-symbolic reasoning is the technology that can give LLMs genuine reasoning capabilities.

The technical white paper from CoreThink, an AI startup and some University of California, San Diego, researchers states that LLMs are “statistical generators not reasoning engines.”

“LLMs ‘overthink’ simple problems and collapse under complexity, even the reasoning-specialized LLMs,” the white paper asserts.

In August 2025, several researchers from Apple came to similar conclusions regarding LLMs and reasoning.

“Large Language Models (LLMs) have demonstrated remarkable capabilities across various domains, including natural language processing, question answering, and creative tasks,” the report, GSM-Symbolic: Understanding The Limitations Of Mathematical Reasoning In Large Language Models, states.

“However, the question of whether current LLMs are genuinely capable of true logical reasoning remains an important research focus. While some studies highlight impressive capabilities, a closer examination reveals substantial limitations,” the Apple report continues.

The answer to giving LLMs true reasoning superpowers lies in neuro-symbolic reasoning, say CoreThink’s founders.

“Most large language models are purely neural. They are statistical pattern matchers. They can mimic reasoning with tricks like chain-of-thought prompting, but they don’t actually reason symbolically. That’s why they break down on multi-step tasks and can’t explain their own decisions,” Vishvesh Bhat, CoreThink co-founder, said in an email to MES Computing.

“Neural networks alone are hitting a ceiling: accuracy collapses beyond a handful of reasoning steps, and costs skyrocket. Neuro-symbolic AI combines the adaptability of neural models with the structure and explainability of symbolic reasoning. That’s the path to scalable, trustworthy AI,” Bhat added. “CoT is an illusion of reasoning, Symbolic is actual reasoning.”

“Parroting is what purely LLMs do. They generate the most likely next word. Symbolic reasoning is different: it creates explicit, auditable logic traces. Instead of just echoing patterns, Symbolic reasoning can explain ‘why’ a conclusion was reached and show the steps that got them there,” said CoreThink co-founder Ram Shanmugam in an email to MES Computing,

CoreThink’s solution for true AI reasoning is General Symbolics, a neuro-symbolic reasoning layer that improves base models’ reasoning by 30-60 percent, the research paper states.

The researchers predict that the industry will move to neuro-symbolic reasoning especially in sectors with demanding needs for data accuracy like health care and law.

Which means in the near future, LLMs will not just memorize data sets, but will learn genuine reasoning strategies, according to the researchers.

But what does this all mean for the midmarket IT executive?

It means the ability to actually deploy LLMs in “compliance-heavy, mission-critical workflows,” said Shanmugam, “not just in prototypes or chatbots.”