ReAct: Synergizing Reasoning and Acting in Language Models

By 2022, two powerful capabilities had emerged in large language models, but they were studied in isolation:

• Chain-of-thought (CoT) reasoning let models "think out loud" — decomposing problems into steps, performing arithmetic, and drawing conclusions. But CoT is a closed loop: the model reasons entirely from its own internal knowledge, with no way to check facts or gather new information. When its knowledge is wrong or incomplete, it hallucinates — confidently generating plausible-sounding but fabricated facts.

• Action generation let models interact with external tools — searching the web, querying databases, or navigating environments. But without reasoning, the model takes actions blindly. It cannot decompose goals, track progress, handle unexpected results, or synthesize a final answer from what it finds.

Humans don't separate thinking from acting. When cooking, you think "I need salt," reach for the salt, discover you're out, think "soy sauce could work instead," and adapt. This tight loop between reasoning and acting is what makes human problem-solving robust. ReAct brings this loop to language models.

The ReAct idea is elegant in its simplicity. Consider an agent interacting with an environment. At each time step $t$, the agent receives an observation $o_t$ and takes an action $a_t$. Normally the action space $A$ contains only environment actions — search, click, navigate.

ReAct extends this action space to $\hat{A} = A \cup L$, where $L$ is the space of natural language. An action in $L$ — a thought or reasoning trace — does not affect the external environment. It produces no observation. Instead, it updates the agent's internal context, helping it plan, reason, and decide.

This means thoughts are "free" — they cost no environment step, generate no side effects, but enrich the context that the model uses to choose its next real action. The model alternates between thinking (in language space $L$) and acting (in environment space $A$), creating a thought-action-observation loop.

ReAct uses few-shot prompting: the model is given a small number of human-annotated trajectories as examples, then generates its own trajectories for new problems. Each trajectory is a sequence of thought-action-observation steps.

For knowledge-intensive tasks like HotpotQA (multi-hop question answering) and FEVER (fact verification), the model alternates thoughts and actions densely — every action is preceded by a thought explaining what to do and why. The action space is a simple Wikipedia API with three operations: search[entity] returns the first 5 sentences from a Wikipedia page, lookup[string] simulates Ctrl+F, and finish[answer] submits the final answer.

The walkthrough below shows the complete trajectory for a real HotpotQA question. Notice how each thought serves a purpose: decomposing the question, extracting relevant facts from observations, reformulating failed searches, and synthesizing the final answer.

ReAct is not limited to question answering. The paper tests it on two interactive decision-making environments where an agent must take many actions over long horizons:

• ALFWorld — a text-based game where the agent navigates a simulated household to complete tasks like "put a clean knife on the countertop." The agent must plan subgoals (find knife → clean it → place it), reason about where household items are likely to be found (a knife is more likely on a countertop or in a drawer), and track which subgoals are complete.

• WebShop — an online shopping environment with 1.18 million real products. The agent must find and purchase a product matching a user's instruction by searching, comparing products, and selecting the right options.

In these environments, thoughts appear sparsely — only at key decision points, not before every action. The model itself decides when a thought is needed. This flexible placement is one of ReAct's strengths: it adapts the density of reasoning to the task.

The paper evaluates ReAct on four diverse benchmarks using PaLM-540B with few-shot prompting. The results reveal a nuanced picture:

On FEVER (fact verification), ReAct clearly outperforms CoT (60.9 vs. 56.3 accuracy), because verifying claims often requires retrieving precise facts that the model's internal knowledge may get slightly wrong.

On HotpotQA (multi-hop QA), ReAct slightly lags behind CoT (27.4 vs. 29.4 exact match), because CoT's free-form reasoning is more flexible than ReAct's structured thought-action-observation format. However, the best overall method is combining both: ReAct→CoT-SC achieves 35.1, and CoT-SC→ReAct achieves 34.2 — significantly outperforming either alone.

On both ALFWorld and WebShop, ReAct dramatically outperforms all baselines, demonstrating that reasoning is most valuable in interactive environments where the agent must plan over many steps.

The authors performed a detailed human analysis of 200 trajectories from ReAct and CoT on HotpotQA. The findings are striking:

Among successful trajectories, 14% of CoT answers were correct by accident — the reasoning trace contained hallucinated facts that happened to lead to the right answer. For ReAct, only 6% had this problem. Among failed trajectories, the picture is even clearer: 56% of CoT failures were caused by hallucination — the model invented facts and reasoned from them confidently. For ReAct, hallucination caused 0% of failures. External grounding completely eliminates fabricated facts.

ReAct has its own failure mode: search result errors (23% of failures), where the search returns empty or unhelpful results that derail the reasoning. This is the trade-off between factuality and flexibility. But the authors show this can be mitigated by combining ReAct with CoT-SC — falling back to internal reasoning when external search fails, or vice versa.

An important finding is how ReAct scales with model size and training. With prompting alone, ReAct actually performs worst among all methods on small models (PaLM-8B, PaLM-62B) — learning both reasoning and acting from in-context examples is too complex for smaller models.

But when fine-tuned on just 3,000 ReAct trajectories with correct answers, the picture reverses dramatically. Fine-tuned ReAct becomes the best method across all model sizes. PaLM-8B fine-tuned with ReAct outperforms every PaLM-62B prompting method. PaLM-62B fine-tuned with ReAct outperforms every PaLM-540B prompting method. The intuition is clear: fine-tuning Standard or CoT teaches the model to memorize (potentially hallucinated) facts, while fine-tuning ReAct teaches the model how to retrieve and reason — a more generalizable skill.

A distinctive advantage of ReAct is transparency. Because the model writes its reasoning in natural language before each action, humans can inspect the decision-making process in real time. This enables two powerful capabilities:

• Diagnosability: when ReAct makes an error, the reasoning trace reveals exactly where the reasoning went wrong — whether the model misinterpreted an observation, hallucinated a fact, or chose the wrong search query. This is far more interpretable than a black-box model that simply outputs a wrong answer.

• Human-in-the-loop editing: because thoughts are just text, a human can edit them mid-trajectory. The paper demonstrates that by simply removing a hallucinated sentence and adding a hint, a failing ReAct trajectory can be steered to success. This enables a new paradigm of human-AI collaboration where the human edits the model's thinking, not its actions.

ReAct's deepest contribution is not a model or an algorithm — it is a pattern. The think-act-observe loop is now so fundamental that it is hard to imagine building an AI agent without it. Every time Claude uses a tool, every time an AI agent searches the web, browses a page, or executes code, the ReAct pattern is at work: reason about what to do, do it, observe the result, then reason again.