Engineering the ReAct Reasoning Framework in Hengshi Agentic BI

Article body

Full article

18 min read

Introduction

Traditional BI follows a predefined analysis logic. Developers write query templates, calculation formulas, and chart configurations in advance, and the system executes along a fixed path after a user triggers it. This model is highly deterministic, but it is not very flexible. Any analysis need outside the predefined scope must be scheduled with the development team.

The vision of Agentic BI is an autonomous analysis agent. Users only need to state their analysis intent, and the agent decides what data to query, how to calculate it, which charts to use, and how to interpret the results. This requires the agent to have multi-step reasoning capability. Instead of producing an answer in one step, it thinks while acting and adjusts subsequent steps based on intermediate results.

The ReAct (Reasoning + Acting) framework is a classic paradigm for enabling this kind of multi-step reasoning. Hengshi Agentic BI has accumulated extensive practical experience while engineering the ReAct framework. This article examines the technical implementation of ReAct in BI scenarios.

1. Core Principles of the ReAct Framework

1.1 Alternating Between Thought and Action

The name ReAct combines two words: Reasoning and Acting. Its core idea is to let an LLM alternate between thought and action, forming a Thought-Action-Observation loop.

A complete ReAct loop consists of the following stages:

Thought: The LLM analyzes the current state and decides what to do next. For example: “The user wants to know why sales revenue declined. I first need to query sales data for the last 3 months to see the trend.”

Action: Based on the reasoning, the LLM calls a tool to perform an operation. For example, it calls query_metric to query sales revenue for the last 3 months.

Observation: The tool returns a result. The LLM observes that result and enters the next round of reasoning. For example: “Sales revenue did indeed decline by 15% in February. I need to further query the breakdown by product line to identify which product line caused the decline.”

This loop continues until the LLM believes it has enough information to answer the user’s question and outputs a final answer.

1.2 ReAct vs. Other Reasoning Paradigms

vs. Chain-of-Thought (CoT): CoT lets an LLM perform a pure text reasoning chain without calling external tools. It suits scenarios such as mathematical reasoning that do not require external data. BI analysis must query data, so CoT is not suitable.

vs. Plan-and-Execute: This approach first generates a complete plan in one pass and then executes it step by step. The problem is that once an intermediate step returns an unexpected result, the entire plan becomes invalid. ReAct has the advantage that each step can change direction based on actual results.

vs. Tree-of-Thoughts (ToT): This approach generates multiple branches at every decision point and performs tree search. It can be effective, but its computational cost is high, making it less suitable for BI scenarios with real-time requirements.

ReAct is the best fit for BI scenarios. It combines reasoning capability for analyzing intermediate results with action capability for querying data through tools, while keeping the cost of each step controllable.

2. Hengshi Agentic BI’s ReAct Implementation

2.1 Prompt Template Design

The core of the ReAct framework is the prompt template. It guides the LLM in switching between thinking, acting, and observing. Hengshi’s prompt template contains the following structure:

System instruction: Defines the agent’s role and capability boundaries. “You are a BI analysis agent. You can call the following tools to answer users’ data analysis questions. Execute only one tool call at a time, wait for the result, and then decide the next step.”

Tool descriptions: List the names, descriptions, and parameter schemas of all available tools. The LLM selects a tool from this list during the Action stage.

Interaction history: The preceding Thought-Action-Observation sequence. This lets the LLM know what it has already done and what results it has obtained.

Current question: The user’s original analysis request.

Output-format constraints: Require the LLM to use a fixed format: first output a Thought paragraph, then output an Action call. The system parses the Action, executes the tool, appends the result to the prompt in Observation format, and triggers the next round of reasoning.

2.2 Quality Control for the Chain of Thought

The effectiveness of ReAct depends heavily on the quality of Thought. If the LLM’s reasoning is superficial, such as “I need to query data,” its action will be blind, such as calling tools at random. Hengshi uses three measures to improve reasoning quality:

Thought guidance: Add rules to the system instruction: “In every Thought, explicitly state the following three points: 1) what information is currently known; 2) what information is still missing; and 3) what should be done next to obtain the missing information.” This structured guidance avoids superficial reasoning.

Reflection mechanism: After every 3 ReAct cycles, insert a reflection step. The LLM reviews the actions and results from the preceding 3 steps and evaluates whether it has deviated from the original objective. If it has, it corrects its direction.

Termination decision: In every Thought, the LLM needs to decide, “Do I already have enough information to answer the user’s question?” If the answer is yes, it outputs a Final Answer. If the answer is no, it continues with an Action. To prevent infinite loops, the system sets a maximum step limit of 10 by default.

2.3 Constraints and Guidance for Action Selection

The LLM chooses a tool to call during the Action stage. To improve selection accuracy, Hengshi adds the following constraints:

Tool allowlist: Restrict the available tools according to the current analysis stage. For example, in the data-query stage, only query tools can be called; chart tools become available only in the result-visualization stage. This stage-specific constraint reduces incorrect tool selection.

Parameter guidance: When the LLM generates tool parameters, the system embeds parameter-acquisition hints in the parameter schema. For example, the hint for a time parameter says, “If the user says ‘last month,’ output the standard date-range format YYYY-MM-DD,” guiding the LLM to produce compliant parameters.

History deduplication: If the LLM has already called the same tool with the same parameters in an earlier step, the system adds a prompt message: “You have already called this query. The result is as follows,” preventing repeated calls.

3. ReAct Execution Trace for a Typical Scenario

3.1 Scenario: Attribution Analysis for a Sales Anomaly

User question: “Why did sales revenue suddenly drop in February?”

Round 1:

Thought: The user wants to know why sales revenue declined in February. I need to first confirm the decline and its magnitude by querying the sales-revenue trend for the last 6 months.
Action: Call query_metric, parameters: metric = sales revenue, time range = last 6 months, granularity = month.
Observation: Returns sales-revenue data for 6 months. February did decline, by 18% month over month and 12% year over year.

Round 2:

Thought: The decline has been confirmed. Next, I need to identify its source: which product line, region, or customer segment caused it. I will first break it down by product line.
Action: Call aggregate_metric, parameters: metric = sales revenue, dimension = product line, time range = January vs. February.
Observation: Returns the January-February sales-revenue comparison for each product line. Product Line C declined by 42%, while the other product lines changed little.

Round 3:

Thought: Product Line C is the main source of the decline. I need to further analyze whether the decline in Product Line C is regional or global, and whether it is related to a specific customer.
Action: Call aggregate_metric, parameters: metric = sales revenue, dimension = region + customer, filter condition = Product Line C, time range = January vs. February.
Observation: Returns sales changes for Product Line C by region and customer. A major customer A in South China fell from 3.5 million to 0.5 million.

Round 4:

Thought: The core cause is a substantial reduction in purchases of Product Line C by major customer A in South China. I need to query this customer’s recent order status to determine whether there is an anomaly.
Action: Call query_dataset, parameters: dataset = order details, filter conditions = customer A, Product Line C, last 3 months.
Observation: Returns the order details of major customer A for the last 3 months. A large February order has the status “cancelled.”

Round 5 (termination):

Thought: I have identified the root cause. The main reason sales revenue declined 18% in February is that major customer A in South China cancelled a large Product Line C order of approximately 3 million. I have enough information to answer the user’s question.
Final Answer: Sales revenue declined 18% month over month and 12% year over year in February. The main cause was that major customer A in South China cancelled a large Product Line C order of approximately 3 million on February 15. Excluding this factor, sales revenue across other business lines was largely flat. It is recommended to investigate why major customer A cancelled the order, which may involve contract expiration, a competitor replacement, or a service issue.

3.2 Trace Analysis

This case demonstrates three core characteristics of ReAct:

Progressive deepening: From confirming the fact, to identifying the source, to narrowing it down to a customer, to finding the order status, each step goes deeper than the previous one.

Data-driven decisions: Every action is based on the observation from the preceding step. If step 2 had found that product lines were not the issue, step 3 would have turned in another direction.

Autonomous termination: In step 5, the agent determines that it has found the root cause and independently decides to provide a final answer, without external intervention.

4. Engineering Challenges and Solutions for ReAct

4.1 Token Consumption Control

ReAct’s multi-round loop means that the prompt grows longer over time. Each round’s Thought-Action-Observation is appended to the prompt. After 10 rounds, the prompt may exceed 10,000 tokens, causing inference costs to rise sharply.

Hengshi’s solution:

Context compression: Compress the context every 5 rounds. The LLM summarizes the key findings from the preceding 5 rounds and replaces the original detailed records. The compressed summary contains only 200-300 tokens, substantially reducing prompt length in later rounds.

Observation-result trimming: If an Observation returned by a tool is a large data table, retain only key aggregate figures instead of placing the complete details into the prompt. For example, “1,200 rows returned; key finding: Product Line C declined 42%” replaces 1,200 rows of raw data.

Early stopping: If the LLM obtains no new information for 3 consecutive rounds, meaning the Observation repeats existing information, the system forcibly terminates the loop and outputs the current best answer.

4.2 Error Recovery

Tool calls can fail because of a query timeout, incorrect parameters, or nonexistent data. The ReAct framework needs to handle these errors gracefully:

Automatic retry: When a tool execution fails, the system automatically retries it once. If the retry still fails, the error is returned to the LLM in Observation format.

Making error messages understandable: Raw error messages, such as SQLSyntaxErrorException: ORA-00904: invalid identifier, are difficult for an LLM to understand. The system converts the error into a natural-language description, such as “Query failed: the field name is incorrect. Check whether the field name is spelled correctly,” helping the LLM make a correction.

Fallback strategy: If a tool fails 3 consecutive times, the system suggests that the LLM use another tool or another query method. For example, after execute_sql fails, it suggests switching to query_dataset, a secure query based on a dataset.

4.3 Explainability of the Reasoning Path

Users of Agentic BI need to know how the agent arrived at a conclusion. It is not enough to provide only an answer; the reasoning path must also be provided.

Hengshi preserves the complete ReAct trace log and presents it in a readable format:

Each Thought is shown in an “Agent Thinking” card.
Each Action is shown in an “Operation Executed” card, including the tool name and parameters.
Each Observation is shown in a “Result Obtained” card, including a summary of key data.
The final answer is shown in an “Analysis Conclusion” card.

Users can expand any step to review its details and verify whether the agent’s reasoning is sound. This transparency is key to establishing user trust. Users will not trust an answer from a black box.

4.4 Parallel Reasoning

In some analysis scenarios, an agent needs to explore multiple directions simultaneously. For example, when analyzing why sales revenue declined, it may need to query product-line, regional, and customer dimensions at the same time.

Serial ReAct can query only one dimension at a time, so it is less efficient. Hengshi supports parallel ReAct:

During the Thought stage, the agent determines that it needs to query multiple independent dimensions at the same time.
The system starts multiple Action calls in parallel.
After all Actions return, the agent combines and analyzes them in a unified Observation.

Parallel reasoning compresses the 3x latency of 3 serial queries into 1x latency, substantially improving the user experience.

5. Evaluating ReAct

5.1 Evaluation Dimensions

The effectiveness of a ReAct agent needs to be evaluated across multiple dimensions:

Task completion rate: Whether the agent ultimately provides a correct and usable answer. On Hengshi’s internal test set, the ReAct agent achieves a task completion rate of 87%, significantly higher than the 72% of single-round Function Calling.

Reasoning steps: The average number of ReAct cycles needed to complete a task. Hengshi’s data shows an average of 4.2 steps: 2-3 steps for simple queries and 6-8 steps for complex attribution analysis.

Tool-call accuracy: Whether the tool and parameters selected at each step are correct. The rate is 93.5% on the internal test set, with errors occurring mainly in multi-tool selection scenarios.

Token consumption: The average number of tokens consumed to complete a task. Through context compression and Observation-result trimming, Hengshi keeps average token consumption within 8,000.

5.2 Comparison with Non-ReAct Mode

For complex analysis tasks involving sales-anomaly attribution:

Metric	Single-round Function Calling	ReAct Agent
Task completion rate	72%	87%
Answer depth	Surface-level cause	Root-cause chain
Average latency	2.1s	6.5s
Token consumption	2K	8K

Key finding: ReAct trades 3x latency and 4x tokens for a 15-percentage-point increase in completion rate and substantially deeper analysis. For BI scenarios that require accurate answers, this tradeoff is worthwhile.

6. Conclusion

The ReAct framework is a technical cornerstone of Agentic BI. It evolves a BI agent from a query tool that answers one question at a time into an analyst that thinks while acting.

Hengshi Agentic BI’s ReAct engineering practice has three core designs:

Structured thought guidance: Use prompt templates to guide LLMs toward structured reasoning and avoid superficial reasoning.
Context compression and early stopping: Control token consumption in multi-round loops and prevent costs from getting out of control.
Transparent reasoning paths: Fully present the Thought-Action-Observation trace to build user trust.

A BI system that provides an answer and shows how it reached that answer moves from an intelligent tool to a trusted analysis partner.