Article body
Full article
Natural language makes questions easier to ask, but it does not automatically resolve metric definitions, data scope, or permissions. HENGSHI Data Agent can assist with on-demand analysis, metric creation, and dashboard generation. Whether an answer is trustworthy still depends on data preparation, retrieval of relevant information, model configuration, and business review. This article follows the sequence of an actual data-question task to explain how these pieces work together.
1. Make Data and Metrics Understandable First
For example, the question “What was sales revenue in East China last month?” requires at least four definitions: whether sales revenue includes tax, whether it deducts refunds, whether the date is the order date or payment date, and which field represents East China. Connecting database tables to the platform cannot replace these business definitions. Teams should first organize data packages, complete the names and descriptions for datasets, fields, and atomic metrics, hide irrelevant objects, and turn common calculations into reusable metrics.
In practice, start with a data-question checklist for each core data package: which datasets support sales analysis; what the order, refund, and regional fields represent; which calculation rules common metrics use; and which fields are only for internal verification. The checklist does not need to become a separate, isolated document. It is more effective to maintain names, descriptions, and examples directly in dataset and metric definitions so analysts and Data Agent use the same information.
For the question about sales revenue in East China last month, first calculate a benchmark manually with the confirmed metric and filters, then ask the question in natural language. If the result differs from the benchmark, check the time range, regional mapping, refund treatment, and join granularity in sequence. This process can reveal both model misunderstandings and inconsistent definitions across existing reports.
This step also determines how to troubleshoot. If the answer uses the wrong field, first check dataset knowledge and field descriptions. If it selects the right field but returns the wrong number, check the metric definition, filters, and authorized scope. Do not attribute every discrepancy to the model.
2. Vectorization Improves Retrieval, but Does Not Replace Business Definitions
When there are many data packages and fields, or when business terminology differs greatly from physical field names, a data package can be vectorized. The product documentation states that text such as the names, descriptions, and field values of fields and atomic metrics can be converted into semantic vectors for Data Agent to retrieve relevant information. This helps identify synonyms and near-synonyms, reducing missed retrievals that result from relying only on keywords.
Vector search is a configurable capability. After it is enabled, the system combines vector search with the large model’s selection results. Configuration also determines whether a large model further filters datasets. For this reason, it is inaccurate to describe every data question as one fixed “vector retrieval to SQL generation” pipeline. Field descriptions, knowledge content, and vectorization tasks need maintenance as the business changes, and retrieval results need validation against actual question sets.
The product documentation provides a Vectorization action for data packages, and task progress can be checked in Task Management under execution plans. After the first run, check whether vectorization tasks update on schedule when field descriptions, atomic metrics, or business terminology change. Large numbers of distinct values are not suitable for indiscriminate indexing; confirm specific quantity and resource limits against the deployed version and configuration.
When the system does not select the expected dataset, troubleshoot in this order: whether the candidate dataset is accessible, whether its name and description are clear enough, whether the business term has synonymous expressions, whether vectorization has completed, and whether the selection rules are appropriate. If retrieval finds the right dataset but the calculation is wrong, return to the metric definitions and join model instead of repeatedly increasing the retrieval count.
Table 1. Troubleshooting order for data-question discrepancies (implementation guidance)
| Data-question symptom | Check first | Acceptance evidence |
|---|---|---|
| The expected dataset cannot be found | Account permissions, names and descriptions, vectorization task | The role can locate the dataset it is authorized to use |
| Fields are correct but values differ | Metric definition, time filters, join granularity | Matches the manual benchmark and confirmed report |
| Different roles see different results | Connection, application, and row and column permissions | Differences match the authorization rules |
| Results for the same question are unstable | Data updates and model and prompt configuration | A fixed sample retests consistently under the same configuration |
3. Validate Model Integration Together With Permissions
HENGSHI supports configuration of cloud-hosted and privately deployed large-model services. During integration, check the request endpoint, key, and interface format. Privately deployed models can use interfaces compatible with the OpenAI API. The availability of different models does not mean they provide identical results, and it does not mean every model can be connected without adaptation. Test model connectivity, response quality, and cost separately in the target environment.
The entry point also depends on the task. The Data Agent sidebar suits continued analysis, creation, and interpretation in the context of the current page. Standalone ChatBI better supports an ongoing conversation about data questions, charts, and dashboards. Agent, Workflow, and API modes do not have identical capability boundaries. Before an external integration, determine whether the need is in-page assistance, a standalone data-question entry point, or an interface called by a business system, then select the model and data-preparation approach.
Data Agent should work only within the data scope already authorized for the user. During acceptance, have different roles ask the same question. Check the visible data packages, connection permissions, application data permissions, and row and column permissions. Then check the metric definition, filters, and chart used in the result. Only then can a team determine whether an answer merely sounds plausible or is correct and compliant.
4. Make Accuracy Work That Can Be Reviewed
Start with a set of samples drawn from real business questions. Cover synonyms, cross-table analysis, time definitions, unauthorized data, and repeat questions after data updates. For each question, record the expected data scope, metric definition, and manual verification result, then adjust data descriptions, vectorization, and model configuration. Use this set of questions to measure accuracy continuously. An unexplained one-time accuracy rate or fixed response time should not stand in for product performance.
The sample set should include both positive and negative questions. Positive questions test whether synonyms, cross-table analysis, and complex metrics can find the correct data. Negative questions test how the system handles unauthorized fields, empty results, ambiguous definitions, and nonexistent metrics. After each configuration adjustment, compare changes against the same question set and retain manually verified query results and feedback. Do not present only the cases with polished answers.
The following is an example of a data-question acceptance record. The fields illustrate how to fix expectations; they are not a HENGSHI product configuration file.
question: What was sales revenue in East China last month?
data_scope: Authorized sales data package
metric: Sales revenue (using the confirmed definition)
time_field: Payment date
expected: Matches the manual benchmark resultPost-launch troubleshooting also needs evidence. For errors, record the complete execution log, the response from the failed request, and the target version. For an answer that appears reasonable but has an incorrect value, retain the original question, user identity, selected data scope, metric definition, and benchmark calculation. Future improvements have direction only after a failure is assigned to a specific stage in data, retrieval, model, permissions, or interaction.
Scenario: Asking About Slow-Moving Inventory at a Retail Chain
A retail chain wants to ask in natural language, “Which stores have the most pronounced inventory backlog this week?” The business team first defines “backlog”: which inventory snapshot date to use, the sales window to consider, how to calculate available inventory, and the store and product granularity for comparison. The implementation team then writes these definitions into field descriptions and metrics for the sales and inventory data packages, and adds common expressions such as “backlog” and “slow-moving.”
When someone asks the question, Data Agent should select relevant datasets and metrics within that person’s authorized data scope. If it cannot find the inventory field, first check descriptions, synonyms, and vectorization tasks. If it selects the right field but ranks results unusually, check the inventory snapshot, sales window, and join granularity instead of adjusting only the model prompt.
For acceptance, hold the same stores, products, and dates constant, use a manual query as the benchmark, then have different store roles repeat the question. The criteria are that rankings and values match the confirmed definitions and that unauthorized stores do not appear in the results. This is a reviewable implementation scenario, not real customer outcome data.
HENGSHI’s practical focus is to place data preparation, semantic retrieval, model capability, and permission review in one delivery chain. AI lowers the barrier to asking questions; reliable analysis still requires business definitions and results that can be checked.