Trustworthy Natural-Language Analytics Needs Five Engineering Guardrails Beyond RAG

RAG solves an important problem: a language model cannot learn a company’s private warehouse schemas, field names, and metric definitions during general training. At query time, the system retrieves relevant schema and business knowledge and supplies that context to the model. Retrieval improves the chance of selecting the right objects, but it cannot guarantee a correct analytical answer.

An enterprise query can fail before retrieval, during retrieval, at generation, or after execution. Trustworthy ChatBI turns the complete chain into observable and testable control points.

What RAG Solves and What It Does Not

A typical flow understands the question, retrieves knowledge, assembles context, generates a query, executes it, and explains the result. RAG avoids filling the model context with hundreds of tables and thousands of fields. It also lets teams update knowledge after a schema change without retraining the model.

RAG does not automatically resolve these conditions:

• the knowledge base contains both approved definitions and obsolete drafts;
• the most similar dataset is outside the current user’s permissions;
• “revenue” lacks a tax, currency, organization, or time definition;
• the SQL is valid, but its aggregation grain or magnitude is wrong;
• a user correction remains in chat history instead of updating a governed asset.

Vector retrieval is one part of trustworthy analytics, not its final guarantee.

Guardrail One: Retrieve Governed Assets

The knowledge base should contain maintained dataset documentation, metric definitions, synonyms, lineage, and example queries. Each knowledge unit needs a version, owner, business domain, effective period, and permission labels. Schema and metric changes should update dependent indexes, while obsolete versions leave the default retrieval set.

Embedding every DDL statement, chat transcript, and historical document mixes drafts, duplicate definitions, and retired fields. Knowledge quality must be governed before it enters the vector store. The model should not be expected to repair it during inference.

Guardrail Two: Combine Semantic Retrieval with Structural Filters

Vector similarity finds concepts with related meaning, but it does not inherently respect tenants, permissions, data grain, or version boundaries. A safe retrieval flow first restricts the candidate set with structured policy, then performs semantic retrieval and reranking.

Filter	Purpose
Tenant and user policy	Exclude invisible datasets, fields, and metrics
Business domain and current application	Reduce false matches across domains
Data version and effective period	Avoid obsolete schemas and definitions
Metric grain and compatible dimensions	Prevent invalid aggregation and joins
Language and terminology mapping	Map Chinese and English questions to the same object

Permission filtering belongs in the retrieval service. Masking the answer after generation is too late.

Guardrail Three: Disambiguate Definitions Before Generation

If the agent finds several revenue metrics, it should show candidate definitions or narrow the set using the current application and user context. It must ask when time, organization, currency, tax treatment, or aggregation grain changes the answer.

Disambiguation does not require a technical form. The system can surface only the decisions that matter, such as order date versus payment date or gross versus net revenue. One short confirmation can prevent an entire report from being rebuilt.

Guardrail Four: Validate the Executed Result

A syntactically valid query can still produce a wrong business result. Different checks belong at different stages:

Stage	Example checks
Before execution	Read-only policy, table and field allowlists, query cost, tenant filters
After execution	Empty result, magnitude, time range, aggregation grain, outliers
Before explanation	Evidence supports the conclusion; definitions and filters are disclosed
High-risk metrics	Cross-check against certified reports or published metric results

An anomalous result should enter correction or human review instead of being converted into a confident statement. The execution identity, query text, and validation outcome must remain bound to the same task record.

Guardrail Five: Feed Corrections into Governance and Audit

A user correction should not remain a chat memory. The team must classify the failure as retrieval, missing semantic definition, query generation, or data quality, then update the knowledge base, metric platform, example library, or test set.

Audit records should include the question, retrieved knowledge, metric version, generated query, execution identity, validation result, and final output. These records support an offline evaluation set that measures retrieval recall, definition accuracy, execution success, and validation interception.

Choose Models After the Engineering Controls

A larger model may improve language understanding and complex reasoning, but it cannot repair inconsistent metric definitions. Build governed assets, permission-aware retrieval, disambiguation, and result validation before comparing model cost, latency, and deployment options.

Private deployments also need versioned embedding models, vector-index backups, offline update procedures, and recovery plans. Models can change; knowledge versions and evaluation baselines must remain. Trustworthy ChatBI is a property of the complete system.

Engineering Details

1. Why ChatBI Needs RAG

1.1 Limits of a Model-Only Design

Enterprise analytics places different constraints on a language model:

Constraint	General LLM task	Enterprise BI task
Knowledge	Broad public corpus	Private schemas, metric definitions, and industry terms
Accuracy	Some creative variation is acceptable	SQL and query results must be deterministic
Schema size	Input often spans a few hundred words	Warehouses can contain hundreds of tables and thousands of fields
Context	Text can be shortened	Table structures, field descriptions, and metric definitions must remain intact
Freshness	A training cutoff can be acceptable	Data and schemas change often

A model stores knowledge in parameters learned during training. Warehouse names, fields, and definitions belong to each enterprise and change outside that training process.

1.2 The RAG Query Path

Question: "What was East China GMV last month?"
        │
        ▼
┌─────────────────────────────────────────┐
│ Step 1: Encode the question             │
│   Convert the request into a vector     │
└──────────────┬──────────────────────────┘
               ▼
┌─────────────────────────────────────────┐
│ Step 2: Retrieve related knowledge      │
│   Find schemas, fields, and metrics     │
└──────────────┬──────────────────────────┘
               ▼
┌─────────────────────────────────────────┐
│ Step 3: Assemble context                │
│   Combine knowledge and the request     │
└──────────────┬──────────────────────────┘
               ▼
┌─────────────────────────────────────────┐
│ Step 4: Generate SQL                    │
│   Use the selected governed context     │
└──────────────┬──────────────────────────┘
               ▼
┌─────────────────────────────────────────┐
│ Step 5: Execute and validate            │
│   Run SQL and return the checked result │
└─────────────────────────────────────────┘

Teams can re-embed knowledge after a schema change without retraining the language model. Retrieval also limits the prompt to relevant schema fragments and records which knowledge units supported each query.

2. Vector Service Architecture

2.1 Components

vector-serve                         vector-postgres
Embedding inference  ◄──────────►   pgvector storage and search
        │                                      │
        │ HTTP /v1/embeddings                  │ JDBC
        └──────────────────┬───────────────────┘
                           ▼
                  HENGSHI SENSE Core
                  ├── Question understanding
                  ├── RAG retrieval
                  ├── SQL generation
                  └── Result validation

vector-serve accepts text and returns embeddings. vector-postgres stores vector representations of dataset metadata, metric definitions, and field documentation, then performs approximate nearest-neighbor search through pgvector.

2.2 Why multilingual-e5-base

Criterion	multilingual-e5-base	OpenAI text-embedding-3	BGE-large-zh
Languages	Chinese, English, Japanese, Korean, and more	Broad language support; weaker Chinese fit for this use	Primarily optimized for Chinese
Private deployment	Fully offline	Cloud API required	Offline deployment supported
Dimensions	768	1,536 or 3,072	1,024
Inference	About 50 ms per sentence	Fast cloud inference	About 80 ms per sentence
Retrieval	Strong multilingual baseline	Strong	Strong for Chinese
Hardware	1 to 2 GB GPU memory	Managed cloud service	2 to 4 GB GPU memory

The 768-dimensional representation balances retrieval quality, storage cost, and compute. Offline deployment addresses data-residency requirements, and one multilingual model supports global organizations.

2.3 Script and Docker Compose Deployment

Connect HENGSHI SENSE Core to the two vector services:

# Edit the environment configuration
vi /opt/hengshi/conf/hengshi-sense-env.sh

# Add the service endpoints
export VECTOR_DB_URL="jdbc:postgresql://<VECTOR_PG_HOST>:54321/postgres"
export VECTOR_ENDPOINT="http://<VECTOR_SERVE_HOST>:3000/v1/embeddings"

For Docker Compose, add the same endpoints to .env:

VECTOR_DB_URL=jdbc:postgresql://vector-postgres:5432/postgres
VECTOR_ENDPOINT=http://vector-serve:3000/v1/embeddings

For Kubernetes, edit the HENGSHI SENSE ConfigMap:

kubectl -n hengshi edit configmap hengshi-sense

apiVersion: v1
kind: ConfigMap
metadata:
  name: hengshi-sense
  namespace: hengshi
data:
  VECTOR_DB_URL: "jdbc:postgresql://vector-postgres:5432/postgres"
  VECTOR_ENDPOINT: "http://vector-serve:3000/v1/embeddings"

2.6 Deployment Checks

Check embedding inference first:

curl -X POST http://localhost:3000/v1/embeddings \
  -H 'Content-Type: application/json' \
  -d '{"input": ["East China sales last month"], "model": "intfloat/multilingual-e5-base"}'

Then verify pgvector and run one test transformation:

psql -h localhost -p 54321 -U postgres -d postgres

SELECT * FROM pg_extension WHERE extname = 'vector';

SELECT vectorize.transform_embeddings(
  input => 'embedding test',
  model_name => 'intfloat/multilingual-e5-base'
);

Error	Likely cause	Check
query vector db fail	The two vector services cannot communicate	Health endpoints and network paths
connection refused	Closed port or firewall rule	Port exposure and network policy
model not found	Model files were not loaded	Model copy and extraction step
out of memory	Insufficient GPU memory	Replica count and GPU allocation

5. Operating Practices

5.1 Vector Store Operations

Practice	Recommendation	Reason
Knowledge updates	Re-embed after each schema change	Prevent retrieval of obsolete structures
Index type	Use IVFFlat or HNSW	Balance recall and latency
Monitoring	Track retrieval latency, recall, and embedding QPS	Detect capacity and quality regressions
Backup	Back up vector-postgres	Rebuilding embeddings has a material cost

5.2 Model Selection

Workload	Suggested model	Reason
High-volume simple queries	DeepSeek or Groq	Low latency and high throughput
Complex analysis	Claude 3.5 or GPT-4o	Strong reasoning
Offline deployment	Qwen2.5-27B or GLM-5	Private deployment support
Limited budget	MiniMax-M2.5	MoE cost profile
Chinese workloads	DeepSeek or Qwen	Chinese language quality

5.3 Accuracy Work

1. Keep vector knowledge synchronized with the metric platform.
2. Maintain a controlled synonym map for business terminology.
3. Add approved examples for common query patterns.
4. Validate result magnitude and shape after SQL execution.
5. Route user corrections into prompts, retrieval assets, and evaluation sets.

6. Summary

HENGSHI SENSE combines vector-serve and vector-postgres to retrieve enterprise schemas, metric definitions, and business terminology. Production trust also depends on permission filters, definition disambiguation, result checks, index maintenance, and audit feedback. Teams should establish those controls before selecting a model by latency, cost, or deployment boundary.

Source and Verification Note

Internal material was used to organize HENGSHI capabilities and engineering methods. Competitor and version information was checked against official pages available on August 26, 2026. Features can vary by release, region, license, and deployment mode; confirm them in the target environment before purchase or publication.

Further reading: HENGSHI SENSE Product and Technology White Paper.