Trust is a fundamental requirement for an analytical system, and something we take extremely seriously with Vantage. If you can’t trust the answers, then you can’t confidently apply them.

When compared with traditional products, the recent advances in Generative AI provide immense power and flexibility with the input to systems, but it comes at the cost of predictability and confidence in the output.

With Vantage, we apply the following approaches to ensure accurate, and consistent results and to maximise the trust in our output.

Using deterministic processes where appropriate

Vantage harnesses a blend of traditional data pipeline steps with state-of-the-art, multi-agent AI-enhanced processes. This fusion allows us to deliver clear and reliable answers through deterministic, AI-informed, and wholly AI-driven processes.

As an example our core “analytical and numerical skills” employ wholly deterministic processes to guarantee predictable answers. These are hand-crafted modules that are rigorously tested to ensure correct, consistent function over a wide range of inputs.

For a closer look at the workings of the full pipeline please see our article: “So how does it work?”

Definition clarity and refinement

Vantage interprets the structure and nature of the data to facilitate the data analysis steps. This is performed at the point the data is connected.

We use two techniques to create these interpretation:

Third party services, with well-defined APIs and structures, have default interpretations that have been hand-optimised by our team.

For data repositories, such as custom databases, we use a fine-tuned AI to investigate the schema and data.

Most critically, this interpretation is fully visible and editable within the product itself to ensure it aligns with your internal definitions.

Best Practice use of ML and LLMs

The field of Generative AI and LLMs is evolving rapidly. We continually refine our use of LLM prompting techniques to maximise the accuracy of evaluations, and to detect any hallucinations or anomalies. These techniques include: chain-of-thought execution, self-reflection, and multi-agent review.

Testing Loops

We consistently test our AI agents with unit-level tools to gauge their role-specific performance, ensuring our changes continue to push us forward.

Knowing what we don't know

Technical reliability and accuracy are paramount to trust. A misleading, or incorrect number is worse than no number at all.

Built into our testing, optimization and general ethos is that it's okay for Vantage to not be able to directly answer a question. Vantage has an understanding of the data, and techniques it has available to it, and will tell the user when it's unable to answer a question.

Insights over numbers

The nature of an analysis is as critical as the output. Put another way, analysis is only meaningful in the context of the methodology used.

For instance - we can forecast that Revenue will increase by 5% year-on-year. But that just invites more questions about how that forecast was done and its accuracy. That 5% doesn’t convey the story.

It's in the "why" behind the forecast - like the assumptions made, driving factors, and accounting for recent changes - where analytical value most frequently lies.

"How can I trust this?" becomes "Here's why this analysis matters..."