MongoDB, Mastra Unveil Autonomous Fraud Detection System Marshal
Ai4
Consolidating data retrieval and workflow state in a single platform allows AI fraud agents to remain fully auditable and compliant under regulatory oversight.
## Background
Ai4 published this video on YouTube on August 18, 2026. According to the page description, the session features Brandon Barros from Mastra and Mohammad Daoud Farooqi from MongoDB presenting "Marshal," an AI-driven fraud investigation console that combines MongoDB Atlas's unified vector, text, and graph retrieval with Mastra's durable workflows and audit trails.
## TL;DR
The presentation addresses the operational and financial challenges of modern fraud detection, where balancing rapid legitimate transaction approvals with fraud prevention is critical. The speakers introduce Marshal, an AI agent system that leverages MongoDB Atlas for hybrid vector search, full-text search, and multi-hop graph retrieval alongside Mastra's durable workflow control plane. They walk through common fraud scenarios—including clean approvals, immediate rejections, A-B-C mule networks, and transaction structuring—to illustrate how deterministic workflows and human-in-the-loop steps operate. Finally, they highlight production readiness through Mastra's Trace Intelligence and runtime observability while advocating for data infrastructure consolidation on MongoDB Atlas.
## Atmosphere & style
The session carries a practical, technical, and solution-oriented tone characteristic of an engineering tech talk. The dynamics between Mohammad and Brandon are collaborative and balanced, moving seamlessly from real-world human stakes and business dilemmas to technical architecture diagrams and workflow mechanics. Watching the talk feels like a structured deep dive for developers, grounding advanced AI agent concepts in concrete enterprise engineering and regulatory compliance.
## Key takeaways
- [00:04](https://www.youtube.com/watch?v=xw2c-M6svKc&t=4s) Mastra equips TypeScript developers with agent primitives, built-in observability, and evaluation metrics to build, track, and refine AI agents.
- [00:04](https://www.youtube.com/watch?v=xw2c-M6svKc&t=4s) MongoDB Atlas unifies OLTP transaction records, full-text search, vector search, hybrid search, and graph lookups in a single data layer.
- [02:29](https://www.youtube.com/watch?v=xw2c-M6svKc&t=149s) Effective fraud detection requires surrounding AI models with multi-source evidence, strict policy guardrails, and audit trails rather than relying on simple LLM prompts.
- [02:29](https://www.youtube.com/watch?v=xw2c-M6svKc&t=149s) When AI agents operate with low confidence, they must gracefully escalate flagged transactions to human review through human-in-the-loop workflows.
- [05:21](https://www.youtube.com/watch?v=xw2c-M6svKc&t=321s) The Marshal console combines MongoDB's hybrid search and graph traversal with Mastra's durable workflows to process flagged payments.
- [05:21](https://www.youtube.com/watch?v=xw2c-M6svKc&t=321s) Marshal detects complex financial fraud schemes like mule account traversal (A-B-C fraud) via graph lookups and amount structuring through historical aggregation.
- [10:00](https://www.youtube.com/watch?v=xw2c-M6svKc&t=600s) Mastra workflows bring determinism to agentic systems through sequential steps, branching, loops, and durable state snapshots.
- [10:00](https://www.youtube.com/watch?v=xw2c-M6svKc&t=600s) State snapshots in Mastra enable execution resilience, suspend-and-resume human approvals, retries, and time-travel debugging.
- [14:02](https://www.youtube.com/watch?v=xw2c-M6svKc&t=842s) Mastra's Trace Intelligence categorizes agent execution traces across Goal, Behavior, Outcome, and Sentiment to drive a self-improving agentic learning loop.
- [14:02](https://www.youtube.com/watch?v=xw2c-M6svKc&t=842s) Consolidating transactional, vector, graph, and full-text search workloads onto MongoDB Atlas prevents enterprise infrastructure sprawl.
## Detailed walkthrough
### [00:04](https://www.youtube.com/watch?v=xw2c-M6svKc&t=4s) Introduction and Session Agenda
Mohammad Daoud Farooqi, Partner Solutions Architect for AI at MongoDB, and Brandon Barros, Founding Technical GTM and Special Projects Engineer at Mastra, introduce their presentation on building fraud detection agents at scale using MongoDB and Mastra. They outline a five-part agenda starting with an overview of both platforms, moving into fraud detection as a business problem, and showing how AI acts as a solution via a live demo of "Marshal"—an AI agentic fraud detection system. They then promise a look under the hood at how MongoDB and Mastra power Marshal, explain how to take a demo into a production-ready system, and wrap up with key takeaways.
Brandon introduces Mastra as a leading TypeScript AI agent framework and platform that equips developers with primitives to build, observe, and iterate on AI agents. He highlights Mastra's built-in observability for tracking execution traces step-by-step, along with evals, metrics, experiments, and trace intelligence to refine agents over time.
Mohammad explains that MongoDB serves as the essential data layer. Its flexible JSON document model functions as an OLTP system of record for application transactions. Beyond raw document storage, MongoDB Atlas provides full-text search, vector search, hybrid search, graph lookups via `$graphLookup` to traverse network hops, ACID transactions, and change streams to reliably trigger and capture every data change.
- Mastra boasts over 1.3 million weekly downloads on GitHub and is used by over 100,000 companies building TypeScript AI agents.
- MongoDB unifies operational data storage with semantic retrieval (vector + full-text) and relationship graph traversal.
---
[02:29](https://www.youtube.com/watch?v=xw2c-M6svKc&t=149s) The Business Challenge of Fraud Detection and the Evolution of AI Solutions
Mohammad paints a vivid scene of the human and financial stakes in fraud detection: it is 4:47 p.m. on a Friday afternoon, and Maya, owner of a small construction business, submits a $3,200 payroll transaction. Her employees are depending on those funds on time for rent and childcare. Simultaneously, a massive $75,000 transaction hits the system and gets flagged. A human analyst faces a tough dilemma: rushing through approvals risks letting actual fraud slip by, but moving too slowly forces innocent people like Maya to suffer severe real-world delay.
The fundamental issue in modern fraud detection is not simply asking an LLM "is this fraud or not?" Instead, it requires surrounding the agent with scattered evidence, clear policy rules, and immutable audit trails, while identifying complex hidden relationships across account networks. Crucially, when an AI agent lacks high confidence, it must gracefully escalate the case to human review. Mohammad maps out how the industry reached this point: starting from basic manual database searches, advancing to Retrieval-Augmented Generation (RAG) for basic context, and culminating in modern autonomous AI agents capable of using tools, retrieving complex multi-source evidence, and logging fully auditable decisions.
- High-impact false positives create major friction for innocent users, while false negatives cause severe financial loss.
- Audit trails must remain clear and intact so that decisions can be inspected by regulators or used for reinforcement learning even 6 months or years later.
---
[05:21](https://www.youtube.com/watch?v=xw2c-M6svKc&t=321s) Introducing Marshal: Features, Workflows, and Specific Scenarios
The speakers present "Marshal," a fraud investigation console and AI agentic system designed to handle flagged transaction queues. Every flagged payment enters a queue where MongoDB executes hybrid searches—blending vector similarity search for semantic matching with full-text search to pinpoint exact account numbers, amounts, and historical patterns—while Mastra orchestrates the agentic reasoning workflows. Policies and governance rules are maintained in MongoDB, enabling the agent to execute decisions against deterministic guardrails or flag low-confidence items for human review.
```
+-----------------------------------------------------------------+
| Flagged Transaction |
+-----------------------------------------------------------------+
|
v
+-----------------------------------------------------------------+
| MongoDB Atlas Data Layer |
| - Full-Text Search - Vector Search - Graph Hops ($lookup) |
+-----------------------------------------------------------------+
|
v
+-----------------------------------------------------------------+
| Mastra Agent Control Plane |
| - Reasoning & Prompts - Durable Workflows - Audit Traces |
+-----------------------------------------------------------------+
| |
v v
[ High Confidence ] [ Low Confidence ]
| |
v v
+-----------------+ +-----------------+
| Approve / Reject| | Human-in-Loop |
| Auto-Resolution | | Review Queue |
+-----------------+ +-----------------+
```
Brandon and Mohammad walk through several distinct transaction patterns handled by Marshal:
1. **Clean Approval:** A routine, recurring payroll transaction to a known vendor is quickly matched against historic patterns with zero red flags and instantly approved.
2. **Strong Rejection:** A payment directed to an unknown overseas merchant immediately following a suspicious account login attempt is flagged and flatly rejected.
3. **Mule Account Traversal (A-B-C Fraud):** Entity A wants to send money to Entity C without triggering fraud alerts, so it routes funds through intermediary Entity B (a known "mule"). Marshal runs graph lookups, identifies previous A-to-B and B-to-C payments, uncovers the hidden link, and catches the scheme.
4. **Amount Structuring:** A user attempts to bypass a $5,000 regulatory reporting threshold (which requires SSN or PAN card details) by splitting transactions into multiple smaller amounts just below $5,000. Marshal tracks and aggregates these split transfers over time, flagging them for human review or rejecting them based on risk scoring.
5. **High-Value Escalation ($75,000 Transaction):** The agent gathers evidence, runs reasoning chains over historical data, and routes the transaction directly to a human reviewer alongside its chain of thought. Once the human takes action, the decision is written to an immutable audit log for future reinforcement learning.
---
[10:00](https://www.youtube.com/watch?v=xw2c-M6svKc&t=600s) Deep Dive into the Architecture: MongoDB Data Layer & Mastra Workflows
Mohammad details how MongoDB serves as the agent's memory and evidence engine while Mastra functions as its brain and control plane. On the database side, MongoDB Atlas provides vector search for semantic recall, `$search` for precise full-text search, reciprocal rank fusion (RRF) to combine text and vector scoring, and `$graphLookup` to trace financial hops across complex network graphs. Furthermore, every action and decision taken by the agent is preserved in a tamper-proof event stream.
On the application side, Brandon breaks down Mastra's core agentic control plane into three pillars: LLM reasoning logic, workflow orchestration, and deep runtime observability. He clarifies that workflows introduce deterministic structure into inherently non-deterministic AI architectures.
- **Sequential Steps:** Linear transitions from step A to B to C.
- **Parallel Steps:** Simultaneous execution of independent sub-workflows.
- **Branching:** Conditional routing based on intermediate logic (e.g., approve vs. suspend vs. reject).
- **Loops:** Repeated retrieval or tool execution steps to gather additional context until confidence targets are met.
- **Snapshots & Durability:** Workflows store full execution snapshots at every state transition. This powers suspend/resume for human-in-the-loop approvals, state retries, time-travel debugging, and multi-server execution resilient to system crashes.
---
[14:02](https://www.youtube.com/watch?v=xw2c-M6svKc&t=842s) Moving from Demo to Production: Observability and Infrastructure Consolidation
Transitioning an AI agent from a controlled demo into a resilient, enterprise-scale production system requires robust observability and simplified infrastructure. Because Mastra integrates both the application framework and observability layer, it captures full execution traces directly at agent runtime—tracking every individual line of code, model invocation, and tool call without relying on manual instrumentation. Developers can evaluate token usage and monetary cost down to specific model, tool, or workflow steps.
Brandon highlights Mastra's newly released **Trace Intelligence** capability, which automatically categorizes execution traces across four dimensions:
- **Goal:** The intended target or objective of the execution run.
- **Behavior:** The concrete sequence of tool calls and actions taken.
- **Outcome:** The final output or resolution achieved.
- **Sentiment:** The tone or "vibes" recorded during the interaction.
Trace intelligence serves as the foundation for Mastra's self-improving "agentic learning loop," creating an automated iteration layer between evaluation metrics and source code. Mohammad concludes by warning against "infrastructure sprawl." Instead of maintaining four separate niche databases—one for transactional records, another for vector search, a third for graph processing, and a fourth for full-text search—enterprises can consolidate all workload types onto MongoDB Atlas to keep architecture simple, maintainable, and easy to scale under regulatory scrutiny.
- Attendees are invited to test the demo app, review the public GitHub repository, or reach out to `mongodbpartners@mongodb.com`.
- Free copies of the AI agent development book authored by Mastra's CEO, Sam, are distributed by team member Ashwin after the session.
## Notable quotes
> [02:29](https://www.youtube.com/watch?v=xw2c-M6svKc&t=149s) it is 4:47 p.m. on a Friday afternoon, and Maya, owner of a small construction business, submits a $3,200 payroll transaction. — Mohammad Daoud Farooqi
> [02:29](https://www.youtube.com/watch?v=xw2c-M6svKc&t=149s) High-impact false positives create major friction for innocent users, while false negatives cause severe financial loss. — Mohammad Daoud Farooqi
> [02:29](https://www.youtube.com/watch?v=xw2c-M6svKc&t=149s) Audit trails must remain clear and intact so that decisions can be inspected by regulators or used for reinforcement learning even 6 months or years later. — Mohammad Daoud Farooqi
> [14:02](https://www.youtube.com/watch?v=xw2c-M6svKc&t=842s) Goal, Behavior, Outcome, and Sentiment — Brandon Barros
## Video description
Brandon Barros - Founding Technical GTM, Masta
Mohammad Daoud Farooqi - Partner Solutions Architect, A.I., MongoDB
Fraud detection in an agentic world is hard: evidence for a decision is scattered, only some calls should reach a human, and all must stay auditable. This talk builds Marshal, a fraud investigation console. It assembles the evidence server-side in MongoDB Atlas, running vector, text, and graph retrieval over one collection. Mastra's durable workflows wrap that in a deterministic flow that suspends for an approver and resumes. You get an audit trail, PII protection, and a clear path to self-improvement from production traces. Come if you're putting an agent where a regulator will look.
Mohammad Daoud Farooqi of MongoDB and Brandon Barros of Mastra demonstrate that production-grade fraud detection requires pairing unified data layers with durable, observable agent workflows rather than isolated LLM evaluations. Using their reference system, 'Marshal,' they show how combining hybrid search, graph traversal, and human-in-the-loop controls resolves complex financial fraud while preserving regulatory compliance.
The Operational Tension Between Speed and Accuracy (2:29)
Modern fraud operations face a continuous trade-off between user friction and risk exposure. Farooqi illustrates this dynamic with a scenario where an analyst must evaluate a routine $3,200 payroll transfer critical to a small business alongside a concurrent $75,000 flagged transaction. Rushing decisions creates severe financial losses through false negatives, whereas over-flagging delays legitimate transfers, disrupting essential customer operations. Farooqi argues that fraud detection cannot rely on simple, ungrounded binary prompts like asking a language model whether a payment is fraudulent.
To address this dilemma, systems have evolved from static database lookups to Retrieval-Augmented Generation (RAG) and, finally, to autonomous agentic architectures. An effective fraud agent must evaluate surrounding evidence, enforce policy rules, trace relational account networks, and maintain immutable audit records. When confidence thresholds are not met, the system must deterministically escalate cases to human specialists so that decisions can be inspected by regulators or used for reinforcement learning months later.
Targeting Complex Schemes with Graph and Pattern Analysis (5:21)
The speakers present Marshal, an agentic fraud console built to evaluate flagged transaction queues through hybrid data querying and structured reasoning paths. Marshal executes deterministic actions for clear-cut items—such as auto-approving historical payroll or rejecting transfers initiated after suspicious logins to unverified overseas merchants—while deploying targeted logic for sophisticated criminal patterns.
For intermediary evasion, known as 'A-B-C fraud,' the system uses graph lookups to discover hidden connections where Entity A routes capital to restricted Entity C through an intermediate mule account. To catch amount structuring (smurfing), the agent monitors and aggregates repeated transfers intentionally kept beneath regulatory reporting triggers, such as the $5,000 threshold that mandates tax identifier submissions. In high-value anomalies, such as an ambiguous $75,000 transfer, Marshal synthesizes historic evidence, details its chain of thought, and routes the case to human analysts, recording the final outcome to an immutable audit log.
Architecting Deterministic Workflows on Unified Data (10:00)
Marshal's technical design splits responsibilities between MongoDB Atlas as the underlying memory engine and Mastra as the orchestration control plane. Mastra introduces deterministic structures to non-deterministic model calls by utilizing sequential steps, parallel branches, and iterative retrieval loops. Barros highlights that Mastra captures complete state snapshots across every transition, which provides fault tolerance against crashes, enables time-travel debugging, and allows workflows to pause indefinitely until a human approves or rejects an escalation.
Simultaneously, MongoDB acts as both the transactional store and the multi-modal evidence engine. Rather than moving data across disparate tools, the agent queries MongoDB using reciprocal rank fusion (RRF) to combine semantic vector search with full-text search (`$search`) for exact account IDs. It also leverages `$graphLookup` to trace entity relationships across multiple transactional hops, feeding real-time context into Mastra while logging every agentic intervention in an immutable change stream.
Production Observability and Consolidating Infrastructure (14:02)
Transitioning agentic systems from prototype to production requires granular observability and architectural simplicity. Barros explains that Mastra captures traces at the framework runtime—tracking every line of code, model invocation, tool call, token tally, and operational cost without separate instrumentation. Mastra's Trace Intelligence feature categorizes execution logs across four dimensions: goal, behavior, outcome, and sentiment, which creates a structured iteration loop to refine agent performance against real-world traffic.
Farooqi cautions against 'infrastructure sprawl,' which occurs when organizations stitch together standalone engines for transactional records, vector search, graph analysis, and full-text search. Maintaining fragmented architectures complicates governance, multiplies failure points, and weakens audit integrity under regulatory scrutiny. Consolidating transactional workloads, vector indexing, text search, and graph traversal into MongoDB Atlas preserves a single source of truth for both autonomous agents and regulatory reviews.
The presentation concludes that scalable agentic fraud detection depends on eliminating fragmented database architectures and bounding autonomous model reasoning with durable, observable workflows. The speakers emphasize that while agents can automate pattern analysis and routine decisions, regulatory accountability demands persistent audit trails and clear escalation paths for human intervention.
[0:29] Mohammad Dawood Farooqi: 这笔钱对她的员工至关重要——有人等着交房租，有人等着付小孩的托儿费。这不仅仅是一笔转账，更是对员工的承诺。
[0:29] Mohammad Dawood Farooqi: 如果审得太快、草率批准，欺诈交易就会溜之大吉；如果查得太慢，像 Maya 这样合法合规的普通人就会深受其害。
[0:29] Mohammad Dawood Farooqi: AI 之所以成为破局关键，是因为欺诈证据往往分散在不同的异构系统中，误报（False Positive）会造成极高的业务影响，且任何决策都必须留存长达数年的不可篡改审计追踪
[0:29] Mohammad Dawood Farooqi: 所有 Agent 作出的自动化决策，必须具备可溯源的推演链条，满足监管机构 6 年甚至更久的合规审计，并为后续的强化学习积累数据。
[0:04] Mohammad Dawood Farooqi: 通过单套引擎整合 OLTP 事务、混合检索与图查询，极大地简化了 Agent 所需的数据架构。
[0:52] 针对试图绕过风控的‘A → B → C’间接交易（B 为骡子中转账户），Marshal 自动通过图查询与历史检索穿透链路，发现潜在的 A 到 C 隐藏关联并标记拦截。
[10:00] MongoDB 构筑了 Agent 的‘记忆与证据链’，Mastra 则是 Agent 的‘大脑与控制平面’。
[10:00] Brandon Barros: 为了在天然具备随机性（Non-deterministic）的 LLM 之上构建出确定性（Deterministic）的风控系统，Mastra 提供了极其丰富的工作流编排原语
[10:00] Brandon Barros: 当需要人工审核时，工作流可直接‘挂起（Suspend）’，待分析师在 UI 上点击确认后再‘恢复（Resume）’，甚至支持历史步骤回放（Time Travel）与失败重试。
[14:02] Brandon Barros: Mastra 兼具框架层与可观测性层，这使其能够深入 Agent 运行时的最底层——不只是捕捉 Agent 主动暴露的信息，还能追踪到每一次工具调用、每一行代码与工作流的具体执行 Trace。
[17:02] Mohammad Dawood Farooqi: 构建一个 Agent Demo 并不难，真正的挑战在于将 Demo 转化为具备合规审计追踪、完整证据链与明确责任归属的生产级系统。
Source quotes
[2:29] Mohammad Daoud Farooqi: it is 4:47 p.m. on a Friday afternoon, and Maya, owner of a small construction business, submits a $3,200 payroll transaction.
[2:29] Mohammad Daoud Farooqi: High-impact false positives create major friction for innocent users, while false negatives cause severe financial loss.
[2:29] Mohammad Daoud Farooqi: Audit trails must remain clear and intact so that decisions can be inspected by regulators or used for reinforcement learning even 6 months or years later.
[14:02] Brandon Barros: Goal, Behavior, Outcome, and Sentiment
Original transcript
[0:04] Thanks you all for joining us today.
Today we'll be discussing about fraud
[0:08] detection agents at scale using MongoDB
and Mastra. I'm Mohammad Dawood Faruqi,
[0:12] a partner solutions architect for AI at MongoDB.
And My name is Brandon. I'm the founding technical
[0:18] go-to-market and special projects engineer at
Mastra. Now to start off, a quick summary of what
[0:24] we're going to go over today. First, we're just
going to do an overview of Mastra and MongoDB.
[0:29] In section two, we're going to talk about fraud
detection as a business problem. In section three,
[0:35] we're just we're then going to go into where AI
comes in as a perfect solution to this problem
[0:40] and demo Marshall, which is our AI agentic system
that helps you detect fraud. In section four,
[0:47] we're going to take a look under the hood and
see how MongoDB and Mastra build out Marshall.
[0:52] And in section five, we're going to journey over
to how you take something like this, a great demo,
[0:57] and get it ready for production. And then
to wrap up, we're just going to talk about
[1:01] some of the takeaways from today's chat.
And so to start off as an introduction,
[1:08] what is Mastra? Mastra is a leading
TypeScript AI agent framework and platform.
[1:14] So it gives you all the tools you need to
build, observe, and iterate on AI agents.
[1:20] First, you can build out great agents with our
framework that has a ton of powerful primitives
[1:24] out of the box. You can use our observability
functionality to see exactly where your agents
[1:30] are at and track their traces every single step
they're taking. And you can also use our evals,
[1:36] metrics, and experiments, and trace intelligence
to help build your agents over time. Mastra has
[1:43] over 1.3 million weekly downloads on GitHub, and
over 100,000 companies are building their agents
[1:48] on Mastra. So if you're looking to get TypeScript
agents into production, Mastra is a great place
[1:54] to get started. And so what is MongoDB giving
you? MongoDB is providing you the data layer.
[2:00] It is giving you the documents, a flexible JSON
document model that can store your transactions,
[2:05] serve as your OLTP system of records for your
app. It gives you search, full-text search,
[2:11] vector search, and a combination of both being the
hybrid search. It also gives you the graph look
[2:16] up, so you can traverse the network hops. And it
gives you asset transactions and change streams,
[2:22] so that every change gets triggered
and captured and stored safely.
[2:29] So, the fraud detection problem. So, consider
it's Friday afternoon around 4:47 p.m. And Maya,
[2:37] who owns a small construction company, makes
a transaction to pay payroll of around $3,200.
[2:43] And one of her employees needed for
rent, the other needs it for child care.
[2:49] It's not just a transaction that Maya made, it's
a promise to her employees that the money will
[2:54] reach on time when they need it. And at the
same time, there is a $75 $75,000 transaction
[3:01] happening in the bank, which got flagged.
Now, the problem for the analyst, it has to
[3:10] analyze all these transactions. And
if he's too quick to approve them,
[3:16] a fraudulent transaction might miss out. And if
he takes a lot of time, people like Maya suffer.
[3:24] Now, it's not the problem that we have to ask the
agent whether it's a fraud or not. It's providing
[3:33] the AI agent with evidence, the auditable trails,
the right policies with consistency, the rules
[3:39] that are hardcoded every time, and making it
get the relationships between the transactions
[3:46] and having the human review it when the AI is not
able to make the decision with a high confidence.
[3:54] Now, how does AI come into picture and how is AI
going to support these financial transactions?
[4:03] Why do we need AI for this one? Because
evidence is scattered across different systems.
[4:10] The false positive cases could
provide a higher impact to a larger uh
[4:20] audience. And ambiguity must be escalated
when it's not sure on how to act clearly.
[4:31] And every decision must must have an audit
trail so that the decision could be used for
[4:35] reinforcement learning and it can be traced when
an auditor is looking into it after 6 months.
[4:42] Now, how did we get from these searches
to the agent state that we are here today?
[4:46] Initially, we just had simple searches.
You can look into your data. The uh
[4:51] the analyst could look into your data sets.
They all the transactions and would have to find
[4:57] uh meaning themselves to figure out whether
these transactions are fraudulent or not. Then,
[5:01] we got the rack. The rack system helped the uh
transaction analyst to analyze with some context.
[5:08] And now we have the AI agents which have the
capability to extract your data using tools,
[5:14] retrieve them, uh show the results,
and have decisions logged properly.
[5:21] Now, we have the Marshall app, a fraud detection
demo that we built on Mastra and [ __ ]
[5:27] Every transaction that gets flagged
is in the queue for a human review.
[5:32] MongoDB helps you retrieve the relevant
transaction based on the full-text searches,
[5:36] the hybrid search that gets you the similar
transactions uh based on the vector similarities,
[5:42] and also combines the full-text search results
for finding exact accounts, exact amounts,
[5:47] and exact transactions that has
happened similarly to the current one.
[5:52] Mastra gives you the brain for the AI agent.
It helps you orchestrate the complete workflow,
[5:59] and when a review is required, the human
in the loop gets the notification for it.
[6:06] The governance and the policies are the rule
sets that we provide are stored into MongoDB,
[6:11] and the agents can retrieve it and take action
according to the rules and policies, and provide
[6:16] a govern governance-ready auditable trail that
is tamper-proof. So, what we get from this one?
[6:24] We get speed, and we get audit audit uh traces,
and we get accuracy over time. And you can
[6:33] justify your decisions even after 6 years or so.
And you can see all this in our live dashboard.
[6:41] All right. Now, we're going to dive in
and actually take a look at Marshall. So,
[6:45] we're going to leave that QR code up for a
moment if you want to actually follow along
[6:49] and play with Marshall yourself, but then I'm
going to dive in into a few specific cases.
[6:55] So, I'll leave it up for a few more moments. I
see a lot of excited people here. Are you trying
[6:58] it out? So, I'll give you all a chance. There
will also be a QR code again at the end. So,
[7:02] don't feel like if you missed it now, you won't be
able to get it. All right. So, let's run through
[7:07] a few example transactions that Marshall tackles.
First, let's start with a simple, clean approve.
[7:14] Here, we have a use case where it's a recurring
payroll payment from a known vendor. So,
[7:19] what Marshall is going to do is going to look for
previous transactions, recognize that this is a
[7:24] common transaction, no red flags at all, can
just approve and move forward. Pretty simple.
[7:32] Now, on the other side, let's look at
something that's a strong reject. In this case,
[7:37] this is a payment to an unknown overseas
merchant that immediately follows a specific
[7:44] a suspicious login attempt from the payee. So,
here we'll see that Marshall is going to go in,
[7:51] figure out all this information, and flag this
as a very clear fraudulent transaction attempt.
[7:58] So, those are two sort of extreme ends of
the spectrum. Let's take a look at something
[8:01] that's a bit more in the middle. Here, we
have a transaction between two vendors,
[8:07] but the vendor paying is known to be a mule.
Which what a mule does is it sits in the middle
[8:13] of two different entities and helps control their
payment. So, if A wants to control wants to pay C,
[8:20] but knows that it will be flagged as fraud,
they'll use B as a payment in the middle to
[8:24] get around that. So, what we'll see here,
and I'll start the video from the front,
[8:32] what we'll see here is this is the B to C
payment. It's going to then search to see
[8:36] if it can find any payments from A to C. It found
them and recognizes this as an ABC fraud attempt.
[8:48] The next one we have is the amount structuring.
[8:55] Consider you're only allowed to uh transact
around $5,000, and above that you need an SSN
[9:02] or a PAN card, usually in India. Someone making
such transaction can just lower the amount just
[9:08] below the threshold and do multiple transactions.
Those transactions could be traced, dragged, and
[9:13] we can have it reviewed, and the LLM could himself
reject those if needed based on the score. And the
[9:19] agents do that as well. The next one. Now, this is
the $75,000 transaction that we are talking about.
[9:28] For this one, the agent is going to retrieve
the relevant transactions, reason over it,
[9:35] and give it for the human review with reasoning
chain whether it should be approved or not and
[9:41] it depends on the human in the loop to approve it,
store it or reject it. And these go into the audit
[9:47] trail if you see on the right hand side and these
will also be used as a reinforcement learning for
[9:53] the future transaction that could come up and
these decisions are in an immutable audit trail.
[10:00] Now, how MongoDB and Maestro are working together
here. MongoDB provides the agent its memory
[10:09] and the evidence trail. Maestro serves as the
agent's brain and provides the control plane.
[10:18] MongoDB provides you with the operational data,
[10:22] the semantic retrieval layer and the
relationship between these transactions.
[10:32] MongoDB is giving you the vector search
capabilities for semantic recall, the dollar
[10:37] search for the full text search retrievals
to find exact accounts and audit trails.
[10:43] The rank fusion combines both of them to give
you a better result sets of best of both worlds.
[10:49] And the graph look up helps you to find
[10:52] uh the hops among accounts and how the money
is going and to track around the trails.
[11:00] MongoDB lets you store every AI decision in an
audible trail for every action that AI agent takes
[11:09] and which is a tamper-proof
[11:11] evidence that could be referred whenever the
auditor needs it and could be reasoned around it.
[11:19] And then on the Maestro end, Maestro
is the whole agentic control plane. So,
[11:24] there's a few different pillars to this. First,
all the agent logic where we're using the LLM to
[11:29] actually discern all the different evidence
we're getting and draw conclusions from it.
[11:34] Also, the workflow orchestration, which was this
deterministic routing we saw in each step that
[11:39] Marshall took when it was analyzing transactions.
And finally, the observability, which helps you
[11:45] get a look into all your previous runs and
get an idea of what your agent has been doing.
[11:51] I know we've all been talking a lot about agents
this week and observability I'm going to touch
[11:54] on a little bit later. So, I'm going to dig
into workflow orchestration a little bit.
[12:01] So, just at a high level, what is a workflow? It's
a deterministic series of steps that you can put
[12:06] into your agentic system to give it determinist
determinism in a naturally non-deterministic
[12:13] infrastructure. So, this is basically just
a high-level 101 of some of the things you
[12:18] can do with workflows in Mostra. You can have
sequential steps where you just go A to B to C.
[12:24] You can have parallel steps, multiple workflows
running at the same time. You can branch into
[12:29] different categories and that's kind of like what
we saw with Marshall where sometimes it would
[12:34] approve a transaction, sometimes it would hold
it, and those would have different outcomes. So,
[12:39] that's a branch step where we go down different
paths depending on the information we receive.
[12:44] And also, loops, which we saw in Marshall when
it would do the retrieve step multiple times to
[12:49] gather more information. And you can build so much
more out from there, but that's just a high-level
[12:55] starting point with workflows. And another thing I
want to highlight with workflows is that they save
[13:00] snapshots, which is basically all the information
and metadata you have at every step of a workflow.
[13:07] This opens the door to be able to do things like
suspend resume functionality, human in the loop,
[13:13] workflow retries, and also time travel or
replaying your workflows from previous steps.
[13:19] So, we saw a lot in previous examples
how Marshall would pause a workflow,
[13:24] wait for human input, and then continue. And
that's possible thanks to Monstrous Workflows.
[13:30] And the last thing I want to highlight here is
we also have durable workflows, which means you
[13:34] can have your workflows distributed across servers
to make them more resilient to different crashes,
[13:41] making sure that they can run not just for
minutes or hours, but days or weeks at a time.
[13:47] You can also integrate into your
infrastructure. We're building out
[13:50] the ability to build out these workflows, these
durable workflows on our no-code agent builder.
[13:56] And when you integrate that with our durable
agents, you make a really robust agentic system.
[14:02] Which is the perfect lead-in to our next
topic, which is getting a demo to production
[14:07] and how you do this reliably with MongoDB and
Monstrous. So, what this demo has showed so far
[14:13] is it solves a real problem and it can run
in this controlled environment. But how do
[14:17] we make sure that it's something that we can
run at scale in production, that we can do
[14:23] not just for minutes or hours, but for days,
and also have the observability around it to
[14:28] make sure we're improving it consistently over
time and tracking what's going on under the hood?
[14:35] And to chat a little bit about how MongoDB and
Monstrous come together to help you get there,
[14:40] to start with Monstrous, as I mentioned
before, durable workflows is one ingredient
[14:45] to that recipe, but I want to touch
a bit more on observability here.
[14:49] Since Monstrous is both the framework layer
and the observability layer, that gives you
[14:54] access to traces that are actually at agentic
runtime. So, rather than your observability just
[14:59] seeing whatever the agent exposes to it, every
single tool call, every workflow, every line of
[15:05] code that your agent is running is something
Monstrous observability can have insight on.
[15:10] You also can set up your evals through Monstrous,
see metrics like your token usage and cost,
[15:17] not just at an agent level, but
at a model level, at a at a level,
[15:21] at a workflow level. So, you get a very
granular look at what's going on under the hood.
[15:26] And we also have a ton of guardrails that you can
build out or use our out-of-the-box solutions.
[15:34] And one thing new to Maestra that I want
to highlight here is trace intelligence,
[15:37] which we just announced last week. And basically,
it categories categorizes your traces into four
[15:43] areas: goal, what was the intent of your trace,
behavior, what actions did it perform, outcome,
[15:50] what was the result, and sentiment, what were
the vibes associated with that agent run.
[15:55] This helps you start to identify patterns where
maybe you notice certain behaviors lead to
[16:00] certain outcomes, and then can start adjusting
your agentic code to handle these bottlenecks.
[16:06] And this is also the first step in Maestra's
agentic learning loop, which is a focus of ours
[16:11] right now, which is creating an agent layer
in between your evals and your source code
[16:16] to consistently automatically iterate on your
agent and have this self-improvement loop.
[16:30] MongoDB is helping you prevent the infra sprawl.
Instead of setting four different databases
[16:36] for your system of records, for your graph
lookups, for your vector searches, and for
[16:41] your search indexes, you can all consolidate them
in a single system. It gives you the complete data
[16:48] foundation layer that your developers will love
and which will be easier to scale and maintain.
[16:57] So, what's the takeaway from this?
[17:02] The hard part isn't getting a demo created
for an agent, it's the demo that's get built
[17:10] and turned into a go-on system with the audible
traces, the evidences, and accountability. MongoDB
[17:18] and Mastra gives you a credible path for creating
your demos into production systems. You can scan
[17:25] the QR code to try out the Mastra app that that is
running the Marshall. And we also have the second
[17:34] QR code as the GitHub repo with the documentation
and the code that you can deploy it on your infra.
[17:44] Thank you all for being here today. If you
have any questions, you can reach out to us.
[17:49] We will be available and you can also
contact us at mongodbpartners@mongodb.com.
[17:55] And you can also fill out the form
that Marshall provides you. So,
[18:00] Yep, and I want to quickly mention we'll both be
around after this talk to chat with you guys if
[18:04] you want to go through anything. And Mastra's
CEO, Sam, actually wrote a book that covers a
[18:09] ton of this stuff with AI agent development. And
I have my coworker, Ashwin, over there who's ready
[18:14] to give you free copy. So, if you're interested
in learning about more stuff like this, feel
[18:18] free to go see Ashwin and get a free copy of the
book. Thank you, everyone. Thank you. [applause]
