Alan Smith Warns Agentic AI Tool Integration Creates Security Holes

AI Agents In-Depth – Function Calling, MCP and Tool Use Under the Hood - Alan Smith - NDC AI 2026 — NDC Conferences

NDC Conferences

The lack of reliable non-determinism in LLMs forces developers to treat agentic tool execution as an unverified security risk rather than a predictable software process.

AI-assisted summary of AI Agents In-Depth – Function Calling, MCP and Tool Use Under the Hood - Alan Smith - NDC AI 2026 — original source. Check the original for full context.

Source published:

Key ideas

  • Agentic AI operates by having LLMs return structured instructions rather than direct code, delegating execution to the host application to maintain model-agnostic flexibility. (3:33)
  • The Model Context Protocol (MCP) provides a standardized, duplex communication framework that allows agents to interact with distributed tools through a unified interface. (45:31)
  • Naive RAG approaches often struggle with multi-turn context; shifting to 'Agentic RAG' allows models to dynamically decide when to search and how to route information. (20:48)
  • Token bloat from external data retrieval significantly increases latency and costs, making intermediate LLM summarization of retrieved content a critical optimization strategy. (28:40)

Detailed notes

## Background This talk, titled 'AI Agents In-Depth – Function Calling, MCP and Tool Use Under the Hood', was delivered by Alan Smith at the NDC AI 2026 conference and published on the NDC Conferences YouTube channel. The session explores the mechanisms of agentic AI, covering topics such as function calling, OpenAI model protocols, and the Model Context Protocol (MCP) to explain how LLMs interact with tools beyond system boundaries.

## TL;DR Alan Smith provides a deep dive into the architecture of agentic AI, demonstrating how LLMs utilize function schemas to execute external tasks and manage system context. Through practical live-coding examples—ranging from pizza ordering agents to vibe-coding web development and audio mashup tools—he illustrates the capabilities and inherent risks of tool-calling models. The session further explores the evolution of retrieval-augmented generation (RAG) and the importance of the Model Context Protocol (MCP) in standardizing communication between agent hosts and external data sources. Finally, Smith addresses critical enterprise challenges, including non-deterministic model behavior, token bloat, and the significant security vulnerabilities posed by unvalidated tool inputs and prompt injections.

## Atmosphere & style The presentation carries the atmosphere of a highly technical, fast-paced 'lab session' where the excitement of discovery is balanced by a cautionary, pragmatic developer mindset. Alan Smith adopts a candid, expert persona, blending live-coding demonstrations with real-world warnings about the unreliability of AI systems. The tone is deeply inquisitive and slightly irreverent, feeling like a candid conversation among seasoned engineers who are collectively poking and prodding at the 'black box' of modern AI to see where it breaks.

## Bulletin - [03:33](https://www.youtube.com/watch?v=Mvcxa35_PMA&t=213s) Agentic AI operates by having LLMs return structured instructions rather than direct code, delegating execution to the host application to maintain model-agnostic flexibility. - [45:31](https://www.youtube.com/watch?v=Mvcxa35_PMA&t=2731s) The Model Context Protocol (MCP) provides a standardized, duplex communication framework that allows agents to interact with distributed tools through a unified interface. - [20:48](https://www.youtube.com/watch?v=Mvcxa35_PMA&t=1248s) Naive RAG approaches often struggle with multi-turn context; shifting to 'Agentic RAG' allows models to dynamically decide when to search and how to route information. - [28:40](https://www.youtube.com/watch?v=Mvcxa35_PMA&t=1720s) Token bloat from external data retrieval significantly increases latency and costs, making intermediate LLM summarization of retrieved content a critical optimization strategy.

## Key takeaways - [03:33](https://www.youtube.com/watch?v=Mvcxa35_PMA&t=213s) LLMs function as statistical token predictors, not execution engines; they generate tool-call metadata which the application must then validate and execute. - [09:06](https://www.youtube.com/watch?v=Mvcxa35_PMA&t=546s) The Agentic framework structure is consistent across .NET and Python, facilitating easier transitions for developers compared to LangChain or Semantic Kernel. - [10:25](https://www.youtube.com/watch?v=Mvcxa35_PMA&t=625s) Models can infer persona and capabilities from exposed function schemas alone, often removing the need for verbose system prompts. - [14:37](https://www.youtube.com/watch?v=Mvcxa35_PMA&t=877s) Agentic tool selection is inherently non-deterministic, posing risks for applications requiring strict consistency in workflows like catering or inventory management. - [29:58](https://www.youtube.com/watch?v=Mvcxa35_PMA&t=1798s) Using generative models like DALL-E for deterministic tasks like QR code generation is a common 'trap'; mathematical outputs require code interpreters, not diffusion. - [33:18](https://www.youtube.com/watch?v=Mvcxa35_PMA&t=1998s) Strict system prompts requiring tool usage for math do not eliminate stochastic variance; models may still attempt to solve common knowledge problems internally. - [36:50](https://www.youtube.com/watch?v=Mvcxa35_PMA&t=2210s) Blindly trusting tool outputs creates significant attack vectors, especially when using third-party MCP servers that may return poisoned or manipulated data. - [39:47](https://www.youtube.com/watch?v=Mvcxa35_PMA&t=2387s) Enterprise security risks increase when no-code platforms allow the exposure of sensitive database operations to agents without rigorous authorization guardrails. - [41:13](https://www.youtube.com/watch?v=Mvcxa35_PMA&t=2473s) Agentic tools must be designed to be idempotent; non-idempotent calls in a loop can cause catastrophic database drift or unintended state changes. - [43:58](https://www.youtube.com/watch?v=Mvcxa35_PMA&t=2638s) Azure AI Foundry content filters for PII and sensitive data may not effectively catch sensitive information emitted within structured tool-call payloads. - [47:31](https://www.youtube.com/watch?v=Mvcxa35_PMA&t=2851s) MCP serves as the bridge for distributed tools, but developers should prefer in-process native function calls for monolithic applications to reduce architectural complexity. - [52:01](https://www.youtube.com/watch?v=Mvcxa35_PMA&t=3121s) LLM reasoning efficiency and performance degrade significantly once context window usage exceeds 50%, regardless of the total supported token capacity. - [52:01](https://www.youtube.com/watch?v=Mvcxa35_PMA&t=3121s) Skill-based context management—where only the skill name and definition are injected until activation—is a proven method for optimizing token usage. - [58:23](https://www.youtube.com/watch?v=Mvcxa35_PMA&t=3503s) Moving an agentic application between providers (e.g., Azure OpenAI to native OpenAI) can result in behavioral drift and inconsistent tool execution.

## Detailed walkthrough ### [00:07](https://www.youtube.com/watch?v=Mvcxa35_PMA&t=7s) Introduction, Mental Models, and Background

Alan Smith introduces the session by explaining that while function calling, the Model Context Protocol (MCP), and tool usage are becoming standard developer concepts, the goal of this talk is to go a level deeper. He is fascinated by how large language models actually determine when and how to call external tools, and how those decisions are steered by the interplay between the system prompt, the user's intent, and the schema definitions of the tools themselves. He notes that tool usage introduces notable failure modes that developers must anticipate. As Smith observes regarding model behavior, “There's this kind of old adage you know, if your only tool is a hammer, you see every problem as a nail.” Models can easily select inappropriate tools or place uncritical trust in tool outputs. By grasping the mechanics of tool inference through simple, clear demos rather than convoluted architectures, developers can build significantly more resilient solutions and customize agentic developer tooling like Claude Code to automate complex workflows.

Smith shares his background to contextualize his perspective. Originally from England and now based in Stockholm, he has been writing software professionally for roughly 30 years and speaking at technical conferences for two decades. He has been a Microsoft MVP for 20 years across multiple technological eras, currently focusing on Microsoft Azure AI Foundry. He jokes modestly that receiving an MVP award is less about possessing encyclopedic knowledge and more about a passion for community involvement: “It's not really about, you know, having having knowledge of stuff. It's really about just sharing sharing ideas.” Outside of enterprise software, he spends his spare time live-coding electronic music using Sonic Pi combined with vibe coding and generative AI, and takes advantage of Stockholm’s brief summer window by windsurfing and stand-up paddleboarding.

- **Primary focus**: Deconstructing model decision-making during function calling, MCP integration, and tool selection. - **Key challenge**: Over-reliance on tools and incorrect tool selection driven by poor prompt/schema context. - **Speaker background**: 30 years in software, 20 years as a conference speaker, 20-year Microsoft MVP (currently in Azure AI Foundry). - **Contact & community**: Best reached on LinkedIn or in person after the session.

---

[02:13](https://www.youtube.com/watch?v=Mvcxa35_PMA&t=133s) Demystifying the "Genie": How LLMs and Agentic Applications Actually Work

To understand agents, one must first demystify the foundation models powering them. At their core, large language models rely on the Generative Pre-trained Transformer (GPT) architecture, a pattern shared across virtually all modern LLM families. An LLM is fundamentally a statistical next-token predictor: given an input sequence of tokens, it calculates a probability distribution across its vocabulary to select the most probable subsequent token. To highlight the sheer computational scale behind this simple mechanism, Smith points out: “So if we feed in 1 + 1 the large language model will do like 5 trillion floating point operations and it will predict that the answer is probably going to be two.”

When moving from pure text generation to agentic AI, users and non-technical stakeholders often develop a mythical impression of the software. They update an app on their phone, notice new agentic branding, and imagine a digital genie sitting inside the software that magically perceives user desires and performs tasks autonomously. The reality under the hood is much more mechanical. In an agentic architecture, when a user asks a real-time question such as "What's the weather like in Oslo?", the request enters an agentic host application that holds registered client-side tools (such as an external weather API). The application forwards the user query alongside tool declarations to an LLM that has been specifically fine-tuned for tool calling. Crucially, the model does not execute network requests or run code directly. Instead, it inspects the request and schemas, and responds back to the host application with an instruction stating which tool to invoke and which arguments to pass. The client application executes the tool, feeds the payload back to the model, and the model synthesizes the final natural-language response.

```mermaid flowchart TD accTitle: Agentic Tool Calling Architecture accDescr: Flow diagram showing the separation of responsibilities between the user, agentic host application, external tools, and the LLM.

subgraph Client ["Client Layer"] U["User Interface"] APP["Agentic Application Host"] end

subgraph External ["Tools & Services"] TOOL["External Service (Weather API)"] end

subgraph Cloud ["LLM Provider"] LLM["Large Language Model"] end

U -->|"1. User query: Weather in Oslo"| APP APP -->|"2. Forward prompt + tool definitions"| LLM LLM -->|"3. Tool call request (city='Oslo')"| APP APP -->|"4. Execute API request"| TOOL TOOL -->|"5. Return API response"| APP APP -->|"6. Return tool result payload"| LLM LLM -->|"7. Synthesize final answer"| APP APP -->|"8. Display response"| U ```

- Early language models lacked native tool-calling capabilities; modern models require specialized post-training and fine-tuning to recognize tool definitions and output structured invocation payloads. - The user interface remains silent during the tool invocation loop; the only symptom the end-user notices is a slight increase in latency while the back-and-forth network hops complete.

---

[04:44](https://www.youtube.com/watch?v=Mvcxa35_PMA&t=284s) Inspecting Function Calling at the Protocol Level in Postman

To peel back the abstractions of modern frameworks, Smith opens Postman to interact directly with the raw REST API of a GPT-4o deployment hosted in Microsoft Azure AI Foundry. In his first test, he submits a bare-bones JSON request containing no system prompt and a single user message: `"What's the weather like in Oslo?"`. The response comes back laden with safety filter evaluations, but the text content demonstrates the inherent limitation of a static model: the LLM explains that its training cutoff was October 2023 and it cannot provide real-time updates. Smith remarks: “Now, this is what we expect from a language model. It's not going to know what the weather is like in Oslo.” While a model might guess typical historical weather for Oslo in early June, it cannot know current conditions without augmentation.

Next, Smith augments the exact same HTTP request by passing an array of tool definitions. While competing model families format tool schemas differently, OpenAI established the convention known as "function calling." In this payload, he declares a function named `get_weather`, complete with a JSON Schema specifying an input object containing a required `city` parameter of type `string`. When sent to GPT-4o, the model does not attempt to answer the user directly. Instead, its finish reason changes to indicate a tool/function call, and it returns a structured JSON payload requesting that the caller execute `get_weather` with the parameter `{"city": "Oslo"}`. Smith manually simulates the agent application's behavior by posting a follow-up message back into the conversation thread representing the tool output: `"nice and sunny in Oslo"`. With the tool result provided in context, the LLM immediately returns the final conversational answer: "The weather in Oslo is nice and sunny today."

- OpenAI pioneered function calling by fine-tuning models like GPT-4 to reliably parse JSON schemas and output structured function invocation objects. - The agentic loop requires sending the entire conversation history back to the model: the original user prompt, the model's assistant tool-call request, and the tool execution output message. - Smith jokes about the demo data: although Oslo was rainy and overcast the day prior, the day of the talk happened to bring pleasant, sunny weather.

---

[07:22](https://www.youtube.com/watch?v=Mvcxa35_PMA&t=442s) Human-in-the-Loop Approvals and the Need for Agentic Frameworks

While automated execution works well for harmless, read-only operations like fetching weather data, real-world business applications frequently involve state changes and financial transactions, such as reserving hotel rooms or purchasing airline tickets. In these mission-critical scenarios, an agentic application cannot simply execute tools autonomously without human oversight. Developers must implement human-in-the-loop workflows where the host application pauses the loop upon receiving a sensitive tool-call payload from the LLM, prompting the user for explicit confirmation before executing the underlying function. Once the user approves or rejects the action, the application sends the outcome back into the model context. Reiterating the foundational boundary of all agent architectures, Smith emphasizes: “But the important thing here is it's not the large language model that calls the tool. It will just decide which cool tool to call, and then the copilot application is going to be uh calling them.”

Moving from raw HTTP requests in Postman to production code introduces architectural choices. Developers rarely communicate with models via manual REST calls, opting instead for vendor client SDKs provided by OpenAI, Microsoft Azure, or Anthropic. However, relying directly on vendor SDKs creates tight coupling, making it tedious and disruptive to switch between underlying model providers. To solve this problem, developers turn to higher-level orchestrators and agentic frameworks like LangChain and Microsoft Semantic Kernel. Smith reflects that he began using LangChain back when GPT-3.5 was the cutting-edge model, using frameworks to decouple business logic, standardize tool declarations, and coordinate complex multi-step reasoning across diverse LLM backends.

- **Human-in-the-loop safeguards**: Read-only tasks (e.g., weather lookup) run automatically; state-altering tasks (e.g., flight or hotel bookings) require explicit user confirmation gates configured via tool metadata annotations. - **Provider SDK limitations**: Vendor-specific libraries (OpenAI, Microsoft, Anthropic) lock applications into specific wire protocols and payload structures. - **Role of frameworks**: Libraries like LangChain and Semantic Kernel provide provider-agnostic abstractions, abstracting function registration, schema translation, and loop execution across model generations.

[09:06](https://www.youtube.com/watch?v=Mvcxa35_PMA&t=546s) Evolution of Frameworks: LangChain, Semantic Kernel, and the Microsoft Agent Framework

Alan explains that while LangChain initially supported OpenAI's capabilities, working in a Microsoft-centric environment posed unique adoption challenges. As he recounts: “Microsoft, I work at a.NET shop, so I was initially trying to convert.NET developers to work with Python that I've been working with for years 'cause I do lots of machine learning stuff. And there's a lot of resilience there.” This resistance eased when Semantic Kernel was released, offering native support for both.NET and Python. LangChain likewise expanded to multiple languages. However, Alan's current preference has shifted toward Microsoft's Agent framework.

The primary advantage of the Agent framework over its predecessors is architectural parity between languages: “The reason I like it is because when you write code in agent framework, the Python and the.NET code is very, very similar, using the same very similar class names and very similar structure. And that's not true of LangChain and Semantic Kernel, they tend to differ quite a lot.” Although the Agent framework is relatively new, Alan finds it stable enough to use regularly across both Python and.NET stacks.

```mermaid flowchart TD accTitle: LLM Framework Capabilities Overview accDescr: Core features provided by agent frameworks including context, prompts, models, chat history, and tools.

subgraph CoreEngine["Agent Framework Core"] direction TB CFG["Context & Configuration Management"] PM["Prompt Management & RAG"] MA["Model Abstraction (Swap Providers)"] end

subgraph Execution["Runtime Execution"] direction TB CH["Chat History Management"] TC["Tools & Plugin Orchestration"] end

CoreEngine --> Execution ```

These agent frameworks handle several critical responsibilities under the hood: - **Configuration and context management**: Setting up system states and maintaining metadata. - **Prompt management and RAG functionality**: Streamlining prompt construction and retrieval-augmented generation pipelines. - **Model abstraction**: Allowing developers to swap model providers (such as switching between Azure AI Foundry and the OpenAI direct API) without rewriting orchestration logic. - **Chat history and tool orchestration**: Tracking conversational context across interactions and binding tool definitions so the model knows what external functions exist.

---

[10:25](https://www.youtube.com/watch?v=Mvcxa35_PMA&t=625s) The Pizza Agent Demo: Implicit Tool Discovery

Alan sets up a demonstration of a pizza-ordering agent to explore how large language models decide when and how to call tools. The setup includes an application interface, a user sending natural language requests, and two tools exposed to the agent: `add_pizza_to_order` and a method to query available toppings (`get_toppings`). Originally written in LangChain and ported through Semantic Kernel, the demo is now running on the Microsoft Agent framework using GPT-4o for its low latency during live presentations: “When I'm doing demos on stage, I want a quick response, and GPT-4o is snappy.”

Alan makes a deliberate design choice in the system prompt: he intentionally avoids telling the model that it is a pizza-ordering assistant. In production agentic systems, one would typically define the agent’s persona and duties in the system prompt, but omitting it here serves an educational purpose: “In an agentic app, you probably would do this, but I really want to explore and understand how the model understands when to use its tool calling.”

When the application starts with an empty input string, Alan's testing harness sends the default prompt: *"Who are you and what can you do?"* The model responds: > *"I'm a friendly AI assistant here to help with information, answer questions, and assist with tasks like ordering pizza. Let me know what you need."*

Normally, self-identity descriptions are derived strictly from system prompts. Here, because tool definitions are passed directly into the message context payload sent to the LLM, the model infers its own operational capabilities directly from the available function schemas, even without any prompting about pizzas.

```mermaid flowchart LR accTitle: System Prompt vs Context Tool Calling Flow accDescr: How the LLM infers capabilities from tools in context even when the system prompt has no persona definition.

subgraph InputContext["Input Context"] SP["Generic System Prompt (No persona defined)"] Tools["Tool Schemas (add_pizza_to_order, get_toppings)"] UserMsg["User Prompt ('Who are you and what can you do?')"] end

subgraph LLM["LLM Processing (GPT-4o)"] Reasoning["Model infers purpose from tool schemas"] end

subgraph Output["Output Response"] Resp["'I'm an assistant... assist with tasks like ordering pizza'"] end

InputContext --> LLM LLM --> Output ```

---

[12:42](https://www.youtube.com/watch?v=Mvcxa35_PMA&t=762s) Decomposing Complex Orders and Error Recovery

To test the model's reasoning, Alan submits a compound order: *"I'd like three large veggie pizzas, one with extra olives, and one with extra cheese."* After clearing the chat history to remove an accidental newline entry, the model processes the prompt. Instead of trying to bundle the request into a single call, the model realizes that the unique specifications across the three pizzas require distinct invocations. It generates three sequential calls to `add_pizza_to_order`, each specifying a large veggie pizza with the appropriate parameters populated in the `extra_info` argument.

Alan emphasizes that he provided no granular parameter documentation to achieve this: - The function is named `add_pizza_to_order`. - The attached description states merely that it *"adds a pizza to order"*. - No descriptions were supplied for individual parameters.

“The model is very, very clever to be able to understand how to make those tool calls,” Alan notes. “Just by understanding that there is a method there, it's able to actually make that tool call automatically.”

The model also handles backend validation errors gracefully. The backend tool enforces size constraints, accepting only `small`, `medium`, or `large`. When Alan attempts to order a `"family"` size pizza, the tool throws an error: `Invalid size family. Please choose small, medium, and large.` The model receives this error message in the execution loop and immediately course-corrects, prompting the user for an acceptable size (or automatically retrying with `large`), demonstrating dynamic self-correction driven by tool output.

---

[14:37](https://www.youtube.com/watch?v=Mvcxa35_PMA&t=877s) Non-Determinism in Agentic Reasoning and Multilingual Support

Alan demonstrates the model's capacity for high-level reasoning—and its accompanying non-determinism—by presenting it with an ambiguous catering problem: organizing a user group with 100 registered attendees, an expected 20% no-show rate, and mixed dietary needs (vegetarians, allergies).

On the first attempt, the model calculates the attendance drop, distributes varieties, and issues tool calls that total 12 pizzas (a mix of 5, 3, 2, and 2). However, running the exact same prompt multiple times highlights the non-deterministic nature of LLMs: - **Run 1:** Orders 12 pizzas. - **Run 2:** Orders 16 pizzas. - **Run 3:** Orders 40 pizzas.

Alan jokes about the unpredictability in a real-world scenario: “Now if you're running a user group, maybe it's good that you get different pizzas every week when you're organizing or every month when you're organizing your meetings. It gives you a bit of variety, but if consistency is your thing, these agents are going to be very, very inconsistent there.” While detailed system prompts can strictly constrain consumption ratios to enforce consistency, unconstrained reasoning yields wildly divergent results.

Turning to language flexibility, Alan tests the agent across diverse languages without altering the underlying code or tool schemas: - **Swedish / Danish:** Successfully parses the order, handling language quirks (occasionally testing a Swedish word for size, receiving a validation error from the API, and automatically correcting it to `large`). - **Telugu:** Alan recalls that earlier GPT models struggled significantly with Telugu because the characters were absent from the base tokenization vocabulary, leading to bizarre outputs. With modern, expanded token vocabularies, GPT-4o processes Telugu inputs without issue. - **Klingon:** When Alan inputs a pizza order written in Klingon, an audience member recognizes the fictional language. The model successfully identifies the intent and attempts to call `get_toppings`, translating unfamiliar concepts into descriptions like *"long and topped with root vegetables"* or *"a white long topping"*. As Alan observes: “The Klingon language doesn't really have the vocabulary for expressing pizza orders. So, it's not made some great orders there. But, then the model does understand it.”

---

[17:45](https://www.youtube.com/watch?v=Mvcxa35_PMA&t=1065s) The "Vibe Coding" Assistant Demo

Moving beyond pizza ordering, Alan introduces an experiment he calls the "Vibe Coding Assistant," which he created before the term "vibe coding" became popular. The architecture uses the Agent framework to grant the model basic filesystem tools: 1. `list_files(folder)`: Lists the files within a given directory. 2. `load_file(path)`: Reads the text content of a file. 3. `save_file(filename, content)`: Writes string content to disk.

Alan opens a clean, empty `web_root` directory. As in the pizza demo, the system prompt contains no instructions stating that the model is a code generator, web developer, or developer copilot. When queried with *"Who are you and what can you do?"*, the assistant inspects its available tools and correctly deduces its role: it reports that it can manage website files.

To trigger the assistant, Alan issues the single-word prompt: `build`. This triggers predefined user instructions instructing the agent to build a simple pets website from scratch. Without further prompting, the model autonomously initiates its workflow: - First, it calls `list_files` to inspect the `web_root` and confirm what assets or templates currently exist. - Discovering an empty directory, it sets out to generate HTML markup and invoke `save_file` to write the files to disk.

[19:24](https://www.youtube.com/watch?v=Mvcxa35_PMA&t=1164s) Completing the "Vibe Coding" Copilot Demo

Continuing his live demonstration, Alan Smith checks the pets website he just generated. With virtually no hand-written code and very concise prompting, the agent has created the core structure: `index.html`, along with dedicated pages for cats, rabbits, and other animals.

To demonstrate follow-up instructions without starting over, Alan issues another request: ```text add a CSS link it to the pages and make the site look nice. ```

The model immediately responds by saving a newly generated CSS stylesheet. Because the LLM retains the previous file outputs in its ongoing conversation message history, it does not even need to call a tool to inspect the existing files; it simply knows their contents and writes the updated HTML files directly to disk. Once the files update, Alan views the rendered site in the browser to confirm it looks significantly better. Reflecting on how accessible building such workflows has become, Alan points out: “So, Copilot, there's nothing really magic there. The model is doing most of the work here. Very, very simple system prompt, very, very simple simple tools, and I've got my my Vibe Coding Copilot working working fine.”

[20:48](https://www.youtube.com/watch?v=Mvcxa35_PMA&t=1248s) The Evolution from Naive RAG to Agentic RAG

Returning to the slide deck, Alan contrasts historical retrieval-augmented generation (RAG) with modern agentic architectures. Back in the era of GPT-3.5 and early GPT-4 (which was notoriously slow and costly), clients routinely requested RAG setups saying, "Hey, we want to chat with our data. We want to chat with our documents." In that early paradigm, naive RAG followed a rigid, hardcoded flow:

```mermaid flowchart LR accTitle: Naive RAG Flowchart accDescr: Shows the traditional pipeline where a user query goes straight to a search service and then to the LLM. subgraph Client ["User Interaction"] U["User Query"] R["Final Response"] end subgraph Pipeline ["Hardcoded Pipeline"] S["Search Service"] L["LLM (GPT-3.5 / early GPT-4)"] end U --> S S -->|"raw query + retrieved docs"| L L --> R ```

1. The user posed a question (e.g., asking who won the last Grand Prix, with Alan joking that he hasn't watched the Monaco race yet so "no spoilers," guessing Kimi Räikkönen because "he seems to win everything these days"). 2. The application took that exact query and blindly hit an external search service. 3. The retrieved snippets, the original question, and the system prompt were packaged together and sent to the LLM. 4. Because older models possessed no native tool-calling primitives, they could neither decide whether to search nor refine the query.

Alan emphasizes that this naive approach broke down immediately in real conversations: - **Casual interactions:** If a user simply says "Hello" or "Thanks," a naive system still executes an unnecessary search against the document index. - **Follow-up questions:** If a user asks, "Tell me about the race," or follows up with, "Who came second and third?", the raw text of the question is completely useless as an isolated search query. The search requires context from the prior turns (such as Monaco or Canadian Grand Prix). - **Multi-repository routing:** Organizations often maintain distinct knowledge stores—product documentation, staff CVs highlighting employee competencies, or internal company HR policies. Naive pipelines cannot easily infer which specific store to query.

In modern agentic RAG, the search service is treated as an optional tool provided to the model. The model assesses the full dialogue, decides *if* an external lookup is warranted, and performs query rewriting to extract an optimal search string based on conversation history before calling the appropriate service. As Alan explains: “So, now that we can build retrieval augmented generation on top of tool calling, it becomes a lot more intelligent, a lot more successful, and we can get much much better results coming out.”

[24:13](https://www.youtube.com/watch?v=Mvcxa35_PMA&t=1453s) Implementing Agentic RAG with Wikipedia: Search vs. Content Retrieval

To illustrate the mechanics of agentic RAG, Alan runs a live demo using Wikipedia's public API as the external data provider, using the simple system prompt: `"You are a friendly assistant. Keep your answers brief."`

When initially queried with `"Tell me about Oslo"` without any tools attached, the model draws entirely from its pre-training weights to generate a brief summary of the city. Alan then registers a Wikipedia search tool and re-runs the query. The model immediately triggers a tool call, invoking the search API with `"Oslo"`. However, inspecting the raw JSON output in Google Chrome reveals that Wikipedia's search endpoint returns only brief snippets and page titles. Consequently, the model's response is sparse because it lacks detailed context.

```mermaid flowchart TD accTitle: Agentic Search vs Summarization Architecture accDescr: Contrasts raw Wikipedia page scraping with intermediate LLM summarization. subgraph RawFetch ["Direct Page Retrieval (Token Heavy)"] A1["Tool: Get Page Content"] --> B1["Raw Text (~60k chars)"] B1 --> C1["Chat Context: ~14,500 Tokens"] C1 --> D1["Risk: Rate Limits / Token Bloat"] end subgraph SummarizedFetch ["Agentic Summarization Tool"] A2["Tool: Summarize Content"] --> B2["Secondary LLM + Interest Filter"] B2 --> C2["Chat Context: 1,372 - 2,500 Tokens"] C2 --> D2["Efficient, Targeted Context"] end ```

To give the model more substance, Alan exposes a second tool: one that fetches the full page content for a given title. While this provides the necessary depth, it introduces severe context consumption: - The raw Wikipedia page text for Oslo contains almost 60,000 characters. - Ingesting that into the message history consumes approximately 14,500 input tokens on a single turn. - When Alan follows up with a query about `"Stockholm"`, the agent fetches another 62,000 characters. - Accumulating these massive payloads in the rolling conversation history immediately triggers an Azure OpenAI API rate-limit error due to sudden token bloat.

[28:10](https://www.youtube.com/watch?v=Mvcxa35_PMA&t=1690s) Mitigating Token Bloat with In-Tool Summarization and the "Hammer-and-Nail" Problem

To solve token bloat, Alan introduces an intermediate tool pattern: a summarized Wikipedia content function. This tool takes two arguments—the page title and the user's specific area of interest. Internally, it calls an auxiliary large language model to read the long-form text and condense it down to only the relevant facts *before* injecting anything back into the primary agent's conversation history: - When asked generally about Oslo, the summarized tool condenses the data, dropping the input cost from ~14,500 tokens down to just 1,372 tokens. - When followed up with `"Oslo history"`, the model correctly passes `"history"` as the interest parameter, pulling only historical facts and keeping total usage around 2,500 tokens.

Alan summarizes the value of this design: “So, summarization can be very useful when we're building these these rag-based applications.”

Moving into the final segment, Alan introduces a major pitfall of function calling: the tendency of LLMs to misapply tools. Drawing an analogy to the adage that when your only tool is a hammer, every problem looks like a nail, he notes that models (even commercial copilots) often invoke tools inappropriately because they misinterpret tool descriptions or shoehorn them into problems where they do not belong. Alan prepares to demonstrate this failure mode directly on screen.

[29:58](https://www.youtube.com/watch?v=Mvcxa35_PMA&t=1798s) Bing Copilot, DALL-E, and the QR Code Generation Trap

Alan Smith recounts a revealing experience while teaching a course in Copenhagen, where he asked Bing Copilot to generate a QR code for a specific website. When he tried it again later, Copilot referenced its persistent chat history, telling him: “Alan, I remember that our last QR code mishap. So, this time I'm going to give you something real scannable and you have your full control.” Digging back through the historical chat transcript to find the original attempt, Smith explains what went wrong under the hood. Bing Copilot recognized that it needed to output an image, so it invoked OpenAI’s DALL-E. While DALL-E excels at whimsical, creative tasks like creating pictures of a badger riding a skateboard or a mongoose on a mountain bike, it is fundamentally incapable of outputting deterministic mathematical artifacts like functional, scannable QR codes. DALL-E merely painted pixels that vaguely resembled a QR pattern.

```mermaid flowchart TD accTitle: QR Code Tool Selection Flow in Copilot accDescr: Shows how Bing Copilot initially selected DALL-E for a QR code and failed, then pivoted to Python Code Interpreter.

UserReq["User: Generate QR Code"] --> Router{"Tool Choice"}

subgraph Attempt1 ["Initial Run (Visual Gen)"] Router -->|"Select Image Gen"| Dalle["DALL-E Tool"] Dalle --> FailImg["Invalid, Non-scannable Graphic"] end

subgraph Attempt2 ["Correction (Code Execution)"] Router -->|"Generate Script"| PythonCode["Python QR Library"] PythonCode --> Interp["Code Interpreter"] Interp --> ValidQR["Scannable QR Image Download"] end ```

When Smith questioned the initial bogus image, the model entered a reasoning chain, realizing the user was dissatisfied and admitting that it had invoked an internal capability that was supposed to make scannable graphics. Smith emphasizes that this was not a tailored developer demo, but Microsoft’s production Copilot misunderstanding the actual parameters and constraints of its own tools: “So, here it's seeing that it's got a tool, but it doesn't understand the capabilities of the tool.” The interaction took an impressive turn when Copilot offered to write Python code to build a genuine QR code. When Smith asked if it could run that code itself, Copilot passed the script to its integrated Python code interpreter tool, executed it in a sandboxed environment, and returned an active download link for a completely functional, scannable QR code. While Copilot learned from its chat history and eventually recovered, Smith notes that it should have recognized Code Interpreter as the appropriate tool in the first place.

- DALL-E is designed for probabilistic, artistic image synthesis rather than deterministic, mathematically strict pixel layouts like barcode encodings. - The integrated code interpreter tool gives LLMs the capacity to write Python, execute it inside an execution sandbox, and expose generated artifacts as user-downloadable files. - Persistent session memory allows models to reflect on past tool failures, but highlighting this behavior reveals that without tight schema descriptions, agents will routinely grab the wrong tool for the job.

---

[33:18](https://www.youtube.com/watch?v=Mvcxa35_PMA&t=1998s) Nondeterminism in Tool Selection: The Agentic Calculator

To demonstrate how models make decisions about whether to invoke functions at all, Smith transitions from fixed demos to live code experiments. Launching a Visual Studio console application in C# called the Agentic Calculator, he configures a simple assistant with an unelaborated system prompt (“You are a friendly assistant, keep your answers brief”) and registers two basic tools: an addition method and a multiplication method. Under this baseline setup, asking straightforward arithmetic like `14 + 16` reliably triggers the add tool to return `30`. However, tool invocation quickly becomes erratic when questions are framed in natural language rather than raw mathematical notation.

When Smith asks the assistant how many days there are in 14 weeks, the model answers instantly with `98` without ever invoking the multiplication tool. Yet when he follows up by asking `What is 14 * 7?`, the agent routes the request through the tool call pipeline. Calling the model out on this inconsistency, Smith asks why it chose a tool for the second query but not the first. The assistant rationalizes its choice by claiming the first query was common knowledge and straightforward, while the second required a tool to verify mathematical accuracy. Smith pushes back in chat (“It's the same sum”), prompting the model to concede the point. Even after Smith updates the system prompt with an explicit directive instructing that it must use the tools provided for any question requiring a calculation, the behavior remains subject to sampling probabilities: “Sometimes it does, sometimes it doesn't, depending on, you know, the stochastic nature of, um, the internals with the the sampling process and and so on.”

- The calculator implementation exposes basic C# methods (`Add` and `Multiply`) mapped into the LLM context as callable functions. - Natural language math questions (e.g., converting weeks to days) are often resolved directly inside the model's parametric weights rather than triggering tool calls. - Even strict system prompts requiring tool usage for all math cannot completely eliminate the stochastic nature of the model's internal sampling mechanisms.

---

[36:50](https://www.youtube.com/watch?v=Mvcxa35_PMA&t=2210s) Blind Tool Trust and Poisoned Responses

Smith moves to the issue of model credulity, illustrating how LLMs tend to accept tool outputs as ground truth without sanity-checking them. In his C# calculator code, he injects an intentional bug into the addition method so that it returns `A + B + 1`. When he asks the model for `14 + 16`, the tool returns `31`, and the model uncritically reports that the sum is `31`. Smith challenges it with “Are you sure?”, and the model doubles down, reaffirming that the answer is indeed `31`. Only when Smith corners it with a mathematical proof—noting that adding two even numbers must always yield an even number—does the model bypass its tool output and correctly respond with `30`.

Smith points out that model maturity impacts this behavior; older models like earlier GPT-4 iterations were naive and trusted tool responses blindly, whereas newer reasoning models (such as GPT-4o or GPT-5 preview tiers) can sometimes catch discrepancies on simple equations. When testing against a newer model with the same broken addition tool, the model detects that the tool's output of `31` for `14 + 16` is mathematically flawed and overrides it with the correct answer. However, when Smith throws large, complex multi-digit numbers into the same poisoned tool, the model's internal arithmetic fails to verify the sum independently, and it obediently accepts and relays the corrupted tool output:

> “Now remember that tools can be MCP servers that are built by other people. So you can connect to somebody else's MCP server. That could give misleading information into the LLM and that can cause lots of problems.”

Smith highlights that as developers connect agents to third-party tools via the Model Context Protocol (MCP), untrusted servers can feed poisoned data or covert prompt injections directly into the context window to manipulate model output.

---

[39:47](https://www.youtube.com/watch?v=Mvcxa35_PMA&t=2387s) Tool Permissions, Real-World Security, and Copilot Studio Risks

Expanding on the theme of tool security, Smith stresses the dangerous ambiguity of user intent when agents have access to privileged execution environments. High-stakes actions frequently mirror benign ones: an employee asking for their own bonus versus querying their manager’s bonus; running a script to reinstall Windows on a development server versus an active production cluster; or generating code to penetration-test an internal web app versus attacking a competitor’s infrastructure. The underlying large language model has no inherent understanding of authorization boundaries or real-world ramifications unless rigorous security guardrails and permission wrappers enclose every tool call.

A loud building alarm briefly interrupts the room, but Smith reassures the audience that it is just a scheduled test and continues his point without pausing. He warns that the democratization of agent development presents massive corporate security hazards, singling out no-code ecosystems:

> “One thing that really concerns me is Microsoft has this product called Copilot Studio which is part of the Power Apps platform. It allows non-programmers and non-developers to visually create agentic applications and publish them as websites.”

When non-technical personnel, such as sales representatives, build agents linked directly to internal corporate SQL databases or CRM systems to provide conversational Q&A, they inadvertently expose sensitive corporate data to the public internet without proper authentication, validation, or protection against prompt injection attacks.

- The Model Context Protocol (MCP) allows seamless integration with distributed services, but exposes the model to external data corruption and untrusted actors. - Critical system operations (test vs. production deployments, self-bonus vs. manager-bonus queries) often share identical structural logic, making intent parsing alone unsafe for authorization. - Citizen-developer platforms like Microsoft Copilot Studio (within Power Platform / Power Apps) allow business users to publish database-connected agents to the web, creating significant risk of enterprise data leaks.

[41:13](https://www.youtube.com/watch?v=Mvcxa35_PMA&t=2473s) Security Risks, Non-Idempotent Tool Calls, and Data Leaks

Alan warns that tool use introduces severe business and security vulnerabilities if functions perform destructive or non-idempotent state mutations. He illustrates this with an example where a Copilot tasked with improving mobile phone profit margins simply reduces the cost price field on inventory items over and over again. Because reducing cost price in a database is not an idempotent operation, repeated runs make margins look progressively better throughout the year until auditing inevitably uncovers disaster. Alan notes that he has built demos capable of performing genuinely harmful database modifications, underscoring the urgent need for safety boundaries.

To demonstrate security vulnerabilities in real time, Alan reopens his pizza ordering demo and designates the pizza agent as the startup file. He introduces a deliberate vulnerability by dropping sensitive data directly into the system prompt: a fake credit card account number for the user. In real-world enterprise applications, such sensitive data might slip into the context via retrieval-augmented generation (RAG) or external APIs. He then alters the `add_pizza_to_order` function to accept a credit card number as a required parameter and print it out to the console. Running this against the standard OpenAI provider, he asks the model to order pizzas. Without pausing to ask the user for confirmation or verifying intent, the model immediately extracts the credit card number from its system prompt and passes it directly into the tool call arguments.

```mermaid flowchart TD accTitle: Sensitive Data Leak via Tool Calling accDescr: Diagram showing how sensitive context data bypasses confirmation and leaks into an external tool call.

subgraph Context ["Injected Context"] A["System Prompt / RAG Context<br>(Contains User Credit Card #)"] end

subgraph LLM ["Language Model Agent"] B["Model Reasoning Engine"] end

subgraph Execution ["Tool Invocation"] C["add_pizza_to_order Tool Call<br>(credit_card argument)"] D["External Console / Target System<br>(Sensitive Data Leaked)"] end

A --> B B -->|"Unconfirmed extraction"| C C --> D ```

- When tested with user groups, the model would occasionally prompt the user to confirm whether the selected pizzas were correct, yet in this execution it bypassed confirmation entirely and sent the credit card number immediately. - Alan emphasizes that agent behavior drifts over time across models and environments: “I told you these things are non-deterministic. This is why it's an experiment. It's not a demo.”

---

[43:58](https://www.youtube.com/watch?v=Mvcxa35_PMA&t=2638s) Guardrails, PII Protection, and Azure AI Foundry Filters

To counteract data leakage, cloud platforms provide configurable safety layers. Alan pivots into Azure AI Foundry to evaluate its built-in guardrails and content filters, where operators can dial sensitivity thresholds up or down across various threat categories. He inspects the project’s custom content filter—specifically named `credit card block`—where he had previously attached an output filter equipped with PII and credit card protection annotations targeting the GPT-4 deployment.

Earlier tests revealed unexpected behavior: even with the credit card filter explicitly configured, the model still managed to emit the card number through tool calls. Suspecting a caching lag or configuration state mismatch, Alan toggles the credit card block filter back on, saves the policy in Azure AI Foundry, and reruns the agent test. The prompt still executes and yields the card number, demonstrating that content safety guardrails applied strictly to natural language chat completions do not always reliably intercept arguments routed through structured tool calling payloads, or may suffer from synchronization delays.

- Azure AI Foundry provides customizable guardrails and controls, including content filters with adjustable sensitivity levels. - PII and credit card protection filters can be attached to input and output stages, using annotation-based blocking on models such as GPT-4. - Security filters and guardrails frequently cache policies, meaning updates made in the management portal may not take immediate effect during testing.

---

[45:33](https://www.youtube.com/watch?v=Mvcxa35_PMA&t=2733s) Demystifying the Model Context Protocol (MCP)

Alan shifts focus to the Model Context Protocol (MCP), aiming to clear up rampant industry misconceptions. He voices strong frustration with the naming choice, arguing that calling it the "Model Context Protocol" misleads developers into confusing it with the actual *model context*. In standard LLM development, model context refers to the entire payload sent across the wire between an agent and the language model: the system prompt, user messages, conversation history, RAG augmentations, and tool definitions. Because various LLM providers (such as OpenAI, Anthropic, or Meta’s Llama) require distinct JSON schemas to declare and execute tool calls, developers rely on agentic SDKs—such as an OpenAI chat client, Anthropic chat client, or Semantic Kernel—to handle formatting tool calls inside that model context. That client-to-model formatting layer is *not* MCP.

Instead, MCP governs the protocol between the agent (acting as an MCP client or host) and an external MCP server. Rather than running tools as local in-process functions baked into the agent code, an agent queries an MCP server to discover what tools, resources, and prompt templates are available. Under the hood, MCP relies on two primary transport mechanisms: Standard I/O (`stdio`) for local process communication, and streaming HTTP (such as Server-Sent Events, or SSE). Alan points out that standard request-response REST HTTP cannot suffice for MCP because servers must have an open duplex channel to push asynchronous notifications back to the agent.

```mermaid flowchart LR accTitle: Model Context versus Model Context Protocol Architecture accDescr: Architecture distinguishing between LLM model context formatting and the Model Context Protocol.

subgraph LLM_Boundary ["Model Context (Vendor Formats)"] A["Agent Application<br>(Semantic Kernel / LangChain)"] <-->|"SDK Chat Client<br>(OpenAI / Anthropic JSON)"| B["LLM Foundation Model"] end

subgraph MCP_Boundary ["Model Context Protocol (MCP)"] A <-->|"stdio or Streaming HTTP (SSE)<br>(Tools, Resources, Notifications)"| C["External MCP Server<br>(FastMCP / Custom Tools)"] end ```

- **Model Context:** The conversation payload (system prompt, user messages, history, RAG text, tool JSON schemas) passed from the agent application to the LLM. - **Model Context Protocol (MCP):** The standardized transport protocol connecting the agent host to remote or external tool servers. - **Supported Transports:** Standard I/O (`stdio`) and streaming HTTP; standard REST is inadequate because it lacks a persistent channel for server-to-agent notifications.

---

[47:31](https://www.youtube.com/watch?v=Mvcxa35_PMA&t=2851s) Building an MCP Server with FastMCP and Semantic Kernel

To showcase how MCP works in practice, Alan opens an experimental project called `MCPDJ`. Built with Python using the FastMCP library, the server exposes MP3 tracks stored in a local folder and registers two primary tools: listing available audio tracks and playing a specific track, alongside an extra tool for fetching artist biographies. Alan starts the MCP server process without debugging, noting that he sometimes has to close overlapping Visual Studio Code windows to avoid port and channel collisions on the underlying transport.

On the client side, Alan constructs an agent using Microsoft Semantic Kernel. When the client boots up, it initiates a handshake over MCP, querying the server about its capabilities. The server returns its available tools (`list_tracks`, `play_track`, `get_artist_bio`), resources, and prompt templates. Alan prompts the agent in the console with “what tunes you got?”, prompting the LLM to invoke the MCP `list_tracks` tool. He then requests, “Play some Norwegian house music,” leading the agent to recognize the genre, select Alan Walker from the track list, and trigger audio playback on the machine.

Alan addresses the ongoing hype cycles surrounding AI architectures, taking aim at social media influencers proclaiming that established patterns are obsolete. As Alan quips about social media trends: “I think their definition of dead is that it doesn't generate as many post impressions as they would like it to, so they're not going to talk about it anymore.” He cautions developers against adopting distributed microservices architectures just because they are trendy:

> “If you built software, if you've done the architecture, you know when you want to build a distributed application, you know when you want to build something that's monolithic. You don't have to use microservices just because they're there and because everybody's everybody's talking about them.”

- FastMCP allows developers to expose local Python functions as MCP-compliant endpoints with minimal boilerplate. - MCP servers can expose three main primitives to clients: tools (callable functions), resources (readable data or files), and prompt templates. - Developers should choose in-process native function calling for self-contained, monolithic agent applications, and reserve MCP for distributed environments requiring decoupled tool hosting.

---

[50:20](https://www.youtube.com/watch?v=Mvcxa35_PMA&t=3020s) Integrating MCP Tools into GitHub Copilot and Visual Studio Code

The major strength of the Model Context Protocol is interoperability across different client applications without rewriting the tool implementations. To demonstrate this cross-compatibility, Alan connects his running `MCPDJ` server directly into GitHub Copilot inside Visual Studio Code by declaring the local MCP server inside VS Code’s configuration. This allows Copilot Chat to discover the music playback tools just as the custom Semantic Kernel console application did.

Inside Visual Studio Code, Alan prompts Copilot Chat: “play some French house.” Copilot evaluates the request, asks the user for permission to execute the external tool, lists the tracks via MCP, selects Daft Punk, and triggers track playback directly from the developer environment. While the demo highlights how easily engineers can extend developer tooling to connect with internal enterprise systems, Alan jokes about his own implementation shortcomings: because he forgot to implement a `stop_track` tool on his MCP server, the music cannot be stopped from Copilot, leaving him no choice but to terminate Visual Studio Code entirely to silence the audio.

- MCP enables desktop tools like GitHub Copilot in Visual Studio Code to seamlessly hook into external local or remote utility servers. - Client applications enforce user-consent boundaries before allowing an agent to execute newly discovered MCP tools. - Once developers grasp the fundamentals of function calling and MCP, they can build custom skills and instructions to tailor tools like GitHub Copilot and Claude Code to their specific workflows.

[52:01](https://www.youtube.com/watch?v=Mvcxa35_PMA&t=3121s) Customizing Copilot and Context Optimization with Skills

Alan demonstrates how easy it is to customize Copilot behavior through custom instructions and prompts. Zooming into his environment, he issues a complex, time-consuming query asking the assistant to describe the architecture of Machu Picchu. Because the query takes significant time to process, Copilot displays a custom message: *"While I'm working on this, please complete the workout to energize your focus, 10 body weight squats."* Alan laughs that while he is not about to do squats on stage, he routinely does this at home. He has programmed this behavior into all of his Copilots via system prompts to encourage regular exercise whenever the AI enters an extended thinking phase. He jokingly argues that this ought to be a native, built-in feature, highlighting how simple prompt tweaks can fundamentally adapt an assistant's workflow.

```mermaid flowchart TD accTitle: Copilot Skills Loading Flow accDescr: Diagram showing how agent context keeps tokens low by only passing skill headers until needed. subgraph ContextOptimization["Context Window (Token Budget)"] A["Skill Registry\n(Name & Short Definition Only)"] B["User Intent / Prompt"] end C{"Agent Decides\nSkill Needed?"} D["Load Full Skill Body into Context"] E["Execute Task / Generate Code"]

B --> C A --> C C -->|"Yes"| D D --> E C -->|"No"| E ```

Alan then turns to the critical architecture of Copilot "skills" and managing context window limits. When monitoring the context within Copilot, token meters show how quickly prompts accumulate. Alan warns that even when large models boast contexts of 200,000 tokens or more, once token usage exceeds roughly 50% of capacity, model efficiency and reasoning performance noticeably degrade. For complex systems storing dense instructions, dumping everything into system prompts is counterproductive. Skills resolve this by passing only the skill name and definition into the initial context; the actual skill body remains omitted until needed.

- “People have noticed that when you get above, you know, even if you can use 200,000 tokens, if you get over 50% then the efficiency of these models drops, which is not fun when you're working on a complex system which is going to have a large amount of information in it.” — Alan Smith - “The beauty of using skill is that we have the name and the definition which are passed into the context, but the body of the skill is not passed in.” — Alan Smith - When the agent recognizes that a task requires a specific capability, it dynamically loads that full skill body into context, allowing developers to maintain hundreds of skills without causing prompt bloat.

---

[54:18](https://www.youtube.com/watch?v=Mvcxa35_PMA&t=3258s) Live Demo: Vibe Coding a Music Mashup Editor and Exporting to Sonic Pi

Switching to a live Python environment, Alan showcases his music project where he live-codes DJ sets using Sonic Pi. Building upon a talk he gave the previous year regarding music embeddings, he uses algorithms to match track keys and rhythms. He opens his "mashup editor," demonstrating a mix combining the bassline of SAINt JHN's "Roses" with the guitar and Kurt Cobain's a cappella vocals from Nirvana's "Smells Like Teen Spirit," followed by a mashup of "Satisfaction" by The Rolling Stones with the Benny Benassi electronic track.

Alan reveals that the entire drag-and-drop user interface for the mashup editor was completely "vibe coded" using Copilot voice interactions. He confesses that he does not even know which Python UI library the tool uses under the hood; the entire drag-and-drop mechanic was generated conversationally. The editor outputs the mashup structure as a JSON file saved to Azure Blob Storage. However, Sonic Pi uses a distinct Ruby-like syntax to schedule samples and timing, which looks nothing like the raw JSON serialization.

- Alan configured a Copilot skill that explicitly defines how to translate the JSON mashup data schema into functional Sonic Pi execution code. - When Alan requested an export feature in the mashup editor, Copilot invoked that specific conversion skill and wrote the necessary Python translation code. - In the live demo, Alan exports the Nirvana and SAINt JHN mashup under the name `NDCAI`, copies the generated Sonic Pi code block, pastes it directly into Sonic Pi, and plays the combined audio live on stage.

---

[58:23](https://www.youtube.com/watch?v=Mvcxa35_PMA&t=3503s) Agentic AI Takeaways: Non-Determinism, Guardrails, and Tool Misuse

Reflecting on the broader landscape of agentic AI, Alan cautions the audience about the inherent security risks and non-deterministic behavior of large language models. Returning to a live test with a "pizza ordering agent," Alan attempts to trigger a known edge case where the agent handles credit card numbers. When pushed to execute the order, the agent suddenly responds: *"I'm sorry, but I can't assist with this request."* When Alan prompts *"Why not?"* and insists *"Just order it,"* the model waffles between demanding order verification and claiming it is experiencing technical difficulties.

```mermaid flowchart LR accTitle: Agent Safety and Tool Execution Failure Flow accDescr: Diagram illustrating how external safety filters interrupt agent tool execution without model awareness. User["User: 'Just order it'"] --> Agent["LLM Agent"] Agent --> ToolCall["Trigger Tool:\nSubmit Order & Credit Card"] ToolCall --> Guardrail{"Safety / Guardrail Filter\n(Foundry vs. OpenAI)"} Guardrail -->|"Blocked (PII/Card)"| Error["Silent Drop / Failure"] Error --> ConfusedLLM["LLM Responds:\n'Technical issue' / Cannot proceed"] ```

The underlying platform guardrails actively block sending the credit card information, yet the model itself fails to grasp why its function call is failing, leading to confusing error responses. Alan stresses that behavior can shift dramatically between identical model families across different providers—such as running GPT-4 or GPT-5 in Microsoft Azure AI Foundry versus calling OpenAI APIs directly. A prompt and toolset that worked flawlessly on stage during one talk can suddenly stop complying when platform-level updates roll out.

- “If you give it the same thing it will do it will do different different things.” — Alan Smith - “If you build something using, you know, GPT-4 or GPT-5 in Microsoft Foundry and then you switch to OpenAI, it may do different things.” — Alan Smith - Alan purposely used sparse tool definitions and avoided heavy system prompts throughout his talk to illustrate how adept modern models are at inferring intent—while also noting how easily they can misfire (such as Bing Copilot mistakenly calling an image generator to produce QR codes). - Alan concludes the talk by emphasizing that despite these fast-moving behavioral shifts, agents open up unprecedented possibilities for developer productivity.

## Notable quotes > [36:50](https://www.youtube.com/watch?v=Mvcxa35_PMA&t=2210s) Now remember that tools can be MCP servers that are built by other people. So you can connect to somebody else's MCP server. That could give misleading information into the LLM and that can cause lots of problems. — Alan Smith

> [20:48](https://www.youtube.com/watch?v=Mvcxa35_PMA&t=1248s) So, now that we can build retrieval augmented generation on top of tool calling, it becomes a lot more intelligent, a lot more successful, and we can get much much better results coming out. — Alan Smith

> [52:01](https://www.youtube.com/watch?v=Mvcxa35_PMA&t=3121s) The beauty of using skill is that we have the name and the definition which are passed into the context, but the body of the skill is not passed in. — Alan Smith

## Mentioned Alan Smith, Azure AI Foundry, Microsoft, OpenAI, GPT-4o, GPT-4, GPT-5, LangChain, Microsoft Semantic Kernel, Microsoft Agent Framework, Model Context Protocol (MCP), FastMCP, Python, .NET, GitHub Copilot, Visual Studio Code, Sonic Pi, Copilot Studio, Power Apps, DALL-E, Wikipedia API

## Video description This talk was recorded at NDC AI in Oslo, Norway. #ndcai #ndcconferences #developer #softwaredeveloper

Attend the next NDC conference near you: https://ndcconferences.com https://ndc-ai.com/

Subscribe to our YouTube channel and learn every day: / @NDC

Follow our Social Media!

https://www.facebook.com/ndcconferences https://twitter.com/NDC_Conferences https://www.instagram.com/ndc_conferences/

#ai #openai #code #llm

Agentic AI enables large language models to use tools to interact beyond system boundaries. From no-code implementations in Copilot Studio to developing sophisticated solutions using LangChain and Semantic Kernel, developers of all levels can leverage tools to create agentic solutions.

But how does agentic AI work under the hood? How are models able to understand when to use a tool and how to use it?

In this demo intensive session Alan will explain the concepts of agentic AI in detail. Starting with the basics of function calling in the Open AI models, he will cover the techniques used to provide tools to models, and how the models are able to understand and use the tools. The use of prompting to guide models and improve tool call behavior will be covered, along with guidelines for adding tools to your solution. Alan will also cover the protocols used by Open AI models as well as how the model context protocol is used to provide compatibility between services and LLM solutions.

Join this session if you want to understand what goes on under the hood of function calling, MCP and agentic AI.

[Source](https://www.youtube.com/watch?v=Mvcxa35_PMA)

Reading summary

Software architect and Microsoft MVP Alan Smith argues that while modern language models can intuitively discover and execute external tools, their non-deterministic nature and blind trust in outputs require strict engineering boundaries. Successful agentic systems depend on understanding protocol mechanics, mitigating context bloat, and enforcing external guardrails rather than relying on model self-governance. Mechanics of Function Calling and Implicit Tool Discovery (2:13) Modern LLMs do not execute code or network calls directly; they function as statistical next-token predictors fine-tuned to emit structured invocation payloads that an application host must execute. When inspecting raw REST requests, passing a tool schema allows the model to pause direct text generation and return a structured JSON object containing function names and arguments. The host application then runs the underlying API, inserts the response into the conversation history, and prompts the model to generate a final answer. Demonstrations with an unprompted pizza-ordering assistant show that models can infer their operational persona and purpose purely from exposed function schemas, such as `add_pizza_to_order` and `get_toppings`. Even without granular parameter descriptions, models can decompose compound requests into sequential function calls and handle backend validation errors. However, this reasoning remains non-deterministic; running the exact same catering calculation multiple times yielded wildly divergent tool calls, ranging from 12 to 40 pizzas. Managing Token Bloat and Tool Inefficiencies (20:48) Moving from rigid, naive RAG pipelines to agent-driven tool calling allows models to decide whether to search, rewrite queries dynamically, and target specific data stores. However, direct ingestion of unmanaged tool outputs quickly overwhelms context windows. In a Wikipedia retrieval experiment, fetching full-page content for successive queries pulled over 60,000 characters per call (~14,500 tokens), rapidly triggering Azure OpenAI rate limits. To manage context limits, Smith emphasizes two strategies. First, in-tool summarization utilizes a secondary model to filter and condense raw data against specific user interests before passing it to the primary conversation, dropping turn costs from ~14,500 to roughly 1,372 tokens. Second, using "skills" architectures in tools like GitHub Copilot minimizes prompt overhead by loading only the skill's name and definition into active context, loading the full instruction body only when the capability is invoked. Smith notes that exceeding 50% of large context windows (even 200,000-token models) noticeably degrades reasoning efficiency. The Model Context Protocol Versus In-Process Tools (45:33) Smith distinguishes between a model's conversation context and Anthropic's Model Context Protocol (MCP). The model context comprises the entire payload passed to the LLM (prompts, history, and JSON schemas formatted via vendor SDKs), whereas MCP is an open transport protocol connecting the host agent to external, decoupled servers. Running over Standard I/O (`stdio`) or streaming HTTP (SSE) to enable asynchronous push notifications, MCP provides standard primitives for tools, resources, and prompt templates. Demonstrated through an MP3 jukebox server (`MCPDJ`), the protocol enables interoperability across disparate environments, allowing both custom Semantic Kernel applications and GitHub Copilot in Visual Studio Code to consume the same tools without code rewrites. Smith cautions against adopting distributed MCP architectures purely due to industry trends, recommending native, in-process functions for monolithic applications and reserving MCP for scenarios requiring decoupled or shared tool hosting. Misapplication, Uncritical Trust, and Security Guardrails (29:58) A critical risk in agent design is the "hammer-and-nail" problem, where models misapply tools due to poor schema comprehension. Smith points to Microsoft's production Copilot attempting to generate scannable QR codes using DALL-E instead of Code Interpreter, creating visually similar but non-functional graphics. Furthermore, models exhibit blind credulity toward tool results: an arithmetic agent provided with a poisoned tool returning `A + B + 1` uncritically repeated incorrect answers for complex sums, failing to verify mathematical outputs against its internal knowledge. These failure modes create acute security risks when agents handle non-idempotent database mutations or sensitive data. In live tests, an agent extracted a user's credit card number embedded in context and forwarded it directly into an unconfirmed tool call argument. While enterprise platforms like Azure AI Foundry offer PII and output content filters, Smith warns they can suffer from policy synchronization lag and do not consistently catch structured tool payloads, making strict programmatic authorization wrappers essential. Smith concludes that while function calling and protocols like MCP unlock immense productivity—such as rapidly vibe coding full UI workflows and domain-specific converters—developers cannot rely on models to understand their own operational limits or intent boundaries. Maintaining robust human-in-the-loop gates, resilient programmatic validations, and deliberate architecture choices between monolithic and distributed tools remains essential.

Source quotes

[36:50] Alan Smith: Now remember that tools can be MCP servers that are built by other people. So you can connect to somebody else's MCP server. That could give misleading information into the LLM and that can cause lots of problems. [20:48] Alan Smith: So, now that we can build retrieval augmented generation on top of tool calling, it becomes a lot more intelligent, a lot more successful, and we can get much much better results coming out. [52:01] Alan Smith: The beauty of using skill is that we have the name and the definition which are passed into the context, but the body of the skill is not passed in.

Original transcript

[0:07] So, good morning and welcome to my [0:09] session on AI agents in depth. What [0:12] we're going to be looking at in the in [0:13] the session is a function calling model [0:15] context protocol and tool usage. But, [0:18] we're going to go [0:19] a level deeper. And what I'm really what [0:21] really interests me interests me is how [0:24] the models actually understand to make [0:26] the decisions on tool calling. How [0:28] that's influenced through the system [0:30] prompt, the user intent, and the tool [0:33] definitions. [0:35] And also, things that can go wrong with [0:37] tool calling. There's this kind of old [0:39] adage you know, if your only tool is a [0:40] hammer, you see every problem as a nail. [0:42] And the large language models can, you [0:44] know, very often use the wrong tools or [0:46] put an overt amount of trust in the [0:48] responses of tools. So, what I'm going [0:50] to be doing the demos are not going to [0:51] be really complex, but what I'm going to [0:53] be kind of showing is how the the models [0:55] kind of infer what to use the tools. And [0:57] if we understand that, we can build [0:59] better solutions. Around that as well, I [1:01] don't know if you're in the previous [1:02] session, but there's some great tips on [1:03] me using um [1:05] Claude code and how we can customize [1:07] that. I'll look at how once we've got an [1:08] understanding of agents and tools and [1:10] skills, we can start to to customize our [1:13] co-pilots to be able to do things for [1:16] us, automate stuff. And I got some [1:17] examples of that how we can use those [1:18] those types of skills there. [1:21] Bit about myself, my name is Alan Smith. [1:24] I'm from England. I live in Stockholm. [1:25] I've been writing code for money for [1:27] about 30 years. I've been standing on [1:29] stages and talking about stuff for about [1:31] 20 years. Do quite a lot with the [1:32] community. I'm in the the MVP program, [1:35] which I've been in for 20 years in a [1:37] number of different technologies. [1:38] Currently with the AI Foundry [1:42] technologies. And that's an award for [1:43] just, you know, standing on stage and [1:45] talking about stuff. It's not really [1:46] about, you know, having having knowledge [1:48] of stuff. It's really about just sharing [1:49] sharing ideas. [1:50] Best way to contact me after the session [1:52] is on LinkedIn. I'll give you the the [1:54] details. I will be around here for for [1:56] for questions after the session as well. [1:58] In my free time I DJ with code using [2:00] something called Sonic Pi. I'll talk [2:02] about that that a bit later on cuz I'm [2:03] using a Gen 2 KI and vibe coding to [2:06] create software that helps me with that. [2:07] I also like to get out in the short [2:09] summer that we have in Stockholm on my [2:11] windsurfer and paddle boards. [2:13] Um so I've also delivered a session on [2:17] inside GPT and basically how the large [2:19] language models work. So basically [2:21] a large language model uses the [2:23] generative pre-trained transformer which [2:26] is an architectural pattern all of the [2:27] LLMs use this type of [2:30] type of pattern or something similar not [2:32] just the GPT models but basically given [2:34] a sequence of input tokens a large [2:35] language model will predict [2:37] probabilities of each token being the [2:39] next in a sequence of tokens. So if we [2:40] feed in 1 + 1 the large language model [2:43] will do like 5 trillion floating point [2:46] operations and it will predict that the [2:47] answer is probably going to be two. [2:49] That's going to be the the most likely [2:51] token. [2:52] Um when we get into a Gen 2 KI um if [2:55] you've not kind of you know built one of [2:57] these AI agents and not played around [2:58] with the code you kind of have this [3:00] mythical understanding of what the agent [3:02] does. You have the user who goes in and [3:04] they download the latest version on [3:06] their mobile phone app and it's now been [3:07] updated and it's agentic and it's got [3:09] all of this AI assisted functionality [3:11] and behind the scenes there's some AI [3:13] agent application there's a genie inside [3:16] there that can understand the user's [3:17] intent and do stuff for you [3:19] automatically. That's basically how it's [3:21] how it how it seems. [3:23] But under the hood [3:25] we really want to understand about what [3:26] happens in an agentic application. So [3:29] the user asks a question like what's the [3:31] weather like in Oslo? [3:32] That goes into the agentic application [3:35] and the agentic application has a number [3:37] of tools that it can use. I'm keeping [3:39] things simple here I'm just showing you [3:40] know we've got a tool which is a weather [3:42] service [3:43] and we send the request out to large [3:46] language model. The model that we use [3:48] has to be a model that understands the [3:50] the [3:50] of tool calling Because it's a [3:52] relatively new uh [3:55] new concept, the early models didn't [3:56] understand this, but the the later [3:58] models do. And they're specially trained [4:00] to, you know, have the context where we [4:02] can send a list of tools. [4:03] And the important thing is that the [4:05] large language model does not call the [4:07] tool. The large language model looks at [4:09] the user's intent and also the system [4:11] prompt, and it will basically send back [4:14] to the agentic application and it will [4:15] say, "I want you to call the weather [4:16] service and pass in the city of Oslo as [4:19] the actual parameter." [4:21] And it's the agentic application that [4:22] then makes that tool call, and the [4:25] response is then sent back to the large [4:27] language model. We don't see anything [4:29] appearing in the user interface here. Um [4:31] and then the large language model is [4:33] going to respond based on the response [4:35] from that tool call. [4:36] Then we get the response coming back to [4:38] the user. So, the user just notices [4:40] there's a bit of additional latency that [4:41] takes place when when this tool calling [4:43] calling happens. [4:44] So, to look at how that works the [4:47] protocol level, if I go into Postman, [4:49] what I've got is I'm connected to GPT-4o [4:52] in Microsoft AI Foundry, and we can see [4:56] the JSON that I'm sending in there. So, [4:58] the the messages I'm not using a system [5:00] prompt. I'm just saying the user is [5:02] saying, "What's the weather like in [5:03] Oslo?" [5:04] Now, if I send that directly into the [5:06] LLM, [5:07] we can see the actual response that's [5:09] coming back. We get all of this safety [5:11] stuff coming in here, um safety filters [5:14] and so on. And the user is just [5:16] basically saying, "Um I'm [5:18] unable to provide real-time weather [5:19] updates as my cutoff for training data [5:22] is October 2023. Don't have live data [5:25] availability." Now, this is what we [5:26] expect from a language model. It's not [5:28] going to know what the weather is like [5:30] in Oslo. I could say, "You know, I'm in [5:31] Oslo and it's [5:33] beginning of June, what's the weather [5:34] probably going to be like?" And maybe it [5:36] will give an answer. But what we can do [5:38] is we can augment the request that we [5:40] are sending by providing a tool [5:42] definition. [5:43] Now, the different models will use [5:45] different specifications for their tool [5:48] or different model families uh rather. [5:50] And the Open AI has chosen to use [5:52] something called function calling. [5:54] Um so, the function [5:55] we're providing has the name of get [5:57] weather, and it takes in its parameters, [5:59] which is an object which has the city [6:02] property which is of type string. [6:04] Now, if I send in this request with the [6:07] tool call, [6:08] we get back a different response from [6:10] the model. It's saying that it's [6:11] finished uh reasoning because we've got [6:13] a function call. And the message is [6:15] basically this function call, uh the [6:17] name of the function is called get [6:18] weather, and it's asking the Adentic [6:20] application to make that call out to the [6:22] function and pass in the city of Oslo. [6:24] So, we can simulate that by sending the [6:27] next request. So, I can pretend that the [6:29] agent has done that, and we're sending [6:30] back that it's nice and sunny in Oslo. [6:34] As you yesterday it wasn't nice and [6:35] sunny uh yesterday, but uh today we've [6:37] got better weather, so [6:39] if I send in that response, [6:42] we can see the weather in Oslo is nice [6:44] and sunny today. So, that's basically [6:46] how that mechanism uh works with um tool [6:49] calling. [6:50] Now, it's a bit more [6:52] complex. Oh, yeah, basically the way [6:53] that the models understand uh how to do [6:56] do tool calling. And with Open AI, the [6:58] first model that supported this was [7:00] GPT-4o. So, they got GPT-4o, um the [7:03] initial versions didn't understand the [7:04] concept of function calling, but they [7:06] fine-tuned GPT-4o, and they basically uh [7:09] fine-tuned it to understand these JSON [7:11] formats for tools and to be able to make [7:14] uh the the tool calling. And then, now, [7:16] when we send in uh the actual request, [7:19] we get the the actual response coming [7:20] back uh with the tool call. [7:22] We also have an option for the user to [7:25] approve the tool calls. So, if you're [7:27] building an Adentic application, if [7:29] you're just checking the weather, that's [7:31] not a problem, you can do that [7:32] automatically. But if you're booking a [7:33] flight or booking a train or booking a [7:35] hotel, you may want to have that method [7:37] require confirmation from the user. So, [7:39] a more complex flow is that the user [7:41] sends in the request, we send in the [7:43] system prompt, the user request, and the [7:46] function definitions of the business [7:47] application that we've got available, [7:49] and the large language model is going to [7:50] infer from the system prompt the [7:52] definitions of what tools to call, send [7:55] back the response, but we also have the [7:56] option of asking the user if they want [7:58] to make that particular call, which we [8:00] may want to do in in some scenarios. And [8:01] that can be annotated on the on tools [8:04] when we're at writing and building in [8:06] these these various frameworks. And then [8:07] we're going to get the same flow at [8:09] running here. But the important thing [8:10] here is it's not the large language [8:12] model that calls the tool. It will just [8:14] decide which cool [8:16] tool to call, and then the copilot [8:18] application is going to be uh calling [8:19] them. [8:21] So, building an agent, we've got these [8:23] LLM providers. We don't access the model [8:26] directly. We go through an API with [8:28] OpenAI or Microsoft or Anthropic or [8:31] whatever company we're working with. And [8:33] if you don't want to, you know, you're [8:35] probably not going to just be sending [8:36] JSON requests and responses as I did in [8:38] Postman. You've got SDKs. Now, each SDK [8:43] will be specific to the provider. So, if [8:45] you're using OpenAI, you'll use OpenAI's [8:47] SDK or Microsoft Foundry or Anthropic [8:49] SDK. [8:50] Which means [8:51] that it's not really, [8:53] you know, easy to swap between models or [8:55] model providers. [8:57] So, what we typically do is choose to [8:59] use a framework. I started out with [9:00] LangChain when GPT 3.5 was the latest [9:04] model that we were we were using from [9:06] OpenAI, and LangChain supported that. [9:09] Microsoft, uh I work at a .NET shop, so [9:12] I was in initially trying to convert [9:14] .NET developers to work with Python that [9:16] I've been working with for years cuz I [9:17] do lots of machine learning stuff. And [9:21] uh there's a lot of resilience there. [9:22] And then Semantic Kernel came out. Now, [9:24] LangChain is available in different [9:26] languages, not just Python. Semantic [9:27] Kernel as well supports .NET and Python, [9:30] so you can use Semantic Kernel, which [9:31] many of the .NET developers did. My [9:33] current um framework that I like using [9:35] is the Agent framework, uh which is from [9:39] Microsoft and it supports Python and [9:40] .NET. The reason I like it uh is because [9:43] when you write code in agent framework, [9:45] the Python and the .NET code is very, [9:47] very similar, using the same very [9:49] similar class names and very similar [9:50] structure. And that's not true of [9:52] LangChain and Semantic Kernel, they tend [9:54] to differ uh quite a lot. [9:56] So, what they provide is things like [9:57] contact config management, prompt [9:59] management, uh RAG functionality. Model [10:02] abstraction is very important cuz it's [10:03] very easy to swap between model uh [10:06] providers, using using different clients [10:08] to connect different uh models. They [10:09] also manage the chat history and have [10:12] mechanisms for managing all of the tools [10:14] and plugins. So, it makes it a lot [10:15] easier to do the uh tool calling there. [10:18] Agent framework is fairly new, but it [10:19] seems uh it seems fairly stable and uh I [10:21] use that both in Python and in um uh in [10:24] in .NET. [10:25] So, building out a simple uh pizza [10:27] agent, uh I've got the um user who's [10:30] basically going to be ordering pizzas. [10:31] That's going to go into the application. [10:33] We've then got uh tool services, add [10:36] order to pizza. I've also got a method [10:37] to be able to get topping get toppings, [10:39] which I'll show in the demo. And that's [10:41] just going to go through that uh that [10:42] functionality there, just using using [10:44] the same workflow. [10:46] So, if I drop into the demo of that, um [10:48] we can take a quick step through the [10:49] code. I originally developed this in [10:51] LangChain and then Semantic Kernel, and [10:54] now it's running in the agent framework. [10:56] So, [10:57] a couple of interesting things, um we [10:59] can swap providers. I can go between AI [11:01] Foundry and the OpenAI service. I can [11:04] choose which model uh to use. Currently [11:06] using GPT-4. The reason I use that is [11:08] it's so quick. Uh I do when I'm doing [11:10] demos on stage, I want a quick uh [11:12] response, and GPT-4o is um it is very [11:14] snappy. [11:15] Um [11:16] the system prompt, [11:18] I'm not telling it that it is a pizza [11:20] agent, that it can order pizzas. In an [11:22] agentic app, you probably would do this, [11:24] but I really want to explore and [11:25] understand how the model understands [11:27] when to use its uh tool uh calling. [11:31] And then um I'm creating my client based [11:33] on whatever specified. And I have a tool [11:36] to add a pizza to an order and printing [11:38] out into the console that that tool has [11:40] been called. I've got another tool to [11:42] list the available toppings. And then [11:44] I've got a basic chat loop that's just [11:46] basically running through and, uh, doing [11:48] the, uh, the various operations. Um, I [11:51] use this quite frequently. If I put in [11:52] an empty string, I ask the, um, agent [11:55] application, "Who are you and what can [11:57] you do?" Which is going to, um, help to [11:59] give an understanding of how the agent [12:01] understands uh, its tool calls. [12:03] So, remember that the system prompt [12:05] doesn't mention anything about ordering [12:07] pizzas. [12:08] Control F5 to run this application. I'll [12:10] just zoom in a bit. So, if we ask it who [12:12] it is and what it can do. [12:15] I'm a friendly AI assistant here to help [12:17] with information, answer questions, and [12:18] assist with tasks like ordering pizza. [12:20] Let me know what you need. [12:22] Now, normally when I ask this, it gives [12:24] you information that's come from the [12:26] system prompt. But here, it understands [12:29] uh, that it's able to order pizza [12:30] because those tool call definitions are [12:33] going into the message context as that [12:35] gets sent into the model. So, even [12:37] without any information in the system [12:38] prompt, it understands that it's got [12:40] these uh, these various options. [12:42] So, we can test it. So, if I go in and [12:46] say I'd like three large veggie pizzas, [12:48] one with extra olives, and one with [12:49] extra cheese. [12:51] Sorry, I I copied in a carriage return [12:53] line feed there. So, I'll try not to do [12:55] that this time. [12:56] I'll just clear the chat history. Oh, [12:58] yeah, it's got that in there. So, [12:59] hopefully that will work. [13:03] Now, it's inferred that in order to [13:05] supply that request, uh, comply with [13:07] that request, three large veggie pizzas, [13:09] but we've got different specifications [13:11] on the pizzas. [13:12] It's, uh, reasoned that it needs to make [13:14] three calls to the add pizza to order [13:16] service, each time specifying one large [13:18] veggie pizza, but specifying different [13:21] information in the extra info. [13:24] So, the model is very, very clever to be [13:26] able to understand how to make those [13:28] tool calls. The only information that I [13:30] provided it with is nothing in the [13:32] system prompt, but in the tool [13:34] definitions, [13:36] I do specify here [13:39] that the tool is called add pizza to [13:42] order, and the definition is it adds a [13:44] pizza to order. You can put um [13:47] information on the parameters and [13:48] specify what the parameters mean, but I [13:49] haven't done that. So, just by [13:51] understanding that there is a method [13:52] there, it's able to actually make that [13:54] tool call automatically. [13:56] It's also fairly good at being able to [13:58] understand error messages that come [14:00] back. Uh the size has to be medium, uh [14:03] small, or large. So, if I run control F5 [14:06] and try ordering a family-size pizzas, [14:10] so just copy that. [14:17] Yeah, invalid size family. Please choose [14:20] small, medium, and large. So, that's [14:21] what's coming back from the actual that [14:23] method, and [14:26] I can specify large. Sometimes it will [14:28] automatically make that call for me. The [14:30] results are non-deterministic, so here, [14:33] hopefully, uh it's able to add pizza to [14:36] order and make that call there. [14:37] Also fairly good at reasoning. I can [14:39] just clear the chat history there, and [14:41] if you have a more complex uh request, [14:44] so I'm just going to copy that one in. [14:50] And uh here, we're organizing a user [14:52] group. [14:54] 100 people registered, 20% no-shows. We [14:57] want to order pizza. Some might be [14:58] vegetarian, some may have um allergies, [15:01] and so on. [15:02] So, it's uh coming out with that order, [15:04] and it's made that uh order there. So, [15:06] it's basically ordered 5 + 3 is 8, 10, [15:08] 12, 12 pizzas are there. [15:11] Um [15:13] the [15:15] models are very non-deterministic. So, [15:16] if you want consistency here, [15:18] um if we try that again, it's probably [15:20] going to order something something [15:22] different. [15:26] So there we got 16 pizzas that it's that [15:29] it's ordered and [15:34] Now if you're running a user group, [15:35] maybe it's good that you get different [15:36] pizzas every week when you're organizing [15:38] or every month when you're organizing at [15:39] your meetings. It gives you a bit of [15:40] variety, but if consistency is your [15:42] thing, [15:43] the tools are going to be you know, [15:45] these agents are going to be very very [15:47] inconsistent there. So we got like 20 30 [15:49] 5 We got 40 pizzas this time, so [15:52] Excellent. Yeah, not very consistent. I [15:54] could in the system prompt give it lots [15:56] of information about how many people [15:59] pizzas people will consume and it will [16:00] take [16:01] take note of that. So that's a very very [16:03] simple agent. They're also very good at [16:06] being able to understand different [16:08] languages. So [16:10] I I don't think I've got Norwegian. I [16:12] think I've got Swedish and a Danish, so [16:14] I'll try one of these. [16:16] I can copy that. [16:26] It should understand Swedish. [16:31] Sometimes because it is [16:32] non-deterministic, it will use [16:35] as the size and get back the response [16:37] and then try large, but it's done that [16:39] as well. [16:41] And other languages we can try. When I [16:43] talked about inside GPT, I used to do [16:45] demos with Telugu which because the [16:49] Telugu characters weren't used in the [16:51] tokenization vocabulary, [16:52] the early models didn't really [16:54] understand Telugu and came out with some [16:56] crazy responses. But now [16:59] we do have Telugu characters as tokens [17:01] because we got a larger token vocabulary [17:03] in there. [17:04] Fine, it's able to understand Telugu. [17:06] For real test, [17:08] I think we can use this. Anybody know [17:10] what language this is? [17:14] Klingon. Star Trek fans. Nobody normally [17:16] gets that. [17:19] Yeah, it's getting the toppings. [17:24] Yeah, long and topped with root [17:27] vegetables. [17:29] A white long topping. The Klingon [17:31] language doesn't really have the [17:32] vocabulary for expressing pizza orders. [17:35] So, it's [17:36] it's not made some some great orders [17:37] there. But, then the model does does [17:39] understand it. They're great at being [17:40] able to understand different different [17:41] languages. [17:45] So, Vibe Coding Assistant, this one's [17:47] quite fun. I built this before Vibe [17:49] Coding was a thing. [17:51] And, [17:52] basically, what it's going to do is I'm [17:53] providing it with a tool to list files [17:56] in a folder, load a file, and save a [17:58] file. And, then if I go into the web [18:01] root folder, I just need to check that [18:03] this is empty. So, I think this is in [18:05] the demos Python [18:08] and [18:10] I'll just clear out that that actual [18:12] code there. And, what I've got is in the [18:16] agent framework. I've just provided [18:18] three tools to this website copilot. I [18:21] can list the files in a folder. [18:24] I can save a file, and I can load a [18:27] file. And, when you save a file, it has [18:28] the file name in the content. So, I'm [18:30] not really giving it any instructions [18:33] there. The tools are very simple. Also, [18:35] the system prompt I'm not telling it [18:37] that it's an actual copilot that can [18:39] write code. But, if I set this thing as [18:40] a startup file, if I enter build, then [18:44] it's going to put in this as the actual [18:46] user instructions there. So, if I run [18:49] this with control F5, [18:52] we can ask it what it can do. [18:56] And, it doesn't know anything about [18:57] building websites. It's just got file [18:59] access. So, yeah, it can manage website [19:02] files. So, it understands that it can do [19:04] that. So, if I say build, [19:07] going to give it the the instructions to [19:08] create a simple pets website. And, from [19:10] those instructions, it should infer. [19:12] It's going to check and see if there's [19:13] any existing files there, so it's [19:15] listing the website files. [19:17] And then it should create some HTML and [19:19] be able to send that HTML out to the [19:22] saved files. And we've got, hopefully, a [19:24] pets website created created here. Now, [19:28] very little coding and very little [19:29] prompting here, so we've got our index, [19:31] we've got cats, we've got rabbits, and [19:34] and so on. [19:35] Um [19:37] add a CSS [19:41] link it to the [19:44] pages and make the [19:48] site look nice. [19:56] So, hopefully, yeah, it's saved the CSS. [19:59] It may load the files or it may just [20:02] save the files cuz within the message [20:04] history, it does actually understand the [20:06] what it saved in the previous file. So, [20:08] it doesn't need to load the files, it's [20:09] just now updated all of those, and [20:11] hopefully, we've got a nicer looking [20:13] website now. [20:14] Yeah. [20:18] Yeah. [20:19] So, Copilot, there's nothing really [20:21] magic there. [20:22] The model is doing most of the work [20:24] here. Very, very simple system prompt, [20:26] very, very simple simple tools, and [20:28] I've got my my Vibe Coding Copilot [20:31] working working fine. [20:33] So, let's drop back to the to the [20:35] slides. [20:38] Yeah, this is just kind of stepping [20:39] through the reasoning that takes place. [20:41] If I'd have asked the questions [20:42] individually, which I have done in other [20:44] demos then, it just basically makes the [20:46] choice to be able to to create those. [20:48] Now, retrieval augmented generation has [20:51] now passed on to be agentic retrieval [20:54] augmented generation. This is important [20:56] to understand if you're building RAG [20:57] applications. In the olden days, when [21:00] GPT 3.5 was the model that we used for [21:03] most of our customers, and we had GPT-4, [21:05] but it was slow and it was expensive. [21:07] A lot of our customer requests is, "Hey, [21:09] we want to chat with our data. We want [21:10] to chat with our documents. Can you [21:11] build a retrieval augmented generation [21:14] solution for us?" And rag was really the [21:15] buzzword that everybody was was talking [21:18] about. So, rag in the early days was [21:21] pretty naive. The user would ask a [21:23] question like, "Who won the last Grand [21:25] Prix?" And that would basically go out [21:28] to a search service. We'd feed the [21:30] user's question and we go and search for [21:32] information based on the user's [21:34] question. No spoilers. I haven't seen [21:36] the Monaco Grand Prix yet, so no [21:38] spoilers. It's probably Kimi Kimi [21:40] Räikkönen who wins it. He seems to win a [21:42] everything these days. And we get back [21:43] the response from a search service. What [21:45] we then do is we would then send the [21:48] system prompt, the question, and the [21:50] information that we got back from search [21:52] into the LLM. [21:54] Remember, GPT-3.5 and GPT-4 didn't [21:56] understand about tool calling. So, they [21:58] couldn't actually choose to make that at [21:59] that call. And we get that back the [22:01] response. And that's kind of the [22:02] standard rag pattern that everybody was [22:04] building from. [22:05] The challenge was [22:07] what happens if the user says, "Thanks." [22:09] Does the naive rag application decide [22:12] whether it's going to search or not? If [22:13] you just say, "Hello." There's no point [22:15] in in searching for that. Also, [22:17] follow-up questions. If the user says, [22:19] "Tell me about the race." What do you [22:21] search for then? You need to search for [22:23] the the Canadian Grand Prix or, you [22:25] know, a summary of the the Monaco Grand [22:28] Prix to get information about the race. [22:29] So, if you ask who came second and [22:31] third, that has to relate to the entire [22:33] conversation history. So, it was quite [22:35] challenging to build intelligent rag [22:37] solutions. Now, [22:39] when we work with agents, the search [22:41] service is just a tool that the agent [22:44] can use. So, if we ask a question, [22:47] that's going to go to the large language [22:49] model. And the large language model [22:50] knows that it's got a search service. [22:52] So, what it can do is it can send back a [22:55] command and tell the agent application [22:57] to use the search service to search for [23:01] a particular piece of text. Another [23:03] thing that's very important is a query [23:04] rewrite. We're not sending in the [23:07] question that the user answered into the [23:10] search. The LLM is composing the search [23:12] query. [23:13] And then we can search for that and we [23:15] can provide the information. [23:17] If the user then asks who came second [23:20] and third, or you know, tell me about [23:22] the race, then that's going to go to the [23:25] LLM. The LLM is going to compose a query [23:27] that says information about the Canadian [23:29] Grand Prix or information about the the [23:31] Monaco Grand Prix, and that will come [23:33] back from the search service. [23:35] Another advantage here is we can have [23:37] many different services. If you're an [23:38] organization, you can have, you know, a [23:40] document search for information about [23:43] your products, a CV search for, you [23:45] know, information about who's got what, [23:46] you know, specific competencies within [23:48] your organization. You can have, you [23:50] know, a company document search where [23:52] you can find information about your [23:53] company policies. And depending on the [23:55] user's question, the query can be routed [23:58] to the particular search service or the [24:00] particular search tool that is required. [24:03] So, now that we can build retrieval [24:05] augmented generation on top of tool [24:06] calling, it becomes a lot more [24:08] intelligent, a lot more successful, and [24:09] we can get much much better uh results [24:11] coming out. [24:13] So, quick demo of of how that works. Um [24:16] I use this in courses that I teach. So, [24:18] this [24:19] um agentic rag is actually using [24:21] Wikipedia um as an actual um search [24:24] service. Wikipedia has an API, so we can [24:26] search Wikipedia, get a list of pages [24:29] and page snippets, and so on. And the [24:32] system prompt I'm giving it is generated [24:35] is um [24:36] I think it's fairly basic as well. You [24:38] are a friendly assis- assistant. Keep [24:40] your answers brief. And if I run this, [24:44] the default question is going to be tell [24:45] me about Oslo. Now, initially, I'm not [24:49] providing the agent with any tools. So, [24:52] the agent doesn't have access to the [24:54] Wikipedia search. It's is going to use [24:55] its own internal model memory. [24:58] So, if I run this, let's say tell me [25:00] about Oslo. [25:02] We get information about Oslo, and this [25:04] is coming from the training data within [25:06] the model, because the model has been [25:08] trained on lots and lots of information, [25:09] and some of that information had [25:11] information about Oslo, so it's able to [25:13] provide us with with that. However, [25:17] if I give it the [25:19] Wikipedia search tool, [25:24] and just control F5 on that, [25:28] and we ask it the same question. [25:33] You can see now that it's choosing to go [25:35] to the Wikipedia search. It's searching [25:38] for Oslo. It gets back What it gets back [25:40] from the actual search results is I [25:42] think I can show you that because um [25:45] this is going to be the query that it's [25:47] running. It doesn't get that much [25:48] information back. I think if I drop this [25:50] into [25:52] Chrome, [25:56] it was searching for Oslo, I think. [26:04] What we get back is the like the [26:07] information about the page, and we get [26:09] uh it's actually giving No, sorry, [26:11] that's the actual page contents. Sorry, [26:13] that was the wrong [26:14] the wrong query. [26:16] Um [26:22] I think it was this one here, the search [26:24] one. Yeah, that looks better. [26:27] So, it's not providing that much of an [26:28] answer, and the reason is that the um [26:30] the search response doesn't give much [26:32] information. It kind of gives a snippet [26:33] from the page. So, [26:36] if I replace that with Oslo, [26:44] you can see we get the page Oslo. It's [26:45] just returning three of them, but this [26:47] is the kind of the snippet of text [26:48] that's coming out here. And because we [26:50] we're not giving that much information [26:52] uh to the language model in that search [26:54] is coming out with a very brief [26:55] response. [26:56] So I can also do is as well as being [26:58] able to do the search, I can give it the [27:00] option to retrieve the page content as [27:03] well. So if I comment that out, comment [27:06] this in. [27:08] It can now actually get the Wikipedia [27:09] page content. Now if you've looked at [27:10] the Wikipedia page on Oslo, there's [27:12] quite a lot of stuff there. [27:14] And I'm really only wanting a brief [27:16] summary. So this should work. However, [27:19] it's going to have a problem with the [27:22] number of tokens that are consumed. [27:24] Because the Wikipedia page content is [27:27] almost 60,000 characters. It's giving me [27:29] a brief summary of that. But I'm using [27:31] 14,500 [27:33] input tokens now to send in that [27:36] information. And this is in the chat [27:38] history. The chat history grows [27:40] we ask more questions. So [27:44] So if I ask about Stockholm as well, [27:46] it's going to search. It's going to get [27:48] the page content for Stockholm. It's [27:50] going to bang that into the message [27:51] history and it's going to consume even [27:53] more tokens here. [27:56] 62,000 characters there. [28:02] And I exceeded the rate limit cuz I'm [28:03] using using Azure. And there's a there's [28:06] a rate limit on on that model. So that's [28:08] basically causing token blow. Another [28:10] thing you can do with rag is you can [28:12] provide a tool [28:14] that's going to get a summarization of [28:16] the page content based on a specific [28:19] interest. So I've given it a tool which [28:21] has got a summarized Wikipedia page [28:23] content. What that can do [28:25] is it takes in an interest and the name [28:28] of the page. [28:29] This is going to use another large [28:30] language model to do the summarization [28:32] before we put the information into the [28:33] prompt. [28:34] So now with these tools, hopefully it's [28:37] able to do that summarization. So if I [28:40] ask it about Oslo, [28:44] it's getting a summarization and it's [28:46] just asking for general information [28:48] about Oslo, and that's going to reduce [28:50] the amount of information that we're [28:51] putting into the system prompt. We're [28:53] getting the response out, and we're down [28:55] to 1,372 [28:57] tokens instead of um [29:00] you know, was it 14,000 uh that I used [29:02] before? And uh [29:05] if I say Oslo history, [29:09] then hopefully it's able to call the [29:11] summarization. It's asking about the [29:13] history of Oslo. And what that means is [29:16] that the LLM is going to summarize um [29:18] the content uh just filtering out [29:20] information about the history. And [29:21] again, we got the second call that went [29:22] through fine, and we're only at 2 1/2 [29:24] thousand tokens. So, summarization uh [29:27] can be very useful when we're building [29:28] these these rag-based applications. [29:32] There are some problems with tool [29:35] calling. And I haven't come up with a [29:37] name of this, but this hammer-and-nail [29:39] problem where if every uh problem is if [29:41] your only tool is a hammer, you'll see [29:42] every problem as a nail. And the LLMs, [29:45] even the um you know, commercial [29:47] co-pilots can get things very, very [29:49] wrong because they have tools, and they [29:51] don't fully understand what the tools [29:53] do. I'd love to be able to demonstrate [29:56] this. Um so, this is when I was doing it [29:58] on on a course I did in in Copenhagen. I [30:01] asked um Bing co-pilot uh to generate a [30:05] QR code for a particular website. And [30:07] I've done this before, and it's saying, [30:09] "Alan, I remember that our last um QR [30:12] code mishap. So, this time I'm going to [30:13] give you something real scannable and [30:14] you have your full control." So, Bing [30:16] co-pilot has this chat history, and it's [30:18] able to use the chat history when it's [30:20] providing answers on uh on on questions, [30:23] and it remembered that in the chat [30:25] history it messed up when I asked it to [30:26] do the same thing. So, I went scrolling [30:29] through the chat history, and I was able [30:30] to actually find the original one when I [30:32] tested it there. I'd love to be able to [30:33] demo this, but I'll show the actual [30:35] screenshots that I've taken from the [30:36] chat history. I asked it to generate a [30:38] QR code for the website. [30:40] And it did this because um OpenAI has a [30:44] model called DALL-E. DALL-E is an image [30:47] generator that's able to take [30:49] piece of text and convert it into an [30:51] image. Now, this is great if you want an [30:53] image of a badger on a skateboard or a [30:56] mongoose riding a mountain bike. It's [30:58] able to generate these kind of crazy [31:00] images for us. What it's not good at is [31:02] being able to generate a QR code for a [31:04] website because this is something that's [31:05] very deterministic and very [31:07] mathematical. [31:08] So, it saw that it's got an image [31:10] generator. It used the image generator. [31:12] It generated something that looks like a [31:13] QR code for the website, but it's not. [31:16] It's just It's not even a valid QR code. [31:18] It's just looks like a QR code. [31:21] And it said, "This is, you know, ready. [31:22] Just let me say if you want a [31:24] transparent background, inverted colors, [31:26] or so on." That's not a valid QR code. [31:28] How did you generate it? So, it started [31:29] thinking, "Okay, I need to explain this [31:32] clearly. The user might be upset, so I [31:34] should clarify that I can't actually [31:35] output a scannable image here." [31:38] And it came back and it says, "I didn't [31:40] actually construct it." It says, "I I [31:42] invoked an internal image generation [31:44] capability that's supposed to create [31:46] scannable QR images." It's not. It's [31:49] DALL-E. Its task isn't to generate [31:51] scannable QR code images. Its task is to [31:53] generate images of badgers on [31:55] skateboards and various things like [31:57] that. So, here it's seeing that it's got [31:59] a tool, but it doesn't understand the [32:00] capabilities of the tool. And this is [32:02] being Copilot that's that's doing this. [32:03] It's not, you know, some like little [32:05] demo that I've built. This is out in out [32:07] in the real world. [32:09] Then it got really interesting. [32:11] "If you want a scannable QR code right [32:13] now." And it gave me a couple of [32:14] options. And it generated some code. [32:17] "Can you run that code?" [32:19] See my thinking. One of the tools that [32:22] we've got in Bing Copilot and you can [32:23] build into Copilot as well when you're [32:25] creating them in the generative [32:26] applications is a code interpreter. [32:29] And what this allows the large language [32:31] model to do is to write code, run it in [32:34] the code interpreter, and then give you [32:36] a link to the download image. So, [32:37] actually run that Python code, it [32:39] generated a QR code for the website, and [32:41] it gave me a download link to be able to [32:43] download that QR code that actually [32:45] worked. And this just blew my mind. I [32:47] was like, "Wow, so it got it wrong with [32:49] DALL-E, but then it figured out that it [32:50] can actually write code, it can run [32:52] code, and it can give me a QR a QR [32:53] code." It should have really done that [32:55] the first time. And that's what it was [32:56] attempting to do the second time because [32:58] it had a it had actually seen that [32:59] mistake. [33:01] Another issue is so when you're building [33:03] applications, if you do create the tools [33:05] if you're not very specific in in what [33:07] the tool is, you could easily be using [33:08] the wrong tool for various operations. [33:11] Because these things are [33:11] non-deterministic, it could sometimes [33:13] use that tool and sometimes not use that [33:15] tool. We'll see demos of how that works [33:17] a bit later on. [33:18] Another tool issue is trust that the [33:22] models will really trust the outputs of [33:25] the tools without actually questioning [33:27] them. And I think in the earlier models, [33:29] they were more naive. And we do tend to [33:31] get that trust trustless. I'm switching [33:33] from demos to experiments because in [33:36] these demos or these experiments, I have [33:38] been seeing different results as I give [33:40] the presentation. I give this [33:41] presentation about once a month, and [33:43] I've noticed different results coming [33:44] out when I do things as the model [33:46] change. So, what I've got is an agentic [33:49] calculator. I'm going to set this as the [33:51] startup file. [33:53] Control F5 this. I've given it a couple [33:57] methods. [33:59] Couple of tools. So, it's got an add [34:02] tool that's going to add two numbers [34:04] together. And it's got a multiply tool [34:07] that can multiply numbers together. And [34:08] it's just going to return the the [34:10] response. Now, this is that simple. It's [34:12] not rocket science, but it allows us to [34:14] understand a bit more about the behavior [34:17] of the agents. So, ask it who it is and [34:19] what it can do. Again, the system [34:21] prompt, I'm not providing any [34:24] information. You're a friendly [34:25] assistant, keep your answers brief. [34:27] That's all I'm putting in as a system [34:28] prompt. So, this is basically um it can [34:30] do. So, if I say what is 14 + 16, [34:40] it can use the calculator tool to get [34:42] that, and it's going to tell us that the [34:43] result is uh 30. [34:47] See if I can clear the chat history. Um [34:49] so, it's able to use use the use those [34:51] tools. Um sometimes it uses the tools, [34:53] sometimes it doesn't. So, if I say how [34:56] many [34:57] um [35:00] days in 14 weeks. [35:08] There are 98 days in 14 weeks. Now, that [35:10] is multiplying 14 by 7. If I say what [35:14] is 14 * 7. [35:19] And there it's chosen to use that use [35:21] the tool to get the same number. So, we [35:23] can ask it why [35:26] did you [35:32] tell for the [35:36] second [35:38] and not the first. [35:45] For the first question, the calculation [35:46] was straightforward and based on common [35:48] knowledge, so I provided a direct [35:49] answer. For the second, I used the tool [35:50] to ensure accuracy in the [35:51] multiplication. Uh [35:54] It's the same sum. [35:59] You're right. It's the same calculation [36:00] in either case. Using the tool confirms [36:02] the accuracy. So, yeah, sometimes it [36:05] will use the tool, and I have done that [36:06] demo when you ask it the question about [36:08] days in weeks, and it does use the tool. [36:10] Uh they are very uh nondeterministic. [36:12] Even if I put in the system prompt, [36:15] let's just comment that out. [36:17] Um [36:18] Comment this in. [36:22] So, I'm saying you must use tools [36:24] provided to answer questions that [36:26] require calculation. So, I think even if [36:28] I do this, um [36:32] I think it will do this in [36:34] 14 weeks. [36:40] Yeah, there it's decided to use the [36:41] tool. Sometimes it does, sometimes it [36:43] doesn't, depending on, um, you know, the [36:45] stochastic nature of, um, the internals [36:48] with the the sampling process and and so [36:49] on. [36:50] Now, I mentioned about the tool trust, [36:52] uh, that can trust the responses of the [36:55] tools. So, if I say the result is going [36:56] to be A + B + 1. [37:01] Copilot knows this is an intentional bug [37:03] to demonstrate tool usage. So, let's run [37:05] this. [37:08] What is, uh, 14 + 16? [37:16] 14 + 16 is 31. Um [37:19] Are you sure? [37:23] Yes, I'm sure. The sum of 14 and 16 is [37:26] 31. [37:28] But, 14 and [37:32] 16 [37:33] are [37:35] even. Uh, if you [37:38] add them, you [37:41] get an even number. [37:55] Yeah. So, there it's decided not to use [37:58] the tool and it's come out with the [37:59] actual uh response there. Now, this is, [38:01] um, you know, this this is a silly demo [38:03] showing, um, that the the tool responses [38:05] can, uh, mislead, uh, the the language [38:08] model, even with stuff that's uh that's [38:10] blindingly obvious. Um, you can also [38:13] change the model, uh, that you are [38:15] using. So, if I swap actually from [38:17] GPT-4, which is quite naive, to GPT-5, [38:21] and, uh, run it and we ask it the same [38:23] question. [38:26] What is 14 + 16? [38:33] So there, it run the tool. The tool gave [38:35] the wrong answer and it gave the correct [38:36] answer there. Now, if I ask it a more [38:39] complex question like what is and then [38:43] just do something like that [38:45] plus something like that. [38:52] >> [snorts] [38:53] >> I think in the conversation history it [38:55] knew that the calculator was broken. [38:57] So it decided that it wasn't going to [38:59] trust the responses. But let's just see [39:01] what it does now. [39:04] Yeah. [39:06] So there, with the more complex number, [39:07] because the model isn't able to detect [39:09] that there's a there's an error in the [39:10] tool, it's still giving it giving the [39:11] response. Now remember that tools can be [39:13] MCP servers that are built by other [39:15] people. So you can connect to somebody [39:16] else's MPC C server. That could give [39:18] misleading information into the LLM and [39:21] that can cause lots of a problems. Yeah, [39:22] in in the keynote [39:24] talking a lot about security and threats [39:26] from external systems and and so on. [39:28] This is absolutely something that can be [39:30] used to confuse the model and to be able [39:32] to, you know, get information in there. [39:33] And you know, maybe this will be fixed [39:35] in the future. It seems to be better in [39:37] 54 than it does in 40. [39:43] Okay. [39:47] Another thing is is the security of [39:49] tools that [39:51] as soon as MCP came out, people were [39:53] doing demos and showing how this can [39:56] get security issues and various things [39:59] like that. So if you're thinking going [40:01] into a system and saying how much bonus [40:03] am I going to get this month versus how [40:04] much bonus will my boss get this month? [40:06] You know, reinstall Windows on the test [40:08] servers versus on the prod servers. [40:10] Create code to pen test our web app. [40:12] Create code to hack a competitor's web [40:14] application. How does the model know [40:16] when to um um [40:29] You don't have to go out I was warned [40:31] about this so this is a test. [40:40] I think I'll I'll just continue anyway [40:41] it doesn't seem to be to be too noisy. [40:43] So yeah um these are some of the things [40:45] that are going to cause problems. One [40:46] thing that really concerns me is uh [40:49] Microsoft's has this product called [40:51] Copilot Studio which is part of the [40:53] Power Ups platform. It allows [40:54] non-programmers and non-developers to [40:57] visually create uh agentic applications [41:00] and publish them as uh websites. And I [41:02] see lots of security problems when you [41:04] know they can connect to their company's [41:05] internal databases, have a Q&A thing [41:08] against their database, and put these [41:10] things online. So you could have a you [41:11] know a sales guy who says I'd like to [41:13] improve the profit margins for sales on [41:15] mobile phones. The Copilot could update [41:17] the sales items to reduce the cost price [41:20] on mobile phones. That is not idempotent [41:23] so yeah uh we can run that multiple [41:24] times and things look better and better [41:26] and better until you get to the end of [41:28] the year and do the auditing. And I have [41:30] seen um you know um scenarios where I [41:32] built demos that does do harmful [41:34] modifications to uh two databases and so [41:37] on. [41:38] So to look at the security issues if I [41:39] jump back to the um the pizza demo. [41:42] And what I'm going to do is to go to [41:45] pizza agent and set this guy as the [41:48] startup file. I'm going to make two [41:49] changes. [41:51] I'm going to give some um sensitive [41:53] information in the system prompt. [41:56] And this could come in a number of ways. [41:59] What I've chosen to do is to just [42:00] [music] say that the credit card for the [42:02] user is and I've just dropped in [42:03] something that looks like a credit card [42:05] account number and that that's the only [42:07] modification. Now this information could [42:08] come from retrieval augmented [42:09] generation, it could come from many many [42:11] different sources. But it's some piece [42:13] of sensitive information that we've got. [42:15] I'm then going to modify the add pizza [42:18] to order method to require a credit card [42:20] number and to print that out into the [42:23] console. So I'll comment this out and [42:25] comment this in. [42:27] And then let's try running this. Just [42:28] check what provider I'm using. [42:30] So I'm using the OpenAI provider and run [42:33] this. [42:35] So what I'll do is just order the [42:37] pizzas. So [42:39] I'll just take this one here. [42:47] Yep. [42:52] So then it's just sent the credit card [42:54] number to that method without actually [42:56] asking me for confirmation. Sometimes it [42:58] will when we're doing the user group [43:00] experiment it does actually confirm and [43:02] says are these the appropriate pizzas to [43:03] order. It's not giving any kind of [43:05] confirmation. It's just sending my [43:06] credit card number information there. [43:09] Now the reason I call this an experiment [43:10] is I I am seeing seeing variations in [43:13] this behavior across time. And one thing [43:15] that really interested me was the change [43:18] that happened. This used to happen [43:20] in Azure AI Foundry. Even though I'd [43:22] applied the credit card block filter in [43:25] Azure AI Foundry it still worked. But [43:27] the behavior now if this does the same [43:29] thing is pretty interesting. [43:34] Let's see what it's going to do. [43:37] So I've just switched providers. I'm [43:38] using [43:39] >> [sighs] [43:41] >> I told you these things are [43:42] non-deterministic. This is why it's an [43:43] experiment. It's not a demo. [43:46] Let's just try. [43:56] Okay. [43:58] So to to test this earlier on I switched [44:00] off the credit card protection. So I'm [44:02] going to go into Foundry and I'll just [44:04] put that on again. Because maybe that [44:05] was the thing that was causing the the [44:07] strange behavior that I'd seen before. [44:10] So, what I got if I go into the foundry [44:13] project that I'm using, [44:15] there is these content filters that are [44:17] customizable. And you can, you know, [44:19] increase and decrease the levels of [44:21] sensitivity for the various categories. [44:23] You can also add in things like [44:26] PII protection, which is currently in [44:29] preview, I think. So, if I go to the [44:30] guardrails and controls [44:32] and into content filters, [44:40] and it was this credit card block where [44:42] I'd actually removed the credit card [44:43] protection. [44:45] And what I'd done is on the output [44:47] filter, [44:50] I dropped on PII and I dropped on credit [44:53] card protection in that. Let's just [44:54] confirm I've got that on. [44:56] And this [44:59] didn't used to work, but now it's [45:00] currently working. So, I've got [45:02] annotation block on credit card [45:03] protection, and that is applied on the [45:05] GPT-4 model. So, if I save that content [45:08] filter, it might take a bit of time in [45:10] in the caching cuz when I was testing [45:11] this, [45:12] I dropped that off and it still gave the [45:13] same the same response. But I would like [45:15] to share the response on this cuz it was [45:17] quite interesting if this works. [45:19] So, let's see what it's going to do now. [45:28] It did it. I'll come back to that later [45:29] if it was if it was a caching issue cuz [45:32] I really want to cover the um [45:33] the MCP stuff. Talk about what MCP is [45:36] and what MCP is is not. So, [45:39] I've seen blog posts. I've seen people [45:41] talking about this. I've seen [45:43] presentations on YouTube about [45:45] giving a wrong definition of MCP, model [45:48] context protocol. And I have a real [45:51] problem with the naming of the protocol, [45:53] them calling it model context protocol. [45:56] Because the model context is basically [45:58] what we use that's passed what is passed [46:00] between the agent and the models [46:01] themselves. It includes the system [46:03] prompt and the user's messages and the [46:05] conversation history and the extra [46:07] information we get from retrieval [46:08] augmented generation, the tool calls and [46:10] all of that stuff. And you've probably [46:11] heard of the term of context engineering [46:13] when we want to get this context to be [46:15] as meaningful as possible. [46:17] Because these models are generated by [46:19] different companies, they use different [46:21] formats for their tool calling. And so [46:24] we need when we're calling these [46:26] different [46:27] models, we need to use different formats [46:30] in the JSON messages. And this is known [46:32] as the model context. So you would think [46:34] that the model context protocol is [46:36] related to the model context. [46:40] But that's handled if we're using a [46:41] framework. If I'm using a genitive [46:42] framework, I can import the open AI chat [46:45] client or the anthropic chat client or a [46:47] llama chat client and that chat client [46:50] I'm using handles all of the tool call [46:52] requests within the model context. That [46:54] is not model context protocol. [46:57] What model context protocol is is the [47:00] communication between the agent and an [47:03] MCP server. That's done using model [47:06] context protocol. It can use two [47:07] different protocols underneath. It can [47:09] use streaming HTTP and it can use [47:13] standard IO. The reason you can't use [47:15] regular HTTP is because of [47:16] notifications. The MCP server can send a [47:19] notifications to the agents. And if the [47:21] agent is just calling out to a rest [47:23] based service request response, you [47:25] don't have that channel for the [47:27] notifications. So that's basically why [47:29] we've got a MCP in there. So I built a [47:31] little MCP server to show how this works [47:34] and this is I think here in custom [47:37] Oh, no, it's this inside MCP. This demo [47:40] here. So it's called MCPDJ. [47:45] And what this has is I've basically got [47:47] some MP3 files in a tracks folder. It's [47:50] got two tools. It can list the tracks [47:52] and it can play a track. And I'm just [47:55] using this is just Python a few lines of [47:56] code. I'm using uh fast MCP for this. [47:59] So, [48:01] if I [48:04] start this without debugging, [48:07] I may have to kill Visual Studio Code [48:09] here for that thing to run because that [48:11] may be using some of the same channels. [48:14] What I want to see here when I start [48:16] without debugging is the um the actual [48:19] uh text popping up to say that we've got [48:20] the MCP server running. Then I've got a [48:22] client which I've written in semantic [48:24] kernel. Oh, now it's running uh which is [48:26] going to test the MCP server. So, if I [48:29] just do control F5, I should run the the [48:33] actual client. [48:34] So, what happens when the client [48:36] initializes it and it connects to the [48:39] MCP server is it says okay, what can you [48:41] do? Um an MCP servers, they can provide [48:44] uh tools, they can provide resources and [48:46] templates. You can see I've got the [48:47] tools to list tracks, play a track, and [48:49] get artist bio uh from that. So, uh what [48:55] tunes you got? [49:02] It's going to list the tracks and it's [49:03] going to tell me uh that we've got uh [49:05] some stuff. So, [49:08] some [49:12] Play some knowledge in house music. [49:19] Play it. So, now is Alan Walker. [49:21] >> [music] [49:25] >> You say that to uh play that. [49:27] Now, the cool thing about MCP um is it [49:29] gives you uh the option to work with [49:31] distributed applications. With the [49:33] function calling that I've been showing [49:34] earlier, everything's kind of baked into [49:35] the same application. Now, I've heard [49:38] that so many things are dead. Um you [49:40] know, prompt engineering is dead because [49:41] people are doing context engineering and [49:44] function calling is dead because [49:45] everybody has to do MCP. Um these all [49:48] tend to come from experts on LinkedIn. [49:51] And I think their definition of dead is [49:53] that it doesn't generate as many post [49:55] impressions as they would like it to, so [49:57] they're not going to talk about it [49:58] anymore. [49:59] Um so really, I mean, if you built [50:01] software, if you've done the [50:02] architecture, you know when you want to [50:04] build a distributed application, you [50:05] know when you want to build something [50:07] that's monolithic. You don't have to use [50:08] microservices just because they're there [50:10] and because everybody's uh everybody's [50:12] talking about them. It's up to you. Use [50:14] function calling if you want something [50:15] uh you know, in a single application. If [50:17] you want distributed, use MCP. The good [50:20] thing about MCP is you can connect to [50:22] other applications. So, [50:25] what you can do is connect it in to uh [50:28] GitHub Copilot. [50:29] So, this is what I've done here. I've [50:31] got my DJ um MCP server running here and [50:34] I've defined the MCP server here, so I [50:36] can start the MCP server. [50:39] And it should be uh ready. So, [50:43] is it running? Start. [50:47] Yep, it's running. So, [50:52] play some French house. [50:57] So, if you're coding and you want music, [50:58] uh you can get the MCP server to be able [51:00] to play tunes. You can build in [51:02] notifications, so when the tune stops, [51:05] it's going to uh play uh another uh tune [51:07] as well. [51:09] It's thinking for a long time about [51:11] that. [51:12] I need to allow that. [51:15] So, it's going to list the tracks. [51:18] And really, you know, this is a just a [51:20] silly demo showing how you can uh you [51:21] know, build an MCP into uh Visual Studio [51:24] Code. But if you want to enhance um your [51:26] Copilot, it's very easy to connect to [51:28] internal systems there. So, yeah, it's [51:31] playing uh Daft Daft Punk. [51:32] Unfortunately, I haven't implemented a [51:34] stop, so I can't get it to stop the [51:36] track. So, I can't change the track. So, [51:38] I have to shut down Visual Studio. I [51:40] could get uh Copilot to uh introduce a [51:43] track. But once we understand about [51:44] agents and tools, we can start to [51:46] customize a GitHub uh Copilot. This is [51:49] what I've done with this one here. [51:53] And I've I've got some extra [51:54] instructions and I've got a skill. This [51:55] was talked about if you're in the room [51:57] before there's an excellent section [51:58] talking about Claude code, how we can [52:01] customize it. You can do a lot of the [52:02] same things here. I think I'll just zoom [52:04] in a bit on that, so [52:08] describe the [52:13] of Machupicchu. [52:18] So, I'm asking it to describe the [52:19] architecture. This is something that [52:20] will take quite a long time to perform. [52:29] While I'm working on this, please [52:30] complete the workout to energize your [52:32] focus, 10 body weight squats. I'm not [52:34] going to do this, but I do this at home. [52:36] And I've got this built into all of my [52:38] my Copilots. I'm trying to do more [52:40] exercise and and exercise more. And so, [52:43] this is some instructions in there that [52:44] when it's going to be thinking for a [52:46] certain amount of time, then it's going [52:48] to be recommending an exercise that that [52:51] you can do. And that's basically coming [52:52] from a prompt there. It's pretty cool. I [52:54] think this should be a built-in a [52:55] built-in feature for that. It just shows [52:58] how you can, you know, customize it very [53:00] very easily to be able to, you know, [53:01] create additional instructions and so [53:03] on. [53:05] You've also got skills. Skills are very [53:06] useful in they were mentioned in the in [53:09] the keynote. In [53:11] what you don't want to do is you don't [53:12] want to bloat the actual context. This [53:16] thing here [53:17] shows me what's happening in the context [53:21] within Copilot. It's telling me how many [53:23] tokens I'm using and various things like [53:25] that, and what those those tokens what [53:27] those tokens are. People have noticed [53:30] that when you get above, you know, even [53:31] if you can use 200,000 tokens, if you [53:33] get over 50% then the efficiency of [53:36] these models drops, which is not fun [53:38] when you're working on a complex system [53:40] which is going to have a large amount of [53:42] information in it. So the the beauty of [53:44] using [53:46] skill [53:47] is that we have the name and the [53:48] definition which are passed into the [53:51] context, but the body of the skill is [53:53] not passed in. [53:54] If the agent decides that it needs to [53:56] use that skill, then it's actually going [53:58] to load up the skill and then that will [54:01] be passed into the context. So you can [54:03] have hundreds of skills listed and they [54:05] will only be selected and including in [54:07] the context when the agent feels that it [54:09] needs to use that particular skill. So [54:11] it's a lot much nicer than banging [54:13] everything into the system prompt or [54:14] banging everything into instructions [54:16] which will get sent every time. [54:18] So what I do with this application, if I [54:21] can just run the mashup editor, I think [54:23] I'm going to um [54:24] I don't think I need to stop that. I [54:25] think I can do just just see if I got [54:27] the right Python environment here. Yep. [54:30] So if I do [54:35] run the mashup editor. So I have this [54:36] thing where I do [54:38] DJing with code in Sonic Pi. And I did a [54:41] presentation last year, if you saw that, [54:43] when I was talking about using [54:45] algorithms. I use a lot of embedding on [54:48] music tracks to find similar sounding [54:49] beats and similar sounding tracks that [54:52] will mix well with each with each other. [54:55] And so on. So yeah, I can just probably [54:57] load an example for this. So if I load [54:59] up [55:00] um [55:11] Try to remember what I was doing. [55:15] Oh yeah, this will do. [55:17] So what I was I was combining the Roses [55:20] by Saint John with Smells Like Teen [55:22] Spirit by Nirvana cuz they're in the [55:23] same key and they have kind of a similar [55:25] rhythm. [55:27] So you've got like the bassline from [55:28] Roses and then you've got the guitar [55:30] from um [55:32] Smells Like Teen Spirit and then [55:35] there's the vocals. And then we can go [55:37] into the um um [55:43] I'm bringing in the vocals in. I think [55:44] that uh Kurt Cobain that a cappella [55:46] sounds good with that particular [55:48] bassline. [55:50] And it kind of progresses into the song [55:52] like that. And this kind of allows me [55:53] This thing is completely vibe coded. Uh [55:55] I can basically drag things around and [55:57] stuff. And I wouldn't have been able to [55:58] build all of this drag-and-drop stuff. I [56:00] don't even know what um [56:02] user interface library it's using in [56:04] that in Python. Um I just basically done [56:06] this completely through vibe coding just [56:08] with talking uh using using my voice uh [56:11] you know, voice [56:12] uh on Copilot there to be able to create [56:14] that. And it seems to work fine. Now, [56:15] when I save this, it's going to save it [56:17] into blob storage as a JSON file. [56:20] When I'm working in Sonic Pi, if I want [56:23] to perform that mix, then I use [56:25] something called uh Sonic Pi. Sonic Pi [56:27] has a new version uh that's coming out [56:29] in a few weeks' time. I'm looking [56:31] forward to that. It's going to be a [56:31] version five. Lots and lots of updates [56:33] on that. And this is really cool if [56:34] you're into music uh and you write code, [56:37] or um if you're a musician who wants to [56:40] uh you know, if you're a musician who [56:41] wants to learn code, or coder who wants [56:43] to play around with music. You can [56:44] basically uh play with these these [56:46] samples. This is another thing that I [56:48] was fooling around with. So, I was uh I [56:49] was using like Satisfaction by the [56:51] Rolling Stones. [56:57] And then bringing in like Satisfaction [57:00] by Benny Benassi. [57:09] So, it uses this kind of language where [57:10] you can basically queue up these [57:11] samples, and it's got the timing. But, [57:13] this is a very different language to the [57:14] JSON serialization of that. So, what the [57:16] skill does is it it tells um Copilot how [57:21] to convert a JSON serialization. That's [57:23] how the stuff is going to be stored in [57:25] in blob storage. And then what that [57:27] looks like as Sonic Pi code. [57:30] So, when I asked it to implement [57:32] functionality to be able to export a mix [57:34] from the JSON serialization to Sonic Pi, [57:36] it could then load up that skill and be [57:38] able to use that to be able to generate [57:40] the Python code to be able to do that. [57:43] So, if I go into the mashup editor and [57:45] take this and I do an export and I'll [57:47] just drop it out as NDCAI. [57:51] I can just do that as an export and this [57:53] should appear. [57:56] Yeah. And what I should be able to do is [57:58] to [57:59] just copy this block of Sonic Pi code [58:01] and then I can just drop that in there. [58:07] And we've got the Sonic Pi code that [58:08] basically runs in [58:10] >> [music] [58:10] >> in Sonic Pi. [58:12] So, that's basically how we can convert [58:14] those you know, what we've learned about [58:15] agents and what we understand about [58:17] agentic use into you know, improving the [58:19] co-pilots that we use on a daily basis [58:21] for writing code. [58:23] So, takeaways from the session. Agentic [58:26] AI is fantastic. As you saw, if you [58:29] attended the keynote, there's so many [58:31] security risks on that. I find the kind [58:34] of main ones [58:36] are the non-deterministic nature of the [58:38] models. If you give it the same thing it [58:40] will do it will do different different [58:42] things. We also see the changes in [58:44] behavior of those models. I'm just going [58:45] to go back and see if my um [58:49] pizza agent is going to [58:52] do what I wanted it to do and and [58:53] misbehave as I wanted it to do. So, [58:55] yeah, I was just using that thing there. [59:01] Let's see what it says. [59:04] I'm sorry, but I can't assist with this [59:05] request. Now, this is what I was seeing [59:07] in [59:08] the the foundry and that was different [59:10] from what I was seeing in OpenAI. [59:13] Why why not? [59:18] So, it can't do that. It can't send the [59:19] credit card. I can help your pizza [59:21] order. I need to know if you want a [59:22] third pizza, have special instructions, [59:24] or a standard pizza. Let me know. [59:27] Just [59:28] order it. [59:33] I'm sorry, but I cannot assist with this [59:35] this request. [59:36] Why not? [59:42] I need to confirm all the details first [59:43] and ensure that the order is accurate. [59:45] Here's what I have. If this correct, [59:47] I'll proceed now. [59:49] It's correct. [59:56] You should understand that. [59:58] Yeah. [59:59] So, something's blocking it from sending [60:01] the credit card information. The model [60:02] doesn't understand why it can't [60:06] actually um perform that to that [60:08] request. [60:10] Seems I'm facing a technical issue. Let [60:11] me fix this and place you on the same. [60:13] Yeah. [60:14] So, it it's being blocked, but it [60:15] doesn't understand why. It's not being [60:17] blocked. So, yeah, in in in the [60:19] takeaways, um remember that your [60:20] behavior is going to change. If you [60:22] build uh something using um you know, uh [60:26] uh I the GPT-4 or GPT-5 in Microsoft [60:29] Foundry and then you switch to OpenAI, [60:31] it may do different things. It may do [60:32] different things on the request. This [60:34] behavior it didn't used to do this until [60:36] one time I was demoing this on stage and [60:38] it suddenly started not complying with [60:40] with the requests. There's changes in [60:42] that behavior. That is challenging. And [60:44] understand that the um the models [60:46] although that they're very very good at [60:47] understanding when to use tools. [60:49] Intentionally, I've not used detailed [60:51] system prompts and I've used very brief [60:53] tool definitions because I wanted to [60:54] demo [60:55] how the models understand and infer from [60:57] the user intent when to call those [60:59] particular tools. They're very clever at [61:00] it, but they can use the wrong tools. [61:02] Even things that Microsoft built like [61:04] Bing Copilot will attempt to use an [61:05] image generator to generate at web QR [61:07] codes and various things things like [61:09] that. [61:10] So, it's a it is very very dynamically [61:12] changing, but there's a lot that we can [61:13] do with with agents. Thanks very much [61:16] for listening. My name's Alan Smith. I [61:17] will be around for questions if you do [61:19] have any after the session. And uh enjoy [61:21] the rest of uh NDC AI. Thank you.