NYC Council Chief Data Scientist Deploys In-House AI for Legislation
Ai4
New York City Council uses open-source vector embeddings and local LLMs to eliminate duplicate legislative submissions and automate municipal operations.
## Background
This video was published on YouTube by Ai4 on August 18, 2026. According to the page description, Alaa Moussawi, Chief Data Scientist at the New York City Council, presents the design, open-source hardware and software development, and on-premises deployment of a locally hosted Retrieval-Augmented Generation (RAG) LLM system within a local government organization.
## TL;DR
Alaa Moussawi discusses how the New York City Council's in-house technical team leverages open-source data science and software engineering to modernize legislative workflows and support evidence-based policymaking. He traces the evolution of their internal tools from a 2018 Word2Vec-powered duplicate proposal checker to modern vector embeddings and semantic search systems. The talk highlights practical operational frameworks, data management practices, and technical methodologies for implementing secure, locally hosted RAG architectures in public sector environments.
## Atmosphere & style
The presentation is pragmatic, technical, and lighthearted, opening with humor to engage the audience before diving into deep technical topics. Moussawi delivers the material in an approachable, grounded manner, drawing directly on his eight years of experience building municipal tech systems. The talk balances complex concepts like high-dimensional vector math with down-to-earth real-world problem solving, making it feel like an authentic, behind-the-scenes look at engineering in local government.
## Key takeaways
- [00:04](https://www.youtube.com/watch?v=2rLb8VOxJG4&t=4s) Alaa Moussawi serves as Chief Data Scientist for the New York City Council, managing in-house software engineering and data science teams to modernize legislative processes.
- [00:04](https://www.youtube.com/watch?v=2rLb8VOxJG4&t=4s) The NYC Council relies strictly on open-source tooling to automate administrative workflows and support evidence-based policymaking.
- [02:14](https://www.youtube.com/watch?v=2rLb8VOxJG4&t=134s) Before launching AI projects, organizations must evaluate technology maturity, data accessibility, user value, and operational constraints.
- [02:14](https://www.youtube.com/watch?v=2rLb8VOxJG4&t=134s) In 2018, the NYC Council built its first production ML tool, the Duplicate Checker, to identify overlapping legislative proposals submitted by 51 council members.
- [02:14](https://www.youtube.com/watch?v=2rLb8VOxJG4&t=134s) The original Duplicate Checker utilized 2013 Word2Vec embeddings to compute similarity rankings in 5 to 10 seconds, drastically reducing manual review errors.
- [05:07](https://www.youtube.com/watch?v=2rLb8VOxJG4&t=307s) Vector embeddings convert text into lists of numbers representing coordinates in high-dimensional vector spaces (spanning 1,000 to 8,000 dimensions).
- [05:07](https://www.youtube.com/watch?v=2rLb8VOxJG4&t=307s) High-dimensional vector dimensions represent learned semantic features discovered by neural networks, allowing mathematical operations on concepts.
- [05:07](https://www.youtube.com/watch?v=2rLb8VOxJG4&t=307s) Semantic vector arithmetic (such as 'royalty' plus 'woman' equalling 'queen') illustrates how spatial proximity maps conceptual relationships.
## Detailed walkthrough
Here is a detailed, faithful walkthrough of the presentation based on the transcript:
[00:04](https://www.youtube.com/watch?v=2rLb8VOxJG4&t=4s) Introduction and the Dual Mission at New York City Council
Alaa Moussawi (who also introduces himself as Al) opens the presentation by setting a lighthearted tone, hoping the audience will stay awake during a deep dive into Retrieval-Augmented Generation (RAG). He serves as the Chief Data Scientist for the New York City Council—the legislative branch of New York City government responsible for passing local laws and approving the municipal budget. Over his eight-year tenure, Moussawi has assembled and managed two distinct in-house technical teams: a data science team and a software engineering team. Together, they build open-source tools designed to modernize how the council interacts with data.
On the data science side, his team focuses on making the legislative process more evidence-based through statistical analysis. On the software engineering side, they develop internal platforms to manage everyday operations and automate workflows. Moussawi highlights that the intersection between these two disciplines—fact-based analytical rigor and process automation—is precisely### Introduction and the Dual Mission at the NYC Council
Alaa Moussawi (who also goes by Al) opens his presentation with a quick joke hoping everyone stays awake as he jumps into the details of Retrieval-Augmented Generation (RAG). As Chief Data Scientist for the New York City Council—the legislative branch tasked with passing city laws and negotiating municipal budgets—Moussawi reflects on his eight-year tenure building an in-house team of software engineers and data scientists from the ground up. The team operates strictly around open-source tooling, aiming to systematically modernize how the local government leverages data.
His organization bridges two distinct operational thrusts. On the data science side, the focus is making the legislative process strictly evidence-driven through statistical analyses. On the software development side, the goal is automating daily administrative workflows. The intersection of these two areas creates the sweet spot where artificial intelligence can genuinely augment public sector operations.
- **Role:** Chief Data Scientist, New York City Council (8+ years building the tech team).
- **Legislative Mandate:** The NYC Council consists of 51 elected officials handling municipal laws and city budgets.
- **Core Strategy:** In-house development using open-source tools to automate processes and enable evidence-based policymaking.
---
[02:14](https://www.youtube.com/watch?v=2rLb8VOxJG4&t=134s) The Philosophy of Project Evaluation & The 2018 Duplicate Checker
Before launching any AI project, Moussawi emphasizes starting with purpose and clear intention. Teams must verify that the underlying technology is mature enough, that necessary data is accessible, and that the final output provides tangible value to end-users within realistic operational constraints.
To demonstrate these fundamentals, he steps back to 2018—prior to the modern generative AI boom—when his team built their first production machine learning tool: the **Duplicate Checker**. Every term, 51 council members submit tens of thousands of legislative proposals through custom in-house software. Because the legal team cannot draft every single idea, staff must quickly conduct a duplicate search to confirm which council member originally submitted a concept.
```
+-------------------------------------------------------------------+
| NYC COUNCIL LEGISLATIVE WORKFLOW |
| |
| 51 Elected Members ---> 10,000s Proposals ---> In-House App |
| | |
| v |
| Duplicate Search |
+-------------------------------------------------------------------+
```
- **Problem:** Council members submit overlapping ideas expressed in vastly different phrasing, making human cross-referencing tedious and error-prone.
- **2018 Baseline:** Implemented using Google's **Word2Vec** (2013 technology) to generate word embeddings.
- **Performance:** Took 5 to 10 seconds to generate similarity rankings—significantly faster than manual review, helping reduce costly legislative errors.
Moussawi simplifies vector embeddings, demystifying the mathematical concepts behind modern AI models. At its core, an embedding is just a list of numbers (a vector) representing a specific coordinate in a high-dimensional space. While human visual intuition is limited to two or three dimensions, computer vector spaces regularly span 1,000 to 8,000 dimensions. Each dimension represents a learned feature deduced by a neural network.
To illustrate spatial math, he uses a classic semantic vector example: adding the vector for "woman" to the vector for "royalty" lands near the coordinate for "queen". Subtracting "royalty" from "queen" yields "woman" once again. While modern systems embed entire sentences using more sophisticated methods, Moussawi's early duplicate
## Notable quotes
> [00:04](https://www.youtube.com/watch?v=2rLb8VOxJG4&t=4s) hoping everyone stays awake as he jumps into the details of Retrieval-Augmented Generation (RAG) — Alaa Moussawi
## Mentioned
Alaa Moussawi, Al, New York City Council, Retrieval-Augmented Generation, RAG, Google, Word2Vec
## Video description
Alaa Moussawi - Chief Data Scientist, New York City Council
Are you curious to see a full deployment, from nuts and bolts (open source) to delivery? This presentation details the design and deployment of a locally hosted, open-source LLM within a government organization, from ordering individual parts and building a server by hand, to developing the software and integrating it into existing systems. It covers LLM fundamentals, an overview of Retrieval-Augmented Generation (RAG), and operational best practices, followed by a practical discussion of secure, on-premises implementation. Attend this talk to see the full picture!
Alaa Moussawi, Chief Data Scientist for the New York City Council, outlines how the municipal legislature combines data science and in-house software engineering to modernize legislative workflows through natural language processing and retrieval-augmented systems.
Municipal Mandate and Technical Strategy (0:04)
The New York City Council operates as the city's legislative branch, comprising 51 elected officials tasked with negotiating municipal budgets and enacting local laws. During his eight-year tenure, Moussawi established two internal technical groups—a data science team focused on evidence-based legislative analysis and a software engineering team dedicated to operational workflow automation. The council relies strictly on in-house development and open-source tooling, targeting intersections where automated systems can support public sector operations without external vendor lock-in.
Pragmatic Project Evaluation and the 2018 Baseline (2:14)
Moussawi argues that deploying machine learning in government requires confirming technological maturity, data accessibility, and clear operational value before implementation. To illustrate this discipline, he points to the council's 2018 'Duplicate Checker,' an internal tool created prior to generative AI to manage tens of thousands of legislative proposals submitted each term. Because council members frequently submit overlapping policy concepts using divergent phrasing, staff required an automated mechanism to confirm submission priority.
Semantic Search via Vector Embeddings (5:07)
The 2018 Duplicate Checker relied on Google's 2013 Word2Vec architecture to convert text into vector embeddings—numerical coordinates positioned across high-dimensional feature spaces typically ranging from 1,000 to 8,000 dimensions in modern systems. By conducting spatial mathematics, such as the semantic relationship where 'woman' combined with 'royalty' maps near 'queen,' the model compared proposal semantics rather than exact keyword matches. The system delivered ranked similarity comparisons within 5 to 10 seconds, cutting manual review time and reducing drafting conflicts among legal staff.
The presentation notes cut off while transitioning from early word-level embeddings to modern sentence-level architectures, leaving the technical details of the council's current Retrieval-Augmented Generation implementation unspecified.
[0:04] Alaa Moussawi: hoping everyone stays awake as he jumps into the details of Retrieval-Augmented Generation (RAG)
Original transcript
[0:04] Hi, my name is Alatam Sawi or Al and
uh today we'll be talking about RAG um
[0:12] and hopefully staying awake
as we do. All right. So, um,
[0:17] yeah, as was mentioned, I am the chief data
scientist for the New York City Council,
[0:22] which is the legislative branch
of the New York City government,
[0:26] meaning that we pass the laws for New York City
and we pass the budget for New York City. Um,
[0:31] and at the New York City Council, I I've been
there for a little over eight years. I run a
[0:35] team of data scientists and a team of software
engineers that I've put together over the past
[0:41] eight years building all sorts of capabilities
inhouse um developing everything we do open-source
[0:48] uh you know building that skill set and slowly
transforming how the council interacts with data
[0:56] in part by on the data science side making the
legislative process more fact-based and evidence-
[1:02] driven and using statistical analyses to help
make decisions and then on the software side
[1:10] by helping develop software that manages
everyday processes and automating things
[1:16] uh left and right and in the space between
the two is where the opportunity for AI lives
[1:23] and that is what I'll be
talking about today. Um so
[1:33] I'll start off by motivating and
explaining the use of vector embeddings
[1:38] uh looking at some of the limitations and
discussing how uh LLMs and AI have impacted
[1:48] uh workflows at the New York City Council.
Then I'll walk through some history of how
[1:54] AI has come to be at the point that it is at
today and so that we can better understand
[2:00] its capabilities and its shortfalls. And finally
uh I'll explain how at the at the council we're
[2:06] using AI to empower staff uh and try to explain
as many technical concepts as I can along the way.
[2:14] So now before we dive into any project,
it's always good to make sure that we're
[2:20] doing it with purpose and with intention and that
there's a feasible outcome at the end of the day.
[2:28] You should also make sure that the necessary
technology exists and it's capable of doing
[2:34] what you're trying to do and that you have access
to all the data necessary to do it. And finally,
[2:42] that given the realistic constraints
and limitations of what you're doing,
[2:47] the end product will be beneficial to the end
user. So, we're going to start off today's talk
[2:54] with a model considered now by today's standards
to be very rudimentary, but one that's very
[2:59] instructional and can get us to understand some
of the most basic concepts of machine learning,
[3:05] which is a term I'll interchange with AI. Same
thing in my head. I apologize if you disagree. So
[3:12] back in 2018 before the explosion of
generative AI was when we first built our
[3:19] first machine learning application. It it's called
the duplicate checker. And so at the New York
[3:24] City Council there's 51 elected officials. They
submit tens of thousands of ideas that they want
[3:31] to become legislation. And they submit these
ideas into software that we've built inhouse.
[3:37] and we can't draft tens of thousands
of ideas. So there's a whole process
[3:43] that goes on and it's managed by the software.
And part of the process is performing what's
[3:49] called a duplicate search. So you have to identify
which council member thought of the idea first.
[3:56] And you can get similar ideas written with very
different words. And so you'll see why this is an
[4:05] an example of something that's very well
suited for vector embeddings to handle. Um
[4:12] and so this is something that is a
challenge for humans and for computers to do
[4:18] um just because of the nature of it.
And so we had to take a stab at it. Um,
[4:27] so the first implementation, and we're going
to get into exactly how all this works,
[4:31] used Google's word, which was 2013 technology
for generating word embeddings. Um,
[4:39] and it would take 5 or 10 seconds to
spit out a list of recommendations.
[4:44] Now, similarity rankings. Now, that's a lot
faster than a human, but it's still slow. But
[4:54] the point is that it can make the human a lot more
efficient, saving a lot of time and increasing uh
[5:00] decreasing the likelihood of errors that are that
can be very costly in the legislative process.
[5:07] And the key technology behind this is the vector
embedding. So what is a vector embedding you ask?
[5:16] A vector embedding is simply a list of numbers.
That's it. It's called a vector in math
[5:23] and it represents the position or a position in
a space. So it can in a three-dimensional space
[5:33] it's a position right here. On a two-dimensional
space it'll be like somewhere on your flat screen.
[5:39] Uh and it can be a bunch of numbers. It could
be like a thousand numbers. It could be 4,000
[5:44] 8,000 numbers. And each number in this vector,
this list of numbers holds some significance. So
[5:54] we don't know, it's very difficult to figure
out what the significance is because the neural
[5:58] network is deducing what the values are, but
it could be like a combination of things. It
[6:04] could be like how intelligent and crunchy you
are. Like we're just making up random things.
[6:09] And that could be one of the numbers. And then
all of them have their own specific meaning.
[6:13] and then all of them coming together in
this vector space represent something.
[6:19] So uh to get the point across let's consider the
vector for royal uh sorry the vector for woman
[6:30] and the vector for royalty. And now
these are in two dimensions because
[6:36] we only have two dimensions on the
screen in your computer. It lives in a
[6:40] very highdimensional space, but
the same math extrapolates. So
[6:46] if we add the vector for royalty to the vector for
woman, let's say we were playing the game taboo
[6:53] and I said royalty and woman, you guys, what
would you guess? We queen Elizabeth maybe.
[7:00] Uh, and yeah, exactly. That is what you get if you
build a good vector embedding model. it should be
[7:08] able to deduce that relationship. So similarly, if
you take queen and you subtract royalty, you get
[7:16] woman. Um, and this is the crucial technology that
was employed in the duplicate checker and it's
[7:24] part of the basis of all large language models
because they have to first convert words into
[7:32] numbers because machines don't understand just
words. Um, and so you use the full set of text
[7:42] and you can train a model and it'll eventually
learn the relationships between words based on how
[7:51] how close to each other they appear in in text
which is a pretty impressive thing. Now similarly
[8:00] if you have the vector for the word
idea and the vector for the word whole
[8:12] I can make something up because we make things
up. Now this is not how they do it in modern
[8:18] LLMs but this is actually how I did it for this
example uh for the duplicate checker that I use.
[8:25] You can just sum up the words for whole idea
and you get the vector for a a sentence. Now
[8:32] whole idea this is not the way it's done in modern
systems but it's a way that you can do it. Uh
[8:41] and then what can you do now? You have one
vector or one point in space for whole idea
[8:48] and you have another vector or another point
in space for another sentence. you can measure
[8:54] the distance between them. And
the closer these two vectors are,
[9:01] the more similar they are. And measuring the
distance is literally as simple as the Pythagorean
[9:07] theorem in two dimensions. And then you uh
extend it out into a higher dimensional space.
[9:16] Now that said, one point I do want to emphasize
is that this duplicate checker doesn't provide any
[9:23] final recommendations. And the most direct
explanation of why it doesn't is because
[9:31] assessing whether two ideas are similar enough
to be considered duplicates of each other
[9:37] is inherently subjective. attorneys within the
organization will disagree on whether two ideas
[9:44] are duplicates of each other. And so if the
if the humans can't deterministically come to
[9:51] a conclusion, then it's lost hope for the
AI. So it's a recommendation system. Um and
[10:03] yeah, it has its limitations. So with no
deterministic way to evaluate the output,
[10:09] a machine certainly never will. Um, and this
really has nothing to do with AI. This this
[10:15] applies anywhere else as well. Anyway, that said,
I'm going to now pivot us slightly and I'm going
[10:22] to take a moment to briefly go through the recent
history of developments in AI to motivate why even
[10:29] though I am a huge fan of AI, I call it stupid.
So about a decade ago, we couldn't rely on Gen AI
[10:38] to reliably take on tasks such as summarization,
which now we take for granted, and now it could
[10:44] like it could be your therapist. It can debug your
code for you. It can develop whatever for you.
[10:49] It can take on different personalities. It can
write you a really nice song that rhymes very
[10:55] well. Um, and so how did we get here to this
amazing point? So the recurrent neural network
[11:05] was the first to have a feedback loop but
it had a large set of issues which were
[11:11] eventually resolved with the long shortterm
memory neural network architecture in 1997
[11:18] um which could remember words that came earlier in
a sequence and the order they came in. uh and it
[11:26] it solved something called the vanishing gradient
problem that the recurrent neural networks had.
[11:32] But then it also had its own issues um which
was that it was only predicting the next best
[11:42] token given the previous sequence. And
then they came up with something called
[11:48] the attention mechanism which you've probably
heard of and that was uh 2017. Um and so
[11:58] it came up with something a little a little better
in that it could pay attention to the positions of
[12:04] all the words in the past and it could understand
how important they were to generate the uh
[12:13] the next token. Now, what's really important
to understand is all of these models, including
[12:23] transformers, sorry, it was
transformers, that was 2017. Um,
[12:30] they're all doing the same thing on the back end.
All these models are doing next token prediction.
[12:37] That means that you feed them. By the way,
token is the same more or less as saying word.
[12:44] Like runner is runer two tokens, but like
chicken is one token u so on and so forth.
[12:52] So all these models you ask it why did
the chicken cross the road and all it does
[12:58] is it and and all of the models we're
using today fable mythos opus whatever
[13:05] they're just predicting the next most likely
token given a sequence of previous tokens.
[13:12] So you ask it why did the chicken cross the road
the model predicts the word 'the' and it's done.
[13:20] Then it takes all of that. Why did the chicken
cross the road? The It pumps it back into itself
[13:24] and it predicts the word chicken. And then
it takes Why did the chicken cross the road?
[13:28] The chicken pumps it back into itself. Crossed.
Repeat. Rinse. Repeat. Recycle. And that's because
[13:35] that's the best way that scientists have thought
of to make a model talk. I mean, these models are
[13:42] talking, which is wild. It's very impressive.
So, but the architecture of how they're talking
[13:51] is very basic and it's based on the same
stuff that we've been using since the 80s.
[13:58] And if you if you really understand that, you
should start to understand that when a model is
[14:04] spitting out words or tokens, it doesn't know what
it's going to say in the future. So, it says the
[14:08] chicken crossed. It has no idea that it's going to
say the chicken crossed the road. It's just saying
[14:13] the chicken crossed because given all the data
that this that it's been trained on which now you
[14:20] have like uh it's been trained on every Wikipedia
article all these like everything trillions of
[14:26] tokens right that that's why the training data
is so important it's picked up these patterns
[14:32] from all this training data and now it's able to
predict the next most likely token in the sequence
[14:38] in such a convincing way that it seems seems to
understand what it's saying. But uh I would argue
[14:49] that that is not true understanding. It's
just predicting a token in a sequence
[14:55] given training data without any concept of what
it's saying. I right now am expressing an idea to
[15:01] you with words and I'm finding the words to relay
that idea and you're understanding this concept
[15:08] that I have in my head that I'm using words as
a tool to just transfer over to you right now.
[15:15] That's not at all what LLM are doing. LLMs
are predicting the next most likely token
[15:21] given a whole bunch of training data and that
is it. So the patterns and the training data
[15:27] have been baked into them. So, it's learned
grammar and persuasion, how to speak politely
[15:33] just because it was in the training data
and and that's all there is to it. Um so
[15:43] yeah, now they get to the point of um transformers
and they're doing things a lot more efficiently
[15:54] now than they were doing in the past. Um,
[16:01] and so really in terms of like serious scientific
breakthroughs and progress since Transformers,
[16:09] nothing I wouldn't say
anything has been like a crazy
[16:14] breakthrough, but they came up they came to
some scientific realizations around 2020. And
[16:28] so what they realized is that there's and this
is going to get a little complex but there's
[16:33] a consistent power loss scaling between compute
this is in the wrong order compute data set size
[16:44] and the number of parameters in your model and the
performance of your model. So this was like a 2020
[16:52] maybe Google paper. I'm not I'm forgetting.
But you see how there's a straight line even
[16:57] though it's on a logged scale graph. The line
kept going down. So what they realized is
[17:04] we can either train on more data. We can either
throw more compute power at it or we can train
[17:16] uh a larger model with like you know how they
say llama 70B 70 billion parameters or like uh
[17:23] quen 27b 27 bill. So you could just
make the model bigger, more neurons,
[17:28] add more data or add more compute and you keep
getting smarter and smarter models. Now remember
[17:33] in 2020 we didn't have chat GPT. Nobody knew that
these machines were going to actually like start
[17:38] talking and they like blow our minds. But then
they convinced I don't know who they convinced
[17:46] to dump a lot of money like millions of dollars
just on electricity and equipment to run and start
[17:53] training these M. And then and then that's how we
got here. It was from dumping massive amounts of
[17:59] first the scientific realization that we could
just dump massive amounts of money into these
[18:07] computers with the current technology and it'll
just keep getting better and better and better
[18:11] and then they did it. And then and then right now
like in my opinion like any advancements we see
[18:17] um although this could be a contested opinion are
just because they're making this for the most part
[18:22] this is not 100% true. they are doing some things
that are but like on the on the cutting edge like
[18:27] you get all these open source models that are
now competing with the closed source models
[18:31] that's just cuz they're like building models
that are their size 2.8 trillion parameters now
[18:35] where we are um and so yeah that that was
it. So, we just made things more efficient
[18:45] and we're still finding ways to make things more
efficient as we go and increasing the size. Um,
[18:53] and so, all right, let's forget about
the history for a second and we're going
[18:56] to go back to applications at the New York City
Council. So, at the New York City Council, we uh
[19:02] we run [clears throat] our stuff locally and we
host it on prem. uh meaning that all the all we
[19:10] pay is for the electricity consumption beyond
the initial cost of some RTX 5090s that we run.
[19:18] Um and given the small size of the models
that we run, it's not so expensive. And
[19:26] so what we wanted to do is we wanted to
utilize a vast repository of legal documents
[19:33] uh that the New York City Council had. These are
documents that contain all of the legal findings
[19:40] from the research that the attorneys and the
policy analysts have performed in the past
[19:44] as they're drafting legislation because in the
future any legislation they draft needs to be
[19:50] consistent with any of their findings from earlier
in time. And so I'm going to make this very quick
[19:57] because I've ran a little over or I will run a
little over. Um so the first step in doing this
[20:07] is to index your documents into a vector
database. Um and this is exactly what
[20:14] we discussed before about we learned about
vectors. So you have a database and you just put
[20:21] your vectors right here into your database and
then the sentence that they are representing.
[20:28] And you do that for every sentence in
your so you take all your legal memos,
[20:34] you chunk them up into sentences or you
chunk them up in an appropriate way. Um,
[20:40] sentences are good because a you
want something that can hold one idea
[20:45] because a vector holds about one idea and a
sentence more or less holds about one idea.
[20:50] So then you generate vector embeddings for all
of them. you put it into your database and then
[21:00] you create a whole pipeline. And so now
what this pipeline is going to do is a user
[21:06] is going to ask a question the query. We're going
to create a vector embedding of the user's query.
[21:13] So now we have this vector embedding and we have
this whole database of vector embeddings. We're
[21:18] going to do a similarity search to find which of
these vector embeddings here are closest to the
[21:25] vector embedding we made from the user's query.
And this is the exact same type of similarity
[21:32] search that I was doing with the duplicate checker
example. And then you take like the top five or 10
[21:37] hits, you find those sentences and they're going
to be sentences within legal memos. So you take
[21:43] the some context around them as well. So, you want
to get like the paragraph they're in because you
[21:49] don't want to just give the sentence because
the LLM needs a little more context than that.
[21:53] And then you take that whole pipeline and you feed
it back into um the LLM with a prompt telling it
[22:05] these this is what I've pulled that's relevant
to the user's query. Answer the user's query
[22:13] and don't make up stuff. only answer based on
the text given da da and that's it and that is
[22:21] what we call retrieval augmented generation.
Um and so yeah exactly with this process
[22:30] uh we deployed a model first very slow then we
got better hardware and now runs a lot faster and
[22:37] it just opened up the ability to go new places
where now we're building out a whole now we're
[22:44] we're focusing more on MCP uh agents and
uh building out like chat bots. This is
[22:52] a chatbot essentially. Everything is a
chatbot uh with other capabilities. And
[22:57] yeah, hope you enjoy enjoyed the talk and
got something useful out of it. Thank you.
