Bloc
Connecting generative language models to character-specific story engines enables unscripted player dialogue in roleplaying games without breaking original lore or quest structures.
AI-assisted summary of Future of Elder Scrolls & RPG - Skyrim NPCs & Inworld AI (like GPT-4 for gaming) — original source. Check the original for full context.
Source published:
Key ideas
- Modder Bloc showcases an integration of Inworld AI and ElevenLabs speech synthesis into Skyrim, enabling real-time, dynamic NPC conversations that adhere to lore and predefined character backgrounds. (0:15)
- The mod replaces standard dialogue trees with freeform text input via a custom UI without stripping out Bethesda's original quests or handcrafted dialogue.
- Demonstrations across Whiterun show NPCs contextually expanding on established lore, including Jon Battle-Born's secret romance with Olfina Gray-Mane and Ysolda's ethical justifications for dealing skooma. (3:54)
- Guard dialogue confirms and elaborates on Skyrim's famous 'arrow in the knee' meme as a literal bandit ambush injury that ended an adventurer's roving career rather than a metaphor. (14:26)
Detailed notes
## Background Published on YouTube by Bloc on April 28, 2023, this video showcases a custom mod integrating Inworld AI and ElevenLabs speech synthesis into The Elder Scrolls V: Skyrim. According to the page description, the system provides NPCs with a custom story engine allowing players to type freeform dialogue prompts while preserving the characters' original personalities and lore.
## TL;DR The video presents an in-game demonstration of dynamic AI-driven NPC conversations throughout Whiterun. The player chats with Lydia outside the city gates before entering Whiterun to question Jon Battle-Born about his secret romance and probe Ysolda on the ethics of her skooma trade. After hearing Carlotta Valentia's complaints regarding an unwanted suitor and uncovering the detailed backstory of a guard who took an arrow in the knee, the tour visits Dragonsreach to speak with Irileth and Jarl Balgruuf about their loyalties and neutral stance in the civil war.
## Atmosphere & style The showcase possesses an exploratory, tech-demo ambiance that seamlessly blends cutting-edge generative AI dialogue with Skyrim's classic, atmospheric high fantasy setting. Interacting via typed custom UI prompts and generated voice synthesis, the flow feels like an organic, inquisitive walking tour through Whiterun where NPCs sound authentic, grounded, and lore-accurate. It balances serious world-building debates on contraband and wartime politics with playful immersion, closing on an endearing, tongue-in-cheek call to action delivered directly by an arrow-struck guard.
## Bulletin - [00:15](https://www.youtube.com/watch?v=d6sVWEu9HWU&t=15s) Modder Bloc showcases an integration of Inworld AI and ElevenLabs speech synthesis into Skyrim, enabling real-time, dynamic NPC conversations that adhere to lore and predefined character backgrounds. - The mod replaces standard dialogue trees with freeform text input via a custom UI without stripping out Bethesda's original quests or handcrafted dialogue. - [03:54](https://www.youtube.com/watch?v=d6sVWEu9HWU&t=234s) Demonstrations across Whiterun show NPCs contextually expanding on established lore, including Jon Battle-Born's secret romance with Olfina Gray-Mane and Ysolda's ethical justifications for dealing skooma. - [14:26](https://www.youtube.com/watch?v=d6sVWEu9HWU&t=866s) Guard dialogue confirms and elaborates on Skyrim's famous 'arrow in the knee' meme as a literal bandit ambush injury that ended an adventurer's roving career rather than a metaphor.
## Key takeaways - [00:15](https://www.youtube.com/watch?v=d6sVWEu9HWU&t=15s) The mod integrates Inworld AI's character platform with Skyrim, allowing players to type freeform queries to NPCs rather than selecting static dialogue options. - [00:15](https://www.youtube.com/watch?v=d6sVWEu9HWU&t=15s) Spoken character dialogue is generated through ElevenLabs speech synthesis and synced with custom subtitle displays rather than standard game subtitles. - Handcrafted base-game quests and native dialogue options remain fully intact, with AI dialogue serving as an immersion layer rather than a replacement. - [00:15](https://www.youtube.com/watch?v=d6sVWEu9HWU&t=15s) Lydia demonstrates situational awareness of nearby game entities, describing her camp by the Khajiit caravan and suspecting them of stealing her mead. - [03:54](https://www.youtube.com/watch?v=d6sVWEu9HWU&t=234s) Jon Battle-Born organically reveals his clandestine relationship with Olfina Gray-Mane, citing clan rivalries as the reason for secrecy while providing directions to Warmaiden's. - [08:07](https://www.youtube.com/watch?v=d6sVWEu9HWU&t=487s) Ysolda defends her covert skooma trade through a free-will philosophical stance, arguing adults should make their own choices while maintaining her business ethics. - [11:31](https://www.youtube.com/watch?v=d6sVWEu9HWU&t=691s) Carlotta Valentia contextualizes her harassment from the bard Mikael, asking the player to intervene and persuade him peacefully without resorting to violence. - [14:26](https://www.youtube.com/watch?v=d6sVWEu9HWU&t=866s) Whiterun guard Rolf gives an extensive backstory to taking an arrow in the knee, detailing an ambush by bandits across Tamriel that forced his transition to city defense. - [18:00](https://www.youtube.com/watch?v=d6sVWEu9HWU&t=1080s) NPCs dynamically react to the player's identity, with guard Rolf expressing reverence and poetic praise upon discovering the player is the Dragonborn. - [21:19](https://www.youtube.com/watch?v=d6sVWEu9HWU&t=1279s) Irileth reaffirms that her unwavering loyalty belongs to Jarl Balgruuf and Whiterun over racial or cultural ties, referencing their shared combat history in the Great War. - [24:06](https://www.youtube.com/watch?v=d6sVWEu9HWU&t=1446s) Jarl Balgruuf articulates a pragmatic, neutral stance on the civil war, refusing to let past Imperial Legion service dictate decisions that could harm Whiterun's citizens. - The technical setup supports speech-to-text input and alternative local TTS engines like xVASynth, though current AI interactions do not directly trigger game script events like starting quests.
## Detailed walkthrough ### [00:15](https://www.youtube.com/watch?v=d6sVWEu9HWU&t=15s) Meeting Lydia Outside Whiterun
The demonstration begins outside Whiterun, showcasing Inworld AI integrated into *The Elder Scrolls V: Skyrim* using custom dialogue prompts and ElevenLabs speech synthesis. Approaching his follower Lydia outside the gates, the player inquires about what she has been doing while waiting for him.
```mermaid flowchart TD accTitle: Player and Inworld AI Interaction Loop accDescr: Diagram showing the flow between the player input, Skyrim mod interface, Inworld AI engine, and ElevenLabs voice output. subgraph GameClient["Skyrim Game Client"] A["Player Approaches NPC & Presses 'Y'"] --> B["Custom UI Text Input"] E["Display Subtitles & Trigger Voice"] --> F["NPC In-Game Action / Idle"] end subgraph AIBackend["External AI Backend"] B --> C["Inworld AI Story Engine<br/>(Context & Personality)"] C --> D["ElevenLabs TTS System<br/>(Voice Generation)"] D --> E end ```
Lydia explains that while camped near the Khajiit caravan, she ran into a minor issue. When asked if everything went smoothly, she reveals her suspicion: “No my Thane, I camped close to the Khajiit camp. I suspect they may have stolen my mead, but I have no proof. Is there anything else you need from me?” When the player expresses concern and offers assistance, she politely declines, preferring self-reliance: “Thank you for your concern my Thane, I can handle the situation myself. I will be more careful around the Khajiit in the future.” When ordered to wait near the city gates while the player enters Whiterun, Lydia loyally assents, reminding the player not to hesitate if danger arises.
[03:54](https://www.youtube.com/watch?v=d6sVWEu9HWU&t=234s) A Secret Romance: Jon Battle-Born
Inside Whiterun, the player encounters Jon Battle-Born strolling the streets. When greeted, Jon introduces himself with his usual swagger: “Well, it's a pleasure to make your acquaintance. Names Jon Battle-Born. I'm a mercenary, bard, an all-around decent chap. So, how can I help?”
The player presses Jon about his well-known family, the Battle-Born clan, and asks what he is currently up to. Jon explains that while his family is indeed wealthy and prominent, he prefers the independent lifestyle of a wandering minstrel and sellsword. Pressed further about why he is lingering around the plaza, Jon lets slip that he is waiting for someone special. When probed about who she is, he confesses: “Her name is Olfina Gray-Mane, but like I said, it's a secret.” Explaining why their romance must remain hidden, he notes: “It's a secret because, well, it's complicated. Let's just say that our families don't exactly see eye to eye on certain matters. But that won't stop me from being her friend and supporting her when she needs it.”
Before parting ways, the player asks Jon for recommendations on purchasing gear. Jon points the player directly toward Warmaiden's: - **Location**: Warmaiden's Smithy is located right by the main entrance gates of Whiterun. - **Proprietors**: Run by Adrianne Avenicci, a skilled blacksmith, and her husband Ulfberth War-Bear, a sturdy warrior. - **Services**: Selling handcrafted weapons and armor, as well as buying used adventurer gear.
[08:07](https://www.youtube.com/watch?v=d6sVWEu9HWU&t=487s) Ysolda's Skooma Trade and Free-Will Philosophy
Moving through the Whiterun market stalls, the player initiates a conversation with Ysolda regarding her mercantile ambitions and rumors of her trade with Khajiit caravans. Ysolda openly confirms that she purchases exotic goods, including spices, moon sugar, and skooma.
When the player challenges her on the morality and legality of trading illicit substances, Ysolda presents a nuanced libertarian defense of personal liberty: “Yes, skooma is illegal in Skyrim. However, I don't think that makes it right to completely ban it. While I don't personally approve of the use of skooma, I believe that adults should have the freedom to make their own choices, even if those choices are harmful to them.”
The player pushes back, asking how she can justify profiting off an addictive narcotic. Ysolda responds that skooma is merely a sideline: “I understand your concerns, but skooma is not my main source of income. I trade in a variety of goods and services, and I only sell skooma to those who are already addicted and need it to function. My priority is to provide for myself and my community while also being responsible and ethical in my business practices.” When the player notes that skooma is legal in Elsweyr, Ysolda acknowledges the regional cultural differences while reaffirming the real danger of addiction, thanking the player for engaging in an honest, thoughtful discussion about ethics.
[11:31](https://www.youtube.com/watch?v=d6sVWEu9HWU&t=691s) Carlotta Valentia's Troubles with Mikael
At the vegetable stalls, the player speaks with Carlotta Valentia, who is visibly exasperated. When asked why she looks distressed, she admits she has been dealing with constant harassment from Mikael, the resident bard at The Bannered Mare. She describes Mikael as an insufferable flirt who refuses to take no for an answer, ruining her peace with his incessant singing and unwanted advances.
The player asks where to purchase mead and where Mikael can be found. Carlotta suggests visiting The Bannered Mare near Arcadia's Cauldron in the center of Whiterun, or speaking to Ysolda. She makes an earnest plea to the player: “And as for Mikael, if you could talk to him and convince him to stop bothering me, I would be grateful. But please don't do anything violent or dangerous. I just want him to leave me alone.” The player promises to handle the bard peacefully, and Carlotta expresses sincere gratitude before returning to tend her market stall.
[14:26](https://www.youtube.com/watch?v=d6sVWEu9HWU&t=866s) The Lore Behind the Knee: Whiterun Guard Rolf
Proceeding toward the upper districts, the player stops a Whiterun guard named Rolf. When asked his identity, Rolf introduces himself proudly: “My name is Rolf, and I serve the honorable duty of protecting Whiterun as a guard. Like a shield that defends against the piercing arrows of danger, I stand vigilant against all threats to our fair city.”
The player brings up Skyrim's most infamous line, asking Rolf if he really took an arrow in the knee. Rolf launches into a vivid recounting of his past: - He was once a roaming adventurer seeking treasure across Tamriel, from the snowy crags of High Hrothgar to the arid wastes of Hammerfell. - Near Whiterun, he was ambushed by a band of bandits and hopelessly outnumbered. - During the struggle, an archer shot him directly through the knee, shattering his mobility and ending his career as an agile adventurer. - Whiterun guards drove off the bandits, carried him to safety within the city, and tended his wounds.
Reflecting on his life shift, Rolf explains that although his days of jumping and running are finished, guard duty provides equal fulfillment: “While I do miss the thrill of adventure, I find that being a guard is a different kind of adventure, one where the stakes are just as high and the rewards just as fulfilling.” When the player reveals that they are the legendary Dragonborn, Rolf is awestruck: “By the Nine, I stand in the presence of a legend! The Dragonborn, possessor of the Voice and slayer of dragons.” He showers the player with elaborate poetic metaphors, wishing them luck on their ascent to Dragonsreach to treat with the Jarl.
``` Rolf's Adventuring & Guard Journey: [Adventurer across Tamriel] ➔ [Ambushed by Bandits] ➔ [Shot in the Knee] ➔ [Rescued by Whiterun Guards] ➔ [Sworn Whiterun City Guard] ```
[21:19](https://www.youtube.com/watch?v=d6sVWEu9HWU&t=1279s) Irileth's Fierce Loyalty to Balgruuf
Entering Dragonsreach, the player is confronted by Irileth, the Jarl's Dunmer housecarl. When asked about her heritage and how she fits into Skyrim's insular society, Irileth makes her priorities crystal clear: “Yes, I am a Dunmer and proud of it. My loyalty is to Jarl Balgruuf and Whiterun, not to any race or culture. And I will protect them with everything I have, regardless of who or what threatens them.”
When asked about those who mistrust Dark Elves, she dismisses them flatly: “Let them disagree. My actions speak louder than their words, and I will continue to protect Whiterun and its people no matter what anyone thinks.” She explains that her bond with Balgruuf was forged during their youth fighting side by side on the battlefields of the Great War against the Aldmeri Dominion. She considers Balgruuf both a great leader and an irreplaceable friend, warning the player sternly that she will be watching every move they make inside the keep.
[24:06](https://www.youtube.com/watch?v=d6sVWEu9HWU&t=1446s) Jarl Balgruuf the Greater on the Civil War
The player approaches the throne to speak directly with Jarl Balgruuf the Greater, questioning him regarding his stance on the simmering civil war between the Imperial Legion and Ulfric Stormcloak's rebellion. Balgruuf rejects being forced into an ideological box, explaining: “My focus is on ruling justly and protecting my people than in the conflict between the Imperials and the Stormcloaks.”
Confirming that he served in the Imperial Legion prior to donning the mantle of Jarl, Balgruuf maintains that past military service will not dictate his policy: “My allegiance lies solely with the citizens of Whiterun. If the war forces me to choose a side, I will do what is best for the people of Whiterun.” He welcomes the player to stay within the city walls as long as peace is respected.
[25:40](https://www.youtube.com/watch?v=d6sVWEu9HWU&t=1540s) A Whiterun Guard's Farewell and Call to Action
The demonstration concludes with a parting farewell wishing the player safety on their travels across Skyrim. Following a brief pause, the iconic Whiterun guard steps in to deliver a humorous, in-character outro that riffs on the game's most enduring meme, addressing the audience directly while staying firmly rooted in the lore and tone of Tamriel.
Adopting the familiar gravelly persona, the guard invokes the pantheon to deliver a custom call to action, playfully claiming that his days of casually browsing videos came to an abrupt halt: “By the Nine Divines, I used to be a YouTube watcher like you, but then I took an arrow in the knee.” Urging the viewer onward as a fellow traveler, he encourages everyone to subscribe to Bloc's channel for future content and directs those seeking legendary status to support the project financially, declaring: “If you want to join the heroes in the halls of Sovngarde, join his Patreon.” He wraps up the showcase with a traditional Nord benediction, wishing that the blessings of Talos guide the player on their continuing path.
- **Meme Homage:** The guard adapts the famous Skyrim line ("I used to be an adventurer like you, then I took an arrow in the knee") into a meta-joke about being a former YouTube watcher. - **Lore-Friendly Call to Action:** Viewers are invited to subscribe to Bloc's channel and join his Patreon, framed humorously as taking a place alongside the honored dead in the halls of Sovngarde under the guidance of Talos.
## Notable quotes > [08:07](https://www.youtube.com/watch?v=d6sVWEu9HWU&t=487s) While I don't personally approve of the use of skooma, I believe that adults should have the freedom to make their own choices, even if those choices are harmful to them. — Ysolda
> [21:19](https://www.youtube.com/watch?v=d6sVWEu9HWU&t=1279s) My loyalty is to Jarl Balgruuf and Whiterun, not to any race or culture. And I will protect them with everything I have, regardless of who or what threatens them. — Irileth
> [24:06](https://www.youtube.com/watch?v=d6sVWEu9HWU&t=1446s) My allegiance lies solely with the citizens of Whiterun. If the war forces me to choose a side, I will do what is best for the people of Whiterun. — Jarl Balgruuf the Greater
> [14:26](https://www.youtube.com/watch?v=d6sVWEu9HWU&t=866s) While I do miss the thrill of adventure, I find that being a guard is a different kind of adventure, one where the stakes are just as high and the rewards just as fulfilling. — Rolf
## Mentioned Inworld AI, The Elder Scrolls V: Skyrim, ElevenLabs, GPT-4, OpenAI, xVASynth, Bethesda, Bloc, Lydia, Khajiit, Jon Battle-Born, Battle-Born clan, Olfina Gray-Mane, Gray-Mane clan, Warmaiden's, Adrianne Avenicci, Ulfberth War-Bear, Ysolda, Elsweyr, Carlotta Valentia, Mikael, The Bannered Mare, Arcadia's Cauldron, Rolf, High Hrothgar, Hammerfell, Dragonborn, Dragonsreach, Irileth, Jarl Balgruuf the Greater, Aldmeri Dominion, Imperial Legion, Ulfric Stormcloak, Stormcloaks, Sovngarde, Talos, Patreon, Nexus Mods, GitHub, uiExtensions, iWidgets
## Video description The following video demonstrates the use of Inworld AI to power NPCs in Skyrim with a custom-built story engine for dialogue interactions.
Download the mod from here: https://www.nexusmods.com/skyrimspecialedition/mods/98481
With Inworld's powerful AI character platform, you can now chat with NPCs in Skyrim on your own terms. Get almost instant responses that stay true to each character's unique personality, and immerse yourself even further in the game's universe. This is the future of roleplaying games.
Watch the video of me wandering around the Whiterun, talking with Lydia about what she does while she is waiting, or asking secret love of Jon Battle-Born, or finding out facts about skooma or learning the actual personality and backstory of our glorious memeguard, "I used to be an adventurer like you, then I took an arrow in the knee"!
Curious about how it works? Check out the source code! I'm not exactly a seasoned Skyrim modder so, I'm leaving the source code for people who know what they are doing. But in a nutshell, the mod creates a connection between Inworld's system and Skyrim, allowing you to approach NPCs and start typing your message. The NPCs are constructed with their own predefined story engine, so you can rest assured that each character will stay true to their original personality.
Some may worry that this AI conversation system will replace the handcrafted characters and quests that make Skyrim so special. But fear not! The current dialogue options in the game are still there, and the countless hours put in by the Bethesda team to craft these personalities are not ignored. The AI conversation system simply adds an extra layer of immersion to keep you engaged even further. Currently, it allows player (you) to approach NPCs and press Y and start typing the message, instead of using E key to interact. When you get the answer from NPC, you can either hear it's voice or read the text (special subtitle - not game's subtitle)
Patreon: https://www.patreon.com/bloctheworker Source code: https://github.com/BlocTheWorker/Inworld-Skyrim-Mod About Inworld: https://inworld.ai/?utm_campaign=bloctheworker-skyrim&utm_medium=referral&utm_source=youtube
Notes: Sorry for occasional typos and agonizing DOF :) Voices are created with ElevenLabs' great system and not part of Inworld. Adding ElevenLabs integration to mod could add extra delays, which wouldn't be fair to Inworld. But given that mod is getting text, any Text-to-speech system can be utilized in this mod. Yes, technically you can even use xVASynth.
FAQ: Can Speech to Text be used instead of typing? Yes, that's possible. Can we have run-time TTS? Yes, you can. Although, for ElevenLabs, you do need quite a bit work to setup. Also you will need some "credits" Can this have effect on gameplay directly (like starting quests etc)? Not at the moment. Can we use Microphone and just talk to them? Yes! Even though in the video I didn't show anything like that, Inworld does come with Speech to Text model. So you don't even need to install anything extra in normal circumstances. Is this using ChatGPT? In a way, yes. Inworld is working with OpenAI under the hood. Arrow in the knee was slang for getting married!!: No it is not. See: https://elderscrolls.bethesda.net/en/article/6KNyeH7wG4FOFIEev8VaX1/how-an-arrow-in-the-knee-made-its-way-to-skyrim "Pagliarulo confirms that the line is literal – “an arrow in the knee” isn’t Skyrim slang for going down on one knee to propose and start a family, for example."
About typos/grammar: On some parts, I intentionally asked ambiguous questions with bad grammar, but some other minor typos were related to me trying to type as fast as I can to avoid the video taking more than ~25 mins with me thinking of a creative improvised question.
Honorable mentions: mrowrpurr - her great tutorials and discord helped me a lot in my journey to Skyrim modding. uiExtensions and iWidgets mods - eased the pain with Skyrim UI modding (it's nightmare) and Inworld team for being cool with this crazy idea!
#skyrim #arrowintheknee #skyrimmods #chatgpt #gpt #gpt4 #gptai #ai #gaming #skyrimchatgpt skyrim chatgpt
[Source](https://www.youtube.com/watch?v=d6sVWEu9HWU)