Microsoft Patents Display Scanning to Gate Game Access Behind Ads

Gamers Nexus

Microsoft’s proposed credit system uses continuous screen and audio surveillance to meter software access, creating an operating-system-level ad engine that platforms like Steam and Linux cannot easily block.

AI-assisted summary of Microsoft’s Terrifying New Patent | LG-Style ACR for Games & Software — original source. Check the original for full context.

Source published: 2026-09-24

Key ideas

Microsoft filed a patent application outlining an ad-funded credit model to temporarily 'rent' software and games, featuring dynamic rate-limiting that increases credit costs during peak weekend gaming hours. (0:01)
Operating similarly to smart TV Automatic Content Recognition (ACR), the system scans screens up to every 250 milliseconds and monitors voice chat audio to detect contextual ad injection opportunities. (2:42)
The architecture introduces a Content Management System (CMS) that intercepts user inputs before they reach the game engine, enabling real-time gaze tracking, haptic delivery, and dynamic product placement. (12:26)
The patent outlines fallback connectivity through Bluetooth and Wi-Fi Direct, allowing systems to bridge ad-hoc connections and bypass users attempting to evade surveillance by disconnecting their internet. (17:24)

Detailed notes

## Background Published on YouTube by Gamers Nexus on September 24, 2026, this investigative video examines a patent filed by Microsoft that proposes an ad-funded credit rental system for games and software. According to the video description, the system functions similarly to Automatic Content Recognition (ACR) on smart TVs by scanning screen displays and audio feeds to dynamically insert advertisements during gameplay pauses, loading screens, and menus across OS, application, and device tiers.

## TL;DR Host Steve Burke analyzes a comprehensive 23-page patent application from Microsoft that outlines continuous audiovisual monitoring and machine learning to scan user screens and audio feeds every 250 milliseconds. The patent proposes an ad-supported credit model where software access is rationed into timed intervals, docking higher credit costs for multiplayer modes and surge hours while forcing commercial pauses into natural gameplay downtime like loading screens. Gamers Nexus breaks down the architectural diagrams, highlighting how the system intercepts physical inputs and can bypass offline configurations using short-range wireless protocols like Bluetooth. Burke evaluates the broader risks to non-gaming productivity applications and explores whether platform barriers like Steam's advertising guidelines or migrating to Linux can truly shield users from operating-system and hardware-level telemetry.

## Atmosphere & style The presentation carries Gamers Nexus's signature investigative gravity, pairing rigorous, dry technical analysis with biting cynicism and deadpan dark humor. Steve Burke delivers the breakdown with measured indignation, methodically citing patent diagrams and legal terminology before puncturing corporate euphemisms with comparisons to dystopian media like Ready Player One and Black Mirror. The atmosphere feels analytical yet unsettlingly urgent, treating the corporate filing not merely as speculative trivia, but as a chilling blueprint for normalizing pervasive surveillance and ad-monetized time-gating across consumer technology.

## Bulletin - [00:01](https://www.youtube.com/watch?v=Smwo493IAfw&t=1s) Microsoft filed a patent application outlining an ad-funded credit model to temporarily 'rent' software and games, featuring dynamic rate-limiting that increases credit costs during peak weekend gaming hours. - [02:42](https://www.youtube.com/watch?v=Smwo493IAfw&t=162s) Operating similarly to smart TV Automatic Content Recognition (ACR), the system scans screens up to every 250 milliseconds and monitors voice chat audio to detect contextual ad injection opportunities. - [12:26](https://www.youtube.com/watch?v=Smwo493IAfw&t=746s) The architecture introduces a Content Management System (CMS) that intercepts user inputs before they reach the game engine, enabling real-time gaze tracking, haptic delivery, and dynamic product placement. - [17:24](https://www.youtube.com/watch?v=Smwo493IAfw&t=1044s) The patent outlines fallback connectivity through Bluetooth and Wi-Fi Direct, allowing systems to bridge ad-hoc connections and bypass users attempting to evade surveillance by disconnecting their internet.

## Key takeaways - [00:01](https://www.youtube.com/watch?v=Smwo493IAfw&t=1s) Microsoft's patent proposes replacing traditional perpetual or digital game licenses with an ad-gated credit model where users 'rent' software in time-limited increments. - [00:01](https://www.youtube.com/watch?v=Smwo493IAfw&t=1s) Credit consumption is stratified and rate-limited, requiring more credits for multiplayer than single-player modes and cutting playable time in half during weekend evenings compared to weekday afternoons. - [02:42](https://www.youtube.com/watch?v=Smwo493IAfw&t=162s) The patent relies on continuous screen scanning—as frequently as every 250 ms (15 frames at 60 FPS)—using machine learning models to identify scene transitions, procedural assets, and pause states. - [02:42](https://www.youtube.com/watch?v=Smwo493IAfw&t=162s) Surveillance extends beyond visual rendering into incoming and outgoing audio, specifically parsing player commentary and voice chat to evaluate context. - [05:40](https://www.youtube.com/watch?v=Smwo493IAfw&t=340s) Migrating to Linux cannot fully protect users because the architecture is designed to operate at the application, runtime, engine, server, or hardware peripheral level. - [10:43](https://www.youtube.com/watch?v=Smwo493IAfw&t=643s) The framework utilizes an Event Detection Module powered by machine learning to identify moments like loading screens, NPC boss defeats, and save menus to inject promotional content. - [11:35](https://www.youtube.com/watch?v=Smwo493IAfw&t=695s) By relying on the broad legal classification of 'interactive software applications', the technology could theoretically be applied to productivity software like accounting tools or photo editing suites. - [12:26](https://www.youtube.com/watch?v=Smwo493IAfw&t=746s) A Content Management System is architected to sit directly between input hardware and the application, intercepting controllers, keyboards, and VR gaze trackers before inputs reach the software. - [12:26](https://www.youtube.com/watch?v=Smwo493IAfw&t=746s) The system can leverage telemetry and user hardware configurations to dynamically serve tailored advertisements and adjust in-game product placement based on perceived socioeconomic status. - [15:15](https://www.youtube.com/watch?v=Smwo493IAfw&t=915s) Figure 12 of the patent outlines granting software access in small 15-to-30-minute slices earned by enduring video commercial interruptions, which Microsoft frames as 'improving' user experience. - [17:24](https://www.youtube.com/watch?v=Smwo493IAfw&t=1044s) The filing incorporates Bluetooth and Wi-Fi Direct protocols to communicate with proximate devices, preventing users from evading telemetry simply by unplugging Ethernet or turning off Wi-Fi. - [23:36](https://www.youtube.com/watch?v=Smwo493IAfw&t=1416s) Valve's Steam distribution policy strictly prohibits games that gate gameplay behind mandatory advertisement viewing, representing a major hurdle for game-level deployment on PC. - [23:36](https://www.youtube.com/watch?v=Smwo493IAfw&t=1416s) A potential loophole exists if Microsoft deploys the screen-scanning and overlay mechanics at the Windows operating system level rather than packaging it inside game binaries distributed on Steam. - [21:08](https://www.youtube.com/watch?v=Smwo493IAfw&t=1268s) While patents often serve defensive legal purposes rather than guaranteed product roadmaps, public pushback remains critical to prevent tech companies from normalizing invasive monetization models.

## Detailed walkthrough ### [00:01](https://www.youtube.com/watch?v=Smwo493IAfw&t=1s) Microsoft's Surveillance Advertising Patent

Microsoft has filed a patent for a surveillance advertising system that actively scans display outputs and audio devices, functioning much like the Automatic Content Recognition (ACR) technologies embedded inside LG, Vizio, and Samsung smart televisions. Rather than merely running basic in-game banner ads, the technology is designed to constantly monitor what is appearing on a monitor and what is playing through speakers. The patent explicitly states that its mechanisms can listen in on conversations to identify opportune moments to display promotional content during software use—specifically targeting video games. Suggested injection moments include loading screens, dialogue popups, and save menus.

```mermaid flowchart TD accTitle: Microsoft Ad-Credit Patent Architecture accDescr: Diagram illustrating surveillance inputs leading to credit deduction or ad insertion for game access.

subgraph Monitoring["Continuous Monitoring"] SCR["Screen Scans (Every 250 ms / 15 frames)"] AUD["Audio Analysis (In-game sound & voice chat)"] end

subgraph Evaluation["Context Engine"] ID["Event & Scene Detection (Trees, menus, dialogue, pauses)"] CR["Credit Valuation Engine (Usage mode & time-of-day multipliers)"] end

subgraph Enforcement["Gated Access"] ACC["Software Access (15-30 min intervals)"] ADS["Ad Injections (Loading screens, save menus)"] end

SCR --> ID AUD --> ID ID --> CR CR --> ACC CR --> ADS ```

The system ties interactive software access to an ad-funded credit model, describing a dystopian arrangement where users willingly subject themselves to advertisements to earn temporary access to applications. Microsoft repeatedly uses the term "rent" in the documentation:

> "The credit allows the user to rent, decrypt, or gain access to the games and applications sometimes for predetermined durations, such as 30 minutes of usage of a video game software application."

The patent outlines heavily stratified credit costs depending on the intensity or tier of the activity. A single credit might grant an hour of practice or tutorial play, but degrade to only 30 minutes of a single-player campaign, or just 20 minutes for multiplayer access. Microsoft also includes rate limiting based on peak periods:

> "One credit provides 1 hour of usage during a weekday afternoon, while the same credit provides 30 minutes of usage on a weekend evening."

This model effectively doubles the cost of playing during the exact hours working people have free time. The premise replaces the traditional model of buying physical discs or digital game licenses with paying for screen time using eyeball surveillance.

- **Explicit access mechanics:** Credits can "rent, decrypt, or gain access" to executables. - **Credit tiers:** Practice mode (60 minutes) vs. Single-player campaign (30 minutes) vs. Multiplayer (20 minutes). - **Surge-style rate limiting:** Halves the duration per credit during weekend evenings relative to weekday afternoons.

---

[02:42](https://www.youtube.com/watch?v=Smwo493IAfw&t=162s) Scanning Your Screen with AI and Voice Chat Listening

The patent specifies that the host system can scan the screen for visual objects and events at an interval as rapid as once every 250 milliseconds—roughly once every 15 frames at 60 FPS. The system employs machine learning to parse live frame data, explicitly highlighting the detection of complex procedural assets, such as procedurally generated trees, to verify user context. In tandem with visual scraping, the host system parses outgoing and incoming audio to detect natural gaps where promotional material can be seamlessly injected.

Ad triggers encompass recognized "events, textures, images, characters, sounds, or inputs." Watching ads awards credits that delay the system from pausing, rate-limiting, or interrupting software execution. These credit allowances can run on simple timers (such as 15 minutes of uninterrupted use) or trigger milestones:

- Surviving or finishing five rounds of gameplay - Beating a single game level - Finishing three online multiplayer matches - Defeating a designated non-player character (NPC) boss - Finalizing a project or workflow task inside a non-gaming interactive application

The scope of compatible hardware reaches beyond Windows PCs. The documentation explicitly targets laptops, desktops, dedicated consoles, set-top boxes, smartphones, VR head-mounted displays, graphics cards, peripheral controllers, and smart TVs.

The patent's audio scope extends into private voice channels:

> "In some embodiment, the audio information includes player or chat commentary from the recording of the video information and audio information... allowing identification of discussion or comments about the game environment."

This broad wording indicates that microphone feeds and party voice communications could be continuously analyzed under the guise of detecting player engagement.

---

[05:40](https://www.youtube.com/watch?v=Smwo493IAfw&t=340s) Why Linux Won't Completely Protect You

While switching an operating system to Linux strips telemetry and operating-system-level monitoring away from Microsoft on a standard desktop, it does not completely circumvent this patent. Because the architecture is built to function at the application, runtime, engine, or device level, any game or cross-platform tool utilizing this framework can execute the exact same tracking and credit gating on non-Windows platforms. Furthermore, hardware-level deployments inside connected peripherals or smart TVs bypass the host OS entirely.

Gamers Nexus pauses to highlight support from their community through their store, which funded independent investigative hardware testing:

- **Campaign:** Self-sponsored Data Dragnet campaign on `store.gamersnexus.net`. - **Merch and backer items:** A foil cyberpunk skeleton t-shirt with a biomechanical rib cage and spine motif; physical Blu-ray sets of the earlier LG data-collection investigation packaged in red metallic 3D-printed cases with companion USB drives; and an end-of-year Disappointment Tour credit listing. - **Investigative funding:** The proceeds are channeled into acquiring fleets of consumer smart TVs from Samsung, LG, and other vendors to monitor their network traffic and ACR surveillance in real time.

---

[07:21](https://www.youtube.com/watch?v=Smwo493IAfw&t=441s) The Patent's Stated Goals and Contextual Triggers

The text of the 23-page filing summarizes its goal as delivering "contextually aware promoted content in interactive software applications." It contrasts games against traditional media, explaining that streaming audio/video formats (like broadcast television) have predictable points where ad interruptions do not harm the experience. In contrast, interactive media presents unpredictable pacing, making it difficult to find non-disruptive moments to forcefully pause the experience.

Microsoft's proposed solution is a persistent background ACR engine that constantly surveys audio-video outputs to detect pauses, save actions, and scene transitions in real time. While hardcoded developer triggers for ad rollouts already exist across mobile software, this patent formalizes a continuous, OS- and hardware-level spy engine that intercepts and watches user activities across two-column dense legal documentation, applying television-style surveillance to consumer software.

[09:20](https://www.youtube.com/watch?v=Smwo493IAfw&t=560s) Why Switching to Linux Won't Protect You

The scope of Microsoft's patent application is remarkably broad, spanning consoles, TVs, set-top boxes, laptops, and various other hardware categories. The architecture is engineered to run server-side, entirely locally, or in hybrid configurations. Furthermore, connectivity is not strictly limited to Wi-Fi or Ethernet; the filing explicitly mentions short-range protocols such as Bluetooth to transmit data and reach devices that users might attempt to keep offline.

While Gamers Nexus strongly advocates switching to Linux to strip Microsoft of market leverage and limit its power, Steve cautions that migrating to an alternative operating system does not neutralize this threat. Because the technology can be embedded directly within individual applications, games, and server backends, the open nature of Linux means developers and publishers could integrate this ad framework into Linux-native or Proton-run software just as easily. Microsoft also holds vast gaming properties—including Activision Blizzard, Mojang, Obsidian Entertainment, Bethesda, and franchises like *The Elder Scrolls*. Steve jokes that perhaps *Starfield* has finally discovered its ultimate use case: transforming its endless parade of loading screens into monetizable "advertising inventory."

- Supported device endpoints: Desktop PCs, laptops, consoles, set-top boxes, smart TVs, smartphones, tablets, hybrid computers, and head-mounted displays (HMDs / VR headsets). - Transport protocols: Standard Ethernet, wireless LAN (Wi-Fi), Wi-Fi Direct, and Bluetooth. - Software integration level: Capable of running OS-wide, local-to-app, or entirely remote/server-side.

[10:43](https://www.youtube.com/watch?v=Smwo493IAfw&t=643s) Architecture: The Machine Learning Event Detection Module

Diving into the patent figures, Figure 1 illustrates how interactive software applications feed into a Content Management System (CMS) alongside promotional advertising material routed from a remote server. This information flows into an Event Detection Module, which the patent notes can reside "local to or remote from the content management system." This module functions by analyzing real-time video feeds, audio data, game state information, and API/event "hints" via a machine learning model to detect monetization opportunities.

```mermaid flowchart TD accTitle: Patent Content Delivery Architecture accDescr: Diagram showing how user input, game state, and remote ad servers interact through the CMS and Event Detection Module.

subgraph Inputs ["Input & Application Layer"] UI["User Input Devices (Keyboard, Mouse, Gamepad, Eye-Tracker, VR)"] APP["Interactive Software Application (Games, Accounting Tools, Desktop Software)"] end

subgraph Intermediation ["Surveillance & Ingestion Engine"] CMS["Content Management System (CMS) (Intercepts inputs & halts gameplay)"] EDM["Event Detection Module (Machine Learning / Language Models)"] end

subgraph RemoteServer ["Ad Distribution Network"] ADS["Remote Ad Server (Promotional Content Repository)"] end

UI --> CMS CMS --> APP APP --> EDM APP --> CMS ADS --> CMS EDM -->|"API hints & event detection"| CMS ```

Although the patent lists various classical definitions of machine learning—even referencing language models as a possible mechanism—Steve emphasizes that in contemporary industry parlance, this simply translates to buzzword "AI." Rather than waiting for a developer to manually program an ad break, an AI model evaluates what is happening on screen and in audio to trigger targeted commercial interruptions dynamically.

[11:35](https://www.youtube.com/watch?v=Smwo493IAfw&t=695s) Turning Productivity and Gaming into "Interactive Software Applications"

Figure 2 and Figure 3 outline the actual execution loop. Microsoft relies on the sterile legal phrase "interactive software application," which essentially means any program. While games are the primary focus, Steve dryly points out that the patent's broad definition covers productivity suites such as QuickBooks. Under this paradigm, an accountant completing a general journal entry could be interrupted by a mandatory commercial break before the software resumes.

Figure 3 explicitly outlines this workflow: the system retrieves a promotional content trigger from the application, forcibly suspends the user's active interaction, delivers an audiovisual commercial, and only then resumes software execution while depositing a time-limited "credit" into the user's account balance.

[12:26](https://www.youtube.com/watch?v=Smwo493IAfw&t=746s) The CMS Sits Between Your Hands and the Screen

Figure 7 reveals an alarming architectural detail: user input devices are mapped to interface with the Content Management System *before* those inputs ever reach the game or application. The CMS intercepts and interprets everything the user does, positioning itself as a middleman between the player's physical hardware and the game engine.

Because the patent explicitly covers head-mounted displays (VR headsets), eye-tracking systems, and camera sensors, Steve highlights parallels to Automatic Content Recognition (ACR) used in smart TVs. In a virtual environment like *Cyberpunk 2077* running in VR, gaze tracking could monitor what in-game items you look at. In-game background props—such as a fictional "Brosef Lager" sitting on a bar counter—could be dynamically swapped out using dynamic product placement.

Steve jokes that if the system detects you are running a budget Nvidia GeForce RTX 3050, it might dynamically serve you a Budweiser ad, whereas if you have an ultra-enthusiast RTX 5090, it might serve high-end luxury alcohol like Johnnie Walker. Because the system can correlate your hardware profile, telemetry, and in-game behavior, it could construct tailored corporate realities and adjust pricing, product placements, and ad delivery based on perceived socioeconomic standing or political leanings.

[15:15](https://www.youtube.com/watch?v=Smwo493IAfw&t=915s) Figure 12: Paying Microtransactions for 15 Minutes of Software Time

Figure 12 synthesizes the entire monetization loop into a timed credit mechanism. Microsoft outlines a sequence where a user obtains a gameplay credit valid for a specific duration—functioning like a digital arcade token. However, rather than offering the satisfying physical sensation of dropping a quarter into an arcade cabinet or pinball machine, users must sit through intrusive video advertisements to earn a measly 15-minute slice of software access.

The system determines a "promoted content window," commences interaction, and then, during that window, halts the software based on an in-game trigger event to force another audiovisual ad onto the screen. Worse, the patent tries to frame these interruptions as a consumer benefit. The filing claims that by employing "contextually aware interruption" that pauses play at appropriate times, "the user's experience may be improved." Steve lambasts this corporate double-speak, comparing Microsoft's filing to connected-TV (CTV) advertising executives and automotive companies that slap unskippable screens across car dashboards and label them "ad experiences" that consumers supposedly want.

[17:24](https://www.youtube.com/watch?v=Smwo493IAfw&t=1044s) Bypassing Offline Protections with Bluetooth and Wi-Fi Direct

In paragraph 0031 on page 11 of the filing (viewed via Justia Patents), Microsoft addresses client-device communications. While standard Ethernet and Wi-Fi connections are enumerated, the patent explicitly details fallbacks:

> "In some embodiment, the communication device is a short-range wireless communication, such as a Bluetooth connection or a Wi-Fi direct connection that allows data communication between the client device and electronic devices in proximity to the client device."

This introduces serious privacy and bypass implications. If a user intentionally disconnects an Ethernet cable or takes a gaming PC, console, or smart TV entirely offline to evade telemetry, the system is designed to use Bluetooth or Wi-Fi Direct to bridge connections through nearby ambient smart devices. Just like ACR technology in modern smart TVs that hoards analytical data locally until any network bridge becomes available, Microsoft's proposed system could store offline interaction metrics and exfiltrate them via ad-hoc wireless handshakes the moment another connected device passes within short-range proximity.

[18:45](https://www.youtube.com/watch?v=Smwo493IAfw&t=1125s) Context-Aware Ad Breaks and Screen Surveillance

Microsoft’s patent document details methods for delivering software and promotions even when users intentionally take their machines offline or stream their software from remote servers. Specifically, the patent describes software that is "stored on and/or executed remotely from the client device, such as on a remote server or remote computing device in which the client device is in network communication." The patent describes promotional delivery methods targeting every sensory avenue available, noting ads could be pushed "via a display, audio speakers, haptic devices, etc."

At the same time, the patent displays an almost comedic level of self-awareness regarding how intrusive this is. The filing explicitly notes: "In some instances, however, constraining, interrupting, suspending user inputs to or otherwise preventing usage of the interactive software application independent of context can cause negative user experiences." The authors highlight that unpauseable multiplayer sessions or persistent single-player role-playing games would be ruined if arbitrary ads froze the user out—resulting in match losses, player deaths, or broken team communications.

```mermaid flowchart TD accTitle: Microsoft Contextual Ad Delivery Architecture accDescr: Flow diagram showing how local or remote systems analyze user context to inject ads without causing disruption.

subgraph InputAnalysis["Screen & Audio Surveillance"] A["Active Display Scan"] --> C["Content Management System (CMS)"] B["Audio & Chat Cue Monitoring"] --> C end

subgraph Processing["Context Evaluation"] C --> D{"Safe Moment Identified?\n(Loading / Lobby / Death Screen)"} end

subgraph Execution["Ad Injection Channels"] D -->|"Yes"| E["Display Video / Static Ad"] D -->|"Yes"| F["Audio Speakers"] D -->|"Yes"| G["Haptic Devices"] D -->|"No"| H["Delay Ad Break"] end ```

To circumvent this disruption, Microsoft’s proposed solution is to evaluate "context from the application, ensuring the promoted content is presented without substantially... negatively impacting the user's experience." The delivery is managed by a content management system (CMS) running natively on a PC, a console, or handled externally via remote servers or third-party devices. However, this creates a major surveillance problem: the system cannot determine contextual moments (like round ends or loading screens) without actively monitoring and scraping everything occurring on screen, alongside scanning game audio and conversational voice or text chat cues.

- The system relies on native or remote Content Management Systems (CMS) running on local hardware or third-party intermediary devices. - Ad delivery mechanisms explicitly encompass visuals, sound cues, and physical haptics. - Audio and chat surveillance remains ambiguous in the patent text, covering potential scraping of both in-game text communication and live voice audio.

---

[21:08](https://www.youtube.com/watch?v=Smwo493IAfw&t=1268s) The Threat Beyond Games: "Interactive Software Applications"

The broader implications of this patent stretch far beyond Microsoft’s first-party game catalog. While switching to Linux might seem like an escape, Microsoft owns an enormous stable of studios and wields massive leverage over hardware and software partners across the industry. Although patents are often filed simply to protect ideas or secure licensing revenue long before reaching commercial deployment, making noise now is critical. Public backlash can force companies to kick the can down the road rather than immediately normalizing intrusive technology.

Furthermore, the patent rarely limits its scope to video games, repeatedly using the broader legal phrase "interactive software application." Research into legal precedent shows that this term is intentionally flexible: - In California Civil Code 1.81.5, Section 1798.136 (CCPA), the law explicitly defines even a standard web browser as an "interactive software application." - Case law against Sony uses the term to describe console video games. - Schrödinger's end-user license agreement (EULA) defines its proprietary life sciences and material sciences simulation suites as "interactive software applications."

By maintaining an intentionally ambiguous definition, the patent could easily be licensed or applied to commercial creative suites. For instance, Microsoft and partners like Adobe could implement an ad-supported tier of Photoshop Basic for $5 a month, interrupting the user with forced video ads every time they attempt to save an image or export a file.

---

[23:36](https://www.youtube.com/watch?v=Smwo493IAfw&t=1416s) Steam’s Anti-Ad Policy and the OS-Level Loophole

Valve’s Steam distribution platform provides some potential defense against these practices. Steam's storefront documentation directly forbids ad-based monetization models: "Steam does not contain any paid advertising, nor are advertising models supported in games distributed on Steam." Valve's guidelines clarify that gating gameplay behind mandatory ad viewing or requiring players to interact with commercial ads is forbidden, and any such mechanics must be excised before a game can be distributed on Steam.

Despite Valve's stance, relying on a private digital storefront to safeguard the user experience is tenuous. A tech giant like Microsoft could easily exploit architectural loopholes. If ad injection is handled at the Windows OS level via background screen scraping—rather than through game binaries uploaded to Valve's servers—it creates an unresolved grey area:

```mermaid flowchart LR accTitle: OS-Level vs Game-Level Ad Execution accDescr: Diagram distinguishing between games packaged for Steam and OS-level display overlays.

subgraph SteamScope["Steam Distribution Boundary"] A["Game Binary on Steam"] --> B["Steam Client Execution"] end

subgraph WindowsOSScope["OS Surveillance Boundary"] C["Windows OS Screen Capture"] --> D["Context Engine"] D --> E["OS-Injected Overlay Ad"] end

B -.->|"Displayed on Screen"| C E -.->|"Covers Display"| B ```

Valve has the authority as a private platform to ban publishers attempting such workarounds, but whether they would actively police OS-level behavior remains an open question. If the ad engine is driven strictly by Windows OS scanning, running games via Steam on Linux might serve as an effective technical escape hatch.

---

[26:26](https://www.youtube.com/watch?v=Smwo493IAfw&t=1586s) Prior Art, Dystopian Tropes, and Closing Thoughts

The patent reveals no technical evidence supporting claims that it requires native game-engine integration. The language is kept deliberately broad, allowing execution at the local device level, the operating system level, or across external cloud servers.

Microsoft is far from the first company to explore absurd advertising patents. Years ago, Sony famously patented an interactive commercial system where a television program would pause mid-broadcast, forcing viewers to stand up from the couch, throw their arms in the air, and yell brand slogans like "McDonald's!" at their television before normal programming could resume. Steve jokes that the industry is rapidly approaching the reality depicted in *Ready Player One*, quoting Nolan Sorrento's plan to fill 80% of a user's visual field with advertisements right up to the point of inducing seizures.

Steve closes out the segment by directing viewers to the Gamers Nexus store for their satirical "Micro$oft" parody shirts, joking about how much users clearly love targeted, forced advertising and frustrating operating systems. He reminds viewers that merchandise and their Patreon page are the best ways to support the channel's independent reporting.

[28:13](https://www.youtube.com/watch?v=Smwo493IAfw&t=1693s) Closing Sign-off

Steve concludes the episode by thanking the audience for tuning in to the investigation. He encourages viewers to subscribe to the channel for future reporting and signs off with his standard closing: "Subscribe for more. We'll see you all next time."

## Notable quotes > [00:01](https://www.youtube.com/watch?v=Smwo493IAfw&t=1s) The credit allows the user to rent, decrypt, or gain access to the games and applications sometimes for predetermined durations, such as 30 minutes of usage of a video game software application.

> [00:01](https://www.youtube.com/watch?v=Smwo493IAfw&t=1s) One credit provides 1 hour of usage during a weekday afternoon, while the same credit provides 30 minutes of usage on a weekend evening.

> [02:42](https://www.youtube.com/watch?v=Smwo493IAfw&t=162s) In some embodiment, the audio information includes player or chat commentary from the recording of the video information and audio information... allowing identification of discussion or comments about the game environment.

> [07:21](https://www.youtube.com/watch?v=Smwo493IAfw&t=441s) contextually aware promoted content in interactive software applications.

> [10:43](https://www.youtube.com/watch?v=Smwo493IAfw&t=643s) local to or remote from the content management system.

> [15:15](https://www.youtube.com/watch?v=Smwo493IAfw&t=915s) the user's experience may be improved.

> [17:24](https://www.youtube.com/watch?v=Smwo493IAfw&t=1044s) In some embodiment, the communication device is a short-range wireless communication, such as a Bluetooth connection or a Wi-Fi direct connection that allows data communication between the client device and electronic devices in proximity to the client device.

> [18:45](https://www.youtube.com/watch?v=Smwo493IAfw&t=1125s) stored on and/or executed remotely from the client device, such as on a remote server or remote computing device in which the client device is in network communication.

> [18:45](https://www.youtube.com/watch?v=Smwo493IAfw&t=1125s) via a display, audio speakers, haptic devices, etc.

> [18:45](https://www.youtube.com/watch?v=Smwo493IAfw&t=1125s) In some instances, however, constraining, interrupting, suspending user inputs to or otherwise preventing usage of the interactive software application independent of context can cause negative user experiences.

> [23:36](https://www.youtube.com/watch?v=Smwo493IAfw&t=1416s) Steam does not contain any paid advertising, nor are advertising models supported in games distributed on Steam.

> [28:13](https://www.youtube.com/watch?v=Smwo493IAfw&t=1693s) Subscribe for more. We'll see you all next time. — Steve Burke

## Mentioned Activision Blizzard, Adobe, Bethesda, Bluetooth, Budweiser, Cyberpunk 2077, Gamers Nexus, Johnnie Walker, Justia Patents, LG, Linux, McDonald's, Microsoft, Mike Gaglione, Mojang, Nvidia GeForce RTX 3050, Nvidia GeForce RTX 5090, Nolan Sorrento, Obsidian Entertainment, Photoshop, Proton, QuickBooks, Ready Player One, Samsung, Schrödinger, Sony, Starfield, Steam, Steve Burke, The Elder Scrolls, Valve, Vizio, Windows

## Video description SUPPORT OUR WORK https://store.gamersnexus.net/data-dragnet Microsoft has filed for a new patent that describes a credit system to "rent" software and games, including rate-limited rentals based on the type of use. In their examples, singleplayer costs more than training mode, but multiplayer costs more credits than both. Users can 'earn' credits by subjecting themselves to more advertisements during loading screens, round win or loss screens, and level loading screens. The patent technology can be applied at the application level, OS level, and device level, making it wide-reaching and affecting more than just Windows machines. In one instance, Microsoft explicitly names Smart TVs. This would be similar, in some ways, to ACR (Automatic Content Recognition) on LG or other Smart TVs, where the patent-pending technology could scan the screen and audio.

The best way to support our work is through our store: https://store.gamersnexus.net/

Like our content? Please consider becoming our Patron to support us: http://www.patreon.com/gamersnexus

TIMESTAMPS

00:00 - Microsoft's Surveillance Advertising Patent 02:42 - Scanning Your Screen with AI 05:40 - LINUX ISN'T ENOUGH TO ESCAPE THIS 07:21 - The Patent's Stated Goals 10:43 - How it Works and Credits 21:08 - Our Thoughts 23:37 - Steam's Ads Policy 26:26 - Conclusion

** Please like, comment, and subscribe for more! **

Links to Amazon and Newegg are typically monetized on our channel (affiliate links) and may return a commission of sales to us from the retailer. This is unrelated to the product manufacturer. Any advertisements or sponsorships are disclosed within the video ("this video is brought to you by") and above the fold in the description. We do not ever produce paid content or "sponsored content" (meaning that the content is our idea and is not funded externally aside from whatever ad placement is in the beginning) and we do not ever charge manufacturers for coverage.

t: http://www.twitter.com/gamersnexus f: http://www.facebook.com/gamersnexus w: http://www.gamersnexus.net/

Our policies, processes, and ethics statements relating to review samples, advertising, travel, errors, and more are transparently and publicly available on this page: https://gamers.nexus/ethics-statements

Steve Burke: Host, Writing Mike Gaglione: Editing

[Source](https://www.youtube.com/watch?v=Smwo493IAfw)

中文简报

微软申请屏幕扫描专利，拟将游戏访问权限与广告挂钩微软拟议的积分系统利用持续的屏幕和音频监控来计量软件使用权限，打造出Steam和Linux等平台难以轻易屏蔽的操作系统级广告引擎。主持人Steve Burke分析了微软一份长达23页的详尽专利申请，该专利概述了利用持续的视听监控和机器学习，每250毫秒扫描一次用户的屏幕及音频流。专利提出了一种广告支持的积分模式，将软件访问权限划分为计时时段，对多人模式和高峰时段扣除更高的积分成本，同时在加载界面等游戏内的自然停顿期间强行插入商业广告。Gamers Nexus详细剖析了其架构图，强调该系统如何在物理输入到达游戏前进行拦截，并如何利用蓝牙等短距离无线协议绕过离线设置。Burke还评估了此举对非游戏生产力应用的更广泛风险，并探讨了Steam的广告准则等平台屏障或转向Linux是否真能保护用户免受操作系统与硬件级遥测的影响。微软提交了一项专利申请，概述了一种通过广告获取资金的积分模式，用于临时“租赁”软件和游戏，该模式具备动态速率限制功能，会在周末游戏高峰时段提高积分消耗成本。 (0:01) 该系统的运作方式类似于智能电视的自动内容识别（ACR），扫描屏幕的频率最高可达每250毫秒一次，并监控语音聊天音频，以检测植入上下文广告的时机。 (2:42) 该架构引入了一套内容管理系统（CMS），可在用户输入到达游戏引擎前予以拦截，从而实现实时视线追踪、触觉反馈传递以及动态产品植入。 (12:26) 该专利概述了通过蓝牙和Wi-Fi Direct实现的备用连接方式，允许系统建立点对点自组连接，从而绕过试图通过断开网络来规避监控的用户。 (17:24)

Reading summary

According to an investigation by Gamers Nexus, Microsoft has filed a patent for a surveillance-driven advertising framework that scans display outputs and audio streams to inject commercial breaks into interactive software. The proposed technology pairs real-time content recognition with a credit-based metering system that temporarily rents software access in exchange for viewing ads. Continuous Display Scraping and Voice Chat Surveillance (2:42) The patent outlines an Automatic Content Recognition (ACR) system that continuously monitors visual outputs and audio devices, functioning similarly to tracking technologies used in smart televisions from LG and Samsung. Rather than relying on static developer-placed banners, the host system scans live frame buffers at intervals as rapid as once every 250 milliseconds—roughly every 15 frames at 60 FPS. The framework uses machine learning models to detect visual state markers, such as procedurally generated assets, menus, and scene transitions. In addition to visual capture, an Event Detection Module analyzes outgoing and incoming audio feeds. The filing explicitly notes that audio scanning extends to player commentary and chat communications, allowing the system to monitor microphone channels to locate natural lulls or pauses—such as loading screens, dialogue popups, and save menus—to inject promotional content without disrupting active gameplay. Ad-Gated Execution and Surge-Priced Software Rents (0:01) The filing outlines a monetization model that replaces traditional perpetual software ownership with time-limited access credits. The patent repeatedly uses the term "rent," describing mechanisms where users submit to ad delivery to decrypt, rent, or unlock software execution for predetermined durations, such as 15- to 30-minute intervals. Execution can also be metered against in-game progression milestones, such as completing five rounds, beating a level, or defeating a designated non-player character boss. Microsoft details stratified credit valuation based on the type of activity and the time of day. Practice or tutorial modes grant longer execution periods per credit (such as 60 minutes), while single-player campaigns yield 30 minutes, and multiplayer access drops to 20 minutes. The architecture also incorporates surge-style rate limiting, where a credit that awards one hour of access on a weekday afternoon degrades to only 30 minutes during peak weekend evening hours. Hardware-Level Interception and Wireless Offline Bypasses (9:20) The scope of the architecture spans desktop PCs, consoles, mobile devices, VR head-mounted displays, and smart TVs, operating locally, OS-wide, or entirely via remote servers. Detailed system diagrams show a Content Management System (CMS) positioned directly between user input devices (including gamepads, mice, and eye trackers) and the application. This setup enables gaze tracking and dynamic product placement, allowing the system to profile hardware configurations and swap in targeted advertising assets. To prevent users from circumventing tracking by taking systems offline, paragraph 0031 of the filing specifies alternative communication protocols. The system is designed to use short-range wireless channels, specifically Bluetooth or Wi-Fi Direct, to bridge connections through nearby ambient electronic devices. Like modern television ACR software that caches telemetry locally, the engine can hoard engagement data and exfiltrate it as soon as a wireless handshake with a passing connected device becomes possible. Productivity Applications and Digital Storefront Conflicts (21:08) While gaming represents the primary scenario in the documentation, Microsoft repeatedly relies on the broader legal phrase "interactive software application." Gamers Nexus notes that legal precedent in privacy statutes and commercial end-user license agreements applies this definition to web browsers, simulation packages, and accounting tools. This opens the possibility of deploying ad-supported tiers across non-gaming commercial software, such as forcing commercial breaks during file exports in creative suites. Valve's Steam distribution policy strictly prohibits in-game ad gating and mandatory commercial interactions, but Gamers Nexus observes that an operating-system-level surveillance engine operating outside the game binary creates an unresolved loophole. Furthermore, while migrating to Linux strips away Windows OS-level telemetry, Gamers Nexus cautions that it offers incomplete protection, as developers or publishers can embed the patent's ad and credit framework directly into cross-platform game engines or remote server backends. While patent applications often function as speculative intellectual property claims that may never reach commercial deployment, Gamers Nexus argues that public scrutiny is necessary to push back against the creeping normalization of intrusive telemetry across consumer software platforms.

Source quotes

[0:01] The credit allows the user to rent, decrypt, or gain access to the games and applications sometimes for predetermined durations, such as 30 minutes of usage of a video game software application. [0:01] One credit provides 1 hour of usage during a weekday afternoon, while the same credit provides 30 minutes of usage on a weekend evening. [2:42] In some embodiment, the audio information includes player or chat commentary from the recording of the video information and audio information... allowing identification of discussion or comments about the game environment. [7:21] contextually aware promoted content in interactive software applications. [10:43] local to or remote from the content management system. [15:15] the user's experience may be improved. [17:24] In some embodiment, the communication device is a short-range wireless communication, such as a Bluetooth connection or a Wi-Fi direct connection that allows data communication between the client device and electronic devices in proximity to the client device. [18:45] stored on and/or executed remotely from the client device, such as on a remote server or remote computing device in which the client device is in network communication. [18:45] via a display, audio speakers, haptic devices, etc. [18:45] In some instances, however, constraining, interrupting, suspending user inputs to or otherwise preventing usage of the interactive software application independent of context can cause negative user experiences. [23:36] Steam does not contain any paid advertising, nor are advertising models supported in games distributed on Steam. [28:13] Steve Burke: Subscribe for more. We'll see you all next time.

Original transcript

[0:01] Microsoft has filed for a terrifying new [0:03] surveillance advertising patent that [0:05] allows them to scan your screen and [0:07] audio devices in a similar way to how [0:10] ACR for LG Vizio and Samsung TVs [0:14] functions. And this patent would enable [0:17] them to know what's on your screen and [0:19] what plays out of your speakers and [0:20] actually explicitly says that parts of [0:23] its functions would enable it to listen [0:25] to conversations that you're having so [0:28] that it can play advertisements while [0:30] using software and in particular while [0:33] playing video games. Microsoft describes [0:35] ideal times for promotional content like [0:38] ads during gaming as being things like [0:39] game loading screens, dialogue box [0:41] popups, save screens, and more. [0:43] Microsoft's patent describes a system [0:45] where users can earn credits by watching [0:47] ads that are injected into software [0:49] applications and into video games. They [0:51] even talk about in the most actually [0:53] cyberpunk dystopian way possible, how [0:56] users can willingly subject themselves [0:59] to more advertisements in order to earn [1:01] more credits to play games for 15 to 30 [1:04] minutes at a time. Quote, "The credit [1:06] allows the user to rent." This is what I [1:09] just want to be clear. They use the word [1:13] rent. [1:14] Been talking about this for a while now. [1:16] They're just saying it anyway. The [1:18] credit allows the user to rent, decrypt, [1:20] or gain access to end quote the games [1:22] and applications sometimes for quote [1:24] predetermined durations end quote such [1:26] as quote 30 minutes of usage of a video [1:28] game software application end quote. In [1:30] Microsoft's Orwellian wet dream, they [1:33] say that not all credits are created [1:35] equal. One credit may give you an hour [1:37] of tutorial or practice play. Or that [1:40] same credit could give you 30 minutes of [1:42] single player campaign play. Or because [1:44] apparently multiplayer is premium, it [1:47] could give you just 20 minutes of [1:48] multiplayer mode. They also describe [1:51] rate limiting. Quote, "One credit [1:53] provides 1 hour of usage during a [1:55] weekday afternoon, while the same credit [1:57] provides 30 minutes of usage on a [1:59] weekend evening." End quote. Considering [2:01] the weekender who's playing the games [2:03] probably just spent an entire week [2:05] exhausting themselves trying to earn [2:07] money to live, it feels particularly [2:09] predatory that you would effectively [2:12] charge them double to play games on the [2:14] weekend. And let's not lose sight of the [2:16] fact that this still talking about [2:19] paying [laughter] [2:20] for playing video games with your [2:22] eyeballs [2:25] with advertisements [2:27] when this is like something you used to [2:29] buy a disc off a shelf and that was the [2:32] end of your interaction with the company [2:35] or even buy it through a digital [2:36] download service. This is this is so [2:40] insane. The Microsoft patent says that [2:42] it can scan your screen for events and [2:45] objects in software, including but not [2:47] limited to video games, describing a [2:49] scanning rate as fast as once every 250 [2:53] milliseconds that it looks at your [2:54] screen or roughly every 15 frames if [2:57] you're at 60 fps. It describes the [2:59] process of scanning and identifying, for [3:01] example, a tree on the user screen, [3:03] including variance in trees such as [3:05] those procedurally generated in games. [3:07] and it describes using machine learning [3:09] functionality to identify objects and [3:11] things on the screen. Microsoft talks [3:13] about interpreting audio from the host [3:15] system to identify ideal gaps to play [3:17] promotional advertising content. The [3:19] software can inject ads and promos by [3:21] detecting quote events, textures, [3:23] images, characters, sounds, or inputs [3:25] end quote. And it can push promotional [3:27] ads in front of users to quote allow the [3:29] user to earn credits for the use of the [3:31] interactive software application end [3:33] quote including games. Microsoft [3:35] describes that credits earned [music] [3:36] for watching ads can delay pausing or [3:39] interrupting your use of software and [3:41] delay further ads which will otherwise [3:43] continue until the credits improve. It [3:45] says quote the credit may have a credit [3:47] duration of a nominal time such as 15 [3:50] minutes. It also says that credits may [3:51] have a time duration based on in-game [3:53] events such as quote five rounds of [3:56] gameplay, completing one level of a [3:58] video game, completing three online [3:59] multiplayer matches, defeating a [4:01] particular non-playable character and [4:03] PC, or completing a project or task [4:06] within the application. End quote. [4:07] Microsoft says that this would work on [4:09] laptops, desktops, consoles, settop [4:10] boxes, phones, VR headsets, GPUs, input [4:13] devices, gamepads, and best of all, [4:15] smart TVs. Event detection is done for [4:17] save screens, dialogue boxes, detecting [4:19] around end screens, and pause and [4:21] loading screens. And the problem with [4:22] all of this, obviously, is that [4:24] Microsoft and its advertising partners, [4:26] and the game studios that are working [4:28] with this type of technology seemingly [4:30] would have the ability to know what's on [4:32] your screen at all times, what's coming [4:34] out of your speaker at all times, and [4:36] possibly what's going into your [4:37] microphone based on what they say in the [4:39] next line. in order to identify the very [4:42] things they're talking about, they would [4:43] need to be listening or watching you at [4:46] all times. Quote, "In some embodiment, [4:48] the audio information includes player or [4:51] chat commentary from the recording of [4:53] the video information and audio [4:55] information." And it further says, [4:57] quote, "Allowing identification of [4:59] discussion or comments about the game [5:02] environment," end quote, and you know, [5:05] literally everything else that you're [5:06] talking about. Uh, so they're straight [5:08] up telling you that they are going to [5:10] spy on your conversations with your [5:13] friends when you're using software. It's [5:14] not just games. This is it talks about [5:16] interactive software applications. [5:18] Broadly, that means games, but there are [5:20] other types of applications that fall [5:22] under that umbrella. And this is at this [5:25] point a 23-page patent filing of horrors [5:28] for surveillance capitalism that seems [5:33] to set the stage for things far worse [5:35] than just ACR on TVs, which is already [5:38] really bad. And just to get ahead of it, [5:40] sadly, installing Linux doesn't save you [5:43] from this. I know that's going to be [5:44] like one of the top things. It helps. It [5:47] takes power away from Microsoft. It [5:50] probably gets you a little away from [5:51] some use cases of this, like just [5:52] Windows at a desktop level. That doesn't [5:54] necessarily save you, though. This [5:56] patent works everywhere. It's not just [6:00] Windows computers. We brought you this [6:01] video with our Kickstarter styled [6:03] fundraiser campaign on [6:04] store.gamersex.net, [6:05] and we're self-sponsored here. You all [6:07] have absolutely blown away our goal for [6:09] the data dragnet campaign. Our backer [6:12] kits include things like our foil [6:13] cyberpunk skeleton t-shirt with a UI [6:16] tech dystopian style rib cage design and [6:18] matching spine that goes all the way [6:20] down the back of the shirt. Some of the [6:21] kits also include our Blu-ray disc packs [6:23] of our LG Investigation with a custom [6:26] fully in-house 3D printed case and a [6:29] special red metallic coloring for the LG [6:31] deep dive specifically. It even includes [6:33] a USB drive just in case you need an [6:35] easier way to watch it and as sort of a [6:37] dead man switch for the DMCA. Also, [6:39] we're duplicating the discs in house and [6:41] even printing the labels on them in [6:43] house with our disc duplicator. So, this [6:46] is actually made here. We also have a [6:47] backer tier with a live streamed Q&A [6:50] session with me and a group of backers [6:51] if you want to ask questions, including [6:53] your name in the credits for that tier [6:55] of the disappointment tour video at the [6:57] end of the year. It's been incredible to [6:59] see the funding support on the data [7:00] dragnet page. And we've already started [7:02] applying that money towards buying even [7:04] more TVs to investigate alongside other [7:06] things. and not just LG. We've spent [7:09] thousands and thousands of dollars and [7:11] have TVs showing up daily right now for [7:13] the next round. Thank you sincerely for [7:15] your support as we continue to expand [7:17] our coverage scope. And now back to it. [7:19] Let's start with the goal of the patent [7:21] and their own summary. The patent is [7:22] described as being for technology that [7:24] quote is contextually aware promoted [7:26] content in interactive software [7:28] applications. End quote. And they [7:30] describe traditional advertising as [7:31] quote streaming audiovisisual content [7:33] can be supported by promoted [7:34] audiovisisual content through periodic [7:36] breaks in the streaming audiovisisual [7:37] content. End quote. So really this is [7:39] just a very wordy way to describe an ad [7:40] break in a TV show. But that's not the [7:42] thing Microsoft's trying to solve. [7:44] They're trying to solve interactive [7:46] content. Quote in non-interactive [7:48] audiovisisual content such periodic [7:50] breaks can be placed at a variety of [7:51] predictable times in the content without [7:54] adversely affecting the user experience. [7:56] End quote. I I would contest that. But [7:58] let's continue. Quote, however, [7:59] interactive software applications which [8:01] stream interactive audiovisisual content [8:04] can provide a more varied user [8:06] experience and predicting when to [8:08] suspend interaction and present promoted [8:11] audiovisisual content can be [8:13] challenging. End quote. So that's the [8:14] premise for the patent. They're trying [8:16] to figure out when they can interrupt, [8:17] that's their word, your use of software, [8:21] games or otherwise in order to shove ads [8:24] in front of you. And again that like [8:26] let's not lose sight of the big picture [8:28] here which is that this requires them [8:32] and their partners potentially their [8:34] game studios potentially to know what's [8:36] on your screen at all times to listen to [8:39] you or your friends or just the audio [8:42] output or processing of the game the [8:44] software at all times. This is a spy [8:48] patent. That is what this is. That's the [8:51] big picture. Some of this programming [8:53] functionality could use just [8:55] pre-programmed triggers in, for example, [8:57] a video game or even a software [8:59] application, meaning that it could [9:00] detect for an event and then pop up an [9:02] ad. That's functionality that already [9:04] exists. That doesn't require a new [9:05] patent. The concerning thing with this [9:07] patent is that every other aspect of the [9:09] 23 pages of two column text that they [9:11] have indicates an ACR system that [9:13] constantly surveills the screen and the [9:16] audio. And this can extend outside of [9:18] just Windows machines because its patent [9:20] applies to consoles, TVs, settop boxes, [9:23] and all kinds of other things they list [9:25] and can even run serverside for software [9:27] or games or locally and can be [9:30] transmitted over Bluetooth if wireless [9:33] internet or Ethernet is not available. [9:36] Maybe you've taken your device online, [9:37] they want to punch through to it. And [9:38] again, just to really emphasize here, a [9:40] lot of people are going to herald Linux [9:41] as a solution. And while we embrace that [9:44] and we support moving to some form of [9:45] Linux for anyone who is able to, we have [9:47] a lot of content on that. I really want [9:49] to caution that that alone doesn't stop [9:53] this from rolling out. This is not [9:55] specific to Windows and really the best [9:57] thing you get out of a move from Linux, [9:58] which we do support, is undermining [10:00] Microsoft's ability to abuse its power [10:03] because you're taking it away from them. [10:04] But that doesn't stop this. So I just [10:06] want to be clear because they list all [10:09] kinds of devices this can be installed [10:11] on. Microsoft owns a ton of game [10:13] studios, although fewer by the day. And [10:15] this can be rolled out in lots of ways [10:16] beyond just an operating system. And of [10:19] course, the nature of Linux being as [10:21] open as it is means that can [10:24] also use it. [laughter] And anyway, [10:26] Microsoft owns a bunch of studios like [10:28] Activision, Blizzard Ken, Mojang with [10:29] Microsoft, Obsidian, Bethesda, Elder [10:31] Scroll series, of course, under [10:32] Bethesda, and plenty more. But maybe [10:35] Starfield finally has its use case, [10:37] which is all of those loading screens [10:39] are advertising inventory. All right, [10:41] enough of that. Let's get into the [10:42] patent and how the technology actually [10:44] works. Figure one shows that the [10:46] software application, which includes [10:47] games, gets fed into a content [10:49] management system alongside promotional [10:51] advertising content from a remote [10:52] server. Those are then fed into an event [10:54] detection module, which is later [10:56] described as quote local to or remote [10:58] from the content management system. It [11:00] quote transmits or provides API hints [11:02] and/or event hints and quote includes a [11:04] machine learning model or system to [11:06] detect and identify events in the video [11:08] information, audio information, game [11:10] state data, or other information [11:12] provided by or from the interactive [11:14] software application to the event [11:15] detection module. End quote. Machine [11:17] learning over the years has morphed into [11:19] the catchall AI buzzword. And when they [11:22] describe what machine learning is and [11:23] they define it in this document, they do [11:25] list language models as being one of the [11:27] possible of many definitions for machine [11:29] learning. But basically by modern [11:31] marketing buzzword terms [11:33] they're talking about AI. Figure two [11:35] details a credit system where players [11:37] earn credits for watching ads in our [11:40] mega dystopian future. The patent [11:42] describes this image as quote a timeline [11:44] of promoted content presentation and [11:47] interactive software application usage. [11:48] End quote. Interactive software [11:50] application is going to be used a lot [11:51] here. It's Microsoft's really clunky way [11:53] of saying a program. It could be games. [11:55] It could be other applications like I [11:57] don't know QuickBooks. So, you can enjoy [12:00] an advertising promotional interruption [12:03] spot and ad break every time you [12:05] complete a general journal entry. Isn't [12:08] that fun? [clears throat] Figure three [12:09] writes that the system quote obtains a [12:11] promoted content trigger from the [12:13] interactive software application, [12:14] suspends a user's interaction with the [12:16] application based on the promoted [12:18] content trigger event, presents [12:20] audiovisisual promoted content to the [12:22] user, end quote, and then resumes the [12:24] application while giving them a credit. [12:26] Figure 7 illustrates user input as [12:29] linking to the CMS before it hits the [12:31] game or the application. The CMS is [12:34] described as quote in data communication [12:36] with an interactive software application [12:38] local to a client device including a [12:40] laptop, desktop, video game console, [12:42] settop box, smart TV, smartphone, table [12:44] computer, hybrid computer, headmounted [12:46] display, or any other electronic device [12:47] capable of presenting audiovisisual [12:49] information to a user. End quote. The [12:50] content management system is responsible [12:52] for suspending or pausing your [12:54] interaction with the games or [12:56] applications in order to allow the [12:57] promotional ads through. From a software [12:59] architecture standpoint, then their [13:00] block diagram is describing that your [13:02] input devices feed into the CMS first [13:06] and then it goes into the application. [13:08] So the CMS sits between you and your [13:10] input and the application receiving it. [13:12] Which means that the CMS is there to [13:14] interpret everything you feed into the [13:17] system including all input devices that [13:19] they could think of listing. If you [13:21] remember the old Intel eyetracking [13:23] technology for games, that would be an [13:25] example of input that could feasibly be [13:26] tracked by such a CMS. Likewise, they [13:29] explicitly list headmounted displays, [13:31] which are VR headsets. So, you could [13:32] imagine a situation where you are [13:34] playing in your frame or your Quest in [13:37] Cyberpunk VR or something. And to use [13:39] LG's example, you are in a bar, you look [13:41] over at the bar, and because it can [13:44] track all of this, maybe you look at the [13:47] in-game Brosef Logger, but maybe maybe [13:51] they need some money, someone in the [13:53] chain here, and they sub out the in-game [13:56] Broph logger for if you're running on an [13:58] RTX 3050 [14:00] Budweiser. And if you're running on an [14:03] RTX 5090, [14:05] um, high-end [14:08] alcohol brands, [14:11] let's see, uh, Johnny Walker. [14:15] Look, I don't know. I don't I don't [14:17] drink, so I unless it's reading LG [14:20] advertising agreements, in which case [14:21] we're very inebriated legally speaking. [14:24] >> Oh I accidentally clicked accept [14:27] on the terms for voice, and I'm [14:28] incredibly inebriated. The point is this [14:30] type of technology just like with LG ACR [14:33] solutions where they talk about a future [14:35] of AI scanning scenes. This is a thing [14:38] that they have talked about to interact [14:41] with onscreen product placement and [14:44] advertising. [14:46] You could end up in a situation where [14:47] it's looking at everything it knows [14:49] about your computer and about you and [14:51] it's giving you a different reality from [14:54] other people who are also using that [14:56] application. And that might be based on [14:58] things like what it thinks of your [15:00] socioeconomic status or your political [15:02] leanings. These are things they can do. [15:06] It's not like it's not imaginative. [15:08] Anyway, figure 9 illustrates training [15:10] data sets for AI and machine learning, [15:12] which are referenced several times [15:13] throughout the patent. Figure 12 is [15:15] where it all comes together. The player [15:17] gets a gameplay credit with some [15:19] duration of time, puts that credit into [15:21] the game like an old arcade token, and [15:23] then instead of getting the fun [15:25] mechanical clunk of the coin going in [15:27] the slot and a pinball being delivered [15:29] to you, you get 15 more minutes of [15:32] playing a video game. I mean, it's like [15:35] it's it's kind of the same idea except [15:38] instead of paying with a coin and then [15:40] getting to play the game, you you pay [15:43] for the game and then you watch an ad [15:46] and then you get to play the game for 15 [15:49] minutes, then watch another ad and maybe [15:51] there's microtransactions or a monthly [15:53] subscription to keep the servers online [15:54] or okay, it's not really it's actually a [15:56] lot worse. Figure 12 describes quote [15:59] determining a promoted content window [16:01] based on the first credit duration end [16:03] quote after which it allows the user to [16:05] actually play the game or use the [16:07] software described as quote commencing a [16:10] user's interaction end quote after which [16:12] it interrupts the player. Quote during [16:14] the promoted content window obtain a [16:16] promoted content trigger event from the [16:19] application based at least partially on [16:21] content of the application. suspend the [16:24] user's interaction with the application [16:26] and present audiovisisual promoted [16:28] content to the user. End quote. The best [16:30] part though is that Microsoft [16:32] acknowledges interruptions, but it says [16:34] that they're going to be the good kind [16:36] of interruption. Quote, "In at least [16:38] some embodiment, systems and methods [16:40] described herein allow for a [16:41] contextually aware interruption of a [16:44] user's interaction with an application [16:47] for the presentation of promoted [16:48] content. by interrupting the user's [16:50] interaction when contextually [16:52] appropriate, the user's experience may [16:54] be improved." End quote. That's right. [16:56] These people are just as sociopathic as [16:59] the CTV advertising executives who say [17:01] that their job is to deliver you better [17:05] ad experiences, [17:07] which are experiences that you want. [17:10] like when you turn your car on and [17:12] you're greeted with an ad on the [17:15] gigantic screen in the middle of the car [17:17] that nobody wanted. But it's not an ad. [17:19] It's a it's an experience to quote Lewis [17:21] Rossman. And concerningly, page 11, [17:24] paragraph 0031 of the filing, which [17:26] we're showing on Justia patents instead [17:28] as it's easier to read, describes the [17:30] networking solutions to communicate with [17:32] the user's device. Those can be the [17:34] obvious, Ethernet or wireless, but it [17:36] can also include the less obvious. [17:38] quote, "In some embodiment, the [17:40] communication device is a short-range [17:41] wireless communication, such as a [17:43] Bluetooth connection or a Wi-Fi direct [17:46] connection that allows data [17:47] communication between the client device [17:49] and electronic devices in proximity to [17:52] the client device." End quote. So, to be [17:54] clear, if your device is offline, you've [17:56] taken measures specifically to keep it [17:59] off of the internet, whether that's a [18:00] computer or maybe a console that you [18:02] have in your living room or a TV. They [18:04] do list those as options. But if that [18:07] device has Bluetooth or a Wi-Fi direct [18:09] connection, then the ad might still be [18:12] able to penetrate through. They might [18:13] still be able to punch through and read [18:15] information if it's running in the [18:16] background. ACR, remember, can run even [18:19] if you don't have Ethernet plugged in. [18:21] In fact, it's sort of a core feature of [18:22] it. And as soon as it receives a [18:24] connection or an ability to exfiltrate [18:25] that data, there is a possibility that [18:28] data is exfiltrated. So unplugging the [18:31] network, just like with TVs, might not [18:33] be good enough if the mechanism is able [18:37] to find some other solution like [18:38] Bluetooth to still interact with the [18:41] device, which Bluetooth is listed [18:43] explicitly, and it's a continuation of [18:45] companies trying to find a means of [18:47] circumventing users who have [18:49] intentionally taken their devices off of [18:51] the internet. They also described the [18:52] software as being quote stored on and or [18:54] executed remotely from the client device [18:57] such as on a remote server or remote [18:59] computing device in which the client [19:00] device is in network communication end [19:02] quote. And they describe ads as being [19:04] shoved down your throat as being quote [19:07] via a display, audio speakers, haptic [19:09] devices, etc. end quote. But Microsoft [19:12] is self-aware enough to recognize the [19:14] problems. Quote, "In some instances, [19:15] however, constraining, interrupting, [19:17] suspending user inputs to or otherwise [19:20] preventing usage of the interactive [19:21] software application independent of [19:23] context can cause negative user [19:25] experiences." End quote. Yes, in some [19:28] instances, it can negatively affect the [19:30] user experience. [19:32] Some doing some pretty heavy lifting. [19:36] Quote, "For example, many online-based [19:37] multiplayer video games and some single [19:39] player video games cannot be paused or [19:41] suspended as the virtual environment of [19:42] the game persists while the user is [19:44] unable to interact with the interactive [19:46] software application. This can result in [19:47] the user losing an online multiplayer [19:49] match, dying in a single player role [19:51] playing game, being unable to [19:52] communicate with or interact with [19:53] teammates or otherwise having a negative [19:55] user experience." End quote. So, why are [19:57] we making this? This is a But don't [20:00] worry, they've got a solution. They say [20:02] the way to fix this is to use quote [20:04] context from the application ensuring [20:06] the promoted content is presented [20:07] without substantially the key word there [20:10] negatively impacting the user's [20:12] experience end quote. To help with that, [20:14] they describe their methods as being [20:15] performed quote at least partially by a [20:17] content management system of a client [20:20] device end quote. And they say the CMS [20:22] may run quote natively on a user's [20:23] personal computer or video game console [20:25] end quote. But of course, they also [20:27] suggest that servers could handle this [20:28] or quote different devices than that [20:30] executing the interactive software [20:32] application. End quote. The more you [20:34] read, the worse it gets. But it all kind [20:35] of boils down to that big picture again, [20:38] which is it's not possible to know the [20:40] context of these things without actively [20:44] knowing everything that's on your screen [20:46] all the time, just like the smart TVs [20:50] sometimes directly state that they do. [20:53] And uh that also includes by the way the [20:55] audio where they describe needing to [20:58] have audio cues and potentially [21:00] conversational or chat cues being [21:03] non-specific as to whether that's voice [21:05] or text. A few thoughts here overall. So [21:08] on Linux, our concern is that given how [21:10] many studios Microsoft owns, this could [21:13] get spread there too. It could mean that [21:16] the Microsoft games are just the things [21:18] you avoid altogether. But Microsoft has [21:20] many partners and they've got kind of a [21:22] strangle hold on those that they work [21:24] with. They own obviously a large console [21:26] platform. Microsoft has leverage where [21:29] this could be deployed outside of their [21:31] own studios. Now, it is a patent filing. [21:33] Patent filings don't always get directly [21:35] applied in technology and not always in [21:37] the immediate future. Sometimes it's [21:38] pretty far out. Sometimes they're trying [21:40] to reserve an idea to maybe license it [21:41] later. But the fact that this exists as [21:44] a patent shows that Microsoft is at [21:46] least interested enough to think that [21:48] they should protect the idea before [21:50] someone else gets to it. That's the [21:52] concerning part to us and that is what [21:54] warrants the discussion now before it's [21:57] too late because it's in that stage [21:58] where they might look at backlash and go [22:01] let's kick this can another five years [22:03] down the road and try again or something [22:05] and delay it. Now, the other thing to [22:07] think about is that Microsoft also has [22:09] kind of a strangle hold over software [22:11] partners, like for example, Adobe. Now, [22:13] Adobee's complicit in that, uh, and [22:15] they're in their own right, but [22:17] we're concerned about the technology [22:18] used outside of just video games. [22:20] Microsoft's patent uses the language [22:22] interactive software application rather [22:24] than game in many places. We couldn't [22:26] find a singular legal definition of the [22:29] phrase interactive software application. [22:31] We did, however, find some past examples [22:33] of its use in law. In California civil [22:36] code 1.81.5 section 1798.136 [22:39] of the CCPA, the law defined even a web [22:43] browser as quote an interactive software [22:46] application and quote a lawsuit against [22:48] Sony also shows interactive software [22:50] application in use, but in that one it [22:52] mostly refers to games. We also stumbled [22:55] across Schrodinger's ULA, which to be [22:57] clear is not a joke, and found that they [22:59] define their software as quote [23:01] interactive software applications end [23:03] quote. And in their case, that includes [23:05] material sciences software and life [23:07] sciences software, not video games. The [23:09] point is that the phrase interactive [23:10] software application is maybe [23:11] intentionally vague, where it can mean a [23:13] web browser, a material science [23:15] application, or a video game. If [23:17] Microsoft wants to work with Adobe to [23:20] one day launch the adup supported [23:23] premium tier of Adobe Photoshop basic [23:25] edition for $5 a month plus ads to then [23:29] pop up an ad every time you go to save a [23:31] file feasibly this patent could be [23:34] applied in such an implementation. Steam [23:36] storefront policy as pointed out online [23:39] technically prohibits ad-based revenue. [23:41] Quote, "Steam does not contain any paid [23:43] advertising, nor are advertising models [23:46] supported in games distributed on [23:47] Steam." End quote. Steam has three [23:49] categories of supported in-game ads. [23:52] Product placement, so maybe a particular [23:54] car model in a racing game, cross [23:55] promotions, including other products, [23:57] and paid advertising outside of Steam, [23:59] meaning normal ads on some random [24:01] website. But Steam makes clear that it [24:03] does not allow quote paid advertising as [24:06] a business model in game, such as [24:08] requiring players to watch or otherwise [24:10] engage with advertising in order to play [24:12] or gating gameplay behind advertising [24:14] end quote. They then say any such [24:16] features must be removed for sale on [24:18] Steam, which by the way is an awesome [24:20] policy. So that's cool to see. That [24:22] doesn't give us a whole lot of [24:24] reassurance in general though. Some, but [24:27] not a lot. For one, Steam isn't the only [24:30] platform out there. to Microsoft is a [24:33] behemoth and we have faith that a [24:36] company with the capabilities and evils [24:39] of their past has means of figuring out [24:42] how to circumvent things like this and [24:43] slime their way around the rules. For [24:45] point three, we shouldn't need to rely [24:48] on a digital storefront to protect us [24:52] from these things because that's crazy. [24:54] That's not how it's supposed to work. Uh [24:56] but still a great policy from Steam. [24:58] What isn't clear to us though is if the [25:01] operating system overrides this. Steam's [25:04] language is pretty clear about the games [25:06] not doing these things. What's not [25:08] clear, maybe because it wasn't an [25:09] option, is if the ads aren't baked into [25:12] the game's binaries or its code or the [25:15] execution of them on Steam, but instead [25:18] they're executing at an OS level where [25:22] at say the layer of abstraction away [25:24] from Steam, the ads can be injected [25:27] through the OS scanning your screen, [25:28] determining the ad break, and [25:30] technically maybe it never actually [25:32] interacts with the game itself or the [25:34] game as its code is uploaded to Steam [25:36] itself. [25:37] and it's all handled sort of external [25:39] from Steam. I don't know. I actually [25:41] don't know. I don't know if that is sort [25:45] of something Steam could at any point [25:47] say, "No, you. That's not okay [25:49] because they're a private company. They [25:50] have private rules. It there's no right [25:53] to any particular distribution if you're [25:56] on Steam. They can block that if they [25:58] want to. But at the same time, it's just [25:59] that question of so where is that line?" [26:03] And it's not really clear. Now the [26:04] positive of that implementation I guess [26:06] is that that would seem to implicate [26:09] Windows specifically. So when such an [26:11] implementation where you're doing [26:12] everything on Steam and if Steam [26:14] disallows that then Linux might actually [26:18] be kind of a pathway out of most of the [26:20] concerns with the exception of whatever [26:21] other applications might use such a [26:23] technology. We saw one user say that the [26:26] patent would require engine level [26:28] implementation. [26:29] I read through the document. I can't [26:31] find anything that suggests this is [26:33] true. Um, in fact, the patent very [26:36] broadly states that it can be [26:38] implemented on any number of devices, [26:40] including those that aren't involved in [26:42] gaming anywhere in the pipeline directly [26:44] anyway, including at the server level or [26:47] at the local level to the device itself. [26:49] So, I couldn't find any language that [26:51] suggested this is strictly a game engine [26:54] level implementation, which is also [26:56] pretty concerning. Microsoft isn't alone [26:57] here. There was a time that Sony [26:59] patented an ad tech solution where shows [27:01] might be paused for an ad, but viewers [27:03] could skip the ad by standing up, [27:05] throwing their arms up, and shouting [27:06] excitedly, "McDonald!" while evidently [27:08] waving their arms around excitedly [27:10] before then being allowed to continue [27:12] watching a character it looks like [27:14] getting shot point blank on screen. And [27:16] soon enough, we'll be at peak ad breaks. [27:18] >> This is the first of our planned [27:19] upgrades once we can roll back some of [27:21] holiday ad restrictions. We estimate we [27:24] can sell up to 80% of an individual's [27:28] visual field before inducing seizures. [27:31] >> And with that, a closing message FROM [27:33] MICROSOFT. [27:34] MICROSOFT. [27:36] >> OKAY. [27:37] >> That's it for this one. Thanks for [27:39] watching. Subscribe for more. As always, [27:40] you can go to store.camesex.net if you [27:42] want to grab a Microsoft shirt. We're [27:44] currently out of a few sizes because oh [27:47] boy, do people really like Microsop. [27:50] just I never knew they they loved [27:52] Microsoft so much. It's just people it [27:56] must be just the love of the game. I [27:59] mean, you know, if the game is like [28:02] super targeted forest advertising and an [28:04] awful operating system, but anyway, [28:06] those are on the store if you want them. [28:07] We have more coming in soon. Uh, of [28:09] course, you can also grab any number of [28:10] our other things there to support us. [28:11] Our patreon.com/gamersexes. Thanks for [28:13] watching. Subscribe for more. We'll see [28:15] you all next time.