Harman Pushes Zero-App AI System to Replace In-Car Mobile Interfaces
Ai4
Autonomous AI agency will eliminate traditional application menus across consumer hardware, forcing automakers and electronics brands to own cross-device orchestration or risk becoming commoditized endpoints.
## Background
Published on YouTube by Ai4 on August 15, 2026, this keynote features Armin Prommersberger, Chief Technology Officer at HARMAN International. The session explores the shift from screen- and app-centric digital experiences toward "Zero-App" AI-orchestrated interfaces, examining the architectural, business model, and UX continuity challenges facing the automotive and consumer electronics industries.
## TL;DR
Armin Prommersberger argues that modern conversational intelligence and autonomous agency are finally ending the app-centric era, shifting the paradigm from user micromanagement to proactive AI orchestration. Harman addresses this fragmentation across smart devices and vehicles through a four-layer architecture—encompassing account federation, orchestration AI, payload delivery, and IoT connectivity—governed by unified avatars like Luna. To succeed in real-world deployments, builders must conquer strict sub-500ms latency thresholds, optimize hybrid edge-cloud hardware costs, and maintain balanced cybersecurity guardrails. Ultimately, this zero-app approach enables frictionless, cross-domain continuity, turning environments like the connected vehicle into intelligent, context-aware nodes.
## Atmosphere & style
The presentation carries a pragmatic, engineering-grounded keynote tone that cuts through generic AI hype with candid industry realism. Prommersberger delivers an authoritative and measured narrative, moving fluidly between high-level ecosystem shifts and tactical constraints like millisecond latencies and bill-of-materials costs. It feels like an insider masterclass for systems architects, instilling a sense of technical urgency while concluding with a confident, open invitation to collaborative innovation.
## Key takeaways
- [00:04](https://www.youtube.com/watch?v=e0_CQUnkNH0&t=4s) The traditional app-centric paradigm has trained users to micromanage dozens of apps, normalizing cross-app friction rather than delivering true convenience.
- [00:04](https://www.youtube.com/watch?v=e0_CQUnkNH0&t=4s) Harman leverages an end-to-end industry vantage point across professional studio production and massive consumer playback telemetry to track consumption shifts.
- [04:42](https://www.youtube.com/watch?v=e0_CQUnkNH0&t=282s) Unlike early virtual assistants, modern conversational intelligence possesses genuine natural language comprehension, setting the foundation for zero-app interfaces.
- [04:42](https://www.youtube.com/watch?v=e0_CQUnkNH0&t=282s) The rise of AI agency allows systems to predict needs and autonomously execute multi-step actions without demanding constant user permission.
- [04:42](https://www.youtube.com/watch?v=e0_CQUnkNH0&t=282s) Uncoordinated multi-device agency risks creating chaos, requiring a centralized AI orchestration front-end (such as Harman's Luna avatar) to govern device ecosystems.
- [08:44](https://www.youtube.com/watch?v=e0_CQUnkNH0&t=524s) Orchestration architectures must reconcile radically misaligned innovation lifecycles: 10-year vehicle platforms, 12-month consumer electronics, and weekly AI model updates.
- [08:44](https://www.youtube.com/watch?v=e0_CQUnkNH0&t=524s) Harman's orchestration framework relies on four modular tiers: Account Federation, Orchestration AI, deterministic MIMO Payload, and IoT connectivity.
- [08:44](https://www.youtube.com/watch?v=e0_CQUnkNH0&t=524s) Security guardrails must be carefully calibrated because over-constraining destroys autonomous agentic delight, while under-constraining creates severe vulnerability.
- [13:59](https://www.youtube.com/watch?v=e0_CQUnkNH0&t=839s) Interaction latency must strictly stay under 500 milliseconds for utterance-to-action responses, as exceeding this threshold causes immediate user distrust.
- [13:59](https://www.youtube.com/watch?v=e0_CQUnkNH0&t=839s) Achieving viable consumer device integration requires a fine-tuned hybrid edge-cloud model that aggressively minimizes hardware bill-of-materials and recurring token costs.
- [13:59](https://www.youtube.com/watch?v=e0_CQUnkNH0&t=839s) Cross-domain continuity enables seamless contextual handoffs, such as instantly resuming media and applying personalized acoustic profiles when moving from the home into a vehicle.
## Detailed walkthrough
### [00:04](https://www.youtube.com/watch?v=e0_CQUnkNH0&t=4s) The Impending Death of the App-Centric Model
Armin Prommersberger, CTO of Harman (a Samsung company), opens his presentation by reflecting on the digital landscape. Having been in the industry for years, he notes that while the "death of the app" has been declared repeatedly over the past two decades without actually materializing, this time is fundamentally different. Harman occupies a unique vantage point across the entire content lifecycle: on one side, its professional audio equipment powers recording and production studios globally, offering early visibility into content creation trends. On the other side, Harman ships hundreds of millions of consumer playback devices annually—including headphones, Bluetooth speakers, and automotive infotainment systems—granting deep telemetry on user consumption behavior.
Prommersberger addresses how the industry collectively built an "app-centric" paradigm that trained users to become the micromanagers of their own digital lives. Modern smartphones typically carry 40 to 80 apps, while industry professionals often exceed 100. Yet despite this abundance, users regularly interact with only a tiny fraction of them daily. The industry normalized this cross-app friction and labeled it convenience. Now, singular modalities and isolated machines are failing to fulfill evolving consumer expectations, necessitating a shift toward **AI orchestration**—a paradigm that seamlessly connects surrounding devices by understanding context across user intent, devices, and the environment.
[04:42](https://www.youtube.com/watch?v=e0_CQUnkNH0&t=282s) Intelligence and Agency: Why This Shift Is Different
Prommersberger outlines two major technological catalysts that distinguish the current moment from past failed promises of app-less interfaces. The first catalyst, which scaled roughly a year prior, is conversational intelligence: systems capable of understanding human language and parsing user intent. Unlike early 2010s virtual personal assistants (VPAs), which were merely basic intent classifiers bolted onto specific applications, today’s artificial intelligence possesses true linguistic comprehension, though it remains largely trapped behind glass screens.
The second catalyst—emerging only months before—is **agency**. Intelligent systems are no longer passive advisors asking for permission; they can now predict needs and execute multi-step actions autonomously without requiring continuous human approval. However, in an environment saturated with dozens of smart, eager-to-please, yet uncoordinated devices, unmanaged agency risks causing chaos rather than delight. Harman’s solution to this fragmentation is an AI orchestration front-end avatar known as Luna (part of Harman's *Ready Engage* product suite), which acts as the unified conversational interface governing the surrounding device ecosystem.
```mermaid
flowchart TD
accTitle: Evolution of Interface Paradigms
A["App-Centric Model<br/>(User micromanages 40-80+ apps)"] --> B["Conversational Intelligence<br/>(Natural language intent parsing)"]
B --> C["AI Agency<br/>(Predictive & autonomous execution)"]
C --> D["AI Orchestration<br/>(Unified context & cross-device fabric)"]
```
[08:44](https://www.youtube.com/watch?v=e0_CQUnkNH0&t=524s) Architecture and System Layers of AI Orchestration
To illustrate the necessity of AI orchestration, Prommersberger shares everyday examples where cross-device experiences ought to be fluid—such as transferring media seamlessly or adjusting ambient settings—yet fail due to ecosystem silos. He emphasizes that while the presentation highlights consumer electronics, the exact same architectural challenges exist in industrial production floors, hospitals, and connected vehicle fleets.
Building an architecture for AI orchestration requires bridging vastly different device lifecycles and hardware capabilities: automotive systems evolve on 10-year cycles, consumer electronics update every 12 months, and AI models iterate weekly. Harman approaches this complex environment through a structured four-layer system architecture:
```mermaid
flowchart TD
accTitle: Four-Layer System Architecture for AI Orchestration
L4["Account Federation Layer<br/>(User identity & external account bridging)"]
L3["Orchestration AI Layer<br/>(User, device, usage & content graphs)"]
L2["Payload Layer<br/>(Deterministic MIMO streaming & sync)"]
L1["IoT Layer<br/>(Device discovery & control protocol)"]
L4 --- L3
L3 --- L2
L2 --- L1
```
- **Account Federation Layer:** Manages user identity and integrates diverse third-party accounts across disparate ecosystems.
- **Orchestration AI Layer:** Serves as the central intelligence node, evaluating user graphs, device graphs, usage telemetry, and content metadata while interfacing with external control systems like SmartThings or Matter.
- **Payload Layer:** Handles deterministic multiple-input, multiple-output (MIMO) data and media distribution, ensuring synchronized playback with video and real-time system actions.
- **IoT Layer:** Facilitates direct machine-to-machine communication and inter-device connectivity.
A major design mandate within this architecture is cybersecurity. Given the expanded attack surface resulting from rich cross-device and external cloud connections, guardrails must be carefully tuned. Over-constraining security destroys agency and "magic," whereas under-constraining it exposes the ecosystem to bad actors. Thus, trust and security are built in as baseline boundary conditions.
[13:59](https://www.youtube.com/watch?v=e0_CQUnkNH0&t=839s) Key Learnings: Latency, Hybrid Processing, and Cross-Domain UX
Reflecting on Harman's deployment journey, Prommersberger highlights hard-won operational insights for builders and system architects:
1. **Multimodal Interaction & Latency Limits:** Interfaces are rapidly evolving beyond touch screens to voice and real-time biosignals (such as stress levels). However, latency remains the primary cause of user disengagement. If system response time—whether utterance-to-utterance or utterance-to-action—exceeds 500 milliseconds, users doubt whether the command was registered. Because a single request triggers hundreds of micro-tasks between edge devices and cloud endpoints, every 10-millisecond optimization is crucial.
2. **Hybrid Architecture & Bill-of-Materials (BOM) Optimization:** To satisfy strict latency requirements while keeping hardware costs viable, systems must balance edge computation with cloud offloading. Cost optimization down to the cent is required when integrating low-cost hardware (e.g., $100 headphones) into an orchestrated AI ecosystem without creating prohibitive token-usage expenses.
3. **Organizational Synergy:** Building these experiences requires bridging the gap between pure AI software companies (which lack deep consumer hardware and UX expertise) and consumer electronics firms (which lack core AI development capabilities).
4. **Cross-Domain Continuity:** The ultimate goal is seamless context transfer across domains. For example, when a user leaves their home audio setup and enters their vehicle (such as a Volvo equipped with a Bowers & Wilkins system), the vehicle greets them, offers to resume their audio session at the exact timestamp, and applies their custom acoustic profile automatically.
Prommersberger concludes by inviting industry partners, content creators, and technology vendors to collaborate with Harman via QR code to build out this unified zero-app ecosystem, finishing his presentation with over a minute to spare.
## Notable quotes
> the vehicle cabin is becoming the most contested surface for AI-led experience orchestration — Armin Prommersberger
> The next interface is not a screen -- it’s intent, mediated by AI. — Armin Prommersberger
> When intent becomes the interface, distribution changes, data control changes, brand value changes, and “who owns the customer” gets rewritten. — Armin Prommersberger
## Video description
Armin Prommersberger - Chief Technology Officer, HARMAN International
For twenty years digital experiences were built around a single assumption: users open an app, navigate menus, and adapt to product’s logic. That assumption is collapsing. The next interface is not a screen -- it’s intent, mediated by AI. We call this shift "Zero-App/AI-First Interfaces." In automotive, this shift is existential: the vehicle cabin is becoming the most contested surface for AI-led experience orchestration. This session examines what model is dying inside the vehicle -- and what must replace it to deliver trust, safety, and genuine delight. The opportunity is massive: the car can become the most context-rich node in a consumer’s daily ecosystem. The risk is just as large: if automotive doesn’t own the orchestration layer, it becomes a commodity endpoint in someone else’s AI universe. The session will examine the gaps between rhetoric and reality across orchestration AI, UX continuity, trust, and operating models. The “Zero-App” shift is not a UX trend... it’s a business model shift. When intent becomes the interface, distribution changes, data control changes, brand value changes, and “who owns the customer” gets rewritten.
哈曼力推“零应用”AI系统以取代车载移动界面
自主AI代理将消除消费级硬件中的传统应用菜单，迫使汽车制造商和电子品牌掌控跨设备编排能力，否则将面临沦为平庸终端的风险。
Armin Prommersberger 认为，现代对话智能和自主代理正在终结以应用为中心的时代，将交互范式从用户的微观管理转变为主动式 AI 编排。哈曼通过涵盖账户联合、编排 AI、有效负载交付和物联网连接的四层架构，来解决智能设备与车辆之间的碎片化问题，并由像 Luna 这样的统一人形智能体进行管理。为了在实际部署中取得成功，开发者必须攻克严格的 500 毫秒以下延迟门槛，优化端云混合硬件成本，并维持均衡的网络安全防护屏障。最终，这种“零应用”方式将实现无摩擦的跨域连续性，把网联汽车等环境转变为具备上下文感知能力的智能节点。
传统的以应用为中心的范式迫使用户去微观管理数十个应用，使得跨应用间的摩擦体验常态化，而非带来真正的便利。 (0:04)
哈曼利用贯穿专业工作室制作与庞大消费端播放遥测数据的端到端行业优势，持续追踪消费习惯的转变。 (0:04)
与早期的虚拟助手不同，现代对话智能拥有真正的自然语言理解能力，为“零应用”界面的实现奠定了基础。 (4:42)
AI 代理能力的兴起使系统能够预判需求并自主执行多步骤操作，而无需反复索要用户许可。 (4:42)
缺乏协同的多设备代理操作容易引发混乱，因此需要一个集中式的 AI 编排前端（如哈曼的 Luna 虚拟形象）来管理设备生态系统。 (4:42)
Reading summary
Armin Prommersberger, CTO of Harman, argues that the digital landscape is transitioning from fragmented, app-centric interfaces to AI-orchestrated experiences driven by linguistic intelligence and autonomous agency.
Catalysts Beyond the App-Centric Paradigm (0:04)
Prommersberger contends that the traditional app-centric model has conditioned users to micromanage 40 to 80 mobile applications, despite only engaging with a small fraction on a daily basis. While previous claims of the demise of applications failed to materialize, he argues that two recent technological shifts make this transition viable. First, conversational intelligence reached scale by offering natural language intent comprehension, advancing beyond early 2010s virtual personal assistants that served as simple intent classifiers.
Autonomous Agency and the Orchestration Layer (4:42)
The second catalyst identified by Prommersberger is machine agency, which enables systems to predict needs and execute multi-step actions autonomously rather than acting as passive advisors. However, he warns that uncoordinated agency across numerous smart devices risks operational friction. Harman's proposed solution is an orchestration front-end avatar called Luna (part of the Ready Engage suite), designed to harmonize interactions across an ecosystem rather than leaving individual devices to operate in silos.
Multi-Layer Architecture Across Diverging Hardware Lifecycles (8:44)
Deploying cross-device orchestration requires bridging sharply mismatched development timelines: connected automotive platforms operate on 10-year lifecycles, consumer electronics refresh every 12 months, and AI models iterate weekly. Harman addresses these disparities through a four-layer architecture comprising an Account Federation Layer for identity, an Orchestration AI Layer governing contextual graphs and protocols like SmartThings or Matter, a Payload Layer managing deterministic multiple-input, multiple-output (MIMO) streaming, and an underlying IoT Layer for machine-to-machine connectivity.
Prommersberger emphasizes that cybersecurity guardrails must be calibrated carefully within this stack. Because cross-device connections expand the attack surface, security settings that are too rigid eliminate system agency and user utility, while insufficient controls introduce serious vulnerabilities.
Latency Constraints, Hybrid Compute, and Continuity (13:59)
Harman's implementations demonstrate that latency remains the principal point of failure for multimodal interactions. Prommersberger observes that when total response time exceeds 500 milliseconds—whether between spoken utterances or from utterance to physical action—users question whether the system processed their command. Because an individual prompt triggers hundreds of micro-tasks across cloud endpoints and edge devices, engineering teams must optimize execution in 10-millisecond increments.
To keep hardware commercially viable, systems rely on a hybrid architecture that balances edge computation against cloud processing. Prommersberger notes that bill-of-materials optimization must be managed down to the cent, ensuring lower-cost devices, such as $100 headphones, can participate in the ecosystem without incurring unsustainable token costs. The operational objective is cross-domain continuity, exemplified by a user transitioning from home audio to a vehicle that instantly resumes media playback at the correct timestamp and applies customized acoustic tuning.
Prommersberger concludes that realizing an orchestrated, zero-app environment requires bridging the organizational gap between pure AI software vendors and consumer hardware manufacturers, urging cross-industry collaboration to establish shared interfaces and protocols.
Armin Prommersberger: the vehicle cabin is becoming the most contested surface for AI-led experience orchestration
Armin Prommersberger: The next interface is not a screen -- it’s intent, mediated by AI.
Armin Prommersberger: When intent becomes the interface, distribution changes, data control changes, brand value changes, and “who owns the customer” gets rewritten.
Original transcript
[0:04] So my name is Armen Promisberger, CTO of Harmon.
Harmon is actually a Samsung company and division
[0:10] and we have a quite interesting perspective on the
market, how consumer behavior changes and we're
[0:15] going to talk about that. So a few of you most
probably have been around just like me for a few
[0:21] years and we have seen the moment where the app
is dying like 10 times over and then it didn't,
[0:26] right? So let's talk about in the next 20 minutes
why we think this time is different, how we come
[0:35] to the conclusion, what does it mean and what
did we learn already that you might take away
[0:40] that can help you. The good news is I'm not
here to sell you anything in the morning.
[0:45] So it's more about starting a really good
conversation of what is coming in the future.
[0:51] Why does Harmon have this unique perspective and
that's very straightforward. We are on both sides
[0:57] of the content. On the one side, we we create
products and solutions that are used to create
[1:04] content in recording studios, production
studios around the world. That gives you
[1:10] gives us an early insight in what the trends
are, what is happening in content creation.
[1:17] On the other side, we are selling hundreds
of millions of devices every year for
[1:22] content reproduction. headphones, Bluetooth
speakers, infotainment systems in your cars
[1:28] and that gives us an insights how people are using
it and now we can fuse both of these signals.
[1:34] One thing that we as an industry all of us now we
are guilty right um that we induced into the users
[1:42] lives is actually this idea of abentricity. The
appentric model is something we all created more
[1:50] or less 20 years ago. So there's an app for
everything, but when you're using your apps,
[1:55] how intuitive is it? Some of them are really
good, but have you ever tried to make two apps
[2:00] work together nicely to create something
together? That might create some friction.
[2:07] What we see in the market is that user
expectations are very much changing. What we see
[2:13] in the market is that a singular modality like the
absentric model will no longer feed these desires.
[2:21] What we also see in the market is that individual
isolated machines devices are no longer feeding
[2:30] the desire of the user. So this is the moment
when we as a group need to start about something
[2:35] that we call actually AI orchestration.
The black bar is actually hiding that
[2:42] So AI orchestration is really for us the moment
when we talk to all of those devices around the
[2:49] users and create a unique experience understanding
context of the user the devices the environment
[2:59] that will lead us to a conversation today
about architecture and there's something
[3:05] about architecture and fabric right that's why
in our lingo we talk we call it architecture
[3:11] because we need to think about how we architect
a full system that is talking to all of those
[3:16] devices and then how we create a fabric that
is eventually allowing the AI, the information
[3:23] and the content to move around as intended.
Let's start with the appentric part, right?
[3:31] What we really did is we trained our users to
be the micromanagers of their digital lives,
[3:39] right? We have all of these wonderful smartphones
and on average there are 40 to 80 apps on every
[3:47] smartphone. I guess in this room with our
profession we are way beyond the 100 right. So
[3:56] we simply normalized friction the friction that
we just talked before and called it convenience.
[4:03] I really believe that AI is
here to rip this idea apart.
[4:09] What is really happening is that out of these
100 apps, you're using a handful every day,
[4:15] maybe not even half of it every month. So, what
is the point? Right? The app per se has been
[4:23] remarkably hard to kill, right? It survived its
own um funeral quite a few times. But why is it
[4:33] different now? Why do we believe? What do we see
in our usage data that this is a different moment?
[4:42] The first thing that arrived about a year ago at
scale right for the broad public is the arrival
[4:50] of very intelligent systems that can converse with
the user in human language parsing their intent.
[5:00] We all remember the 2014s 15s with the advent of
the what we called back at the time VPAs but they
[5:07] were not really very intelligent. They were pretty
much I would say intent classifiers bolded onto
[5:13] applications. Now intelligence finally arrived
but it's still kind of trapped behind glass.
[5:23] The second thing that is very different and
that is only month old in the public view right
[5:29] is these intelligence systems now became or
they gained agency. They can do things now. They
[5:36] can act and they can act and predict without
you even being in the loop without you being
[5:43] oh I recommend you take this action and this
is what you need to do. So this is why we think
[5:49] it's very different right now that we have two
moments that create a unique moment in time.
[5:59] The intelligence system can converse with
you like a human being and it is able to act.
[6:08] So what does this mean and why should this
be interesting to our users, our consumers?
[6:14] Consider a situation like this in a year from
now. You most probably won't see any robots
[6:18] sitting around. However, you will be surrounded
by 10, 20, 30 pretty intelligent devices.
[6:27] Now, we all dealt with all these beautiful AI
systems, right? And we're doing it all the time.
[6:33] And one thing we learn is like they are very
eager to please, right? They really want to
[6:38] please you. They want to help you. They want to
get reconfirmation. Now think about a situation
[6:43] when you are in your living room. You have
20 devices. They are not working together.
[6:47] They are all monolithic and all 20 want to help
you and and want to please you. That doesn't sound
[6:52] like delight. It sounds pretty much like potential
chaos, right? This is the moment when we believe
[6:58] actually that AI orchestration is entering the
room. And what you see right there in the middle,
[7:08] Luna, is actually not a fancy character we
created for AI4. If you're interested in what she
[7:14] really is, search the web for Ready Engage Harmon
International. She actually is, what you see here
[7:22] is a front-end avatar of our orchestration
AI system, and she is a skin on a rig.
[7:31] The moment we can orchestrate
all those devices around us
[7:38] is the moment when magic happens. Now why would
the user be interested and not just the tech nerds
[7:45] and the engineers and maybe some product managers
and marketing people? Because this is the reality
[7:50] for example of us as a company. What you see
here is a representation, an insight in how
[7:57] our product portfolio looks like and the world
of our users. We are not living in the matrix.
[8:02] We are still living in a very physical world. We
are dealing with physical devices in real spaces.
[8:10] We touch the lives of our users throughout the
day multiple times with headphones, Bluetooth
[8:15] speakers, speakers at home, infotainment systems
in your cars. And then if you put our mothership
[8:20] into the mix right with Samsung then there are
mobile phones, TVs, intelligent um appliances, all
[8:27] of that. We are much more than just a device in
the corner. We are literally part of the family.
[8:33] And this is the power of the insights that
we have. How the devices, how people are
[8:37] using the devices, what is changing over
time as a step function or gradually.
[8:44] This now is a representation of how this could
look like. We selected a few very simple use
[8:50] cases. If you read quickly through those
use cases, you will start nodding and say,
[8:55] "Yeah, this is how it should be." Then please
think for a second, how often does this work
[9:00] seamlessly and frictionfree in your home, in your
environment today? Most probably it does not,
[9:07] right? The orchestrations AI's job is not only
to reduce the overload of the intelligence and
[9:14] the eagerness to please. It's actually there to
anticipate your needs, understand for example
[9:20] your intention, your aspiration, the context of
the devices, your environment and your content
[9:26] and then make decisions of what you might
be liking next as a magical experience.
[9:33] One thing for all of us because I think
we have as many builders as people who
[9:37] actually are responsible for infrastructure
and tool chains and workflows.
[9:42] This looks very consumer electronics focused.
You can replace this picture simply with
[9:49] a production floor, a hospital, a car fleet
and you still run into the exact same problem.
[9:56] You have a lot of intelligent devices that you
need to orchestrate to make sense of for the user.
[10:04] So now let's talk about what we need to
architect. So how are we dealing with
[10:07] this? What is the architecture? What is
the fabric that we are applying? Right?
[10:13] We need to consider one thing. The systems
that we are building are continuously moving.
[10:20] AI is moving incredibly fast. If you're in
automotive, your world moves in 10 years.
[10:26] If you are in consumer electronics, your world
moves in 12 months. If you're dealing with AI,
[10:32] your world moves like in weeks. So now we need to
bring all of that together. The way how we think
[10:38] about it at Hormon is very simple. The foundation
to us is physical devices and control apps. They
[10:42] are still there at this point in time. As you can
see, these devices are all very different. Some
[10:48] have massive computational power. Some don't. They
all need to now engage with the same ecosystem.
[10:55] So, what we are thinking of is a system of four
layers. The first two layers are IoT and payload,
[11:01] which is really what the devices are directly
talking to because IoT is obviously required so
[11:07] the devices can talk to each other, understand
each other, and communicate. And then payload
[11:11] for us is where the content comes in and is
distributed across all of your environments,
[11:15] all of your devices which is to us a myo which
is a multiple inputs, multiple output structure,
[11:21] right? It must be very deterministic because
most of the times we need to synchronize it with
[11:26] some video or other actions that are happening in
the system. From the top what is coming to us is
[11:33] actually what we call an account federation layer.
Why is it account federation or just account? In
[11:38] our world, we deal with multiple accounts
and most probably some of you will do too.
[11:42] We deal with a user identity. The user will bring
multiple accounts with them. In the middle of
[11:48] all of that sits the orchestration AI and she is
actually talking and working with the user graph,
[11:53] the device graphs, but eventually also for example
with usage and content graphs. We are overall then
[12:02] connecting all of this to the outside world.
We are not building this for ourselves,
[12:07] right? The user must be making this part of their
his own ecosystem. So we need to connect it to
[12:14] other accounts. We need to connect
it to other services on the AI layer.
[12:19] Very important. The all the content needs to
come in. And then there are control systems.
[12:24] It is important that by the way this here
is not a replacement for small things matter
[12:30] or anything like that. Actually it's working
with them. Right? We want to actually talk to
[12:34] smart things to matter to other such systems
because to create a really good experience
[12:41] I want to control the room not only the
audio and the video I want to control the
[12:45] lighting or in the car the HVAC to create
the experience that my user actually needs.
[12:51] Now some of you are mo most probably in cyber
security and you look at this picture and you say
[12:56] oh my god and I can agree because the
attack surface of such a system is massive.
[13:06] the parameter that you now need to defend
with all these connections inside the system,
[13:11] outside of the system. We can try to guard
rail all of this, but if we getting too tight,
[13:18] then the system is actually losing its agency
and the magic goes away. If we don't do that,
[13:23] actually it might overstep and we are opening up
for bad actors. So one of the key design choices
[13:29] that we made is that trust and security are
a boundary condition and no longer an option.
[13:39] Now it's very easy for me to stand here and say
okay this is it this is the problem everyone not
[13:46] off we go next session. Um most probably that
would not be very helpful. So let me tell you
[13:51] a little bit about what we found on our journey
so you might not have to make the same mistakes.
[13:59] The first and foremost is like this is the
picture from the beginning, right? We firmly
[14:03] believe that the world of the abentricity
is gone and everything goes multimodal.
[14:08] Will the app move and and go away just tomorrow?
No, most probably not. But the advent of the
[14:14] voice interaction together for example with bio
signals in our world bio signals are a big thing.
[14:19] If I know how stressed my user is or how relaxed,
I can adjust actually my experience. I can also
[14:26] test in real time how successful the decision of
my orchestration AI was. The challenge if you ever
[14:33] build a system like this, I guess you know is
everything about latency. What keeps me up at
[14:38] night is latency. After 500 milliseconds
the user disengagements whether this is
[14:43] from utterance to utterance so the machine or the
user is asking something the machine is answering
[14:48] or is utterance to action where the user is asking
for so for example play this and that in my living
[14:54] room and if it takes seconds and seconds it's like
the user is asking himself or herself did it work
[15:01] right is something wrong if you consider a
sequence flow of such a single task there are
[15:07] hundreds of tasks going up and down between the
edge devices and multiple cloud components to make
[15:13] the single thing work. So every 10 milliseconds
of optimization are very valuable to us.
[15:21] The way how we approach it is that we decided for
a very strong hybrid system architecture where
[15:28] the edge itself is quite powerful in the
constraints of the bill of material. We are
[15:33] talking about consumer devices and those consumer
devices are optimized on the bill of material
[15:39] down to the scent. And then we optimize also for
example token usage because again this can become
[15:46] very much cost prohibitive in a world where a
headphone of $100 is also joining your ecosystem
[15:54] and is incurring for example token usage.
So again this is down optimized to descent.
[16:01] The second thing and this is for me the most
exciting moment I think for the last two decades
[16:07] in consumer electronics in automotive
is really orchestration AI.
[16:11] What this will do to the user experience we
can see already the first indications today the
[16:16] the probability that this will land is really
really high. We are preparing for that.
[16:22] This is a technical challenge. Yes. But however,
the biggest challenge you're going to face is the
[16:30] understanding of the organization. AI companies
most probably are not the consumer parsing,
[16:37] UX parsing company at their core, while the
consumer electronics companies are not necessarily
[16:44] the AI companies at their core. Nonetheless, you
need both of them. If you don't have both of those
[16:49] insights and both of the knowledge and both of the
expertise you will not build the adequate system
[16:56] and then eventually where we are very clear the
experience needs to be cross domain. The magic
[17:02] happens when you stop listening to music at your
home on your power system. You go into your Volvo,
[17:09] your power system greets you, say, "Shall we um
complete now the listening session where we where
[17:14] you dropped off? And by the way, shall I bring
over your preferred audio settings into the car?"
[17:21] This is very powerful once you experience such a
demo. So to sum this up, I think you get already
[17:32] an idea that this is a big journey and this is
a journey that we believe we cannot walk alone.
[17:42] So if you believe based on what you saw that
hey I have a solution that would really help
[17:48] such a system, I have technology, I have content
or I simply want to be part of creating that.
[17:55] Well, you should take a photograph,
right? Use the QR code, send us a ping,
[18:01] and then let's start with the conversation. And
with that, I think I am 1 minute 16 down. You're
[18:07] running a very tight ship, guys. And that is
it. Everything I had. Thank you very much.
