Muratori Scraps Retained GUI Trees for Direct Per-Frame Execution

Muratori弃用保留模式GUI树，转向逐帧直接执行 消除冗余的控件状态并逐帧以过程化方式评估控件，不仅去除了回调同步的样板代码，还恢复了统一的应用程序控制流。 Muratori详细介绍了即时模式图形用户界面（IMGUI）的架构。该范式由他在RAD Game Tools开发工具期间开创，旨在取代传统保留模式框架中脆弱的回调、冗余的状态缓存以及样板代码。该系统直接类比即时模式3D图形管线，利用标准编程语言的控制流，在每一帧同时评估布局、渲染和用户输入。通过利用带有“hot”和“active”控件ID的极简全局上下文来跟踪交互，应用程序无需复杂的控件对象层级即可完全掌控自身数据。尽管该方案带来了一帧布局延迟以及在C/C++中需要应用层进行粗粒度剔除等权衡，但它极大地简化了自定义控件、动画动态界面以及3D渲染集成的实现。 Casey Muratori正式阐述了即时模式图形用户界面（IMGUI），这是他在2002年左右开发的一种替代性UI范式，它摈弃了保留模式的控件层级、初始化例程和事件派发回调，转而采用逐帧过程化执行。 (0:01) IMGUI系统的整个交互模型仅由一个跟踪两种状态的全局UIContext管理——即“hot”（悬停/即将发生交互）和“active”（当前正在交互）——从而消除了非活动控件中缓存的冗余状态。 (12:02) IMGUI通过将UI视作命令式绘制流来契合现代图形管线，无需场景图同步即可无缝嵌入3D渲染循环和动态动画变换。 (20:43) 关键的工程权衡包括：在缺乏内省特性的C/C++等语言中，全局布局计算可能存在一帧延迟，同时粗粒度剔除的职责也转移到了应用程序代码上。 (32:46)

Casey Muratori introduces immediate-mode graphical user interfaces (IMGUI), arguing that treating UI like immediate-mode graphics—re-evaluating input and drawing procedurally every frame—eliminates the fragile state synchronization, callbacks, and boilerplate that plague traditional retained-mode frameworks. The Retained-Mode Breakdown and Graphic Parallels (3:18) Traditional retained-mode GUI architectures fragment program execution across three distinct phases: initialization, an opaque internal update loop, and asynchronous event callbacks. In this model, developers must register control IDs, coordinate dynamic creation and destruction calls across files, and continuously synchronize widget data copies with application memory. Muratori compares this friction to the graphics industry's shift away from retained-mode scene graphs in the 1990s and early 2000s, where game engines replaced rigid object hierarchies with immediate per-frame rendering loops to maintain flexibility and direct control. Unified Procedural Control and Redundant State (9:25) In immediate mode, interface elements are evaluated inline within the main application loop, allowing standard language control flow (such as `if`, `switch`, and `for` constructs) to govern control visibility and dynamic lists directly. If a condition changes or an item is culled, simply bypassing the widget function call removes its visual presence and disables user interaction without requiring explicit deletion or state synchronization. Muratori argues that retained-mode widgets hoard redundant state—such as multiple edit boxes storing individual cursor and selection data even though a user interacts with only one at a time. An immediate-mode architecture exploits this by collapsing interaction tracking into a single global `UIContext`. Widget interactions are managed using two core identifier fields (`UIID`): `hot` (the item hovered or highlighted for interaction) and `active` (the item currently being manipulated, such as holding down a mouse click). Dynamic Layouts and Direct 3D Subsystem Integration (29:01) Because the user interface hierarchy is submitted anew on every frame, immediate mode naturally accommodates dynamic, animated layouts. Widgets can track dynamic 3D joint transforms or floating animations without encountering partitioning bugs or coordinate synchronization issues. Furthermore, custom controls can bypass conventional 2D axis-aligned bounding box hit-testing to invoke 3D renderer ray-casting or picking routines directly, allowing 2D and 3D UI flows to unify into a single procedural pass without subclassing complex vendor frameworks. Language Limitations and Engineering Trade-offs (32:46) The immediate-mode model introduces specific trade-offs, largely driven by the lack of reflection and introspection in languages like C and C++. Because an application cannot inspect the aggregate dimensions of a UI tree without executing it, global layout calculations that require advance sizing must either accept a single frame of latency or introduce multi-pass execution (measuring before rendering), which adds architectural complexity. Additionally, because the interface lacks a static scene graph, fine-grained dirty-rectangle repainting is difficult, requiring the application developer to implement coarse-grained culling by skipping inactive panels. Muratori notes that while retained-mode toolkits have benefited from decades of industrial evolution, his implementation of IMGUI was assembled ad hoc during spare time to meet practical game development needs. Given its success in those environments, he expects the paradigm to mature as broader community experimentation and outside writing—such as upcoming articles by Sean Barrett—address its remaining limitations.

Casey Muratori 提出立即模式图形用户界面（IMGUI）范式，借鉴实时计算机图形学的渲染理念，通过在主循环中以内联过程化调用的方式合并交互判定与界面绘制，旨在消除传统保留模式 UI 中繁复的层级对象管理与冗余状态同步问题。 传统保留模式UI的生命周期割裂与状态冗余 (3:18) 传统保留模式（Retained-Mode）UI 架构通常将代码拆分为三个相互脱节的阶段：初始化阶段、内部更新阶段与事件回调阶段。开发者必须使用可视化编辑器生成布局文件，或通过大量样板代码显式实例化按钮、对话框等控件并注册全局 ID。在运行时，UI 库将自身封闭为黑盒处理点击与排版；一旦产生用户输入，系统便触发异步回调，开发者被迫在不同的事件处理函数中根据 ID 查找控件指针，手动将控件状态同步回应用程序底层数据。此外，动态隐藏、启用或销毁控件也需要在各种回调里穿插操作，导致控制流极其分散。 Muratori 进一步指出，保留模式在各个控件中驻留了大量冗余状态。例如一个普通的单行输入框，即使在非焦点状态下也完整维护着光标位置、选择高亮与鼠标拖拽历史，甚至还复制了一份本属于应用层的数据字符串。但物理上用户同一时刻只能与一个控件交互，这种让每个控件都持有一套完备交互状态的设计不仅带来了代码维护负担，还极易导致数据不同步与性能浪费。 核心交互模型：Hot与Active状态机制 (12:02) IMGUI 的核心是将所有瞬时交互状态集中于一个全局轻量结构 UIContext 中，彻底剥离控件对业务数据的占有权。应用数据始终保存在原生内存结构中，UI 仅负责每帧按需读取并呈现。为了处理复杂的鼠标与键盘交互，系统抽象出两个基础状态字段：hot 与 active。hot 指向当前光标悬停或处于高亮焦点、即将被操作的控件，负责提供视觉反馈；active 则指向用户已实际发起输入动作（如按下鼠标左键未放）的控件，该控件将独占后续输入流，直至鼠标释放。 在此模型下，控件代码被精简为纯粹的过程化函数（如 DoButton）。函数在每帧执行时就地完成激活状态检查、鼠标松开判定以及边界命中测试；若判定点击成功，函数直接返回布尔值 true，使业务逻辑在调用点当前的作用域内立即执行。由于界面逻辑不跨文件持久化，系统无需笨重的全局 ID 映射表，而是直接利用函数指针、父级数据结构指针或循环索引作为临时标识符，大幅降低了接口的胶水代码量。 过程化控制流与3D子系统的无缝集成 (27:01) IMGUI 与现代 3D 渲染管道的运作方式高度同构。正如现代游戏引擎摒弃了静态场景图（Scene Graph）并转为每帧动态遍历实体以提交几何体，IMGUI 允许开发者直接借助原生语言的控制流来构建界面。条件渲染无需依赖事件通知和树节点的挂载与拆卸，只需编写原生的 if 或 switch 语句；生成动态列表也仅需遍历底层数据数组并调用控件绘制函数。若某项元素因逻辑改变而无需显示，程序只需跳过该帧的调用，其对应的视觉表现与交互行为便自动消失，彻底消除了界面存续性与业务逻辑之间的同步鸿沟。 这种设计特别适合游戏及专业 3D 创作工具的开发。传统面向对象 UI 极难与非 UI 子系统融合，往往导致 2D 界面与 3D 视口采用两套完全割裂的交互框架。而在 IMGUI 中，控件可以绕过传统的 2D 矩形命中测试，直接内嵌 3D 渲染器的视口射线拾取（Ray-picking）逻辑。同时，界面元素能够直接读取骨骼变换矩阵等动态坐标，在每一帧随角色骨骼流畅运动，避免了保留模式在处理高动态、富动效界面时产生的沉重追踪开销。 架构权衡：语言内省缺失与粗粒度剔除 (32:46) 尽管开发效率显著提升，IMGUI 在 C 和 C++ 等缺乏原生内省、代码即数据（Code-as-data）和闭包支持的编译型语言中仍存在固有的工程妥协。由于系统必须实际执行代码才能知晓界面的布局与尺寸，如果某个父容器需要预先测量整体子控件的外围边界来进行自适应排版，往往会产生一帧的排版延迟（Layout Latency）。虽然开发者可以通过在 UIContext 中引入多阶段标记（先执行测量遍再执行绘制遍）来消除延迟，但这不可避免地增加了系统复杂度，偏离了 IMGUI 的极简原则；不过在实时交互环境中，单帧延迟通常极难察觉。 另一项权衡在于渲染管线的重绘成本。保留模式能够利用稳定的对象树精确计算并仅重绘局部脏矩形（Dirty Rectangles），而动态生成的 IMGUI 则缺乏这种细粒度隔离能力。因此，IMGUI 将防范重复计算与跳过隐蔽区域的责任转移给了宿主应用程序——开发者必须效仿 3D 图形引擎的视锥剔除，在应用层面手动执行粗粒度裁剪（如判断窗口关闭或移出屏幕时跳过整个 UI 代码块的执行），以保障复杂界面的运行效能。 Muratori 总结指出，传统保留模式 UI 历经数十年学术界与工业界的演进，而 IMGUI 当时仅由他利用业余时间在实践中摸索完善，却已在实际项目中展现出极高的可用性与精简优势。他认为随着更广泛社区的尝试与探讨（例如 Sean Barrett 当时计划在《Game Developer Magazine》发表的系列解析与示例代码），开发者将逐步攻克边缘工况，推动即时模式理念在图形界面工程中获得更深入的发展。

## Background 该视频由 Casey Muratori 于 2016 年 5 月 12 日上传至 YouTube 平台（记录的是其 2005 年的早期分享）。视频简介提供了其个人博客的延伸阅读链接，整体内容围绕他在 2002 年左右开创并实践的“即时模式图形用户界面”（IMGUI）设计范式展开。 ## TL;DR Casey Muratori 深入对比了传统保留模式 UI 与即时模式 GUI（IMGUI）的核心架构差异，尖锐指出了传统模式中割裂的状态同步、冗长样板代码和繁琐回调带来的工程灾难。他结合代码实现剖析了 IMGUI 的核心工作流：利用极简的全局 UI 上下文管理 Hot（悬停就绪）与 Active（交互激活）状态，将控件的声明、排版、绘制与事件触发直接收敛进应用程序的主执行路径中。此外，演讲客观探讨了 IMGUI 在 3D 渲染和动态动效集成上的天然优势，坦诚剖析了其在 C/C++ 缺乏语言自省下的一帧延迟与局部重绘难题，并呼吁开发者共同推进该领域的演化。 ## Atmosphere & style 整场讲座透着极具极客风格的务实感与松弛感，Casey 站在镜头前仅凭几张手绘白纸草图侃侃而谈，没有任何花哨的幻灯片包揽修饰。他的语言坦诚、锐利且充满工程师的直觉洞察，对传统保留模式界面的设计缺陷吐槽精准而不失幽默，谈及 IMGUI 带来的简洁与解脱时则难掩兴奋。整场分享节奏连贯、抽丝剥茧，带给观众一种在车库白板前听资深黑客传授底层设计秘籍的纯粹智性享受。 ## Bulletin - [00:01](https://www.youtube.com/watch?v=Z1qyvQsjK5Y&t=1s) Casey Muratori 提出即时模式图形界面（IMGUI）范式，将 UI 声明、渲染与事件判定整合进统一的单向执行流，彻底消除了传统保留模式界面中初始化、黑盒更新与事件回调割裂的架构灾难。 - [09:25](https://www.youtube.com/watch?v=Z1qyvQsjK5Y&t=565s) IMGUI 将交互状态简化为极简的全局 UI 上下文，利用“用户同时间只能与单一控件交互”的事实，仅通过 hot（悬停预备态）与 active（激活态）两个核心状态就足以处理完整的复杂交互流程。 - [23:55](https://www.youtube.com/watch?v=Z1qyvQsjK5Y&t=1435s) 由于不需要维护持久化的控件对象树，IMGUI 依靠普通控制流（如 if、for 循环）就能天然实现条件渲染与零同步，但受限于 C/C++ 缺乏代码自省能力，在复杂自适应布局时通常需接受一帧的时间延迟。 - [29:01](https://www.youtube.com/watch?v=Z1qyvQsjK5Y&t=1741s) 该范式能与 3D 射线拾取、实时骨骼动画系统完全无缝整合，避免了传统图形软件中 2D 与 3D UI 系统各自为政、难以复用的垃圾设计问题。 ## Key takeaways - [03:18](https://www.youtube.com/watch?v=Z1qyvQsjK5Y&t=198s) 传统保留模式 UI 将初始化、底层黑盒渲染与异步回调割裂为孤岛，迫使开发者编写数百行无业务价值的胶水代码来维持控件状态与数据同步。 - [06:31](https://www.youtube.com/watch?v=Z1qyvQsjK5Y&t=391s) 即时模式 GUI（IMGUI）仿效了现代图形渲染从保留模式到即时模式的演进，取消控件私有状态，单帧按需根据应用自身数据就地构建界面。 - [09:25](https://www.youtube.com/watch?v=Z1qyvQsjK5Y&t=565s) 传统控件内部充斥着庞大的冗余状态（如闲置输入框维护的交互计时与鼠标状态）；IMGUI 抓住“用户同一时刻只能操作单一对象”的物理事实，通过单点全局状态大幅精简系统。 - [12:00](https://www.youtube.com/watch?v=Z1qyvQsjK5Y&t=720s) IMGUI 核心交互模型建立在 hot（即将交互/高亮）和 active（正在交互/捕获输入）的分离之上，极大降低了控件状态管理的复杂度。 - [13:29](https://www.youtube.com/watch?v=Z1qyvQsjK5Y&t=809s) IMGUI 的控件标识（UI ID）无须头文件预先枚举，可通过函数指针、调用者对象地址或循环索引就地充当盲数据生成，与 OpenGL 反馈缓冲区理念一致。 - [16:49](https://www.youtube.com/watch?v=Z1qyvQsjK5Y&t=1009s) do_button 的底层逻辑仅通过比对当前 ID 与全局 active/hot 状态、检测鼠标按键抬起与落点即可裁决点击，无需事件监听器或跨帧消息队列。 - [18:23](https://www.youtube.com/watch?v=Z1qyvQsjK5Y&t=1103s) 有效的按钮点击语义依赖双重校验：在按钮区域内按下确立 active，且在松开鼠标时依然保持 hot 状态才算成功触发。 - [21:26](https://www.youtube.com/watch?v=Z1qyvQsjK5Y&t=1286s) 在 IMGUI 体系下，应用代码在当前帧不调用某个控件，该控件的交互与绘制便即刻凭空消失，无需传统 UI 繁琐的销毁、注销或隐藏操作。 - [23:55](https://www.youtube.com/watch?v=Z1qyvQsjK5Y&t=1435s) IMGUI 带来三大核心架构优势：逻辑高度集中的单点代码、与业务结构完全重合的统一流程控制，以及无需调用 SetWindowText 式的绝对零状态同步。 - [27:01](https://www.youtube.com/watch?v=Z1qyvQsjK5Y&t=1621s) 动态显示控件或根据数值排布界面，在 IMGUI 中仅仅是一条普通的 if 语句或循环，消除了传统框架下层层注册回调修改显隐的复杂灾难。 - [29:01](https://www.youtube.com/watch?v=Z1qyvQsjK5Y&t=1741s) IMGUI 可以直接嵌入 3D 渲染循环，利用 3D 射线拾取和实时动画骨骼数据驱动按钮位置，消灭了 2D 与 3D 交互系统割裂的架构缺陷。 - [32:48](https://www.youtube.com/watch?v=Z1qyvQsjK5Y&t=1968s) C/C++ 缺乏运行时自省（代码即数据）是 IMGUI 面临的主要局限，导致系统无法提前获知 UI 全貌，必须通过执行代码才能计算总尺寸，因而引发一帧布局延迟（One Frame Lag）。 - [36:15](https://www.youtube.com/watch?v=Z1qyvQsjK5Y&t=2175s) IMGUI 的动态特性使得增量局部重绘（Dirty Rectangles）更难在框架层自动实现，复杂的渲染开销优化需要应用层在粗粒度级别自行进行类似视锥剔除的控制。 - [36:55](https://www.youtube.com/watch?v=Z1qyvQsjK5Y&t=2215s) 尽管当时 IMGUI 缺乏商业级宣传和完善标准库，但个人短时间实践所展现出的高可用性，充分印证了该架构替代传统复杂面向对象界面的巨大潜力。 ## Detailed walkthrough ### [00:01](https://www.youtube.com/watch?v=Z1qyvQsjK5Y&t=1s) 视频缘起与即时模式GUI的诞生背景 Casey Muratori 在视频开篇向大家问好，并表示希望这能成为一系列小型编程视频的开端。他在 Molly Rocket 负责新游戏的开发，工作中经常尝试许多异想天开、前沿大胆的技术。然而，撰写正式的学术论文或录制高规格的专业讲座极其耗费时间，为了将那些他认为真正成功且有趣的全新方法传播出去，他决定采用最务实低廉的形式：在巨大的白纸上画简易图纸，站在摄像机前直接讲解。他探讨的这项技术实际上早在 2002 年左右便已成型，被称为“即时模式图形用户界面”（Immediate-Mode Graphical User Interfaces，简称 IMGUI）。无论是通用软件还是游戏应用，这种界面编写范式都远比当时主流的面向对象、类库驱动的设计方式更加易用、直观且条理清晰。 在此之前，Casey 曾在 Windows、X Window 以及各类自定义图形系统中编写过大量用户界面代码，但体验始终非常糟糕。传统的 UI 编写过程令人极其厌烦，开发者往往被迫将大把时间浪费在低效的控件布局工具上，还要艰难处理分散在各处的事件回调、控件 ID 以及复杂的组件生命周期绑定。这种架构导致哪怕只是在屏幕上呈现一个极其简陋的界面，都需要堆砌海量令人头疼的代码。2002 年他在 RAD Game Tools 为 3D 骨骼动画引擎 Granny 开发模型查看器（Granny Viewer）时，面对其中繁杂的界面需求，他突然顿悟并意识到现有的 UI 流程完全可以大幅精简；尝试后的效果彻底颠覆了以往的开发模式，自那之后，即时模式成了他编写 UI 的唯一选择。 正如 Casey Muratori 回顾这段经历时总结道：“I've done user interface coding several times in the past and I was never happy with the way it came out.” - 视频制作背景：Casey 在 Molly Rocket 研发游戏期间尝试了大量激进新技术，因缺乏时间写论文而选择用白纸手绘草图录制短视频。 - 技术起源：大约在 2002 年于 RAD Game Tools 开发 Granny 3D 查看器时首次构想并实践了这一架构。 - 适用场景：不仅限于常规应用程序界面，在游戏内置 UI 以及各类自定义渲染工具中均可通用。 --- ### [03:18](https://www.youtube.com/watch?v=Z1qyvQsjK5Y&t=198s) 传统保留模式UI的困境与反思 为了对比即时模式的优势，Casey 详细拆解了传统（保留模式）UI 库的标准工作流程。在传统 UI 框架中，整个生命周期被割裂为几个互不连贯的孤岛阶段：首先是在初始化阶段（Init Time），应用程序需要调用各种 `create` 函数，或者加载界面编辑器生成的配置文件，一口气实例化所有的对话框、按钮与属性页。紧接着进入更新渲染阶段（Update Time），但此时控制权完全被 UI 框架接管，应用代码根本不知道、也不关心框架内部是如何重绘控件的，完全沦为一个黑盒。最后是事件响应阶段，当用户点击按钮或展开菜单时，框架会通过预留的回调函数（Callbacks）或消息队列通知程序。 这种分散的架构带来了一系列严重的工程灾难。应用程序在收到回调后，往往需要借助外部 ID 去框架内部查询控件的当前数值，或者根据业务状态在运行途中动态创建、销毁、启用或禁用其他控件。由于定义、渲染与业务逻辑散落在编辑器、初始化代码和各个回调函数中，唯一的纽带就是共享的控件 ID。Casey 直言，优秀的 UI 库也只能勉强缓解这些痛点，而在诸如微软那一套工具链中，开发者往往要写上几百行毫无实际业务意义的胶水代码，仅仅是为了把这些支离破碎的控件状态串联在一起。 Casey Muratori 在批评传统工具的混乱时指出：“You end up writing hundreds of lines of code that essentially does nothing other than keeping all of this stuff kind of wrangled together.” ```mermaid flowchart TD accTitle: 传统保留模式UI的工作流程 accDescr: 展示传统保留模式UI在初始化、黑盒更新与事件回调阶段之间的分散状态与数据交互 subgraph InitStage["初始化阶段 (Init Time)"] A["加载UI配置 / 调用Create函数"] --> B["实例化Button、Dialog、Property Page等控件"] end subgraph UpdateStage["更新/渲染阶段 (Update Time)"] C["UI框架黑盒处理"] --> D["内部绘制按钮与控件"] end subgraph EventStage["事件响应阶段 (Callbacks / Messages)"] E["接收回调/消息"] --> F["使用全局/共享ID查询控件值"] F --> G["动态创建、销毁或禁用其他控件"] end B -.->|"转交控件管理权"| C D -.->|"用户触发事件"| E G -.->|"修改运行时状态"| C ``` - 传统架构的核心弊端：初始化、渲染黑盒与回调处理高度分散，状态难以统一追踪。 - 隐形依赖：业务层与 UI 层严重依赖外部生成的数字 ID 或名称进行弱关联绑定。 - 冗余代码量：微软等传统 UI 体系需要开发者编写大量模板与胶水代码来维系状态一致性。 --- ### [06:31](https://www.youtube.com/watch?v=Z1qyvQsjK5Y&t=391s) 即时模式GUI的核心机制与图形学类比 在对比传统模式之后，Casey 展现了即时模式 GUI（IMGUI）截然相反的代码形态。在即时模式下，顾名思义，一切都是“即时处理”的——这里没有专门的 UI 初始化代码，更没有任何繁琐的异步回调函数，整个 UI 的驱动完全依托于由开发者亲自书写的主更新路径（Main Update Path）。在这条主执行线上，UI 的声明、绘制与逻辑触发在同一时刻完成。例如，只要写下一行简单的条件语句 `if (DoButton(...))`，该函数不仅负责在屏幕对应位置绘制按钮，还会立即返回布尔值：如果被用户点击了就返回 `true`，否则返回 `false`。开发者可以直接在 `if` 块内部编写点击后的响应动作，彻底终结了跨文件、跨函数的事件跳转。 不仅如此，开发者能够毫无限制地使用编程语言原生提供的各种控制流结构。你可以使用标准的 `if` 语句根据程序状态切换界面，利用 `switch` 拆分页面，甚至能在一个 `while` 或 `for` 循环里一次性动态绘制出整排按钮。这与图形学领域从早期保留模式图形 API（Retained-mode Graphics）演进到现代即时模式绘制完全是同一个道理。在图形学中，没人愿意先通过一大堆初始化配置静态构建所有几何体再全权交给渲染器；相反，开发者更希望在自己的主循环中根据当下数据直接提交绘制调用。即时模式 UI 彻底摒弃了控件对象对状态的私有存储，让界面完全根据应用自身的数据随时按需构建。 Casey Muratori 一针见血地概括了两者的本质差异：“That's the only key difference between immediate mode user interfaces and standard user interfaces is they're just trying to do what immediate mode graphics did as compared to retained mode graphics because we all pretty much know how hard retain mode graphics suck.” - 代码直观性：通过 `if (DoButton(...))` 将控件渲染与事件触发就地合并，省去中间状态存储与消息监听。 - 原生控制流：直接通过语言自身的条件分支（`if`、`switch`）与循环（`for`、`while`）精确控制 UI 的生成与排布。 - 数据驱动：UI 状态不存储在控件黑盒中，而是直接与应用程序自身的实时数据绑定，单帧即时计算。 ### [09:25](https://www.youtube.com/watch?v=Z1qyvQsjK5Y&t=565s) 传统保留模式中的冗余状态与 UI Context 的引入 Casey Muratori 指出，立即模式 GUI（Immediate-Mode GUI，IMGUI）有无数种具体实现方式，他所分享的只是自己的惯用方案。这种方案的核心是一个被称为 `UI context` 的结构体，其核心职责是在任意给定时刻追踪用户与系统的交互状态。 许多人在审视图形用户界面（即便是复杂的界面）时，往往忽略了一个关键事实：传统的保留模式（Retained-mode）GUI 中充斥着惊人且庞大的冗余状态。Casey 以界面中的文本输入框（Edit box）为例进行了生动说明：在一个保留模式界面中，即便用户当前完全没有和某个输入框进行任何交互，这个输入框依然携带着为了编辑文本所必须维护的全部状态与包袱。比如屏幕上有两个输入框，用户绝不可能同时在两者里面打字编辑；那个处于闲置状态的输入框，此时此刻所唯一需要知道的仅仅是文本内容本身，以便完成绘制，但它内部往往还不得不保存着鼠标状态、上次点击时间戳等一堆交互数据。 虽然在普通的桌面 GUI 中，内存占用通常并不算什么生死攸关的大问题——不过 Casey 也调侃道，用过 Visual Studio 的人肯定都遇到过点击展开某个东西时弹窗提示“展开可能需要很长时间，确定继续吗”的糟糕体验。Casey 强调，指责冗余状态浪费内存并非重点，重点在于我们可以充分利用“用户在同一瞬间只能与单一控件交互”这一物理事实。既然交互的唯一性是天然存在的，我们完全可以将其收拢为一个全局性的单点结构，只在里面记录当前交互状态，从而写出极为精简、高效且易于理解的代码。应用程序的数据原本就归属于应用程序自身（输入框中的文本在应用数据结构中有具体含义，本就不该冗余复制在 UI 控件内部），UI 本身根本不需要长期驻留这些数据，它真正需要记录的仅仅是交互发生时的极简状态。 > Casey Muratori 总结道：“The point is that redundant state is actually something that can be exploited to make a much more efficient piece of code that's simpler and easier to understand and what that is is you can say well you know what the user can only be interacting with one thing in a given time so let's just make a single structure that's kind of global to the user interface that's the state of user interface that is the state of the user's interaction because really that's all we need we don't need anything else.” ```mermaid flowchart TD accTitle: IMGUI 交互状态核心概念 accDescr: 展示用户输入如何驱动 UI Context 中的 Hot 和 Active 状态转换 subgraph InputState["输入感知"] Hover["鼠标悬停 / 键盘焦点到控件"] MouseDown["鼠标在 Hot 控件上按下"] MouseUp["鼠标在同一控件上松开"] end subgraph UIContextState["UI Context 状态标识"] direction TB Hot["Hot Item (悬停预备态)\n展示高亮视觉效果"] Active["Active Item (激活交互态)\n正在捕获用户动作"] end subgraph AppResult["交互裁决"] TriggerAction["触发点击事件 (返回 true)"] end Hover -->|"捕获 ID"| Hot Hot -->|"按下"| Active Active --> MouseDown Active -->|"松开且仍 Hot"| TriggerAction MouseUp -->|"重置 Active"| Hot ``` --- ### [12:00](https://www.youtube.com/watch?v=Z1qyvQsjK5Y&t=720s) 控件交互模型：Hot 与 Active 状态的分离 聚焦到最简单的按钮（Button）或菜单项（Menu item）逻辑上，UI Context 最核心的成员主要有两个：`hot`（焦点/悬停项）和 `active`（当前激活项）。这种抽象不仅适用于按钮，更可以推广到任何类型的复合控件。 在设计逻辑上，用户与界面元素的交互天然被切分为前后衔接的两个阶段： 1. **“将要发生交互”的就绪状态（Hot）**：例如用户的鼠标指针移动到了某个控件上方，或是通过键盘 Tab 键将焦点导航到了该元素。此时控件的视觉表现通常需要高亮显示，以给予用户明确的视觉反馈，提示“你可能正打算与我互动”。 2. **“正在发生交互”的激活状态（Active）**：当用户明确发起动作（例如在处于 Hot 状态的按钮上按下了鼠标左键），该元素便升格为 Active。这代表用户已经进入实际操作阶段，界面逻辑必须锁定该元素来捕获后续输入。 通过将交互拆解为 Hot 和 Active 两个简单的标识，系统无需为成百上千个界面元素各自分配独立的事件监听器和上下文存储，只要维护全局的一对状态，就能完成按钮从鼠标掠过高亮、按下凹陷、拖拽移出再移入，直到松开触发的全部完整体验。 > Casey Muratori 针对这两个关键状态给出定义：“So hot means like about to be interacted with and active means I'm actually interacting with it okay and we'll see why we how we use these two things in a second but that's basically the the majority of the information that we need to store for this for basic UI.” --- ### [13:29](https://www.youtube.com/watch?v=Z1qyvQsjK5Y&t=809s) 立即模式下的轻量级元素标识（UI ID） 在传统的保留模式界面中，控件通常依赖一个预先全局注册或在头文件中统一定义的唯一 ID（比如控件句柄或全局常量）。之所以需要这种全局显式 ID，是因为传统的架构往往把界面构造代码与事件响应、数据提取代码割裂在完全不同的模块甚至文件之中；开发者在 A 处创建控件，必须在 B 处的事件回调里用该 ID 跨模块寻址并读取控件状态。 而在立即模式 GUI 中，这种跨模块数据拉取的通信需求根本不存在。所有界面的渲染、排版和状态判定全都在同一帧的代码流程里一气呵成，底层 GUI 并不持有应用层业务数据，复杂的逻辑关联直接在应用程序自身的数据侧完成。因此，IMGUI 对标识符的唯一诉求，仅仅是为了在“上一帧与这一帧之间”辨识用户正在与哪个具体的控件互动。 Casey 表达了自己对编写冗长头文件来枚举 ID 这种传统做法的厌倦。在 IMGUI 中，控件标识（UI ID）可以通过调用端随手可得的上下文信息直接就地生成： - **函数指针（Function Pointer）**：如果一个按钮点击后只负责触发某特定函数，该函数本身的内存地址就可以天然作为全局唯一的 UI ID。 - **调用者指针（Owner Pointer）**：当一组相同功能的控件需要被明确区分时，可以传入驱动这组控件的父级数据指针，或者渲染函数的指针。 - **数组索引（Index）**：当通过循环动态遍历渲染一组数组列表时，直接将当前的索引值作为盲数据（Blind data）传入组成复合 ID。 Casey 进一步类比道，这种标识方式与 OpenGL 等立即模式图形渲染系统中的反馈缓冲区（Feedback Buffers）机制极为神似：底层库根本不在乎你传入的具体 ID 究竟是数字还是指针，它只把这些数据当成黑盒标识用于做等值比较，使得调用方可以极其自由、无开销地拾取具体元素。 > Casey Muratori 说明了 IMGUI 标识符与保留模式的本质区别：“For immediate mode UIs we don't really care about that so much because of the immediate mode nature of it we can always handle all of the interrelations the complex interrelations on the app side because it's all using app data we don't have to have there be ways of getting that information out of the GUI because GUI is not storing any of that information for you it's all clean and on your side.” --- ### [16:49](https://www.youtube.com/watch?v=Z1qyvQsjK5Y&t=1009s) `do_button` 的底层逻辑实现 Casey 随后以 UI 库内部的 `do_button` 函数为例，剖析了底层实际运转的代码逻辑。需要注意的是，这是 UI 库本身的实现代码，而不是上层业务应用调用的代码。业务侧只需关心 `do_button(...)` 返回的布尔值——为 `true` 则说明按钮被用户有效点击，应用立刻执行业务逻辑。 在 UI 库内部，一个极简按钮的处理逻辑清晰明了： 1. **Active 校验**：调用 `UI context` 查询当前按钮的 ID 是否与全局记录的 `active` ID 相匹配。如果一致，说明该按钮当前正处于被按下的活跃状态。 2. **释放判定**：在处于 Active 状态的前提下，检测用户是否在此帧松开了鼠标按键（Mouse Up）。若鼠标松开，逻辑进一步判断当前按钮是否同时依然是 `hot`（即松开鼠标瞬间，光标是否仍然停留在按钮区域内）。若仍为 Hot，说明这是一次完整的“按下并在控件内释放”的有效点击，函数将结果置为 `true`，以通知调用方执行响应行为。 3. **状态清理**：无论鼠标松开时当前按钮是否仍处于 Hot 状态，只要鼠标按键已抬起，按钮的 Active 状态都会被立即重置释放，交出交互控制权。 整套处理完全基于帧与全局状态的简单比对，没有回调函数的注册绑定，也没有跨帧的复杂事件队列，以极简的代码结构实现了完备的按钮交互语义。 > Casey Muratori 解释 Active 的判断方式时提到：“Basically we just say if we're the active item and in order to determine if we're the active item that's a call that usually goes into the UI context which Compares our ID versus the active ID and if they're the same then we're the active item and if they're not then we're not the active item very simple.” ### [18:23](https://www.youtube.com/watch?v=Z1qyvQsjK5Y&t=1103s) 按钮交互状态的处理：从 Active、Hot 到绘制渲染 Casey 深入解释了按钮行为的标准化定义：当用户在按钮上按下鼠标左键，并在释放时鼠标指针仍然停留在该按钮上方（即保持 `hot` 状态），该操作才被判定为一次有效的点击，函数返回 `true`。如果用户在按钮外释放鼠标，则不应被视为点击。“It's a standard UI thing obviously you don't have to do it that way but that's kind of the convention so it'd be weird if you did something else, your users would get a little confused although many games often don't implement that correctly, what are you going to do.” 如果当前控件尚未处于 `active` 状态但处于 `hot` 状态，逻辑只需检测用户是否按下了鼠标按键。一旦用户在 `hot` 状态下按下鼠标，该控件便成为 `active` 控件，进入后续帧的主动交互跟踪。至于判断光标是否在自身包围盒内（hit-test），只要确认命中，控件就会请求 UI 核心将其设为 `hot`。但 Casey 强调，`set hot` 与直接赋值不同，它会先检查当前全局是否已经存在 `active` 控件；如果有某个控件正在被交互拖拽，其他悬停的控件绝不能抢占高亮或进入 `hot` 状态。在完成这套仅几十行的输入裁定后，函数末尾执行常规的图形绘制，无论是画带圆角的矩形边框还是渲染文本，全部由底层库封装完成，调用方代码依然保持极致的精简。 ```mermaid flowchart TD accTitle: 立即模式按钮的交互决策流 accDescr: 展示立即模式按钮如何根据当前鼠标位置、Hot 状态及 Active 状态做出返回 true 或更新状态的判断流程 subgraph Evaluation ["状态判定阶段"] A["调用 DoButton(id, rect)"] --> B{"当前控件是否 Active?"} B -->|"是"| C{"鼠标按键是否释放?"} C -->|"是"| D{"光标是否仍在 rect 内 (Hot)?"} D -->|"是"| E["清空 Active; 返回 true (触发点击)"] D -->|"否"| F["清空 Active; 返回 false"] C -->|"否"| G["保持 Active 状态"] B -->|"否"| H{"当前控件是否 Hot?"} H -->|"是"| I{"鼠标按键是否按下?"} I -->|"是"| J["将当前控件设为 Active"] end subgraph HotUpdate ["Hot 状态竞争与绘制"] K{"光标在 rect 内且无其他 Active?"} K -->|"是"| L["设为 Hot 状态"] K -->|"否"| M["忽略 Hot 请求"] J --> N["绘制按钮视觉 (背景、文字、高亮)"] L --> N end ``` - 按钮的标准交互模型依赖双重验证：按下时光标在按钮内（确立 active），释放时光标仍在其内（处于 hot），方才视为有效触发。 - 复杂的控件（如文本编辑框 `edit box`）除了基础的 hot 与 active 标识，还需要在 UI 上下文中临时记录光标位置（cursor position）与文本选中范围（selection region）等中间状态，但整个状态机的核心骨架与按钮完全一致。 --- ### [21:26](https://www.youtube.com/watch?v=Z1qyvQsjK5Y&t=1286s) 立即模式图形与界面的同构哲学与应用端自主权 Casey 进一步将立即模式 GUI 与现代实时渲染管线进行类比。在过去，图形编程曾误入“保留模式”（Retained Mode）的歧途，但工业界最终认识到应用层必须对几何体的处理拥有完全的调度控制权：遍历每个物体与批次、切换材质、剔除遮挡物、优先排序半透明物体并在最后绘制，或者自行遍历 BSP 树。所有这些分支控制必须位于应用层，而非死板地沉淀在底层渲染器内部。“This code does not live in the renderer that was a very important part of the progress of graphical coding is realizing that retained mode was not a good thing because the way that the app needs to express how things get drawn is necessarily going to look like this.” 在立即模式 GUI 中，这一设计模式得到了原汁原味的继承。无论是弹出窗口、多级菜单还是三维场景里的交互元素，应用层可以自由决定循环遍历与调用顺序，完全不需要向 UI 库预先注册控件树。如果应用层某一行代码决定在当前帧不调用某个控件，该控件的交互逻辑与视觉渲染就会即刻凭空消失，无需经历传统 UI 繁复的“查找节点、调用隐藏函数、注销监听器或销毁内存”流程。底层 UI 库仅仅充当一组纯粹的交互行为与绘制原语的提供者，对业务流程为何出现、何时展现全然无须感知。 - 应用层可以直接通过迭代自身内部数据（无论格式是动态数组、配置文件加载而来的树状结构还是内存对象）动态生成 UI，无需预先创建任何对应的镜像控件节点。 - 对界面的动态“剔除”（Culling）变得极其廉价：仅需通过一个 `if` 条件语句略过函数调用，该控件在此帧中便不再接收点击，也不再消耗绘制资源。 --- ### [23:55](https://www.youtube.com/watch?v=Z1qyvQsjK5Y&t=1435s) 立即模式 GUI 的架构优势：单点代码、统一流程与零同步 谈及立即模式 GUI 相较于传统保留模式 GUI 的巨大优越性，Casey 提炼出了三大核心支柱：单点代码集中（Single Piece of Code）、统一流程控制（Unified Flow Control）与绝对的零状态同步（No Synchronization）。在传统框架下，定义一个界面往往分散在可视化编辑器布局文件、构造初始化代码、事件响应类与样式表等数个碎片中；而立即模式让控件的布局定位、渲染绘制、交互处理与触发后的业务响应全部聚拢在单一块代码里。开发者需要调整逻辑时，再也不用满项目检索，代码量呈数量级下降，杜绝了“一个控件对应一个对象实例”的类泛滥。 ```mermaid flowchart LR accTitle: 保留模式与立即模式的控制流对比 accDescr: 对比传统保留模式由 UI 事件循环被动调用与立即模式由应用主循环主动调用的架构流向 subgraph RetainedMode ["传统保留模式 (Retained Mode)"] direction TB R_App["应用程序数据"] <-->|"痛苦的双向状态同步"| R_Tree["UI 控件树对象"] R_Loop["底层 UI 消息队列 / 事件循环"] -->|"异步回调/发送通知"| R_App end subgraph ImmediateMode ["立即模式 (Immediate Mode)"] direction TB I_App["应用程序单向控制流"] -->|"每帧直接调用 DoWidget() 并就地取回结果"| I_Lib["无状态 UI 功能库"] I_App -->|"状态完全归属于业务逻辑"| I_App end ``` 更核心的跃迁在于控制权的倒置：传统 UI 框架强制要求应用程序屈从于框架自带的事件循环机制（Message Queuing / Callbacks），而立即模式让应用逻辑始终稳坐驾驶席。“You are always in control on the immediate mode system, you have a single path for your UI that works exactly the way you want that mirrors the structure behavior of your app and then you're always just calling out to the library and it immediately returns back to you the information that you need.” “There's no synchronization which just comes with any immediate mode system which is nice and this is including existence,” Casey 兴奋地指出。保留模式开发中令人窒息的开销几乎全来自于同步状态——调用大量的 `SetWindowText`、启用或禁用状态、设置可见性。而在立即模式下，业务数据即界面：哪怕只是因为某个游戏角色做了一个动作、或者全局计时器走过了 10 秒导致某个数据变量发生改变，下一帧只需分支执行调用另一个面板函数，前一个面板就瞬间化为乌有，开发者彻底摆脱了将内部状态费力对齐到外部 UI 模型的泥潭。 - **单点维护性**：逻辑高度局部化，消除了业务代码与界面对象绑定的中间粘合层（Glue Code）。 - **统一流控（Unified Flow Control）**：所有交互直接返回实时结果给调用者，代码结构与应用的业务执行树完全重合。 - **零存在性同步（No Existence Synchronization）**：控件的存在与否完全受普通控制流控制，无须为了管理生命周期编写枯燥且毫无实际信息价值的增删改查样板代码。 ### [27:01](https://www.youtube.com/watch?v=Z1qyvQsjK5Y&t=1621s) 复杂控件与动态关联的极简实现 在传统的保留模式 UI 库中，实现带有特殊交互行为的复杂定制控件往往极其痛苦。Casey Muratori 指出，当开发者需要为自己的应用程序编写某种高度定制的专属控件时，标准 UI 库的表现通常非常差劲。但在即时模式图形用户界面（IMGUI）中，这完全不是问题：“因为你随时随地都能轻而易举地写出自己的交互逻辑和控件代码”。开发者甚至可以直接调用自己的辅助函数库，而这一切唯一需要遵守的规则，就是保证它与 UI 上下文中的 `active` 和 `hot` 机制和平共处，不发生冲突。 一旦自定义控件检测到交互触发（例如鼠标按下），它只需将全局的 `active` 项设为自己；在它释放激活状态之前，界面上的其他所有交互都会自动挂起。开发者拥有绝对的自由：“无论你怎么渲染、无论你怎么处理鼠标，全都由你说了算”。 ```mermaid flowchart TD accTitle: IMGUI 控件状态与执行流 accDescr: 展示用户输入如何通过条件语句与 active/hot 状态直接驱动逻辑，无需复杂的回调机制。 subgraph Logic["常规应用逻辑与条件控制"] Check["条件检查: if (value == 5)"] Exec["直接顺序执行该区域内所有控件代码"] Check -->|"条件为真"| Exec end subgraph State["IMGUI 全局状态控制"] Input["接收用户交互输入"] ActiveCheck{"当前控件是否被激活?"} SetActive["设置 UIContext.activeItem = 当前控件ID"] BlockOthers["其他 UI 控件交互暂停响应"] Release["鼠标释放: 清空 activeItem"] Input --> ActiveCheck ActiveCheck -->|"是"| SetActive SetActive --> BlockOthers BlockOthers --> Release end ``` 同样，传统 UI 在处理控件间的复杂关系时也极易陷入混乱，比如动态重排按钮顺序、勾选复选框后隐藏大片面板，或是仅在某个变量达到特定数值时才显示一组控件。在传统体系中，这类需求通常需要注册监听回调，在回调中判断变量，再层层调用外部 API 去设置其他控件的显隐或强行销毁重建对象。Muratori 评价道：“在标准控件库里，这种事情通常被搞得无比复杂，……简直就是一场灾难”。而在 IMGUI 中，逻辑自然得就像日常写代码一样： - 如果想在某个数值等于 5 时才显示一整组控件，只需写一个普通的条件分支：`if (value == 5) { /* 调用相关控件代码 */ }`。 - “这就该是它原本的样子，因为你本来就没要求做什么复杂的事情。” - 界面层级与显隐状态不再需要双向同步，代码本身的结构就是 UI 的结构。 ### [29:01](https://www.youtube.com/watch?v=Z1qyvQsjK5Y&t=1741s) 游戏引擎、3D 渲染与实时系统的无缝集成 即时模式最让游戏程序员和 3D 编辑工具开发者兴奋的特性，是它能与完全不了解、甚至根本不想参与 UI 的底层系统实现完美集成。例如渲染器、射线检测（Ray picking）和动画系统，它们在设计之初从未考虑过图形界面，但它们天生就具备渲染自身和拾取物体的能力。 在 IMGUI 中，将一个 3D 渲染对象包装成可交互控件轻而易举：开发者只需编写一个自定义的按钮行为函数，把原本矩形碰撞测试的 `isInsideRectangle` 换成渲染系统的 3D 射线拾取调用即可。这段交互代码可以直接嵌入到极其复杂的渲染循环内部，与摄像机矩阵变换、材质绑定并列存在。而在传统面向对象的保留模式系统中，实现类似效果往往需要继承某个庞大基类，祈祷它提供了足够的消息响应钩子，但结果“几乎无一例外都无法如愿”，开发者不仅拿不到合适时机下的回调，连搭建上下文都困难重重。Muratori 甚至愤怒地吐槽：“以至于在专业的 3D 应用程序中，经常存在两套完全独立的系统来运行用户界面：一套负责 2D 界面，另一套负责 3D 界面。那简直是垃圾设计！为什么？不应该存在两套不同的系统，它们都只是用户界面而已。” 这种紧密契合同样延伸到了实时动画系统： - 每一帧程序都在重跑 UI 代码并执行绘制，数据是即刻消费的。 - 若要将一个按钮固定在动画角色的头顶，每一帧只需直接从动画骨骼系统读取当前位置坐标，随即就地调用绘制按钮代码即可。 - 整个流程单轨运行（Single Path），没有任何数据不同步或分层架构撕裂的烦恼。 - 面对现代界面对动态动效、浮动元素的高要求（如 macOS 系统动效或 Flash 交互），传统保留模式焦头烂额，而“即时模式图形界面对此毫无压力，它生来就是干这个的，甚至它做动态交互比做静态界面还要出色”。 ### [32:48](https://www.youtube.com/watch?v=Z1qyvQsjK5Y&t=1968s) 核心权衡：语言内省缺失与一帧延迟问题 任何架构选型都有代价，即时模式最大的权衡根源并非来自于模式本身，而是源于底层编程语言的局限。Muratori 指出，游戏开发与底层系统主要采用 C 和 C++，但这些语言“根本不尊重‘代码即数据’这一事实”。C/C++ 在运行时自省（Introspection）、作用域理解和闭包支持上极其贫瘠，“在这些场景下你基本上是在盲人摸象”。 由于缺乏对代码结构的外部探知能力，系统无法在不执行代码的前提下提前知道整个 UI 的全貌。“你必须实际执行 UI 代码，才能知道 UI 到底长什么样。”这带来了 IMGUI 最显著的一个架构缺陷： - **一帧延迟（One Frame of Lag）**：任何依赖全局布局（Layout）或依赖界面总尺寸才能进行定位的逻辑，都必须承受一帧的时间差。因为必须跑完所有布局代码，系统才能收集齐最终的尺寸信息并应用到下一轮计算中。 - **多通道规避方案（Multi-pass）**：虽然可以通过让 `UIContext` 区分“渲染通道（Render Pass）”和“布局通道（Layout Pass）”来消除延迟（在布局阶段 `doButton` 仅记录尺寸并通知布局引擎），但 Muratori 认为这打破了 IMGUI 纯粹、简单易实现的初衷。 - **实用主义态度**：Muratori 坦言自己更倾向于接受这一帧延迟，因为“在实时应用程序中，极少有什么东西是无法承受一帧延迟并正常工作的”。相比之下，保留模式能随意对静态树结构进行多次循环计算，在纯静态布局的表达能力上确实具备这种天然优势。 尽管存在一帧延迟以及让应用程序在需要局部刷新（不按每帧全量重绘）时负担略有增加的问题，Muratori 依然强调：“从如今的传统图形界面切换到即时模式图形界面，绝大多数地方都是彻头彻尾的胜利。” ### [36:15](https://www.youtube.com/watch?v=Z1qyvQsjK5Y&t=2175s) 渲染优化权衡与应用层剔除 当图形界面的视觉元素极其复杂、绘制成本高昂时，开发者往往不希望每帧都将整个界面重新绘制一遍，而是只想绘制发生变化的部分。Casey Muratori 指出，在即时模式图形界面（IMGUI）中实现这种增量绘制（Dirty Rectangles / 局部更新）要困难得多。因为 IMGUI 在本质上是依赖代码执行流来动态构建用户界面的，各个元素难以被彼此隔离以单独检测变化并触发局部刷新。 因此，如果应用程序确实需要这种极高要求的绘制优化，这一职责最终只能落到应用层代码上。Casey Muratori 将这种情况与 3D 渲染中的流程进行了对比：“And that's kind of akin to what happens in 3D app with culling and so forth: the application is responsible for doing those sorts of things at the coarse level, and the same would be true of IMGUI.” 应用程序需要像处理 3D 视锥剔除一样，在粗粒度级别自行决定哪些界面的代码路径需要执行或跳过绘制。 ```mermaid flowchart TD accTitle: IMGUI 粗粒度跳过与渲染流程 accDescr: 展示应用层如何自行判断是否执行 IMGUI 渲染分支，以实现类似于 3D 剔除的优化 subgraph AppLogic["应用层决策 (Coarse-level Logic)"] A["检测界面模块是否可见/需要更新"] -->|有变更或可见| B["执行 IMGUI 控件代码流"] A -->|无变更且隐藏/静态| C["跳过控件代码分支"] end subgraph IMGUIRender["渲染提交"] B --> D["生成绘制指令并提交渲染"] C --> E["沿用缓存或不绘制"] end ``` ### [36:55](https://www.youtube.com/watch?v=Z1qyvQsjK5Y&t=2215s) 即时模式 GUI 的潜力与演进展望 在总结 IMGUI 的优劣后，Casey Muratori 强调采用这种模式依然有非常多极其强大的理由。传统的保留模式图形界面库经历了漫长岁月的演进和行业打磨，而 IMGUI 在当时仅仅依赖他个人非全职的有限业余时间，就已经发展到了相当成熟实用的地步。这让他深信这是一种极其优秀的设计方向，因为只要稍微尝试就能轻松取得显著进展。 他由衷希望更多人能够参与实验，并期待整个技术社区共同探索：“I suspect that as more people use it, if it becomes popular, then there will be a lot more things that people find out that I don't know about yet, because I've only been doing it for my limited uses.” 他十分期待当大量开发者开始深入打磨、构建标准库并提出更巧妙的技巧时，IMGUI 所能释放出的全部潜力。 - 传统保留模式 GUI 拥有行业数十年的积累与演进历程。 - IMGUI 仅凭单人零散时间的开发尝试，就已达到极高可用性，证明了该理念的轻量与高产。 - 更多人参与实践将带来更完善的标准库、设计模式与未被发现的实用技巧。 ### [37:51](https://www.youtube.com/watch?v=Z1qyvQsjK5Y&t=2271s) 延伸资料与 Sean Barrett 的深入文章 关于 IMGUI 的更多参考资料，Casey Muratori 透露著名图形程序员 Sean Barrett 即将在《游戏开发者杂志》（*Game Developer Magazine*）上撰写一系列专门讨论即时模式图形界面的深度文章。虽然两人在 IMGUI 上的技术实现风格可能随着各自的开发产生了一定分化，但他认为 Sean Barrett 的文章将比自己这场临时讲座更加严谨、彻底，并且文章会附带实际运行的示例代码供开发者研究学习。 Casey Muratori 坦承自己此前并没有做太多工作来推广和传播 IMGUI，市面上几乎见不到任何系统性的相关文章，因此外界能找到的资料极度匮乏。他提到未来可能也会出现商业级的 IMGUI 类库，并给出了几处获取后续进展的网络途径： - 《游戏开发者杂志》官方网站：`www.gdmag.com` - Sean Barrett 个人站点：`www.nothings.org` ### [39:38](https://www.youtube.com/watch?v=Z1qyvQsjK5Y&t=2378s) 讲座答疑与 Molly Rocket 论坛交流 在讲座尾声，Casey Muratori 解释称本次分享较为粗略，主要是概念性的梳理。为了方便大家进一步探讨和答疑，所有讲座内容都会发布在 Molly Rocket 官方论坛的讲座板块中（网址为 `www.mollyrocket.com/forums`）。 他欢迎所有对此感兴趣的开发者前往论坛提出疑问、交流使用心得，或者共同探讨他在演讲中未能覆盖到的技术死角。在轻松坦诚的告别中，他向观众表达了感谢：“I hope to see you on there, and I hope you enjoyed this kind of makeshift presentation.” 鼓励大家亲自尝试编写即时模式界面。 ## Notable quotes > [32:48](https://www.youtube.com/watch?v=Z1qyvQsjK5Y&t=1968s) 从如今的传统图形界面切换到即时模式图形界面，绝大多数地方都是彻头彻尾的胜利。 — Casey Muratori > [29:01](https://www.youtube.com/watch?v=Z1qyvQsjK5Y&t=1741s) 在专业的 3D 应用程序中，经常存在两套完全独立的系统来运行用户界面：一套负责 2D 界面，另一套负责 3D 界面。那简直是垃圾设计！ — Casey Muratori ## Mentioned Casey Muratori, Molly Rocket, RAD Game Tools, Granny 3D, Windows, X Window, Visual Studio, OpenGL, Sean Barrett, Game Developer Magazine ## Video description For more details, see https://caseymuratori.com/blog_0001 [Source](https://www.youtube.com/watch?v=Z1qyvQsjK5Y)

[32:48] Casey Muratori: 从如今的传统图形界面切换到即时模式图形界面，绝大多数地方都是彻头彻尾的胜利。 [29:01] Casey Muratori: 在专业的 3D 应用程序中，经常存在两套完全独立的系统来运行用户界面：一套负责 2D 界面，另一套负责 3D 界面。那简直是垃圾设计！

[23:55] Casey Muratori: All you have to do is just write a single piece of code which encapsulates where something's drawn, how it's drawn, how it's interacted with, and what you want to do when that interaction comes through; it's all in one place.

[0:01] okay hello everybody and welcome to what [0:04] is hopefully the first in a series of [0:07] small programming videos um essentially [0:10] I do a lot of crazy wacky new stuff at [0:13] Molly rocket as part of the development [0:16] of our upcoming game and hopefully [0:18] future games and I don't really have any [0:21] time to write up papers or produce [0:24] professional quality lectures that talk [0:25] about some of the neur techniques that [0:27] I've been trying that I think are [0:28] successful or interesting so the goal [0:31] here is to see if just writing out some [0:33] sort of makeshift slides on a giant [0:34] piece of paper and talking about uh you [0:38] know the methods for a while in front of [0:40] a video camera will be sufficient to get [0:42] the idea across and seminate the [0:44] information um so it's something that I [0:46] can actually have time to do so uh this [0:50] this first one is actually a pretty old [0:52] idea um it's something that I started [0:54] doing in 2002 or roughly thereabouts and [0:58] it's called Uh immediate mode graph user [1:00] interfaces and really it's a style of [1:04] coding a user interface um or making the [1:06] code for a user interface of some kind [1:08] doesn't have to necessarily be an [1:10] application could be for a game [1:11] application anything you want um that I [1:14] think is much easier to use and easier [1:17] to read and more logical uh and better [1:20] laid out than the current mechanisms [1:23] which uh are mostly based around some [1:25] kind of [1:26] objectoriented um sort of class Library [1:29] sorts of things for you user interface [1:30] design and so forth so I wanted to uh go [1:34] over all of the techniques that in this [1:36] method and give you sort of the overview [1:39] and so that the word can get out and I [1:41] can see if if it's something that a lot [1:43] of people find useful if for something [1:44] that's like totally specific to my [1:46] coding style um so first let me tell you [1:49] why I sort of went down this path and [1:52] the reason is because I've done user [1:55] interface coding several times in the [1:56] past and I was never happy with the way [1:59] it came out [2:00] uh I've done Windows user interfaces [2:02] I've done custom user interfaces I've [2:04] done x windows user interfaces and sort [2:08] of invariably the user interface code is [2:10] always a pain in the butt to write it's [2:12] not fun to write it's annoying uh you [2:14] often times are spending a lot of your [2:16] time dealing with some crappy widget [2:18] layout tool and all of these different [2:21] linkages different parts of what happens [2:23] for the UI different places call backs [2:26] IDs all this stuff and it just always [2:28] feels like a morass it feels like takes [2:30] way too much code to get a very simple [2:32] user interface up on the screen and that [2:35] was sort of what I was trying to address [2:38] I didn't intentionally try to address it [2:40] I wasn't set out to try and fix this [2:41] problem but what happened was I was [2:43] working on um at Brad game tools I was [2:46] working on doing the viewer for granny [2:49] um which had a lot of different kind of [2:50] UI components to it and as I was working [2:53] on it it sort of hit me that the way [2:55] that I was doing things could be [2:57] simplified quite a bit um and so I [3:00] started trying to do some of the simpli [3:03] the simplifications and I found that [3:05] just it was It was kind of just like a a [3:07] completely new a completely fresh way of [3:10] coding the user interfaces that was just [3:12] so much more natural to me and so ever [3:14] since then that's the only way I've ever [3:15] done user interfaces so let me go over [3:18] basically what I think a normal UI [3:21] Library tends to look like and then I'll [3:23] show you what that same sort of where [3:25] everything goes in an immediate mode [3:27] user interface uh Library so in a [3:30] typical UI uh Library what you have is [3:33] at init time the app code the stuff that [3:36] you're writing assuming that someone [3:37] else wrote the library or that you [3:38] already wrote the library the stuff that [3:41] you're writing is essentially a bunch of [3:42] create calls um or maybe you've got some [3:45] file that you loaded from an editor that [3:47] has like those equivalents of create [3:49] calls that Define um what's going to be [3:51] created in the user interface you might [3:52] like create some buttons here create a [3:54] dialogue some buttons in it property [3:56] Pages whatever you know you name it [3:58] whatever you wanted to to create you've [4:00] created it at update time typically you [4:03] have no idea what's going on at update [4:05] time that's usually the UI codes perview [4:08] you don't have any actual code that's [4:09] executing there it's not your [4:11] responsibility it handles redrawing the [4:12] buttons it handles whatever maybe if [4:14] you're doing owner draw or something [4:15] like this uh then you occasionally know [4:18] about what happens in here but for the [4:19] most part assuming you're not doing [4:21] anything terribly complicated you don't [4:22] know what's going on in there and don't [4:23] care and then finally there's usually uh [4:26] you know what happens when you actually [4:27] need to get information back um the user [4:30] clicks on a button the user does [4:32] something anything selects a menu item [4:34] you're going to get called back or [4:36] you're going to get a message in some [4:38] kind of a queue so there's there's kind [4:39] of this in here some stuff happened that [4:42] we don't really know what it was but [4:43] then we're going to get called back [4:45] eventually or get a message that tells [4:47] us what happened and then as a result of [4:49] that message we might get some [4:51] information back from the widgets that [4:52] we want um usually using some kind of ID [4:55] linkage like oh well what was that uh [4:58] you know what's the value of this widget [5:00] right now cuz I'm going to go set some [5:01] variable to be uh that value or [5:03] something like this um I might uh sort [5:07] of delete or create controls dynamically [5:10] or maybe just enable and disable them uh [5:13] based on what happened because sometimes [5:14] you know I'm switching to a new property [5:16] page and I need to do something or I'm [5:18] bringing up a new dialogue whatever so [5:20] even though we did a lot of creation at [5:22] a nit time we also probably have a lot [5:24] of creation that sort of goes on in [5:26] between and it's unclear where that's [5:28] necessarily all supposed to happen but [5:30] you typically throw it around wherever [5:32] you know what what persists when is is [5:35] entirely up to you when you're coding [5:36] the user [5:37] interface now for a lot of reasons this [5:39] is a big pain in the butt I won't go [5:41] into them all now but I mean you know [5:43] just for starters the fact that this [5:45] stuff is all so decentralized is usually [5:48] problem number one I've got some editor [5:50] or some code somewhere that's creating [5:52] these things I've got some people who [5:54] need to switch between those things and [5:56] get values out of them and the only way [5:57] they can identify them is by using some [5:59] ID which is now shared between multiple [6:01] pieces of code and possibly an external [6:03] editor um and it's just okay you know it [6:06] just gets hairy and ugly really fast the [6:10] good UI libraries try to do their best [6:12] to minimize a lot of this stuff and some [6:13] of them do a decent job but on the whole [6:15] it still ends up being pretty messy even [6:17] in the good case and in the bad case [6:19] like if you're using say for example any [6:20] of the Microsoft tools you're a deep [6:23] trouble you end up writing hundreds of [6:25] lines of code that essentially does [6:26] nothing other than keeping all of this [6:28] stuff kind of wrangled together [6:31] so on the reverse side [6:34] here uh we basically have what happens [6:37] in immediate mode graph of user [6:38] interface code what happens in in the [6:41] immediate mode code is much like it's [6:43] name would imply everything is immediate [6:44] mode which means that there is no init [6:47] code unless you want there to be [6:49] something on your end and there is no [6:51] callbacks there's no nothing like that [6:52] there's just a main update path written [6:55] by you and what that is is it's just [6:59] like you would expect in immediate mode [7:00] if you were doing Graphics or something [7:01] like that you go through your code and [7:03] you start doing the things that you want [7:05] to do to get your interface up on the [7:07] screen um so for example I might do [7:09] something like uh if do button right [7:12] let's say that this was going to be a [7:13] button I might call the do button thing [7:15] and it's going to return me true if the [7:17] button was clicked or false otherwise so [7:20] if it was clicked then I'll just do [7:21] whatever I wanted to do when that button [7:23] was clicked right here there's no need [7:25] for call back there's no need for [7:26] anything else I'm just going to do it um [7:28] and if not then I won't do anything with [7:30] that button but I mean I still drew it [7:31] on the screen by using this call um and [7:33] then I could do other things I could [7:35] have ifs that say oh if this state is [7:37] this that I'm going to do this or I [7:38] could do some kind of uh switch or call [7:41] a function that does some other parts of [7:42] the UI if I want to I can have four and [7:44] while Loops I can do a while loop and [7:46] draw a button on each Loop whatever I [7:48] want to do and it's exactly akin to what [7:50] you would do if you were doing immediate [7:53] mode drawing right um you typically in [7:55] Med mode drawing right you don't need to [7:57] have a nit code that sets up all the [7:59] stuff that you're going to draw and then [8:01] you just map they called Draw and it all [8:02] draws itself no you have an update Loop [8:05] which pulls from your data not some data [8:08] that's defined by the renderer but your [8:10] data and you decide what how the user in [8:13] face gets done on each update [8:15] yourself okay that's basically all there [8:18] is to it now granted I'm going to have [8:20] to explain a bunch of other things [8:21] because this may look totally strange to [8:22] you and that's understandable this isn't [8:24] the way user interface are normally [8:25] coded but honestly really despite the [8:29] fact that I'm might spend 20 more [8:30] minutes talking about this that's the [8:32] only key difference between immediate [8:34] mode user interfaces and standard user [8:36] interfaces is they're just trying to do [8:39] what immediate mode Graphics did as [8:42] compared to retained mode Graphics [8:44] because we all pretty much know how hard [8:46] retain mode graphics suck that's why we [8:49] use immediate mode Graphics same is true [8:51] of immediate mode user interface I think [8:54] regular graphic interes which are [8:55] retained you do a nits you have them [8:57] they know State they know what what [8:59] they're button is whatever uh I think [9:01] those suck and immediate mode graph Poli [9:03] interfaces are kind of the answer in my [9:05] opinion okay [9:09] so how does this stuff actually work how [9:12] do you actually do [9:16] code in this manner um well the primary [9:21] uh the primary idea that motivates how [9:24] this stuff is going to work and again [9:25] this is just one implementation um the [9:28] core idea I've already said so you know [9:30] there's a million different ways you can [9:31] implement this here's how I tend to [9:34] implement it the core structure is [9:36] called the UI context and what that does [9:39] is that tracks the state of the user's [9:42] interaction with the system at any given [9:45] time now there's a key Point here that I [9:48] think a lot of people fail to realize [9:50] about user interfaces even complicated [9:52] ones and that is that there's a massive [9:56] massive amount of redundant State [9:58] typically stored in a retain mode graph [10:00] user [10:02] interface it's not uncommon at all for [10:05] something like say an edit box in a user [10:07] interface to retain all of the [10:10] information necessary to edit the [10:11] information in the edit box even though [10:14] the edit box is not being engaged by the [10:16] user at any in any way at a particular [10:19] time right so if I have two edit boxes [10:20] on the screen the user cannot be editing [10:22] the text in both edit boxes at the same [10:24] time obviously however both edit boxes [10:27] typically carry out all of the baggage [10:29] that they would need to do so even [10:31] though one of them the one that's not [10:33] being interacted with at this time only [10:35] needs to know the text that's all it [10:36] needs to know because that's all it's [10:37] doing it's just going to draw that text [10:39] doesn't need to know anything else but [10:41] it still probably carries around you [10:42] know stuff like oh you know what's was [10:45] the mouse dat when was the last time I [10:46] clicked on all you know all these pieces [10:48] of data that it would need to interact [10:50] with that's fine we're not really [10:52] concerned with memory so much and graph [10:53] user interfaces typically that's not [10:55] that big of an issue although those of [10:57] you who have ever used Visual Studio May [10:59] have noticed that sometimes you click on [11:01] something and it says this may take a [11:03] long time to expand are you sure you [11:05] want to do it or things like that so I [11:07] don't know maybe sometimes you are but [11:08] for the most part you're not so I'm not [11:09] trying to say that that's a bad thing [11:11] the point is that redundant state is [11:15] actually something that can be exploited [11:17] to make a much more efficient piece of [11:19] code that's simpler and easier to [11:20] understand and what that is is you can [11:22] say well you know what the user can only [11:24] be interacting with one thing in a given [11:26] time so let's just make a single [11:28] structure that's kind of global to the [11:30] user interface that's the state of user [11:32] interface that is the state of the [11:34] user's interaction because really that's [11:36] all we need we don't need anything else [11:39] everything else is data that's coming [11:40] from the application and that [11:42] application is storing that data in its [11:44] own way the text in an edit box should [11:46] not be stored in the edit box the text [11:48] that's there means something to the [11:49] application and is stored in some [11:50] application data structure so we clearly [11:53] don't need all of that so what do we [11:56] actually need to store in the UI and the [11:58] answer is just the minimal amount [12:00] necessary to interact with it so I'm [12:02] going to go over the simplest set of [12:03] interactions that you can do which is a [12:05] button or a menu item um but this [12:07] extrapolates to basically any kind of [12:09] widget that you want now what I've done [12:13] here is I've shown the two most [12:14] important members of this UI context uh [12:17] which is what is the hot item in the [12:20] user interface and what is the active [12:22] item um now basically these two things [12:25] um are using What's called the uiid [12:28] struct but really all that means is some [12:30] method of identifying the item and I'll [12:32] talk about what that means in a second [12:34] here but we have two things hot and [12:36] active because typically user interface [12:38] elements have sort of two states there's [12:41] about to be interacting with this item [12:43] and actually interacting with this item [12:45] if I'm about to be interacting with the [12:47] item simp that like I move the mouse [12:49] over that item let's say or I'm using a [12:51] keyboard interface and I've highlighted [12:53] the item so typically means the item [12:55] wants to draw itself highlighted or show [12:56] up in some way that suggests to you that [12:59] hey you know what maybe you want to [13:01] interact with me right now maybe not but [13:03] maybe you do then if the user actually [13:05] starts the interaction they click the [13:07] mouse on it or do whatever it takes to [13:08] get that widget going then that's an [13:12] active item by my terminology here so [13:14] hot means like about to be interacted [13:16] with and active means I'm actually [13:17] interacting with it okay and we'll see [13:20] why we how we use these two things in a [13:22] second but that's basically the the [13:24] majority of the information that we need [13:25] to store for this for basic uh UI now [13:29] the way that we identify items is kind [13:32] of interesting in uh immediate mode UI [13:35] because it has a very different property [13:37] from the way that we identify them in [13:39] standard uis in a standard UI the [13:41] typical reason that you have an [13:43] identifier like I showed on one of the [13:45] previous slides was because there are [13:46] different sections of code that need to [13:48] talk to each other about a particular [13:51] item I created the item over here but I [13:54] need to access the data from the item [13:56] somewhere else right makes sense [13:59] in that case what you're talking about [14:00] is some globally unique identifier that [14:02] is set up beforehand or whatever however [14:04] you want to do it right for mediate mode [14:06] U eyes we don't really care about that [14:08] so much because of the immediate mode [14:10] nature of it we can always handle all of [14:12] the interrelations the complex [14:14] interrelations on the app side because [14:16] it's all using app data we don't have to [14:18] have there be ways of getting that [14:20] information out of the guei because GUI [14:22] is not storing any of that information [14:23] for you it's all clean and on your side [14:26] so we do need identifiers however [14:28] because we need to remember things like [14:30] what you know element the user was [14:32] interacting with last frame for example [14:34] um or even this Frame so we need a [14:36] method of identifying them so typically [14:38] what I do is I don't want to actually [14:40] have to make a header file with these [14:42] IDs and I don't want to have to make [14:44] them will be I don't want all of that [14:46] headache that's uh sort of associated [14:49] with the old way of doing the user [14:50] interface stuff so what I do is I [14:52] typically use just kind of when you call [14:55] in to do to you know do a button action [14:59] or any of that sort of stuff you just [15:00] pass in some bits of information that [15:03] will uniquely identify you and this is [15:05] almost always readily available to you [15:09] um except in very complicated scenarios [15:11] which we can talk about later um so in [15:14] the basic scenario let's say I'm just [15:15] doing some buttons and so on and maybe [15:17] you know when I do a button that I'm [15:18] going to go call some function that's [15:20] going to do some stuff right for the the [15:23] the global identifier for that button [15:24] could easily just be the function [15:26] pointer for example right because we [15:27] know that that's globally unique and we [15:29] know that any user interface element [15:31] that is going to be interacted with [15:33] that's going to do this action we [15:34] probably want them to always share that [15:36] so that way you know when one of them [15:37] becomes hot then multiple of them could [15:40] light up for example just you know that [15:42] they're the same activity of course [15:43] normally you don't want user interfaces [15:44] that have lots of buttons that do [15:45] exactly the same thing on them but [15:46] that's the point um and furthermore [15:50] sometimes you want to be very specific [15:51] and have have even two things that [15:53] basically do the same thing or have the [15:54] same meaning uh be separated in terms of [15:57] the hot item and so on so sometimes you [15:59] keep more information that which is like [16:00] for example the owner um which is like [16:03] oh when I went in to draw this group of [16:06] user interface elements I'll use the [16:08] pointer to that function or I'll use the [16:09] pointer to the data that I was being [16:12] driven from for that group of items like [16:14] the menu or whatever I'll use that as [16:16] well so they're just basically pieces of [16:18] blind data that the application passes [16:20] in index is another one in case I was [16:22] doing an array of things you know [16:24] whatever and it's very very similar [16:26] again uh continually drawing comparison [16:28] very very similar to the way it would [16:30] work in a immediate mode graphic system [16:32] when you need to get feedback from the [16:34] hardware or from the library for example [16:37] feedback buffers that sort of thing the [16:39] the library does not care how you want [16:41] to identify these things you can use any [16:43] identifiers that you want it's just [16:44] there so that you can uniquely pick out [16:46] a particular [16:49] item so here's an example of this is [16:53] again this is code in the UI Library so [16:56] this is not code that you would be [16:57] writing unless you were implementing the [16:59] library this is the code that you'll be [17:00] calling so I I talked about do button [17:02] and I was saying that it returns a Bo [17:04] which is uh true if the uh button was [17:07] clicked so you know to do something when [17:09] you called it right um and the example [17:12] here uh you pass in the context [17:14] obviously some piece of identifying [17:16] information and again that's you know [17:17] your decision but the struct that UI [17:19] struct as an example uh say the text of [17:21] the button I don't know and some other [17:22] crap maybe where you want the button [17:23] placed I don't know maybe there's some [17:25] Auto layout happening who knows um again [17:27] all that's dependent on how you want [17:29] Implement your library um but the logic [17:31] is incredibly simple basically we just [17:33] say if we're the active item and in [17:35] order to determine if we're the active [17:37] item that's a call that usually goes [17:38] into the UI context which Compares our [17:40] ID versus the active ID and if they're [17:43] the same then we're the active item and [17:44] if they're not then we're not the active [17:46] item very [17:47] simple if we were active then if the [17:49] mouse went up meaning the user released [17:51] the mouse button if we were hot which [17:54] again is the same as active but a [17:55] different ID in the context so just [17:57] Compares against that and says it that [17:59] means the user was still over us then [18:02] the result of this interaction is true [18:03] so we're going to return true and [18:05] eventually when we fall out of here and [18:07] uh that's going to go ahead and tell the [18:09] calling code that it's time to do the [18:11] activity for this button if we weren't [18:14] active then if we were hot uh well okay [18:18] skip one here um regardless of what [18:20] happens when the mouse goes up we set [18:21] ourselves to not be active anymore [18:23] that's because the definition of a [18:24] button behavior of course is when you [18:25] click down on it then when you release [18:27] If You released on top of the button [18:29] meaning it was hot then the result is [18:31] true if you're released away from the [18:33] button somewhere then you don't consider [18:36] that a click uh it's a standard UI thing [18:38] obviously you don't have to do it that [18:39] way but that's kind of the convention so [18:41] it'd be weird if you did something else [18:43] uh your users would get a little [18:44] confused although many games often don't [18:46] Implement that correctly what are you [18:47] going to do um now okay so if we weren't [18:50] active but we were hot then we just need [18:52] to know did the mouse go down did the [18:54] user click because if the user clicks [18:56] while we're hot then we become active [18:58] right because we were about to be [18:59] interactive with as a button we know the [19:01] user was over us cuz we're hot so if we [19:03] click on it we become active and now [19:04] we're the active item and now we're [19:05] going to be in here next time uh the [19:08] user inace does something and maybe when [19:11] maybe the mouse will go up and we'll be [19:12] able to uh return true and finally [19:15] regardless of what happens up here if [19:17] the the mouse is inside us if we were [19:19] the hit if we were the hit region um [19:21] which we you know presumably the user [19:23] pass is where we're supposed to be here [19:25] then we we set ourselves as hot and the [19:28] way that that works works is we call [19:29] into the UI code and say set us we're [19:31] the we're the new hot item what that [19:34] does is not directly to it doesn't [19:37] directly set it as it does with the [19:38] active thing um right here with with the [19:41] Bounce Down set active which usually [19:42] just directly sets the the thing as [19:44] active um when we do the set hot it [19:46] looks to see if there is an active item [19:49] if there is an active item it doesn't [19:50] set us as hot right because that item is [19:53] being interacted with and we're not [19:54] going to be hot right if there's an item [19:56] being interacted with and you move your [19:57] mouse over other widgets they should [19:58] should not become um highlighted or [20:00] anything uh so this is simply a request [20:03] into the UI system if there was no [20:05] active item make us the hot item because [20:07] we are hot finally after this I haven't [20:10] written anything in here but you would [20:12] this would you know presumably draw some [20:14] stuff for the button and whatnot as well [20:15] um you know you display the button [20:17] however you want to display it with a [20:18] frame rounded for the text you know who [20:20] knows whatever same exact code you would [20:22] have in a regular graphic laser [20:23] interface where at the time when it's [20:24] time to to to code it so again this is [20:27] not something that you write on the app [20:29] side this is something that the library [20:30] is doing and it's a function that the LI [20:31] that gets called um by the calling [20:35] code very very simple stuff [20:39] here [20:43] okay so you can imagine more complicated [20:47] widgets than just that uh that require [20:49] more than just hot and active um like a [20:51] button does but there's a surprising [20:53] number that actually do that scroll bars [20:55] are pretty much the same um you know [20:57] most of these things are pretty much the [20:59] same the only difference is some more [21:01] compli ones which are like edit box text [21:03] edit boxes and so on um will require [21:05] interim information like where the [21:07] cursor position was and what the hot [21:09] selection region is and that sort of [21:10] thing so sometimes you have more you [21:12] have some widget specific data um that [21:14] the UI context will store for that [21:16] widget during its active period um but [21:19] again I wanted to keep this relatively [21:20] simple so I'm not going to go into how [21:22] that stuff works it's pretty easy to [21:23] extend to do them um so I just wanted to [21:26] show here the example of of [21:29] just kind of draw some more parallels [21:30] between how you use this code and how [21:32] you use Graphics code the mediate mode [21:34] Graphics code uh looks you know like oh [21:37] for each object and each batch in the [21:39] object I'm going to set the material and [21:40] draw some triangles and maybe I'll do [21:42] the transparent ones last and I'll sort [21:44] them and do some batches there or you [21:46] know maybe this gets kind of complicated [21:47] and I do some checks like if something [21:49] was called then I'm not going to draw it [21:51] uh maybe I call batches maybe I've got [21:53] some kind of a bsp on traversing who [21:55] knows this is entirely in the app side [21:58] of things right this code does not live [22:01] in the renderer that was a very [22:02] important part of the progress of [22:05] graphical um kind of like graphical [22:08] coding is realizing that retain mode was [22:10] not a good thing because the important [22:12] way the way that the app needs to [22:13] express how things get drawn is [22:16] necessarily uh going to look like this [22:18] it needs the flexibility there to do [22:20] whatever it wants um because each game [22:22] um or each application is very different [22:24] when it comes to exactly how the flow is [22:26] going to to work um and so UI is no [22:29] different in my opinion and you get the [22:31] same sorts of things like I want to just [22:33] be able to say oh for all the menus and [22:34] the items in the menus I'm going to do [22:35] an item or maybe I have this as kind of [22:38] a thing which I just call for do menu [22:40] and I can do the same thing for pop-ups [22:41] if I want to and I can make sure that I [22:42] do those after or I could you know do [22:44] them during or whatever and I can do 38 [22:47] elements too it doesn't matter I can do [22:48] anything I want to do here um and again [22:53] this is totally up to me how I want to [22:54] do it I can do culling quote unquote [22:56] meaning if at any time I don't feel like [22:58] drawing some of the items I just don't [22:59] draw them right I just don't call do [23:01] item which gets rid of the user [23:03] interactivity of them and gets rid of [23:04] their visual representation I don't have [23:06] to delete the item I don't have to [23:08] disable it I don't have to hide it I [23:10] just simply don't call it that frame [23:12] furthermore I don't even have to know [23:13] how many of these there are I don't have [23:14] to create ones for it ahead of time I [23:15] enough do any of that stuff I can [23:17] iterate over my own internal data which [23:19] could be any format I want generate uis [23:22] out of that right I could load a file [23:24] and generate DUI right from that file [23:26] could do anything I wanted to do it's [23:28] enti up to me and the same Library can [23:30] facilitate all of that the library does [23:32] not have to be aware of how I'm deciding [23:35] what would just are on the screen and [23:37] when and how they're being interacted [23:38] with and why none of that stuff has to [23:40] be known by the library the library [23:41] simply provides you a number of [23:43] potential interactions and potential [23:45] ways of drawing those interactions and [23:46] you call them when you want that's the [23:48] whole idea um and and a lot of the power [23:52] so um some stuff [23:55] uh see here reasons [23:59] why this is cool and why I like it a lot [24:02] better than the previous method for [24:04] starters there's a single piece of code [24:06] for every logical piece of your [24:07] interface there it's not broken up it's [24:09] not broken up between an editor some an [24:12] in code some uh Response Code Etc ET [24:15] there's no breaking up of it it's all in [24:17] one place um if you wish to load [24:19] something from a file you could but [24:21] that's entirely optional to you for the [24:23] most part all you have to do is just [24:25] write a single piece of code which [24:26] encapsulates where something's drawn how [24:29] it's drawn how it's interacted with and [24:31] what you want to do when that [24:32] interaction comes through it's all in [24:34] one place no matter how complicated it [24:35] gets it's all in one place and that's [24:37] very important because it keeps the code [24:39] much cleaner and it makes it much easier [24:41] to edit when you know that if you want [24:43] to edit a particular piece of the UI [24:44] changes Behavior you only have to go to [24:46] one place in your code it's not [24:47] scattered right it doesn't lead to code [24:49] proliferation there's not an object per [24:51] widget there's none of this stuff it's [24:53] all very very [24:55] straightforward uh and similar similarly [24:58] kind of going along with that there's [24:59] unified flow control which means that [25:01] the uh graphical user interface library [25:03] is not in control of what's happening at [25:06] any time it it is in a regular user [25:08] interface it's got kind of an update [25:09] Loop that's going to do and it's going [25:10] to sort of either call you back or be [25:12] message queuing back to you and so that [25:14] does not happen in the immediate mode [25:16] system you are always in control on the [25:17] immediate mode system you have a single [25:19] path for your UI that works exactly the [25:21] way you want that mirrors the structure [25:23] behavior of your app and then you're [25:25] always just calling out to the library [25:27] and it immediately returns back to you [25:29] the information that you need uh to [25:31] process the to the activity there right [25:34] there um there's no synchronization uh [25:38] which just comes with any immediate mode [25:39] system which is nice and this is [25:41] including existence like including [25:43] whether or not things actually exist on [25:44] the screen or exist um to be interacted [25:47] with there's never any synchronization [25:49] that happens and that's very important [25:50] because typically a lot of the overhead [25:52] in user interface stuff also comes from [25:54] this ridiculous amount of [25:55] synchronization calling all these set [25:57] calls enabling this disabling that [25:59] putting this here making this that it's [26:01] like none of that stuff happens at all [26:02] in immed mode user interfaces if your [26:04] code executes and it calls out to [26:06] display a particular property page [26:08] that's what gets displayed if you don't [26:09] want to display that one next frame [26:10] because some variable changed which has [26:12] nothing to do with user interface maybe [26:14] it was like some variable that changed [26:15] because like 10 seconds went by whatever [26:17] whatever you want or you know some [26:19] character did something on the screen [26:21] doesn't matter you just call a different [26:23] thing that frame or don't call that [26:24] function at all and the property page [26:26] disappears poop no problem so it's [26:29] really nice that way because now you [26:30] only have to worry about your own part [26:32] of the code your own logic you don't [26:34] have to worry about making sure that [26:35] your own logic can be mapped onto the UI [26:37] logic in some way which typically is a [26:39] huge headache and involves a lot of [26:41] extraneous code that is boring and [26:44] really contains no valuable [26:47] information uh okay so those are kind of [26:50] some things about the actual coding [26:52] process now one of the nice things about [26:54] this in terms of as UI get more [26:56] complicated is this uh this style of [26:58] coding scales really well to very [27:01] complicated widgets with specialized [27:03] behaviors uh and that's something that [27:05] standard UI libraries handle very poorly [27:07] so I'm going along and I'm using you [27:09] know some of the things in my UI library [27:11] and then all of a sudden I want to do [27:13] something complicated that's you know a [27:14] very specific kind of widget that only [27:16] my apps really going to have uh typical [27:18] UI libraries handle that incredibly [27:20] poorly but immediate mode libraries [27:22] that's really no problem because it's [27:23] trivial for you to just do some of your [27:26] own interactivity and your own widget [27:28] coding right there uh anywhere you want [27:30] or even make add you know your own [27:31] function calls that you call out to that [27:33] are kind of your own secondary Library [27:35] um because all you have to do is make [27:36] sure that it plays well with the active [27:38] and hot items or if there's some other [27:40] items there that you you know depending [27:41] on how you structure your UI context as [27:43] long as you can make sure that it [27:44] doesn't step on the toes of that which [27:46] is Trivial to do then you're fine you [27:48] just as soon as you feel like your [27:50] widget became active you said to the [27:51] active item everything else in the UI is [27:54] shut down until you release that [27:55] activeness so you can write any code [27:57] that you want doesn't matter what don't [27:59] care how you render it don't care what [28:00] you're doing with the mouse up to [28:03] you um and similarly it scales well to [28:07] the relationships between items because [28:10] uh and which again is something that's [28:11] very uh poorly handled in typical uis if [28:14] you want to do all kinds of crazy stuff [28:15] rearrange the orders of buttons hide [28:17] huge sections when something's checked [28:18] off uh only show something when a [28:20] particular item is set to a particular [28:22] value those things are as simple as the [28:25] code necessary to check them if you want [28:27] to only display some huge set of widgets [28:29] when a particular uh value is set to [28:32] five it's an if value equals five you [28:35] know Open brackets do all the items [28:37] that's all it is it's just wrapping up [28:39] the code in one if right that's the way [28:41] it should be because you're not asking [28:42] to do anything complicated normal widget [28:45] libraries typically make this kind of [28:46] thing incredibly complicated they [28:48] involve you having a call back on the [28:50] change in that item where you check to [28:51] see if it's that and if it is you call [28:53] to set the other widgets to something [28:55] else or remove them it's just a disaster [28:57] in immedi goey no problem no worries [29:01] whatsoever now uh these are particularly [29:05] uh in of interest to game programmers I [29:07] think or people who write 3D uh editing [29:10] applications of some kind uh which is [29:13] that IMM mediate mode guies really [29:15] integrate fabulously with pieces of code [29:18] that don't actually have any knowledge [29:20] of user interface or don't even want to [29:22] be involved in user interfaces at all uh [29:25] say for example your rendering Library [29:27] um your an system all of these things [29:30] are typically not written with user [29:31] interface in mind but almost always they [29:35] have some way of doing say for example [29:37] rate picking um and they can always [29:39] render themselves and that's all you [29:41] really need to turn them into an [29:43] immediate mode widget because all you [29:45] have to do is you just have to go okay [29:47] well you know I'll make my own little do [29:49] button Behavior kind of function um and [29:51] instead of using is inside rectangle as [29:53] my test I can just use did the mouse hit [29:56] this item using the you know picking [29:58] call of my renderer or whatever and that [30:00] stuff just integrates perfectly because [30:02] you can do it in the middle of some very [30:04] complicated rendering Loops that involve [30:06] setting the camera position and all [30:08] these sorts of things all of that stuff [30:10] you can just write the way you would [30:11] normally write it all in the middle of [30:12] your normal application and you can just [30:15] make the immediate mode gooy calls right [30:17] there and it just works so it's really [30:19] really easy to tie any other complicated [30:22] code into the immediate mode user [30:24] interface that is extremely difficult to [30:26] do in typical UI system that often [30:28] involves dering from somebody's class [30:30] and hoping they have enough messages or [30:32] enough data things or whatever you know [30:34] God only knows the hell's in there [30:36] hoping that that stuff will facilitate [30:38] your ability to do what you need to do [30:40] it almost invariably doesn't you don't [30:42] get the call backs at the right time you [30:44] have trouble setting up all your context [30:45] things it's just it's just a disaster to [30:47] the point where even in professional 3D [30:50] applications often times you have two [30:52] totally separate systems running the [30:54] user interface one for the 2D user [30:56] interface one for the 3D user interface [30:59] that's crap why there should not be two [31:01] different systems it's just a user [31:03] interface and mediate mode goey lets it [31:05] be a single system single path no [31:08] problems finally um immediate mode is [31:11] perfect for real time as you might [31:13] imagine immediate mode in real time hand [31:15] in hand um and the reason for that is [31:18] that you're passing down everything [31:19] every frame you're going through the [31:20] user interface every frame and drawing [31:23] it while you're interacting with it in [31:25] your own code so if you want to do crazy [31:28] stuff like have your animation system [31:30] animate some guy and it puts the [31:32] freaking button on wherever his you know [31:36] above his head or something no problem [31:38] because each frame you just get wherever [31:40] that position was out of the animation [31:41] system and you call the draw button code [31:43] right there and it just magically works [31:45] right again no synchronization issues [31:48] you can simply do a single path where [31:51] you would normally be doing all your [31:52] stuff and you can just put the calls to [31:54] the interface right in there and you [31:56] don't have to worry about all these [31:58] sorts of normal partitioning problems [32:00] that you would otherwise have which make [32:01] this sort of thing incredibly difficult [32:04] um and so I think that that part is [32:06] actually really interesting especially [32:07] going forward when a lot of times now [32:10] people even for simple user interfaces I [32:12] think partially thanks to like Macos 10 [32:13] for example and also uh flash uh on the [32:17] web people are looking to make uh sort [32:20] of more glitzy user interfaces which [32:22] have more action on the screen um things [32:25] floating around and moving and doing all [32:27] these crazy things [32:28] uh that creates enormous problems for [32:31] today's typical uis immediate mode guy [32:33] no problem it's made for that in fact it [32:35] does that better than a stationary stuff [32:38] um because hey stationary stuff doesn't [32:40] use part of the power of the immediate [32:42] modess of it all so um I think there are [32:45] a lot of really compelling reasons to [32:46] use it now that said obviously there are [32:48] drawbacks whenever you switch from one [32:50] thing to another um you you know you're [32:53] making tradeoffs and in immediate mode [32:55] guy's case I think that it's [32:57] overwhelmingly in favor of immediate [32:58] mode goys however there's definitely [33:00] some tradebacks you trade-offs that you [33:02] should be aware of and primarily where [33:04] these trade-offs come from is uh in the [33:08] fact that particularly the languages [33:10] that we use which like for example CN [33:12] C++ which what I primarily use what most [33:14] game developers use what most developers [33:15] use actually I guess until you start [33:17] getting into more of like Webby kinds of [33:19] things or whatever um is that it uh C [33:23] and C++ do not respect the fact that [33:25] code is also data um C and C++ are [33:30] notoriously bad languages for things [33:32] like introspection uh for things like [33:35] understanding where I am in my code and [33:38] so on CN C++ can't do anything [33:40] resembling anything for those sorts of [33:43] uh situations uh no closures no [33:45] introspection no nothing you're Flying [33:47] Blind essentially and what that means is [33:50] there's really no way for you to have [33:54] code that thinks about the whole UI in [33:56] any real way it cannot Traverse it in [33:59] any way other than actually just calling [34:01] it right um you have to actually execute [34:04] the UI to know what the UI is and that's [34:07] where a lot of the power of immediate [34:08] Mode graphes come from but it also means [34:11] there are some drawbacks for example [34:13] anything that you want to do that's [34:14] based on layouts or you know looking at [34:17] the size of the total UI and then doing [34:19] something based on that size anything [34:22] you want to do regarding that has to be [34:24] one frame of lag because in order to [34:26] look at what happened at the end end of [34:28] of all the layout you have to actually [34:31] have called the code um now there's some [34:33] ways that you can code around this um [34:36] which is to make it so that your uh code [34:39] can be called and the UI context which I [34:42] the slide is gone but that UI Contex [34:44] stores the active and hot items that can [34:46] store whether or not you're on a render [34:47] pass or on a layout pass and it can have [34:51] the functions respond accordly so when [34:52] you call do button if it's in layout [34:54] mode it will just record its size and [34:56] send that to the layout engine or [34:57] whatever those things can be done but [35:00] again that starts to get a little more [35:01] complicated and introduces some more [35:03] complexity there um where away from the [35:06] pure Simplicity and ease of [35:08] implementation and ease of use uh that [35:10] come with uh the immedi graphical user [35:12] interface so I tend to prefer to just [35:15] kind of like let that stuff slide um [35:17] because very very rarely is there [35:19] anything in a real-time application that [35:21] can't have one frame of lag and be fine [35:23] with it so I tend to just accept that [35:24] but if you really don't want the one [35:26] frame of lag there's ways you can make [35:27] things work um but I'm not super I'm not [35:29] super into that I don't directly [35:31] Advocate it that sort of thing on the [35:33] other hand is not a problem for retain [35:34] mode things because retain mod things [35:36] can Loop over their stuff uh the sort of [35:39] code code that's in charge can Loop over [35:40] it as many times as it wants um and [35:42] that's kind of a little more powerful uh [35:44] in that sense either way uh other than [35:47] that I don't think there are really any [35:49] uh drawbacks in immediate Mo there's [35:51] just that one but that leads to some [35:53] problems um most everything else is a [35:56] clear wi when going from the graphical [35:57] user interface of today to an immediate [35:59] mode grass user interface I think you [36:00] pretty much get uh all all wins um the [36:04] only other one I can think of is that uh [36:07] it becomes a little more there's a [36:10] little more onuses on the application if [36:11] you want the user interface not to [36:13] update every frame uh so for example if [36:15] for some reason these is very [36:17] complicated and drawing it was a big [36:19] deal so that you didn't want to have to [36:20] draw it every frame you only wanted to [36:22] draw the parts that changed for example [36:24] um that becomes also much more difficult [36:26] because you're kind of rning on code [36:28] flow to generate the user interace in [36:29] the first place it's hard to isolate [36:31] things and do changes so again the app [36:34] side ends up having to be more [36:36] responsible about that sort of thing if [36:37] they really want it um because they're [36:38] the only people who can really do [36:39] anything about that um and that's kind [36:41] of akin to uh what happens in 3D app [36:44] with culling and so forth the [36:45] application is responsible for doing [36:47] those sorts of things at the course [36:49] level um and the same would be true of [36:51] uh of I am gooey so again a couple [36:55] trade-offs there um so in closing um I [36:59] think there's a lot of really compelling [37:01] reasons to use this and I hope that uh [37:03] people experiment with it as I have and [37:05] I suspect that as more people use it if [37:08] it becomes popular uh then there will be [37:11] a lot more things that people find out [37:13] that I don't know about yet because I've [37:14] only been doing it uh in my you know for [37:17] my limited uses and I've only figured [37:19] out the things that were apparent to me [37:21] I'm sure that as other people use it [37:22] they will figure out more things and [37:24] come up with more clever ways to make [37:26] this stuff work standard graph user [37:27] interface libraries have had many many [37:29] many years of evolution and this has [37:30] only had you know basically my time um [37:33] that you know which has not even been [37:35] dedicated to it and it's already gotten [37:37] pretty far which makes me think that [37:38] it's a decent way of doing things [37:40] because it's so easy to make progress [37:41] there but I'd love to see what would [37:43] happen if a lot of other people started [37:45] banging on it and coming up with their [37:46] own ways of doing things and making [37:48] standard libraries and so on um I think [37:51] that would be really cool uh in terms of [37:53] more information about immedi graphical [37:55] user interfaces uh I know that Sean [37:58] Barrett is going to be writing up some [38:01] uh articles in game developer Magazine [38:03] on uh immediate mode graphic user [38:04] interfaces I don't know very much about [38:07] what he's done um along these fronts I [38:10] don't even know exactly how uh much R2 [38:13] styles of doing Medi mode graphic [38:14] experences uh have diverged since we [38:17] last talked about them um but uh that [38:20] would be a great place to look because I [38:21] think he's going to do a much more [38:22] thorough job than I did I know he's [38:23] going to I think put together some code [38:25] for that so you actually see how his [38:27] implementation actually runs and take a [38:29] look at it and he's going to actually [38:30] write some articles that go into it in [38:31] depth much better than I've done here um [38:34] so uh definitely look for that game [38:36] developer magazine um I think their [38:38] website is www.gm mag.com um and I don't [38:42] know if Sean will post them on his site [38:43] but his site is [38:45] www.s.org um either way and I also think [38:50] that maybe there might be some [38:52] commercial libraries coming up uh in the [38:54] future that will do this I uh uh know [38:58] that some people are possibly thinking [38:59] of it uh but I don't know what the St [39:02] says on that uh hopefully that would be [39:03] really nice to be able to have a [39:04] commercial library that was doing it um [39:07] but you know I don't know what to say [39:09] that hopefully that'll happen so that's [39:11] really all the information there is [39:12] again I haven't done much to disseminate [39:14] this and I apologize for that so as a [39:16] result there's not that much available [39:18] about it there really aren't any [39:18] articles um the only people who I know [39:21] um who've done anything with it or [39:22] people who I was you know kind of talked [39:25] with about it and who thought it was [39:26] cool just from you know cursory [39:28] description and so I think that you know [39:31] hopefully there'll be more about it [39:33] coming out soon that you can get your [39:34] hands on but I can't really point you to [39:35] it at this time um so that's about it [39:38] and uh this will all go on the Molly [39:41] rocket forums uh at [39:43] www.ol rocket.com m o l y r o c [39:47] k.com uh and SL forums um down in the uh [39:51] in the lectures forums so that you can [39:53] discuss it you can ask me questions if [39:55] you want or ask other people questions [39:56] on there um and sort of if you there's [40:00] things I didn't cover and this is very [40:01] cursory uh hopefully um I can clear up [40:04] some things in there that I haven't [40:05] touched on and so forth uh so I hope to [40:07] see you on there and I hope you enjoyed [40:09] this uh this kind of makeshift [40:10] presentation