Dice Accelerates Product Bets to Monthly Pace via AI Prototypes
Ai4
DHI Group increased feature experiment velocity from twice yearly to three times monthly by replacing legacy backend development with generative AI prototypes.
## Background
This presentation was uploaded to YouTube by the channel Ai4 on August 24, 2025. According to the page description, David Swegart, VP of Engineering at DHI Group, outlines how SaaS companies can shift mindset, policy, and process to move from AI experimentation to an AI-enabled organization.
## TL;DR
David Swegart shares DHI Group's journey of embedding AI across its product, people, and internal platforms. To tackle organizational challenges, Dice introduced an 'AI Amnesty' program to govern shadow IT, used generative AI as a temporary backend to dramatically accelerate product experimentation, and repositioned its data science team to evaluate vendors and build managed prompts. Furthermore, the company drove developer adoption through peer support rather than mandates, explored spec-driven development and 'vibe coding,' and highlighted emerging strategies like Generative Engine Optimization (GEO).
## Atmosphere & style
The presentation is practical, candid, and candidly modern, delivered without a presenter view as Swegart navigates his slides directly on stage. The narrative style balances pragmatic engineering insights with empathetic leadership lessons, making complex organizational changes feel accessible and grounded. Watching it feels like listening to an experienced tech leader openly share real-world playbooks, successes, and future-looking frontiers rather than a polished corporate sales pitch.
## Key takeaways
- [00:04](https://www.youtube.com/watch?v=u35Djx0bJCY&t=4s) David Swegart, VP of Engineering at Dice, outlines DHI Group's transition from ad-hoc AI experimentation to embedding AI into daily SaaS workflows.
- [01:07](https://www.youtube.com/watch?v=u35Djx0bJCY&t=67s) Dice declared an 'AI Amnesty' to invite non-punitive self-reporting of unsanctioned AI tools and establish a pragmatic cross-functional policy.
- [05:28](https://www.youtube.com/watch?v=u35Djx0bJCY&t=328s) To increase experiment velocity, Dice began using Generative AI as a temporary backend to rapidly validate product bets before committing heavy engineering resources.
- [05:28](https://www.youtube.com/watch?v=u35Djx0bJCY&t=328s) By leveraging AI for rapid prototyping, Dice increased feature experimentation from 1–2 bets per year to 2–3 active bets per month.
- [09:09](https://www.youtube.com/watch?v=u35Djx0bJCY&t=549s) The internal Data Science team was repositioned to advise business units directly and audit third-party software vendors selling thin API wrappers.
- [12:34](https://www.youtube.com/watch?v=u35Djx0bJCY&t=754s) Rather than enforcing top-down mandates, leadership used adoption data to pair high-performing AI power users with reluctant engineers via peer training.
- [14:50](https://www.youtube.com/watch?v=u35Djx0bJCY&t=890s) Automatic documentation generation in VS Code and lightweight 'vibe-coded' internal dashboards help DHI Group track tool adoption and derive tangible ROI.
- [15:33](https://www.youtube.com/watch?v=u35Djx0bJCY&t=933s) The future of software development is shifting toward spec-driven development, where engineers focus on defining high-level specifications for AI agents rather than writing manual code.
- [15:33](https://www.youtube.com/watch?v=u35Djx0bJCY&t=933s) As non-engineers gain the ability to build functional software with AI, tech organizations must rethink the primary role and core value of professional engineers.
- [15:33](https://www.youtube.com/watch?v=u35Djx0bJCY&t=933s) Companies must adopt Generative Engine Optimization (GEO) to optimize content visibility within conversational LLMs as users shift away from traditional search engines.
## Detailed walkthrough
### [00:04](https://www.youtube.com/watch?v=u35Djx0bJCY&t=4s) Introduction and Speaker Overview
David Swegart, Vice President of Engineering at Dice (dhi Group), presents on the transition from AI hype to organizational habits within a SaaS company. Managing the adoption of AI across engineering, product, and business units, Swegart highlights Dice’s role as a career marketplace for technologists in the United States, serving over 7.5 million members.
Following the public release of ChatGPT 3.5, organization-wide questions regarding the practical implementation, safety, and business value of artificial intelligence began emerging rapidly from executive leadership, technical teams, and internal stakeholders.
- **Speaker:** David Swegart, VP of Engineering at Dice.
- **Experience:** 15 years in software and data engineering.
- **Platform Reach:** Dice connects tech talent with companies and recruiters, boasting a membership base exceeding 7.5 million technologists.
---
[00:44](https://www.youtube.com/watch?v=u35Djx0bJCY&t=44s) The Four Operationalization Challenges
To transition through Gartner's AI maturity model, Dice identified four operational challenges that needed structured solutions:
```
+-----------------------------------------------------------------+
| AI MATURITY AT DICE |
+------------------------------------------------### Introduction and Speaker Background
David Swegart, Vice President of Engineering at DHI Group (Dice), introduces his session at AI4 titled "From Hype to Habits: Becoming an AI-Enabled SaaS Company." Facing the stage without a presenter view, he navigates his slides while outlining his background: 15 years in software and data engineering, currently leading AI adoption across Dice's engineering, product, and business teams.
Dice (dice.com) serves as a primary tech career marketplace in the U.S., connecting tech talent with companies and recruiters across a network of over 7.5 million members.
* **Speaker:** David Swegart, VP of Engineering at DHI Group (Dice).
* **Scale:** 7.5+ million technologist members on dice.com.
* **Mission:** Transitioning from early AI hype and ad-hoc experimentation to operational habits embedded in daily SaaS workflows.
---
[01:07](https://www.youtube.com/watch?v=u35Djx0bJCY&t=67s) Overcoming the AI Governance Challenge: AI Amnesty & Policy
Since the launch of ChatGPT 3.5, organizations have faced rapid adoption of unsanctioned AI tools. Business units and employees at Dice independently integrated widely available AI applications into daily routines, often outpacing centralized governance, legal policies, and IT visibility.
```
[ Unsanctioned AI Use ] ──► [ Declare AI Amnesty ] ──► [ Cross-Org AI Policy ]
(Shadow IT) (No Penalties) (Legal, Product, Eng)
```
To establish governance without hurting productivity or morale, Dice declared an "AI Amnesty." Employees were invited to disclose all unsanctioned tools they were using without fear of reprimand. This inventory revealed no severe security or PII leaks, enabling leadership to collaborate with legal, product, engineering, and business units to craft a pragmatic policy. Rather than stripping away tools, Dice sanctioned valid workflows, provided viable alternatives where needed, and instituted structured enablement programs with engineering code reviews.
* **Challenge 1:** Rapid adoption of shadow AI outstripping official governance and policy.
* **AI Amnesty:** Non-punitive self-reporting mechanism to uncover real-world usage patterns.
* **Policy Creation:** Cross-functional collaboration across Legal, Product, Engineering, and Business leads.
* **Enablement:** Technical reviews and direct guidance from engineering to ensure secure integration with internal systems.
---
[05:28](https://www.youtube.com/watch?v=u35Djx0bJCY&t=328s) Accelerating Product Bets with AI as a Temporary Backend
Traditional SaaS feature validation requires significant engineering effort, especially within legacy systems. Dice, operating with a 30-year codebase footprint, historically faced long feature release cycles. While industry benchmarks—such as those highlighted by Marty Cagan of the Silicon Valley Product Group—suggest top engineering teams run 10 to 20 bets per week, internal audits revealed Dice was executing only 1 to 2 experiments per year.
```
TRADITIONAL ML:
[ Data Gathering ] ──► [ Build ML Model ] ──► [ Deploy Feature ] (Months)
AI-ENABLED BETS:
[ Generative AI Backend ] ──► [ Live Experiment ] ──► [ Long-term Model ]
(Rapid Prototype) (User Validation) (If Validated)
```
By substituting Generative AI as an interim backend, product and design teams now build, prototype, and validate end-to-end features before engaging core engineering teams. This shift increased experiment velocity from 1–2 bets per year to 2–3 active bets per month.
* **Benchmark Shift:** Moved from 1–2 bets per year to 2–3 bets per month.
* **Target Goal:** Aligning toward high-performing benchmarks (10–20 bets per week).
* **Architecture Strategy:** Generative AI acts as a flexible, temporary backend to test customer demand before committing heavy machine learning or backend refactoring.
* **Early Risk Detection:** Product and design teams spot structural issues early in the prototyping phase, reducing unnecessary engineering back-and-forth.
---
[09:09](https://www.youtube.com/watch?v=u35Djx0bJCY&t=549s) Repositioning Data Science & Auditing Vendor Facades
Building heavy, custom machine learning models can take months before reaching customer-facing viability. Additionally, third-party software vendors often sell traditional ML features that are little more than thin wrappers around foundational model APIs (e.g., OpenAI, Anthropic), leading to unnecessary licensing overhead.
Dice repositioned its internal Data Science team to serve as the "tip of the spear" for AI innovation. Data scientists now engage directly with internal business units, lead internal technical forums, and evaluate third-party tools to identify redundant vendor wrappers.
* **Team Evolution:** Data Science transformed from a backend model builder into an active advisory and prototyping group for business teams.
* **Vendor Audits:** Direct evaluation of third-party vendors to replace costly API wrappers with internally managed LLM integrations.
* **Cost Reduction:** Replacing redundant ML vendor contracts with direct, managed prompt interfaces.
---
[12:34](https://www.youtube.com/watch?v=u35Djx0bJCY&t=754s) Driving Consistent Developer Adoption via Peer Training
Rolling out AI tooling across engineering and product teams revealed a performance split: high adopters achieved significant productivity gains (the "3x or 10x engineer"), while others abandoned the tools after initial trials due to unclear workflows.
To drive consistent adoption, leadership avoided top-down mandates or punitive usage metrics. Instead, usage data was used to identify adoption gaps, pairing AI power users with reluctant engineers through hands-on training sessions, pair programming, and open office hours.
* **Adoption Variance:** Disparity between power users experiencing exponential productivity gains and engineers dropping usage after minor friction.
* **Empathetic Approach:** Usage tracking was used to provide targeted assistance rather than performance penalties.
* **Enablement Strategy:** Establishing peer-led office hours, cross-training, and pair programming to transition teams away from manual coding toward LLM-assisted development.
***
How would you like to build on this summary?
[14:50](https://www.youtube.com/watch?v=u35Djx0bJCY&t=890s) Automated Documentation and Vibe-Coding Internal Dashboards
David Swegart highlights how developer productivity tools inside VS Code are transforming everyday engineering workflows by automatically generating documentation that engineers can easily review and approve. This streamline approach has led to a noticeably strong adoption rate across the organization. Taking this rapid prototyping philosophy a step further, Swegart shares a story about one of his directors who "vibe coded" a custom metrics dashboard to track usage statistics. Using ChatGPT, the director generated the entire application with essentially no manual coding effort.
While GitHub Copilot provides its own out-of-the-box telemetry, DHI Group needed a unified, tailored view as they begin incorporating other tools like Claude Code. Building their own lightweight dashboard allows management to blend key metrics across various platforms, keeping a close pulse on adoption, usage patterns, and real ROI with minimal initial overhead.
- VS Code integrations generate initial documentation drafts automatically for human review, resulting in strong team adoption.
- A director successfully "vibe coded" a custom Copilot tracking dashboard using ChatGPT without writing manual code.
- The custom dashboard was created to consolidate usage metrics from both GitHub Copilot and upcoming tools like Claude Code into a single view.
- The overarching goal of the internal dashboard is to continuously evaluate who is using which AI tools and ensure the organization derives tangible value from them.
[15:33](https://www.youtube.com/watch?v=u35Djx0bJCY&t=933s) Emerging Frontiers: Spec-Driven Development, Non-Engineering Builders, and GEO
Looking ahead, Swegart identifies three major shifts on the horizon for SaaS companies and tech organizations. The first is spec-driven development—a paradigm shift moving engineers away from writing individual functions toward defining precise requirements and specifications that AI agents execute against. Second, the rise of powerful AI tools means non-engineers are now capable of building functional software. This prompts fundamental strategic questions for technologist-focused organizations like Dice regarding how the software engineering role will evolve within the company.
Finally, Swegart introduces Generative Engine Optimization (GEO) as the AI-era counterpart to traditional Search Engine Optimization (SEO). As users transition from traditional search engines to conversational LLMs to discover products and services, companies must rethink how their content surfaces in AI-generated answers and proactively optimize for LLM visibility.
- **Spec-Driven Development:** Focus shifts from writing line-by-line code or specific functions to high-level requirements engineering.
- **Democratized Building:** Non-technical staff can now create software, forcing organizations to re-evaluate the primary value and core duties of professional engineers.
- **Generative Engine Optimization (GEO):** The strategy of optimizing content and brand presence so LLMs accurately recommend and cite the company in AI search responses.
[16:15](https://www.youtube.com/watch?v=u35Djx0bJCY&t=975s) Next Steps and Related Sessions on AI Talent
Concluding his presentation, Swegart addresses audience members who came to learn about broader recruitment, candidate trends, and the job market. He plugs an upcoming session hosted by his colleague, Brooke Loren, titled "Finding the Right Talent in the AI Area." The presentation leverages Dice's extensive data ecosystem to explore best practices for hiring, retaining, and upskilling talent to execute an organizational AI strategy.
- **Upcoming Presentation:** "Finding the Right Talent in the AI Area" by Brooke Loren at 4:25 PM in Room 163 (Level 1).
- **Core Topics Covered:** Data-backed strategies on retaining, training, and recruiting tech talent tailored to an enterprise AI strategy.
## Notable quotes
> [00:04](https://www.youtube.com/watch?v=u35Djx0bJCY&t=4s) From Hype to Habits: Becoming an AI-Enabled SaaS Company — David Swegart
> [05:28](https://www.youtube.com/watch?v=u35Djx0bJCY&t=328s) 1 to 2 experiments per year — David Swegart
> [05:28](https://www.youtube.com/watch?v=u35Djx0bJCY&t=328s) 2–3 active bets per month — David Swegart
> [12:34](https://www.youtube.com/watch?v=u35Djx0bJCY&t=754s) 3x or 10x engineer — David Swegart
> [16:15](https://www.youtube.com/watch?v=u35Djx0bJCY&t=975s) Finding the Right Talent in the AI Area — David Swegart
## Mentioned
David Swegart, Dice, DHI Group, ChatGPT 3.5, Gartner, Marty Cagan, Silicon Valley Product Group, OpenAI, Anthropic, VS Code, GitHub Copilot, Claude Code, ChatGPT, Brooke Loren, AI4
## Video description
David Swegart, VP of Engineering, DHI Group
Abstracts
How do you take a SaaS company from experimenting with AI to operating as an AI-enabled organization? With AI reshaping how we build software, organize teams, and deliver value to customers, SaaS companies need to shift mindset, policy, and process. In this talk, I’ll share what we’ve done at Dice to lean into AI across our product, people, and internal platforms starting with recognizing what tools people were already using and building guardrails instead of roadblocks. I’ll explain how we approached developer training versus broader org enablement, how we handle AI tools responsibly, and where we’re headed as AI begins writing more of our code. You’ll leave with practical ideas for applying AI in your own SaaS environment and a better understanding of what it really means to transform, not just bolt on.
Dice借助AI原型将产品试验加速至月度节奏
DHI Group通过用生成式AI原型替代传统后端开发,将功能试验速度从每年两次提升至每月三次。
David Swegart分享了DHI Group将AI融入其产品、团队和内部平台的历程。为应对组织层面的挑战,Dice推出了“AI特赦”计划以管控影子IT,利用生成式AI作为临时后端大幅加速产品试验,并重新定位其数据科学团队以评估供应商及构建受管提示词。此外,公司通过同行支持而非强制命令推动开发者的采纳,探索了基于规格的开发与“氛围编码”(vibe coding),并着重介绍了生成式引擎优化(GEO)等新兴策略。
Dice 推行了“AI 特赦”(AI Amnesty)机制,鼓励员工在免受处罚的前提下主动上报影子 AI 工具,以此联合法务、产品与工程团队制定务实的合规治理政策。 (1:07)
通过将生成式 AI 作为临时后端进行快速原型验证,Dice 将产品功能试错速度从过去每年的 1–2 次大幅提升至每月 2–3 次活跃实验。 (5:28)
内部数据科学团队被转型为直接面向业务的创新顾问,负责自研 LLM 整合方案并审查第三方供应商,淘汰溢价严重的大模型 API 套壳产品。 (9:09)
针对开发团队采用率不均的问题,Dice 拒绝自上而下的强行摊派,而是利用指标安排熟练用户与普通工程师结对辅导,并通过“氛围编码”(vibe-coding)自建仪表板追踪真实 ROI。 (12:34)
面向未来,软件研发正转向“规范驱动开发”(Spec-Driven Development),工程师的工作重心正从手工编写代码演进为向 AI Agent 输出高层次的规格与需求定义。 (15:33)
Reading summary
David Swegart, Vice President of Engineering at Dice (DHI Group), outlines how the tech career marketplace transitioned from ad-hoc AI experimentation to structured organizational habits by modernizing governance, product testing, internal developer workflows, and data science operations.
Uncovering Shadow IT Through AI Amnesty (1:07)
Following the release of ChatGPT 3.5, staff across Dice adopted external AI tools independently, outpacing corporate IT visibility and centralized legal policy. To resolve this governance gap without suppressing productivity or morale, leadership instituted an 'AI Amnesty,' inviting employees to disclose all unsanctioned tools without fear of reprimand.
The resulting inventory revealed no severe personal data or security leaks. This transparency allowed cross-functional leaders from legal, product, and engineering to establish a formal AI policy that sanctioned viable workflows, replaced risky services with compliant alternatives, and added engineering code reviews to guide secure implementation.
Accelerating Experiments With Generative AI Backends (5:28)
Hampered by a 30-year codebase footprint, Dice historically executed only one to two product experiments per year, falling far behind top industry benchmarks cited by Marty Cagan of 10 to 20 bets per week. Building conventional machine learning models required months of development before teams could validate customer interest.
To compress this cycle, product and design teams began using generative AI as a temporary, flexible backend. This architecture allows teams to test live prototypes directly with users before committing core engineering resources to heavy model development or backend refactoring, raising Dice's experiment velocity to two to three active bets per month.
Refocusing Data Science and Developer Enablement (9:09)
Dice shifted its Data Science team from traditional, isolated model building into an advisory and prototyping unit. The team audits commercial software vendors to identify thin wrappers around foundational model APIs, allowing the company to replace costly external licensing with direct, internally managed prompt interfaces.
Simultaneously, engineering rollouts revealed wide adoption disparities, with power users accelerating output while others stopped using the tools after minor friction. Rather than applying top-down usage mandates, leadership used telemetry to identify adoption gaps, pairing advanced users with hesitant engineers through peer training, open office hours, and pair programming.
Lightweight Telemetry and Emerging Engineering Shifts (14:50)
To monitor adoption across multiple platforms like GitHub Copilot and Claude Code, an engineering director 'vibe coded' a unified telemetry dashboard using ChatGPT without manual coding. This prototyping approach mirrors broader workflow gains, such as automated documentation generation in VS Code, which has seen strong internal adoption.
Swegart points to three upcoming industry shifts: spec-driven development that directs engineers to write requirements rather than line-by-line functions, the ability of non-engineers to build functional software, and Generative Engine Optimization (GEO) to preserve brand discovery within conversational search engines.
While interim generative backends and peer enablement have measurably accelerated Dice's experimentation velocity, the emergence of spec-driven development and non-technical software builders leaves open the strategic question of how professional engineering roles will fundamentally redefine their value.
## Background
该视频由 Ai4 频道于 2025 年 8 月 24 日发布在 YouTube 平台上。视频记录了 DHI Group 旗下 Dice 的工程副总裁 David Swegart 在大会上的主题演讲,主要分享了 SaaS 企业 Dice 如何从最初的 AI 炒作与零星尝试,逐步转向在产品开发、团队组织和内部平台中全面拥抱并融入 AI 的转型历程。
## TL;DR
演讲者深入探讨了 Dice 在成为 AI 赋能的 SaaS 公司过程中面临的四大核心挑战与应对策略。Dice 率先通过实施“AI 特赦”和建立联合 AI 政策,解决了未经授权工具滥用的问题;接着利用 AI 作为临时后端突破工程瓶颈,将新功能验证速度从一年 1–2 次提升至每月 2–3 次;同时重新定位数据科学团队职责以取代高昂的第三方套壳 API,并通过结对编程与答疑辅导解决员工 AI 工具使用率不均的问题。此外,演讲还展望了规范驱动开发、非技术人员构建能力以及生成式引擎优化(GEO)等前沿趋势。
## Atmosphere & style
整场演讲风格专业且不失幽默接地气,David 随和亲切的表达拉近了与听众的距离。他结合自身丰富的实践案例,坦诚直面传统企业在落地 AI 时遭遇的阵痛与尴尬,通过生动的对比和扎实的数据让分享兼具说服力与启发性。听众能在紧凑有力的节奏中感受到清晰的战略思考与充满活力的实操体验。
## Key takeaways
- [00:04](https://www.youtube.com/watch?v=u35Djx0bJCY&t=4s) Dice.com 拥有超过 750 万名技术会员,是美国领先的技术人才职业市场,致力于连接人才、用人单位与招聘人员。
- [01:04](https://www.youtube.com/watch?v=u35Djx0bJCY&t=64s) ChatGPT 发布后,企业在回答关于 AI 落地疑问的过程中,推动了自身在 AI 运营化上的成熟度演进。
- [02:35](https://www.youtube.com/watch?v=u35Djx0bJCY&t=155s) 未经授权的 AI 工具爆发会导致缺乏可见性,Dice 采取“AI 特赦”鼓励主动申报,而非一刀切禁止。
- [04:00](https://www.youtube.com/watch?v=u35Djx0bJCY&t=240s) Dice 通过联合法务、工程和业务干系人制定联合 AI 政策,并对复杂工具提供技术审查与替代方案。
- [05:26](https://www.youtube.com/watch?v=u35Djx0bJCY&t=326s) 传统 SaaS 遗留代码导致开发验证周期极长,Dice 历史上每年仅能验证 1 到 2 个新想法。
- [06:52](https://www.youtube.com/watch?v=u35Djx0bJCY&t=412s) Dice 利用 LLM 作为临时后端,让产品经理与设计团队快速验证功能原型,成功将验证速度提升至每月 2 到 3 个。
- [09:09](https://www.youtube.com/watch?v=u35Djx0bJCY&t=549s) 传统数据科学项目过于复杂,且许多高价 AI 供应商只是在 LLM API 外加了封装外壳。
- [10:05](https://www.youtube.com/watch?v=u35Djx0bJCY&t=605s) Dice 重新定位数据科学团队为 AI“尖刀部队”,带头探索新模型并自主开发 API 封装以优化采购成本。
- [12:34](https://www.youtube.com/watch?v=u35Djx0bJCY&t=754s) 员工对 AI 工具的使用呈现两极分化,Dice 采用数据驱动培训,通过明星工程师结对编程与答疑辅导帮扶员工。
- [14:50](https://www.youtube.com/watch?v=u35Djx0bJCY&t=890s) 技术总监通过“Vibe Coding”利用 ChatGPT 快速构建了针对 GitHub Copilot 等工具的指标分析仪表盘。
- [15:33](https://www.youtube.com/watch?v=u35Djx0bJCY&t=933s) 规范驱动开发(Spec-Driven Development)将成为趋势,未来更注重根据高层级规范由 AI 生成系统逻辑。
- [15:47](https://www.youtube.com/watch?v=u35Djx0bJCY&t=947s) 非技术人员借助 AI 拥有构建软件的能力,促使企业重新思考专业工程师的角色与核心价值。
- [16:00](https://www.youtube.com/watch?v=u35Djx0bJCY&t=960s) 生成式引擎优化(GEO)成为新焦点,企业需研究并优化产品信息在各类 LLM 生成结果中的展现方式。
## Detailed walkthrough
### [00:04](https://www.youtube.com/watch?v=u35Djx0bJCY&t=4s) 演讲背景与 DICE 平台简介
DHI Group 旗下 **Dice.com** 的工程副总裁 **David Swegart** 在 Ai4 大会上发表演讲,主题为《从炒作到习惯:将 DHI Group 打造为 AI 赋能的 SaaS 企业》(*From Hype to Habits: Becoming an AI-Enabled SaaS Company With DHI Group*)。演讲开头,David 幽默地解释由于现场没有演讲者提示屏(Presenter view),自己可能会因为记不住所有 Slide 细节而显得姿势略微僵硬。
David 在软件与数据工程领域拥有约 15 年的工作经验,目前在 Dice 主要领导工程、产品以及业务团队对人工智能技术的全面落地应用。Dice.com 是美国领先的技术人才职业市场(Career marketplace),拥有超过 750 万名技术会员,致力于连接技术人才、用人单位与招聘人员。正如他总结的:“如果你的企业需要技术人才,Dice 拥有你所需的一切。”
- **演讲者**:David Swegart(Dice 工程副总裁),拥有 15 年软件与数据工程经验。
- **平台规模**:Dice.com 拥有超过 750 万技术领域会员。
---
### [01:04](https://www.youtube.com/watch?v=u35Djx0bJCY&t=64s) 企业面临的 AI 疑问与成熟度演进路径
自 ChatGPT 3.5 发布后的 6 到 12 个月起,Dice 内部以及高层、业务干系人之间开始频繁涌现关于 AI 的各种疑问。企业在寻找这些问题答案的过程中,逐步推动了自身在 AI 运营化(Operationalization)上的成熟度演进。
David 引用了 Gartner 提出的 AI 成熟度模型(Maturity Model),指出 Dice 在推进 AI 运用的过程中,总结出了四大关键挑战以及对应的解决方案。这四个核心挑战涵盖了从未经授权工具的使用、工程瓶颈与新功能验证、传统数据科学模式的滞后,到 AI 工具普及率不均等实际难题。
---
### [02:35](https://www.youtube.com/watch?v=u35Djx0bJCY&t=155s) 挑战一:未经授权 AI 工具的爆发与特赦治理
当团队意识到生成式 AI 的价值后,各部门员工开始迅速尝试并引入各式各样的第三方 AI 工具。在 Dice 内部,这种未经授权的工具使用(Unsanctioned AI tools)呈爆发式增长,其推广速度远超中央政策与安全审查的更新步伐。安全与数据团队面临的主要困难在于 Lack of Visibility(缺乏可见性):
- 无法清晰掌握谁在何时使用了哪款工具。
- 难以确定输入给工具的数据是否合规、是否存在泄露风险。
- 部分员工甚至通过公司发票流程(而非个人信用卡)直接采购工具,但未能确保其最大化产生业务价值。
针对这一困局,Dice 没有选择“一刀切”的封杀模式,因为一味禁止会极大地破坏员工已经建立的高效工作流。Dice 借鉴了 Built In 的思路,采取了以下“安全且合法”的推进策略:
1. **宣布“AI 特赦”(AI Amnesty)**:明确告知全员,此前使用未经授权的工具不会受到任何惩罚或处分,鼓励大家主动申报正在使用的工具。事后调查显示,并没有员工在工具中泄露敏感的 PII(个人身份信息)。
2. **制定联合 AI 政策(AI Policy)**:基于特赦收集到的工具清单,工程团队联合法务、高层领导、产品团队及各业务单元干系人共同制定了合规政策。对能够产生实际价值的工具予以正式收录;对不合规的工具则提供安全的替代方案。
3. **结构化赋能与技术审查(Structured Enablement)**:推出内部培训与沟通机制。针对大型部门引入的复杂 AI 工具,由工程团队开展技术审查,帮助其进行正确的架构对接,确保 AI 能够与内部系统无缝连通并发挥最大效益。
---
### [05:26](https://www.youtube.com/watch?v=u35Djx0bJCY&t=326s) 挑战二:破解工程瓶颈,加速新功能验证
在传统的 SaaS 软件开发中,即便是轻量级的功能验证与测试,也往往依赖大量工程资源的投入。Dice 作为一家拥有近 30 年历史的企业,其平台中不可避免地积累了许多遗留代码(Legacy code)。在老旧架构上开发新功能耗时极长,限制了快速迭代的能力。
根据 Silicon Valley Product Group(硅谷产品集团)负责人 Marty Cagan 的观点,顶尖工程团队每周通常能够进行 10 到 20 次新功能尝试(Bets)。当 David 向产品负责人询问 Dice 的测试频率时,得到了令人吃惊的对比:
```
Marty Cagan 行业标准: ████████████████████ 10 - 20 次 / 周
Dice 预期目标: ██ 1 - 2 次 / 周
Dice 实际历史状况: █ 1 - 2 次 / 年
```
为了改变一年只能验证 1 到 2 个新想法的困局,Dice 利用 **AI 作为临时后端(AI as the Backend)** 重塑了开发路径,成功将验证速度提升至**每月 2 到 3 个**。
在传统机器学习(ML Engineering)模式下,团队需要先经历繁琐的数据收集、模型训练与部署,才能将产品呈现在用户面前。而在新型 AI 工程(AI Engineering)模式下,团队可以直接利用 LLM 作为产品的临时后端,快速构建出具备完整交互功能的原型并直接推向客户收集反馈。
```
传统 ML 开发流程:
[ 数据收集 ] ──> [ 模型构建/训练 ] ──> [ 交付产品原型 ] ──> [ 客户测试反馈 ]
(耗时数月,验证周期极长)
AI 驱动开发流程 (Dice 模式):
[ LLM 临时后端 ] ──> [ 快速交付产品 ] ──> [ 客户测试反馈 ] ──> (验证成功后) ──> [ 沉淀为长期架构 ]
(耗时数天/数周,快速试错)
```
这种模式甚至赋予了产品经理(PM)和 UI/UX 设计师端到端(End-to-End)自主构建完整功能原型的能力。他们在无需工程团队编写任何底层代码(Hand touches keyboard on code)之前,就能提前发现设计逻辑中的纰漏,大大缩短了跨部门摩擦与沟通成本。
---
### [09:09](https://www.youtube.com/watch?v=u35Djx0bJCY&t=549s) 挑战三:传统数据科学模式转型与供应商成本优化
传统的机器学习与数据科学项目往往过于庞大和复杂。即便客户需求非常明确,数据科学团队也需要数月时间才能将模型调整到“不跑偏”的稳定状态。同时,市面上涌现出大量宣称提供“AI 解决方案”的第三方供应商,但经过深入评估,Dice 发现许多供应商本质上只是在 OpenAI 或 Anthropic 的 API 外面套了一层“外壳”(Facade over API),收费却极其昂贵。
为了应对这一挑战,Dice 重新定位了数据科学团队的职责与使命:
- **充当 AI 的“尖刀部队”(Tip of the spear)**:数据科学家对最新模型最敏感,也懂得如何安全、高效地实施部署。Dice 鼓励他们从繁重的传统机器学习维护中抽身,直接带头探索最新的 AI 模型并开展快速验证。
- **促进跨部门协同**:以往很少与数据科学团队直接沟通的业务部门,现在通过内部论坛、Guilds(公会)及联合项目,频繁向数据团队请教 AI 应用方案,大大提高了数据团队的参与度与影响力。
- **审查供应商与优化成本**:数据科学团队被赋予了一项新任务——评估高价供应商的底层逻辑。如果供应商只是简单封装了 LLM API,数据团队就会评估是否可以在内部通过自主开发 API 封装来直接替代,从而为公司节省大量不必要的软件采购支出。
---
### [12:34](https://www.youtube.com/watch?v=u35Djx0bJCY&t=754s) 挑战四:打破 AI 工具使用壁垒,促进持续普及
尽管 Dice 已经向全员及工程/产品团队开放了 AI 工具,但实际的使用情况却呈现严重的“两极分化”:
- 一部分积极拥抱 AI 的工程师成为了“3x 甚至 10x 工程师”,生产力大幅飞跃。
- 另有一部分员工在初次尝试后,认为 AI 只是个“酷炫的小把戏”,没有看到深层价值,随后便放弃了使用。
由于部分工具按实际使用量(Usage-based)计费,使用率低虽然不会造成额外的软件许可浪费,但却无法达成企业预期的整体效能提升。
为了解决普及率不均的问题,Dice 采取了数据驱动的培训策略。管理层通过提取后台使用指标(Metrics),精准定位未熟练使用 AI 的员工。这种排查并非为了惩责或考核(“你不用工具你就被开除”),而是为了主动问诊与帮扶。Dice 安排了对 AI 充满热情的“明星工程师”作为导师,通过**结对编程(Pairing)**和**设置 Office Hours(答疑专场)**的方式,手把手教其他同事如何在日常工作中高效利用 ChatGPT 等 LLM 工具,引导大家逐步摆脱“每行代码都必须手写”的传统习惯。
### [14:50](https://www.youtube.com/watch?v=u35Djx0bJCY&t=890s) VS Code 工具集成与 GitHub Copilot 数据看板
在讲解 AI 工具在开发流程中的落地时,演讲者提到了他们在 VS Code 中所使用的生成式工具。该工具可以直接为开发者生成代码或文档,随后开发者只需对输出结果进行评审与确认即可。通过这种流程,团队观察到了非常显著的使用率提升和效率增长。
为了监控这些 AI 辅助工具的实际应用效果与投资回报,团队的一位技术总监(Director)通过“Vibe Coding”(氛围编码)的方式,利用 ChatGPT 在极短时间内构建出了一个无缝挂载在 GitHub Copilot 之上的分析仪表盘(Dashboard)。尽管 GitHub Copilot 本身提供了一套内置的指标统计,但 DHI Group(Dice)计划未来进一步引入 Claude Code 等其他开发工具,因此需要一个能够将多源工具数据融合统一的分析看板。这种由总监通过 AI 零投入/极低成本快速构建出的分析面板,帮助管理团队时刻掌握“谁在使用什么工具、使用频率如何、是否真正产生了业务价值”等核心问题。
- **VS Code 工具体验**:生成式工具自动完成代码/文档构建,开发者转为“审查者”角色,显著提高了工具采纳率。
- **Vibe Coding 实操**:技术总监完全利用 ChatGPT 编写生成了一个针对 GitHub Copilot 的数据统计仪表盘,几乎“零额外工作量”。
- **多工具指标融合**:看板不仅监控现有的 GitHub Copilot 使用数据,还为未来融合 Claude Code 等其他 AI 编码工具的指标奠定了基础。
---
### [15:33](https://www.youtube.com/watch?v=u35Djx0bJCY&t=933s) 规范驱动开发、非技术人员构建与生成式引擎优化(GEO)
面对 AI 技术的快速演进,演讲者提出了团队正在密切关注与积极布局的三个前沿趋势:
1. **规范驱动开发(Spec-Driven Development)**:区别于以往手动编写逐行代码或仅让 AI 辅助编写特定单体函数的模式,未来的开发将更加以“需求和规范”为核心驱动力,由 AI 根据高层级规范来生成大块系统逻辑。
2. **非技术人员的构建能力(Non-Engineers Who Can Build)**:Dice 作为技术人才领域的专家企业,敏锐地注意到非工程背景的人员正借助 AI 拥有软件构建能力。这一变化引发了深层的组织思考:在人人皆可构建的时代,专业工程师将扮演什么新角色?他们在企业组织中的核心价值又该如何重新定义?
3. **生成式引擎优化(Generative Engine Optimization, GEO)**:作为传统搜索引擎优化(SEO)在大语言模型(LLM)时代的衍生概念,GEO 正在成为全新的关注焦点。团队正深入研究产品信息与内容如何在各类 LLM 模型的生成结果中展现,以及如何针对这些生成式引擎进行针对性的优化与曝光提升。
---
### [16:15](https://www.youtube.com/watch?v=u35Djx0bJCY&t=975s) 演讲结语与后续 AI 人才讲座推荐
在演讲的尾声,David Swegart 向现场听众做出了贴心的延伸推荐。他提到,如果参会者来到现场是希望深入了解候选人、招聘方、IT 招聘市场趋势以及相关数据,那么他的同事 Brooke Loren 将在随后的一个小时内带来精彩分享。
Brooke Loren 的演讲主题为“在 AI 时代寻找与留住合适人才”(Finding the Right Talent in the AI Area),涵盖了如何招募、培训、留存人才,以及如何构建匹配企业 AI 战略落地的人才梯队。该演讲基于 Dice 作为专业技术招聘平台所收集的庞大行业数据。大卫强烈推荐感兴趣的听众前往 1 楼 163 房间(Level 1, Room 163)参加该场讲座,并表示自己也会亲自到场聆听。
- **推荐讲座主题**:AI 时代的人才发现、培训、留存与 AI 战略落地执行。
- **演讲人与时间地点**:Brooke Loren;下午 4:25,1 楼 163 房间(Level 1, Room 163)。
- **数据支撑**:演讲内容基于 Dice 平台长期积累的真实技术人才市场大数据。
## Notable quotes
> [00:04](https://www.youtube.com/watch?v=u35Djx0bJCY&t=4s) 如果你的企业需要技术人才,Dice 拥有你所需的一切。 — David Swegart
> [02:35](https://www.youtube.com/watch?v=u35Djx0bJCY&t=155s) AI 特赦 — David Swegart
> [05:26](https://www.youtube.com/watch?v=u35Djx0bJCY&t=326s) 10 - 20 次 / 周 — Marty Cagan
> [05:26](https://www.youtube.com/watch?v=u35Djx0bJCY&t=326s) 1 - 2 次 / 年 — David Swegart
> [05:26](https://www.youtube.com/watch?v=u35Djx0bJCY&t=326s) 每月 2 到 3 个 — David Swegart
> [05:26](https://www.youtube.com/watch?v=u35Djx0bJCY&t=326s) AI 作为临时后端 — David Swegart
> [09:09](https://www.youtube.com/watch?v=u35Djx0bJCY&t=549s) 尖刀部队 — David Swegart
> [12:34](https://www.youtube.com/watch?v=u35Djx0bJCY&t=754s) 3x 甚至 10x 工程师 — David Swegart
> [16:15](https://www.youtube.com/watch?v=u35Djx0bJCY&t=975s) 在 AI 时代寻找与留住合适人才 — David Swegart
## Mentioned
David Swegart, Dice.com, DHI Group, Ai4, Gartner, Built In, Marty Cagan, Silicon Valley Product Group, OpenAI, Anthropic, ChatGPT, VS Code, GitHub Copilot, Claude Code, Brooke Loren
## Video description
David Swegart, VP of Engineering, DHI Group
Abstracts
How do you take a SaaS company from experimenting with AI to operating as an AI-enabled organization? With AI reshaping how we build software, organize teams, and deliver value to customers, SaaS companies need to shift mindset, policy, and process. In this talk, I’ll share what we’ve done at Dice to lean into AI across our product, people, and internal platforms starting with recognizing what tools people were already using and building guardrails instead of roadblocks. I’ll explain how we approached developer training versus broader org enablement, how we handle AI tools responsibly, and where we’re headed as AI begins writing more of our code. You’ll leave with practical ideas for applying AI in your own SaaS environment and a better understanding of what it really means to transform, not just bolt on.
[Source](https://www.youtube.com/watch?v=u35Djx0bJCY)
Source quotes
[0:04] David Swegart: 如果你的企业需要技术人才,Dice 拥有你所需的一切。
[2:35] David Swegart: AI 特赦
[5:26] Marty Cagan: 10 - 20 次 / 周
[5:26] David Swegart: 1 - 2 次 / 年
[5:26] David Swegart: 每月 2 到 3 个
[5:26] David Swegart: AI 作为临时后端
[9:09] David Swegart: 尖刀部队
[12:34] David Swegart: 3x 甚至 10x 工程师
[16:15] David Swegart: 在 AI 时代寻找与留住合适人才
Source quotes
[0:04] David Swegart: From Hype to Habits: Becoming an AI-Enabled SaaS Company
[5:28] David Swegart: 1 to 2 experiments per year
[5:28] David Swegart: 2–3 active bets per month
[12:34] David Swegart: 3x or 10x engineer
[16:15] David Swegart: Finding the Right Talent in the AI Area
Original transcript
[0:04] All right. So, hyped to habits, becoming
[0:05] an AI enabled SAS company. It's what
[0:08] we're going to talk about. Uh, just FYI,
[0:10] hope everybody's enjoying the AI4
[0:12] presentation so far. I don't have a
[0:14] presenter view here, so I'm going to be
[0:15] kind of a little bit awkwardly
[0:17] positioned because I don't have all my
[0:18] slides completely memorized. So, um,
[0:23] all right,
[0:25] next slide.
[0:28] All right. So, a little bit about me
[0:29] before we start. Um, I'm the vice
[0:30] president of engineering at DICE. Uh,
[0:32] I've been working in software and data
[0:34] engineering for about 15 years. Um, and
[0:37] right now I'm leading our adoption
[0:38] across engineering product and business
[0:40] teams at DICE for AI. Next slide. So,
[0:44] little about DICE before we start. Um,
[0:46] dice.com is the leading career
[0:47] marketplace for technologists in the
[0:49] United States with over 7.5 million
[0:51] members. So, we help um technologists
[0:54] and companies and recruiters connect uh
[0:56] and find jobs and and place candidates.
[0:59] Uh if you need tech tal Oh, sorry. If
[1:01] you need tech talent, DICE has it. Next
[1:04] slide. All right. So, getting into our
[1:07] content problem. AI is everywhere and so
[1:10] are the questions. So, it's likely that
[1:12] you've had some of these questions or uh
[1:14] your business has some of these
[1:15] questions um or your stakeholders or
[1:18] people that you're talking to have some
[1:19] of these questions. um you know since
[1:20] chatbt 3.5 released you know about 6 to
[1:24] 12 months after that we I started
[1:25] getting some of these questions as well
[1:27] as asking some of these questions myself
[1:29] um and so as part of this journey in
[1:32] answering these questions next slide
[1:35] um and then there's an animation so I'll
[1:37] go ahead and say say click
[1:40] all right so in answering these
[1:42] questions that kind of took us on our
[1:43] journey and what um you know an easy if
[1:46] you if you've read any of the Gartner
[1:47] content you've seen these maturity
[1:49] models that are out there before.
[1:50] There's plenty of these. Um, this one's
[1:52] their one for AI. Um, and so, you know,
[1:55] in answering these questions, we kind of
[1:57] matured our operationalization of AI at
[1:59] DICE. And so, I wanted to talk through
[2:01] next slide our four kind of four
[2:03] challenge four challenges and the
[2:05] solutions to those challenges to give
[2:06] you a little bit of sense of how we
[2:08] operationalized
[2:10] AI at DICE and are getting value from
[2:12] that. Um so first we're going to talk a
[2:14] little bit about the use of unsanctioned
[2:16] AI tools um outpacing governance at the
[2:18] company. Uh engineering bottleneck and
[2:20] new feature validation then traditional
[2:22] data science approaches that can't keep
[2:24] pace with AI. Uh and then all finally
[2:27] we'll talk about AI tools um being
[2:29] available but not consistently adopted
[2:31] building on some of the previous topics
[2:33] we talked about.
[2:35] So okay so ch so so challenge one um use
[2:38] of unsanctioned AI tools outpacing
[2:40] governance. Um, so I'm guessing
[2:42] everybody here, you know, as soon as as
[2:44] soon as we realized the value of these
[2:46] tools, everybody started diving in using
[2:47] them. At DICE, it was no different. We
[2:49] we had plenty of people that were
[2:51] looking to use these tools. Um, plenty
[2:53] of uh, let's say unsanctioned use of
[2:56] these tools, and it was it was an
[2:57] explosion of these tools. um talking
[2:59] with some of our um security folks, our
[3:02] data folks, um even even people that I
[3:04] work with on a daily basis, um we saw
[3:06] departments quickly uh embracing widely
[3:10] available AI tools, often outpacing
[3:12] centralized policies reviews. In
[3:14] addition, um we had some visibility
[3:16] challenges. We didn't know who was using
[3:17] what, what were they using it for, and a
[3:19] lot of times people even had permissions
[3:21] to use certain tools. They had uh
[3:23] invoicing process set up. It wasn't on a
[3:25] personal credit card. um but we weren't
[3:27] sure that they were feeding the right
[3:28] data into those tools and leveraging
[3:30] those the right way to get value for a
[3:32] business and have the right impact. So
[3:35] next slide. So what did we do about
[3:37] that? Um so we enabled safe sanctioned
[3:39] AI adoption. Um I think the biggest one
[3:41] and I'll thank uh uh builtin.com for for
[3:44] this for this idea was the AI amnesty.
[3:47] So we basically declared AI and amnesty
[3:49] said that's okay. You know it's it's we
[3:52] understand you've been using
[3:53] unsanctioned tools. you know, in digging
[3:55] in, just to be clear, there wasn't any
[3:56] any like egregious use of people pumping
[3:58] all sorts of uh PI or anything like that
[4:00] in these tools. Um, but we we said
[4:03] that's okay that you're using these
[4:04] tools. There's no penalty. Um, just tell
[4:06] us what you're using them. And we we
[4:07] basically had an AI MST where we allowed
[4:09] everybody to kind of declare that they
[4:11] were using these tools. We used that as
[4:13] kind of a base of what our AI policy
[4:14] was. So we we dovetailed that with a
[4:16] with a policy we put together with legal
[4:19] um with leadership uh the engineering
[4:21] team's product as well as stakeholders
[4:22] from each of our business units and
[4:24] built a policy that we kind of was found
[4:27] founded on that amnesty and the tools
[4:29] that people were using. So we said look
[4:30] we understand you're using this we can
[4:32] see how this fits in in in your daily uh
[4:35] life. again turning those things off
[4:37] would have had ma you know once people
[4:38] realized the value they would have had
[4:40] massive impact um to their daily uh
[4:43] workflow if we would just come through
[4:44] and said no you can't use that anymore.
[4:46] So in certain cases we did offer
[4:47] alternatives to folks. Um but uh we we
[4:51] did we did end up scooping up quite a
[4:52] few of those tools and and putting them
[4:54] in our policy. Um on top of that we uh
[4:57] also had a structure enablement program.
[5:00] So we deployed some training internally.
[5:02] We also had um communication and and we
[5:05] also had technical reviews for the
[5:06] larger implementations for some of the
[5:08] departments of the AI tools that we're
[5:10] they were using and we look through with
[5:12] some of the folks on the engineering
[5:13] team and said what are you doing? Can we
[5:16] you know can we help you implement this
[5:18] correctly? Can we make sure it connects
[5:19] to the right systems and is getting the
[5:21] right answers because we want this to be
[5:22] to have the maximum impact for you.
[5:26] All right, next next issue or next
[5:28] challenge um engineering bottleneck and
[5:30] new feature validation. So um you know
[5:33] for for those that are in engineering or
[5:35] work closely with engineering teams um
[5:37] even even lightweight feature feature
[5:39] validation rolling them out um uh you
[5:42] know requires engineering work. At DICE
[5:44] we have as with most places we have a
[5:47] platform we have legacy code legacy
[5:50] things running right legacy code is just
[5:52] code that somebody else wrote. Uh but we
[5:54] we have a lot of those things there and
[5:56] and we have to deal with those. Dice has
[5:57] been around for about 30 years and we
[5:59] have in those in dealing with things in
[6:01] those platforms we have to um be
[6:03] cognizant that adding new features
[6:05] things like that is going to take time
[6:06] and so you know if you think about a SAS
[6:09] company you're like well you should be
[6:10] experimenting all the time and maybe you
[6:12] are and and good for you know I showed
[6:14] the Gartner matrix or the Gartner uh
[6:17] maturity model at the beginning we're in
[6:19] we were in the process of growing that
[6:21] and you know for years we'd always
[6:23] talked about wanting to do more
[6:24] experiments quickly if you think about
[6:26] Marty K Kagan um who's in charge of the
[6:29] Silicon Valley product group is kind of
[6:30] a thought leader in that area. He says
[6:32] um top engineering teams typically do 10
[6:35] to 20 bets a week. So I went to my
[6:36] product counterpart and said that's a
[6:39] really interesting stat. I know we're
[6:40] not there. What do you think we're at?
[6:41] And she said about 1 to two. And I said
[6:43] wow 1 to two a week. We've got a little
[6:45] ways to go but we're doing okay. And she
[6:46] said no one to two a year. So we had a
[6:48] ways to go um to do that. And what we
[6:51] ended up doing was um uh oh sorry I
[6:56] meant to go to the next slide. What we
[6:58] ended up doing was shifting from testing
[7:00] one to two two ideas a year to two two
[7:03] to three a month by using AI as the
[7:04] backend. So I like this model from the
[7:06] pragmatic engineer um which kind of
[7:09] shows that traditionally we were focused
[7:11] on taking you know building get getting
[7:13] data putting it into a model um and then
[7:16] the product would come out even if you
[7:17] were or or you'd be able to launch the
[7:19] product as an experiment to customers
[7:20] even if you were doing this relatively
[7:23] um quickly it would still take time it
[7:25] would still take time to gather that
[7:26] data and have something that was
[7:27] effective for your customers. And when
[7:30] you switch from what was traditionally
[7:31] kind of AI AI engineering to or excuse
[7:34] me ML engineering to AI engineering, you
[7:36] can use that as the backend for the
[7:38] product quickly and roll that out. And
[7:40] we've already launched several several
[7:42] things to put in front of our customers
[7:44] um that that are based on those just by
[7:46] essentially taking AI having it serve as
[7:49] the the interim or temporary backend
[7:51] while we while we brought that actual
[7:53] product in front of customer and said do
[7:55] you like this? Do you like using this?
[7:56] Give us some feedback. And then if
[7:58] that's something that they like that
[7:59] that top model doesn't necessarily go
[8:01] away. It's just moved further out and
[8:03] something you use a little bit um down
[8:05] down the road once customers show
[8:07] interest in that. But you can have that
[8:08] kind of fully working model in front of
[8:10] them. Additionally, a lot of these tools
[8:12] are allowing folks from product and
[8:14] design to also uh build models end to
[8:18] end or sorry build features end to end
[8:20] themselves before they hand them off to
[8:21] engineering teams or even in some cases
[8:23] like I said place those bets to um
[8:26] customers and or or excuse me take those
[8:28] bets and give them to customers and see
[8:30] if they like them before we engage
[8:31] engineering teams. So um we're actually
[8:34] working with um a couple of folks on our
[8:36] product and design team who are trying
[8:37] to um build things almost end to end get
[8:40] that feedback and then roll it out. The
[8:42] the nice part too is that a lot of times
[8:44] those uh engineer or those product and
[8:46] design folks end up uh being able to see
[8:49] the trouble with their idea first before
[8:51] they get engineering involved. And if
[8:53] anybody's been engineering, they know
[8:54] they're like, "Well, this won't work
[8:56] this way. You can't do it this way." And
[8:57] then that that's the cycle and the back
[8:59] and forth. And that's fine. That's part
[9:00] of the natural process. But a lot of
[9:02] times these product and design folks can
[9:04] realize that sooner before engineers
[9:06] engage in a hand touches keyboard on
[9:07] code.
[9:09] Next slide. All right. So uh next
[9:12] challenge traditional data science
[9:13] approaches can't keep pace with AI. So
[9:16] you know I think when we we we had a
[9:19] data science team we were maintaining
[9:21] and launching a number of like
[9:22] structured machine learning models. um
[9:25] they were relatively complex and I'm not
[9:28] going to propose as part of this
[9:29] presentation that that that workflow
[9:31] doesn't go away. I'm just proposing
[9:33] maybe um that's fit for larger um more
[9:37] complex implementations and that you
[9:38] should take a step back and look at that
[9:40] before you before you commit to building
[9:43] something that large. So um again with
[9:46] uh you know even when customer demand is
[9:48] clear it can take months for these
[9:50] projects to be in a good enough state to
[9:51] put in front of customers because that
[9:53] that data science machine learning back
[9:56] back end can be very complex can require
[9:58] a lot of input just to get it to the
[10:00] point where a customer doesn't feel like
[10:01] it goes off the rails immediately with
[10:03] AI. Think a lot of that's changing. One
[10:05] of the other things I wanted to add too
[10:06] is not just about data science teams,
[10:08] but if you look at vendors providing
[10:09] data science solutions, what what's
[10:11] happening is we're with a lot of these
[10:13] LLM tools, we're seeing many traditional
[10:16] honestly, we look at a lot of these
[10:17] vendors, we're just like this is just a
[10:19] facade over the top of OpenAI's API or
[10:21] or anthropic or something like that. And
[10:23] we're like, what are we paying for here?
[10:25] So, a lot of these um in in in in our
[10:28] growth at DICE, a lot of these things
[10:29] have come up and we've started to look
[10:31] at um what you know, why are we paying
[10:34] so much for these things when these LLMs
[10:36] offer almost um similar similar features
[10:41] for the same or less amount of money.
[10:44] So for our solution on this um our data
[10:47] science teams kind of served as and and
[10:49] this may be obvious for some folks but
[10:51] there were a lot of questions when these
[10:52] tools really started to take off about
[10:54] what is the future of a data science
[10:55] team. Um you know I think uh the the the
[10:59] for us we start I we started saying
[11:01] let's start biasing these folks towards
[11:03] being the tip of the spear with AI. They
[11:05] know they're excited about a lot of
[11:07] these models. They know how to implement
[11:08] them. They know how to implement them
[11:10] safely. They can quickly test these
[11:11] things. let's have them serve as the tip
[11:13] of the spear and start taking um some of
[11:15] those challenges on themselves. And um
[11:17] for us at DICE, we ended up learning
[11:19] that uh the customers and and the the
[11:22] internal people would actually come to
[11:24] them and and ask questions and now
[11:25] they're more engaged. It have to manage
[11:28] their time obviously because once people
[11:30] found out they knew what they were
[11:31] talking about, everybody wanted a piece
[11:32] of the action. But, you know, we we
[11:34] found a balance there. But a lot of our
[11:36] internal business folks hadn't been
[11:37] talking to the data science group and
[11:39] now talk to them quite frequently as
[11:41] they're part of certain forums or guilds
[11:42] or even projects together. And we found
[11:45] that the data science team kind of
[11:46] leading the way there um was a great was
[11:49] a great model. Um, additionally, as I
[11:51] said, we we tasked them with having to
[11:54] be uh
[11:57] vet vendors that we thought were were
[12:00] paying quite a bit of money. I'm not
[12:01] going to mention any names because they
[12:02] may be here and they've been great
[12:04] partners to us, but we've had them look
[12:06] at vendors that are providing certain
[12:08] like machine learning type solutions for
[12:09] us and say like could we get almost as
[12:12] good or better performance as we just
[12:14] strapped, you know, a facade on top of
[12:16] uh ChatG, OpenAI or Anthropic ourselves
[12:19] um and and did this. And that's actually
[12:21] a a real use case that's happening right
[12:23] now in our company is we're looking at
[12:25] several of these vendors and saying we
[12:27] don't think we need to pay these costs
[12:28] anymore. So, they're helping with cost
[12:29] reduction.
[12:31] All right, next slide.
[12:34] So, and then finally, AI tools were
[12:36] available but not consistently adopted.
[12:38] So, um tools, you know, once we rolled
[12:41] out these tools to everybody in the in
[12:43] our company, um and talking about here,
[12:46] it's a little bit more specific to
[12:47] engineering and product, these were
[12:49] available to everybody everybody in the
[12:51] in on the team. But we noticed that some
[12:53] people leaned in. They were generating
[12:56] massive amounts. you know the the talk
[12:58] about three or 10x engineers we noticed
[13:00] that with some folks and with other
[13:01] folks we didn't notice as much of
[13:03] increase if any at all and I think um
[13:05] that some people had tried it and they
[13:07] were they felt like it was you know the
[13:09] usage was inconsistent they didn't see
[13:11] necessarily a value they're like this is
[13:12] a cool you know trick a tool that can
[13:15] help me maybe a little bit but um but
[13:18] they dropped it after a little while um
[13:20] and then you know without that adoption
[13:22] I think we weren't getting the value
[13:24] from some of these tools even though
[13:25] some of them are built
[13:27] usage based, which means we weren't
[13:28] paying. But when we looked at
[13:30] implementing AI at DICE, we were like,
[13:32] unless these engineers and some of these
[13:34] product folks pick this up, we're not
[13:35] going to have the impact that we were
[13:37] expecting from this. So, um, what we did
[13:42] was we took a look at what metrics we
[13:44] could get out of there, and I'll show a
[13:45] slide here next in a second that that
[13:47] shows you what we built. Um, but we we
[13:49] we took a look at what metrics
[13:51] available. So, taking a look at who's
[13:52] using what. We're not looking to beat
[13:54] people over the head and saying you're
[13:55] not using it. you know, please ple
[13:57] you're doing your job wrong, you're
[13:59] fired or something like that. We we
[14:00] actually took the took the took the tool
[14:03] and we're looking at for like how, you
[14:05] know, going to these people and saying
[14:06] like, "Hey, why aren't you using it?"
[14:08] And what we found was that a lot of
[14:09] these folks again just didn't see how
[14:11] they were being used. So, we took our
[14:13] engineers that were very excited about
[14:14] them and we put them, we kind of paired
[14:16] them or had them take um uh had them
[14:19] attend an office hours, things like
[14:21] that, and we had them um train these
[14:24] other folks for uh how to use these
[14:26] tools. And I would say this is in
[14:28] progress. This is this is very fresh.
[14:29] This is with even with the last month or
[14:31] two, this is what we've been doing. But
[14:32] we've seen some great results from this
[14:34] where we've shown them like, hey, you
[14:36] know, we're shifting away from having to
[14:37] write every function uh ourselves. were
[14:40] shifting into like the the LLM, you
[14:42] know, in in this case, um, uh,
[14:46] forgot the name, uh, chat GBT or or the
[14:50] the tool that we're using in VS Code
[14:52] will generate it for you and you you can
[14:54] then review the output of that. And so
[14:56] that we've noticed a a pretty good
[14:57] pickup from that.
[14:59] So, and then one of my directors uh,
[15:02] essentially vibe coded this dashboard
[15:03] that sits on top of co-pilot was the
[15:06] name I was looking for. Uh, escaped me
[15:08] for a second there. um we just pull we
[15:10] no no no no work involved he essentially
[15:12] had uh chatgpt generate this whole thing
[15:14] for you and the reason why we pulled out
[15:16] co-pilot has its own but we're going to
[15:18] start um we we also want to use clog
[15:20] code and we can start blending some of
[15:21] those metrics together in here so again
[15:23] just something that we're keeping an eye
[15:24] of like who's using what what what's
[15:26] going on are we getting any value of
[15:28] this this was something that helped us
[15:29] and it was very minimal work again one
[15:30] of our directors vibecoded this next
[15:33] slide a couple things we're looking
[15:34] forward to so spec driven development if
[15:37] you haven't heard of that yet pretty
[15:38] interesting things coming up in that
[15:39] area where it's a little bit more like
[15:41] requirements driven uh development than
[15:43] it is uh that you know writing
[15:45] individual lines of code yourself um or
[15:47] or having it work on a specific
[15:49] function. Um we're starting to see again
[15:51] and we are dice. We are we are experts
[15:54] on technologists. We're starting to see
[15:55] non-engineers who can build. What does
[15:57] that mean for engineers? What role do
[15:59] they play? What role do they play in the
[16:00] organization? Those are starting to be
[16:01] questions that we're that are coming up
[16:02] that we're having to answer. And then
[16:04] finally, if you haven't heard of it,
[16:05] generative engine optimization, which is
[16:08] the LLM partner to SEO. Those are things
[16:10] that we're starting to look in as how
[16:11] those results would show up and how we
[16:13] optimize for that.
[16:15] Uh, if you came here being like, "Dice,
[16:18] I want to hear more about uh candidates
[16:20] and and recruiters and the job market
[16:21] and things like that," you're in luck.
[16:22] Um, my uh my colleague Brooke Loren is
[16:25] doing a a talk here in just an hour or
[16:28] so uh finding the right talent in the AI
[16:30] area. So that's around retaining,
[16:32] training, hiring uh talent in the in in
[16:35] to execute on your AI strategy. Um and
[16:37] so shout out or plug for her at 425 on
[16:41] uh in room 163 on level one. You can
[16:43] find more out about that. And that's all
[16:45] based on a lot of the data that we
[16:46] collect as DICE. Um and I'll be there
[16:48] excited to see that one. So
[16:52] that's it. Thank you.