## Detailed walkthrough
### [00:00](https://www.youtube.com/watch?v=hckVR-Aa-vs&t=0s) The Core Dilemma: When Optimizing Local Metrics Hurts the Whole App
In modern big tech organizations, individual product teams are designed to be relentlessly metric-driven. Every surface is incentivized to hit its own goals, and on paper, every team appears to be succeeding quarter after quarter. Yet this dynamic introduces a quiet and dangerous tension: a scenario where every team hits its targets, but the product as a whole feels worse to the user.
Around 2019, Facebook seemed to be thriving based on top-line aggregate health, but deeper cohort clustering, trend evaluations, and experiment results revealed a concerning pattern among young adults. This core challenge defines the role of the Ecosystem Data Scientist (Ecosystem DS). Rather than simply optimizing existing metrics, their mandate is to step back, question whether those metrics reflect holistic health, and see the forest through the trees.
```mermaid
flowchart TD
accTitle: Local Optimization vs Ecosystem Health
accDescr: Diagram contrasting how individual siloed teams optimize local metrics versus how Ecosystem DS evaluates holistic product impact across the entire platform.
subgraph SiloedTeams["Siloed Product Teams"]
direction TB
F["Feed Team<br/>Focus: Engagement"]
V["Video Team<br/>Focus: Watch Time"]
M["Marketplace Team<br/>Focus: Transactions"]
end
subgraph LocalSuccess["Local Optimization"]
F -->|Hits KPI| S1["+ Feed Metrics"]
V -->|Hits KPI| S2["+ Watch Time"]
M -->|Hits KPI| S3["+ Listing Views"]
end
subgraph EcosystemDS["Ecosystem Data Science Lens"]
S1 & S2 & S3 --> Eval{"Holistic Health Evaluation"}
Eval -->|"Trade-off Check"| Q1["Are friend interactions down?"]
Eval -->|"Demographic Check"| Q2["Are young adults churning?"]
Eval -->|"Regional Check"| Q3["Is cross-app utility healthy?"]
end
```
The host introduces Gene Tabach (formerly Head of Ecosystem Data Science at Facebook, later leading a 60-person DS team across growth and ads at Instacart, and former Head of Data at Grow Therapy) alongside Mujtaba Tirmizi (Head of Marketplace Data Science at Facebook, leading both Product DS and Ecosystem DS teams). Mujtaba highlights an operational challenge with strategic data work: organizations frequently revisit perennial questions every few years—such as regional prioritization and upcoming headwinds—run the same analyses, and see the same patterns, yet fail to act because of organizational inertia. The hardest part of the role is often grappling with the gap between delivering clear data and driving real leadership action.
- “In a big tech-type company, every team is laser-focused on hitting their own KPI. They are always winning, and they are winning by improving the KPI. What if optimizing some user metrics makes the entire product feel worse?” — Host
- “I think the issue that I've run into, usually, is we present the same data, but we don't change our decision. For me, the biggest problem has always been how do I come to terms with that.” — Mujtaba Tirmizi
- The fastest way for data science to demonstrate real impact is sometimes recommending that a team or initiative be eliminated, but such recommendations require influence two or three leadership levels above direct managers rather than at the front-line level.
---
[02:20](https://www.youtube.com/watch?v=hckVR-Aa-vs&t=140s) Former Manager and Skip-Level: The Ecosystem DS Archetype
The conversation shifts into the dynamic between Gene and Mujtaba. Gene was previously Mujtaba’s skip-level manager within the ecosystem data science organization at Facebook, and Mujtaba credits Gene’s vision for making the team compelling enough for him to join. Gene jokes that once Mujtaba arrived, there was nothing left for him to do because Mujtaba took care of everything.
Reflecting on the personality traits required for an ecosystem data scientist, Gene explains that their team deliberately targeted individuals who combined deep analytical rigor with a unique disposition: they looked for people who were “somewhat sassy, low-arrogance individuals.” Because the role requires challenging established product teams, digging into unowned gray zones, and speaking truth to power without institutional ego, this low-arrogance, high-candor profile was essential in building an all-star team.
- “Yes, Gene used to be my skip on ecosystems. He is one of the reasons I came and joined the team—he made it so compelling.” — Mujtaba Tirmizi
- “Yeah, I think more of our team specific, I think we elected to hire a bunch of somewhat sassy, low-arrogance individuals.” — Gene Tabach
---
[03:24](https://www.youtube.com/watch?v=hckVR-Aa-vs&t=204s) Siloed Trees vs. the Forest: What Ecosystem DS Actually Does
The host offers an initial framing: both "data scientist" and "ecosystem" are notoriously abstract concepts. In large tech companies with sprawling surfaces, numerous teams, and complex processes, the job is not to optimize an isolated touchpoint, but to optimize the connective tissue, cross-surface value, and interactions between systems.
Gene agrees and details why this role naturally emerges in massive organizations. In big tech, companies structure themselves into vertical pillars. Facebook users perceive a single unified application, but under the hood, Feed, Groups, Marketplace, Notifications, and Video operate as semi-autonomous companies-within-the-company. Each unit runs at “10,000 miles an hour” toward its specific KPI, with little incentive to look around.
The Ecosystem team steps in to assess whether the sum of these local optimizations actually serves the whole platform. For example, if the video team ships an improvement that doubles watch time, it looks like an unalloyed victory. But if that shift cannibalizes friend-to-friend social interactions by 10%, is it genuinely good for the long-term health of Facebook? The defining characteristic of ecosystem work is not complexity for its own sake, but tracking the systemic connectivity and trade-offs between disparate products.
Mujtaba emphasizes the organizational motto: “we don't ship the org chart,” even though companies are structured by it. Users do not care which pillar built a surface; if individual teams over-optimize their spaces, the user simply feels that “Facebook just feels weird to me.” Beyond balancing internal trade-offs, the ecosystem team acts as a vital safety net for critical strategic questions that fall through the organizational cracks. These include cross-app dynamics—evaluating how Facebook interacts with external competitors like TikTok or sister platforms within the Meta family like WhatsApp and Instagram. Because those horizontal issues involve video, messaging, and identity simultaneously, no single vertical pillar owns them.
- “It's in some ways a company within a company, and each of these teams, especially a company like Facebook, is running 10,000 miles an hour at their objective, and they're not actually taking a lot of time to look around and see the forest for the trees.” — Gene Tabach
- “We talk about this a lot, like we don't ship the org chart, even though internally we're all looking at the org. The user just looks at the overall experience, and they're not going to notice whether or not groups is different or video is different; they're going to be like, 'Facebook just feels weird to me.'” — Mujtaba Tirmizi
- “So that it's also beneficial to have a team like ours that has someone with a horizontal understanding of the whole ecosystem that can tackle some of these questions that kind of like fall through the cracks.” — Mujtaba Tirmizi
---
[07:37](https://www.youtube.com/watch?v=hckVR-Aa-vs&t=457s) The India Problem: Why Vertical Pillars Cannot Solve Holistic Issues
Gene shares a concrete historical example that highlights the structural limitations of functional silos. The ecosystem data team identified concerning trends in India: overall relevance and application health metrics were eroding compared to the rest of the world. At the time, Fidji Simo, who was Head of the Facebook app, recognized the threat and tasked her vertical pillar leads with solving the "India problem."
The response clearly demonstrated why siloed product pillars struggle with horizontal issues:
- When she asked the head of News, the leader returned with a dedicated news strategy.
- When she approached the lead for video and gaming, the response was a gaming-focused proposal on how games could capture the Indian market.
Each leader reflexively viewed the problem through the narrow lens of their own product surface, rather than analyzing the broader systemic dynamic. Because neither strategy answered what the Facebook app needed holistically, this gap directly catalyzed the formation of a dedicated product ecosystem team. Without a cross-cutting data science team sitting horizontally across all verticals, organizations inevitably make disjointed decisions.
- “And then I think she went to the person who's running News at the time, and that person came back with an amazing news solution. She's like, 'No, no, I need a holistic solution.' Then she went to the person who was running, I think like video or games at the time, maybe it was VJ actually, and she was like, 'Hey, can you please help me fix this India problem?' He came back with an amazing game strategy, how games could be number one in India, but that wasn't what she was looking for, and that I think is emblematic of the challenges of this type of thing.” — Gene Tabach
- “This is a very common problem, and like I have tons of examples of when we've made weird decisions because we didn't have like a data team sitting across all these areas.” — Gene Tabach
Mujtaba Tirmizi 指出，在很多组织架构中，Ecosystem 团队与 Growth 团队的关系其实非常紧密，常常是并肩协作的。Growth 部门通常包含两块业务：一块偏向具体的产品抓手（product levers），比如负责 SEO 获客、邮件营销活动（email campaigns）或具体产品功能的落地；另一块则是纯粹从宏观上理解转化漏斗（funnel）、掌握整体产品健康度、进行前瞻预测与轨迹推演（forecast and trajectories）。后者的定位与 Ecosystem DS 理解并追踪产品整体健康的核心职责高度重合。因此在 Facebook 或其他头部应用中，这两个团队从来不会坐得太远。
```mermaid
flowchart TD
accTitle: Relationship between Growth and Ecosystem DS
accDescr: Growth splits into product levers and ecosystem analytics, with company stage shifting focus between profitability and growth.
subgraph GrowthDept["Growth Function"]
PL["Product Levers<br/>(SEO, Acquisition, Email campaigns)"]
Funnel["Funnel & Health Tracking<br/>(Forecasting, Trajectories)"]
end
subgraph EcosystemDS["Ecosystem DS"]
EcoHealth["Holistic Product Health"]
end
Mujtaba Tirmizi 笑谈自己曾经开过的一个玩笑：“At one point in time, I propose we just run a survey of our executive team, a satisfaction survey and the degree to how happy we do a good job, but that's not like that far off.” 他强调，数据科学的核心在于“影响力”（influence），而 Ecosystem 的主要利益相关方就是最高领导层（leadership team）。衡量 Ecosystem DS 影响力的维度主要包括：
1. **解答核心问题的能力**：团队是否切实解答了管理层抛出的关键疑问。
2. **主动拓展认知边界**：团队是否超越了管理层的既有提问，主动向外推进公司对复杂生态系统的理解。
3. **质性评价与反馈**：Mujtaba 坦言，管理普通产品团队主要看业务指标；而管理 Ecosystem 团队时，他极大程度地依赖跨功能团队（XFN）的同行反馈以及高管的直接引语（quotes from leadership），评估领导层在多大程度上感到“自身决策得到了高质量的信息支撑”。
针对主持人抛出的尖锐质询——“如果领导层的观点本来就是错的，或者你讨好了一个持有错误意见的高管怎么办？”——Mujtaba 回应称这种情况在 Facebook 和 Instagram 屡见不鲜。但他认为，优秀的 DS 从来不是一味迎合上级：“A lot of times it's not about alignment or, like, satisfying the questions some leads have. It's about the degree to which we are making sure they're informed on what's exactly going on in the ecosystem and the trends and the health of the app.” 只要 DS 能够基于扎实的数据给出清晰严谨的论证，哪怕最终推翻了高管的原有假设，也能赢得极高评价；加之重大决策往往依赖多元的管理层群体，单一领导者的个人偏见通常会被集体信号所平衡。
Gene 坦言，这正是 Ecosystem 团队无法在过低层级设立的原因：“This is why sometimes I've seen ecosystem teams form at much lower altitudes, and it just doesn't work that well because you don't have as much of that connection to the incentives of that leader to be intellectually honest about their products.”
主持人随后分享了一个 Meta 内部的经典老梗：“The quickest way for DS to make impact is to kill a team.” 要砍掉某个团队，绝不可能在那个团队的经理级别推动，必须跨越两到三层找到激励相容的更高层管理者。主持人还打了一个生动的比喻：如果一家公司像人体一样运转，当大脑发出“减肥”的单边 KPI 时，如果手臂只按自身利益响应，最快达成 KPI 的方式反而是把整条手臂砍掉——单向指标的盲目追逐往往直接戕害生态的整体健康。
[18:19](https://www.youtube.com/watch?v=hckVR-Aa-vs&t=1099s) Top-Down vs. Bottom-Up: Persuading Execution Teams
Gene explains the top-down playbook he found most effective when operating in Ecosystem DS: identify the highest-level decision-maker for the entire surface (such as the head of the Facebook app), persuade them of the core problem, establish shared principles, and then bring that leader and the relevant product lead (such as the head of Facebook Groups) into the room together to get buy-in. While this top-down alignment drives swift mandates, Gene acknowledges that many peers prefer an "across-and-up" bottom-up model. In that approach, ecosystem data scientists collaborate deeply with the data scientists, product managers (PMs), and engineering managers (EMs) inside individual product pillars to understand the day-to-day product realities before escalating. Both methods can be blended, but pushing solutions from the outside creates a severe downstream issue: credit attribution.
```mermaid
flowchart TD
accTitle: Influencing Product Execution
accDescr: Diagram contrasting the top-down mandate route with the bottom-up pre-alignment route used by Ecosystem DS.
Mujtaba emphasizes that relying purely on top-down direction is dangerous because individual product organizations naturally push back. When proposing that the head of the Facebook app instruct the Groups team to take an action, Mujtaba prefers to anticipate that pushback by pre-aligning with ground-level peers first. Talking directly to the data scientists and PMs embedded within those teams provides essential product context that Ecosystem DS naturally lacks from a high altitude. Surfacing their objections beforehand allows the ecosystem analyst to present a balanced view directly to executive leadership: laying out the pros, the cons, and the exact trade-offs already debated, which removes surprises and builds a resilient proposal across organizational boundaries.
- Gene notes that when an external team executes an ecosystem-originated idea, the executing team predictably claims all the wins: “The people who work on the groups team, they're going to say, 'Hey, we did big groups initiative number two. It was amazing. It led to 5 million group connections. Please promote me.'”
- Mujtaba highlights that ecosystem teams routinely lack depth of surface-specific context, making bottom-up conversations essential: “You don't have the depth of context in all these orgs that the product does, so there is value in just going and talking to some of these people to figure out, like, am I missing something?”
- Host note: Backchanneling—or "pre-aligning"—is crucial in large tech firms and startups alike because major meetings are usually rubber-stamp announcements rather than true decision forums.
---
[19:24](https://www.youtube.com/watch?v=hckVR-Aa-vs&t=1164s) "Calling Your Shot": Earning Credit for Strategic Influence
Because the engineers and product teams who write the code and ship the features naturally capture measurable execution credit, Gene developed a deliberate tactic to ensure his foundational influence was recognized during Performance Summary Cycle (PSC) reviews. He called this technique "calling your shot." Weeks before an initiative surfaced publicly, Gene would explicitly alert his manager about what was broken, what product pivot needed to take place, and that he was going to make it happen within a given timeframe. When a director from a product group later posted about launching that exact initiative, Gene's manager had unambiguous proof of who catalyzed the shift.
Gene and Mujtaba reflect that calibrating Ecosystem DS work requires managers to ask a counterfactual question: what would have happened if this individual had not been here for the last six months? In ecosystem projects, the answer is often that the entire strategic shift or headcount reallocation simply would not have occurred. While execution teams deserve significant credit for ideation, engineering, and launching, the initial spark deserves equal recognition. Mujtaba points out that Gene and his peers spent years intentionally cultivating a culture across Facebook App leadership where strategy and foundational analyses were recognized as promotion-worthy impact, rather than defaulting strictly to lines of code shipped or direct feature metrics.
- Gene humorously summarizes his proactive alignment tactic: “I would always tell him, like a couple weeks—I have time, I'd be like, 'Hey, I think this should be happening in the product. I'm going to go make it happen. So, you know, watch this space in three weeks. The thing that I want to happen is going to happen.'”
- On performance calibration philosophy, Gene asserts: “If this person didn't exist—if they weren't here for the last six months, what would have happened? I think on an ecosystem team, very often you will find that if that person didn't exist, that whole strategic pivot wouldn't have happened.”
- Mujtaba highlights that executive buy-in justifies the credit: “We're not saying that the execution and everything is perfect and everything launched because of their analysis, but it's very evident that they wouldn't have put the headcount investment there if we didn't do this analysis beforehand.”
---
[25:42](https://www.youtube.com/watch?v=hckVR-Aa-vs&t=1542s) The Young Adults Analysis: When Aggregates Conceal Reality
The host asks for concrete examples where an Ecosystem DS recommendation directly collided with a VP or Director's key performance indicators (KPIs). Gene clarifies that direct adversarial clashes are rare for Ecosystem DS compared to Integrity teams, whose safety or quality metrics often run directly counter to growth targets. For Ecosystem DS, the ultimate high-level objective is usually shared; the friction lies in uncovering where aggregate team wins are masking systemic vulnerabilities.
Around 2019, Facebook's headline metrics appeared uniformly strong, but the Ecosystem DS team ran deep cohort clustering, trend studies, and experiment breakdowns across demographics. They uncovered an alarming divergence: while the app was flourishing in aggregate, young adults (roughly aged 17 to 29) were flatlining. Major A/B tests that posted positive aggregate results were actually being driven entirely by older user cohorts, while young adults were finding zero relevance and using the platform in fundamentally unhealthy or disengaged ways. The ecosystem team bundled these disparate behavioral patterns into an executive-level analysis, proving to leadership that the platform's overall growth was an illusion propped up by a single aging segment while completely losing the next generation.
- The young adult cohort identified in the study spanned ages 17 to 29 (the post-millennial/older Gen-Z adult range).
- Ecosystem DS used behavioral clustering, longitudinal trends, and demographic-sliced A/B test distributions to expose that core company-wide experiment wins showed zero positive movement among younger demographics.
- Gene emphasizes how headline numbers deceive organizations: “Facebook is doing really well, but it's these people over here are driving all of it. This group here, they're the younger generation, nothing coming, they're not finding relevance here at all.”
> “But I've learned that sometimes it's not that the data changes, but like the leadership circumstances change. Like there's a different level of risk tolerance, there's a different level of external pressure, there's a different level of like competitive pressure that results in us changing decisions, and sometimes it's okay if I'm pulling the same data or refreshing analysis, because there might be external factors changing that, leading to a different decision versus like I'm not just kind of like beating the same drum.” — Mujtaba Tirmizi
> “Basically, your area is your house, right? And like if you saw dirt in the corner of your house you would pick it up. Um, and if you saw a leak in your house you wouldn't clean up the leak. You would be like, 'Why is it leaking?' Um, and if you're, you know, data scientist whose product is, uh, all of Instagram or all of Facebook, then it gets hard because it's a very large house that you're owning. But it is leaks and dirt and small animals or whatever one does with large house, you know.” — Gene Tabach
[36:49](https://www.youtube.com/watch?v=hckVR-Aa-vs&t=2209s) The Skillset Spectrum: Why Ecosystem DS Is Not for Everyone
Working in ecosystem data science demands a distinct capability mix that sets it apart from traditional product DS. While individual problem spaces vary—some requiring exceptionally refined executive presentation skills and the ability to hold spontaneous, high-stakes back-and-forth dialogues with leadership—the broader archetype is defined by where an individual's intrinsic motivation lies. Some talented data scientists simply find that ecosystem analysis does not build the professional muscles they care about.
Mujtaba recounts an illustrative experience with a team member who partnered closely with him on a major, game-changing ecosystem growth analysis:
> “I remember there was like one quarter where that person worked super closely together on analysis and they were doing a great job, I thought. And this was probably one of the top like five that we put together. You mean, changed the way that we thought that growth. And at the end of this, this person talked: ‘I'm going to move to a different team. I really enjoy working with you. I like this result, but this is not the type of work I want to be doing. This isn't like growing the type of muscles that like I want to be growing.’”
That individual transferred to a product team where they could sit in a room side-by-side with an engineer and hammer out ML model improvements full-time, a role in which they excelled. Mujtaba reflects that he personally would never be able to sustain that type of modeling role full-time either. Both Mujtaba and Gene emphasize that identifying a candidate’s genuine appetite for ecosystem-level outcomes matters more than raw aptitude; there are plenty of stellar product data scientists who have the requisite strengths but fundamentally love shipping tangible user features, which an ecosystem team does not do.
[39:49](https://www.youtube.com/watch?v=hckVR-Aa-vs&t=2389s) Tangible vs. Intangible: Defining the Work Along the Spectrum
When asked to anchor the opposing ends of this interest spectrum, Gene frames the distinction as one between the **tangible** and the **intangible**:
```mermaid
flowchart LR
accTitle: Spectrum from Tangible to Intangible Technical Work
accDescr: A continuum showing engineering at the tangible extreme, followed by product DS, zero-to-one product DS, ecosystem DS, and management consulting at the intangible extreme.
SWE --> ProdDS --> ZeroToOne --> EcoDS --> Consulting
```
Engineers sit on the far tangible side, writing code to build direct functionality. Right alongside them are product data scientists who can point to a concrete feature and say:
> “As a product data scientist, you are working on group search and you're like, ‘Hey, I work on group search. Look at this group search. Check it out. Boom, boom, boom. I built that.’ Then you have ecosystem data science, which is much more intangible, right? To the point where it's hard for me to explain to my parents.”
Further toward the abstract end beyond ecosystem data scientists are management consultants. Mujtaba adds that another core dimension is **the level of ambiguity and how well-defined the scope is**. Mature product areas feature well-established metrics and abundant telemetry, whereas zero-to-one spaces carry more ambiguity.
Mujtaba frequently tells his team that these boundaries are fluid: a product data scientist can step back to run a strategic ecosystem project without being a full-time ecosystem specialist. However, at senior levels (such as Meta IC levels L7 and L8), ecosystem thinking becomes nearly inescapable:
> “I feel like a lot of the very senior data scientists, like if you look at like L7, L8, unless you are like a deep technical expert and that's like your one thing, you can't not do some type of ecosystem work.”
[41:43](https://www.youtube.com/watch?v=hckVR-Aa-vs&t=2503s) Seniority, Onboarding Hurdles, and the Critical Role of Managers
Because the work lacks immediate operational guardrails, ecosystem teams typically exhibit a much higher concentration of senior individual contributors (ICs) than product data science teams. Product teams carry a heavy volume of day-to-day operational mechanics required to keep features running, which makes them well-suited for junior data scientists to ramp up and contribute. In contrast, parachuting someone directly into an ecosystem role at a complex company like Meta presents a massive onboarding challenge:
> “Onboarding just to a product team at Facebook is hard, because you're like, ‘I have to learn everything about Marketplace right away.’ But now, if I tell you to learn everything about everything before you have, like, it is very overwhelming. So, I would say, like, the learning curve is much higher.”
For this reason, Mujtaba generally prefers internal transfers who already understand the company's organizational context and data architecture over external hires, giving external transfers a significantly longer runway before expecting major impact.
Gene highlights that the structural linchpin of any ecosystem team is its **engineering/data science manager**, who must effectively operate as both a people leader and a product manager for a group of data scientists:
- **Dual-Hat Leadership**: Junior DS hires (such as L3s and L4s) cannot be expected to navigate organizational politics and uncover ambiguous, multi-team problems on their own. The manager must proactively source questions, gather bottom-up feedback from product groups, absorb strategic priorities from leadership meetings, and construct an actionable roadmap for their ICs.
- **Selective Manager Hiring**: While Gene was happy to hire junior ICs externally—many of whom grew into top performers under proper mentorship—the hiring bar for ecosystem managers was set exceptionally high: “Like I turned down so many very talented people who just weren't at the 99%.”
- **Sustaining Leadership Trust**: Because the ecosystem manager embodies the team's credibility with senior executives, a lack of executive trust at the managerial level renders the entire team ineffective.
[44:28](https://www.youtube.com/watch?v=hckVR-Aa-vs&t=2668s) Why There Are No "Ecosystem PMs"
When asked whether ecosystem teams have counterpart roles like dedicated Ecosystem Product Managers, Ecosystem Software Engineers, or Ecosystem Designers, Mujtaba and Gene explain why the organizational design fundamentally differs from a standard cross-functional squad.
In standard product development, data science acts as a partner function supporting PMs and engineering to launch user-facing features. On an ecosystem team, the dynamic is inverted:
> “The closest thing we have to an ecosystem PM is the leadership team... Just like for most product DS, we are a support function. At the end of the day, you are the support function for the PMs and the engineering team. The ecosystem team, to some level, is a DS support function to the leadership team.”
Executive leadership brings open-ended questions about overall business health and high-level strategy directly to the ecosystem data team. Beyond data science, the only common cross-functional partner is **User Experience Research (UXR)**:
- Many ecosystem explorations delve into spaces with weak quantitative instrumentation or sparse data logging.
- Combining qualitative user research with macro data science fills coverage gaps where telemetry alone fails to explain why users behave the way they do.
Gene agrees, noting that apart from rare organizational experiments, an ecosystem group must inherently be an insights and analytics team—uniting data science, data engineering, and UXR—rather than an engineering- or design-heavy product squad.
数据科学团队在生态系统（Ecosystem）工作中扮演核心角色的一个底层原因在于“中立性”。与背负业务 KPI、功能交付压力或个人设计直觉的产品经理及业务负责人不同，生态团队必须跳出既得利益、部门议程（Agendas）和特定激励，专注在底层基本盘。产品领导者在依赖直觉、个人喜好或主观意见决策时往往自带立场，但在生态层面的博弈中，一旦个人偏见占据主导，分析就会丧失公信力。Gene Tabach 坦言自己在这方面吃过亏：“Like, that team must be unbiased, and I think data is really the only way you can be unbiased.”
```mermaid
flowchart TD
accTitle: Product Lead vs Ecosystem DS Mindset
accDescr: Comparison between subjective product intuition and data-driven ecosystem neutrality.
Mujtaba Tirmizi 则指出，生态数据分析并不需要华丽或极其复杂的动态可视化，一张极简的折线图往往最有说服力，真正的胜负手在底层逻辑：“I think a simple line chart goes really far, but I think what really matters is what's on the y-axis, what's the metric you're looking at.” 团队必须极度严谨地界定 Y 轴指标究竟度量了什么、不能说明什么，以及它如何真实映射用户体验。
对话转入对预测（Forecasting）与“过桥图/未来瀑布图”（Bridges）的探讨。Gene 谈到了对这类分析的矛盾心态：复盘历史归因的后验瀑布图确实很有用，但如果是前瞻性预测未来的所谓“过桥图”，在实际业务中“几乎从来就没准过”（almost always wrong）。Gene 甚至笑称，如果有人逼他把一张充满不确定性的未来趋势折线图进一步拆解成各项确定无疑的细分因子瀑布图，他宁可辞职不干：“Predictions. It requires a different episode, another episode, other things that data scientists are asked to do but actually do not make sense fundamentally; I think prediction is one of them.”
Mujtaba 则给出了不同维度的务实看法。他认为前瞻预测在生态战略中依然具备重要意义，关键在于从业者要清楚这场讨论到底需要多高的精度：“The most important thing is to understand what is the level of precision you need for the conversation you're trying to have.” 很多时候高管并不是要一条置信区间极窄的完美数学曲线，他们核心想了解的是数量级（Magnitude）和机会空间（Ballpark）：
- 这是一个只有 1% 的局部边际改善（Headroom）？
- 还是一个足以彻底扭转大盘增长轨迹的拐点（Angle change）？
## Notable quotes
> [48:25](https://www.youtube.com/watch?v=hckVR-Aa-vs&t=2905s) The most important thing is to understand what is the level of precision you need for the conversation you're trying to have. — Mujtaba Tirmizi
> [46:29](https://www.youtube.com/watch?v=hckVR-Aa-vs&t=2789s) I think a simple line chart goes really far, but I think what really matters is what's on the y-axis, what's the metric you're looking at. — Mujtaba Tirmizi
> [44:28](https://www.youtube.com/watch?v=hckVR-Aa-vs&t=2668s) The ecosystem team, to some level, is a DS support function to the leadership team. — Mujtaba Tirmizi
## Background
该视频由 YouTube 频道“课代表立正”于 2026 年 10 月 23 日发布。视频围绕 Meta 等科技大厂中生态数据科学（Ecosystem Data Science）的定位展开，邀请了曾任职于 Facebook 和 Instacart 的数据高管 Gene Tabach 与 Mujtaba Tirmizi，探讨该角色如何穿透各业务团队的指标烟囱、发现大盘掩盖的全局问题、撬动高管战略决策以及应对绩效归因与职业发展等难题。
## TL;DR
在大型科技公司中，各个垂直业务团队（如 Feed、Groups、Marketplace）往往都在优化自己的局部 KPI，但这种局部最优可能掩盖甚至损害整个产品的长期全局健康。Ecosystem DS 充当高管层的战略数据智囊，通过聚类分析与复用其他团队的 A/B 测试实验，识别出跨业务孤岛的宏观危机（如 Facebook 早期年轻用户群体的流失）。由于不直接拥有具体功能页面，生态数据科学家必须借助自上而下对齐全局激励高管、自下而上提前摸底的混合策略推动执行，并依靠“提前叫牌”和反事实影响来证明战略贡献。这一角色要求极高的自主性、刨根问底的好奇心与大局主人翁意识，其核心价值在于以中立的数据事实对抗组织偏见，用把握数量级的宏观预测为公司战略转向提供指引。
## Atmosphere & style
整场对谈氛围轻松自如、坦诚务实且充满了硅谷大厂资深“老兵”心照不宣的幽默感。两位嘉宾既是行业顶尖专家，又是曾经的上下级伙伴，彼此抛梗接梗极为自然，在谈到“搞死一个团队才是 DS 最大影响力”、“年年拉同样的数据管理层却三年后才听”等痛点时，流露出真实而深刻的职场共鸣。对话没有空泛的理论说教，而是用生动的行业隐喻直击跨部门博弈与大厂考评制度的核心本质，听感既有拆解复杂战略框架的烧脑深度，又带着揭秘内部幕后运作的爽快与亲切。
## Bulletin
- [03:24](https://www.youtube.com/watch?v=hckVR-Aa-vs&t=204s) Ecosystem DS（生态数据科学家）的核心职责是穿透部门深井，防止垂直业务线因过度优化局部 KPI 而损害产品的全局健康度与长期用户体验。
- [14:05](https://www.youtube.com/watch?v=hckVR-Aa-vs&t=845s) 在推动组织战略落地时，Ecosystem DS 应优先对接考核机制与大盘整体健康绑定的高管层，而非利益冲突的基层业务线负责人。
- [19:24](https://www.youtube.com/watch?v=hckVR-Aa-vs&t=1164s) 针对战略分析难以量化归因的难题，DS 可采用提前两至三周向经理预告业务变化的“提前叫牌”（calling your shot）策略，并在绩效校准中依赖反事实检验来证明不可替代性。
- [28:57](https://www.youtube.com/watch?v=hckVR-Aa-vs&t=1737s) 同一份分析可能需要持续更新多年，直到外部竞争加剧或管理层的风险偏好发生质变时，数据才能最终促成实际决策转变。
## Key takeaways
- [00:00](https://www.youtube.com/watch?v=hckVR-Aa-vs&t=0s) 部门烟囱（silos）导致各团队盲目追求局部指标大涨，Ecosystem DS 必须横向审视全局，质疑指标本身并承接跨部门战略盲区。
- [07:37](https://www.youtube.com/watch?v=hckVR-Aa-vs&t=457s) 以 Facebook 印度问题为例，垂直团队往往各执己见试图用自己的业务方案解决问题，促成了超脱于具体业务线的 Product Ecosystem 团队的设立。
- [09:12](https://www.youtube.com/watch?v=hckVR-Aa-vs&t=552s) Ecosystem 团队的职能形态随企业发展阶段灵活调整，例如 Instacart 团队从初期的盈利与单位经济拆解逐步转型为增长与漏斗分析。
- [10:39](https://www.youtube.com/watch?v=hckVR-Aa-vs&t=639s) Ecosystem DS 不直接负责具体产品功能，其核心价值在于对高管决策的战略影响力与对大盘系统性趋势的清晰阐释。
- [14:05](https://www.youtube.com/watch?v=hckVR-Aa-vs&t=845s) 基层业务负责人受制于局部考核指标容易产生防御心理，而高层管理者的激励机制天然与整个 App 的长期健康绑定，更易接受全局优化建议。
- [18:19](https://www.youtube.com/watch?v=hckVR-Aa-vs&t=1099s) 推进方案落地需兼顾自上而下与自下而上的混合模式，在正式汇报前与一线产品人员摸底对齐，化解执行团队的抵触情绪。
- [19:24](https://www.youtube.com/watch?v=hckVR-Aa-vs&t=1164s) 通过“提前叫牌”（calling your shot）策略提前锁定预期变化，结合反事实假设评估，能有效证明战略分析在撬动人力编制与业务转型上的贡献。
- [25:42](https://www.youtube.com/watch?v=hckVR-Aa-vs&t=1542s) 2019 年 Facebook 年轻用户危机表明，大盘整体向好的繁荣数据极易掩盖特定核心群体的黏性断层，需要依靠深度聚类与实验复盘发现真相。
- [27:37](https://www.youtube.com/watch?v=hckVR-Aa-vs&t=1657s) 推动转型的阻力并非目标背道而驰，而是要求业务团队打破舒适区，在未经验证的高难度新方向上重新排布优先级。
- [28:57](https://www.youtube.com/watch?v=hckVR-Aa-vs&t=1737s) 同样的战略分析即使结论稳定也需周期性刷新，因为最终促成决策的触发点往往是管理层风险容忍度或外部竞争格局的改变。
- [30:48](https://www.youtube.com/watch?v=hckVR-Aa-vs&t=1848s) 优秀的 Ecosystem DS 具备极强的自主性、怀疑精神与“大房子”主人翁意识，主动在没有现成需求的情况下规划高价值工作路线图。
- [35:56](https://www.youtube.com/watch?v=hckVR-Aa-vs&t=2156s) Ecosystem DS 团队没有唯一人才画像，既需要攻关底层指标与分析框架的技术专家，也需要擅长高管沟通与战略推演的业务型人才。
- [41:43](https://www.youtube.com/watch?v=hckVR-Aa-vs&t=2503s) 由于上下文门槛极高，生态团队倾向于内部转岗，而团队经理需要承担双重角色，兼任数据科学经理与指导初级 DS 的产品经理。
- [44:28](https://www.youtube.com/watch?v=hckVR-Aa-vs&t=2668s) 公司不需要设立专门的 Ecosystem PM，因为业务高管层本身就是事实上的产品经理，而生态团队必须作为中立客观的数据力量发挥智囊作用。
- [46:29](https://www.youtube.com/watch?v=hckVR-Aa-vs&t=2789s) Ecosystem DS 最倚重的工具是复用其他团队的 A/B 测试，通过定义并监控全局健康度指标，审视各类实验对整个生态系统的净影响。
- [48:25](https://www.youtube.com/watch?v=hckVR-Aa-vs&t=2905s) 战略决策中的业务预测无需过度工程化或盲目依赖复杂机器学习，利用二八法则快速给出数量级判断和大致区间即可满足高管需求。
## Detailed walkthrough
### [00:00](https://www.youtube.com/watch?v=hckVR-Aa-vs&t=0s) 各团队指标飘红，整体产品却在变差？
大型科技公司的运作机制天然带有某种结构性盲点：每个业务团队都被紧紧绑定在各自的 KPI 上，而且每个团队都在不断“赢”——他们的局部实验指标全线飘红、持续上升。然而，一个致命的拷问随之而来：如果单纯优化某些局部的用户指标，反而让整款产品变得更糟糕了呢？
Gene Tabach 回顾了大约在 2019 年的前线观察：当时 Facebook 整体大盘的数据看起来非常出色，但团队通过聚类分析（clustering）、宏观趋势观察以及深度复盘实验数据后发现，整个产品在年轻群体（young adults）中的表现其实已经在亮起红灯。
主持人指出，正是这种“局部最优掩盖全局危机”的核心矛盾，催生了大厂内部最强有力却也最容易被误解的职位——Ecosystem Data Scientist（生态数据科学家）。他们的天职从来不是盲目追逐某项现成指标，而是去质疑指标本身。
```mermaid
flowchart TD
accTitle: Local Optimization vs Global Health
accDescr: Sub-teams optimizing their own silos create conflicting user experiences. Ecosystem DS steps in to measure cross-surface impact.
subgraph Silos["各业务垂类（局部优化）"]
V["Video 团队<br/>追逐：观看时长 ⬆️"]
G["Groups 团队<br/>追逐：群组活跃 ⬆️"]
N["News / Feed 团队<br/>追逐：内容曝光 ⬆️"]
end
User["终端用户体验<br/>'整款产品感觉变糟了'"]
EcoDS["Ecosystem DS 团队<br/>跳出局部指标，衡量整体健康度与连接价值"]
V --> User
G --> User
N --> User
EcoDS -.->|"横向审视 / 发现冲突"| Silos
EcoDS -->|"维护全局体验"| User
```
- 嘉宾 Gene Tabach 曾统领 Facebook 的 Ecosystem Data Science，其后管理 Instacart 增长与广告线规模达 60 人的数据团队，并曾任 Grow Therapy 的数据负责人。
- 嘉宾 Mujtaba Tirmizi 现任 Meta 数据科学负责人，长期领导 Facebook Marketplace 的 Product DS 与 Ecosystem DS 跨职能团队。
- 业务团队周期性地会面临根源性追问：诸如“每隔几年就要重审是否优先倾斜某些区域？我们的投资大方向是否仍然成立？该不该为潜在的宏观逆风买单？”
- Mujtaba 坦言，做这种战略性评估最痛苦的经历之一在于：“I think the issue that I've run into, usually, is we present the same data, but we don't change our decision.”（很多时候我们呈递了同样的数据，但在决策层面却什么都没能改变）。
- 数据科学家最能立竿见影产生巨大业务影响力的动作其实是“砍掉不健康的团队（kill a team）”，但单靠单条业务线管理者根本推不动，必须向上推至两到三个层级的核心高管；而 Ecosystem DS 往往正是提出这种大刀阔斧建议的“吹哨人”。
---
### [02:20](https://www.youtube.com/watch?v=hckVR-Aa-vs&t=140s) 嘉宾重聚与 Ecosystem DS 的特异人设
Gene 和 Mujtaba 此前曾是上下级关系，Mujtaba 坦承 Gene 曾是他的隔级经理（skip manager），也是吸引他当初加入 Facebook Ecosystem 团队的关键原因——Gene 将这个职位的愿景描绘得无比引人入胜。Gene 则打趣道，其实是 Mujtaba 一入职就把所有脏活累活全摆平了，自己反而“无事可做”只能换个坑。
谈及什么样的性格特质才能在 Ecosystem 团队里活下来并做出成果，两人达成了一个高度默契的观察：这个岗位需要特定维度的性格底色。
Gene 指出，在他负责组建团队时，倾向于招募那些兼具尖锐反叛与低调谦逊特质的人：“I think more of our team specific, I think we elected to hire a bunch of somewhat sassy, low-arrogance individuals.”（我们特意挑选了一群带点傲娇叛逆劲儿、但又没有傲慢自负架子的顶尖怪才）。因为面对盘根错节的各个强势业务方，如果一个人唯唯诺诺不敢挑刺，或者反过来趾高气扬试图对别人指手画脚，都不可能把生态层面的系统问题理清楚。
- 团队构筑强调“Sassy yet low-arrogance”（既敢于直言不讳、挑出盲区，又具备共情与谦逊的协作心态）。
- Ecosystem 团队的工作高度依赖非实体化的跨部门人际连接，而非行政指令。
---
### [03:24](https://www.youtube.com/watch?v=hckVR-Aa-vs&t=204s) 穿透部门烟囱：Ecosystem DS 到底在解决什么？
主持人提出，在大厂里，无论是“数据科学家”还是“生态系统”，拆开来看都极度抽象难懂，放在一起更让人费解；大厂充满了繁复的业务表面（surfaces）、流程机制和人际网络，Ecosystem DS 似乎不是去死磕某个单点的优化，而是优化不同节点之间的连接和全局价值。
Gene 赞同这个定位，并指出大厂普遍存在且难以自愈的组织病灶：部门深井（silos）。以 Facebook App 为例：
- 用户打开 App 时，感知到的是一整套流动统一的产品体验。
- 但在代码和组织架构底层，Groups（群组）、Marketplace（商城）、Notifications（通知系统）、Feed（信息流）全都是独立割裂的业务单元（units）。
- 每个团队都拥有排他的考核目标、独特的做事风格和独立的小生态，“如同大公司里寄生的小公司（a company within a company）”。
- 每个团队都在以“时速一万英里”的速度朝着自己的专属 KPI 狂奔，根本没有人停下脚步去看这片森林到底还健不健康。
Gene 给出了一个极为具体的业务冲突案例：如果 Video（视频）团队做了一组极具侵略性的调整，使用户观看视频的时长翻了一倍，单从视频团队的 KPI 来看，这简直是一场辉煌的胜利；“amazing, they're doing a incredible job—but if that's making people have 10% fewer interactions with their friends, is that still a good job? How do you know?”（但如果这导致用户与现实好友之间的互动直接跌了 10%，这还能算是一件好事吗？你究竟要如何衡量与得知？）这就是 Ecosystem 团队必须给出答案的核心课题。复杂的并非数学模型本身，而是指标间的联通与系统性牵引。
Mujtaba 补充强调，业界常说“团队交付的不能是公司的组织架构图（we don't ship the org chart）”，哪怕内部组织架构分得再细，用户感知到的永远只有一个整全的 Facebook。一旦产品变得别扭，用户只会抱怨“Facebook 整体变味了”。此外，Ecosystem DS 还要扮演“接盘者（catch the extra stuff）”：承接一切横跨在现有业务裂缝中间、没有任何单一团队愿意或能够认领的宏观战略课题。
- **跨 App 竞争与分流**：Facebook 怎样受 TikTok 挤压？它与同一家族（Family of Apps）内部的 WhatsApp 或 Instagram 又存在怎样的互动与蚕食？
- 这些横向课题既涉及视频，又混杂私信，还牵涉熟人社交关系网，任何垂直团队（如 Video 或 Messaging）都无法给出全局解答，必须依靠 Ecosystem 团队的横向视野（horizontal understanding）穿透盲区。
---
### [07:37](https://www.youtube.com/watch?v=hckVR-Aa-vs&t=457s) Facebook 的印度难题与“锤子找钉子”的窘境
Gene 分享了一段耐人寻味的真实业务往事。当年 Ecosystem 团队从全局大盘数据中率先预警：Facebook App 在印度的相关性（relevance）与整体健康度指标正在急速恶化，其健康态势明显落后于全球其他地区。
当时的 Facebook 负责人 Fidji Simo 试图寻求破局之策，她找到负责各个垂直业务支柱（pillars）的高管，下达了“给我解决印度问题”的指令。然而现实却演变成了一场各说各话的典型大厂闹剧：
1. Fidji 找到负责新闻（News）线的主管，新闻主管带着一整套精彩纷呈的“新闻拯救印度”方案回来汇报。
2. Fidji 摇头表示这不是她要的整全方案；转头又找到了当时分管视频与游戏（Video & Games）的主管，结果对方反手端出了一套让人挑不出毛病的“Facebook Games 怎样打穿印度登顶第一”的雄伟策略。
3. 每个业务负责人都在拿着自己擅长的“锤子”寻找印度这颗“钉子”，没有任何一个垂直团队能够从整体生态的角度给出真正解法。
这起事件成了极具代表性的样板戏。Gene 回忆道，正是这次教训直接促成了公司在业务侧平行设立专门的 Product Ecosystem 团队。正因为垂直业务团队天生带有利己的目标偏向，企业必须养育一支超脱于具体产品业务线、横跨全局的数据团队，否则大厂就会在局部指标的假象繁荣中不断做出违背全局利益的荒谬决策。
### [09:12](https://www.youtube.com/watch?v=hckVR-Aa-vs&t=552s) 从盈利到增长：Ecosystem 与 Growth 的边界演变
Mujtaba Tirmizi 强调，Ecosystem（生态系统）与 Growth（增长）团队在许多业务场景下关系极其紧密，甚至并肩作战。在增长职能内部，一部分是偏向具体落地杠杆的“产品侧”（Product side），比如负责 SEO 用户获取、邮件召回活动或特定的功能迭代；而另一部分则专注于全局视角——理解用户漏斗、评估整体健康度、制定预测与轨迹推演，这与 Ecosystem 数据科学的核心目标几乎完全重叠。在 Facebook 乃至整个应用矩阵中，这两类职能往往并不割裂，而是高度协同。
Gene Tabach 结合自己的职业履历指出，Ecosystem 团队的具体形态往往取决于公司当时所处的生命周期与核心矛盾。他以加入 Instacart 后的经历为例：最初团队聚焦于“盈利能力”（Profitability），因为那是公司当时面临的生死关头。Ecosystem 数据科学家便深度扎根于 Shopper（配送员）端业务，与财务团队一起逐项拆解损益表（P&L）。然而，一旦盈利问题被彻底解决、不再是主要掣肘时，增长瓶颈便浮出水面。团队的重心随即调整，整体转向贴近增长与营销领域。当时团队中的一些老成员甚至不得不“重塑自我”（reinvent themselves），从精通财务与利润核算的专家，转型为增长与投放策略的行家里手。这证明了生态数据团队的定位从来不是静态的，而是随着业务核心矛盾的转移而灵活变形。
```mermaid
flowchart LR
accTitle: Instacart 生态团队重点转移示意图
accDescr: 展示 Instacart 生态数据科学团队随着业务发展从盈利重点转向增长重点的演变过程。
subgraph S1["阶段一：破解生存瓶颈"]
A["核心挑战：盈利能力 (Profitability)"]
B["合作对象：Shopper 团队 & 财务团队"]
C["分析重点：拆解 P&L 损益与单位经济效益"]
A --> B --> C
end
subgraph S2["阶段二：突破规模瓶颈"]
D["核心挑战：业务增长 (Growth)"]
E["能力重塑：转型为增长与营销专家"]
F["分析重点：漏斗转化、用户留存与大盘健康度"]
D --> E --> F
end
C -->|"盈利问题解决后"| D
```
---
### [10:39](https://www.youtube.com/watch?v=hckVR-Aa-vs&t=639s) 不直接做产品，怎样衡量 Ecosystem DS 的工作成果与影响力？
相较于 Growth 团队可以通过 A/B 测试、提升留存率（Retention）或证明增量（Incrementality）来拿出清晰亮眼的硬指标，Ecosystem 数据科学团队往往不直接拥有某个具体产品页面，其产出衡量一直是一个痛点。主持人由此抛出疑问：怎样去客观评估一个 Ecosystem DS 究竟有没有发挥价值？
Mujtaba Tirmizi 笑着回忆，自己甚至一度半开玩笑地提议“对高管团队做一次满意度问卷调查”。但他指出，数据科学的本质是“影响力”（Influence），对于 Ecosystem 而言尤为如此，因为其最核心的利益相关方就是高管领导层（Leadership team）。衡量该团队的标准主要落在以下几个维度：
1. **关键问题的解答质量**：能否高质量回答高管们迫切关心的系统性问题。
2. **利益相关方认知深度**：高管层与跨职能团队（XFN）是否因此对整个 App 正在发生的变化建立起清晰透彻的理解。
3. **前瞻性与主动性**：能否跨越领导层主动询问的范围，自主向前推演，发现潜在盲点。
Mujtaba 坦言，在管理 Ecosystem 团队时，他极为依赖同行评价（Peer feedback）以及来自领导层的真实引言，用以评估管理层到底在多大程度上“获得了清晰的信息输入”。
针对主持人提出的尖锐质问——“如果领导层的判断本身是错的，而 DS 只是迎合错误意见换取好感怎么办？”Mujtaba 回应称，这种情况在 Facebook 屡见不鲜。但他认为，优秀的反馈并不建立在盲从之上，而在于 DS 是否通过强有力的数据论据（Data-backed argument）清晰地指出了为什么领导层的先验假设站不住脚。他强调：“It's about the degree to which we are making sure they're informed on what's exactly going on in the ecosystem and the trends and the health of the app.” 只要团队保持客观透明，哪怕分析结果推翻了高管原有的直觉判断，依然能够赢得尊重。此外，大公司机制的一大优势在于战略拍板不会取决于单一一颗“脑袋”，广泛采集多位管理者的交叉反馈，能够有效熨平单点偏差。
---
### [14:05](https://www.youtube.com/watch?v=hckVR-Aa-vs&t=845s) 想改变决策，先找到激励机制一致的高管
Gene Tabach 顺着这一话题深入剖析了 Product DS 与 Ecosystem DS 之间最本质的差异——他们所交互的层级不同，而不同层级的利益相关方背负着截然不同的**激励机制（Incentives）**。
Gene 解释道，如果对接的是 News Feed（信息流）Ranking 的资深产品总监（Senior Director PM），该负责人的核心考核指标可能只是他所负责的特定局部指标（例如 Feed 的时长或交互）。如果一名基层 DS 跑过去告诉该总监：“我设计了一个对全 App 用户体验极佳的新策略，但它会砸掉你这半年的考核 KPI”，该总监大概率只会回以拒绝甚至给出一份负面绩效评价，因为其考核机制本质上并非为全局负责。相反，Ecosystem DS 通常直接对接 App 最高层的负责人（如 Facebook App Head 或 C-Level/VP 级别高管），这些掌舵者的激励机制天然与整个 Facebook 的大盘长期健康牢牢绑定。因此，当你向他们阐明“Feed 的某个局部数据会略微受损，但整个 App 的大盘体验会因此显著提升”并拿出扎实证据时，高管在利益上是绝对有动力去倾听并做出调整的。
针对这一利益博弈，现场爆发出幽默的共鸣与讨论：
- 主持人提到 Meta 内部流传的一个经典玩笑：“数据科学家产生影响力最快的方式，就是搞死一个团队（The quickest way for DS to make impact is to kill a team）。”但基层团队经理绝对不可能同意砍掉自己，这种颠覆性决策只能由高出两三层的管理者依托全局激励来推行。
- Mujtaba 笑着回应：“I don't think we killed any teams—you can run me if I'm wrong—but yeah, maybe opportunity, not for not trying. I'm sure we probably wanted to.”
- 主持人进一步用身体代谢作比喻：如果把一家公司比作人体，但各个器官像独立部门一样割裂运转，当你设定了“减肥”的单点 KPI 时，“你的胳膊可能会直接选择把自己砍下来，因为那是减重最快的方式。”在公司管理中，单维盲目追求盈利或单一指标，往往会导致局部团队为了刷亮个人战绩和晋升资本，做出严重噬咬整体生态健康的自残行为。
```mermaid
flowchart TD
accTitle: 组织层级与激励机制错位对比
accDescr: 对比局部业务线总监与高层管理者的激励机制差异，说明为什么生态团队必须在高层寻找对齐点。
subgraph LowAlt["局部业务线（如 Feed Ranking）"]
L1["考核目标：特定业务模块 KPI"]
L2["行为倾向：保护本组指标，抗拒牺牲本组数据的全局优化"]
L3["DS 提议：全 App 体验好，但本组指标下跌"]
L3 -->|"利益冲突"| L2
end
subgraph HighAlt["高层管理者（如 Facebook App Head / VP）"]
H1["考核目标：整个产品的长期大盘健康与总留存"]
H2["行为倾向：具备知识诚实性，愿意用局部妥协换取整体大胜"]
H3["Ecosystem DS 提议：局部受损换取大盘显著增长"]
H3 -->|"利益一致"| H2
end
```
---
### [17:25](https://www.youtube.com/watch?v=hckVR-Aa-vs&t=1045s) 怎样让具体执行团队真正落地你的建议？
当高层管理者被数据说服并点头认可后，真正的执行难点才刚浮出水面：方案必须由底层的业务域负责人（Domain owners）或页面负责人（Surface owners）一行行代码去写、一个个功能去改。如果仅有高层批示而无法调动一线团队的积极性，战略就只会停留在 PPT 上。
对于如何跨越层级推动落地，Gene 透露他在实践中总结出两种行之有效的工作方法，不仅能够促成项目执行，更是解决 Ecosystem DS 个人晋升与战略归因的核心抓手。首要的一种方法被称为**“跨越与穿透”（Across and down）**路径——先在横向跨组织层面找到具有共识的联盟，再由上至下打通阻滞环节……
### [18:19](https://www.youtube.com/watch?v=hckVR-Aa-vs&t=1099s) 自上而下与自下而上：推动业务团队落地的两种路径
在推动业务团队（如 Facebook Groups）采纳建议并付诸行动时，数据科学团队通常存在两种推进路径。Gene Tabach 倾向于“自上而下”的方法：首先在组织内找到层级最高的决策者（例如 Facebook App 的最高负责人），向其论证某项改变势在必行，且该事项会影响到具体的业务团队；随后，让业务线负责人（如 Groups 业务负责人）一同坐到会议室中，基于中立客观的数据事实与已建立的核心原则达成共识，最终促成执行。另一种常见路径则是“横向渗透再向上汇报”（Across and up），即生态数据科学家深入业务团队一线，与对方的产品数据科学家（Product DS）、产品经理（PM）或工程经理（EM）密切协作，彻底摸清该产品的业务机制并达成利益一致。
然而，Gene Tabach 指出，单纯依赖自上而下推动存在一个极其现实的职业发展瓶颈——业绩归属（Credit Attribution）问题。当生态 DS 提出构想，业务团队实际落地并取得亮眼成绩后，业务团队的所有成员都会在绩效评审（Performance Review）中全盘认领功劳。对此，Gene Tabach 尖锐地剖析了这种职场困境：“Like the people who work on the groups team, they're going to say, 'Hey, we did big groups initiative number two. It was amazing. It led to 5 million group connections. Please promote me.' And then I'm going to go, 'Well, it was my idea... promote me as well.' And that gets challenging, right?”
针对这一痛点，Mujtaba Tirmizi 提出应采用“自上而下与自下而上相结合的混合模式”（Hybrid Approach）。纯粹的自上而下往往会引发执行团队的强烈防御心理——直接向高管建言往往会导致被点名的团队本能地反弹。因此，在正式向上汇报前，生态 DS 必须先下沉到一线，同对方团队同级别的 DS 或 PM 进行非正式预沟通（Preview），提前摸清对方可能的反对意见与顾虑。由于生态团队通常缺乏业务团队那样深度的垂直上下文，这种前期摸底不仅能补齐盲区，还能将潜在的反驳意见提前纳入提案，在最终向高管汇报时客观呈现各项利弊，扫清决策阻力。
```mermaid
flowchart TD
accTitle: Top-down and Bottom-up Alignment Paths
accDescr: Comparison of Gene's top-down approach and Mujtaba's hybrid approach for ecosystem data science recommendations.
subgraph TopDown["自上而下路径 (Top-Down)"]
A["高层决策者 (Head of App)"] -->|"确立原则与方向"| B["业务负责人 (Head of Groups)"]
B -->|"下达执行"| C["一线产品团队"]
end
subgraph BottomUp["一线预沟通 (Bottom-Up 摸底)"]
D["生态 DS (Ecosystem DS)"] <-->|"预先摸清顾虑与阻力"| E["业务线 DS / PM"]
E -.->|"补齐业务上下文"| D
end
BottomUp -->|"消除盲区、准备应对预案"| A
```
---
### [19:24](https://www.youtube.com/watch?v=hckVR-Aa-vs&t=1164s) “提前叫牌”与绩效归因：证明战略分析的不可替代性
面对成果被其他团队完全占有的难题，Gene Tabach 分享了他屡试不爽的实战策略——“提前叫牌”（Calling your shot）。在正式推动某个战略事项的两三周前，他会提前找到自己的直属经理打预防针，明确说明：“我认为产品应该发生这样的变化，我接下来要着手去推动。请留心观察，三周之内，我期望的事情就会落地。”三周后，当相关业务线总监在公司内网正式发帖宣布启动该项目时，经理自然心领神会。这种在结果出现前便预判并告知行动路径的方式，有效解决了隐性战略贡献难以被度量的难题。
主持人立正将这种私下沟通与摸底行为总结为大厂与初创公司通用的生存法则——“私下串联与前置对齐”（Backchanneling / Pre-aligning）。大公司的大型评审会议往往只是“宣布决定”的过场，真正的决策早在私下的小范围对齐中尘埃落定。而在 Meta 的绩效评估（PSC, Performance Summary Cycle）中，如何量化这种无形的影响力，考验着数据团队的核心评估逻辑。
在主持管理层校准会议（Calibration）时，Gene Tabach 衡量高阶生态 DS 价值的核心标准只有一条，那就是评估该员工的“不可替代反事实”：“If this person didn't exist—if they weren't here for the last six months, what would have happened?” Gene Tabach 强调，在生态团队中，事实往往是：如果没有这位数据科学家，某项重大的公司级战略转向（Strategic Pivot）根本就不会发生。虽然最终的落地依赖于数十乃至数百名研发人员的推进，但正是生态 DS 点燃了最初的火花（Initial Spark），奠定了战略的大方向。
Mujtaba Tirmizi 补充道，这种战略价值能被认可的前提，是组织内部必须自上而下建立起重视生态分析的文化。如果管理层缺乏对这种宏观战略工作的信任，校准会议上很容易出现“凭什么把整个战略转向的功劳归于某个数据分析师”的质疑。管理层需要认清的关键逻辑在于：尽管工程执行并非由生态 DS 完成，但如果没有他们前期的全局分析，管理层甚至根本不会在这个方向上投入编制与预算（Headcount Investment）。
- **提前叫牌策略**：在项目推动落地前 2-3 周向管理者明确说明预期的产品变动与推动节奏，为后续 PSC 归因打下明确的时间线印记。
- **不可替代性衡量**：管理层校准考核的核心思想是反事实检验——剔除该人员后，战略转向是否还会发生。
- **资源投入归因**：战略分析的最大影响往往体现在撬动组织的人力编制（Headcount）与资源重新分配，而非单纯计较上线了多少行代码。
---
### [25:42](https://www.youtube.com/watch?v=hckVR-Aa-vs&t=1542s) Facebook 年轻用户危机：大盘指标掩盖下的断层
针对生态 DS 是否会经常与业务负责人的个人 KPI 发生剧烈冲突的问题，Gene Tabach 澄清道，生态团队与负责安全合规的 Integrity 团队不同。Integrity 团队的目标往往在客观上抑制某些产品指标，而生态团队的大方向与业务团队高度一致，大家都在追求 Facebook App 的健康与成长，分歧主要在于“到底哪个战略方向对于全局未来更重要”。
为了具象化生态 DS 的工作，Gene Tabach 详细拆解了团队在 2019 年左右主导的“年轻用户（Young Adults）危机”分析。彼时，从公司宏观大盘来看，Facebook App 的整体增长势头极佳，所有核心指标都在向好发展。然而，生态团队通过深度聚类分析（Clustering）、趋势洞察以及对历史 A/B 测试实验数据的全面拆解，发现了一个被总指标严重掩盖的致命问题：在 17 至 29 岁的年轻群体中，用户的使用行为已经与健康活跃的成熟模式脱节。
更令人震惊的发现来自于对大型产品实验的细分复盘。Gene Tabach 指出，当时公司内绝大部分取得显著正向结果的大型 A/B 测试，其增长几乎完全是由中老年等非年轻用户群体拉动的；在被视作产品未来的年轻用户群体中，那些带来全盘大涨的实验表现完全是一片平坦（Flat）。生态 DS 团队将这些碎片化的数据拼接成一套系统的战略论证，直接向管理层指出：看似繁荣的大盘指标背后，年轻一代用户正对 Facebook 失去共鸣与黏性。这种跨越单一业务线孤岛、揭示大盘虚假繁荣的诊断，正是生态数据团队能够撼动公司核心战略走向的典型范例。
Gene Tabach 用最直白的话总结了当时向高管层汇报的核心结论：“Hey, Facebook is doing really well, but it's these people over here are driving all of it. This group here, they're the younger generation, nothing coming, they're not finding relevance here at all.”
### [27:37](https://www.youtube.com/watch?v=hckVR-Aa-vs&t=1657s) 并非直接对立，而是推动团队调整优先级与攻克硬骨头
在探讨推动团队变革时，Gene 澄清了一个普遍的误区：Ecosystem DS 与具体产品团队之间并不是剑拔弩张的死对头。很少会出现某个产品负责人跳出来坚决反对说“这是一个糟糕透顶的主意，我的产品只做50岁以上的老年人”。实际的阻力在于，Ecosystem DS 提出的建议往往要求产品团队转向一个更难执行、且他们过去未能成功攻克的战略方向。因为如果他们原本就能轻易做到，大家也就根本不需要坐在一起开会讨论了。
这种挑战在于能力与预期之间的落差。Gene 举了一个生动的例子：如果你跑去对新闻（News）业务的负责人说：“你今年必须把日活跃用户数（DAU）提高4%，但你必须专门针对年轻用户群体来实现这个目标。”负责人的反应往往是：“我去年确实做到了4%的增长，但我是靠大盘用户冲上去的，在年轻群体里我的增长是零，我现在根本不知道怎么做年轻群体。”因此，真正的摩擦点并非目标背道而驰（比如“你要DAU，而我要利润”），而是要求团队在充满未知的新方向上调整业务优先级并重新寻找打法。
主持人和 Mujtaba 补充指出，这个针对年轻群体的分析至今影响深远。主持人提到，自己当时在 Marketplace 团队工作，如今大家都意识到 Marketplace 承担起了为 Facebook 吸引年轻用户的重任。Mujtaba 则开玩笑回应了一句“不客气”（You're welcome），并表示在绝大多数情况下，数据科学团队与业务线负责人并不会在核心指标的有效性上正面对撞，关键永远在于“优先级的重新排布”（Prioritization）。
### [28:57](https://www.youtube.com/watch?v=hckVR-Aa-vs&t=1737s) 同一份分析做三年：为什么领导层在第三年才改变决策？
Mujtaba 深入分享了 Ecosystem DS 经常面临的一大现实困惑：回答周期性的“长青问题”（Perennial questions）。例如，团队经常要讨论“区域优先级”，比如为何必须将资源重度倾斜在美加市场（US/Canada）——因为那里承载着巨大的商业化收入、文化影响力以及新趋势的早期采纳。每隔几年，Ecosystem 团队都会被要求重新审视相同的问题：我们在全球某些快速增长的区域表现优异，但在竞争激烈的核心人群中表现疲软，我们现有的投资组合是否合理？我们是否应该将重心转向那些正在逼近的逆风风险？
最让数据科学家感到抓狂和自我怀疑的，莫过于“呈现的数据年年相似，但管理层的决策却纹丝不动”。Mujtaba 坦言，自己曾连续三年针对同一个问题拉取几乎一模一样的数据、做同样的刷新分析，一度甚至觉得自己是不是疯了。然而到了第三年，管理层突然决定采取行动。他由此领悟到：**决定发生转变，往往不是因为底层数据发生了剧变，而是领导层所处的外部环境改变了。**
```mermaid
flowchart TD
accTitle: 战略决策触发机制
accDescr: 展示Ecosystem DS分析在不同外部变量变化下最终促成管理层决策的过程
subgraph Input["持续输入（三年保持稳定）"]
A["Ecosystem DS 周期性分析与数据"]
end
subgraph Triggers["外部环境变化（第三年发生质变）"]
B["领导层风险容忍度 (Risk Tolerance) 改变"]
C["外部与宏观市场压力加剧"]
D["竞对威胁 (Competitive Pressure) 迫近"]
end
subgraph Output["业务决策落地"]
E["管理层正式调配资源并采取战略行动"]
end
A --> E
B --> E
C --> E
D --> E
```
在总结这种心路历程时，Mujtaba 深刻地反思道：
> “I've learned that sometimes it's not that the data changes, but like the leadership circumstances change. Like there's a different level of risk tolerance, there's a different level of external pressure, there's a different level of like competitive pressure that results in us changing decisions, and sometimes it's okay if I'm pulling the same data or refreshing analysis, because there might be external factors changing that, leading to a different decision versus like I'm not just kind of like beating the same drum.”
> （我明白了一个道理：有时候并不是数据变了，而是领导层所处的环境变了。可能是风险容忍度不同了，可能是外部压力不同了，也可能是竞争压力的加剧导致了决策的转变。有时候哪怕我只是在拉取相同的数据或刷新分析也是有意义的，因为外部因素的变化会导向不同的决策，我并不是在徒劳地敲同一面鼓。）
- **长青问题的本质**：公司会周期性地重新审视区域分配与人群痛点，需要 DS 在没有新结论时耐心地提供稳定的事实基础。
- **推动改变的时机**：战略转向需要数据事实与外部催化剂（竞争、收入压力、管理层更替）的交汇。
### [30:48](https://www.youtube.com/watch?v=hckVR-Aa-vs&t=1848s) 优秀 Ecosystem DS 的特质：自主性、刨根问底与“大房子”主人翁意识
主持人随后梳理了做好 Ecosystem DS 所需的特质：面对高度不确定性的主动性（Proactive / Initiative）、面对众多跨产品领域的复杂度处理能力（Dealing with complexity）、以及在没有职权的情况下去施加影响并建立杠杆关系（Influence without authority）。
Mujtaba 非常赞同这些特质，并将其形象地比喻为“空降雇佣兵”（Mercenaries）：Ecosystem DS 必须具备极强的自主性（Autonomy）和足智多谋（Resourceful）的解决能力，经常需要空降到一个完全陌生的领域（比如突然跳进视频业务分析其数据），迅速摸清门道并发挥影响力。除了这些，Mujtaba 强调 Ecosystem DS 必须具备“刨根问底”（Inquisitive）的特质。由于 Ecosystem 团队没有专属的产品经理每天塞给他们路线图和需求，DS 必须自己去提出问题、质疑产品战略，并规划属于自己的工作路线图。
这种刨根问底往往表现为一种“健康的怀疑精神”。Mujtaba 调侃道，Ecosystem DS 每天盯着整个 App 的宏观走势看，通常是整个组织里对数据最持怀疑态度的人（Most skeptical）——他们会不断反思：业务真的有表面数据看起来那么好吗？或者真的有那么糟吗？
```mermaid
flowchart LR
accTitle: 普通产品团队与Ecosystem团队的工作驱动对比
accDescr: 展示产品DS被动接收需求与Ecosystem DS主动自驱探索的差异
subgraph ProductDS["具体产品线 DS (Product-Specific DS)"]
PM["产品经理 / 跨职能团队 (XFN)"] -->|"提供明确问题与需求"| P_Roadmap["产品路线图优先级排序"]
end
subgraph EcoDS["生态数据科学 (Ecosystem DS)"]
Inq["刨根问底的直觉与好奇心"] -->|"自主质疑业务假设与宏观大盘"| E_Roadmap["自我定义并规划高价值路线图"]
end
```
Gene 则将核心特质归结为求知欲（Intellectually curious）与极高要求的主人翁意识（Ownership）。如果一个 DS 负责极小的产品功能，因为拥有直接控制权，保持主人翁意识相对容易；但面对 Facebook 或 Instagram 这样宏大且缺乏直接控制力的业务大盘，维护 Ownership 极其艰难却至关重要。Gene 提出了著名的“大房子隐喻”：
> “Basically, your area is your house, right? And like if you saw dirt in the corner of your house you would pick it up. Um, and if you saw a leak in your house you wouldn't clean up the leak. You would be like, 'Why is it leaking?' Um, and if you're, you know, data scientist whose product is, uh, all of Instagram or all of Facebook, then it gets hard because it's a very large house that you're owning.”
> （基本上，你负责的领域就是你的房子，对吧？如果你看到房子的角落有脏东西，你会把它捡起来；如果你看到屋里漏水了，你不会只是把水擦干净，你会问：‘为什么会漏水？’如果你作为一名数据科学家，你的产品是整个 Instagram 或整个 Facebook，这就会变得很难，因为你拥有一栋非常大的房子。）
Gene 回忆起自己初入团队做初级数据科学家（Junior DS）时的经历：他当时着魔般地死盯每一个监控看板，只要看到任何异常波动，就会立刻深挖原因并在跨团队的工作群组里发帖追问“这里或那里到底出了什么状况”。虽然当时可能让一些同事觉得有点烦人，但 Gene 坚信，那些最终在生态分析中取得巨大成功的人，骨子里都带着这种“令人讨厌的高主人翁意识”（Annoying in that high ownership kind of way）。这种探究不仅排除了隐患，更帮助 DS 在脑海中拼凑出整个生态系统各模块如何联动的真实心智模型（Mental model）。
### [35:56](https://www.youtube.com/watch?v=hckVR-Aa-vs&t=2156s) 团队画像与评估：技术大牛与多元化人才结构
针对主持人提出的“如何衡量这种工作的质量以及提拔什么样的 DS”，Mujtaba 指出，**Ecosystem DS 并不存在唯一的标准画像（Single standard version）**。生态团队所面临的问题谱系极其宽泛，因此团队内部需要多元化的技能专长搭配。
例如在 Facebook 内部，团队会设立类似“技术专家/技术大师”（Tech gurus）的角色类型。因为 Ecosystem DS 面临着一整套非常严苛的技术难题，比如全公司级别分析框架的标准化、全局性数据指标工具链的开发与统一，这些高难度底层工程确实需要技术极其过硬的极客来攻克。但 Mujtaba 也明确指出，一个团队绝不可能全部由十个纯技术专家组成，生态团队既需要能够深潜底层技术的大牛，也需要具备敏锐战略直觉与卓越沟通能力的业务型人才，两者的结合才能应对复杂的生态系统难题。
### [36:49](https://www.youtube.com/watch?v=hckVR-Aa-vs&t=2209s) 生态数据科学家的技能谱系与人员匹配
Ecosystem 数据科学团队内部面临的问题截然不同，因此并不存在单一的技能模型。某些业务问题高度依赖极强的高管沟通（executive communication）能力，要求 DS 在直面高层汇报时具备极高的表达打磨度（polish），能够在高管的频繁追问与多轮互动中清晰表述，这种要求与普通的 Product DS 岗位完全不同。反过来，许多技术过硬的 DS 并不适合生态团队。Gene Tabach 分享了一个真实案例：两年前他团队里的一位成员做出了极其出色的成果，完成了一份足以改变整个团队对增长认知的深度分析，位列团队排名前五；但项目结束后，该成员却主动提出转岗，直言这并不是自己想锻炼的技能，随后转去了一个只需与工程师坐在一起、专注于底层模型迭代优化的团队，并在那里大放异彩。
在招聘时，这种个人意愿与热情的匹配度甚至超过了单纯的能力考核。许多分布在公司各处的优秀 DS 虽然具备做生态的所有潜质，但他们本质上热爱产品，享受亲手把功能上线发布（ship stuff）的成就感，而生态团队恰恰无法提供这种即时产出的反馈。Mujtaba Tirmizi 强调，如果生态数据科学家所能带来的宏观成果无法让你感到兴奋，那么你大概率不适合这个岗位。
```mermaid
flowchart LR
accTitle: 数据与工程岗位的具象与抽象光谱
accDescr: 展示从底层工程师到管理顾问在工作产出形态上的有形与无形分布。
subgraph Tangible["具象端 (Tangible)"]
ENG["软件工程师<br/>(编写代码 / 实体系统)"]
PDS["产品数据科学家<br/>(上线具体功能，如 Group Search)"]
end
subgraph Mid["过渡带 (Ambiguity)"]
ZTO["0 到 1 新业务探索 DS<br/>(大赌注 / 范围模糊)"]
end
subgraph Intangible["抽象端 (Intangible)"]
EDS["生态数据科学家<br/>(全局度量 / 跨业务协同)"]
MC["管理顾问<br/>(纯战略 / 外部建议)"]
end
ENG --> PDS --> ZTO --> EDS --> MC
```
- **职能兴趣光谱的两极**：Mujtaba Tirmizi 用“具象”（tangible）与“抽象”（intangible）来定义这个连续光谱。工程师写代码造物处于极具象端；Product DS 负责具体功能（如夸赞“看，我做出了社群搜索功能”）紧随其后；而 Ecosystem DS 则偏向极度抽象，甚至抽象到“很难向父母解释自己到底在干什么”的程度，更外侧则是纯粹的管理顾问（management consultants）。
- **资深人员的生态属性**：Gene Tabach 补充道，光谱的本质还在于“模糊度”（level of ambiguity）与范围的清晰度。在 Meta，当 DS 晋升到 L7、L8 级别的高阶阶段时，“除非你是走纯深度技术路线的特例专家，否则你不可能不做某种形式的生态工作”。
> “I really enjoy working with you. I like this result, but this is not the type of work I want to be doing. This isn't like growing the type of muscles that like I want to be growing.” —— 前团队成员向 Gene Tabach 坦言转岗原因
---
### [41:43](https://www.youtube.com/watch?v=hckVR-Aa-vs&t=2503s) 资历密度、内部转岗与经理的“双重角色”
生态团队的高级个人贡献者（Senior IC）密度天然高于产品团队。产品团队日常有大量维持业务运转的操作性工作（operational work），能够消化吸收较多初级数据科学家；而生态团队如果直接从零招募初入职场的新人，新人往往很难直接撬动高层决策或证明自身影响力。因此，Mujtaba Tirmizi 明确表示在组建生态团队时，内部转岗（internal transfers）通常远优于直接社招。在 Facebook 这样产品架构庞大复杂的公司，单是弄懂某一个业务线（如 Marketplace）的业务上下文就已极其困难；如果让一个完全没有公司背景的新人一进来就掌握跨所有业务的全部全局逻辑，其认知负荷会令人不堪重负。因此，生态团队必须为新成员提供更宽容的适应期（longer runway）。
在这种高难度与高模糊度的背景下，生态团队经理（Ecosystem Manager）成为了维系团队生存的最关键角色。Gene Tabach 指出，生态团队的经理实际上在承担“双重职责”（double duty）：他们不仅是数据科学经理，更扮演着整个数据科学家团队的“产品经理”（Product Manager）。当代号 L3、L4 的初中级 DS 缺乏自下而上独立挖掘题目的视野时，生态经理必须主动从跨团队的基层交流或高管决策会议中捕捉信号，将零散的想法转化为严密的路线图，并拆解为适合初级 DS 执行的具体项目。
- **经理招聘的极高门槛**：Gene Tabach 提到，虽然他很乐意在执行层招募外部初级人才，并在长期将他们培养成顶梁柱，但对于生态团队经理的社招门槛要求极为严苛，“拒绝了无数非常优秀的人才，因为他们没有达到前 1%（99th percentile）的水准”。
- **高管信任的纽带**：生态团队缺乏落地实物的硬性交付物，其生死完全取决于领导层对该团队的信任度；经理正是这种信任与支持（buy-in）的代言人。一旦经理无法在高层建立威信，整个团队的价值就会大打折扣。
> “I think basically an ecosystem leader is doing double duty, right? They are both data science managers, but they are also I think product managers of a team of data scientists.” —— Gene Tabach
---
### [44:28](https://www.youtube.com/watch?v=hckVR-Aa-vs&t=2668s) 为什么只有 Ecosystem DS，而没有 Ecosystem PM？
面对主持人提出“公司里是否存在生态产品经理（Ecosystem PM）、生态工程师或生态设计师”的疑问，两位嘉宾给出了否定的回答。Mujtaba Tirmizi 解释，普通的 Product DS 本质上是具体 PM 和工程团队的支撑职能（support function）；而 Ecosystem DS 服务的不是基层产品组，其直接的对接方和支撑对象就是公司的高管领导层（leadership team）。可以说，公司的业务领导层自身就是那个事实上的“Ecosystem PM”，高管在战略视野中产生的全局数据疑问，都会指派给生态团队进行解答与推演。
Gene Tabach 进一步明确，生态团队从形态上几乎必须是一支“数据洞察型团队”（data team）。这并不局限于数据科学（Data Science），还可以吸纳数据工程（Data Engineering）、用户研究（UXR, User Research）等职能。特别是在很多缺乏日志数据覆盖的新兴领域或盲区中，单纯依靠埋点数据无法描绘全貌，必须引入 UXR 等定性洞察来补充研究支撑。除了极少数极为特殊的业务阶段外，企业通常不需要设置专职的生态 PM 或生态工程师，数据团队本身就是探索全局战略最核心、最自然的承载者。
- **服务对象的本质倒置**：普通 DS 服务于项目团队；生态 DS 则是高管层专属的“战略数据智囊”。
- **跨职能协作的边界**：生态团队很少配备专属的研发工程师或设计师，但对定性研究（UXR/Insights）有极高的依赖度，以此弥补海量数据无法覆盖的盲区。
> “The closest thing we have to an ecosystem PM is the leadership team... The ecosystem team, to some level, is a DS support function to the leadership team.” —— Mujtaba Tirmizi
> “I think an ecosystem team has to be a data team, and I don't mean just science, engineering, UXR, insights, whatever you want to call that group of people.” —— Gene Tabach
### [04:45:48](https://www.youtube.com/watch?v=hckVR-Aa-vs&t=17148s) 为什么是数据团队：以中立性对抗主观偏见
谈到为什么生态系统的战略审视工作必须由数据科学（DS）团队主导，而非由产品经理（PM）牵头时，Gene Tabach 指出，这背后的核心在于团队的中立性以及专注于底层基本面的能力。在大型组织中，各个业务部门往往被各自的指标考核、利益诉求或特定路线图所牵引，而审视整个生态的团队必须绝对客观。Gene 强调：“Like, that team must be unbiased, and I think data is really the only way you can be unbiased.”（这支团队必须保持不偏不倚，而我认为数据是让你保持无偏见的唯一方式）。
如果由产品负责人凭借设计直觉或主观偏好来做判断，难免会带入不同程度的偏见。在生态系统的全局工作中，观点固然有一席之地，但在陈述事实和做战略决策时，必须让客观数据处于主导地位，个人偏好则要退居其后。Gene 自嘲自己在这方面踩过几次坑，深有体会：对于非洞察（insight）类的业务职能部门而言，要完全剥离主观意图、让位于纯粹的数据驱动，往往极其困难。
### [04:46:29](https://www.youtube.com/watch?v=hckVR-Aa-vs&t=17189s) 选对指标：借用其他团队的 A/B 测试评估生态健康
当主持人立正询问生态数据科学家（Ecosystem DS）最倚重的分析工具是什么时，Gene 打趣说他们最有效的工具其实是“其他团队的 A/B 测试”。在像 Facebook 这样结构庞大复杂的应用中，衡量用户到底认为产品有多大价值是一件极难做到且必须保持客观的事情。相比于 Instacart 可以直截了当地使用交易额（GMV）或利润来衡量，社交网络没有单一的交易行为，因此 Facebook 的生态数据团队耗费了大量心血去构建能够真实反映整个应用健康度的核心指标。一旦定义好这套生态指标，DS 团队便可以直接复用各业务团队运行的成百上千个实验，站在全局视角审视这些测试对生态关键指标的综合影响。
```mermaid
flowchart LR
accTitle: Ecosystem DS 借力实验分析流程
accDescr: 展示生态DS如何通过定义全局健康指标，复用业务团队的A/B测试数据进行跨产品归因分析。
subgraph Design["指标定义与基准"]
A["定义底层生态指标<br/>(App Health Metrics)"]
end
subgraph Experiments["业务线实验池"]
B["Feed 团队测试"]
C["Marketplace 测试"]
D["Groups 团队测试"]
end
subgraph Analysis["全局生态评估"]
E["跨团队实验数据复用"]
F["评估整体产品健康<br/>与长期生态影响"]
end
A --> E
B --> E
C --> E
D --> E
E --> F
```
Mujtaba Tirmizi 补充道，在可视化呈现上，生态 DS 不需要炫技，最简单的折线图往往最能说明问题。真正的核心难点不在图表形式，而在于纵坐标（Y 轴）到底放了什么指标——团队必须精准厘清自己要衡量什么，并对该指标“能说明什么、不能说明什么”有清醒的认知。在实际向管理层汇报时，Mujtaba 个人非常青睐瀑布图（Waterfall Chart）：
- 业务指标上升或下降时，瀑布图能够清晰拆解出各项正面和负面的驱动因素。
- 虽然 Gene 对瀑布图抱有“爱恨交加”的复杂情绪，但两人都认同它在引导高管讨论、理清原因归因时极为有效。
### [04:48:25](https://www.youtube.com/watch?v=hckVR-Aa-vs&t=17305s) 预测需要多精确？80/20 法则与商业决策的现实权衡
关于预测未来，现场爆发了一场充满共鸣又带点自嘲的行业争论。Gene 提到，如果把瀑布图用于前瞻性预测（即所谓的“桥接图”Bridge），几乎每次都会被现实打脸：“At Instagram, seeing a backward-looking waterfall that explains something—sure. We often have forward-looking waterfalls that project what will happen, and we call these bridges, and they are almost always wrong.” 他甚至直言，如果有人要求他把一条预测折线图拆成各驱动因子的前瞻瀑布图并给出绝对自信的预测，他宁可“辞职不干”。在数据科学界，常常存在让数据科学家做各种复杂预测、但在商业底层逻辑上根本立不住脚的怪现象。
Mujtaba 则从高管决策的实际场景出发提出了不同视角。他认为预测在生态数据工作中依然非常重要，但关键是理解业务对话到底需要多高的精度：
- 领导层并不需要极度收敛的误差范围（tight error bars）或精确无误的置信区间。
- 决策者真正想知道的是影响的数量级和大致范围（magnitude and ballpark）——这究竟是一个仅有 1% 的局部边际空间，还是足以彻底改变整体大盘走向的增长拐点。
- 生态 DS 必须建立商业直觉，把握“二八法则”（80/20 rule），用合理的精力快速给出数量级判断，避免在不必要的细节上过度工程化。
Gene 对此表示认同，指出如果是做规模估算（Sizing），用简单的 Excel 梳理假设变量、评估逻辑传导关系是非常健康且合理的做法。然而，若非要套上复杂的机器学习模型来粉饰预测的“科学性”，不仅无法提供增量价值，而且“你表现得越自信，最后往往被现实打脸打得越惨”（the more confidence you are, the more burned you are going to be）。
## Notable quotes
> [41:43](https://www.youtube.com/watch?v=hckVR-Aa-vs&t=2503s) I think basically an ecosystem leader is doing double duty, right? They are both data science managers, but they are also I think product managers of a team of data scientists. — Gene Tabach
> [19:24](https://www.youtube.com/watch?v=hckVR-Aa-vs&t=1164s) If this person didn't exist—if they weren't here for the last six months, what would have happened? — Gene Tabach
> [44:28](https://www.youtube.com/watch?v=hckVR-Aa-vs&t=2668s) The closest thing we have to an ecosystem PM is the leadership team... The ecosystem team, to some level, is a DS support function to the leadership team. — Mujtaba Tirmizi
> [04:35:48](https://www.youtube.com/watch?v=hckVR-Aa-vs&t=16548s) Like, that team must be unbiased, and I think data is really the only way you can be unbiased. — Gene Tabach
## Mentioned
Meta, Facebook, Instagram, WhatsApp, TikTok, Instacart, OpenAI, Grow Therapy, Gene Tabach, Mujtaba Tirmizi, Fidji Simo, 课代表立正, Facebook Marketplace, Facebook Groups, News Feed, Excel
## Video description
各团队的KPI都在涨，产品就一定更好吗？在Facebook，Feed、Groups、Marketplace各有目标，但用户面对的是同一个产品。Ecosystem Data Science要回答的，就是单个团队的“成功”，到底有没有变成整个产品的进步。
这期我和Gene Tabach、Mujtaba Tirmizi，从Facebook和Instacart的实际工作出发，拆解Ecosystem DS怎样发现总指标掩盖的问题、影响公司战略，并推动其他团队采取行动。Gene分享了Facebook如何识别年轻用户的使用问题，以及自己怎样在推动变化之前先向经理讲清预期结果，让难以归因的战略贡献被看见。Mujtaba则讲到：同样的区域分析可能连续做三年，直到管理层的风险容忍度与外部压力改变，才真正推动决策。
我们也谈了这个角色最实际的难题：说服高管之后，怎样争取执行团队的支持？成果最终由其他团队上线，提出方向的DS怎样证明贡献、获得晋升？什么样的人适合Ecosystem DS，为什么它的经理还需要像产品经理一样，为团队寻找问题、建立路线图？最后回到工具与方法：怎样定义产品健康度，如何利用其他团队的A/B测试，以及预测究竟需要多精确，才足以支持决策。
嘉宾背景与近况（更新于2026年10月22日）：
Gene Tabach：曾负责Facebook的Ecosystem Data Science，后任Instacart增长与广告数据科学高级总监、Grow Therapy数据副总裁。目前已加入OpenAI的B2B团队，并从管理岗回到个人贡献者（IC）岗位，参与帮助企业采用AI的工作。
Mujtaba Tirmizi：Meta数据科学负责人，访谈时负责Facebook Marketplace的Product DS与Ecosystem DS团队。目前仍在Meta从事数据科学领导工作，2026年曾公开分享其团队参与推出Marketplace的Seller应用及AI功能。
访谈录制于2025年8月26日（美西时间），为英文对谈，配有中英字幕。嘉宾近况已按2026年10月22日的公开信息更新；视频中的职位介绍为较早版本。
嘉宾主页：
Gene：https://www.linkedin.com/in/genetabach/
Mujtaba：https://www.linkedin.com/in/mujtabatirmizi/
章节：
00:00 各团队的KPI都在涨，产品就一定更好？
02:20 嘉宾登场：曾经的上下级，再谈Ecosystem DS
03:24 Ecosystem DS到底做什么？
07:37 Facebook印度问题：为什么单个团队给不出完整答案
09:02 从盈利到增长：Ecosystem与Growth的关系
10:39 不直接做产品，怎样衡量DS的影响？
14:05 想改变决策，先找到激励一致的管理层
17:25 怎样让其他团队真正执行你的建议？
19:24 Calling your shot：让难以归因的贡献被看见
22:35 没有这位DS，这次战略转向还会发生吗？
25:42 Facebook年轻用户：总指标掩盖了什么？
28:57 同一份分析做三年，为什么第三年才改变决策？
30:48 好奇心、独立性与跨团队的主人翁意识
35:56 技术、沟通与兴趣：Ecosystem DS没有唯一画像
41:43 新人与资深DS，怎样进入Ecosystem团队？
44:28 为什么是数据团队，而不是Ecosystem PM？
46:30 选对指标，利用其他团队的A/B测试
48:22 预测需要多精确，才能帮助决策？
[Source](https://www.youtube.com/watch?v=hckVR-Aa-vs)
Source quotes
[41:43] Gene Tabach: I think basically an ecosystem leader is doing double duty, right? They are both data science managers, but they are also I think product managers of a team of data scientists.
[19:24] Gene Tabach: If this person didn't exist—if they weren't here for the last six months, what would have happened?
[44:28] Mujtaba Tirmizi: The closest thing we have to an ecosystem PM is the leadership team... The ecosystem team, to some level, is a DS support function to the leadership team.
[275:48] Gene Tabach: Like, that team must be unbiased, and I think data is really the only way you can be unbiased.
Source quotes
[48:25] Mujtaba Tirmizi: The most important thing is to understand what is the level of precision you need for the conversation you're trying to have.
[46:29] Mujtaba Tirmizi: I think a simple line chart goes really far, but I think what really matters is what's on the y-axis, what's the metric you're looking at.
[44:28] Mujtaba Tirmizi: The ecosystem team, to some level, is a DS support function to the leadership team.
Original transcript
[0:00] In a big tech-type company, every team
[0:02] is laser-focused on hitting their own
[0:04] KPI. They are always winning, and they
[0:06] are winning by improving the KPI. What
[0:08] if optimizing some user metrics makes
[0:10] the entire product feel worse?
[0:12] >> I want to say like 2019 or something.
[0:14] The Facebook app was doing really well
[0:16] overall, but one thing we found was it
[0:18] wasn't doing quite as well for young
[0:20] adults. Like, we did clustering, we
[0:22] looked at trends, we looked at
[0:23] experiment results and found like...
[0:25] >> This is the central problem that leads
[0:27] to one of the most powerful yet
[0:29] misunderstood roles: the ecosystem data
[0:31] scientist. Their job isn't to chase
[0:33] metrics but to question them. To impact
[0:35] this role, I have two brilliant guests:
[0:38] Jin, who was the head of ecosystem data
[0:40] science at Facebook, then he led a 60-
[0:42] person data scientist workforce at
[0:44] Instacart's growth and ads, and now he
[0:46] is the head of data at Growth Therapy;
[0:48] and Mushtaba, the head of marketplace
[0:51] data science at Facebook. He led both
[0:53] the product team and the ecosystem DS.
[0:55] We have perennial questions that we
[0:57] address every few years, and we had
[0:59] this question on, like, "Should we be
[1:01] prioritizing certain regions over
[1:03] others?" And every few years we kind of
[1:05] answer the same question, we look at
[1:07] the same trend, we look at the same
[1:09] prioritization of, like, "Are we still
[1:11] okay with where we're investing? Or
[1:13] should we be prioritizing some of these
[1:16] up-and-coming headwinds?" I think the
[1:18] issue that I've run into, usually, is
[1:20] we present the same data, but we don't
[1:22] change our decision. For me, the
[1:24] biggest problem has always been how do
[1:26] I come to terms with that. Of and, used
[1:29] to work, they are going to review how
[1:31] they see the forest from the trees. The
[1:33] quickest way for DS to make impact is
[1:35] to kill a team, but you cannot kill the
[1:37] team at the team manager level. You
[1:39] have to go like two, three levels above
[1:42] . To fence writing center, probably
[1:44] ecosystem, DS would have to make a lot
[1:46] of such recommendations to the proper
[1:48] leaders. I don't think we killed any
[1:50] teams—you can, you can run me if I'm
[1:52] wrong—but yeah, maybe, maybe missed
[1:54] opportunity, not for not trying. I'm
[1:57] sure we probably wanted to.
[2:00] >> How to influence company-wide strategy
[2:02] and why often the most influential work
[2:04] is the ones that are intangible. We
[2:06] also talk about what kind of
[2:08] organizations need ecosystem data
[2:10] scientists, why it is ecosystem data
[2:12] scientist, not ecosystem engineer or
[2:14] product managers, and what kind of
[2:16] person fit this role. Let's go. Jin and
[2:20] Mushtaba, it's the second time for you
[2:22] to be on my podcast and you mentioned
[2:24] Jin used to be skill manager. Yes, Jin
[2:27] used to be my skip on ecosystems. He is
[2:29] one of the reasons I came and joined
[2:31] the team—he made it so compelling.
[2:33] That's a huge, that's how I thought you
[2:35] joined. So I'm glad to hear that. He
[2:37] then left soon me afterwards so I had
[2:39] to learn it quickly on my own, but it
[2:41] was 100%one of the reasons I joined.
[2:43] Well, once Mushtaba came on board,
[2:45] there was just nothing left for me to
[2:47] do, you know, he kind of took care of
[2:49] everything. Tell the personality of an
[2:51] ecosystemist.
[2:53] >> Yeah, I think more of our team specific
[2:55] , I think we elected to hire a bunch of
[2:58] somewhat sassy, low-arrogance
[2:59] individuals. Um, I think it is
[3:01] important for that kind of role. So
[3:03] maybe you're right as a common thread
[3:05] of teams. Um, uh, yeah, I think we did
[3:07] an incredible job hiring and so at
[3:09] least when I was there, I think we, you
[3:11] know, overall had an all-star team.
[3:14] >> I'll give you kind of my prejudice or
[3:16] my understanding of what is ecosystem
[3:18] DS and why that personality is an
[3:20] important aspect. So my understanding,
[3:24] like data scientist is very abstract
[3:26] and hard to describe. Ecosystem is
[3:29] abstract and hard to describe. So when
[3:33] you put those two things together, what
[3:35] does it mean? In my head, it is you
[3:38] work on something so complex which
[3:40] exists in companies like Meta or other
[3:42] big tech. So this could be challenged.
[3:46] So challenge me if you feel it's strong
[3:49] . But mostly in big companies, you have
[3:53] so many surfaces, and you have so many
[3:55] people, and you have so many, you know,
[3:57] process, program, and now is ecosystem,
[4:00] and it is someone's job to make things
[4:02] better. And there is a data aspect to
[4:06] this, the ecosystem DS is working on
[4:09] this. And because people work with each
[4:13] other through relationship and actual
[4:15] connections, so instead of optimizing
[4:17] any point, you're optimizing the
[4:19] connection or the value, so to speak.
[4:23] >> Yeah, I think that's generally the
[4:24] right perspective. I'll get my perspect
[4:26] ......the actual work these days, and
[4:30] I'm kind of curious what you think. So
[4:32] I think you are right that it is
[4:34] something unique but very common across
[4:36] big companies, and I think part of the
[4:38] reason is why most people I talk to in
[4:40] big companies, they found that they're
[4:42] structured in a very specific way where
[4:44] basically surfaces or units are siloed
[4:46] away. So, to use Facebook as an example
[4:48] : everyone uses the Facebook app, and
[4:51] then when you use the Facebook app, you
[4:53] kind of just think of it as one big app
[4:55] . But under the hood, there's, you know
[4:57] , Groups, Marketplace, Notifications,
[4:59] the Feed—all of these are their own
[5:01] separate and siloed teams, with their
[5:03] own separate and siloed objectives and
[5:05] things that they optimize for, and
[5:07] working styles, and everything. It's in
[5:09] some ways a company within a company,
[5:11] and each of these teams, especially a
[5:14] company like Facebook, is running
[5:15] 10,000 miles an hour at their objective
[5:17] , and they're not actually taking a lot
[5:20] of time to look around and see the
[5:21] forest for the trees. And so, what the
[5:24] ecosystem team's role, specifically
[5:26] using a Facebook example, would have is
[5:28] getting an understanding of whether the
[5:30] things that these teams are running
[5:32] toward, trying to accomplish, is good
[5:34] or the best thing that can be done for
[5:36] Facebook, the whole product. And so you
[5:38] know as example if let's say the video
[5:40] team is doing something to make people
[5:42] watch twice as much video—amazing,
[5:44] they're doing a credible job—but if
[5:46] that's making people have 10%fewer
[5:48] interactions with their friends, is
[5:50] that still a good job? How do you know?
[5:52] Um, and it's things like an ecosystem
[5:55] team understands. So it is complex, but
[5:57] I don't think the complexity is the
[5:59] defining characteristic; I think it's
[6:01] more the the connectivity a defining
[6:02] characteristic.
[6:03] >> Yeah, I think Jin and you both
[6:05] described it really well. I think that
[6:07] is a lot of like the motivation behind
[6:09] why I've seen most ecosystem teams
[6:11] created. It's like this idea that like
[6:13] no one's taking a step back to actually
[6:15] look at the overall health of the area.
[6:17] Everyone's kind of really just marching
[6:19] on their orders to whatever goals they
[6:20] have, whatever objectives they have,
[6:22] and no one's taking a step back to
[6:24] understand is it healthy. We talk about
[6:26] this a lot, like we don't ship the org
[6:28] chart, even though internally we're all
[6:30] looking at the org. The user just looks
[6:32] at the overall experience, and they're
[6:34] not going to notice whether or not
[6:36] groups is different or video is
[6:37] different; they're going to be like, "
[6:39] Facebook just feels weird to me." So we
[6:41] want someone to be thinking about that
[6:43] perspective. At least I've noticed on
[6:45] our side more recently, there's an
[6:47] added layer of like certain questions
[6:48] just make sense to have someone focus
[6:50] on who is on an ecosystem team that
[6:52] don't really fit squarely into any
[6:54] individual product team. So we also act
[6:56] to catch like the extra stuff that
[6:58] doesn't really fit into a product team.
[7:00] Example like a lot of the questions we
[7:02] get on our team look at, uh, how does
[7:04] our app interact with other apps. You
[7:05] can think about how does Facebook
[7:07] interact with TikTok or how does
[7:09] Facebook interact with WhatsApp or
[7:10] Instagram and other apps within the
[7:12] family. It doesn't squarely fit into
[7:14] video because there's a video component
[7:16] , but there's also a component of
[7:18] messaging. There's also component of
[7:20] friends and family, and it doesn't fit
[7:21] into any of those individual teams. So
[7:24] that it's also beneficial to have a
[7:25] team like ours that has someone with a
[7:27] horizontal understanding of the whole
[7:29] ecosystem that can tackle some of these
[7:31] questions that kind of like fall
[7:33] through the cracks.
[7:34] >> I remember like a pretty amusing story,
[7:36] and I think you were around for this.
[7:38] There was a time when our team
[7:40] identified that there were some trends
[7:42] in India that were pretty concerning
[7:44] and that their general relevance or the
[7:46] health of the Facebook app in India
[7:47] wasn't, uh, doing as well as it was in
[7:49] other parts of the globe. And I
[7:51] remember Fidji was the head of Facebook
[7:54] at the time, and she basically went to
[7:56] everyone of her leads who was a lead of
[7:58] a specific pillar area, and she's like,
[8:00] "Hey, I need you guys to fix or solve
[8:02] the India problem for me." And then I
[8:04] think she went to the person who's
[8:06] running News at the time, and that
[8:08] person came back with an amazing news
[8:09] solution. She's like, "No, no, I need a
[8:12] holistic solution." Then she went to
[8:14] the person who was running, I think
[8:16] like video or games at the time, maybe
[8:18] it was VJ actually, and she was like, "
[8:20] Hey, can you please help me fix this in
[8:22] your problem?" He came back with an
[8:24] amazing game strategy, how games could
[8:26] number one India, but that's how she
[8:28] was looking for, and that I think is
[8:30] emblematic of the challenges of this
[8:32] type of thing. Um, and actually I think
[8:34] from that, I think we actually formed
[8:36] like a little product ecosystem team on
[8:38] the side just directly out of that work
[8:41] . I don't know if that team's still
[8:42] around, but, uh, this is a very common
[8:45] problem, and like I have tons of
[8:46] examples of when we've made weird
[8:48] decisions because we didn't have like a
[8:50] data team sitting across all these
[8:52] areas. Um, and so it is just like a
[8:54] very necessary part of the companies. I
[8:56] agree with the necessary part, and
[8:58] later I'm going to challenge you with
[9:00] how to actually move people with
[9:02] incentives and with the right KPI or
[9:04] responsibility scope. But description
[9:06] reminds me of another function: growth.
[9:08] It's almost like growth is the kind of
[9:10] "push the ceiling" and the ecosystem is
[9:12] "push the bottom."
[9:13] >> I think they actually are a lot more
[9:15] similar, like I've seen in a lot of
[9:17] scenarios, and I've seen orgs where
[9:19] ecosystem sits with the growth like
[9:21] very hand in hand. I think within
[9:22] growth there is a product side to it,
[9:24] like those are teams who focus on SEO
[9:26] acquisition, like there's actual
[9:28] product levers with how we grow,
[9:29] whether that's email campaigns or
[9:31] actual product features. But then
[9:33] there's also the part of growth that's
[9:35] focus on just understanding our funnel,
[9:37] understanding our health, understanding
[9:39] our forecast and trajectories, and I
[9:41] think that's a lot closer to like that
[9:43] core ecosystem role of understanding
[9:44] and tracking health of the product. So
[9:46] I think within Facebook and probably at
[9:48] apps, I don't think those functions,
[9:50] those teams sit that far apart or don't
[9:52] work together. Yeah, I, I found it to
[9:54] vary by kind of the place where the
[9:56] company is at the time. Um, so when I
[9:58] joined Instacart, I led ecosystem team
[10:00] there too, because I couldn't quite get
[10:02] away from it. Um, when I joined
[10:03] Instacart we had an ecosystem team that
[10:05] was very focused on profitability
[10:07] because that was the biggest challenge
[10:09] for us at the time. Um, and so that
[10:11] team spent a lot of time within the
[10:13] shopper space, uh, with the finance
[10:14] team understanding the P&L. Um, but as
[10:16] we really nailed profitability and that
[10:18] wasn't an issue anymore, we came to a
[10:20] problem of growth. And then the team
[10:22] morphed to sit more, uh, closely with
[10:24] the growth space. And there are some
[10:26] people who are still in the team now
[10:28] who've had to kind of like reinvent
[10:30] themselves and move from being like an
[10:31] expert on profitability—now they have
[10:33] to be an expert on growth or marketing.
[10:35] Um, so it does kind of morph with the
[10:37] needs of the business as well.
[10:39] >> Nice. But when I think about it, like,
[10:42] growth has a lot of visibility, like
[10:44] growth hackers and growth DS—the hot
[10:46] positions. But ecosystem is much less
[10:48] obvious, and I think it's because
[10:50] growth is very easy to measure, right?
[10:53] You increase retention, and you can do
[10:55] a lot of AB tests to show the growth,
[10:57] and you took initiative, and you can
[10:59] show incrementality. Ecosystem DS is
[11:01] much harder. So, how would you evaluate
[11:04] whether ecosystem DS is effective or
[11:06] not? It's a good question. At one point
[11:08] in time, I propose we just run a survey
[11:10] of our executive team, a satisfaction
[11:13] survey and the degree to how happy we
[11:15] do a good job, but that's not like that
[11:17] far off. We talk a lot about, like, the
[11:20] data science function really being
[11:22] about influence, and I think, to some
[11:24] degree, that's like the most important
[11:26] thing for ecosystems because our
[11:28] biggest, our main stakeholders are a
[11:30] leadership team, and the degree to
[11:31] which the app feels like they know
[11:33] what's going on and they understand
[11:35] what's going on. So, when it comes to
[11:38] measuring success or tracking, like,
[11:40] the influence of the success of our DS,
[11:43] it's a combination of how well do we
[11:45] feel like we are answering the
[11:47] questions we're getting, and how
[11:48] satisfied are our leadership team, our
[11:51] key stakeholders are XFN, and how much
[11:53] are we pushing forward our
[11:54] understanding beyond the areas that
[11:56] they just asking. So, there's a couple
[11:59] axes, like, I've managed both on
[12:01] ecosystems and outside of ecosystems.
[12:03] On ecosystems, I rely way more on peer
[12:05] feedback and quotes from leadership on,
[12:08] like, the quality to which they feel
[12:10] like they're being informed by our team
[12:12] , and I think that's a very significant
[12:14] signal in success of our team, the
[12:16] satisfaction of, like, the stakeholders
[12:18] we have and their understanding the
[12:21] ecosystem. Before answer, may I
[12:22] challenge: what if the leader is wrong?
[12:25] What if you get more affinity from a
[12:27] leader who has a wrong opinion? Oh,
[12:29] that's happened a lot, and I think Jin
[12:31] can attest this time on the ecosystem
[12:33] both at Facebook and, uh, maybe at
[12:36] Instagram. I think I've been wrong,
[12:38] stakeholders who are wrong who could
[12:40] still give good feedback. It's about
[12:42] whether or not they felt like I made a
[12:44] very clear case to them, or we
[12:46] presented well to them on why we think
[12:48] their opinion is wrong, and did they
[12:50] feel we had a strong data-backed
[12:52] argument? So, a lot of times it's not
[12:54] about alignment or, like, satisfying
[12:56] the questions some leads have. It's
[12:59] about the degree to which we are making
[13:01] sure they're informed on what's exactly
[13:03] going on in the ecosystem and the
[13:05] trends and the health of the app. But
[13:08] this requires the leadership to have
[13:10] intellectual honesty and even when you
[13:12] disagree the value of merit.
[13:14] >> It's a fair point and I appear to hear
[13:16] your opinion, like dealing with leads
[13:18] who may have intellectual honesty at
[13:20] times. But in my experience, I think
[13:22] it's the advantage we have is like we
[13:24] don't have one single person who has a
[13:26] say in any of this, not one single
[13:28] leadership person who kind of sways all
[13:31] of this. So I've been able to get a
[13:33] pretty good balanced signal across
[13:35] people where at times some people might
[13:37] not be satisfied but we've generally
[13:39] been able to get signal across the
[13:41] leadership team; at least the
[13:42] information representing and the
[13:44] roadmap that pushing on is helping them
[13:46] feel informed even if it's not
[13:48] validating the hypothesis opinion they
[13:50] already had.
[13:52] >> So, I think you touch on a really
[13:53] important point, which is why I think I
[13:55] agreed with you eventually in the end.
[13:57] But I think your first idea, and maybe
[13:59] you picked it when I was there and I
[14:01] said no, but your first idea of like
[14:03] running a survey of execs didn't quite
[14:05] sit that well with me. I think to get
[14:07] to your point, I think there is an
[14:08] underlying assumption, very important,
[14:10] is that the difference in a product
[14:12] data scientist and ecosystem data
[14:14] scientist is almost the level at which
[14:16] you operate. You might think there's a
[14:18] big distinction between like working
[14:20] with like a C-level exec VP versus like
[14:22] a director/senior director, but one of
[14:24] the big differences is where their
[14:25] incentives are, right? And so if you
[14:27] are the senior director PM of newsfeed
[14:29] ranking and your goal is not
[14:30] necessarily to make Facebook better,
[14:32] your goal is to agree on what metric
[14:34] you're going to measure your
[14:35] performance—that everyone agrees that
[14:37] if you do this metric you make Facebook
[14:39] better—and then you just improve that
[14:41] thing, and if you have a data scientist
[14:43] who's working with you, they're going
[14:45] to try to work with you to improve that
[14:47] specific thing. And so if—and this
[14:49] has happened sometimes at Facebook—
[14:51] where if your data scientist on feed
[14:53] ranking comes and says, "Hey, I devised
[14:55] this new thing and it's going to hurt
[14:57] our metric for the half," I think it's
[14:59] amazing for the user, that leader
[15:00] probably will say "no thank you."
[15:02] Here's some negative peer feedback
[15:04] because that's not what their
[15:05] incentives to do, right? That's not how
[15:07] setup. But the person you're working
[15:09] with the ecosystem level, like when we
[15:11] were Facebook, we worked with a bunch
[15:13] of people—I think I was there for the
[15:15] longest—we worked with the Facebook
[15:17] app and their incentives should be to
[15:18] make Facebook as good as possible. And
[15:20] so, if you come to that person with
[15:22] idea that's like, "Hey, I have an idea
[15:24] that's going to make feed ranking a bit
[15:26] worse, but it's going to make me out
[15:28] better, and here, let me prove it to
[15:30] you," I think they should always be
[15:32] interested in hearing that. They should
[15:34] always have that intellectual honesty
[15:36] because the incentives are aligned, and
[15:38] I think in most cases, the incentives
[15:40] are aligned. From my memory, Facebook,
[15:42] they always were. I will say, there are
[15:44] times, care-wise, they weren't always
[15:46] aligned, and then it gets challenging.
[15:48] Um, and they could be misaligned for a
[15:50] variety of reasons which I won't get
[15:52] into in the public podcast. But I do
[15:54] think like having those incentives
[15:55] aligned is vitally important, and this
[15:57] is why sometimes I've seen ecosystem
[15:59] teams form at much lower altitudes, and
[16:01] it just doesn't work that well because
[16:03] you don't have as much of that
[16:05] connection to the incentives of that
[16:07] leader to be intellectually honest
[16:08] about their products. But I think that
[16:11] is a very, very important requirement.
[16:15] I remember there was a joke at Meta: "
[16:17] The quickest way for DS to make impact
[16:19] is to kill a team." But you cannot kill
[16:24] the team at the team manager level; you
[16:26] have to go like two, three levels above
[16:28] to fence the right incentive to do that
[16:30] , and probably ecosystem DS would have
[16:32] to make a lot of such recommendations
[16:34] to the proper leaders.
[16:37] >> I don't think we killed any teams—you
[16:40] can, you can run me if I'm wrong—but
[16:42] yeah, maybe opportunity, not for not
[16:44] trying. I'm sure we probably wanted to.
[16:47] Okay. So that's what you're working on.
[16:49] Okay. Now remember.
[16:51] >> Yeah. So I actually have this analogy:
[16:53] if your brain works like a company and
[16:56] not like a holistic unit, and you try
[16:58] to lose weight and you set KPI of
[17:00] losing weight, what would your arm do?
[17:03] Like your arm, probably would just chop
[17:05] off arm because that's the quickest way
[17:07] to lose weight. And I do think similar
[17:10] things happen as a company. If you just
[17:14] chase profitability or chase a very
[17:16] simple KPI, people will do things to
[17:18] hurt ecosystem health in order to get
[17:20] their win and promotion because their
[17:22] incentive is on their own thing. So
[17:26] that kind of gets to my next question:
[17:28] you can influence the leaders, and
[17:30] hopefully the leader, the high-level
[17:32] leader, their incentive is a
[17:34] well-incentive. But if you want to
[17:37] execute the recommendation, if you want
[17:40] to do things, still comes down to the
[17:42] like domain owners or surface owners,
[17:44] the specific things. And sometimes, if
[17:48] you really want to get the real impact
[17:50] —not only I submitted recommendation,
[17:52] leader took my recommendation—if you
[17:54] want to see it executed, you have to
[17:56] influence like those owners as well.
[18:00] And is there an effective way of doing
[18:02] so?
[18:02] >> Um, I can, I can take this one first. I
[18:05] found two methods of doing this, and
[18:07] I'll share what's an important aspect
[18:09] of these methods, because I think you
[18:11] might be getting into topics of career
[18:12] which are also important. Um, so the
[18:15] first method is kind of, uh, like
[18:16] across and down, where you would find a
[18:19] person that you work with, hopefully at
[18:21] whatever is the highest decision-making
[18:23] level. Like a Facebook app example, you
[18:25] convince them that something needs to
[18:27] happen, um, and that something will
[18:29] affect the groups team, and then you
[18:31] with that person, and ideally the head
[18:33] of the groups team in the room as well,
[18:35] will all agree based on the fundamental
[18:37] principles you established based on
[18:39] your, uh, neutral J people decision,
[18:49] and get them to do it. That's the way I
[18:52] operated. I found the most effective
[18:54] way. I know a lot of people on the team
[18:57] go across and up, which is they work
[18:59] really closely with—they find often
[19:01] data sciences, but sometimes PMs and or
[19:03] EMs on those teams, and they work
[19:05] closely with those PMs to really
[19:07] understand the product and they align
[19:15] Facebook groups, yet both methods can
[19:17] work. You can do both at the same time
[19:19] as well. I think you just need to find
[19:21] what is effective for you. I found the
[19:22] biggest challenge with the first method
[19:24] , and this will get into, you know,
[19:26] topics of career growth: If you propose
[19:28] the thing and the thing works and that
[19:30] team does the thing, everyone on that
[19:32] team is going to take credit for it in
[19:34] their performance reviews, right? Like
[19:36] the people who work on the groups team,
[19:37] they're going to say, "Hey, we did big
[19:39] groups initiative number two. It was
[19:41] amazing. It led to 5 million group
[19:43] connections. Please promote me." And
[19:45] then I'm going to go, "Well, it was my
[19:47] idea, the group—the groups initial
[19:49] member, to promote me as well." And
[19:51] that gets challenging, right? And so I
[19:52] have a pretty funny method for doing
[19:54] this. I would always do like calling my
[19:56] shot. Um, my former manager is
[19:58] listening to this. I think he'll be
[20:00] amused. Um, I would always tell him,
[20:01] like a couple weeks—I have time, I'd
[20:03] be like, "Hey, I think this should be
[20:05] happening in the product. I'm going to
[20:07] go make it happen. So, you know, watch
[20:09] this space in three weeks. The thing
[20:11] that I want to happen is going to
[20:12] happen." And three weeks later,
[20:14] suddenly a director in groups makes a
[20:16] post about exactly the thing I want to
[20:18] happen. Had that happen—who knows? I
[20:20] called my shot, I usually get credit
[20:22] for having done it, and I feel like
[20:23] that worked pretty well. But uh, very
[20:25] often, you learn these techniques after
[20:27] you run into situations where you don't
[20:29] get credit for things that actually
[20:31] were your idea to begin with.
[20:32] >> Yeah. I think the one thing I would add
[20:35] to the first part of Jean's answer is
[20:37] like, in my experience, I like to kind
[20:39] of do a hybrid of both top-down and
[20:41] bottom-up. I think the main thing I
[20:45] like to avoid with just the top-down is
[20:46] : if I go to the head of Facebook app
[20:48] and I suggest that he needs groups to
[20:50] do something, I can already guess what
[20:52] groups is probably going to push back
[20:53] with. And the more I do bottom-up and I
[20:57] reach out to people either at my level,
[20:58] the DS on that level, I kind of like
[21:00] preview my proposal and figure out:
[21:02] what is their push back going to be?
[21:05] What are they going to say about what
[21:06] I'm going to—what are they going to
[21:08] be? And how do I address it before I go
[21:11] to the app, and he's disputing, and we
[21:12] get back? And I just present the pros
[21:15] and cons and be like, "You already know
[21:17] what they're going to say, now you can
[21:19] make the decision you need to make." I
[21:21] think in my, and I think this happens
[21:23] more especially when working like
[21:25] cross-org. So if you're doing Facebook
[21:27] and I, you maybe get a bit more
[21:29] pushback sometimes when the orgs are
[21:30] different, their incentives and goals
[21:32] are different. Getting some steps ahead
[21:36] on what likely will be the pushback and
[21:37] figuring that out by talking to some
[21:39] people on the ground can just like
[21:40] strengthen the proposal and just make
[21:42] sure you cover your bases. Like, is
[21:45] there anything that I'm missing that I
[21:47] kind of like not got in my context,
[21:49] which happens a lot on ecosystems? You
[21:52] don't have the depth of context in all
[21:53] these orgs that the product does, so
[21:55] there is value in just going and
[21:57] talking to some of these people to
[21:58] figure out, like, am I missing
[22:00] something? Am I not aware of context
[22:02] that talking to you will help resolve?
[22:05] >> Nice, like those are great
[22:07] recommendations. Backchanneling, I feel
[22:10] like, especially in a large company—
[22:12] actually in a startup as well—like if
[22:14] you want to have a successful meeting,
[22:16] you have to do the backchanneling. Most
[22:19] decisions are made in small rooms, not
[22:21] in big rooms. Like in big rooms, it's
[22:24] just announcement instead of making the
[22:26] actual decisions. Backchanneling has a
[22:28] bit of a negative context. I will call
[22:31] it, discuss more of, pre-aligning,
[22:33] which I think is in some ways healthy.
[22:35] Yeah, pre-aligning and like the calling
[22:38] the...but how would you write that on
[22:40] your PSC? I called the shot and it
[22:42] happened.
[22:42] >> If you read my PSC...I was a very
[22:44] interesting PSC writer for sure. I
[22:47] think you just have to, you know, be
[22:48] honest about the things that you worked
[22:50] on and the things that you proposed and
[22:52] their outcomes. And I asked this
[22:54] whenever I do calibrations as a manager
[22:56] , I always ask this question for folks
[22:58] even on product teams. It's like, "If
[23:00] this person didn't exist—uh, if they
[23:02] weren't here for the last six months,
[23:04] what would have happened?" I think an
[23:06] ecosystem team, very often you will
[23:08] find that if that person didn't exist,
[23:10] that whole strategic pivot wouldn't
[23:12] have happened. And, obviously, the
[23:13] success of the pivot was based on the
[23:15] work of dozens or hundreds of people.
[23:17] Or, absolutely, they should get tons of
[23:19] credit for execution and ideation, but
[23:21] there is some element of like that
[23:23] initial spark that I think, you know,
[23:25] you should take credit for. And,
[23:27] especially if it was a good idea, the
[23:29] execution matched what you had
[23:30] originally put forward in your idea, I
[23:32] think you should take some credit for
[23:34] that as well. Um, it's not that
[23:36] dissimilar to being a very senior IC on
[23:38] a product team and having like a really
[23:40] big idea for a product, just you're
[23:42] kind of operating a bit more high level
[23:44] because you're specifically asking
[23:46] about PSC. One thing I would also say,
[23:48] and maybe this is just giving credit to
[23:50] Gene, I think you also have to create a
[23:52] culture within your organization that
[23:54] this type of work is valued. Like, I
[23:56] think across the management chain, like
[23:58] there has to be buy-in across like
[24:00] senior leadership with the DS that this
[24:02] type of work is important. I've been in
[24:04] rooms where people who have not any
[24:06] ecosystem DS close to them or near them
[24:08] are really wondering like, "Are we
[24:09] really giving this person credit for
[24:11] this whole strategy?"
[24:14] >> And I was like, we're not saying that
[24:16] the execution and everything is perfect
[24:17] and everything launched because of
[24:19] their analysis, but it's very evident
[24:21] that they wouldn't have put the
[24:22] headcount investment there if we didn't
[24:24] do this analysis beforehand. And I
[24:26] think, over the years, Gene and like
[24:28] all the predecessors on ecosystems have
[24:30] done a really good job creating a
[24:32] culture at Facebook App that this type
[24:34] of work is valued. performance reviews
[24:38] on. It's clear to draw the connection
[24:41] between the analysis and outcomes. We
[24:44] more just have to show that the outc—
[24:46] but there's no cause—important
[24:48] because you have to establish the
[24:50] matter of whether the job is
[24:52] fundamental, available, especially for
[24:54] this kind of qualitative work. Like,
[24:57] you cannot put everything in "what you
[24:59] did,""what you built,""how many lines
[25:02] of code you shipped,""how much outcome.
[25:04] " Then, at the fundamental level,
[25:07] people have to believe in this. I think
[25:10] this is very interesting to people who
[25:12] had experience, but to kind of someone
[25:14] new this may sound too abstract and I
[25:16] want to get a couple, like, dollars
[25:18] from you. I don't know how much you can
[25:22] share, but if you put what you just
[25:24] said into a concrete story, can you
[25:26] share one or two stories?
[25:28] >> Yeah, I don't work there anymore so I
[25:30] can, I can share whatever I want. Try
[25:32] and think of what represented story
[25:34] would be. I think you—I told the one
[25:36] about India earlier, I think that was a
[25:38] pretty solid example.
[25:40] >> Let me ask you a very specific question
[25:42] . Tell me a story when your
[25:45] recommendation goes against a
[25:47] director's KPI, for example, or VP KPI.
[25:50] "I want to increase this for my product
[25:53] ," but ecosystem DS tells me it's not a
[25:55] good idea, or maybe I should do
[25:57] something else, or maybe, you know, I
[25:59] should change my KPI. You were at odds
[26:03] with another leader owner.
[26:05] >> I found that, that what you're
[26:07] describing doesn't often happen on
[26:08] ecosystem team. That does often happen
[26:11] on integrity teams that have outcomes
[26:13] that are, in some ways, counter to the
[26:15] product teams 'outcome. For us, it was
[26:18] more like our high level goal was the
[26:20] same as every product team; it was more
[26:22] in the direction that we thought was
[26:24] important to go into. So, I think one,
[26:26] uh, really big push that we made, this
[26:29] was in, I want to say, like 2019 or
[26:31] something, um, was, uh, the Facebook
[26:33] app was doing really well overall, and,
[26:35] but one thing we found was it wasn't
[26:37] doing quite as well for folks in the,
[26:39] for the exact age range, was like 17 to
[26:42] 29, or like folks of, of that kind of,
[26:44] like, not quite, not quite millennial,
[26:46] like that kind of middle age range
[26:48] adults.
[26:49] >> Yeah, young adults, um, we found that
[26:51] they weren't, uh, kind of using Face—
[26:53] it's not that they weren't doing well,
[26:56] just they weren't even using Facebook
[26:58] in the same way that we would call to
[27:00] be like a successful, healthy use of
[27:01] Facebook. Um, we discovered through
[27:03] analysis, like we did clustering, we
[27:05] looked at trends, we looked at, um,
[27:07] experiments, and found, like, pockets
[27:09] where experiments were being driven by
[27:11] one segment that was not this segment
[27:13] —this segment was actually flat of
[27:15] our biggest experiment, um. And so
[27:17] there are a lot of things together, and
[27:19] you know, we kind of packaged this into
[27:21] analysis that basically suggested, like
[27:23] , "Hey, Facebook is doing really well,
[27:25] but it's these people over here are
[27:27] driving all of it. This group here,
[27:29] they're the younger generation, nothing
[27:31] coming, they're not finding relevance
[27:33] here at all." And I don't think anyone
[27:35] said, like, "Oh no, well, this is a
[27:37] horrible idea. This is the opposite of
[27:40] this, my product only focuses on 50
[27:42] year olds." But it's more that for them
[27:44] , they would have to shift their
[27:46] strategy in a direction that is harder
[27:47] to do, and that they haven't been able
[27:49] to do, because if they had been able to
[27:51] do, you wouldn't be having this
[27:53] conversation. Um, and so it's more just
[27:55] harder for them to do and they didn't
[27:57] know if they could succeed at the same
[27:59] level, right? So, if you told the head
[28:01] of News, "Hey, you have to increase
[28:03] daily active users by 4%. But you have
[28:05] to do it for this group specifically,"
[28:07] they'd be like, "Well, last year I did
[28:09] 4%. I did over for this group. I did
[28:11] zer...I don't know how to do that yet."
[28:14] And so, that is more of the challenge
[28:16] that we have rather than, "You're doing
[28:18] daily active users, but we want you to
[28:20] profit," or something like that. See,
[28:22] so not against, but shifting priorities
[28:24] and focus.
[28:25] >> Exactly.
[28:28] >> I remember that young adult analysis
[28:30] very much because I was working on
[28:32] marketplace, and today's day I hear
[28:34] people paying increasing attention to
[28:36] marketplace because marketplace is able
[28:38] to draw youngers to Facebook.
[28:41] >> You're welcome. Yeah, I think to Je's
[28:43] point, like there's not that many
[28:45] examples of like you directly going
[28:46] head to head with the lead where
[28:48] they're like trying to increase a
[28:50] metric and you try to say like "No,
[28:52] that metric doesn't work." I think a
[28:53] lot of times it is prioritization. I
[28:55] think, along the same vein, like we had
[28:57] a lot of conversations on just regional
[29:00] and prioritization. Focus. to focus on
[29:06] US/Canada. Like, there are other
[29:08] reasons: like a huge revenue focus, a
[29:10] huge cultural relevance focus, a lot of
[29:12] early adoption of, and trends happen in
[29:14] US/Canada. So, I think one thing we run
[29:16] into a lot, at least on ecosystems and
[29:19] maybe this is, uh, another question
[29:20] going to come up, is like, we have
[29:22] these perennial questions that we
[29:24] address every few years. And we had
[29:26] questions on, like, "Should we be
[29:28] prioritizing regions over others?" And
[29:30] every few years, we kind of answer the
[29:32] same question. We look at the same
[29:34] trends, we look at the same
[29:36] prioritization of, like, "Hey, we're
[29:38] doing really well in certain parts of
[29:40] the world where we're growing really
[29:42] quickly, but we're weaker in certain
[29:44] cohorts where we have a lot more
[29:45] competition—are we still okay with
[29:47] where we're investing, or should we be
[29:49] prioritizing some of these
[29:51] up-and-coming headwinds?" I think the,
[29:53] the issue that we run into usually is
[29:55] we present the same data, but we don't
[29:57] change our decision. For me, the
[29:59] biggest problem has always been of like
[30:01] , how do I come to terms with that?
[30:04] This is the same question we're going
[30:05] to get asked a year from now and I
[30:07] don't think the data is going to
[30:09] significantly change. But I've learned
[30:11] that sometimes it's not that the data
[30:13] changes, but like the leadership
[30:15] circumstances change. Like there's a
[30:18] different level of risk tolerance,
[30:19] there's a different level of external
[30:21] pressure, there's a different level of
[30:23] like competitive pressure that results
[30:25] in us changing decisions, and sometimes
[30:27] it's okay if I'm pulling the same data
[30:29] or refreshing analysis, because there
[30:31] might be external factors changing that
[30:33] , leading to a different decision
[30:35] versus like I'm not just kind of like
[30:37] beating the same drum. So that's a
[30:39] bigger issue I've had where, like, I've
[30:41] had to like answer the same questions
[30:43] for three years in a row, and then at
[30:45] some point I thought I was just crazy,
[30:48] then the third year, we decided to do
[30:50] something. Well, into my next set of
[30:52] questions: What makes a good ecosystem
[30:54] DS, and how to quantify that work? And
[30:56] just based on our previous discussion,
[30:58] I can already think about many traits
[31:00] would be required for a successful
[31:02] ecosystem DS. Uh, such as proactive,
[31:04] taking initiative, taking a step back
[31:06] means there are a lot of uncertainty
[31:08] and you, there are many directions to
[31:10] go, you have to take the initiative and
[31:12] do this kind of big analysis or
[31:14] holistically. And being able to deal
[31:16] with complexity, I think is one,
[31:18] because you are working with so many
[31:20] different product areas. You don't have
[31:22] to understand them as well as each
[31:24] product-specific DS, but you need to
[31:26] have the basic understanding or deep
[31:28] enough understanding, and you need to
[31:30] kind of be able to influence without
[31:32] authority. You need to kind of find the
[31:34] right way of using leverage, right? You
[31:37] need to figure out how to move, how to
[31:39] build the relationship, and how to not
[31:41] hurt relationships in order to do
[31:42] anything. Anything I missed that to
[31:44] consider with this one? I think you
[31:46] covered it pretty well, like, I think
[31:48] autonomy, being independent, being
[31:50] fairly resourceful—we oftentimes have
[31:53] to go, like, jump into video. We don't
[31:55] know video data. So you have to be able
[31:57] to like ramp up pretty quickly in some
[31:59] of these scenarios. Me and my manager
[32:01] used a joke about it. It's like you're
[32:03] mercenaries where you just like
[32:05] parachute into an area, quickly ramp up
[32:07] , figure out what you need to know to
[32:09] be impactful. I think the one other
[32:11] thing I would add beyond the autonomy
[32:13] and communication skills and the
[32:15] complexity you talked about is you have
[32:17] to be inquisitive. I think ecosystem DS
[32:19] have to be able to have a strong
[32:21] intuition and question things when it
[32:23] comes to ideas in that sense. Because a
[32:25] product has a lot of XFN to you, have a
[32:27] lot of PMs and who are giving you
[32:29] questions and asking questions, and as
[32:31] an ecosystem DS, you kind of have to do
[32:33] that yourself. So the most successful
[32:36] ecosystem DS in my experience have also
[32:38] been the more generative ones who are
[32:40] constantly asking questions, fairly
[32:42] inquisitive on like the user experience
[32:44] , the product strategy, the direction
[32:46] we're going in, because it helps them
[32:48] go a bit a few layers deeper into their
[32:50] own analysis and actually drive their
[32:52] own road. On ecosystem, that's one of
[32:54] the hard things I think that you have
[32:56] to deal with, where on a product
[32:58] roadmap usually I have a million
[33:00] questions, "How do I prioritize what I
[33:02] need to do?" On ecosystem, what is the
[33:04] most important thing of your own
[33:06] roadmap? That I think at times people
[33:08] underestimate as complex part of the
[33:10] job. Question: "Like why do you choose
[33:12] the word inquisitive rather than
[33:14] curious?" Ah, I don't think a strong
[33:16] reason—vocabulary—I think it's
[33:18] either work for me. It's inquisitive is
[33:20] more about like asking questions. I
[33:22] think it's the type of person who
[33:24] usually when they see data or when they
[33:26] see an analysis, I joke about it. I
[33:28] think ecosystem DS are the most
[33:30] skeptical about anything. They're
[33:31] generally the most optimistic ones in
[33:33] the team. They spend so much time
[33:35] looking at apps and trends that they
[33:37] probably are the most skeptical in the
[33:39] org. And I think that comes from a
[33:41] place of questioning a lot of things.
[33:43] Questioning our assumptions,
[33:45] questioning our positive trends, but
[33:47] also questioning our negative trends.
[33:49] Like are we ever as good as we think we
[33:51] are? And are we ever as bad as we think
[33:53] ? Yeah, I think the questioning and
[33:55] asking the question is the difference
[33:57] between those two words, and I think
[33:59] you all, the descriptions like
[34:01] intellectually curious is probably the
[34:03] most important thing. Um, I think
[34:05] there's an element of ownership that's
[34:07] very important as well. I don't say
[34:09] it's easy because it's still not easy
[34:11] to like have a strong sense of
[34:13] ownership when you're working on a very
[34:15] small product surface because you have
[34:17] a lot of control over it. Continuing to
[34:19] exhibit that sense of ownership over a
[34:21] large surface area that you probably
[34:23] don't have a lot of control over is
[34:25] pretty challenging, but it's something
[34:27] that I like to encourage people to
[34:29] really lean into. I say to some of the
[34:31] product teams I support as well.
[34:33] Basically, your area is your house,
[34:35] right? And like if you saw dirt in the
[34:37] corner of your house you would pick it
[34:39] up. Um, and if you saw a leak in your
[34:41] house you wouldn't clean up the leak.
[34:43] You would be like, "Why is it leaking?"
[34:46] Um, and if you're, you know, data
[34:47] scientist whose product is, uh, all of
[34:50] Instagram or all of Facebook, then it
[34:52] gets hard because it's a very large
[34:54] house that you're owning. But it is
[34:59] leaks and dirt and small animals or
[35:00] whatever one does with large house, you
[35:03] know.
[35:06] >> Exactly. Yeah. When I was a junior data
[35:09] scientist on the team, I would like
[35:10] obsessively check every dashboard all
[35:12] the time, and any time anything went
[35:14] wrong or looked funny, I like always
[35:16] look into it. I would be like posting
[35:18] in the groups, "Know going out this,
[35:20] this, or that." It was probably kind of
[35:22] annoying for people that did this back
[35:24] in the day. Um, but I do think like the
[35:26] most successful people I've seen have
[35:28] all been done that and been annoying in
[35:30] that high ownership kind of way.
[35:32] >> Yeah, the curiosity, and also probably
[35:34] you were curious because you were
[35:36] building your mental model, right? You
[35:39] were trying to figure out what is
[35:41] abnormal, what is, you know, expected,
[35:43] and from the model you get the true
[35:45] understanding of these parts and how
[35:47] they work together.
[35:48] >> Yeah, absolutely.
[35:50] >> Nice. That is a lot of traits, and
[35:53] probably rare traits to find on someone
[35:55] . Then what makes a more successful
[35:58] ecosystem DS? Let's actually look from
[36:01] that perspective, like how do you
[36:03] evaluate the quality of work? What kind
[36:07] of ecosystem DS would you promote out?
[36:10] >> I think the one thing to clarify is I
[36:12] don't think there's a single standard
[36:14] version of an ecosystem DS. Um, I think
[36:16] Je probably seen this a lot in his time
[36:18] and manage a lot. Like, I try to
[36:20] recruit a variety of expertise because
[36:22] I do think you need different types of
[36:24] people on ecosystems. Some of them are
[36:26] a lot more technical in nature. I think
[36:29] like different companies have different
[36:30] archetypes. At Facebook, we call these
[36:33] tech gurus because I think there's like
[36:35] a suite of problems, especially that
[36:37] you face on ecosystems, which is like
[36:39] standardization of analysis and tools
[36:41] that you do need someone more technical
[36:43] for. But do you need a team of 10
[36:45] technical gurus? Probably not. But I do
[36:47] think that you kind of have a certain
[36:49] set of problems that you need a certain
[36:51] skillset for, that's not the same as
[36:53] all other problems. Like, some of our
[36:55] problems really require someone with
[36:57] strong communication skills because all
[36:59] of their work is really about executive
[37:02] communication. And then there's like a
[37:04] really high level of polish required
[37:06] when you're sitting in front of this
[37:08] audience, and your ability to like
[37:10] address questions and go back and forth
[37:12] , communicate clearly, that it's not
[37:13] the same as the product or somewhere
[37:15] else. So I do think that even within
[37:18] ecosystems, yes that's an archetype,
[37:20] there's like a spectrum of the type of
[37:22] skills you Yeah, I think that's right.
[37:29] SK, if you look at the inverse, there
[37:31] are folks who have been on ecosystems
[37:33] teams who are just—it's not the right
[37:35] fit for them. I think there's like one
[37:37] example of someone who's on my team
[37:38] about two years ago, they moved to a
[37:40] product team, and I remember there was
[37:42] like one quarter where that person
[37:44] worked super closely together on
[37:45] analysis and they were doing a great
[37:47] job, I thought. Um, and you, this was
[37:49] probably one of the top like five that
[37:51] we put together. You mean, changed the
[37:53] way that we thought that growth. And at
[37:55] the end of this, this person talked: "
[37:57] I'm going to move to a different team.
[37:59] I really enjoy working with you. I like
[38:01] this result, but this is not the type
[38:03] of work I want to be doing. This isn't
[38:05] like growing the type of muscles that
[38:07] like I want to be growing." Um, and
[38:09] they moved to a team where they can
[38:10] just kind of like sit in a room with
[38:12] like an engineer and just like hammer
[38:14] out model improvements, and that's the
[38:16] type of thing they want to, which
[38:17] they're now amazing at. Um, but it's a
[38:19] very, very different work, just like
[38:22] anyone. I've tried to do that or step
[38:24] into that type of role. It really
[38:25] didn't fit what I wanted to do. Like I
[38:27] would not be able to do that type of
[38:29] role full time. Um, and so to some
[38:31] degree like it's less like archetypes,
[38:33] it's more like what the spectrum of
[38:34] things people gravitate towards. I
[38:36] think like the characteristics we
[38:38] talked about earlier are just like the
[38:40] important ones to try and tease out,
[38:42] and then like a willingness and
[38:43] excitement about this type of work.
[38:45] Like if you think of some of the, you
[38:47] know, best outcomes you can help as an
[38:49] ecosystem data scientist, is it
[38:50] exciting to you to have those outcomes?
[38:52] And if it's not, then probably just not
[38:54] good for that specific type of role.
[38:56] >> Yeah, there's some data scientists
[38:58] across a company who have tried to
[39:00] recruit who I think would be perfect on
[39:02] ecosystems, like they have all the
[39:04] strengths, but they just really love
[39:06] product. They just love being able to
[39:08] ship stuff, and that's what we don't
[39:10] get to do. That's skillsets that are
[39:12] really valuable, but also like an
[39:14] interest in the type of work we do.
[39:16] This interest is so important that I
[39:18] want to kind of ask a difficult
[39:19] question. You both talk about spectrum.
[39:22] If you put words on the end of spectrum
[39:23] , what would you choose?
[39:25] >> And is this a spectrum of like
[39:27] ecosystem product or what?
[39:28] >> No, like the spectrum of interest, like
[39:31] for example like you gravitate towards,
[39:33] like for example, execution and biggest
[39:36] problem like new challenges. I don't
[39:40] know what are the universe of these
[39:41] words and I don't know what I would put
[39:43] on the spectrum, but it's the spectrum
[39:45] of interest that would define different
[39:46] .
[39:49] >> The words I would use off the top of my
[39:51] head are probably tangible and
[39:53] intangible. And so when you look at
[39:54] product data scientists, I think they
[39:56] are closer to tangible. If you go
[39:58] beyond that, you have like engineers
[40:00] who are actually physically writing the
[40:02] code to build the things. As a product
[40:04] data scientist, you are working on
[40:06] group search and you're like, "Hey, I
[40:07] work on group search. Look at this
[40:09] group search. Check it out. Boom, boom,
[40:11] boom. I built that." Then you have
[40:14] ecosystem data science, which is much
[40:16] more intangible, right? To the point
[40:17] where it's hard for me to explain to my
[40:19] parents. Exactly. Engineers beyond that
[40:26] . Management consultants beyond that,
[40:32] and so I think that is the much broader
[40:34] spectrum, and I think that ecosystem DS
[40:36] is further to the more intangible part
[40:39] of that spectrum.
[40:41] >> Yeah, that's a great point.
[40:43] >> Yeah, I think it's a level of ambiguity
[40:45] , or like how defined the scope is.
[40:47] Because somewhere in the middle you
[40:49] probably also have like people who work
[40:51] on zero-to-one spaces. Like, there's a
[40:53] clear product, but it's like much more
[40:55] intangible than, like, a very mature
[40:57] product space where you kind of have
[40:58] very clear defined metrics and data.
[41:00] And there's people who are working on
[41:02] new bets or big bets, and I think
[41:04] there's like a lot more smaller things
[41:06] that kind of fall in that spectrum.
[41:07] Where I tell this to my team all the
[41:09] time: like you could be a product DS,
[41:11] yes, and do an ecosystem project. Like
[41:13] you could be focusing on one project
[41:15] where it's like taking a few steps back
[41:17] to look at the overall ecosystem, but
[41:19] you don't have to be a full ecosystem
[41:21] person. Yes, like that doesn't have to
[41:23] be your full-time job all the time.
[41:24] It's like, on a project-to-project
[41:26] basis, it might vary as well depending
[41:28] on the maturity of the project and the
[41:30] need of the strategy at that point in
[41:32] time. I feel like a lot of the very
[41:34] senior data scientists, like if you
[41:35] look at like L7, L8, unless you are
[41:37] like a deep technical expert and that's
[41:39] like your one thing, you can't not do
[41:41] some type of ecosystem work.
[41:43] >> Yeah. This makes me feel like the
[41:45] ecosystem is actually a lead-career
[41:47] senior role. It's hard to start as an
[41:50] ecosystem DS because it's very hard to
[41:52] justify impact or influence the right
[41:54] level of leadership as your mission,
[41:56] right? Into your former ecosystem team,
[41:59] from a low altitude, the team is hard
[42:01] to survive.
[42:02] >> I think you're right about like
[42:04] probably a higher proportion of more
[42:06] senior ICs on, and like the density is
[42:08] higher on an ecosystem team than a
[42:10] product team because I think to some
[42:11] level product teams just have so much
[42:14] operational work day-to-day to keep the
[42:16] product moving, you can have a lot more
[42:18] junior data scientists to support
[42:20] versus an ecosystem team. I think
[42:21] that's one of the factors for sure. But
[42:23] when it comes to ecosystems, I
[42:25] generally do prefer people who have
[42:27] been at the company, like internal
[42:29] transfers, versus new hires. I think
[42:31] the biggest thing for that, especially
[42:33] with a company like Facebook, is
[42:35] because there's so much context and
[42:36] complexity about the app, that
[42:38] onboarding to all of that, like
[42:40] onboarding just to a product team at
[42:42] Facebook is hard, because you're like,
[42:44] "I have to learn everything about
[42:46] Marketplace right away." But now, if I
[42:48] tell you to learn everything about
[42:49] everything before you have, like, it is
[42:51] very overwhelming. So, I would say,
[42:53] like, the learning curve is much higher
[42:56] . But that's why I would probably, like
[42:58] , ease expectations a lot more, like, I
[43:00] would give them a longer runway on an
[43:02] ecosystem team than I would on a
[43:04] product team, before I feel like they
[43:06] will have a strong grasp of being able
[43:08] to have impact, and so excellent
[43:09] framing here is actually an example of
[43:11] what I'm about to say, which is I think
[43:14] the most important role on ecosystems
[43:16] are actually the ecosystem managers,
[43:17] and I think basically an ecosystem
[43:19] leader is doing double duty, right?
[43:21] They are both data science managers,
[43:23] but they are also I think product
[43:25] managers of a team of data scientists.
[43:27] And so very often like you can have a
[43:29] team of L3s, L4s, and there's no way
[43:31] they're able to go and like find the
[43:33] work by themselves of course. Um, but I
[43:35] think it is that manager's job to then
[43:37] find and source projects for them that
[43:39] are either ideas or what they hear from
[43:41] their discussions with other teams, you
[43:43] know, bottom-up as we discussed, or
[43:45] from leadership meetings, and they
[43:47] effectively source and create those
[43:49] roadmaps. And it's hard as well that we
[43:51] only have one manager on the team. Um,
[43:53] it's important to hold an extremely
[43:55] high bar for manager hires for the team
[43:57] . I was like more than happy to hire
[43:59] folks externally like at a junior level
[44:01] . Um, those are actually like I think
[44:03] eventually became some of the strongest
[44:05] members of the team over time. Um, but
[44:07] for hiring a manager, the bar helped
[44:09] was like astronomical. Like I turned
[44:11] down so many very talented people who
[44:14] just weren't at the 99%.
[44:15] >> That's true. Building and maintaining
[44:18] the buy-in is so hard, but so necessary
[44:20] . The manager is kind of the
[44:22] representation of that, right? Like, if
[44:24] the manager does not have trust from
[44:26] the leadership, then the entire team is
[44:28] worse. I just thought of this question,
[44:31] but I, I'm amazed that I didn't have to
[44:33] prepare for this question: Is there an
[44:35] ecosystem PM, or ecosystem engineer, or
[44:37] ecosystem designer?
[44:39] >> No, well, the way I describe it a lot
[44:41] is the closest thing we have to an
[44:42] ecosystem PM is the leadership team,
[44:44] like that you are. Just like for most
[44:46] product DS, we are a support function.
[44:48] At the end of the day, you are the
[44:50] support function for the PMs and the
[44:52] engineering team. The ecosystem team,
[44:54] to some level, is a DS support function
[44:56] to the leadership team. Leadership
[44:58] questions and any data questions they
[45:00] have end up coming to the ecosystem
[45:01] team, and that's the ones that we end
[45:03] up operating on. But there's not really
[45:05] a form of, like, maybe some companies
[45:07] have a model for this, but at least on
[45:09] ...We don't have like product. We have
[45:13] a......with us because there is, I
[45:17] think, a lot of need for some of the
[45:19] problems we deal with to have, like,
[45:21] user research, because sometimes we're
[45:23] working in spaces that have very little
[45:25] data coverage, and we kind of need a
[45:27] lot of external and internal research
[45:29] coverage, but not too many of the other
[45:31] functions.
[45:31] >> Yeah, except in very rare cases, and I
[45:33] think we had one of those at Facebook
[45:35] for a little while. I think you almost
[45:38] can't have another team. I think an
[45:40] ecosystem team has to be a data team,
[45:42] and I don't mean just science,
[45:43] engineering, UXR, insights, whatever
[45:46] you want to call that group of people,
[45:48] because the neutrality and the ability
[45:50] of that team to focus on fundamentals
[45:52] rather than incentives or agendas or
[45:54] whatever is so, so important. Like,
[45:56] that team must be unbiased, and I think
[45:59] data is really the only way you can be
[46:01] unbiased. Really, I think as a product
[46:03] leader, you are biased in one way or
[46:05] another if you're kind of using your
[46:07] design intuition or your opinions or
[46:09] things like that. I think there's often
[46:12] a lot of places for opinions in
[46:14] ecosystem work, but when presenting the
[46:16] data, and you know I've been burned by
[46:18] this a couple of times, especially, I
[46:20] think it is important to kind of let
[46:22] your opinions take a back seat to the
[46:24] data; it's hard, I think, for functions
[46:26] that aren't insight functions to do
[46:28] that.
[46:29] >> Makes sense. So, speaking of data, what
[46:32] are your most effective tools? For
[46:35] example, as a product DS, I feel like
[46:37] my most effective tools are charts and
[46:40] AB testing, and for, like, economists,
[46:43] mostly Python machine learning models.
[46:46] What about core ecosystem DS? Other
[46:50] teams, AB testing was one of the most
[46:52] effective tools. Yeah, I'm joking, but
[46:54] honestly, I think doing my analysis of
[46:56] experiments, especially creating the
[46:57] right metrics on the ecosystems at
[46:59] Facebook, we spent so much time
[47:01] creating the right metrics to measure
[47:03] the health of the Facebook app. It's
[47:04] like we obsessed over this. Um, much
[47:06] easier, I think we just use like
[47:08] transactional value or profit. So, you
[47:10] know, you make an order, you make an
[47:12] order. Um, at Facebook, getting someone
[47:14] to show you how valuable they think
[47:16] your app is, is actually very difficult
[47:18] to measure in an agnostic way. Um, and
[47:20] so we spent so much time on that, but
[47:22] then when teams run experiments, you
[47:23] can then kind of go into those
[47:25] experiments and look at what they've
[47:27] been able to accomplish on the metrics
[47:29] that you think are important. Apart
[47:30] from that, I mean, I don't think
[47:32] there's like a huge difference, like
[47:34] Cursor and Cloud, DBT and like a lot of
[47:36] these other things as well.
[47:37] >> Yeah, I think you had it right. Like I
[47:40] don't think there's high complexity and
[47:41] like visualizations or stuff that I
[47:43] kind of lean into. I think a simple
[47:45] line chart goes really far, but I think
[47:47] what really matters is what's on the
[47:49] y-axis, what's the metric you're
[47:51] looking at. And I think we get very
[47:53] specific on what we're trying to
[47:54] measure and very clear on what it can
[47:56] and cannot actually say about the
[47:58] product and user experience. And then,
[48:00] the simplest forms of visualizations
[48:02] for me have been the most successful. I
[48:04] also will say personally I'm biased
[48:06] towards a waterfall chart. I very often
[48:08] have to explain why something is going
[48:10] down, why something is going up, so the
[48:12] more I can break it out into a
[48:13] waterfall, it looks very...
[48:15] >> Love a good waterfall. I have a
[48:19] love-hate relationship with the
[48:21] waterfall, but yeah, I recognize how
[48:23] it's very important in framing the
[48:25] discussion.
[48:25] >> At Instagram, seeing a backward-looking
[48:28] waterfall that explains something—
[48:30] sure. We often have forward-looking
[48:32] waterfalls that project what will
[48:34] happen, um, and we call these bridges,
[48:36] and they are almost always wrong. And I
[48:38] have...Love a bridge. Me too.
[48:45] Relationship-wise, I already have
[48:47] concerns with like the accuracy of our
[48:50] forecasts at times, as line charts,
[48:52] what they try to say about like the
[48:54] future. And if someone asked me to take
[48:57] that forecasted line chart and break it
[48:59] into a waterfall of very confident
[49:01] predictions of all the individual
[49:02] drivers, I would just give up. I would
[49:05] stop, quit my job. We need to do a
[49:07] second podcast episode on this topic
[49:09] for sure. Predictions. It requires a
[49:12] different episode, another episode,
[49:14] other things that data scientists are
[49:16] asked to do but actually do not make
[49:18] sense fundamentally; I think prediction
[49:20] is one of them.
[49:22] >> I think they make sense, to be fair,
[49:24] and not to go off on a tangent. I do
[49:26] think they make sense. I think,
[49:27] especially on ecosystems, you do a lot
[49:30] of it. The most important thing is to
[49:32] understand what is the level of
[49:33] precision you need for the conversation
[49:35] you're trying to have. And in many
[49:37] situations, it's not like you need very
[49:39] tight error bars and a very tight
[49:41] confidence interval in the prediction
[49:42] you're trying to make. It's your
[49:45] leadership trying to understand what is
[49:47] the magnitude and the ballpark that
[49:49] we're trying to talk about, like how
[49:50] big of an opportunity is this, is this
[49:52] a 1%headroom, or is this an angle
[49:54] change to our trends? And I think when
[49:57] you have that context and you're aware,
[49:59] and that's the type of intuition you
[50:01] have to build as an ecosystem data
[50:02] scientist, then you know how much
[50:04] effort to put into your forecast
[50:05] because I think a lot of it is really
[50:07] just like 80/20 answers and we just
[50:09] want to know what is the magnitude that
[50:11] we're looking at.
[50:13] >> Yeah. Yeah. I mean sizing, I think,
[50:15] makes sense. I think it's important to
[50:17] kind of quantify people's assumptions
[50:19] prior and understand the relationship.
[50:21] So actually using Excel to build a
[50:23] model, like put different assumptions,
[50:25] get the output, I think that actually
[50:27] makes sense. But when we try to be
[50:29] scientific about it and we try to use
[50:32] machine learning models to make
[50:33] predictions, that practice to me is
[50:35] like: it doesn't give you useful
[50:37] information, and when you are around,
[50:39] like, the more confidence you are, the
[50:41] more burned you are going to be.
