本次模拟面试的候选人是拥有三年工作经验的刘瑞。他提到自己本科学的是 IT，后来转行写代码，平时没有系统学习过算法与数据结构，基本都是在实际工作中逐渐积累的。他此前看了公开课受到启发，希望通过这次模拟面试让主持人 Scott 帮忙找出深层次的问题。刘瑞透露自己在年初（二三月份左右）面试 Uber 时遇到系统设计题表现很不理想，当时稀里糊涂设计了一个“奇奇怪怪”的架构；今天他特意带来了自己之前做的一份类似 Instagram 的设计方案，希望能先接受 Scott 的检验与指点。
Scott 笑着回应称大家直接叫他 Scott 就好，叫“老师”听着不习惯，因为自己“平时比较不正经，很难蹭得上老师的称号”。刘瑞开启共享屏幕展示他的设计方案：他利用 Redis 支持 Linked List（链表）这一数据结构的特性来组织推文。在他的设想中，当用户 A 发布内容（Post）时，系统就用一个 Linked List 依序存入；例如先发了 Post 2（昨天），再发了 Post 1（今天），新发内容直接追加到链表头部或尾部，天然满足按时间逆序排列的要求。当用户 B 想拉取关注者的动态流时，系统只需对多个用户的链表执行一次 Merge K-Sorted Arrays（合并 K 个有序数组），取前 10 条数据直接返回给客户端。
subgraph WrongWay["错误设计路线：外形匹配"]
A["需求：按时间排序的列表"] --> B["看API外形：Redis List 支持链表操作"]
B --> C["脆弱系统：单线程写入瓶颈、内存易失性导致丢数据"]
end
subgraph CorrectWay["正确设计路线：质地匹配"]
D["承重需求：海量数据写入、高可用性、持久化"] --> E["选材质：Cassandra / DynamoDB（金属承重轴）"]
E --> F["Partition Key = UserID<br/>Sort Key = Timestamp"]
F --> G["高吞吐写入与自动按时间排序"]
end
```
听完刘瑞的分享，Scott 直截了当地指出该设计在根本思路上是错误的，并用“搭积木”的经典隐喻进行了深入剖析：系统设计确实像搭积木，积木不仅有形状，更有其内在的“质地”——有些是金属或不锈钢的，有些则是塑料、纸糊甚至是玻璃做的。积木的“形状”相当于系统的 API，而“质地”则是其底层存储引擎、数据持久化与吞吐能力等核心机制。设计系统时，最致命的错误就是“看 API 的形状来做架构”，看到某个系统提供了看似方便友好的 API，就把它放在了承重轴的位置上，结果就是“拿塑料球当承重柱，墙体直接垮掉”。
Scott 指出，Redis 的 List API 虽然表面上契合了按时间追加帖子的形状需求，但用在这里是严重的误判：首先是持久化与可靠性问题，用户刚发了帖子，一旦内存实例故障数据就丢了；其次，Redis 的写入是单线程串行处理的（Serialized Write），针对大吞吐量的发帖场景，其 Write Throughput（写入吞吐量）非常糟糕。工业界（如 Facebook/Instagram）处理这类多帖子用例的标准解法是使用金属质地、专为写入优化的分布式数据库（如 Cassandra 或 DynamoDB），以 `UserID` 为 Partition Key，以 `Post Timestamp` 为 Sort Key，天然且完美地在底层实现按时间排序的高吞吐写入。
此外，Scott 还纠正了一个关于性能瓶颈的常见误区：很多工程师热衷于在内存中优化 List 合并等 CPU 计算逻辑，但实际上，哪怕是复杂度为 $O(N^2)$ 的动态规划算法，现代 CPU 每秒也能轻松完成 $10^9$ 次运算。在线事务处理（OLTP）系统中，最慢、最耗时的部分永远是磁盘 I/O、索引查找和数据库读取，这部分往往占用了整个 API 请求耗时的 90% 以上。花大把精力去省去几毫秒内就能完成的列表排序，优化的只是不到 1% 甚至不到千分之一的非关键路径（Non-critical Path）。
- “积木理论”：API 只是积木的形状，存储引擎的 Durability、Reliability 和并发模型才是积木的材质；绝不能拿塑料做承重梁。
- 真实的性能瓶颈：数据库 Look-up 和 Disk I/O 占请求总时延的 90% 以上，纯内存数据结构的归并计算耗时往往不足 1%。
- Python 与 Go 的性能比照：曾有系统 Profile 显示，在某些极端场景下高开销脚本语言（如 Python）执行时间占总耗时高达 30%，改写为 Go 后语言本身的计算开销迅速降至 1% 以下。
1. **Background（背景经历）**：候选人过去做过什么规模的项目，是否具备独立设计架构的实践经验。
2. **Articulation（表达与切入方式）**：如何陈述问题、拆解逻辑链路，思路是从 A 顺理成章推导到 B 再到 C，还是跳跃混乱；若切入逻辑反常或违背工程常识，面试往往直接被否决。
3. **Knowledge Accumulation（知识积累与深度）**：对底层分布式系统、数据库原理、并发模型的理解扎实度，能否敏锐预判系统在大规模场景下的单点瓶颈与失效模式。
4. **Project Experience（实际工程落地考量）**：在系统取舍（Trade-off）时所展现的技术成熟度。
- 异步写扩散（Fan-out-on-write）机制：在异步任务中将推文推送到关注者的 Timeline 缓存（如 Redis），用高频异步写置换快速的常数级读，是 Feed 流系统的常见模式。
- 系统设计面试的评价重心：绝非死记硬背某种框架，而是在于结构化的表达能力（How to articulate the problem）和合理的推演路径。
---
[25:00](https://www.youtube.com/watch?v=NFZs5fu4bSk&t=1500s) 正式开考：设计 Google Calendar 需求梳理
闲聊与预热答疑告一段落，模拟面试正式拉开帷幕。Scott 切换至英文环境，抛出了当天的核心设计考题：“Design Google Calendar（设计谷歌日历）”。Scott 特别提示，日历系统的核心概念包含定期重复的周期性日程（Set of frequency）以及事件调度（Schedule the event）。
刘瑞作为候选人开始梳理系统功能需求。由于对系统级要求的边界尚不够完全熟悉，他一边思考一边尝试建立用例清单。他首先指出了两个最直观的基础交互流：其一是用户作为发起人创建日程，并将事件发送给参与者（Send an event to the participants who will join this event）；其二是参与者作为接收端客户端（Recipient client），能够准确接收并同步该事件。围绕事件的创建、派发与同步，面试正式进入到核心功能需求的 Clarification（需求澄清）阶段。
面试官与候选人（Rui）继续明确 Google Calendar 的需求边界。在基础功能方面，日历支持跨设备同步（例如用户从一台电脑同步到另一台电脑），以及查看好友或同事的可见日程事件（visible events）。除基础日程同步外，另一项核心功能是**为日程添加会议室（Add Room）**，前提是该会议室在目标时间段内处于空闲状态（available）。
```mermaid
flowchart TD
accTitle: Meeting Room Booking Conflict Scenario
accDescr: Shows User A and User B attempting to book the same room simultaneously, causing a race condition without concurrency control.
subgraph Users ["并发请求"]
UA["用户 A (User A)"]
UB["用户 B (User B)"]
end
subgraph Service ["日历与会议室服务"]
Check["查询会议室状态 (Check Room 1: Free)"]
Lock{"并发一致性控制<br/>(Lock / Isolation)"}
Book["预订成功 (Book Room 1)"]
end
UA -->|"1. 检查可用性"| Check
UB -->|"2. 检查可用性"| Check
UA -->|"3. 提交预订"| Lock
UB -->|"4. 提交预订"| Lock
Lock -->|"成功预订"| Book
Lock -.->|"冲突拒绝 (409 Conflict)"| UB
```
```mermaid
flowchart LR
accTitle: Solution 1 vs Solution 2 Lock Granularity
accDescr: Contrasts coarse-grained queue locking with fine-grained in-memory set locking for room reservations.
面试官指出在写入流程中如果出现从 Web 服务器到数据库或缓存的长暂停（Long Pause），会导致事务迟迟无法提交（Commit）。候选人对此提出了超时淘汰策略：如果整个处理流程超过 1 秒，系统就直接将当前请求踢出并触发重试。面试官随即敏锐地指出，如果卡在 1 秒超时，系统的最大吞吐量就会受到严重制约，并追问当前的加锁粒度到底是锁了什么。候选人解释道，之前的方案一相当于锁住了整个数据库，而修改后的方案二则是把锁的粒度细化——每个房间（Room）分配一把独立的锁，每个房间维护属于自己的内存队列（In-memory Queue）。
面试官梳理后认为，方案二在本质上与方案一并无二致，核心逻辑完全一致，只是将并发控制的粒度从“整个数据库级别”缩减到了“单个房间级别”（reduce the granularity to per room）。从房间的视角来看，每个房间都挂着一个专属的内存队列，系统借助这个队列来严格判定谁先来、谁后到。候选人随后提出，也可以通过记录请求到达时打上的时间戳来确定先后顺序。
- **方案一与方案二对比**：方案一是在整个数据库层面排队加锁；方案二则将粒度细化为单房间，每个 Room ID 拥有独立的锁和队列。
- **超时机制**：单次处理如果超过 1 秒阈值即被判为超时并踢出（kick out），释放资源供后续请求处理。
[53:15](https://www.youtube.com/watch?v=NFZs5fu4bSk&t=3195s) 时钟同步陷阱与按 Room ID 固定分区路由
```mermaid
flowchart LR
accTitle: Room ID Partition Routing
accDescr: Requests for Room A are routed through a partition proxy to a single dedicated server hosting Room A in-memory queue.
subgraph Routing["网关与路由层"]
GW["Consistent Hashing / Proxy<br/>按 Room ID 路由"]
end
subgraph Worker["Server 1 (负责 Room A)"]
Q["In-Memory Queue<br/>(Room A 专属队列)"]
Lock["Room A Lock"]
Q --> Lock
end
R1 --> GW
R2 --> GW
GW -->|"固定路由"| Worker
```
当候选人提出“服务器在收到请求的数据包上打上时间戳来排序”时，面试官立即提出了一个经典的分布式系统挑战：“谁来提供这个绝对时间？”面试官设想了一个场景：有两个预订房间 A 的并发请求（Request 1 和 Request 2），在分布式环境下几乎不可能保证这两个请求恰好打在同一台服务器上。如果 Request 1 去了 Server 1，Request 2 去了 Server 2，而不同机器之间常常存在严重的“时钟漂移”（Clock Drift，极端情况下甚至能慢两个小时），这就意味着依赖本地时钟排序会导致某些机器上的请求永远排在后面、永远抢不到房间。
面对时钟漂移的诘问，候选人调整了架构策略：预订同一个房间的请求，必须强制路由到同一台特定的服务器上。面试官顺着这个思路进一步追问容灾与就近接入问题：如果为了保证同一个 Room 总是落在同一台机器，那么无论用户是从中国发起的跨国出差预订，还是在美国本地发起的预订，都无法采用就近接入（Nearest Server）策略，而是必须将所有请求静态哈希（Partition based on Room ID）到同一台固定的服务器上。一旦这台 Server 1 宕机崩溃，其内存中的队列状态将全部丢失，此时只能通过迁移机制把路由切换到备用节点，并要求客户端重新发起重试。
面试官开始帮助候选人把整套方案的写入链路从头到尾清晰梳理出来：首先客户端发起预订请求；网关层根据 Room ID 将请求分区路由（Routing to server partition by Room）；对应的服务器接收请求后，将其压入该房间专属的内存队列（In-memory Queue）；当轮到该请求处理时，再去锁定数据库中包含该 Room ID 数据的记录并执行写入。
面试官 Scott 针对候选人在应用层维护锁的方案进行了深入的正确性（Correctness）推敲。Scott 指出，从性能上看，候选人试图通过外部系统给会议室加锁（Lock on Room），其性能甚至比传统的两阶段锁（Two-Phase Locking, 2PL）还要糟糕。更严重的问题在于系统的正确性保证。Scott 现场构造了一个极具说服力的边缘案例（Edge Case）：假设系统有两台应用服务器，即主节点 Server 1 和备用节点 Server 1'（Backup Server）。此时，第一个预订会议室 Room 1 的请求 $i$ 到达 Server 1，Server 1 在本地加上了锁，并将请求转发到数据库开启事务执行“先读后写”（Read then Write）。
就在数据库正在执行该事务、尚未提交（Commit）的窗口期内，Server 1 突然发生宕机（Crash）。集群的故障转移（Failover）极其迅速，可能仅耗时 1 毫秒，流量随即切换到备份节点 Server 1'。紧接着，第二个预订同一房间的请求 $j$ 到达。由于 Server 1' 是全新接管的节点，本地根本没有请求 $i$ 的锁状态信息，因此请求 $j$ 也被顺利放行并送入数据库开启事务。如果此时底层数据库因为写入性能抖动或网络延迟花了数十毫秒，两个并发事务就会在数据库层产生重叠与冲突，从而导致灾难性的重复预订（Double Booking）。面对候选人的困惑，Scott 梳理了整个时间线，强调只要节点存在故障切换且锁状态没有强一致性保障，应用层锁就必然会出现漏洞。
```mermaid
flowchart TD
accTitle: Failover Race Condition Sequence
accDescr: Sequence diagram showing how application server crash leads to double booking in database.
subgraph Clients["客户端请求"]
R1["请求 i (Book Room 1)"]
R2["请求 j (Book Room 1)"]
end
subgraph AppServers["应用服务层"]
S1["Server 1 (主)<br/>加本地锁并转发"]
S1Crash["Server 1 宕机 💥<br/>丢失锁上下文"]
S1Prime["Server 1' (备)<br/>无锁信息，直接放行"]
end
subgraph Storage["数据库事务层"]
DB1["事务 i：Read then Write<br/>(耗时数十毫秒)"]
DB2["事务 j：并发执行<br/>(产生数据冲突！)"]
end
由此引申出核心架构原则：为什么绝不能在应用服务器搞复杂的外部有状态锁？应用层（App Server）必须保持无状态（Stateless），以便随时根据流量水平伸缩或随时销毁替换；而真正的并发冲突控制，必须沉降到具备强一致性保障的底层数据库中。数据库内部本身就经过了高度优化，支持索引锁（Lock on Index）、行级锁等成熟机制。如果为了避免并发而自己造轮子在内存或外部弄一套状态同步，一旦遇到网络分区或节点宕机，反而会引发难以收拾的系统漏洞。
## Mentioned
Scott Shi, 刘瑞 (Rui), Uber, Instagram, Google Calendar, LinkedIn, Outlook, Redis, Cassandra, DynamoDB, Elasticsearch, Kafka, MySQL, MongoDB, AWS ElastiCache, Go, Python
## Video description
Mock interview of design google calendar
doc could be found here: https://docs.google.com/document/d/1IdEKaXNoCwUaSx1lZtBD5kbPkERjUbc0mUi8sLt3g30/edit
[0:00] 我们今天interview candidates是刘瑞是吧?然后你要不要先简单介绍一下自己好,简单介绍一下自己就是有三年工作经验, 其实本身是学IT的, 转到写码, 没怎么学过算法跟数据结构, 还是在工作期间慢慢学的。听Scout的课挺有感悟的吧, 上次觉得那个志愿者真的是表现挺好的, 希望这次Scout能多多找出我的问题, 多给我提意见, 我在今年,1月份还是三月份吧可能就二三月份左右面了一回Uber然后我当时系统设计觉得特别不好所以稀里糊涂的给人家设计了一个奇迹怪怪的系统然后今天打算带着这个系统来也想让那个Scott老师帮我指点一下就是我知道我这个设计的不好但是我觉得思路挺奇特的这个对叫Scott就好我之前可能说过在群里面说过一下就是叫Scott好叫老师或者是什么我都不习惯叫Scott就好好,我很难蹭得上老师,我这个人比较不正经,所以经常很难蹭得上老师。然后我们打开摄像头,然后讨论开始讨论好了。然后我记得你跟我聊天,我看你昨天给我发了一个东西,但我没空看。你还跟我说你就是设计了个Instagram,想分享一下,然后你现在来讨论一下。是稍等我这个摄像头好像卡住了好了,应该能看见我了。您想先说哪个呢?还是?先说你的好了。先说我的,好的。先说你的。你能给我个权限,我share一下屏幕吗?好像应该有。好的有这个是我设计的一个系统,是一个Telegram,然后这些数据当然都是瞎编的了,然后我当时拿到这个题目的时候,我想的就是因为RIDIS是可以存LinkedList的,所以我基于这个原理设计了这个系统。比如说如果UserA他post了一些东西的话, 我就用一个linked list把它存进去。比如说他的post1,然后我们copy一下。谢谢大家他的Post 2, 我觉得这样的好处就是,他post的2,这样做的好处就是,他恰好能够根据时间顺序排列。也就是说,2是他比如说昨天po出来的,然后1是他今天po出来的。那么就是再泡出来就往如果它破出来新的,那么就往这个位置加就好了。基于这样一个原理,而且我觉得这样一个原理,当userb想get所有的post的时候,那么只需要那个 该只需要 more 指 k-thoulted array谢谢大家比如说我想要前10个,那么我只需要more case out of the array,more to the 10th,然后return给candidate端。就好了。这块我想说几件事啊我觉得我这块说可能说的稍微不客气一点就是这个设计我觉得是不对的啊然后我周末的时候刚跟那个instagram一个同事啊我一个同学我之前研究生同学他他在facebook工作五年了然后他上周末我们一起去hacking然后他说他走到instagram的这个Instagram Infra-群了,然后他们maintain这个就是整个世界上最大的Cassandra他们组织有三个人,然后门顿市场最大的Cassandra Server有4000多个 nodes,然后就讲了讲他们怎么用RockDB搞的这块有几个很明显的问题啊就是我想提一下说的不客气一点不好意思啊说直观一点没关系您尽管说对就是首先最大的一个问题就是说设计系统设计的时候不能看API来设计就是你永远不能看API来设计所有的API看起来都很friendly然后看起来都很人畜无害但是你看API设计是最有问题的一个事情我之前的时候
[5:00] 早期的时候讲系统设计课的时候,最早期的时候我经常说是,你讲系统设计是,你就是搭积木的一个过程,然后你积木有不同的形状,有不同的质地,有不同的质地,然后积木的形状是API,就是它有圆形的API,它有方形的API,然后积木的质地它可能是木头的,可能是金属的,是吧,可能是不锈钢的。有可能是塑料的这个是它的质地,你不能说这个形状看起来适合,然后本来应该放了一个层纵轴,然后就放了一个玻璃的上去,玻璃的形状。玻璃质地的东西放进去,这个是明显不对的。也就是说,你系统的设计永远不能用看著API来设计应该用什么,不能说这个 system,这个 API 看起来 work,然后我就把这个 system 放过去,这个是绝对不对的,这是系统设计中,基本上一上来就被 毙掉的这个东西,就一上来被 毙掉的。这个什么呢可以接受的是什么呢?就是可以接受说这块车从纵走,那我放一个不锈钢的那我放个不锈光,但形状不行,那我把形状稍微tick一下然后这个是可以的,但你不能说这个是个承重轴,我放个形状像是可以的,但是这个它是个纸做的或者是个塑料做的,那这个是完全不行的。你懂我意思吧,就是它这个database的质地,它Database的它背后的这个怎么设计的它这个质地是远远大于它API的API 是它的形状,可能这个Database没有你想要的API无所谓 但它质地如果是金属的你把它稍微Polish一下稍微修改一下 是吧然后你还是可以把它放进去做乘重轴的但你不能说这个是个圆形的我需要一个圆形的然后看这个塑料球可以我把它放过去然后这个墙直径就垮掉了然后这块的话我觉得属于墙会垮掉的一种情况,为什么呢?你这个user post data 的时候,你刚 post 完,然后发现 data 丢掉了好然后就是对吧,Post完以后对他丢掉了。然后其次的话,我们周四的时候会讲Redis的RightThroughput是很糟糕的。Redis 的 write 是很糟糕的,它是 serialized 的 write,然后它讲究的是非常小的 write,它的这个 write skill 是不好,它 read 是比较好,它是 focus on read 的,然后然后这个情况下的话,所以这个用Redis,我觉得不是一个很好的选择,我觉得不是很好的选择。然后最后的最后这个这个这个Use Case就是你一个User post的多个Post这个Use Case呢是用Cassandra或Dynamo就可以完美实现的Facebook用的是Cassandra完美实践的,你把partition可以变成userID,然后这个sot可以变成posedtimestamp,这个是可以完美实践的。所以这个是所以这个是不太合适的设计然后我再提一下这块设计中常犯的一个错误是我觉得这块有一个list 是吧我读个list 放到list更方便但是在这整个系统执行中这个时间是不一致的你如果从,我这么说一下好了,就是如果你做一个普通的算法来做,就是什么意思?普通算法呢?就比如说,你做一个,我举一个算的比较慢的吧,我想想什么算的比较慢一点。DP吧,Dynamic Programming,N-square的算法复杂度,N-square算法复杂度它没面也能算十的九次方,机器可以很轻松的算十的九次方实际10的9字方是什么概念?就是你把所有的user全部scan一遍,也就是10的9字方。假设你有user data,你把user全部进行一个operation,全部把user id 加起来,你把你所有的user id 加起来也就是10的9字方。这个算的是很快很快的,就是计算这方面是很快很快很快的。最慢的是什么?当你读database的时候最慢的,就是你所有的transaction系统,最慢的部分永远是这个data lookup和从database里面读取,就是说白了是你来一个request,然后我看根据你的request找件我要找的一个index,然后就是disk嘛,如果不在memory的话,很多时候在disk上,首先找见disk在哪里,然后把disk的poyote到那个地方,然后从上面读取data,这个部分是最慢的。这个部分经常是几十几毫秒,就是占了整个API request的90%以上。你在优化的计算这部分,在的是可能不到1%很多时候是不到1%的你知道吧,就是这个计算,把这个list全部读出来,然后把它12345这样list标好,这个salt计划是不到1%的,甚至比1/1000都要少我觉得我之前只读到一篇文章,他是做高性能应算的,他说这个,他当时是用的当时用的Node.js还是Python来着,他说可能是Python,他说Python慢的几乎和database一样慢了。
[10:00] Python居然占了database总时间的30%database占70%Python 占了 30%,然后把 Python 转换成 Go 以后,这个 language 占的时间就有就变成基本上1%以下了所以这个majority时间,如果你把那个做一下System Profile的话,majority时间,基本上都在做这个database read和read,所以这个你优化的这个是1%的时间啊,这个其实是没有必要的所以是两个问题,第一个是优化的不是最主要的,System中不是最主要的Critical Path,其次的话不能看这个积木的形状来决定它用不用它,必须看积木的质体,必须看要的这个Service,这个Mojo的质体,不能看它的形状来,不能看它的API来进行设计。好的明白了,这个是彻底明白了。从设计的最开始的点就是错的。对,我觉得用Redis一般来说,Redis 设计,还有之前有人问我为什么不能拿 Elasticsearch 作为 main database 是吧?就是直接把 Elasticsearch 我记得上周的时候有一个人问我说为什么不能把 Elasticsearch 作为 main database因为这个Elexig Search它设计的时候,它Durability和这个Reliability就不是作为Media-Base设计的Elasticsearch一般来说,你都有个其他的SourceWord Truth Database,你不管是Seql,Mongo,Cassandra都可以,你把这个有个SourceWord Truth Database,然后用Elasticsearch呢,是提供它更多像一个index,你知道吗?Elexizer,我直观感觉更多是index,它设计的时候也这么设计的然后他说更多像一个index就是你用Elexic Search帮你作用辅助的search,但是你不能把Elexic Search作为major storage,这个本身就是很有问题的。然后你会看到很多文章,database的时候,就是你database这块会看到很多文章就是说,Electric Search, Reliability也提高了,也可以用作什么经常是一些Sales的文章,然后我觉得最常见的就是这个我去参加Kafka Conference的时候整个 Kafka 这个 team 不厌其烦的说我们要把 Kafka 转化成一个 database 是吧?但是我觉得谁会把 Kafka 做 database 做得好运 是吧?你会看到很多cells的文章Sales文章他会尝试把它推销成A推销成B,B推销成C,但是这个都是没有任何参考性的。我觉得这个对好的,那确实明白了。好的,那我们来看一下我们出的那道题好了。我们今天就不讨论,不问这个Challenge Project了,我觉得之前的话也问过几次。我们就看一下那个题好了。你可以点一下Google Doc在这个Chat里面。好哦,ok稍等然后我刚才说的话大家有什么额外的想补充的吗谢谢大家谢谢大家您好,Scott,有一个问题,您刚才说Redis它的读的性能高,但不是用来写的,如果我们需要写的话,有一个比较好的代替品是什么?写的话一般,也不是说完全不用来写的,就是我们下周讲Transaction的时候,你会看到Redis的话它写是single side的写,所以它推荐的写是什么呢?它这个写是非常small data,small size data,然后你这个写的话是非常快的完成,然后你这个同时你这个写的话你的这个Redis,这个Rest throughput是有上限的,因为它是single side写的。但是一般来说,假设你问我的话说有一个database或者database-like的这个system一个基目什么是优化写的话那肯定是Dynamo和Cassandra,Dynamo和Cassandra是纯粹为写优化的好的,谢谢你。Denimal和Cassandra,看你怎么做configuration了,你可以把read和write的corum改的不一样点。但是我记得我之前看的那个效果的话,write是0.5毫秒。Rate是0.5毫秒以下,然后那个Rate是比Rate要稍微慢一点,大概就是一两毫秒,就是一毫秒左右。但是它那个Rate其实是比Rate要快几倍的,甚至快一个数量几倍的,所以它是专门对那个Rate进行优化的。你可以看到更多benchmark我看的benchmark比较早了可能是15年还是16年的benchmark好的,谢谢Scott,我想问一下那要是我就是我会把一些
[15:00] 有一点点复杂的object写到Reddit上面去做serialize和deserialize,那这个cost会很高吗?没有,这就是in-memory database 的优点比Memory Database和传统的SQL比,虽然它是从Memory读肯定会快的但是它其实最大的快的地方就是你memory database 你不需要把它拆分成不同的module 然后存到这个这个 SQL 里面你直接就存在一个 list 存在一个什么的就是你可以把它就是你可以把你自己的 in-memory 的 data structure 直接 dump 到 in-memory database 里面这个 serialization 和 deserialization 可以省很多时间就是它这个很少时间的部分一方面是你不小心写disc我们之前讨论过就是写Memory的话有点像写到Memory有点像你从这个San Francisco到这个去吧 这样是麦的sacramento 但是你要写到disc上就感觉到土星那种感觉就这个distance感觉到土星这种感觉差的他的至少10倍可能100倍1000倍有可能但至少差10倍好的 那什么样的WriteSize是比较合适呢?Redis吗?对,对的,是Redis。你如果Redis size很大的话,你后面所有的Red都有一个overhead,你如果Ride size很大的话 顺便所有的Ride都有overhide 因为它一个一个鞋嘛嗯那我觉得Redis的话,其实很难说出来,这个可能还是要做一些Benchmark 以后才能看出来,但总体来说的话可能不适合就是写一个非常大的上级你可能得把他的Data Model再Break 细一点好的,谢谢。你不能上来写一个megabyte 肯定还是比较糟糕的,但是break 细一点的话,几十k,几k的话,我觉得应该问题不大。好的,那应该没有那么大,好,谢谢。对,一般来说,一个密碗拜次数比较大的。几十k,几十k的话,二十来k其实不是很大。就大概10个object左右吧。10个object,对,那不上你打,那可能就是个10k到100k。好的,谢谢。Scott,在AWS里面,这个是用elastic cache,然后那个是distributor的redis吗?这个话是不是也难道也是serialized的鞋吗?Electric cache谢谢大家Elastic cache can be used for Redis, there are two types, one is main cache and one is Redis. Oh, I wanted to use elastic search, I thought how about Redis? Elastic cache, because Redis is not the它是每一个redis 你可以认为一个partition是有一个single write thread吗但是你可以partition了,比如说你现在Aless的话,比如说ID的话,你可以把ID你可以把你要写的user这个space,然后partition到比如说一万个server,1000个server都可能是吧,然后每个server相当于你有一个writing thread这个skill起来是比较简单的每一个partition就只能写就三个thread的写但是你可以partition成比如说根据user partition这样子对对对每个partition是单个thread写的但是你可以partition很多份是吧你比如说你user id是假设举个例子从1到100万的话你比如说从1到1万可以到partition1,然后1万到2万到partition2,当然这个partition是错的,你可以参考我们之前的partition那一节但是这样的话,不是position了100个node吗?那你这样的话,其实是你有100个writing thread可是我理解的Redis它是有16000多个slot就说然后slot然后再去划分到register cluster里面的,我有点不太明白这个过程。因为有这样的slot?嗯因为我的理解就是说Redis是有固定的slot吧,是16000多个,对不对?然后比如说你有10个Redis的机器的话,然后它就把这16000多个slot给分成10份嘛,就是分成10个range,然后每个range放到一个机器上是这样的吗?才是我理解错了对这个的话可能跟你使用相关吗?你也可以只有一个Redis instance啊?你这个不是建的时候可以说?虽然只有一个Redis instance 但它还是有16000个slot在里面它好像是Redis internal的设计吧对吧?它就这么做的对这个我不知道这个我确实不知道啊如果你有rading的话你会有点问题。因为我一直很困惑就是说如果我送了16000多个slot,然后每个slot就是说比如说slot1到1000就送到第一个机器,1000到2000送到第二个机器上,然后这个是我存的比如说我存一个data,比如说我的data,
[20:00] 用户上的User ID来传的,那么你是说根据User ID来去划分到哪个机器上去?哦哦,我知道了,这个16000个slot可能是它partition的一种分的方式,partition有三种分的方式吗?一种是上来就先划分成固定的partition我估计这16000个partition就是它提前划好的partition但是这个partition可能在同一台机器上这个是Partition方式不是有三种吗?一般来说有三种,一种就是你上来就固定成一个具体的数量的Partition16000个的话比如说有3000个机器那你可能一个机器五个是吧4000个机器一个机器四个盘队上然后这是一种,然后其次的话还有一种partition划分方式和nodeproportional的,像Cassandra的话是partition是proportionaltonode来划分的还有一种partition对在有不同的partition化分方式我觉得这个partition下来的我知道我不是说这个Redis的因为我的印象中Redis是对Redis 是固定的 partition。对,这个固定的固定是帮你标定了这个horizontal scalability了。但是如果是这样的,比如说如果它的Redis固定了这个slot的话,然后我在想问的几个问题,一个问题就是说,比如说我拿Redis来cache我的user data,对不对?比如说每个User有UserID,就是UserID应该怎么跟这个slot做一个Mapping这样的关系?第二点就是说我拿到这16000个slot,我要怎么把它distribute到我的class里面?对这个就是说对啊这个的话你很多时候你需要一个routing layer 你可以看参考一下这个你可以参考一下,我这块没有部署过,我没有部署过1万多6千,我没有部署过这个这个Bredis的这个但是如果你就是想那个时候的话你可以看一下这个就是之前讲partition那一张就是很多时候它是你有个你需要有个routing layer然后这个routingLayer是决定几个的,一个是它其实就决定这几个事情一个是给一个ID然后他找见哪个partition是这个ID对应的哪个partition然后其次的话是给一个partition它找见这个partition在哪个node上就是那个node name是什么然后最后的话是给一个node name然后找见IP address最后这个是在那个就是DNS在解决的,domain name server解决的。但是前面这两个一般是有个routing layer,它可能需要有一个像有的那个比如说 rulekeeperI'd like keep routing them对啊,但这个始终还是个general的一个law,一个partition的一个布置,但是我就是想针对redis,因为我们在谈redis嘛,就想针对,因为redis多了这么一层的一个slot,没关系,如果我们也许可以恢后讨论,但是我觉得我还是不太清楚,对。因为我一直看得对如果Redis已经提供你partition的话我估计它也提供这个它可能它自己可能会wrap一层etcd或者是wrap一层roupaper它实际上就帮你做了这个routing他应该是帮你做肉丁,他如果做这个的话,他没有道理不帮你做肉丁。但我这话不熟啊,确实不懂啊,你可以搜一下,或者你发一下,我可以帮你再offline看一下。好的,我只是知道它有固定的Slot,但是怎么去分的我也不熟悉,对好的好的好我们还有其他问题我觉得有很多很index好了我觉得还挺有气的Scott我之前看到过一个design是instagram的design是这么做的他的主DB是用SQL做然后把存法跟刚刚说的是差不多的然后这样的话但是慢就会慢在比如说我pull newsfeed的时候他需要去join table根据我的follower然后去join对吧就是可能要join两个table或者三个table然后他解决这个问题的方法就是一步的搞一个就是一步的搞一个每次你比如说有新的post以后你就会一步的去Redis上面把每一个人应该看的东西写在那上面然后你每次看的时候其实只要查Redis就可以了就是看那个人有什么post要看的或者然后你follow的时候你也是更改Redis然后就把Redis当成cache这样一个层在这里,其实这样也是写的操作在我看来也是挺多的,只是它是异步的。对,我记得我们讨论slack的时候,我当时propose的slack就是你进去那个界面也是放到Redis上的,它这个data也是一个user,然后在channel也是放到Redis上。放Redis上本身是没有错的我觉得还是我们讲System Design的时候是讲这切入点我觉得刚才我们讨论这个问题的时候刘瑞同学这个切入点是不对的就是说我看到这API以后然后讨论到Redis
[25:00] 支入点是不对的啊对我在 chat 里面发了一下我刚收到的 Redix 怎么做 partition 的就是说对,它跟一般的 partition 做的有点不一样这个没有这个是标准partition做法之一这个就是固定数量的partition是这么做的它会根据你的ID然后把它hash过来这样的话其实你从data到partition这部分复杂度是基本上固定死了但是你从partition到node这块是要dynamic adjust的就是标准的这个fixed partition的做法是这么做的如果这块有点糊涂的话,或者其他东西没有context,或者有点糊涂的话,你可以看一下就是我几周前吧,两周前讲的partition那一张,三周前讲的partition那一张那块的话是讲了很多partition的做法和怎么做partition好,我还是说一下,我感觉好像也有很多看见很多新加入来的同学你个是这样子就是嗯讲System Design的时候,有很多讨论的方式,但是System Design主要考虑的是四个方面,我再强调一下讨论四个方面。第一个方面是看这个就是被面试同学的这个background是吧就是看他之前做过什么项目,独立设计过什么项目,然后做过什么项目,然后其次的话是看这个表达方式是吧,你就是怎么切入这个表达的,你是怎么样表达的,这个表达方式很重要那你如果是表达方式是从这个思路,就是从A到B到C的话,如果这个思路和这个就是常见的思路不一致的话,就是这个思路会有问题的话,多半会就是否决掉了。和表达的方式有关,就是articulate,你怎么样articulate这个problem。然后其次,再其次的话是看knowledge,这个knowledge积累怎么样是吧,你对于这个系统的理解怎么样,那你对这个系统可能出现的问题怎么样,这个是怎么样。四分之一很多的时候是之前的一些部分是很重要的一部分然后问怎么看之前内容在那个群公告里面,在那个群公告里面有,如果你不在那个群的话,你得找人拉一下,就是说,找别人拉一下,在那个群里面。好,我们还有什么额外问题吗?好像讨论时间有点久了,我们是不是可以开始Systems计算了?考量了二十分钟看有没有额外的问题,如果还有的话再回答最后一个,再讨论最后一个。谢谢大家谢谢大家谢谢大家谢谢大家好,我们开始好了。OK, so let's switch to English. So here I would like to ask you to design a Google Calendar. In this case,你也可以说什么是个订阅的订阅的订阅也可以说什么是个订阅的订阅to a set of frequencyand schedule the event嗯OK,I think I used Google Calendar before, but I'm not pretty familiar with your request.with the systemthe requirements.So now I list the requirements.So I just谢谢大家I think I count all the requirements嗯 嗯 嗯是正确的?我认为第一件事我们需要去做和去做的事Send an event to the number who will join this eventand for another kind of anotherClient, the guy will receive the event.I will say these two
[30:00] This two match the I had a guest and will my create a I think we need to create first. And then, so... So number three is download all in one. Meeks Such like I have two computers. I need a Yeah. from i need to download my calendar from one computer to another computer and sometimes we need to see my friend's event my friend's schedule so it's also download also with also with visible event? I will say it meets this request. Thank you. And we also need the. I the room. add room on our event if the room is available. So we will talk about the this request for this request after we finish the traffic. And so I think the system will be a reliable system. And the, at here we, Uh... we related, so we need to check the room available and the son book the room. So we need a consistent consistent system we don't want to two user who maybe clicks the book room together but we just have one room we cannot give this one room to the two guys at the same time Okay, that makes sense. You don't want to have two people go to the same meeting room with totally different purposes. Yes. Okay. So exact meat of your requirement. - Yeah, so probably I want to add availability there as well. You don't want the system to be not responding? Yes. And also I will separate two two part ways like feature requires the other is like a system request something like that. Okay, system requirement. Mm-hmm. Mmm. System... I Requirements. Thank you. Okay. So here is a few feature requirements. Add room, We just talked about "Wisability" Thank you. Okay, so can we ask how many users, how many active users for this system? Yeah, let's assume that it's Google style. So you probably have a A billion user. Let's say you have a billion user. Ability user. Thank you. um and how many times for one user create a event for one day yeah let's assume that you create uh uh one event a day One event Thank you. or They. So can we assume I will risk. I will resale three event per day. Yeah, makes sense. Interaction. Yes, even pretty. Thank you. Thank you. Next things, I will talk about the storage. Shhh. 1 billion users. Thank you. Sure. - One... - Is it cracked? Yeah. Yes. I think so. It's 10 to the power of 9. Sorry. 10 to the power of 9. That's 1 trillion. Oh, OK. Time for one event. And can I assume 200 KB for one event? I think it's too big. Probably around 10 kb is more likely. - Turn KB. - Sure.
[35:00] 200 kb is very big, yeah. Okay. Mm-hmm. Uh... 疼KB! Thank you. Thank you. This is for one year. So how many years we will... For beginning how many year let's assume that you want to stop five years data five years Mm. Thank you. All right. Thank you. Thank you. Thank you. P.G. Goodbye. I think. Yeah, so close to that, yes. Yeah. So what is next? I will say QPIs. About the QPIs, so... I think it's a 15 PB instead of 150 PB. It's a 15 PB. Faith. 15. Yeah. Yeah. So the QPS about the right. AIDS All right. Thank you. Thank you. Thank you. . Thank you. Hmm. Almost the Lexis. Thank you. Sure. Probably close to a thousand. Close to thousand. Yeah, it's probably close. Average is one thousand, one kQPS. Thank you. Thank you. Okay, 2K QPS and write. So read. Thank you. I will say it's Thank you. Thank you. Thank you. 2K is right, but not only you, not only you, I will say, read, it's not only from my from my co-worker sent to me. But sometimes I will such like double check it, I will I will check my friends calendar. So I will time for right here. Yeah, I think it's really the value probably really it's probably one level magnitude larger than right so probably 20 K or something more reasonable. Okay, 20K. Yeah, 20k per second. 20k per second. So here I just saw the conversation. Well, in your interview, probably your interviewer wants to mention anything. You need to go on this calculation yourself. But here just to speed up the process a little bit. And also we can do some assumption here. For example, if average QPS is 1K, then we can say peak QBIS is probably 2K to 5K. And if If a write is like 1K, rate here, like a typical rate here, could be like times 10 or something. So it's a user common sense, and then we can roughly estimate the number. Okay. Make sense. Thank you. So now I will talk about my solution. Mm-hmm. Thank you. Thank you. Thank you. Can I assume? I think I need to assume one more thing. So when people can draw two or three meeting. But sometimes the people just go to one. And sometimes maybe such like such like me, in my home, just open the meeting and do something other things. So can I assume one people can go to multiple meetings? Yeah, that's normal. So sometimes you need to go to multiple meetings. That's a pretty normal thing. Okay. And one room, one physical room. It's just for one meeting. Yeah, you cannot share a room. Yes. The solution. Thank you. Okay. Thank you. Hmm.
[40:00] So, - Uh, I will say the solution one, the request one, two, one and two and three is a normal request. So we just need a database to save all of the basic information. Right. Sounds good. Okay. And I think the hard point is the So we need to check the room, So is this rumor available? Um... Yeah. The hard point. The hard point I'd hear such like this is time. Thank you. Thank you. We have a user A and another user A. B. Mm. Thank you. A check room one. Thank you. A check room 1 at here and A book room 1. at here. So between a check and book, B It's maybe a by the guy. B check after A, but B book before A book room. So I will say in this point, this is a consistent problem. Where is my solution? My solution. Thank you. My solution will do... My solution will do write after read. So every time when we write something, we will add Uh... Read at here. So when we do read and the time to Right. Thank you. So how is the difference? So at here, there is pretty close. no so you cannot consume you can under some uh yeah yes so i will i will do something i will do something more at here just to explain the read the right after read um so in this case so i will add a In this case, I will add a database locker at here. So at here we will combine the read and write together. Just to make sure between the read and the right, Thank you. Between this time, only B can access database. Inserate and write only Only one user can access the database. Okay, so here, what's the log granularity? What you're going to log here? - Uh, The lock, we just need a variable. If we just need a variable, if it is true, it's locked. It means someone used the database. If it's false, it means the databases are it's open, I mean, we can access it. So you are saying, you're not using database default. Like, database provider log, you're not using that. You actually use another system to provide log. We can use the database function Okay, so... Mm-hmm. Yeah, so if you're a database log, what's the granularity you're going to log on? So what's the resource granularity you're competing here? Okay. Thank you. For example, I'm a user-- Let's actually draw a few examples, like work through a few examples and say, If we can have this sorted out, let's say, you know, Scott come here on to book room one. - Today's October 4th. 7:00 PM, right? So I want to book room one at 7:00 PM. So break on here. You want to book room one All right. Thank you.
[45:00] October 4th. 7 p.m. Okay, so what are you going to lock here? We will add a message queue. So who is going to the message queue first? So who will get the locker? - Message queue, so you actually introduce another system to do the log. Okay, so could you see yes. Yeah. Could you walk me through how you're going to use message to do the lock. So such like scout, it's go first. At here we have a message queue and Scott is go first. So Scott in the first position and then Rui in the second position. Maybe we have other guys such like Jim. So Scott is the first guy, so now Scott can use the database first. And the one Scott finished the record when the scout finish read and write. So it's race time. So read your check. Review use database second. So you're going to log the whole system to one? Like only starting? This is the solution one. Yes, in the solution two, I will change it. Solution one is lock all the database. Okay. Yeah. Let's see what's the other solution here. Okay. Now solution two. I know solution one is pretty slow. So solution two, I will create a site. So site is intake. It's the Room ID Thank you. Oh, oops. So such like Scott, Thank you. Scott want to book room one. Thank you. Okay, one scout in this couple of time. So we will see we will save we will save we will save room one in this side. So where is this side stock? This side stock is Maybe... Thank you. Just a memory. in memory in the memory yes I think for For general companies, they maybe have 1K. They maybe have 1K rooms. 1k physical rooms so memory is enough to hold off it okay so let's let's actually think about some scenario here it's gonna be it's gonna be fun okay so okay let's say you know let's take the first example as QS example So in this part, you actually, you know, you basically use Q to implement, you know, it's like a, People come to start for your service, so you use a queue. to to to say you know people line up right so what if uh what if scott want to book this room and after long delay, whatever reason. The delay could be network delay, It could be network delay, it could be, you know, Okay. somehow you have this process pause and it could be, you know, you basically go garbage collection on your node suddenly. Whatever is the reason. So you delay for a few minutes, right? Could be one minute. And then about it. What are you going to do for the rest of the request? So first of all, I will talk about the where where's the delay it hates. So let me Hold on one second. Let me drop something. How about draw something at here? Hmm. Make it bigger. If some delay is from client to the web servers, I know. So, Rui will come to the web servers first than Scott. No, no. Scott already arrived first. So, my request already arrived first. Yeah. Okay. But if Scott is delayed, Scott also arrived first. Yeah. Assume that Scott
[50:00] But there is a long pulse. Let's not call it. Lung pulse from the from the web server to the database or to the memory cache. Long pause from your starting right to committed. Thank you. Okay, okay. Thank you. Um. So in this case, I will send a a time. So such like if Scout do all of do all of this all of this process longer than One second. We will... just click scout out. and other things. Let's redo these things. Okay, so you mentioned something very interesting here. So you basically say, you know, we can time out, right? So we can time out at one second. So because of this, so I think this max throughput you can process is... this kind of request per day. Thank you. Thank you. uh but for one room is not Bye. - So I think you're locked in the whole thing, right? You locked the whole database, right? In the second one, it's locked the whole database. In the second, in the In the section one, lock the whole database. In the section two, we just lock the room Each room has one locker. - Oh, okay. So each room has one locker and each room is gonna be its own queue, is Right? Yes, correct. Okay, yeah, this is gonna be a little bit challenging, challenge to for me to challenge you. Let me think about some fun case here. Thank you. Thank you. Thank you. First of all, this solution is not working, don't be a bad one. This solution is not working, this is a database transaction. This is a database transaction. You can do two different transactions, this is a problem. Uh, Usually, you cross-theta system for transaction. And you can use memory queue to do any problem. I think I have a way to think about the way I would like to think of edge cases, counter cases, I don't like to think of edge. And this system will fill. I feel the more precise, I think. If you have any ideas, you can ask. Now, we can put the room ID on memory. and then the room ID is in memory and then the queue is in queue. for the inclusive request. Thank you. Thank you. Thank you. Thank you. Thank you. Thank you. Thank you. Thank you. Thank you. Thank you. Thank you. I'll put it in the middle of the screen. . Okay, so I think fundamentally the solution to and solution one are same. You just break down the granularity. Previously, the granularity is a whole database. Now the granularity is per rule. Correct. Yeah. So you basically just reduce the granularity to per room. So still, if we if we view from room perspective, so each room you're going to have a queue. And you're going to use this in memory queue. to help you to understand who is coming first and who is coming second. Thank you. Yes. Thank you. As well, we can use the calm time to do it. We cannot use calm time, right? Who is going to provide the calm time? Once the server gets a request, the server will add a time in the package. the server at PyTag, but you'll request them. So, for example, let's give an example. So you may have a book room request. You may have a book room request, and you may have a request one, and you have a request two, right? You have two book room requests, but it's all book room A. Thank you. Thank you. There is no guarantee. It's almost impossible, right? There's two requests. There is no guarantee these two requests will go to the same server. Oh, yes. So this may go to server one and this may
[55:00] go to a server tool. And if your server, somehow the server tool may be clock drift. Clock drift a lot. So it means that the clock maybe drift a lot like the B and Two hours. Two hours is slower than It means that all the requests go to server 2.0, you'll never be able to book a room. Request to survey two, we will never be able to book a rule. Yes, but I will talk about if If the request want to book room one, so the request should go to the server one. So OK, OK, request go to serverline is fine. Let's assume that normally you're going to You're going to to root request to server. What if the server crash? Server one crash, it means we will lose all of the requests. We need another server. go to server one's position and do the work for server one. Yeah, so you need to You need to root request, right? Uh... I don't want to request, I want to move also request to another server. I'm not sure the root request is split to different server or root all of the requests to another server. I will select the second one. Thank you. Okay, so still assume this, I think this assumption may not be doable. The reason is that, for example, request from China, so you want to do a business trip. and request to me come from us so are you saying that's no because you want to because you want to make sure the same room always go to the same server so no matter where your request come from you're always going to surveyed using a fixed server. Thank you. So you're not going to take the nearest server, you're going to just fix it to the server. Okay, we want to fix it for the server. Thank you. So basically, the Zoom request partition is based on the Room ID. Partition is based on the Room ID. That's what you're saying? Yes. Thank you. Thank you. Podession based on ROM ID. Thank you. Thank you. Thank you. So... I need to think about what case you need. I think it's a bit of a problem. This board Thank you. Thank you. Thank you. Thank you. Thank you. Thank you. - Okay. So, you're also going to assume that if server ground crash, If server one crash. then all the requests will be asked to plan to retry again, right? Thank you. Yes. Maybe not retry. We just We just delete all the requests if the request already in the server. If the requests do not arrive at the server, we just go to the new server. Thank you. So I need to think through this. So basically, let's assume that you have room booking your class. - Booking a request. So what you'll do is that first server accepts the request. Accept the request, and then, sorry, first is routing to, routing to, uh, routing to server partition. by rule This might be fine. Honestly speaking, I think this might be fine. Okay. Okay. So, I mean, say, just rotate the base on the room ID might be fine. So then Then basically, server accepts your request. and then put into in memory. Putting tool in memory. Q. Okay, so once it's put into in-memory queue, and then you can claim log to the database. which contains room ID. roommate. data. Okay. Mm-hmm.
[60:00] your turn, if it is your turn. Okay. Thank you. then you put Aim your jaw. Thank you. Thank you. Mm-hmm. OK, what about read? Let's see. What about read request? In this case, you're not going to-- how are you going to search read requests here? I still need to sync through some write-requests here. In my photo, we have two read requests. So one read request is from, could you see the, I share the screen? One is this one and another is this one. So which read request you ask? No, not a read request before the write, because you lock the whole database. You lock the database on your room, so you don't -- you're not going to have any -- you're not going to have this right screw issue. So this problem in transaction is called a right screw. or phantom write. So you're not going to have phantom write or write school issue, because your local database is very pessimistic locking, but you still need to read, right? You need to get a list of room which is available, so you can see which room you can send a request. I will say, so this locker is just a lock. This locker is just a lock, the bondage. I mean, if I bond it together, it's just a use for Let me write down something. Hmm. It's just the work at here and the work at here. Thank you. So they do not care about the read request if it is a single read. Thank you. Sorry, could you repeat? Yes. So could you see my Google, Google dryer? Yeah. Yes. So we have to to read it here. Why read it read? no right and another one is read and write together. Okay. Okay. So the locker is just the work at the read and write together. OK, so are you going to block rates there? I just block this bond. If you just a single read, it's okay, doesn't matter. So I'm asking when you execute this bundled operation, are you going to block it? Yes, I will block it together. Not... It's a little hard to say. In this couple of time, no one can access database based on this room ID. Okay. Thank you. Okay, let's say this. I think I got something very interesting. I finally come up with something very interesting here. Okay, so, I finally came up with something very interesting here. This is a pessimistic lock. You are very pessimistic lock. Pessimistic lock. There is a problem. You put it in memory, I just used to say it in the middle, because it's a bit complicated. You will put it in memory. You put it in memory, right? First, you said I have a request is claim in memory, is it right? I said request is claim, so I want to write I'll go to the server, right? I'll go to the room. Room A, and then in memory, you can actually lock it in the room A. Right? You can do Room A logs for it. If your server crashed, what happened? My first thought is that these requests are not required. That is the room A, or is it not being locked? Because the lock is in the server. That if it's broken, But if you have room A
[65:00] Hvað er hvað kalla? Make sense, OK, this room A be locked, so room A be released, is it? This is okay. That we can see this. Pessimistic locking is the most common problem. That we now room A be locked. 然后呢,你去写database og við átabase. 然后database crash 了,然后你怎么 handle Það er það er það er það. Optimism, luck, eðað er sér. Það er það er það er það. - Ja. - Það er að Timemort og Db Request. Um. Það er það er það. Það er það er það. Það það það það það það það það það það það. Speedness, just your speed, or efficiency. Data base is a lock. Einstund er to-facelocking. Ígæði það er það er að ræst-cum problem. Material conflict var material conflict. Ítri er lokk on index er hættið hættið hætti. Ítri er lokk on room. Ítos þú er a loom á loom. Ítos þú er loom á loom. Háða. Ígert er dyrt að er að hæðað er að hæðað. Ígert er að hæðað er að hæðað. það erum það erum konflikta. það erum það erum konflikta. To face locking的话,是 read和 write 相互 block 的 就是 read and run 是 block 的
[70:00] 影响着,就两个之间是不影响的谢谢大家所以你在写的时候就是好你在写的时候这块的话你实际上是你的performance是two-phase locking 的solution performance 然后甚至比two-phase locking performance 还要糟糕 因为你是用了一个外面的系统来进行优化的但是你的这个实践Solution有点像Lock on Room A这种感觉嗯,所以就是从performance来讲的话是肯定会比较糟糕的,但是crackness角度我现在还真没想到一个方法说,就是crackness 是有问题的。谢谢大家谢谢大家谢谢大家好我再花一分钟时间,我再花一分钟,因为这个时间也花了挺久了,我看我能不能想到一些crackness方面是有问题的地方,如果不能的话,我们就继续看了,路往好了KracknessFormation谢谢大家好我想到一个crackness有问题的地方这个应该是个valid case好 你现在是这样子的 你有两个server server 1和server 2Server 1 和Server Backup 叫 Server 1 Prime这个是 Application Server,你有两个 Application Server这个可以吧然后你第一个request呢,是request呢,是到server1的。然后你要求book的什么呢我们把刚才的上面的也copy下来就是入目万吧啊就是我们吧嗯对第一个request的start server也是bookroom match吧然后呢你这个request呢删到db了然后database开始处理了就是database开始清理什么呢then write 是吧可以吧?然后这时候发生了一个server一 crash 掉了然后这时候就变成AppServer二是主要的server room 1的什么了,就是AppServer eCrime 了然后这时候你有第二个request 来第二个request来谢谢大家谢谢大家谢谢然后呢这个是你自己book同一个room的,但是这个呢,因为你去这个prime外,然后没有你信息了,所以这个也过去了。这个时间可以很短比如说一毫秒之内嗯那你这样的话这两个还是conflict谢谢大家这两个还是conflict的时间不是观念,这个时间可能因为你这个write这个可能这个performance有degradation的话很糟糕,这个可能会花几十毫秒但是这个Server Recover找Prime 1的话,这个可能一毫秒就转化了你这两个会conflict?首先我想问,我没太懂您这个例子,我想知道这个database, read then write,我有写进去吗?你在写的过程中,我们画一个时间图,可能更好理解一点。我们把上面这个叫requestRequest 1好了,下面叫 E' 是吧?嗯。然后 1 是这样子, 1 到了,不能叫 1, 1 的话重复,我们叫什么呢?叫爱好了Request i 然后 i 到了 e-server 是吧?然后开始了这个 db transaction 整个这个是 db transaction谢谢大家DB transaction好可以吧? 嗯,然后Transaction,Transaction,Transaction,Transaction,Transaction,然后Commit谢谢大家是吧可以吧好Commit以后就返回了那我们就是假设这样子好了然后呢这个时候呢这个第二个request就要按按好了这种情况下到1的时候1已经挂了,只能去1'了他去 E' 的时候他是没有这个 lock information 的。所以它会什么呢?它已经transaction三三十二然后你同时两个transaction就可以发生各种各样的事情谢谢观看就是在这两个时间点之间这两个时间点之间E挂掉了,然后E'作为它的backup server好的 我好像明白了好
[75:00] шутку. Ну И в уже в прошлом году, и в прошлом году, и в прошлом году. И второе request就到 E''''. E''''没有 E' информation, и то есть. И мы это называемый таймлайн. Продолжение следует... Продолжение следует... Продолжение следует... Ух. Окей, хорошо. Это крактность, и перформансность. И я наконец-то получил пример. Продолжение следует... Субтитры сделал DimaTorzok Это еще более улучшительный. У меня был в мебельце. Я в исследовании этого строительства, я увидел такой вопрос. Мы в... В этом разнообразии, Вы говорите, что 10КБ уже достаточно. Но я хочу спросить, если мы начали 10КБ, но если вдруг появится значение, то это действительно больше 10КБ. Или в будущем, когда 10КБ не будет больше, то это будет больше 10КБ. Это будет больше, чем 10КБ. Это будет больше, чем это будет больше. Это будет больше, чем это будет больше. Это будет больше, чем это будет больше. Средиземный крышек не будет. Сейчас мы строим большую большую решение, это скалабильность обычно очень скалабильность. Это 15 кг пппо, это действительно очень страшно. Если 15 кг пппо, то можно будет 3 миллиона север. 30000000 сервер, это очень много сервер, это очень страшно. Поэтому это учитывая, что это учитывая, что у нас есть более широкая. Но обычно, на самом деле, на самом деле, все системы строительства, 基本上, на самом деле, на самом деле, на самом деле, на самом деле, на самом деле, Так что это не важно. То есть 3,000,000 заверий тоже можно сделать. Только это, как бы, чтобы сохранить, но это еще не так. Я видел, что Лисия уже встал. И он просто... Это мантаин 4,000-двыксервера. Ну, я хочу спросить, если я начинал 10КБ, и после этого 10КБ не используется? та То есть, если я хочу стоить 100 кб, то я как-то так делаю? Это выгоднее, это выгоднее. Выгоднее, это выгоднее. Это выгоднее, например, 1 мегабайт. Но выгоднее, это выгоднее. - Ты может быть 1 мегабайт, а в большинстве есть 1 кб, и 1 мегабайт, и вы 1 мегабайт, и вы 1 мегабайт, и вы 1 кб. О, хорошо, я понял. Продолжение следует... - Хорошо. Но это не проблема, потому что лок не было никаких проблем. Это не было никаких проблем. Поэтому, если вы хотите использовать лог, то есть это создание системы лог. Не нужно найти создание внешне лог. И также, в этом случае, в этом направлении, что все это все серверы, которые все это сделано. Апп сервер, это можно было бы убрать. Датабаз сервер, это может быть более более详细. И еще больше нужно гарантии, например, датабайс сервер, можно не расширить. И если не нужно, то можно сделать это вверх, то можно сделать это вверх, чтобы сделать это вверх. Примеры. Но в обычном случае Датабейс сервер, вы можете видеть его выводы более высокого. И в транзакции, вы можете использовать его датабейс, который выводит в транзакции. И вот, чтобы повторить все это, потому что мы говорим о трансляция. Это было повторно, как вы создали один лунг, что вы создали один лунг? Локон индекс, и локон room индекс. И это снова создавая локон. В этом месте будет много проблем. Я не думаю, что это не проблема. Я не думаю, что это проблема. Я думаю, что это проблема. Я думаю, что это проблема. Ум. И еще раз, я думаю, что это было очень хорошо. Я думаю, что это было так, что я думаю, что это было так. Я думаю, что это было так. Я думаю, что это было так. Я думаю, что это было так. Я думаю, что это было так. Я думаю, что это было так. Поэтому я думаю, как бы это было, как бы это было. Семьян, он
[80:00] 我这个想法确实不对的好的然后我现在的话再go over 一下这个题好了我再go over 一下这个题你建议往上scroll一下我现在的话只有五分钟时间go over 一下这个题好了这个题的话我觉得开始的时候是没有问题的包括这个constraint design计算也是没有问题我计划constraint给的过于大了但是无所谓很多时候就希望给你个大概300个server然后1000个server能藏下的data然后这块的话我们看到这个QPS本身其实是不大的QPS本身是不大的但是这个大的是Data Size这个Data Size过于大了QPS本身很小2K的话或者20K的话你一个Server就处理下来了但是你这个Data Size15个PB的话你如果用MySQL存的话你可能需要3000多个Server一个MySQL500GB的话0.5TB的话你可能需要3000个甚至3000到3000个Server这个conference我给的不太好给的太随便了但是我们去上面看这个题的话我们看这上面这个题的话这个schedule是正常的clude的操作就是正常的create, read, update, delete 操作因为这个transaction要求比较高所以可能可能你还是要用 MySQL database或 SQL database因为它 transaction 要求确实有点高然后如果这个 event 的话其实其他好fills 都比较好 populate就是 meeting room 这个确实比较难然后 meeting room 这块它有一个非常难的地方就是你怎么样去像 Uber 的 meeting room 永远是满的就是我任何时候看基本都没有 meeting room available这个是比较复杂的那Meteorome这块的话有几种方式像Two-Face Locking可以解决然后或者是Material of Conflict也可以解决我周四的时候都会讲一下然后其次的话你可以就是Lock on Index你可以Lock on Room A或者Lock on Time Slot这块的Lock 所有Lock一定要Database的Lock你不要自己搞一个Lock 问题很多就一定要DatabaseLock因为它本来就为这个优化的然后DatabaseLock 比如说SelectRow什么ForUpdate的话你就把那个Role update,就是Log 住了然后就是用Data Base Lock然后最后的话,其实这个Schedule 这块还是有问题的你比如说你Schedule的时候,你Schedule 是Schedule Weekly 是吧然后你这个Meeting Room第一次的Meeting 是没有Conflict那第二次Meeting 有Conflict了那你这块怎么处理,是有一些Business的Challenge比如说你这个Schedule这个meeting是weekly在周一下午weekly at 6点是吧周二下午weekly at 6点你第一周是没有conflict然后到第二周的时候就有人跟你competing这个room然后把这个room已经提前站好了那你应该return什么是return一个partial failure呢还是return一个success但是第二周被fuel掉了是吧这个是有点需要考虑的地方我看google这个calendar处理是只要你第一周的这个room能book到之后不管有多少conflict都是reject你会收到很多邮件说week2roomreject,week3roomreject,week4roomreject所以它会有个这样的邮件或者它可以reserve几周就当你看room的时候会reserve几周这个可能会reject然后这块还有一个非常潜在的问题其实没有讨历到是你这块的readrequest是远远大于writerequest所以你这个read和database的选择的时候你的read和read不应该相互block你需要做non-blocking的readread所以你可能需要一些你的transaction level就是你的isolation level可能需要一些snapshot isolation所以two-face locking基本上这个系统就挂了你如果用two-face locking的话你这个系统基本上performance很糟糕所以很多时候你可能更多的是要material conflict或者是lock on indedex这个话我周四会讲更多一点就是 Two-Face Lock-In 是 Read 和 Read 相互 Block 的你这块的话 Read 很多那你可能 performance 会很糟糕好 这就是这个问题的话几个比较难的考点几个比较难的地方大家有什么想法或者什么 comments我们还有几分钟可以稍微拖持几分钟讨论一下今天讲的是非常难的今天讲的你需要对database有很深的了解然后至少这个什么two-faced login怎么样子transact是怎么样子是要比较深的了解的这个是如果是new-grads或者没有学过database的话还是比较难的我觉得确实的话是有点难的需要很多那个就是我对于可能google calendar不是很了解但是我之前在linkedin这边我们做过Outlook calendar的这些东西我们当时 calendar就是room service我们其实做的是一个non-blocking call如果你add guest的话你是一个blocking call但是你的add room是一个non-blocking他之后具体怎么做不是很清楚但是我想可以比如说我们sending一个卡卡以外然后我们按照那个room来做一个那个hash
[85:00] Kafka partition shop. It can ensure the request is a good thing And then it's in DBs. Then there are problems, first person already wrote, second person will go back to the file, then I can just reject it. And And then... - Yes, I think you should use a non-blocking account. I think you should use optimistic locking. You will resolve it. You will reject one. You will reject one. You will reject one. You will reject one. You will reject one. You will reject one. You will reject one. You will reject one. Optimistic Locking is a bit more than that. Google Calendar is Google Calendar is pessimistic lock, you claim it is very little. You will be rejected. You will be rejected. You will be rejected. You will be rejected. And we will be rejected. And we will be rejected. It is a good thing. If you have rejected, you will be rejected. You will be rejected. If you accept the room, you will be accepted. If you accept the room, you will be accepted. It's a little bit too No, they're too busy. You will be able to do the same thing. So in a while, it can be done with the data. If you read it, you prefetched it, this would be more difficult. You prefetched it, you can lock the room or not lock it? - The user is now in which time it is already booked, and you can choose to choose the room. Because now the Outlook is not so much. - Yes, in which time the book is not a problem? But how do you deal with the available room? You read it after all, you can do two people with the same time and book. And then you can reject who? How will reject it? Oh. Right? You have any time. The idea of the计算机 is a relative. You can do one minute. You can do one minute. One minute. You can do one minute. In any time you can add the amount of time to the same time. It's like a common sense of the same thing. You have the amount of time to the same time. You have the amount of time to the same time. When you read it, it's not the same time. When you book it, you can do this. How do you deal with conflict? Actually, you can see that the case is not crash. So we have two-phase locking or lock on index to solve this problem. Thank you. Thank you. So there are other ways to be able to solve this. It's just that you can do this rate and rate is the same operation. Actually, the idea of the thought is that you can do the same operation. Thank you. Okay, this is the problem. Room lock is low, but the volume is very small. So every room... And then that way, it can be. on. Room lock is small, but you have to book today, there are people who book today This is also a good thing. It's just like room and lock. This is also a good thing. It's just like index. This is a good thing. This solution is okay. But from the general perspective, Okay. This is the index. What other questions to do to discuss today? We may need to discuss the transaction. This is a difficult question. Thank you. Thank you. If we haven't, we'll just end this. And then - Okay. - Thank you Scott. You're getting a lot of money. - Okay. Thank you. - Thank you. - I think the design is quite a creative. I'm more confident. I'm more confident in the experience of the experience. But there are some experience I think I need to add some more.
[90:00] 对就是比如说这个有一些这个比如说这个怎么样选mojo不应该通过API来想是吧应该通过他这个这个database的性质来学。然后其他的确实有一些经验的东西,比如说我真的在今天面试之前,我真的不知道数据库里面还有这么一个锁的东西。对的,数据库锁,database里面其实它中间经过了几十年的发展,从1970年到现在经过了50年的发展,然后它其实这开Base里面有很多很强大的functionality只不过它这个 MySQL 的 database, SQL 的 database 它发展都是单机发展,就是 single node 的发展,所以它这个在这个 cross node 的时候,或者是 cross distributed system 里面,这个就是很多地方是不适用的,但是它 single node 的这个 transaction是做得非常非常好的,Single Node Transaction Support做得非常非常好,有很多 functionality做得很好。那好那如果这道题我在面试的时候我就是直接跟老师说直接跟面试官说那个就从这个数据库里边有个锁啊就调用这个锁就行了就是这样是吗?对,但是首先你说这个Database是什么样的isolation level,是吧?你是Snapshot isolation read 还是这个问你这道题的面试官其实就想考察你这个database,其他的简单的crude你可以先来解决了,就是这种简单的什么entity的data model design,这种crude design你可以先解决了,这个花不了多久的,就是20分钟就解决了,就是你设计一个什么event crude,设计一个user crude这个就是C-R-U-D,就是Create,Read,Update,Delete,这种end point,这个可以很快解决了,然后,但是它其实考察你主要难点就是你怎么样,看到这些 transaction 所以问你这条题的用户他面试官还是想考察一下你怎么样然后另一方面来讲这个面试题确实对三天工作经验是有点难了点这个需要很久的这个对他Base很深的了解所以比较难一点好的谢谢谢谢拜拜谢谢大家谢谢大家
![[Chinese] Mock interview: Design calendar — Scott Shi](https://reducer.xgoose.org/media/thumbnail/3885.jpg)