Rob Pike Decouples Concurrency from Parallelism to Speed Multicore Code

Rob Pike 解耦并发与并行以提升多核代码性能 当工程师混淆程序组合与同时执行时，多核处理器上的软件性能就会下降；而将系统构建为独立的通信过程，则可以在无需锁的情况下释放硬件并行能力。 Rob Pike 排除了并发与并行之间普遍存在的混淆，解释称并发是一种通过独立进程同时处理多件事情的架构工具，而并行则是关于物理上同时做多件事情的执行细节。Pike 借助囊地鼠搬运并焚烧废弃编程手册的直观比喻，阐释了并发分解如何提高吞吐量并实现自然扩展。随后，他演示了 Go 的核心原语——goroutine、类型化通道和 `select` 语句——如何将这些结构化理念转化为实践，并详细介绍了无锁优先队列负载均衡器和多副本数据库查询分发器。最终，Pike 指出，优秀的并发设计无需依赖多个处理器来保证其正确性，而是作为基础，让并行变得简单、健壮且可扩展。 Rob Pike 将并发定义为独立执行进程的组合以及程序结构的架构工具，并将其与并行（即多个计算在物理上的同时执行）进行了对比。 (1:42) Go 通过极其轻量级且动态多路复用到底层操作系统线程的 goroutine 原生支持并发，能够在生产系统中扩展至超过一百万个实例。 (17:13) 一等类型化通道允许构建复杂的异步架构（例如优先堆负载均衡器和多副本数据库查询聚合器），而无需显式互斥锁或锁定机制。 (20:02) 并行并不是并发的主要目标，但清晰的并发分解在本质上会自然释放出可扩展的并行能力，将其作为执行层面上的自由变量。 (8:46)

Rob Pike argues that concurrency is about program structure rather than physical execution, serving as a compositional tool that enables, but does not require, hardware parallelism. Structure Versus Execution (1:42) Pike defines concurrency as the composition of independently executing processes and parallelism as the simultaneous physical execution of computations. In his rule of thumb, concurrency is about dealing with many things at once, whereas parallelism is about doing many things at once. An operating system kernel managing a mouse, display, and network drivers on a single CPU core is fully concurrent despite running on sequential hardware. Conversely, executing a vector dot product across hardware processing elements exhibits parallelism without a concurrent structural model. When developers run concurrent programs on multiple processors and observe degraded performance, Pike asserts that their conceptual model—conflating structural design with raw parallel throughput—is flawed. Decomposition and Scaling Dynamics (6:50) Using an applied metaphor of gophers hauling books to an incinerator, Pike shows that dividing a monolithic task into independent stages—loading, carting forward, unloading, and carting back—yields a concurrent pipeline. Counterintuitively, introducing extra steps like a dedicated return carrier increases the total operations performed while dramatically improving system throughput, potentially achieving up to a fourfold speedup when stages stay balanced. Once a task is decomposed into independent procedures, scaling can proceed along multiple axes, including full workflow duplication or introducing intermediate staging buffers. Language Primitives in Go Go implements concurrency through goroutines, channels, and the select statement, drawing heavily on C. A. R. Hoare's 1978 Communicating Sequential Processes (CSP) framework. Goroutines are lightweight execution contexts dynamically multiplexed over operating system threads, allowing applications to sustain tens of thousands of active routines—and over a million allocated instances—with minimal memory overhead. Channels serve as first-class, typed conduits for passing data, while the select statement acts as an arbitration switch that multiplexes channel operations non-deterministically without explicit locking mechanisms. Lock-Free Systems and Distributed Patterns (21:40) Pike illustrates that structured concurrency eliminates classic multithreading pitfalls like mutexes and condition variables. In an advanced load balancer, incoming request objects encapsulate their own private reply channels, allowing worker routines to compute tasks and return results point-to-point without routing traffic back through the balancing heap. A replicated database query pattern similarly spawns concurrent goroutines to query separate shards simultaneously into a buffered channel, returning the fastest response and safely discarding lagging queries. In both cases, concurrency inherently unlocks hardware parallelism as an intrinsic property of system architecture. Pike concludes that concurrency is not parallelism, but rather a superior structural methodology that makes parallelism straightforward to extract. As an unresolved production detail in his slides, Pike notes that while the replicated query pattern cleanly accepts the fastest response, complete production implementations still require explicit cancellation mechanisms to tear down slower ongoing backend operations.

Rob Pike 认为并发与并行本质不同：并发关注程序的结构，是组合独立执行过程的技术；并行则关注执行，是同时运行多个计算。良好的并发设计能有效组织系统，从而自然地使并行执行变得更简单、更具扩展性。 概念辨析：结构组合与物理执行 (1:42) Pike 指出，许多开发者误以为并发编程等同于并行加速，甚至在增加处理器后发现程序变慢，根源在于混淆了二者的概念。并发（Concurrency）是在宏观上应对（dealing with）很多事情，属于架构层面的程序结构设计，源自 Tony Hoare 1978 年的通信顺序进程（CSP）理论，通过独立过程及其间的通信来组合系统。并行（Parallelism）则是在物理上同时做（doing）很多事情，关注的是执行机制。例如单核操作系统中各设备驱动程序的协同管理属于纯粹的并发架构，即便没有多核并行也能正确运转；而向量点积运算利用微处理单元并行计算，则属于缺乏并发结构的纯并行操作。并发的目标并非并行本身，而是通过良好的解耦结构，赋予系统在硬件具备条件时按需拓展并行的能力。 地鼠隐喻：流水线解耦与吞吐提升 (4:26) 为了具象化并发设计的演进，Pike 提出了地鼠搬运手册至焚化炉的隐喻。单个地鼠推车往返效率低下，若仅仅增加地鼠而不配套推车，就会因争抢端点资源产生协调瓶颈；而将整个设置成倍复制虽然能达成物理并行，但本质仍是独立并发过程的组合。更具架构价值的做法是将任务分解为多阶段流水线：分别由不同地鼠负责装载、运送、卸载焚化以及返还空车。尽管返还空车增加了额外的物理动作与过程，但在各阶段节拍平衡的前提下，所有工序得以重叠展开，整体吞吐量不仅显著提高，还可实现数倍于初始设计的处理速度。系统随后引入中间暂存堆等缓冲结构，并可在多个维度自由复制扩容。这种地鼠、推车、暂存堆与焚化炉的协同模式，直接对应了现代 Web 架构中 CPU、网络序列化、代理缓存和客户端之间的解耦与协作关系。 Go 语言的原语基石：轻量协程与通道调度 (13:48) Go 语言为并发架构提供了原生语言级支持。通过在函数调用前添加 go 关键字，程序即可启动一个轻量级的独立任务——Goroutine。Goroutine 的初始栈空间极小，由运行时动态伸缩并多路复用调度到系统线程上，即使单个阻塞也不会卡死整个执行上下文。Pike 提到在生产系统调试中曾观察到 130 万个已创建协程、其中约 1 万个活跃运行的案例，说明其开销远低于传统操作系统线程。在通信与协调方面，Go 提供了类型化管道（Channel）以及 select 语句。select 类似于专为通道通信设计的控制开关，支持多路复用监听、分支执行、非阻塞的 default 回退机制，以及在多分支就绪时的伪随机仲裁，结合第一类词法闭包，使异步逻辑能够以极简的代码结构清晰表达。 实战应用：无锁负载均衡与副本查询 (21:40) Pike 进一步通过实际模式展示了并发如何自然引申出并行。在工作池模型中，多个协程从共享通道拉取任务并回传结果，无需显式互斥锁或条件变量即可避免经典死锁与竞争问题，且并行度可自由伸缩。在更高级的负载均衡器中，Go 的通道作为一类公民（First-class values）可直接封装进 Request 结构体充当私有返回地址；均衡器内部借助堆（Heap）优先队列管理工作协程的负载计数，利用双向 select 循环分发请求和接收完成信号，而工作协程计算完毕后直接沿着请求自带的通道点对点回复调用方，完全绕过了均衡器的转发路径。此外，针对多副本数据库查询，程序只需为各副本并发启动查询协程并写入同一带缓冲通道，最先返回的结果即可被主流程直接读取，以极低的代码复杂度解决了分布式场景中的竞速与容错问题。 Rob Pike 强调，并发本身不是并行，但它是使并行变得可控且高效的前提构架。并发着眼于将复杂问题分解为独立、可理解且易于正确验证的组件并进行组合；当软件架构具备正确的并发解耦后，底层的硬件并行能力即可成为一个可按需调配的自由变量。除展示生产级方案外，演讲未深入探讨极度严苛场景下的超时取消机制，并建议读者参考 CSP 原著及相关学术文献以深化理解。

[29:55] Rob Pike: Concurrency is not parallelism, but it enables parallelism and it makes parallelism easy, and if you get that, then I’ve done my job.

[0:03] you if you looked at the programming [0:05] languages of today you probably get this [0:07] idea that the world is objectoriented um [0:11] but it's not it's actually parallel um [0:13] you've got everything from the lowest [0:15] level like multicore machines and up [0:18] through networking and so on so but it [0:19] gets all the way up to users who knows [0:21] planets the universe there's all these [0:23] things that are happening simultaneously [0:25] in the world and yet the Computing tools [0:27] that we have are really not very good at [0:30] expressing that kind of worldview and [0:32] that seems like a failing but we can fix [0:35] that if we understand what concurrency [0:37] is and how to use it now I'll assume [0:40] that most of you at least heard of the [0:41] go programming language is just what [0:43] I've been working on at Google the last [0:44] few years and go is a concurrent [0:46] language uh which means that it has some [0:48] things in it to make concurrency useful [0:51] like it's got the ability to execute [0:52] things concurrently the ability to [0:54] communicate between things that are [0:56] executing concurrently and it's got this [0:59] thing called a SEL statement which is a [1:00] multi-way concurrent control switch if [1:02] that doesn't make any sense to you yet [1:04] don't worry but when we announced go [1:07] which was about 2 years ago all these uh [1:09] programmers out there said oh concurrent [1:11] tools I know what to do I can run stuff [1:13] in parallel yay uh but that actually [1:15] isn't true concurrency and parallelism [1:17] are not the same thing it's a commonly [1:20] misunderstood problem and I'm here to [1:22] try to explain why and to show you that [1:24] concurrency is actually better what [1:26] would happen to these people who were [1:28] confused was they take a program they'd [1:31] run it on more processors and it would [1:33] get slower and they think this is broken [1:35] it doesn't work I'm going away um but [1:37] what was really broken was the worldview [1:39] and I hope I can fix that uh so what is [1:42] concurrency well concurrency as I'm [1:44] using it and as it's intended to be used [1:47] in the computer science world is a way [1:49] to build things it's the concurrent it's [1:51] a composition of independently executing [1:54] things typically functions but they [1:55] don't have to be and we usually Express [1:58] those as the the interacting processes [2:00] and by process I don't mean a Linux [2:02] process I mean the sort of General [2:03] concept that embodies threads and co- [2:05] routines and processes the whole thing [2:07] so think in the most abstract possible [2:09] sense so it's the composition of [2:11] independently executing processes [2:13] parallelism on the other hand is the [2:15] simultaneous execution of multiple [2:18] things possibly related possibly not and [2:21] if you think about it in sort of a [2:22] general handwavy way concurrency is [2:24] about dealing with a lot of things at [2:26] once and parallelism is about doing a [2:28] lot of things at once [2:30] and those are obviously related but [2:32] they're actually separate ideas and [2:33] there's a little confusing to try to [2:35] think about them uh if you don't have [2:37] the right toolkit so one is really about [2:39] structure concurrency and one is about [2:41] execution Paralis and I'll show you why [2:44] those are important so concurrency is a [2:46] way to structure a thing so that you can [2:49] maybe use parallelism to do a better job [2:52] but parallelism is not the goal of [2:54] concurrency concurrency goal is is a [2:56] good structure so here's an analogy you [2:58] might be familiar with if you're running [3:00] an operating system it might have a [3:01] mouse driver a keyboard driver display [3:04] driver network drivers whatever else and [3:06] those are all managed by the operating [3:07] system as independent things inside the [3:10] kernel but those are concurrent things [3:12] they aren't necessarily parallel if you [3:14] only have one processor only one of them [3:16] is ever running at a time and so there's [3:18] a concurrent model for these IO devices [3:21] but it's not inherently parallel it [3:23] doesn't need to be parallel whereas a [3:24] parallel thing might be something like a [3:26] vector dot product which you can break [3:28] down into microscopic operations that [3:29] you execute on some fancy computer in [3:31] parallel very different idea not the [3:33] same thing at all so in order to make [3:36] concurrency work though you have to add [3:37] this idea of communication which I'm not [3:39] going to focus on too much today but [3:40] you'll see a little bit about it so [3:42] concurrency gives you way to structure a [3:43] program into independent pieces but then [3:45] you have to coordinate those pieces and [3:47] to make that work you need some form of [3:49] communication and Tony in 19 1978 [3:52] wrote a paper called communicating [3:53] sequential processes which is truly one [3:56] of the greatest papers in computer [3:57] science and if you haven't read it if [4:00] anything out of this talk SNS in is that [4:02] you should go home and read that paper [4:03] it's absolutely amazing but based on [4:05] that a lot of people with lesser Minds [4:07] have followed and built tools to use [4:09] these his ideas into concurrent [4:11] languages like llang is another one [4:13] that's great go has has some of these [4:15] ideas in it but the key points are all [4:17] in that original paper with with a [4:19] couple of minor exceptions which I'll [4:20] I'll come up to but look this is all way [4:22] too abstract uh we need [4:26] Gophers so let's get some Gophers going [4:29] here's a real problem we want to solve [4:31] okay we have a pile of ancient obsolete [4:34] manuals say the C++ 98 manuals now that [4:37] C++ 11 is out or maybe it's a C++ 11 [4:40] books and we don't need them anymore [4:42] whatever point is we got to get rid of [4:43] them they're taking up space so we have [4:44] a gopher whose job is to take the books [4:47] from one pile and move them into the [4:48] incinerator and get rid of them okay but [4:51] with only one gopher it's going to take [4:52] a long time if it's a big pile and also [4:54] Gophers aren't very good at moving books [4:56] although we've given them a cart um so [4:58] let's put another gopher in the problem [5:00] except he's not going to get any better [5:02] right because he needs the tools and [5:03] this this is kind of pointless we need [5:06] to give him all the parts he needs in [5:07] order to do this so This gopher needs [5:09] not only the ability to to be a gopher [5:12] but he also needs the tools to get the [5:13] job done so let's give him another cart [5:16] now that's going to go faster we're [5:17] definitely going to be able to move [5:18] books quicker with two Gophers pushing [5:20] the carts but of course there may be a [5:22] little problem because we're going to [5:23] have to synchronize them they can get [5:24] stuck at the incinerator at the book [5:26] pile getting each other's way running [5:27] back and forth so they're going to need [5:28] to coordinate a little bit so you can [5:30] imagine the Gopher sending s little Tony [5:32] messages saying here here I am I [5:34] need space to put the books in the [5:35] incinerator or whatever it is but you [5:38] get the idea this is silly but I want to [5:40] make it really clear these ideas are not [5:41] deep they're just they're just good okay [5:44] well how do we make them go faster well [5:46] we double everything we put two Gophers [5:47] in and we double the the pile uh the [5:51] piles and the incinerators as well as [5:53] the goers and now we can move twice as [5:55] many books in the same amount of time [5:57] that's parallel right but think of it [5:59] instead of parallel as really the [6:01] concurrent composition of two goer [6:03] procedures moving books so concurrency [6:06] is how we've expressed the problem these [6:08] two this this gopher guy can do this and [6:11] we paralyze it by instantiating more [6:12] instances of this gopher procedure and [6:15] that's called the concurrent composition [6:17] of processes or in this case Gophers now [6:21] um this design is not automatically [6:24] parallel because sure there's two [6:27] Gophers but who says they both have to [6:29] work at the same time I could say that [6:31] only one gopher is allowed to move at [6:32] once which would be like having a a [6:34] single core computer and the design is [6:36] still concurrent and correct and nice [6:38] but it's not intrinsically parallel [6:40] unless I can make both those Gophers [6:41] move at once that's when the parallelism [6:43] comes in having two things executing [6:45] simultaneously not just having two [6:47] things okay that's a really important [6:50] model but once we've decided that we [6:51] understand we can break the problem down [6:53] into these concurrent pieces we can come [6:55] up with other models so here's a [6:56] different design okay now we got three [6:58] Gophers on in the in the picture it's [7:00] the same pile of books the same [7:01] incinerator but now we've got three [7:03] Gophers right there's a gopher whose job [7:05] is just to load the cart there's a job [7:08] there's a gopher whose job is just to [7:09] carry the cart and then presumably [7:11] return the empty back again and then [7:12] there's a gopher whose job is to load [7:14] the incinerator so three Gophers it's [7:16] going to go faster might not go much [7:18] faster though because they're going to [7:19] get blocked you know the cart with the [7:20] books is going to be in the wrong place [7:22] and there's time to bring the Gopher [7:23] running back with the Mt where nothing [7:25] useful is getting done with the cart so [7:27] um let's clean that up by having another [7:29] go for return the empties okay now this [7:32] is obviously silly right but I want to [7:34] point something fairly profound that's [7:35] going on here this this version of the [7:37] problem will actually execute better [7:40] than this problem this guy even though [7:42] we're actually doing more work by having [7:44] another goer running back and forth in [7:45] here so once we've got this concurrency [7:48] idea we're able to add Gophers to the [7:50] picture and actually do more work but [7:53] make it run faster because the [7:55] concurrent composition of better managed [7:57] pieces can actually run faster and it's [8:00] pretty unlikely that things will work [8:02] out just perfectly but you could imagine [8:04] that if all the Gophers were time just [8:06] right and the PS were just right and [8:07] they knew how many books to move at a [8:08] time this thing could actually keep all [8:10] four Gophers busy at once and it could [8:12] in fact move four times faster than our [8:13] original [8:16] version unlikely but I want you to [8:18] understand that it's possible so here's [8:21] an observation that's really important [8:23] and it's kind of subtle we improve the [8:25] performance of this program by adding a [8:27] concurrent procedure to an existing [8:29] design design we actually added more [8:31] things but the whole thing got faster [8:33] and if you think about it that's kind of [8:35] weird it's also kind of not weird [8:37] because you added another gopher and [8:38] Gophers do work but if you forget the [8:40] fact that he's a gopher and think of it [8:41] as just adding design adding things to [8:44] the design can actually make it more [8:46] efficient and that parallelism can come [8:49] from better concurrent expression of the [8:50] problem it's a fairly deep Insight that [8:52] doesn't look like it because there are [8:54] gophers involved but that's okay um so [8:57] we have four concurrent procedures [8:58] running here right there's a gopher that [9:00] loads things into the cart there's a [9:02] gopher that takes the cart and TR trucks [9:04] it across towards the incinerator [9:06] there's another gopher who's unloading [9:08] the carts contents into the incinerator [9:09] and there's a fourth gopher who's [9:11] returning the empty carts back and you [9:13] can think of these as independent [9:14] procedures just running as independent [9:16] things completely and we just compose [9:18] those in parallel to construct the [9:21] entire program solution but that's not [9:23] the only way we could do it here's a [9:25] completely different design sorry not [9:27] complet we we can here's a same design [9:29] made more parallel by putting another [9:31] pile of of books another incinerator and [9:33] four more Gophers but you see the key [9:35] point is here we're taking the idea that [9:37] we have how we break the problem up and [9:39] once we understand it its concurrent [9:41] decomposition we can actually paralyze [9:43] it on different axes and get better [9:45] throughput or or not but at least we [9:48] understand the problem in a much more [9:49] fine grain way we have control over the [9:51] pieces in this cases if we get [9:52] everything just right we've got eight [9:54] Gophers working hard for us burning up [9:56] those C++ manuals um or maybe there's no [9:59] paralyzation at all who says that all [10:02] these Gophers have to be busy at once I [10:04] might only be able to run one gopher at [10:06] a time in which case this design would [10:08] only run at the rate of a single gopher [10:10] like the original problem and the other [10:12] seven would all be idle while he's [10:14] running but the design is still [10:17] correct and that's a pretty big deal [10:20] because it means that we don't have to [10:21] worry about parallelism when we're doing [10:23] concurrency if we got the concurrency [10:24] right the parallelism is actually a free [10:27] variable that we can decide just how [10:28] many go goers are busy or we could do a [10:31] completely different design for the [10:32] whole thing let's let's forget the old [10:35] pattern put in a new pattern and we'll [10:36] have two Gophers in the in the in the [10:38] story but instead of having one gopher [10:41] carried it all the way from the pile to [10:42] the incinerator we put a staging dump in [10:45] the middle so the first gopher carries [10:47] the books to the to the dump drops them [10:49] off runs back gets more the second guy [10:52] sits there waiting for cart for books to [10:54] arrive to the pile takes them from there [10:55] moves the incinerator and if you get [10:57] this right you got two goer procedures [10:58] running but they're kind of different [11:00] procedures they're kind of the same but [11:02] they're subtly different they have [11:03] slightly different parameters but if you [11:04] get this system running right at least [11:06] once it's started uh it can in fact in [11:08] principle run twice as fast as the [11:10] original even though it's a complet [11:11] different design in some in some sense [11:13] to the original one but of course once [11:16] we've got this composition we can go [11:18] another way we can paralyze the usual [11:20] way run two versions of this whole [11:21] program at once and double again now we [11:24] got four golers maybe up to four times [11:26] the throughput or [11:29] we could take a different way again and [11:31] uh put the staging Pile in the middle [11:33] into the original concurrent multi- [11:35] gopher problem so now we got eight [11:37] Gophers on the Fly and books getting [11:38] burned at a horrific rate but that's [11:41] still not good enough because we can [11:42] paralyze on another dimension and Go [11:44] full on so here's 16 Gophers moving [11:46] those books to the burning pile and it [11:49] obviously this is is very very [11:51] simplistic and silly but it's got [11:53] Gophers in it so that makes it good um [11:55] but I want you to understand that [11:57] conceptually this this is really how you [12:00] think about running things in parallel [12:02] you don't think about by running in [12:03] parallel you think about how you break [12:05] the problem down into independent [12:07] components that you can separate and [12:09] understand and get right and then [12:11] compose to solve the whole problem [12:13] together so what does this all mean well [12:17] first of all there are many ways you [12:18] could do this I showed you just a couple [12:19] if you sit there with a Sketchbook you [12:20] can probably come up with 50 more ways [12:22] to have gopher move books there's lots [12:24] of different designs they're not [12:25] necessarily all equivalent but they can [12:27] all be made to work and you can then [12:30] take those concurrent designs and [12:32] refactor them rearrange them scale them [12:34] in different dimensions to get different [12:35] abilities to process the problem and [12:38] it's nice because however you do this [12:40] the correctness of your algorithm for [12:42] doing this is easy there's really it's [12:44] not going to break I mean they're just [12:46] Gophers but um you know the design is [12:48] intrinsically safe because you've done [12:51] it that way however it's this is [12:54] obviously a stupid problem this has no [12:56] bearing on real work well actually it [12:57] does because if you take this problem [12:59] and you change the book pile into some [13:01] web content you change the Gophers into [13:03] CPUs you change the cart to the [13:06] networking or the marshing code or [13:07] whatever it is you need to run to move [13:09] the data and then the incinerator is the [13:11] web proxy or browser whoever you want to [13:13] think about the consumer of the data [13:14] you've just constructed the design for a [13:16] web serving architecture and you don't [13:19] probably don't think of your web serving [13:21] architecture as looking like that but [13:23] the fact this is pretty much what it is [13:25] right and you can see by substituting [13:27] the pieces this is exactly the kind of [13:28] designs that you think about and when [13:30] you talk about things like proxies and [13:31] forwarding agents and and buffers and [13:34] all that kind of stuff uh scaling up [13:36] more [13:37] instances they're on this drawing they [13:40] you just don't think of them that way so [13:41] they're not intrinsically hard things to [13:43] understand if Gophers can do it so can [13:45] we right so um let me now show you how [13:49] to use these ideas a little bit in [13:51] building things with go now I don't I'm [13:53] not going to teach you go in this talk I [13:54] hope some of you know it already I hope [13:56] lots of you go and learn about more uh [13:58] more about it afterwards but I'm going [14:00] to try to teach you a little tiny bit of [14:02] go and then hope the rest kind of gets [14:04] absorbed as we as we do it so go has [14:06] these things called go routines which [14:08] you can think of as being a little bit [14:10] like threads but they're actually [14:12] different and I rather than go into the [14:13] details of how they're different let's [14:14] just say what they are so let's say we [14:17] have a function that in this case takes [14:18] two arguments if we call that function [14:20] our in our program then we wait for the [14:23] function to complete before the next [14:24] statement executes that's very very [14:26] familiar you all know that but if [14:27] instead you put the keyword go before [14:29] you call that function then what happens [14:31] is that function starts running but you [14:33] get to run right away at least [14:34] conceptually not necessarily remember [14:36] concurrency versus parallel but [14:38] conceptually your program keeps running [14:40] while f is off here doing his thing [14:42] right and you don't have to wait for the [14:44] F to return and if you that seems [14:46] confusing just think of it as being a [14:47] lot like the Amper sand in the Shell so [14:49] this is like running F Amper sand off in [14:51] the background okay now what exactly is [14:55] a go team well they're kind of like [14:56] threads right because they run together [14:58] uh they're in the the same address space [15:01] within a program they are at least but [15:02] they're much much cheaper uh it's pretty [15:04] easy to to make them and they're very [15:06] cheap to to create and then they get [15:08] multiplexed dynamically onto operating [15:09] system threads as required so you don't [15:12] have to worry about scheduling and [15:13] blocking and so on the system takes care [15:14] of that for you and when a when a go [15:17] routine does need to block like doing a [15:18] read system call or something like that [15:20] no other go routine needs to wait for it [15:21] they're all scheduled dynamically so [15:23] they feel like threads but they're a [15:25] much much lighter weight version of them [15:26] and it's not an original idea other [15:29] languages and systems have done things [15:30] like this but we we gave them our own [15:32] name to make it clear what they are so [15:33] we call them go routines okay now I [15:35] mentioned we have to communicate between [15:37] these things [15:39] right so to do that we have these things [15:41] in go called channels which are like a [15:43] little bit like pipes in the Shell but [15:44] they have types and they have other nice [15:47] properties which we're not going to go [15:48] into here today but here's a fairly [15:50] trivial example we create a timer [15:52] channel uh and say it's a Channel of [15:54] time.time values and then we we launch [15:57] this this uh function in the background [16:00] that we sleeps for a certain amount of [16:02] time delta T and then sends on the timer [16:05] channel the time at the at that instant [16:07] timer time. now and then the other [16:10] process because the other go routine [16:12] because this one was launched with a go [16:13] statement can doesn't have to wait it [16:15] can do whatever it wants and when it's [16:16] ready to to hear that the other guy is [16:18] completed he says I want to receive from [16:21] the timer Channel whatever that value is [16:23] and the that go routine will block until [16:25] there's a value to be delivered and once [16:27] it is completed that will be get get set [16:29] to the time at which the other go [16:31] routine completed trivial example but [16:33] everything you need is in that one [16:35] little slide and then the last piece is [16:37] a thing called [16:38] select and what it does is it lets you [16:41] control your program's Behavior by [16:42] listening to by looking at multiple [16:44] channels at once and seeing who's ready [16:45] to communicate and you can decide to [16:47] rate in this case between channel one or [16:50] channel two and the program will behave [16:51] differently depending on whether channel [16:52] one or channel two is ready in this case [16:54] if neither's ready the default clause [16:56] will run which is me lets you sort of [16:58] fall through if nobody's ready to [16:59] communicate if the default clause is not [17:01] present in the select then you'll wait [17:03] until one or the other of the channels [17:05] is ready and if they're both ready the [17:06] system will just pick one randomly so [17:09] this this will come up a little later [17:10] but it's it's pretty much like a switch [17:11] statement but for communications and if [17:13] you know D's guarded commands it should [17:15] seem fairly familiar now I said go [17:18] supports concurrency and I mean it it [17:19] really supports concurrency it is [17:21] routine in a go program to create [17:23] thousands of go routines and we are once [17:25] debugging actually live at a conference [17:28] uh uh go thing that was running in [17:30] production that created 1.3 million go [17:32] routines and had something on the [17:33] neighborhood of 10,000 actually active [17:35] at the time we were debugging it and to [17:37] make this work of course they have to be [17:38] much much cheaper than threads and [17:40] that's kind of the point so they're not [17:42] free there's allocation involved but not [17:44] much and they grow and Shrink as needed [17:46] and they they're sort of well managed [17:47] but they're very very cheap and you can [17:49] think about them as being you know as [17:51] cheap as [17:52] Gophers uh you also need closures I [17:55] showed you a closure sort of under the [17:56] covers before um here's just proof that [17:59] you have them in the language because [18:00] they're very handy in concurrent [18:02] expressions of things to create nons [18:03] procedures so you can create a a [18:05] function that here in this case composes [18:07] a couple more functions returns a [18:08] function it's just to show that it works [18:10] and and their real closures Ino so let's [18:14] use these elements to build some [18:16] examples and I hope you'll learn a [18:18] little bit of concurrent go programming [18:20] by osmosis which is the best way to [18:22] learn uh so let's start by launching a [18:25] demon um you can use a closure here to [18:27] wrap some background operation you want [18:28] done but not wait for it so in this case [18:31] we have two channels input and output [18:33] and for whatever reason we have to [18:34] deliver input to Output we but we don't [18:36] want to wait until the copying is done [18:38] so we say go funk for a closure and then [18:40] have a for Loop that just reads the [18:42] input values and writes them to the [18:44] output and the four range claws in go [18:48] will drain the channels so run until the [18:50] Channel's empty and then exit so this [18:52] little burst of code uh just drains the [18:54] channel automatically and does it in the [18:56] background so you don't have to wait for [18:57] it and you has a little bit of boiler [18:59] plate there but you get it's not too bad [19:01] and you get used to it um Let Me Now [19:04] show you a very simple load balancer a [19:06] very simple one and if there's time [19:08] which is I'm not sure there will be I'll [19:09] show you another one um but this is a [19:11] simple one so imagine you have a bunch [19:12] of jobs that need to get done and we've [19:14] abstracted them away here or maybe [19:16] concretize them into a work structure [19:18] with three integer values that you need [19:20] to do some operation [19:22] on so worker tasks what they're going to [19:25] do is compute something based on these [19:27] values and then I put a sleep in there [19:29] so that there's we guaranteed that we [19:31] have to think about blocking because [19:33] this this worker task May block an [19:34] arbitrary amount of time and way we [19:36] structure it is we have the worker task [19:38] read the input channel to get work to do [19:41] and have an output channel to deliver [19:43] the results so those are the arguments [19:44] to this function and then in the loop we [19:47] range over the input values doing the [19:49] calculation sleeping for some [19:51] essentially arbitrary time and then [19:52] delivering the output to the respon to [19:54] the the output to the guy who's waiting [19:57] so we have to worry about blocking so so [19:59] that's got to be pretty hard right well [20:00] there's the whole solution um the reason [20:03] this is so easy is that the channels and [20:06] the way they work along with the other [20:07] elements of the language let you express [20:09] these concurrent things and compose them [20:11] really well this what this does is [20:14] creates two channels an input Channel [20:15] and an output Channel which are the [20:16] things connected to the worker they're [20:17] all reading off one input Channel and [20:19] delivering to one output Channel and [20:21] then you just start up some arbitrary [20:23] number of workers notice the go clause [20:24] in the middle there all these guys are [20:25] running concurrently maybe in parallel [20:29] and then you start another job up that [20:31] says generate lots of work for these [20:32] guys to do and then you hang around in [20:35] this call function call receive lots of [20:37] results which will read the values [20:39] coming out of the output channel in the [20:40] order that they complete and because of [20:42] the way this thing is structured whether [20:44] you're running on One processor or a [20:46] thousand the job will run correctly and [20:48] completely and we use the resource as [20:50] well it's all taken care of for you and [20:53] if you think about this problem it's [20:54] pretty trivial but it's actually fairly [20:56] hard to write concisely in most [20:58] languages without concurrency [20:59] concurrency makes it pretty compact to [21:01] do this kind of [21:03] thing and more important it's implicitly [21:06] parallel although not you don't have to [21:08] think about parallelism if you don't [21:09] want to but it also can scale really [21:11] well there's no synchronization or [21:13] nonsense in there num workers could be a [21:15] huge number and the thing would still [21:16] work efficiently and the tools of [21:18] concurrency therefore make it easy to [21:20] build these kind of solutions to fairly [21:21] big problems uh also notice there was no [21:24] locking no mutexes all these things that [21:26] people think about when they think about [21:28] the old concurrency models they're just [21:29] not there you don't see them and yet [21:32] this is a correct concurrent and [21:34] parallelizable algorithm with no locking [21:36] in it that's got to be good right um but [21:40] that was too easy so let's see how we [21:42] doing yeah I got I got time to do the [21:44] harder one this is a little trickier but [21:45] it's the same basic idea but done much [21:47] more realistically so imagine we have a [21:49] load balancer that we want to write [21:51] that's got a bunch of requesters [21:52] generating actual work okay and we want [21:55] we have a bunch of worker tasks and we [21:57] want to distribute all these requesters [21:59] workload onto an arbitrary number of [22:02] workers and make it all sort of load [22:04] balance out so the work gets assigned to [22:07] the least lightly loaded worker so you [22:08] can think that the workers have may have [22:10] large amounts of work they're doing at [22:11] once it's not just one at a time they [22:13] may be doing lots and there's lots of [22:14] requests going on so it's a very busy [22:16] system this could be maybe on one [22:18] machine which is how I'm going to show [22:19] it to you but you could also imagine [22:20] that some of these these lines represent [22:22] network connections they're doing proper [22:24] load balancing architecturally the [22:26] design is still going to be safe the way [22:28] we do it so the what a request looks [22:31] like is very different now we have some [22:32] arbitrary function closure if you like [22:35] that represents the calculation that we [22:36] want to do and we have a channel on [22:39] which we're going to return the result [22:41] now notice a channel is part of the [22:42] request in go unlike a lot of the other [22:45] languages like erlang the channel idea [22:48] is there and it's a first class value in [22:50] the language and that allows you to pass [22:52] channels around and they're kind of like [22:54] file descriptors in the sense that if [22:55] you have the channel you can communicate [22:57] with someone but any no one with anyone [23:00] who does not have the channel is not [23:02] able to so it's like you know being able [23:04] to pass a phone call to somebody else to [23:05] do or to pass a file descriptor over a [23:07] file descriptor it's pretty a pretty [23:10] powerful idea so the idea is you're [23:11] going to send a request with a [23:13] calculation to do and a channel on which [23:15] to return the result once it's [23:18] done so what here's an artificial but [23:21] somewhat illustrative version of the [23:23] requester uh what we do is we we have a [23:26] Channel of requests that are coming in [23:27] and we're going to generate uh stuff to [23:30] do on that work channel so we make a [23:33] channel which is going to go inside each [23:34] request to come back to us for the [23:36] answer we do some work which I've just [23:39] represented here is sleeping who knows [23:41] what what it's actually doing um and [23:43] then you send on the work Channel a [23:45] request object with the function you [23:47] want to calculate and whatever that is I [23:50] don't care and a channel on which to [23:53] send the answer back and then you wait [23:55] for the on that channel to get the [23:56] result to come back and once you've got [23:58] that you probably have to do something [23:59] with the result so this is just [24:01] something generating work at some [24:02] arbitrary rate it's just cranking out [24:04] results but it's doing it by [24:05] communicating on Channels with with [24:07] inputs and [24:09] outputs uh and then the worker it's [24:11] which is on the other side of this [24:13] picture remember we've got requesters [24:15] oops we've got requesters delivering [24:16] data to the balancer which is the last [24:17] thing I'm going to show you and workers [24:19] on the right and what the workers have [24:21] in them is a channel of incoming [24:23] requests and then a count of pending [24:25] tasks and which is going to represent [24:27] the load that that worker has the number [24:29] of tasks he's actually got busy and then [24:32] an index which is part of the Heap [24:33] architecture which'll show you in a [24:34] second so then what the worker does is [24:38] receive work from his request or his [24:40] request Channel which is part of the [24:42] worker object uh call the function on [24:45] the worker side so you pass it request I [24:47] guess from your point pass the actual [24:49] function from the requester through the [24:50] balancer into the worker he does the [24:53] answer and then he returns the the [24:55] answer back on the channnel now notice [24:57] that unlike lot of other load balancing [24:59] architectures the channels from the [25:01] worker back to the requester do not go [25:02] through the load balancer once the load [25:05] once the requester and the worker are [25:06] connected the bouncers is out of the [25:07] picture and the worker and the request [25:09] are talking directly and that's because [25:11] you can pass channels around inside the [25:13] the the system as it's [25:14] running okay uh and then if you want to [25:18] you could also put a go routine inside [25:21] uh put a go statement inside here and [25:22] just run all these requests in parallel [25:24] on the worker it would work just fine if [25:25] you did that but I that's enough going [25:27] on at once already [25:29] okay and then the balancer is kind of [25:30] magical uh you need a pool of workers [25:32] and you need some balancer object you're [25:34] going to put the balancers methods on [25:35] and that includes a pool and then a [25:37] single done Channel which is how the [25:39] workers tell the balancer that they' [25:41] finished their most recent calculation [25:43] so then the balancer is pretty easy what [25:46] it does is it just forever does a select [25:48] statement waiting either for more work [25:50] to do from a requester in which case it [25:53] dispatches that request to the most [25:54] likly loaded worker or a worker tells [25:57] him he's done in which case you update [25:59] the data structure by saying the [26:00] balancer is that worker has completed [26:02] his task so it's just a simple two-way [26:05] select uh and then we just have to [26:07] implement these two functions and to do [26:09] that we actually what we actually do is [26:11] construct a heap uh I'll skip that bit [26:13] it's not very exciting you'll get the [26:15] idea um dispatch all dispatch has to do [26:19] is grab the least load of worker which [26:20] is a standard priority que [26:21] implementation on a heap so you pull the [26:23] most lightly loaded worker off the Heap [26:26] you send it the task by writing the [26:28] request to its request Channel now you [26:30] increment the load because he's got one [26:31] more guy you know about and that's going [26:33] to in influence the the loading [26:35] distribution and then you push it back [26:37] onto the same spot on the Heap and go [26:40] and that's it you've just dispatched it [26:41] you've updated the data structure and [26:43] that's what four executable lines of [26:45] code and then the completion task which [26:48] is when the work is finished you've got [26:50] to do the sort of inverse you there's [26:52] one F guy on this workers CU so you [26:54] decrement his pending count you pop him [26:56] from the Heat and you put him back on [26:57] the heat and that'll put him back fact [26:58] where belongs in the priority que and [27:00] that's a complete implementation of a [27:02] semi-realistic load balancer but the key [27:05] Point here is that the data structures [27:07] are using channels and go routines to [27:10] construct this concurrent thing and the [27:13] result is scalable it's correct it's [27:16] very simple it's got no explicit locking [27:19] and yet the architecture just sort of [27:21] makes it all happen and concurrency is [27:23] therefore enabled parallelism intrinsic [27:25] in this thing and you can actually run [27:27] this program I have this is all [27:28] compilable and runable um and it works [27:31] and it does low balancing per and that [27:33] things all stay at exactly uniform load [27:35] module quantization it's pretty good and [27:38] you can of course have I never said how [27:39] many workers there are or how many [27:41] requesters there are there could be one [27:43] of each and 10 of the other or a [27:44] thousand of each or a million of each [27:46] the scaling still works and it still [27:48] behaves [27:50] efficiently one more example um which is [27:54] somewhat more uh surprising but it fits [27:56] on a single slide so it's a nice one to [27:57] finish imagine you had a a replicated [27:59] database so you got a database with with [28:02] uh the same data in each of multiple uh [28:05] what we call shards at Google same same [28:07] instance right and what you want to do [28:11] is deliver a quest to all of the [28:13] databases and a query and get back the [28:16] result but they're all going to be the [28:18] same you're using this to go faster by [28:20] picking the first guy answer as first [28:22] guy to come back with the answer is the [28:24] one you want so if one of them's down or [28:26] disconnected or something you don't care [28:27] because somebody else will come in so [28:30] here's how to do that this is the full [28:31] implementation of it um you have some [28:34] array of connections and some query you [28:36] want to [28:37] execute um you create a channel uh which [28:41] is buffered to the number of elements [28:44] the number of replicants inside this [28:46] replicas inside the the query database [28:49] and then you just run over all of the [28:51] connections to the databases and for [28:54] each one of them you start a go routine [28:57] up to deliver the query to that channel [29:00] to that database and then get the answer [29:02] back by this due query call and then [29:05] deliver the answer to the single channel [29:08] that that's holding the result for all [29:10] of these guys and then after you've [29:12] launched them all you just wait on the [29:14] bottom line there and we wait for the [29:16] the first guy that comes back on that [29:17] channel is the answer you want you [29:19] return it and you're done and the thing [29:22] is this looks like a toy and it kind of [29:25] is but it's actually a complete correct [29:26] implementation the one thing that's Miss [29:28] is is clean tear down you want to sort [29:30] of shut tell the servers that haven't [29:32] come back yet when you've got an answer [29:33] that you don't need them anymore and you [29:35] can do that it's it's more code but not [29:37] an unreasonable amount more but then it [29:39] wouldn't fit on the slide so I just want [29:40] to show you this is a fairly [29:42] sophisticated problem to write in a lot [29:44] of systems but here it just sort of [29:46] Falls naturally out of the architecture [29:47] because you've got the tools of [29:48] concurrency to represent a fairly large [29:51] distributed complex problem and it works [29:54] out really [29:55] nicely so I 5 Seconds left that's good [29:59] uh conclusion concurrency is powerl but [30:01] it's not parallelism but it enables [30:04] parallelism and it makes parallelism [30:06] easy and if you if you get that then [30:09] I've done my job so if there if you want [30:11] to read more there's a bunch of links [30:12] here uh there's a golang.org has [30:14] everything about go you want to know uh [30:16] there's a nice history paper that Russ [30:18] put together that's linked there uh I [30:20] gave a talk a few years ago that led up [30:22] to us actually doing go which you might [30:23] find interesting uh Bob Harper at CMU [30:26] has a really nice uh blog posting called [30:28] parallelism is not concurrency which is [30:30] very similar to the idea that [30:31] concurrency is not parallelism but not [30:33] quite uh and then there's a couple other [30:35] things the most uh surprising thing on [30:37] this is the concurrent power series work [30:39] that Doug math my old boss at Bell Labs [30:41] did which is an amazing amazing paper [30:44] but also if you want a complete [30:45] different spin on it the last link on [30:46] the slide is to another language called [30:48] sawal which I did at B at at Google [30:52] shortly after coming there from Bell [30:54] labs and it's remarkable because it is [30:56] incredibly parallel language but it has [30:58] absolutely no concurrency and by now I [31:00] think you might understand that that's [31:02] possible so thanks very much uh for [31:05] listening and thanks to herok for [31:07] inviting me and uh I guess it's time to [31:09] have some drinks or something