Francois Chaubard Predicts Backpropagation Death

Francois Chaubard预言反向传播走向终结 数字硬件与Transformer注意力机制的物理扩展极限，正促使计算基质向光学、神经形态和生物方向转变。 现代人工智能受制于软硬件的协同适配问题，以及数据中心因在光电信号之间转换而消耗99.99%电量的能耗瓶颈。为绕开GPU和反向传播的局限，Francois Chaubard提出了零阶优化（SPSA）与SOMA，仅依靠前向传播来训练分片的小型专家混合模型。Ilker Oguz展示了无源光学器件如何利用物理光传播以线性能量扩展特性运行扩散模型，尽管目前该技术仍面临ADC/DAC转换瓶颈。Alok Vasudev探讨了神经形态计算源自20瓦生物功耗的启发、漏电积分-发放动力学，以及通过解决互连电容问题而具备商业可行性的存内计算架构。最后，Sean Cole详细介绍了Cortical Labs如何基于Karl Friston的自由能原理，利用闭环强化学习和特意受限以防止硅芯片作弊的线性解码器，训练微电极阵列上的活体人类脑细胞玩《毁灭战士》（Doom）。 过去两年中，GPU每焦耳GigaFLOPs的能效提升已趋于停滞，这推动了对非反向传播范式以及光学、神经形态和生物计算等非传统物理基质的研究。 (0:07) 光子矩阵-向量乘法的能耗呈线性扩展（O(N)），而非像电子计算那样呈二次方扩展（O(N^2)），但光学处理器仍受到ADC/DAC信号转换所产生的巨大能耗和延迟开销的制约。 (15:06) 像d-Matrix这样的数字存内计算架构如今在商业上取得成功，是因为它在标准CMOS工艺中将SRAM直接与逻辑单元集成在一起，从而规避了互连电容带来的能耗代价。 (54:24) 在59通道微电极阵列上培养的活体人类脑细胞在Karl Friston自由能原理的指导下学会了玩《毁灭战士》（Doom）；在接受可预测的低熵与随机的高熵电反馈时，它们会重组物理突触。 (64:24)

As silicon-based GPUs encounter energy efficiency plateaus driven by transformer scaling and memory bandwidth limits, researchers are exploring alternative non-von Neumann paradigms including zero-order optimization, analog optical processing, neuromorphic hardware, and in vitro biological compute. Presenters at the YC Paper Club argue that achieving the extreme efficiency of biological brains requires replacing GPU-adapted backpropagation with substrates and algorithms co-designed for physical dynamics. Overcoming Limits of Backpropagation and GPUs (3:38) Francois Chaubard argues that AI hardware efficiency gains in GigaFLOPs per joule have plateaued over the past two years because accelerator design pivoted from raw compute to managing the memory capacity and bandwidth demands of transformer attention mechanisms. In contrast, the biological brain operates on roughly 20 watts via feedforward mechanics without backpropagation, avoiding the biological "weight transfer problem." Because alternative media such as optical paths and biological wetware lack native mechanisms for gradient accumulation or digital weight storage, moving beyond GPUs requires abandoning analytical backpropagation. Photonic Computing Advantages and Domain-Conversion Bottlenecks (15:06) Ilker Oguz explains that optical computing leverages photons to achieve roughly 10,000 times lower loss and 10,000 times greater bandwidth than electronic wires, enabling passive, interference-free parallel processing through space and optical frequency combs. However, commercial adoption is bottlenecked by the energy tax of digital-to-analog (DAC) and analog-to-digital (ADC) converters, thermal calibration, and the absence of rapid, non-volatile optical memory. These interface penalties led firms like Lightmatter to pivot compute engines toward optical interconnects. To demonstrate a viable optical workload, Oguz’s team developed a diffractive optical diffusion model for image generation on datasets like MNIST. By dividing 1,000 diffusion steps into 10 subunits of 100 steps with fixed, passive weights, the system achieved power-law scaling and single-image energy advantages over GPUs without requiring dynamic weight changes in hardware. Neuromorphic Principles Versus Silicon Physical Constraints (42:54) Alok Vasudev identifies three core neuromorphic principles: intertwined memory and compute, event-driven signaling via mixed-signal spikes, and dynamic structural plasticity. Modern deep learning abandoned biological fidelity because digital silicon scaled most effectively through synchronous matrix math and backpropagation. In physical chips, interconnect trace capacitance exponentially exceeds transistor gate capacitance, forcing accelerators to burn the majority of their energy simply moving bits across wires. Neuromorphic R&D currently splits between pragmatism and frontier exploration. Practical approaches like d-Matrix co-locate SRAM directly next to compute logic in standard CMOS to bypass interconnect bottlenecks during inference. Frontier research explores coupled oscillators, memristors, and thermodynamic computing, treating circuit noise not as an error to suppress, but as an active computational resource analogous to Boltzmann machines. In Vitro Biological Computing and Closed-Loop Learning (64:24) Sean Cole presents work with Cortical Labs training human neurons cultured on 59-channel multi-electrode arrays to play the video game Doom. Moving beyond the handcrafted logic of earlier Pong demonstrations, the complex 54-action Doom environment required an end-to-end closed-loop Proximal Policy Optimization (PPO) architecture. A CNN-MLP encoder translates game states into electrical frequency and amplitude signals, while the silicon decoder is deliberately constrained to a strictly linear readout with bounded weights to prevent silicon biases from playing the game autonomously. Training biological cultures relies on Karl Friston's Free Energy Principle, where living cells self-organize to minimize entropy and surprise rather than seeking reward. Predictable, synchronous stimulation serves as positive feedback, while asynchronous stimulation delivers negative feedback. Because biological tissue introduces massive intrinsic stochasticity, the training framework utilizes entropy penalties rather than exploration bonuses, with a silicon critic modulating feedback via temporal difference errors to avoid overwhelming the living culture. The presenters conclude that while optical, neuromorphic, and biological substrates demonstrate massive theoretical energy advantages over GPUs, all remain bounded by severe physical bottlenecks—including domain conversion overhead, volatile storage, fabrication limits, and unproven scalability. Displacing silicon accelerators requires co-designing architectures around the native physics of alternative media rather than forcing transformer backpropagation onto non-native substrates.

[3:38] Francois Chaubard: burn 99.99% of the jewels [10:17] Francois Chaubard: Don't worry, the reviewers, they always separate the wheat from the chaff. There's like, you know, perfect F1 score. So I trust that. [10:17] Francois Chaubard: make my PhD worth something [13:45] Sean Cole: No, it's just an item in Dota. [22:23] Ilker Oguz: letting the physics do its work, [23:54] Ilker Oguz: Unfortunately, still we are using backpropagation to do that, but maybe it will change, [38:59] Alok Vasudev: I thought we would never find a use for this stuff, and you know, here we are. [39:00] Alok Vasudev: several nines degrees of accuracy [47:04] Alok Vasudev: ultimate model-hardware co-design [54:24] Alok Vasudev: They're kind of makeable now. It works... and it feels like you can land the plane in terms of the market taking them up. [67:48] Sean Cole: every single moving part in the system is actively affecting how the cells process information and give us outputs. [80:10] Sean Cole: shitty answer to that would be I don't know how scalable it is [80:10] Sean Cole: maybe a better kind of answer to that is we're trying to find out

[0:07] Okay, welcome to Alternative Compute [0:10] Club. So, this night started with a [0:13] dinner with a close friend of mine who [0:16] um posed this prompt to me. I thought it [0:18] was a really interesting prompt that you [0:20] guys can take with you to your dinner [0:21] parties. Um it was like, if you could [0:23] only ask the aliens one question, these [0:26] super intelligent aliens, one question, [0:28] what would you ask? And so we're [0:30] thinking about, you know, maybe how do [0:33] you solve power? And I was kind of [0:34] thinking, I think we kind of know Dyson [0:36] spheres or I nuclear or something like [0:38] that. Um but I thought it was more [0:40] interesting about how do you how do they [0:43] compute their flops, their floating [0:44] point operations. And so, um, [0:50] I think that we're so far down this [0:52] rabbit hole of hardware, software [0:54] co-adaptation, or more specifically, [0:56] hardware architecture, and optimizer [0:58] co-adaptation. And I'm going to give [1:00] I've been doing deep learning since [1:01] roughly 2012. I'll give a little bit of [1:03] like my understanding of what happened [1:06] in the last, you know, 14 years. Um, and [1:10] so [1:12] roughly, let's say 2012, uh, Alex Net [1:15] comes out. Uh, CNN's probably had their [1:17] era up until maybe 2019, 2020. And at [1:22] that time, I don't think per personally, [1:24] you know, we Focal Systems, my company, [1:27] went through this um, Nvidia, you know, [1:29] um, kind of thing. They gave us $125,000 [1:32] for winning this like, you know, [1:33] whatever competition. We interfaced [1:35] somewhat with Jensen. My understanding [1:37] is that he really didn't care about AI [1:40] at all until 2020. Um before then it was [1:44] there was a really hot topic uh in [1:46] Silicon Valley called crypto and that [1:48] made a lot more money until SPF. [1:51] Um and you know and so uh I think he was [1:55] most mostly focused on flops on getting [1:58] more compute per jewel more flop flops [2:02] per jewel. um and specifically for CNN's [2:06] since they were not memory bound and [2:08] they're not uh bandwidth bound um and [2:11] also for um uh not Bitcoin as Alok uh [2:16] corrected me on but Ethereum. Okay. So, [2:19] and then after 2020, um, they started to [2:22] figure it out and they, I think it was [2:25] the GPT3 stuff, maybe GPT2 really put it [2:28] on his radar and said, "These [2:29] transformers demand more memory capacity [2:32] and bandwidth than ever before." And [2:34] then they launched uh, Ampear was really [2:37] the first one of of of this of this um, [2:40] ilk. And so what happened is they [2:42] started to care a lot less about compute [2:43] efficiency and a lot more about memory [2:45] capacity and memory bandwidth. and [2:47] memory capacity and bandwidth are in [2:48] this exponential. I wonder why. Well, [2:51] attention is n squ. So that's probably [2:53] why. And compute efficiency, well, as [2:56] long as it produces the flops, it's [2:58] going to, you know, uh, burn up all the [3:00] oceans in the world, but that's okay. [3:02] Um, and so this is roughly what's been [3:04] going on. And so this is hardware [3:07] architecture optimizer, code adaptation, [3:10] um, uh, happening in real time. So, and [3:14] then even worse, this is like looking at [3:16] uh from a G-flop G-lop G-flop perjule [3:20] over time, we've kind of petered out. In [3:23] the last two years, there really hasn't [3:24] been that much um improvement on this. [3:27] And so, we need something else. And I [3:29] spent my entire PhD thinking about one [3:32] really, how does the brain work as as as [3:35] number one, but two, what else? What are [3:38] there? There's no way that you know I I [3:40] was around and doing deep learning [3:41] before people knew about back prop. Um, [3:43] I think that in 10 years we'll stop [3:45] using Backdrop. And so I wanted to start [3:47] working on something else. And we'll get [3:49] into my stuff in a little bit, but [3:52] uh the the the premise is that the cost [3:56] per gigaflop has to come down by [3:59] significant amounts for to be human [4:03] level. The human brain draws like 20 [4:05] watts. Um, that's like a light bulb. Um, [4:08] and it can do insane amounts of of [4:11] things that the presenters tonight are [4:13] going to talk about. Um, and there's [4:15] some hope. There are people now that we [4:16] have this bifurcation of chips that are [4:19] training chips and that are inference [4:21] chips, then uh people are focused on [4:23] different things and not trying to make [4:24] everyone happy, but they're largely [4:26] still focused on the transformer [4:28] architecture. So, they're going to have [4:29] lots of um SRAMM uh maybe not HBM, but [4:32] definitely SRAMs will have a lot of [4:34] memory, which you know maybe isn't [4:36] required. Um, and so the brain is massly [4:39] feed forward. And so, uh, any argument [4:42] that I've tried to that I've listened to [4:44] that the brain does back propagation [4:47] where the would require that these [4:48] neurons that the this neuron here fires [4:51] forward and then it would have to fire [4:52] backwards. It would have to turn off the [4:54] brain and then fire backwards so that we [4:56] can update some the syninnapse here. Um, [4:58] and it certainly doesn't do that. Uh, it [5:01] certainly does freed forward. And to [5:02] make Hebian learning work or any of [5:04] these models, um there's something [5:06] called the weight transfer problem where [5:07] it does a forward pass and then in in uh [5:10] one uh synaptic pair and then has to [5:13] come back through a different one and [5:15] you have the weight transfer problem [5:16] where then weights have to be exactly [5:17] equal, which is how is that going to be [5:19] possible? Um it turns out that if you [5:21] put an activation thing in there, um and [5:25] they both see all the same signals, then [5:27] it kind of works, but it's still not [5:29] back prop. that's just a cyclic graph. [5:31] Um, which you know doesn't need doesn't [5:33] require retention of activations and all [5:35] those other things like that. Um, [5:37] there's some chemistry happening. So [5:38] there's some residual after the the [5:40] activation and things and timings and [5:42] things like that. Um, the other cool [5:44] thing that I'll reference later in that [5:45] happens in the brain that people don't [5:47] know about is that these things it's not [5:48] just big cobweb of connections. There's [5:52] massive inhibition in the brain. And so [5:54] the these are these are cortical [5:56] columns. There's bunch many of them in [5:58] the brain and they're basically these [5:59] assemblies of that are repeated all [6:02] throughout the brain with massive [6:03] inhibitions all around to allow each um [6:06] uh assembly to be uh to learn [6:09] independently. [6:11] Um and so there's some hope here. [6:13] There's there's a lot of ideas to one [6:17] use the brain that four and a half [6:18] billion years of evolution already [6:19] figured out to do with Tony Watts. Um we [6:22] have Sean who's who's going to be [6:24] talking about that teaching a brain to [6:26] to play Doom which is super super cool. [6:28] Um we have uh Ilker to talk about this [6:31] opt optical computing. I think if I make [6:34] an argument why optics is probably the [6:36] right answer. [6:38] Um [6:40] we communicate to the data center in [6:41] light then we hit a switch and then we [6:44] switch it back to electrons to tell it [6:46] where to go. we switch it back to light [6:48] and then we you know back to to the the [6:50] DAG DAC is that the DAC uh digital [6:53] analog converter and then it goes back [6:54] to light uh and then uh we switch back [6:58] with a a ADC and then we do burn 99.99% [7:03] of the of the jewels and then we switch [7:05] it back to light. Why don't we just do [7:07] the whole thing in light and it would be [7:09] instantaneous and it would draw no [7:12] watts. And so that's what I really think [7:14] the aliens are probably doing. But I'm [7:16] open to a lot of different ideas and so [7:18] neural computing is obviously really [7:20] really exciting and cool as well. Um and [7:22] then neuromorphics we have a LO uh [7:24] presenting on that as well which you [7:26] know is is basically trying to mimic um [7:28] what the brain is doing uh in analog uh [7:31] circuitry um which is super cool as [7:33] well. And then we still aren't done yet [7:36] because if we're trying to do back prop [7:37] in any one of those substrates it [7:38] wouldn't work because these things [7:40] storing bits in light is actually very [7:43] expensive. It's very hard. Um and you [7:46] know we'll have Ilker talk about that. [7:48] Um so the properties of this are really [7:50] really interesting. The cost per [7:52] gigaflop is way way low to do [7:55] computation in light is like free. You [7:57] can do lots of it. You just can't it's [7:58] hard to like store stuff. You have all [8:00] these other non-ifferiability uh uh um [8:04] activation issues and things like that. [8:06] Um but so the cost of VRAM of of VRAM of [8:12] per per gig of VRAM is very expensive [8:14] and bandwidth with light is pretty much [8:17] instantaneous with neural compute. No [8:19] neuromorphics just copper wire probably [8:22] could could do it as well. These will [8:24] have different properties. So [8:27] there's I spent my entire PhD so far [8:29] working on alternative optimizers. Um [8:32] again I was around before people knew [8:34] back prop. there's a bunch of other [8:36] learning procedures and I was curious [8:37] around what are the other how far could [8:39] we take um non-gradient-based [8:43] methods and so I spent I grabbed a zero [8:46] order to textbook and I literally [8:47] implemented every single method in the [8:50] my first six months of my PhD and this [8:53] the setting was train a 1 billion [8:55] parameter LSTM to do next token [8:56] prediction that's it all I want to do [8:58] what will get the best results and the [9:02] learning procedure the the update rule [9:03] that worked the best is something called [9:05] SPSA that pretty much no one knows about [9:07] except James Ball and his students that [9:09] invented this um learning update rule. [9:12] And basically if you remember calc 1 you [9:14] have two forward passes of some theta uh [9:19] some some theta model that you're going [9:21] to perturb in the random direction with [9:23] some radius epsilon uh plus and then [9:26] minus and you have a finite difference [9:27] and you scale you update by that random [9:31] direction by the finite difference and [9:33] you do that number of pertabbation [9:35] times. If the finite difference is zero [9:37] then you don't uh uh scale that [9:40] direction at all. If there's a huge [9:41] finite difference, then you scale in [9:42] that direction a lot. And that's [9:44] basically what happens here. So it's [9:45] called finite difference method, SPSA, [9:47] central difference method. These are all [9:49] kind of the same names for the same [9:50] thing. And what's cool about this, the [9:52] other property is there's no gradient. [9:54] So I can optimize 01 loss landscapes [9:56] completely fine. If this is a famous one [9:58] called a function and you can look at [10:01] back prop where it gets stuck in a local [10:03] minima [10:04] pretty quickly. Um, and uh, zero order [10:08] finds the the bottom of the of this very [10:10] non-lipshits bowl very quickly. Um, so [10:14] there's some cool properties there. So [10:15] we have the optimizer, we have a good [10:17] optimizer. Well, what do we try? What do [10:19] we train? What's the architecture? And [10:20] then on what compute are we going to [10:22] train it? The architecture I've been [10:24] playing around with a bunch. Why LSTMs? [10:26] LSTMs were designed to be to allow for [10:28] gradients to flow through it. So we [10:30] don't care about gradients anymore. [10:31] We're just doing forward passes. We [10:32] don't there's no gradient flowing [10:33] anywhere. Um and so maybe not uh LSCMs [10:37] and transformers I've tried it you know [10:39] transformer was really built for back [10:41] propagation is you basically are have [10:43] this parallelization in and during [10:45] during training which is so beautiful um [10:48] it was trained with back prop in mind it [10:49] was trained with GPUs in mind in fact um [10:51] I may go farther to say that so um there [10:55] might be and so I've been working on [10:57] this is what I'll be submitting to ilear [10:59] on Friday along with 60,000 other people [11:02] Um, [11:05] don't worry the reviewers they they [11:07] always separate the wheat from the [11:08] chaff. There's like, you know, perfect [11:10] F1 score. Um, so I trust that. Anyway, [11:13] um, this idea I call soma sharted [11:16] optimization mixture of assemblies. So [11:18] with this cortical column idea in mind, [11:20] what if we just take our common crawl [11:22] data set, uh cluster it with TF and some [11:26] SVD thing, cluster the data set and then [11:29] ship a parquet file to a GP to n GPUs [11:32] all throughout the world with n [11:33] clusters. So one uh parquet file per GPU [11:37] and just train a little expert, a little [11:38] LSCM, small one like 41K. And then at [11:41] test time you pull them all together and [11:43] then you can route with that little [11:45] router and you can it's like the same [11:47] benefit of except it's mixture of models [11:49] not not um uh just one layer feed feed [11:53] forward uh uh feed forward uh layers and [11:57] so this is basically the the the premise [11:59] how far can this go and the beauty in [12:01] this is one of the things I didn't say [12:03] here is this solver this optimizer the [12:08] error in the gradient estimate [12:11] uh becomes gets uh scales worse. It gets [12:16] worse and worse, more less and less [12:17] accurate as the model size grows and [12:19] bigger and bigger and bigger um linearly [12:21] so. And so you can't train big models [12:24] with this. If you do a 10 billion [12:25] parameter model, the number of [12:27] pertabbations you'll have to do is [12:29] extremely large, extremely infeasible, [12:31] certainly more than a forward pass and a [12:33] backward pass um in terms of flops. So [12:37] it gets really really uh compute [12:38] inefficient if you shard it. Then that [12:41] gradient noise is capped to some amount [12:43] by the size of each expert. And so you [12:45] only need maybe 64 pertabbations uh per [12:48] step or maybe even less um and it works [12:50] completely fine. And the cool thing is [12:52] we have these scaling laws that happen [12:54] in all three dimensions. We knew about [12:56] pertabbations. We knew about increasing [12:58] batch size to reduce the grain estimate [13:00] uh noise. We didn't know that if you [13:02] shard the model more and more, it [13:05] continually gets better and better and [13:06] better. And so why does this lose the [13:09] back prop? Because the cost of forward [13:11] pass is still really, really expensive. [13:13] Well, what if it wasn't? If we're doing [13:14] this in light, forward passes are [13:16] really, really cheap. It's just beams of [13:17] light. It's just doing computation. It's [13:19] like free. Um, if we're doing this in [13:21] neurons, it's 20 watts. If we're doing [13:24] this in in analog, it's whatever amps [13:27] you put it through, I guess. Um, and so [13:30] I'm really excited about hosting this [13:32] one uh because it will make my PhD worth [13:35] something. Um, so if none of these uh [13:37] people are successful, then my PhD will [13:39] be meaningless. And we should I should [13:41] have done back prop the whole time. So [13:43] I'm hoping on you to to solve it. So uh [13:46] I have three we have three great [13:47] speakers. Let me just introduce them a [13:49] little bit. Uh Ilker is a a PhD out of [13:53] EPFL in Switzerland. uh has worked on [13:56] optical uh AI computing for how many [13:59] years? [14:01] Five, six years. Um so probably one of [14:03] the the forefront uh uh thinkers on this [14:06] this concept um and is now doing his [14:08] postto for the last year uh at Stanford [14:11] uh where we got to meet. Um Aloc is a [14:15] PhD at a double E worked on [14:17] semiconductors and nano optics um and [14:20] has been a wildly successful uh venture [14:22] capitalist and now runs a $ 1.5 billion [14:24] fund. no big deal. Um, and uh, really [14:28] focuses on the frontier and early stage. [14:30] Um, and just like hang out with them. [14:32] Uh, he's, you know, brilliant and he can [14:34] gro any concept that we throw at him. [14:36] So, he offered to do neuromorphics. Um, [14:39] and then Sean just came through the bash [14:41] NYC. He got his masters in neuromorphics [14:44] and now the CEO of a cool company called [14:46] Parasma, which I asked him like, "What [14:48] does that mean? Is there some like deep [14:49] meaning of like what uh, you know, maybe [14:51] some neuroscience term that I didn't [14:53] know?" He's like, "No, it's just an item [14:54] in Dota." And that's kind of funny. All [14:58] right, Elker, you're up. Thank you so [15:00] much. [15:06] >> So, as France um introduced very well, [15:10] um we today do most of our data [15:14] communication in optics and we know it's [15:16] very efficient for that purpose. But [15:18] today I want to float the idea of how we [15:22] can use light for computing. What are [15:25] the advantages? What are the challenges [15:27] out there? And as a use case um I'll [15:31] give our study as an example in which we [15:33] show that we can program the light [15:35] propagation for doing diffusion based [15:38] image generation. [15:42] So as we all know photons are great for [15:44] carrying information and that's why we [15:46] have a field called photonix or optics [15:48] and communication is one of the biggest [15:51] applications of it. Nearly all of the [15:54] transmission between um AI data center [15:57] racks are now done with optical fibers. [15:59] So information is transmitted as photons [16:02] and 99% of the intercontinental data [16:06] transfer is also optical. [16:09] Why? And the answer is very simple and [16:12] comes from the fundamental physics. [16:14] Photons [16:16] has compared to uh electrons has about [16:20] 10,000 times lower loss and 10,000 times [16:24] larger bandwidth uh compared to [16:26] electrons. And that's why when we want [16:29] to transmit some information today [16:31] nearly more than one meter, we always [16:34] convert it to photons and then send it [16:36] over optical fibers. generally. [16:40] So if it provides such a low loss and [16:43] high bandwidth, why don't we process [16:46] information with photons as well? And [16:49] all uh what we have seen is that [16:52] actually the technologies out there [16:53] right now is at a convergence point [16:57] because historically electronic [16:58] processors were using very high [17:00] precision 32bits 64 bits representations [17:04] for doing information processing and [17:07] then these symbols would be converted to [17:09] lower bit depth uh two levels or four [17:12] levels and then transmitted in optics. [17:14] But what we see recently is that with [17:16] the um [17:19] emergence of AI is that uh we can [17:22] actually do our tasks with lower and [17:24] lower bit precision uh to be able to do [17:28] much more operations go faster and at [17:30] the same time optical communications are [17:34] able to support larger and larger bit [17:36] depths so that we can see that there is [17:38] an interface that is appearing between [17:40] these two domains. [17:43] Another fundamental advantage of optics [17:46] uh that exists for the computational [17:48] purposes is that by definition it is [17:52] massively paralyzable because photons [17:55] don't interact with other photons. So in [17:58] at a given space um as many as optical [18:01] beams can coexist without affecting each [18:04] other. On the other hand in electrical [18:06] circuits we have to define different [18:08] paths for each pet of signal. [18:12] So a quick comparison in terms of the [18:14] principles of compute in electronics. [18:16] Generally what we do is that for for [18:18] instance for matrix vector multiplier [18:20] operations uh we charge wires uh we [18:25] effect those wires with transistors and [18:27] then discharge them and at each step of [18:29] these computation we spend energy. So [18:32] for n dimensional inputs and n [18:34] dimensional outputs our energy [18:36] consumption generally correlates with n [18:38] squ on the optical end what we can do is [18:42] and what's a general practice in the [18:44] emergent studies is that we can modulate [18:48] directly our input information on n many [18:51] modulators and let it propagate through [18:54] a designed weight mask so that the [18:57] weights of this operation can modulate [19:00] the light beam passively without [19:01] spending any extra active energy and [19:03] then at the end the result of this [19:05] computation can be summed on the [19:08] detectors again through physics and the [19:11] sum values can be read out directly on [19:13] these detectors. So again the premise [19:15] there is the number of input units or [19:20] output units which is called n [19:22] correlates with the energy consumption [19:23] not n squ. [19:27] But if this is so great then why don't [19:29] we have optical computing everywhere? [19:31] Why isn't widespread and already there [19:35] are many companies out there trying to [19:37] make it a reality and there are very [19:40] competitive results coming out of them [19:41] as well. An example is light matter in [19:44] which they show them electrical opto [19:47] electronical PCIe board where they have [19:50] the compute numbers comparable to GPUs [19:52] and in terms of power efficiency as [19:54] well. And if you take a closer look into [19:57] their energy budget, what we see is that [19:59] actually what we spend on photonics is a [20:02] small portion of all the uh complete [20:05] budget of energy. Why? And the answer to [20:09] this will actually tell us what are the [20:11] challenges out there for optical [20:12] computing. [20:15] So uh one issue is that today our data [20:18] or models are in digital electronic [20:21] memories and if you want to transfer [20:23] them to our optical um processing units, [20:26] one of the challenges is that we pay [20:28] this large price for converting digital [20:31] information to analog and then optical [20:33] domain. And then once we do our [20:35] computation, we have to go back to that [20:37] those memories or transmit the [20:38] information which requires us to convert [20:40] the light to analog and then digital [20:43] domain again. and we pay a big price for [20:45] it and we see that in the uh budget [20:47] breakdown as well. Another um [20:51] expenditure item is that to be able to [20:54] program these devices and keep them [20:56] calibrated and precise, we also have to [20:59] actively spend energy to uh put the [21:01] parameters or control there. And another [21:05] bigger challenge especially for AI [21:07] workloads is that the passive optics [21:11] support linear operations in massive [21:13] scales. But when we want to do the [21:15] nonlinear activation functions, they [21:17] require additional mechanisms. They are [21:19] not as easily accessible as in the case [21:22] of transistors or digital electronics. [21:26] One of our tries was that we used short [21:28] pulses of very intense light in multim [21:30] mode fibers and the light matter [21:32] interactions there created optical [21:34] nonlinearities. But as you can see it [21:36] adds another layer of complexity to have [21:38] these nonlinear activations. So here the [21:41] take-h home message that I would get [21:43] from the first part is that for the [21:45] first demonstrations of useful optical [21:47] compute we have to carefully choose the [21:50] use case and the algorithm that we would [21:53] like to run on the optical machines so [21:55] that we can create a tangible advantage [21:58] compared to electronics and as a try on [22:02] this I'll talk about our ner paper um in [22:05] my previous laboratory in EPFL and it [22:07] was a collaboration with Google in which [22:09] we try to take the task of image [22:12] generation in the diffusion based models [22:14] and we will want to do it by programming [22:16] optical propagation as a den noising or [22:19] generation uh inference unit. [22:23] But first a very brief introduction to [22:25] diffusion based generative models just [22:27] to set the scene. They work by mapping [22:30] after learning a uh train neural network [22:32] they work by mapping a random [22:33] distribution of data. It could be [22:35] pixels, it could be action items, it [22:37] could be videos. [22:39] And these models take steps on these [22:41] random distributions with this train [22:43] neural networks to gradually transform [22:46] the distribution to a desired data [22:49] distribution. This could be images, this [22:50] could be other types of data sets. And [22:52] for the image case, what you can see is [22:54] that we get a random set of pixels. And [22:57] by repeatedly applying this neural [22:59] network on this random set of pixels, we [23:02] can create clean and realistically [23:04] looking images. Our proposition here is [23:07] that the using a general purpose [23:09] electronic hardware for this task [23:11] creates a big redundancy because we are [23:14] using a single train neural network on a [23:17] general purpose hardware again and [23:19] again. Sometimes it can go up to [23:20] thousand times for creating a single [23:22] sampling of a single image. And what we [23:25] propose here is that let's make an [23:27] application specific and passive device [23:29] so that the physics of the propagation [23:31] of light inside this device takes a step [23:34] on the diffusion procedure. And by re uh [23:38] iterating this architecture or [23:41] reiterating on this optical system, we [23:44] can start from a random distribution of [23:45] pixels and by just letting the physic do [23:47] its work, we can create uh new samples [23:50] from the desired data set. [23:54] How does it work? Just giving a a bit of [23:56] a closer look into the system. It just [23:59] depends on the definition and the main [24:01] components that are required are that we [24:04] know the how the physics work and we [24:06] have a way of manipulation manipulating [24:08] it and unfortunately still we are using [24:10] back propagation to do that but maybe it [24:12] will change. But then the goal is that [24:14] we define a target function as in the [24:16] neural networks case and we let the [24:19] physical system to be designed so that [24:21] they perform the task at hand. Here it [24:23] is. We provide noisy samples to this [24:27] physical system and we would like to [24:30] have the predicted noise term in those [24:32] images so that as the light propagates [24:35] through the system, what we end up is a [24:38] filtered version of the image and by [24:39] using it over and over again, we can [24:41] subtract the noise term and at the end [24:43] create a clean image with the system. [24:46] And what you see as the layers are [24:48] basically transparencies that modulate [24:50] the phase of the light. So that as they [24:54] prop as the light beam propagates [24:55] through this um set of layers uh they [24:59] defract and only the information that we [25:02] would like to end up with is [25:03] transmitted. Uh one key property of [25:06] optics that I mentioned was that it's [25:08] hard to program it or hard to change the [25:11] weights and we designed our generation [25:14] algorithm around it. Instead of adding [25:16] uh time dependent activation [25:18] manipulations like the digital neural [25:20] networks do in this case we created [25:23] different subset of um schedule [25:27] sub uh units let's say and if you are [25:30] taking 1,000 steps over time we divided [25:33] these 10,000 steps to 100 steps each uh [25:36] subunits and for each of these 100 [25:40] subunits we kept the parameters fixed. [25:42] So this means that if you have 10 of [25:44] these devices, you could basically use [25:46] one for 100 times, pass the activation [25:48] to the other one, use the other one for [25:50] 100 times and overall without changing [25:52] any weights on the uh physical hardware, [25:54] we could do all this uh procedure of [25:58] different noises and uh denoising of [26:01] over the process. For now the because of [26:04] the limited number of parameters we show [26:07] small data sets like Mnest digits or [26:10] fasion emnest. But what we can clearly [26:12] see is that this model or this optical [26:15] device behaves as an diffusion model and [26:18] by starting from a random distribution [26:20] it can gradually generate images uh that [26:22] are similar to the original data set and [26:24] we can see that with uh image generation [26:27] quality metrics. So we have both the [26:28] diversity and quality and that is shown [26:31] by the FID going lower. But maybe one [26:34] more interesting property is that uh [26:36] when we looked into the behavior of the [26:38] system in terms of the number of [26:39] parameters when we scale the number of [26:41] defractive layers on this system or [26:43] maybe change the number of pixels at [26:45] each layer. What we have seen is that it [26:48] shows a similar power low scaling uh [26:52] behavior when compared to the digital [26:54] neural network. So this means that we [26:56] expect for these models to scale in a [26:59] similar way as we add more parameters to [27:01] it and as you remember the promise of [27:05] optical computing is that as we add more [27:07] par parameters actually the energy [27:08] consumption scales more preferably and [27:11] we could also show that by adding these [27:14] extra parameters we can in a uh [27:17] expectable way the performance improves [27:19] in terms of the image quality. So what [27:22] is the bottom line? The bottom line is [27:24] that when we take took this small model [27:26] and compared with similarly performing [27:30] models on the digital neural network on [27:32] a commercial GPU what we have seen is [27:34] that for generating a single image we [27:38] had a significant energy performance [27:40] energy advantage when we used our [27:42] optical system here there's an asterics [27:44] we assumed our weights passive meaning [27:47] that u just micr fabricating these [27:51] weight layers and putting them there so [27:53] that once they are trained they are [27:55] fixed and they're eternally the same [27:57] which is uh currently easily can be [27:59] done. In our experimental implementation [28:03] uh we took an easier prototyping [28:05] approach. We used a device called [28:07] special light modulator. This is [28:09] generally used also in uh light [28:11] projectors and by coupling it with a [28:13] mirror we could actually show that we [28:15] can do a deep network on a single device [28:17] and um looking at our camera images and [28:20] input laser distributions we can do back [28:24] propagation on spot and update our [28:26] weights so that we don't need to go [28:29] through different phases of micr [28:30] fabrication and trying again. So it gave [28:32] us a useful means for prototyping this [28:35] system. But of course in the future for [28:38] discussing a useful optical compute [28:40] unit, we would want to produce these [28:43] layers and put it in place and [28:45] literature already shows that this is [28:47] quite accessible. Finally, um the points [28:50] that I want to make in this presentation [28:51] is that we could show that propagation [28:54] of light can be programmed for an [28:56] application specific relevant AI task [29:00] which is in noising diffusion based [29:01] image generation and I believe the [29:04] opportunity in optical computing is [29:07] through designing hardware and [29:09] algorithms together. In our case, we [29:12] hand tuned our generation algorithm and [29:15] we made sure that we repeatedly use the [29:17] same weights over and over again so that [29:20] we can really tap into the area where [29:22] optics is useful. And I think uh for [29:25] making this technology more practical [29:27] the next milestone is demonstrating an [29:29] end-to-end system and to do it in a very [29:33] large scale like a billion parameter [29:35] scale model so that we can really claim [29:37] these energy advantages or latency [29:40] advantages survive when the systems are [29:42] scaled out and um I can say that we show [29:45] the first premises that when we add more [29:48] parameters we expect them to perform [29:50] better as well as in the digital neural [29:51] network case. Thank you very much. [30:01] Sorry. Nice. [30:03] >> I worked on NY light matters uh the [30:05] first AI put silicon photonics [30:07] accelerator. One of the challenges I [30:10] worked exact exactly on the the DAC and [30:12] the ADC um subsystem that would do that [30:16] for the their what is the photonic [30:19] score. uh that would basically be the [30:21] offload engine for most tensor ops. One [30:24] of the problems was basically the ADC [30:26] and DAC resolution uh like you know [30:28] accuracy that would eventually end up in [30:30] like you know prediction correctness [30:32] over time basically in the operational [30:36] you know in field. Um there were there [30:39] was a project that basically I cannot [30:41] talk about but eventually that was [30:42] observed and which was which was the [30:44] reason why light matter eventually [30:46] pivoted to you know interconnects for [30:48] HBMs [30:49] >> um and kind of like uh like kind of like [30:53] stopped work on basically their [30:56] multisspectral compute uh arm of [30:58] research work. [30:59] >> Yeah, I I agree. I totally agree. uh [31:01] when there is a large number of [31:03] parameters or large large amount of data [31:06] compared to the number of parameters has [31:08] to be transmitted to the optical system [31:10] then the advantages generally gets [31:13] overtaken by the uh disadvantages that's [31:16] why I believe the first approach to this [31:19] type of hardware is not a general [31:21] purpose computing hardware like CPUs but [31:23] it's more AIC type uh application [31:26] specific approaches where we can really [31:28] go and minimize the as you said the bit [31:30] depth [31:31] the precision or the data rate as well. [31:34] >> So for the back prop, how do you [31:36] actually do that? Because you're saying [31:37] that I can do a forward pass where I [31:39] keep hitting the same uh like F pass [31:42] many times. I do I do my outer loop and [31:44] for my diffusion t times and so then but [31:47] then I need to go backwards. How how are [31:50] you do you flip it and then send the [31:52] beams backwards? How do you how does it [31:54] work? [31:54] >> There has been some work on that as [31:56] well. But in our case, we actually [31:58] calibrate a fully digital twin and [32:01] actually back propagation happens in [32:02] digital electronics which is a normally [32:04] the biggest bottleneck currently. [32:06] >> So um the other thing too that you could [32:08] do so basically diffusion is an RNN that [32:12] is teacher forced at every single step. [32:14] And so then if you remove that then [32:16] there is no like L1, L2, L up to L10, [32:20] LT. Um you can just like do treat it as [32:23] an RNN completely. um have you tried [32:26] that and and is there any advantage to [32:28] that um from like the ADC DAC issue of [32:32] reading things off and computing loss? I [32:36] haven't tried but I I agree that that's [32:38] also another very interesting future [32:40] direction because the usefulness of [32:42] optics I think depends on how small [32:45] interface that you do with electronics [32:47] or digital environment and in that case [32:49] if we can find a way of looping the [32:51] information in the optical system or in [32:54] the analog system for longer uh time [32:57] period or longer uh time steps then [33:00] definitely I think the real advantage [33:02] wouldn't be like seven times that we [33:03] showed but it could scale scale up to [33:05] hundreds or even more. [33:07] >> I'm very curious if you try to do this [33:09] with much much much much much larger uh [33:13] sort of uh weights uh what is the actual [33:17] physical challenge with manufacturing [33:20] kind of the relevant sort of components [33:22] and kind of how does that scale and what [33:24] kind of bottlenecks do we hit? [33:26] >> Mhm. So manufacturing wise I think it [33:28] uses a very similar technology stack [33:30] with electronics. So people showed with [33:34] etching and deposition that type of uh [33:36] simple chemical operations the [33:38] manufacturing could happen. Um of course [33:41] light has a longer wavelength compared [33:43] to electrons. So the unit size a bit [33:46] larger but I think those are all um [33:50] addressable challenges. Maybe one [33:52] limitation is that the nonlinearities [33:54] because if you want to have a very large [33:56] number of parameters what we benefit [33:59] from in the digital case is making deep [34:01] networks and in this case we need non. [34:04] So here the nonity is actually feeding [34:05] back to the modulator again. So we [34:09] should come up with uh cheap ways of [34:11] doing that nonity or maybe doing a [34:13] massive operation and finding a model [34:15] where it's useful. I think uh that's the [34:18] challenge making it very deep network [34:20] for increasing the parameter count. [34:21] Thank you. [34:22] >> Thank you. [34:24] >> Just a quick question. Electrons are [34:26] much smaller than like the typical like [34:29] wavelengths of visible light. So are [34:31] these kinds of photonic like chips [34:33] limited by that size? Like what is the [34:36] theoretical like minimum unit size in [34:38] optical compute? And is there like are [34:40] people trying to make that smaller? [34:43] >> Yeah. um people definitely try to look [34:46] into different directions in terms of [34:48] making better use of the space I would [34:51] say. So in electrons generally it's um [34:55] as as we were discussing the bandwidth [34:57] is smaller. So one for instance one [34:59] approach that uh many people took was [35:02] that using the same unit size which is [35:03] on the order of micrometers as opposed [35:05] to nanometers in the electronic trans uh [35:08] transistors but to use [35:11] for instance something called com [35:13] optical com. So this means that you have [35:16] hundreds of different wavelengths and [35:18] there are ways of encoding different [35:20] weights on each one of these wavelengths [35:23] so that you could do multiplexing not in [35:25] space but on the wavelength dimension. [35:28] Uh but essentially that's true that um [35:32] yeah the feature size should be larger [35:34] so that you can interact with the [35:35] visible light. Thank you. So I'm [35:38] traditional computer architect. Um so my [35:41] question is looks like we have optics [35:43] based uh processing units optics based [35:47] interconnects. What are your thoughts [35:48] about optics based storage in [35:51] traditional computing? We are working on [35:53] that from last like 70 80 years and we [35:55] make significant progress but if we can [35:57] have optics based storage we can bypass [35:59] that ADC at all. So what are your [36:02] thoughts on that? [36:03] >> Definitely I think that's one of the [36:05] biggest bottlenecks of the technology as [36:06] well. Um so actually optics as you know [36:10] CDs, DVDs is a great storage medium for [36:14] archival storage or for long-term [36:16] storage. Um it works quite well and [36:18] there are u many different ways of doing [36:21] that. But of course I think the [36:23] challenge is how can we quickly and [36:26] cheaply write and read memory in the [36:29] optical domain. Um historically there [36:31] has been lots of attempts on that. uh [36:34] holographic memories is one like [36:36] photorefractive materials there are [36:38] attempts at it but I think so far none [36:40] of them was as cheap and as accessible [36:43] as the electronics so that's why it's [36:46] one of the reasons why electronics [36:47] actually stayed as the main uh medium of [36:50] compute but I would say um the reason [36:54] why for instance we took application [36:56] specific and fixed weight is that is [36:59] exactly that because it's very expensive [37:01] to rewrite or keep the weights in the [37:02] optical domain Again u another line of [37:05] research that is looking into this is [37:07] phase change materials. They use them in [37:09] electronics but also in optics as well. [37:11] And uh this could be a way to solve that [37:13] too. So there are attempts at it but [37:16] none of them is as major and um easy to [37:19] use as NAND or like electronic based [37:22] memories. [37:23] Thank you. [37:25] >> Hi. Uh so one question that I had was uh [37:28] have you guys considered I think non [37:30] nonlinearity is something that is [37:32] difficult with light but I was curious [37:34] if you guys considered uh like you know [37:35] polarized filters which almost have like [37:38] a trigonometric relationship and like [37:40] potentially composing uh several like [37:42] polarized filters to do like you know [37:44] like almost like a how forier is able to [37:47] construct any signal with with [37:49] trigonometry. I was curious if that was [37:50] something that you guys had considered [37:53] uh working with. I think that that's a [37:55] good idea. So polarization is another um [37:58] dimensionality. So light has different [38:00] degrees of freedom. Uh intensity is one, [38:02] phase is one, those we used. But [38:04] polarization is another one. And any [38:07] manipulation there actually would [38:08] manifest itself on the amplitude domain [38:11] which we generally use with our sensors [38:13] or detect with our sensors as a [38:15] nonlinearity. I think that's a good one. [38:18] I think one has to think about also how [38:21] deep we can go with it because once we [38:23] modulate the polarization uh we get a [38:25] nonlinear response but if you want to do [38:27] a deep neural network it has to act on [38:31] each one of those layers each one of [38:32] those levels and I'm not sure once we [38:35] filter it how we can uh keep making it [38:38] nonlinear so [38:39] >> but I I don't know at least personally I [38:42] don't know a big research on that so [38:43] that could be very interesting yeah [38:45] >> thanks [38:46] >> thank you [38:47] >> all right thank you so [38:48] Thanks. You guys can chat more AFTER [38:59] I'm Aloque and um today we're going to [39:02] talk about neworph computing. I was told [39:04] 10 minutes so I'm going to try and keep [39:06] it under 10 minutes. [39:08] Um a bit about me. I am a co-founder and [39:12] a partner at a venture firm called [39:14] Standard. We're one and a half billion [39:16] in assets under management and we work [39:18] on early stage technology companies. Um, [39:21] new types of computers have been a [39:22] common thread in what I've worked on [39:24] over the course of my career and I don't [39:26] think there's been a more interesting [39:27] time to [39:30] build new computers than today. Um, a [39:33] little bit of street cred establishment. [39:36] um uh three degrees in double E [39:38] undergrad at UT Austin, master's PhD [39:40] here at Stanford. And my focus in my in [39:43] my research and work experience was [39:45] semiconductors, fetonics, and [39:47] nanomaterials. [39:49] Um [39:50] and part of why I left was I I thought [39:53] we would never find a use for this [39:55] stuff. And you know, here we are. Um, [39:59] okay. [40:01] So, uh, to to start talking about [40:04] neuromorphics, you always have to start [40:05] by glowing up the brain a little bit. [40:08] Um, it's miraculous. It's a marvel of of [40:11] evolutionary engineering. [40:13] Some of its properties, it's capable of [40:16] language, reasoning, and creativity [40:19] all at once. Um, it's multimodal. It's [40:23] real time. You can you can see it sees [40:25] it hears it acts um visual audio [40:30] physical embodiment all of these things [40:32] just just come with it and it's real [40:34] time. You can have conversations with [40:36] people and your brain is precomputing [40:38] what to say back and you have you know [40:40] millisecond hundreds of milliseconds [40:42] response times back and forth. Um it's [40:44] just a remarkable computer. Um it keeps [40:47] learning all throughout your life. No [40:50] context windows. um it just it just [40:53] keeps going and and adapts and learns [40:54] new things. Even things that evolution [40:56] never really prepared it for. Um cars, [40:59] computers, all these things uh the brain [41:01] didn't really have in its pre-train. And [41:03] and yet we can do it. Um it can just [41:07] learn from a couple examples. Uh my [41:09] favorite example here is if you show um [41:12] a 16-year-old a stop sign once or twice, [41:15] they'll recognize stop signs forever [41:17] with several nines degrees of accuracy. [41:20] But how many different angles and [41:22] pictures and shadows of a stop sign do [41:24] you need to show to a autonomous car to [41:26] get it to identify them reliably? Um, a [41:30] lot. And so again, it's it's a marvel of [41:32] of engineering. And um and and perhaps [41:35] the most miraculous thing is that it [41:37] just operates on about 20 watts, which [41:39] is an iPhone charger. Um and so um in [41:43] the in the late 1980s was we we [41:46] basically had enough advances in [41:49] neurobiology [41:51] understanding how the brain itself works [41:53] and enough advances in in electronics um [41:56] we could start to make chips, we could [41:58] start to make circuits and and do them [41:59] at scale where um um people started [42:03] thinking hey we should we should these [42:05] communities should intersect and and [42:06] maybe there are things we can learn in [42:08] designing our circuits and designing our [42:09] chips um from how the brain works. And [42:12] so, uh, Carver Meade at at Caltech in [42:15] the late 80s, who's a one of the [42:17] godfathers of of of the semiconductor [42:19] industry, um, he coined the term [42:21] neuromorphics and and his group did a [42:24] bunch of the seminal work and and [42:25] starting to really tease apart how the [42:27] brain works and mapping it onto circuit [42:29] models, um, for these things. Um, so [42:33] what is neuromorphic computing? Um at at [42:37] the highest level it is can we borrow [42:40] the brain's organizing principles [42:42] somehow in designing circuits and chips [42:46] and and things that we use to do [42:47] computation. Um what you will find if [42:50] you if you dig into the field more [42:51] deeply is that there are many people [42:54] have many different definitions for what [42:55] it means and depending on your goal you [42:58] kind of stretch the definition in a way [43:00] that either includes or excludes what [43:02] you're doing. Um but we can abstract a [43:06] handful of of highle principles um about [43:09] how the brain works um that we can kind [43:12] of use as as as [43:14] concepts um to anchor on for [43:16] neuromorphics. What are those [43:18] attributes? Uh number one it is [43:20] intertwined memory and computation. So [43:24] in the brain, the connections have [43:27] memory, they have weights, they play a [43:29] role in in computation itself rather [43:31] than just simply relaying signals back [43:33] and forth. Um that's something that's [43:35] really unique. Uh that's something that [43:37] we don't have in in in digital [43:39] electronics, for example. Um the way [43:41] that communication occurs and and almost [43:44] like state transition occurs is via [43:45] events. We call them spikes in the [43:48] brain. And um and it's it's a continuous [43:51] machine that that takes sensory input [43:54] and spikes get generated and propagated [43:57] throughout and uh and and that's kind of [43:59] the um the engine that drives things and [44:02] and spikes themselves are these bizarre [44:05] things that we don't really have any [44:08] sort of perfect mirroring to and in and [44:09] how we do computation otherwise. Um it's [44:13] it's this quasi it's you know mixed [44:16] signalish if you were thinking about it [44:18] like a circuit. It has parts of it that [44:20] are kind of digital, parts of it that [44:21] are kind of analog. Um and they all fit [44:24] together in this in this unique way and [44:26] and that's how the brain works. Um and [44:28] I'll get into that um a little bit in [44:30] into more depth in in a toy model of how [44:32] this works because it's a really [44:34] fascinating concept. And then um with [44:36] experience uh with learning it changes [44:39] both structure and function. um like [44:42] your brain is constantly rewiring itself [44:45] um as you learn things. And you know [44:47] when we make chips, they're they're [44:49] frozen. And so this idea that this this [44:51] this brain can can both logically and [44:54] and even physically adapt itself over [44:56] time in response to experiences um it's [44:58] it's a it's a remarkable um it's a [45:01] remarkable occurrence. [45:02] So [45:04] people have tried to apply these [45:07] learnings from the brain um onto onto [45:10] designing chips and and of course like [45:12] the neural network right the name neural [45:14] network obviously um [45:17] pays homage to this concept. Um and uh [45:23] oh sorry uh not there yet. So the uh I [45:26] wanted to go a little bit into the into [45:27] a toy model of what a spiking neuron [45:29] looks like. And so this is what I mean [45:30] by this kind of bizarre behavior that's [45:32] not quite analog, not quite digital. Um, [45:34] and so if you're if you're a circuit bro [45:36] like me, then uh you can you can think [45:39] about it like a leaky capacitor is one [45:42] way to think about it, but it's a leaky [45:43] capacitor that when certain charge [45:46] accumulates above a threshold, then it [45:48] fires something that's kind of digital, [45:50] a spike out. Um, and so, uh, these [45:53] spikes is basically like this this pulse [45:55] train where pulses come in and charge [45:58] accumulates on on the membrane of a [46:00] neuron, but then over time it leaks, it [46:02] dissipates. So, you see these curves, it [46:04] starts with a spike, but then it drops [46:06] off. But then if you have a certain [46:08] number of spikes that come in [46:10] coincidentally [46:11] where they the the charge that you're [46:13] accumulating on on on the on the [46:15] membrane of a neuron sums up really [46:18] quickly above a certain threshold then [46:20] the neuron emits a spike. Um but if not [46:23] enough get there then it kind of relaxes [46:25] and it resets. And so we we don't have a [46:28] circuit like that. And it you know it's [46:30] kind of analog in the sense that you [46:31] have these decays and leaks. Uh but it's [46:34] also kind of digital in the sense that [46:35] it it emits a spike in a in a discrete [46:38] way. Um you know the relationship in [46:40] time between when these spikes occur [46:43] matters a lot. That's actually [46:44] continuous variable. The amplitude of [46:46] the spike matters a lot. And so it's [46:48] it's just this this bizarre consequence. [46:50] But this is what's happening in your [46:51] brain over and over and over again. and [46:54] um and and and this these are one of [46:56] these uh this is one of the core [46:57] concepts that people have explored [46:59] various iterations of you know can we do [47:01] this in in electronics um okay so the [47:04] point I was making earlier which is um [47:06] you know people have tried to think [47:08] about what parts of this could actually [47:09] apply to doing useful computations and [47:12] doing things and the neural net is is [47:14] the most famous of course it is um brain [47:18] inspired [47:20] but that's about it [47:22] Um so when we think about like modern [47:24] AIs and and modern neural networks um [47:28] some concepts are absolutely neuro [47:30] inpired and borrowed from biology. Um [47:33] those include having large networks of [47:36] interconnected neurons. Um [47:40] as we learn we manifest those learnings [47:43] as changes in connection strengths [47:45] between these neurons. Um and when we [47:48] think about how information is encoded [47:49] in these networks um it you know [47:52] patterns across many neurons are how we [47:54] contain pockets of knowledge and pockets [47:57] of information. Um but one of the common [48:01] threads in in neuromorphic computing um [48:04] is that that it turns out that that [48:07] digital electronics is is really good [48:10] and we got really good at making them [48:11] and scaling them and making them [48:12] parallel and and it turns out that that [48:14] going native and actually leaning into [48:16] what can you do with this like highle [48:19] concept but its implementation in in [48:21] digital silicon that's where all the [48:23] real innovation has taken place and when [48:25] you look at a modern AI you have these [48:27] concepts like back propagation, [48:29] transformer attention, how we train them [48:32] at at massive scale um on digital [48:34] hardware, like none of these map to what [48:36] we have in the brain. So, we've already [48:38] kind of gone off off course and and and [48:40] we're doing things native to the [48:42] manifestation of these neural networks [48:43] in silicon rather than than kind of [48:46] staying staying true to um you know, [48:49] faithful to the brain. And so this to me [48:54] is is probably like the biggest takeaway [48:56] I had in neuromorphics. Um which is the [49:00] brain is this ultimate model hardware [49:04] co-design that's taken place like the [49:06] hardware and the software are one and [49:08] the same and um and they really have [49:10] co-evolved in this really unique way. [49:12] And so if you are going to take pieces [49:15] of it and try to apply it onto other [49:18] computing substrates, onto other types [49:20] of physics, you have to be very careful [49:22] because maybe some concept will apply at [49:25] a high level, but it turns out that [49:26] whatever is native to that form factor [49:28] is probably going to be the right way to [49:31] take it to the next level and improve [49:32] it. Um, [49:35] and so um, you know, where are we with [49:38] neuromorphics today? um in 2026 like my [49:41] assessment is that we are still squarely [49:43] in in R&D land and I think we are trying [49:46] to figure out again this this kind of [49:48] matchmaking problem of where can we take [49:50] the right level of inspiration from what [49:52] we find in the brain and where do we [49:54] pair it with some sort of physical [49:55] implementation and and figure out if we [49:58] can actually do some magic with it. Um [50:00] so there there are kind of three [50:02] highlevel ideas or premises that that [50:04] that I see people exploring right now. [50:07] Um, some of them are we've got a problem [50:10] that we're observing that we think that [50:12] we have an opportunity to solve by [50:14] leveraging some of these these [50:15] biological insights. Others are we're [50:18] coming up with a bunch of new toys to [50:20] play with and they have weird [50:21] properties. Um, maybe we can exploit [50:24] these weird properties by by combining [50:28] them with something that we derived from [50:30] biology. So, uh, the first one is is [50:32] obviously people look at the brain. I [50:33] mentioned it 20 watts extraordinarily [50:35] energy efficient. um modern AIS are not [50:38] quite energy efficient and so the idea [50:40] is is there something we can take away [50:42] from that property in order to um to to [50:45] make AIS far more energy efficient. Um [50:48] and so some of the ideas that are being [50:50] tried right now are co-mingling memory [50:52] and computation. Um [50:55] um people are exploring analog ideas for [50:57] even training and inference. Um there [51:00] you know and and I would say that this [51:01] is where like you stress the definition [51:02] of neuromorphics because you know this [51:04] is you can purely digitally is is how [51:06] people are thinking about co-mingling [51:08] memory and computation right you can [51:10] combine SRAMM and logic for example um [51:13] you know Dmatrix is is a startup that's [51:15] that's done some good work there. They [51:17] had a great hot chips presentation a [51:18] couple weeks ago. Um IBM has has done [51:21] some work here. Um but you know it's [51:22] purely digital. So is it is it [51:24] neuromorphic? Like yes if you if you [51:26] have a very broadtent view of of you [51:29] know intertwined memory and computation [51:30] as being neomorphic. We also have um you [51:33] know on on the edge for thinking about [51:35] different types of um chips and hardware [51:37] they need to sense and react and adapt. [51:40] Um, can we can we take some inspiration [51:42] from um the continuous [51:45] sensing that the brain does and apply [51:47] that to um how we have um a sensor on a [51:51] drone, something that's doing something [51:52] in real time? And so um people are [51:54] looking at at spiking networks. Um can [51:57] you train a neural net on on spikes [52:00] rather than conventionally? Um can you [52:03] have devices that are ondevice learning? [52:05] Can can the network itself actually [52:06] self-improve based on what it perceives? [52:09] Um, and and can we have like event- [52:11] driven computation in the same way the [52:13] brain works as there? Um, you know, [52:15] Intel has probably done the most work [52:17] here. They've they've got a a chip out [52:19] of their R&D lab um that that they've [52:22] done some some, you know, they've got a [52:23] drone that can fly itself um that that [52:25] can fly with it. And then last is [52:29] uh what I mentioned, we've got these new [52:32] toys, right? We've got we've got [52:33] fetonics. We've got a bunch of exotic [52:36] electronic devices that have been cooked [52:38] up in in semiconductor R&D labs over the [52:40] years. Things like resistive memories, [52:42] things like meristers. Um we've we've [52:45] got circuit ideas like coupled [52:46] oscillators. And all of these systems [52:49] exhibit very interesting new physics and [52:52] new characteristics. Um they have they [52:54] have some analog properties um [52:56] nonlinearities. And so the question is [52:58] can we can we start with these as as [53:00] kind of a why now technological [53:02] substrate and can we start to build [53:05] models around the specific physical [53:07] behavior of these systems um in order to [53:09] co-design something that actually uh has [53:12] a very interesting result or property. [53:16] This is the 30,000 ft view of the field. [53:19] Um I think it's been it's been a very [53:22] interesting field. it's been, you know, [53:23] perpetually a source of inspiration, but [53:25] then it's a little kernel of it here and [53:27] there, but then for the most part, um, [53:29] you know, we don't have anything that [53:30] really resembles the brain that we're [53:32] using in practice. And, um, it's to me [53:35] it's a very open question whether we [53:37] will. Um, but but I I think it's still a [53:40] really interesting research direction to [53:42] explore. Um, and you know, think about [53:45] something like neural nets, right? they [53:46] were they were very sleepy for basically [53:48] the first you know you know many many [53:50] years of of their existence and then it [53:52] took a while before we actually figured [53:54] out how to make them useful and so I I [53:57] there's just a gestation period for any [53:59] of this stuff and and so I think as a [54:01] research direction it's it's it's super [54:03] compelling um but you have to be patient [54:05] and and you have to be a little bit [54:06] lucky um so these are some of the [54:10] resources um that that that I found [54:12] really useful and helpful for directions [54:14] to go forward Um but with that, thank [54:17] you. [54:24] >> Hi, thank you for coming. Um my question [54:26] is uh given uh all that you've seen in [54:29] neuromorphic computing, what's the most [54:31] promising startup or project or [54:34] direction that you've seen uh take hold? [54:37] >> So I think I have to use the really [54:38] broad tent definition of neuromorphic in [54:42] order to answer that. Right. I think a [54:43] lot of the um a lot of the stuff that [54:45] you would consider like very brain [54:47] inspired is is still pretty early. Um I [54:50] think it's probably around the um either [54:52] the um co-mingling of of memory and [54:57] compute. Um I think I think there there [55:00] are bunch of startups that are doing [55:01] that again purely digitally. Um Dmatrix [55:04] is one um that feels like it's it's [55:08] they're kind of makeable now. It works. [55:11] um we can make we can manufacture them [55:14] and and it feels like you can land the [55:16] plane in terms of the market taking them [55:18] up. So that that's probably the answer [55:21] there. It's probably like a copout [55:22] answer to some degree because it's none [55:24] of like the real sci-fi stuff. Um on the [55:26] on the sci-fi stuff, I think that there [55:28] there a couple very ambitious startups. [55:30] Um um Naveen Ralph's company, for [55:32] example, is trying to do coupled [55:34] oscillators in pure semos. and and I [55:38] think that at least like they're holding [55:39] the constraint that we need to be able [55:40] to make these things at scale. Um, but [55:42] very interesting types of dynamics and [55:44] physics that you can come out of these [55:45] systems. So, I think that's a really [55:47] interesting direction, but again, it's [55:48] he's he's the kind of guy that can raise [55:50] enough money to actually go after it. [55:59] >> Thank you. Thank you for that elegant [56:01] presentation. I learned a lot. Thank [56:02] you. Um uh coming off of the hot chips [56:05] uh presentation from Dmetrix I think the [56:07] innovation around that is basically kind [56:09] of like co- packaging and I'm bringing [56:10] the SRAMM uh really close in the memory [56:13] hierarchy closer to the compute on the [56:14] decode side so that you know as we have [56:17] uh we are hitting the decode ceiling [56:19] they're trying to solve that problem. So [56:20] my question is that like um [56:23] having that basically increases the [56:25] surface area of the packaging and that [56:28] basically means that you have you have [56:31] uh you have lesser servers in the same [56:34] data center uh amount of data centers. [56:36] My question is that do you think that [56:37] there's innovation currently going on in [56:40] neuromorphic uh world to kind of address [56:43] you know packaging efficiency uh around [56:46] around that? I don't know if I consider [56:48] that neuromorphic [56:50] in terms of addressing that issue. I [56:53] think that there are a bunch of really [56:55] interesting solutions for addressing [56:57] that, right? Like the core problem is at [56:59] the end of the day um you know you have [57:02] the capacitance of an interconnect is [57:04] far greater than the capacitance of of [57:06] the gate of a transistor. And so the [57:09] bulk of your of your energy expenditure [57:11] is on moving bits, you know, across [57:15] wires far and wide rather than like [57:16] flipping the bit um of an individual [57:18] transistor. Um I [57:22] I personally think optics is going to be [57:25] how we address this. [57:29] >> If I just uh give a question here. So [57:33] I find it very uh like it seems like the [57:37] the goal of neuromorphics is to develop [57:40] circuitry or analog or optics or [57:43] whatever that um [57:46] outbrains the brain and like just [57:48] becomes the brain. And it's kind of like [57:49] trying to estimate a sinoid with a bunch [57:52] of lines. It's like the best way to [57:54] approximate a sinoid is just with a [57:56] sinoid. Like just use a sinoid. So like [57:59] then if we're really trying to mimic the [58:00] brain, then just use the brain, right? [58:04] Just use neurons, right? Is that is that [58:06] the best hope we have for neuromorphics [58:08] is neuro? [58:10] >> It's a it's a great question. Um and and [58:12] I think we've got a I think we've got a [58:14] talk that might speak to that coming up [58:16] soon. Um [58:17] >> that was a segue. [58:18] >> Yeah. No, but it's a it's an interesting [58:20] question. I think that there are [58:21] limitations [58:22] um of the brain, right? There are things [58:24] that we can do in computers that that [58:27] are way better than what we can do with [58:28] the brain, right? For example, like the [58:30] clock speed um of a computer is so much [58:32] faster than what we can do with a brain. [58:35] Um I think even in terms of memory, [58:37] right? We have like you know the [58:38] retrieval and things like that are [58:40] tougher but but you can you have [58:41] infinite storage capacity with a [58:43] computer whereas like you know we have [58:45] limitations biologically. Um, and so [58:49] like it's possible, but then the [58:50] question is like if we can get it, can [58:52] we scale it? Like maybe, but that's [58:55] certainly like a direction of of of [58:57] research that we that we've not really [58:59] gone down yet. Uh, thanks so much for [59:01] the great talk. Uh, my name's Arun. Um, [59:04] you talked a lot about software hardware [59:07] co-optimization when it comes to setting [59:10] up next generation comput systems. And I [59:12] was wondering given that the cycles the [59:15] iteration cycles for both uh are [59:17] collapsing uh and getting shorter and [59:19] shorter. Do you think it's more [59:21] important to iterate a model in software [59:24] and then pick the hardware platform that [59:26] best supports it or do you think it's [59:27] more important to find the best hardware [59:29] platform you can and then design a model [59:31] around that? [59:32] >> Oh, that's a good question. Um like the [59:34] the copout answer is it [59:36] depends, which is it does. Um, and I [59:40] think it depends on are you starting [59:41] with like an insight around the physical [59:44] property of some system and you know [59:46] we're really good at making an optical [59:47] computer and we deeply understand the [59:50] constraints of the system but we we [59:51] deeply understand where we can really [59:53] lean into its advantages and and [59:56] therefore we can't blindly apply a model [60:00] architecture to it. We need to work [60:02] within the strengths and weaknesses of [60:03] the system in order to design it. I [60:05] think similarly if you're if you're [60:06] approaching it the other angle saying [60:08] hey I've got a really unique [60:09] architecture that I think can really [60:11] work um then like now I need to pick [60:15] what's the right like physical [60:16] manifestation in in which to bring it to [60:18] life and and and so I think it just kind [60:21] of depends on on where your where your [60:23] angle comes from. [60:26] >> Yeah. [60:28] >> Thank you for this presentation. So my [60:30] question is u I I mean looks like all of [60:32] our electronics are silicon based [60:35] whereas brain are fundamentally carbon [60:38] based. Do you think like a lack of [60:39] research in that direction is why we [60:42] have to take uh like learnings from [60:44] brain and simulate on silicon powered [60:47] devices. [60:49] >> We're just really good at doing [60:51] computation in silicon. So I think it's [60:53] more like if we're trying to mirror it [60:55] as like an electronic circuit of some [60:56] sort then like you're doing it in [60:58] silicon. Um, I think that like over [61:00] time, if you look at our our the types [61:03] of materials we've incorporated into [61:05] chips, it's just gotten more and more [61:08] exotic over time, and that should [61:10] continue. Um, you know, there's supposed [61:12] to be there was a um, you know, there's [61:14] some there's a team that was supposed to [61:15] give a talk here tonight that's doing u [61:17] they're working on diamond as a [61:19] substrate. And so there you go, that's [61:20] carbon, but um, but but you know, so I [61:23] think that like new materials, this and [61:24] that, but I I wouldn't overly index on [61:27] like silicon versus carbon. And it's [61:29] more about like manifestation as a as a [61:32] um as electronics and silicon happens to [61:35] be our our workhorse for doing that. [61:39] >> All right, cool. We got time for one [61:40] more. [61:43] So, I'm actually really interested in [61:46] another factor that I would associate [61:47] with neuromorphic computing that maybe [61:49] sort of wasn't mentioned, but then I'm [61:51] interested what you think about it, [61:52] which is the role of noise and [61:54] reproducibility in computation. Uh and [61:58] because at anytime we're doing anything [61:59] with sort of traditional silicon [62:01] computing at the bit level, the [62:03] intention is for the computation to [62:04] either be reproducible or extremely [62:07] close to reproducible because of [62:08] parallelism sort of stuff like that like [62:10] little edge cases. Uh whereas if you [62:14] think about the grain brain it's in some [62:16] sense it's not trying to have a [62:17] perfectly like low-level reproducible [62:19] computation at the lower level. And so [62:21] I'm curious sort of what role that plays [62:22] both in the field as you've described it [62:25] um and sort of more broadly for the kind [62:28] of ideas uh discussed. [62:30] >> Yeah, it's it's a great question. Um [62:33] even just even determinism within [62:35] digital computation is very interesting, [62:36] right? As we get to the limits of [62:38] floating point precision, then anyway, [62:40] it's it's a very deep question. Um I [62:43] would say that there's one category of [62:45] of of of attempt that I've seen um is [62:49] thermodynamic computing which is again [62:52] actually if you look back to the origins [62:53] of neural nets people were even Jeff [62:55] Hinton was looking at these things [62:56] called Boltzman machines back in the day [62:58] and so I think this is almost like a [62:59] reincarnation of some of those ideas um [63:01] it's in analog circuitry but the idea is [63:04] can you can you put noise to work for [63:06] you in a useful way and the hard part is [63:08] you want some of the noise not all the [63:10] noise and and the noise introduced in [63:12] the system itself is can can muddy the [63:14] picture. Um but but to answer the [63:16] question I actually I think that's a [63:18] really interesting um place to think [63:19] about and um and and there are some [63:22] directions that that are leveraging that [63:24] and thinking very deeply about it but um [63:27] I think it's a very profound question [63:30] as per SPSA as long as you can make the [63:34] noise reproducible and it's premise off [63:37] the same seed then it actually is [63:38] completely learnable. Um you can do [63:41] forward difference if you make it [63:42] reproducible just twice if you do one to [63:45] two the perturb and then one to scale [63:47] the gradient and three times if you want [63:49] to do central difference and then it's [63:51] fine but the issue is making it [63:52] reproducible. I spent a lot of time with [63:54] Shaw Drman at Stanford who's uh one of [63:56] the leading neuroscientists over there [63:58] and he's like I can make it reproduc I I [64:00] can't make it reproducible is is really [64:01] the issue. I can't justify how you get [64:04] this seed to a vector twice. That's [64:07] really the the the the main reason why [64:09] zero order um is may perhaps not [64:12] biologically plausible. It's a good [64:13] question. All right, we got to move on [64:15] to neuromorph or to to to brain [64:18] inspired. Thank you and luck. [64:24] >> Hi everyone. Um so guys, some of you may [64:27] have seen human brain cells playing [64:29] Doom. So that's some of the work that I [64:31] did in collaboration with Cortical Labs. [64:33] So, uh, they kind of put brain cells on [64:34] chips and I wrote the algorithms for it. [64:36] Um, I think my goal here today is to [64:40] kind of [64:42] impart some of my thinkings of how we're [64:45] treating this as a dynamical systems [64:47] problem. um rather than trying to force [64:49] the brain to um or force the brain with [64:52] innate calculations like matrix [64:53] multiplications um we want to treat it [64:55] as kind of like an input output machine [64:57] where um some set of stimuli can you [65:00] know produce some set of intelligence [65:01] that allows us to get some form of [65:03] output that's useful computationally. [65:06] So a a rough kind of example of this is [65:09] you know how we talk to brain cells. Um [65:11] this is primarily through electrical [65:13] stimuli. So um in Doom in particular we [65:15] have a game state. Uh so this is stuff [65:17] like you can think of it as like scalar [65:20] vectors and so in Doom for particular we [65:23] have stuff like ammo health um but we [65:26] also have you know kind of an image of [65:27] the screen um so we we take all these [65:29] variables we encode them into a set of [65:31] stimulus so this can be um which channel [65:33] do we stimulate on this 2D multi-elerode [65:35] array as well as stuff like ampl [65:37] amplitude and frequency so we're [65:39] controlling all these variables um of [65:41] which you know I think we saw with the [65:43] kind of leaky integrated fire neurons [65:46] not exactly the same. Uh but these [65:47] neurons do, you know, kind of accumulate [65:49] charge and spike over time. Uh what we [65:51] do in Doom for this specific instance is [65:53] we decode those spikes into a specific [65:56] action in the game. So that's kind of [65:57] how we get useful compute out of these [65:59] cells. [66:01] Um I'll talk a bit about some of the [66:02] previous work. So sorry if this is a bit [66:04] small, but um so what Corticle did about [66:07] four years ago was Pong. So they got the [66:08] brain cells to play Pong. Um and Pong is [66:11] a relatively simple game compared to [66:13] Doom. Um what this means is you can kind [66:15] of hardcode how you want to do encoding [66:18] and decoding. Um so for Pong in [66:20] particular um they figured out you know [66:22] kind of which were the best encoding [66:23] channels. So if you stimulate these [66:25] channels it would give you more diverse [66:27] um outputs. So more kind of a separable [66:30] outputs for your decoder. Um and then [66:32] your decoder would kind of move the [66:33] region or move the paddle to the region [66:35] where it thinks the ball would be going. [66:37] Um so it's very much a handdesign thing [66:39] and you can do that because it's a very [66:40] low bit game. Um, for Doom it's a lot [66:43] harder because you know it's a 3D game [66:45] for one kind of it's early Doom so it's [66:47] like kind of 2D but you get it. Um, so [66:50] you know instead of one ball and a [66:51] paddle you have you know multiple [66:53] enemies in a field that it has to kill. [66:55] Um, and of course you know your action [66:57] space is much larger in Doom. So instead [66:59] of pong where your paddle goes up or [67:00] down uh we have a lot of joint actions. [67:02] So, you know, kind of strafing, turning, [67:04] attacking, moving. Um, these are things [67:07] that we would have to account for. Um, [67:09] and this makes it kind of impossible to [67:11] do encoding and decoding or at least a [67:13] kind of a hand mapping for it because we [67:15] only have 59 channels and becomes really [67:16] really difficult um to get separable [67:19] outputs for 54 different actions on 59 [67:21] channels because some neurons might be [67:23] on similar uh channels. So, you might [67:25] get very coupled behavior. Um so what we [67:28] realized was um for harder tasks you [67:30] know if we want to scale this to GPT [67:32] level intelligence encoding and decoding [67:35] have to be learned end to end we need to [67:36] learn some form of a way for us to [67:38] stimulate the brain and we need to learn [67:41] the decoding or the decoder as well for [67:43] us to convert that into a useful action [67:45] uh for compute. [67:48] Um so kind of the what we did initially [67:50] was a closed loop PO architecture. So PO [67:53] is proximal policy optimization is RL [67:56] kind of algorithm for it. Um so we had [67:58] Doom, we had the encoder uh you know we [68:00] talked about encoder before. So it would [68:01] turn that into a stimulus for the neural [68:03] culture. Uh the spikes would then be [68:04] sent to the decoder and then we'll [68:06] convert that to action in game. Um we'll [68:08] talk a bit more about why we have a [68:09] critic later. Um but this is primarily [68:12] to keep feedback to the cells stable. So [68:14] the thing with the cells is we need to [68:16] provide feedback to it over time. You [68:18] know how good it's doing, how bad it's [68:20] doing. Um and we found that the critic [68:22] for PO was particularly well aligned [68:24] with how we wanted to do feedback. [68:26] Um so kind of how the percept looks like [68:29] um you know we take the doom [68:31] observations encode it neural hardware [68:33] um put all the spikes to a decoder and [68:36] then we update it every you know 2,000 [68:38] steps or so. [68:40] So the encoder network looks something [68:41] like this. I think we talked about um [68:43] image and scaler observations but this [68:45] is kind of what it looks like. Uh so we [68:47] pass the image to a CNN and we you know [68:49] kind of concatenate it with the scalar [68:50] observations. Um we pass it through MLP [68:53] and then that kind of spits out the [68:55] frequency and amplitude we should [68:56] stimulate each channel with. I think um [69:00] if I'm correct in the code, so this is [69:01] open source code. You guys can take a [69:03] look at this afterward. Um I believe in [69:05] this specific code it did not pick the [69:07] channel. Um all it did was pick the [69:08] frequency and amplitude. But I think in [69:10] subsequent experiments we experimented [69:12] with um having the MLP pick different [69:14] channels to stimulate as well. So a bit [69:16] more freedom for the encoder network. Um [69:18] this is kind of what it ends up looking [69:20] like. So you know you can see the [69:22] encoding electrics we have particularly [69:24] the ones in green. Um and you can see [69:27] that we are sending kind of different [69:29] frequencies and different amplitudes to [69:30] these electrodes in hopes that uh this [69:33] specific um setup would encompass the [69:36] game state well enough for the neurons [69:38] to actually act on it. Um at least far [69:40] better than we can kind of manually hand [69:42] program it. uh so expect for you know [69:45] further and further intelligence you [69:46] know think GPD2 and beyond on these [69:48] cells what you would have to do is [69:50] something that's very similar you know [69:51] we can't handcode 50,000 vocabulary [69:54] words um this is something that we'll [69:56] probably have to do and what we can [69:57] control instead is you know frequency [69:59] and amplitude [70:01] um so for the encoder you know for us to [70:03] actually train the encoder because we [70:05] can't propagate gradients uh through [70:07] kind of the the multi-elerode array here [70:10] um what ends up happening is we have to [70:12] do do some form of stoastic stimulation. [70:14] Uh so you know kind of beta sampling to [70:16] figure out what's the best amplitude and [70:17] frequency to stimulate it at. Um I will [70:20] say that this is fairly different from [70:22] reservoir computing. Um the the primary [70:25] difference is your culture is changing. [70:27] Um so if you're sending something of [70:28] this frequency and this amplitude uh [70:30] that changes the culture in a different [70:32] way every single time. Uh so you don't [70:34] really know how it's going to perform. [70:35] So the beta sampling in a way is also [70:38] part of a policy learning algorithm for [70:40] the cells. This is changing the physical [70:42] structure of the cells and changing the [70:44] outputs of the cells over time. So you [70:46] can think of it as every single moving [70:47] part in the system is is actively [70:49] affecting how the cells process [70:51] information and give us outputs. [70:54] Um decoder we use a linear readout. So [70:56] we had a few issues with the decoder [70:58] initially. I think uh what we did was we [71:00] oversized the decoder bit too big and we [71:02] we started realizing that oh it could [71:04] play on itself if you just zeroed out [71:06] the spikes. The decoder could play [71:08] itself if you had enough biases. Um, so [71:10] what we did was we kind of like [71:11] massively undersized the decoder, ran a [71:14] bunch of ablations to make sure that the [71:16] spikes themselves actually carried [71:17] information to play the game. Um, and [71:19] then we set the bias of the the kind of [71:21] linear decoder at this point. Kind of a [71:23] linear readout, I would say, is a better [71:25] way to put it, um, to zero. [71:28] And that kind of gave us better results [71:29] where the decoder wasn't overfitting the [71:31] task and it was able to start making [71:33] progress on the game. Um this is [71:36] particularly important because if your [71:38] silicon decoder overfits what ends up [71:40] happening is the cells never learn. They [71:42] never have to learn. They can keep their [71:44] stochastic behavior and then the decoder [71:46] itself can just play the game. So this [71:47] is particularly important and um kind of [71:50] want to hammer down on it because I [71:52] think this is something that you know [71:55] startups and companies get wrong a lot [71:56] of the time. Um it's very easy to cheat [71:58] performance out of these systems. All [72:00] you have to do is, you know, massively [72:02] oversize your decoder, run back [72:04] propagation on it, and suddenly you have [72:06] brain cells that do anything, play [72:07] anything. Um, I think most recently we [72:09] saw this with, you know, fly brains, [72:11] right? You know, you have these like [72:12] connetos, and what people do is they [72:14] just have these like massively oversized [72:16] decoders. It's never the fly doing [72:18] anything. It's just silicon, right? You [72:19] have this decoder that's doing [72:21] everything for it. Um, so I would say be [72:23] very careful with that and kind of [72:24] information that propagates that idea um [72:27] over time. Um so yeah this is kind of [72:29] what the decoder looks like is a few [72:31] sets of behavior. So forward backward [72:34] none um strafing left and right turning [72:36] left and right and attacking. Um so [72:37] every kind of action step we do a [72:39] softmax over the joint probabilities of [72:41] the actions. Um so feedback is something [72:44] that um a lot of people got wrong. Uh [72:47] this was fairly widely covered by media. [72:49] Um but I think not a single outlet got [72:52] it right. They got very very confused [72:54] over how it happened. Um so to provide [72:57] good and bad feedback to cells um how we [73:00] define good and bad feedback is um [73:02] through asynchronous and synchronous [73:04] stimulation. Um it is our kind of belief [73:06] through the free energy principle Carl [73:08] Fristen that um asynchronous [73:11] stimulations would is something that the [73:14] brain wants to avoid. Um so you can [73:15] think of it as if it does something bad [73:17] give it asynchronous stimulation. If it [73:19] does something good give it a [73:20] synchronous stimulation. Um, however, [73:23] because we're doing it in a kind of [73:24] reinforcement learning style, it gives [73:26] us a very unique problem, which is, um, [73:29] it sucks. You know, at the start of the [73:31] game, it really sucks. It does a lot of [73:32] things that would give it negative [73:34] stimulation. And the cells that we have [73:37] right now aren't good enough for, you [73:38] know, long-term credit assignment. Um, [73:41] so what we had to do was, um, we had to [73:44] predict surprise essentially. So we use [73:46] that critic um in the silicon decoder to [73:48] figure out how far is it away from the [73:51] silicon critic, how surprising is this [73:53] action specifically and we scale our [73:55] feedback based on the surprise. So you [73:57] know if it expected this action was [73:59] going to be good and in reality it [74:00] picked something and was really bad then [74:02] would give it negative feedback. So this [74:04] allows us [74:06] to modulate feedback in a much more I [74:09] would say equal way where you know we [74:12] don't keep stimulating with negative [74:14] feedback. we actually allowed it to [74:16] build uh connections over time and the [74:18] way in which it builds those connections [74:20] is through minimizing surprise uh to the [74:22] silicon uh critic. So that's kind of [74:24] what we do and of course you know the we [74:27] have different stimulation patterns for [74:28] each of these. So um on our website we [74:31] have this and it's kind of interactive [74:33] in a way. Uh so you actually see the [74:35] channels light up and see the different [74:37] simulation policies. Um so you know [74:39] synchronous obviously this would be [74:40] synchronous and then asynchronous for [74:42] negative feedback these would be firing [74:44] at different times at random. [74:47] Um and then of course on top of that we [74:49] scale it by surprise. Um so we kind of [74:51] take the um what I would say the on on [74:56] top of the surprise that we had here uh [74:58] we still scale it by surprise over time [75:00] using the TD error and that specific [75:03] scaling is on the frequency and [75:04] amplitude. So not only do we have kind [75:07] of the reward sign that is modulated by [75:10] the critic uh but we also change the [75:12] frequency and amplitude of that feedback [75:13] that we send to these cells. Um so I [75:16] think in the diagram here we have [75:19] different channels for feedback and [75:21] different channels for encoding. Um I'm [75:23] not sure the code if it's exactly the [75:26] same. Um but you know definitely check [75:29] out the code on GitHub. I think this is [75:31] my last slide. Um if you have any [75:33] questions FOR [75:40] >> YEAH. I was uh curious about like the [75:42] positive and negative signals. So it [75:44] seems like different frequency like uh [75:46] like electricity is applied to the [75:48] neurons or something. Yeah. [75:49] >> I was curious like what like does it [75:51] feel does like it feel bad the like when [75:54] you apply a 40 Hz thing like what [75:56] exactly does it mean negative versus [75:57] positive uh for the neuron? [76:00] >> Yeah. So the neurons themselves don't [76:02] feel bad. You know, they have no pain [76:03] receptors. There's no kind of like [76:05] noxious poison we're introducing to [76:06] these cells. Um when I say, you know, [76:08] positive and negative feedback, that's [76:10] in relation to the game. So, you know, [76:12] if it does something bad in the game and [76:13] we don't want it to do that action, we [76:16] send it asynchronous feedback to [76:17] reinforce that we don't want it to do [76:19] that specific action. Um so the brain [76:21] itself, you know, at least this culture, [76:23] it doesn't optimize for feeling good or [76:25] feeling bad. It optimizes for kind of [76:27] reducing the entropy in the culture [76:28] itself. [76:29] >> Okay? And so like the different like 40 [76:31] Hz thing increases the entropy of the [76:34] culture. [76:35] >> Yeah. I guess it it wants to reduce the [76:39] amount of times that it gets kind of [76:42] this high entropy stimulation basically [76:45] like it wants to reduce the rate at [76:47] which it gets this asynchronous [76:48] stimulations. Um and it would kind of [76:50] self-organize to prevent that. [76:52] >> Got it. That makes sense. If you're [76:54] scaling the TD error is what you mean by [76:57] scaling the surprise, right? [76:58] >> Yes. [76:59] uh aren't you like and you don't have a [77:02] static reward model itself. So you're [77:04] changing the reward model itself. How [77:06] does it reach an equilibrium? [77:08] >> I think this is the interesting thing. [77:10] It never reaches an equilibrium. Um I [77:12] forgot to mention this in the slides [77:13] actually but um you can think of it as [77:16] you know you have these two dynamical [77:17] systems that are almost fighting for [77:19] control in some ways. Um and your your [77:22] enemy is entropy. um entropy would [77:24] disrupt these two systems kind of [77:26] pushing themselves towards you know [77:27] semiattractor states. Um so you know in [77:30] our code actually we uh did something [77:32] called entropy penalties. Um this is [77:34] very very abnormal for PO. So [77:36] traditionally in reinforcement learning [77:37] um you actually want a bit of an entropy [77:39] bonus so you can start exploring the [77:40] space around it. Um we actually actively [77:42] penalized entropy um because we found [77:45] that it was helpful and the cells [77:46] provided sufficient entropy for that. Um [77:48] so you can think of it as you know two [77:50] dynamical systems that are pushing [77:51] themselves uh to better and better [77:53] outputs over time. Um in terms of you [77:56] know the sample efficiency of that I you [77:58] know we never truly measured that down [78:00] uh to the wire but that's uh that's what [78:03] it looks like [78:04] >> for the um stimulation you used. How [78:07] different is it to the real life [78:10] electrical signals you might get from [78:11] your eyes and ears? [78:13] >> I think it's very very different. I [78:15] think the way that I want to frame this [78:18] is we're using this as a material and [78:20] you know kind of going off on a tangent [78:22] here. I think that human intelligence is [78:25] not optimal for compute. Uh we're [78:27] optimized for survival, you know, [78:28] reproduction and you know the existence [78:30] of our race. Um but we're not optimized [78:32] for compute. It just happens that the [78:34] fundamental substrate that we operate on [78:36] is really good and is a good host for [78:38] intelligence. Um so I would say kind of [78:41] it would be very different uh compared [78:43] to you know traditional like I inputs [78:46] you know it could evolve to a point [78:47] where it starts looking similar and that [78:49] might be a bit scary. Um but my bet is [78:51] that it will continue diverging and what [78:54] we will see is a different form of [78:55] intelligence arise on the same substrate [78:57] that we see intelligence on ourselves. [79:00] >> Could I ask a followup? Could you train [79:03] on real life electrical signals from the [79:05] brain or is that like just not [79:08] >> like you know like grow some eyes and [79:09] then [79:10] >> yeah like record it and then maybe put [79:12] it in a person. [79:13] >> I think I think some people have tried [79:15] because you can grow like you know mini [79:17] like eye organoids and attach it to the [79:18] brain. Um but I think that starts to get [79:22] very scary because then you'd be kind of [79:23] recreating a human um from first [79:25] principles maybe. [79:27] >> Yeah. [79:28] >> Okay. Okay. Thank you. [79:32] How scalable is such a solution? [79:35] >> I think that's [79:38] probably one of the hardest questions um [79:41] that we have to answer for ourselves [79:42] which is you know I see this you know [79:45] solution of using biomp computing as [79:47] having two main problems which is um you [79:50] need to get frontier levels of [79:51] intelligence on these cells and then you [79:53] need to figure out how do you serve this [79:55] intelligence to billions of people in [79:56] the world. um we kind of know how to get [80:00] intelligence on cells like we've we've [80:01] seen it you know we're all intelligent [80:02] beings on the same substrate um what we [80:04] haven't seen is distributed computing [80:06] right we we fundamentally don't know how [80:07] to do this yet um so you know kind of a [80:10] shitty answer to that would be I don't [80:12] know how scalable it is um but maybe a [80:14] better kind of answer to that is um [80:17] we're trying to find out um and we've [80:19] kind of assembled a fantastic team uh [80:22] toward finding and making steps toward [80:24] scaling this technology [80:26] >> thank Thank you. Thank you. [80:29] >> Thank you.