Finance Scholars Warn AI Algorithms Risk Market Collusion

金融学者警告AI算法存在市场合谋风险 强化学习模型会催生默契合谋并威胁人类研究角色，除非学者转向高层次的问题界定。 该研讨小组探讨了生成式AI和强化学习如何重塑金融经济学、市场动态及学术研究。与会学者审视了市场中的算法合谋、大语言模型作为人类知识外化先验分布的效用，以及AI在实现概念突破方面的理论极限。讨论兼顾了针对终身教职前青年学者的实际应用与职业建议，同时对比了关于人类研究人员长远来看会走向淘汰还是实现人机协同的鲜明预测。 Xavier Gabaix 指出，AI 在经济与金融建模中的应用高度依赖具有可验证奖励的强化学习，能够赋能宏观预测、自动化信用评级以及构建理性基准。 (18:55) 在评估人类学者长期角色时，Gabaix 借助 CES 生产函数说明资本与劳动力替代弹性决定了人类是被 AI 替代还是被赋能，而人类在确立因果关系和提出原创研究问题上仍具比较优势。 (22:50) Itay Goldstein 展示了自主强化学习算法在金融市场中能够在无显式沟通的情况下达成默契合谋，在低噪声下依靠前瞻性价格触发策略维持，在高噪声下则因学习偏差产生过度剪枝。 (39:36) Goldstein 的多期规划交易模型显示，AI 能够通过操纵市场制造资产泡沫与崩盘来获利，且在信号精度处于中等水平时其表现会显著超越人类交易员。 (48:48) Dacheng Xiu 将生成式 AI 视作凝聚人类知识的高容量概率模型与可复用的丰富先验，宛如“口袋里的专家图书馆”，但其本质是模式编译而非严谨证明，因而擅长增量发现却难以独立提出有效市场假说等概念性突破。 (59:00)

At an American Finance Association panel, financial economists examine generative artificial intelligence not as an autonomous replacement for foundational theory, but as an externalized prior and predictive tool that reshapes empirical modeling, market dynamics, and academic research norms. Externalized Priors and Embeddings in Weak-Signal Environments (59:00) Dacheng Xiu characterizes large language models as structured, lossy compressions of accumulated human knowledge that function as an externalized prior rather than definitive answer machines. In financial economics, where signals are notoriously weak—demonstrated by roughly 20,000 published R-squared statistics from top 2022 journals heavily concentrating near zero—unstructured data requires strong regularization. AI priors and high-dimensional representations allow researchers to extract meaningful structure from corporate disclosures, news narratives, and portfolio holdings. Illustrating this empirical utility, Xavier Gabaix presents evidence from joint research mapping institutional portfolio holdings into high-dimensional vector spaces using masked transformer architectures. In out-of-sample quarter tests, these holdings-based asset embeddings explain approximately 60% of relative valuation variance, substantially outperforming traditional Fama-French characteristics (around 15%) and CAPM beta (approximately 2%). Gabaix notes that powerful predictive engines can operate effectively without incorporating foundational economic theory. Autonomous Algorithmic Collusion in Financial Markets Itay Goldstein explores how reinforcement learning agents, such as Q-learning algorithms, alter trading environments by learning non-competitive behaviors autonomously. In stylized market models extending the Kyle (1985) framework, algorithmic agents learn tacit collusion without direct communication or explicit human programming, posing major challenges for regulators whose enforcement rules depend on evidence of explicit coordination. Goldstein identifies two distinct mechanisms behind this algorithmic collusion. Under low to moderate market noise, trading algorithms sustain non-competitive profits through forward-looking price-trigger strategies that punish aggressive deviations. Under extreme market noise, collusion re-emerges through learning bias and over-pruning: algorithms encounter catastrophic payoffs by chance during early aggressive trading attempts and permanently prune those actions from their decision matrices. Computational Assembly Versus Paradigm-Shifting Conceptual Breakthroughs (67:36) Using Shizuo Kakutani's matrix of research difficulty, Gabaix suggests AI will rapidly automate the 'Easy and Complicated' quadrant—assembling disparate models and resolving complex technical mechanics—while the 'Simple and Hard' quadrant requiring foundational human inventiveness will resist automation longer. The panelists argue that predictive capacity does not equate to conceptual discovery. Xiu questions whether generative models could independently conceptualize foundational principles like Eugene Fama's Efficient Market Hypothesis or Albert Einstein's General Relativity. While machine learning discovers return anomalies and Sharpe ratios at scale, it struggles with open systems and paradigm shifts. Because generative AI optimizes over known symbolic training data and statistical associations, Xiu contends it cannot independently formulate new primitives or equilibrium framings without explicit human guidance. Academic Labor, Journal Gatekeeping, and Obsolescence Risks (111:13) The panel presents starkly contrasting forecasts regarding the future of human researchers. Applying a Constant Elasticity of Substitution framework, Gabaix speculates a 90% probability that human researchers become functionally obsolete within 10 to 20 years if AI capital serves as a complete substitute for human labor. Conversely, Xiu and Goldstein maintain that humans and AI will coexist, citing the necessity of human framing, the degradation that occurs when AI is trained on synthetic data, and the absence of an objective, unchanging 'ground truth' in social sciences. Operationally, the surge in AI-generated papers threatens to overwhelm traditional academic infrastructure. Goldstein and Xiu caution that standard metrics like citation counts and SSRN downloads will become increasingly vulnerable to distortion. Xiu advocates for immediate desk rejections of fully AI-generated submissions, while Gabaix predicts that an unmanageable abundance of papers may collapse legacy journal systems and their five-to-ten-year publication delays in favor of fast, preprint-driven distribution models. The panel concludes that while generative AI lowers the cost of technical execution, it sharply raises the premium on domain judgment, institutional context, and original problem formulation. Because current models compile statistical patterns rather than subjective human utility functions or causal frameworks, human oversight remains essential for interpreting results and navigating open-ended economic questions.

在美国金融学会（AFA）关于生成式AI与金融学术研究的专题研讨中，学者们指出AI正在显著提升经验表征与工程预测能力，但由于缺乏客观基准真理且受制于训练语料分布，AI难以独立实现范式级别的理论跃迁。面对算法默契合谋风险与激增的论文产出，人类学者的核心价值正从繁复的模型装配转向高维度的概念创新与批判性判断。 外部先验与弱信号下的理论局限 (59:00) 修大成（Dacheng Xiu）提出，生成式AI本质上并非绝对的真理机器，而是人类知识在符号数据上的有损压缩，可作为金融研究中高度丰富且可复用的“外部先验”（Externalized Prior）。金融经济学本质上属于“弱信号环境”，测量风险溢价、收益可预测性及因果机制均伴随巨大噪声。修大成展示的实证数据显示，2022年顶刊发表的约20,000个金融与经济学实证 $R^2$ 统计量高度集中在接近于零的区域，样本外真实信号更为微弱。这要求研究必须施加正则化或结构约束，而大语言模型（LLM）提供的先验正成为一种结构化处理弱信号的关键介质。 然而，修大成强调AI极难自主发明类似“有效市场假说”（EMH）的基础理论。尽管现有AI能规模化挖掘异象与预测信号，但有效市场假说是将价格定义为信息竞争均衡结果的范式重构。当前生成式模型依赖已知语料与统计共现，缺乏在训练数据分布之外确立新原语（Primitives）的能力；加之模型缺乏人类主观效用与真实体验，在面对缺乏客观损失函数的开放系统时，无法完成根本性的理论奠基。 资产嵌入表征与预测引擎构建 (8:01) 格瓦克斯（Xavier Gabaix）从方法论切入，阐明了利用自然语言处理中的嵌入（Embedding）技术将资产与投资者映射至高维连续向量空间的路径。借鉴大模型的掩码语言建模（Masked Modeling）方法，其团队对机构持仓数据进行头寸掩码并训练神经网络预测隐藏资产，从而直接从持仓数据中提取潜在资产特征与投资者偏好。在单季度样本外基准测试中，持仓嵌入对相对估值残差的解释力（$R^2$）达到约60%（与传统特征结合后达70%），远超资本资产定价模型（CAPM）贝塔的约2%与法玛-弗伦奇（Fama-French）特征的约15%。 在理论与预测的关系上，格瓦克斯认为高精度的预测引擎往往不需要基础理论支撑，正如AlphaFold无需解薛定谔方程即可预测蛋白质结构。结合分类论文维度的“角谷矩阵”（Kakutani Matrix），格瓦克斯预测AI将迅速胜任“简单但繁复”（Easy and Complicated）的模型拼接与工程求解工作，使相关专业技能加速贬值；而需要深邃人类独创性的“简单且艰难”（Simple and Hard）领域依然具备韧性。金融学界亟需转向构建能解决宏观预测与信用评级等现实问题的工程化应用。 强化学习交易与算法默契合谋 (28:56) 戈德斯坦（Itay Goldstein）关注强化学习（RL）算法对金融市场动态的改变，探讨自主交易智能体是否会在无人类显式编程的情况下自发形成非竞争性合谋。在扩展经典Kyle（1985）框架的市场模拟实验中，基于Q-learning的知情投机者算法展现出在缺乏沟通或协议的前提下维持合谋并损害市场流动性与定价有效性的能力。这种基于算法自主试错形成的默契合谋，直接挑战了传统依赖人类直接沟通证据的反垄断与市场操纵监管框架。 该研究进一步揭示了算法维持合谋的两种截然不同的机制：在低至中度市场噪声环境下，算法采用“价格触发策略”，利用价格作为信号机制并对激进交易施加惩罚，体现出前瞻性的前沿协调；而在市场噪声极高的环境下，联合利润非但没有崩溃至纳什均衡，反而意外逼近“完美卡特尔”基准。戈德斯坦解释，极端噪声导致早期激进交易偶尔遭遇灾难性随机亏损，促使算法出现严重的“学习偏差”或过度剪枝（Over-pruning），永久性规避了激进策略。 学者替代风险与学术评价重构 (25:12) 关于AI是否会导致人类研究者被替代，学者们展现出显著分歧。基于不变替代弹性（CES）生产函数框架，长期结果取决于人类劳动与AI资本的替代弹性。格瓦克斯持更为激进的预期，预测未来10至20年内人类研究者在职能上有90%的概率走向过时，并对高校提供40年终身教职的财务模式表示担忧；修大成与戈德斯坦则认为人类与AI将长期共存，理由是纯AI生成数据训练会导致模型性能退化与分布崩溃，且社会科学缺乏确定“对与错”的客观基准真理（Ground Truth），极度依赖人类在经验、直觉与问题提出上的比较优势。 伴随AI应用带来的论文生产爆发，传统同行评审体制面临严峻冲击。戈德斯坦与修大成警告，自动化生成的论文正在充斥投稿通道，学术期刊必须建立严格的直接拒稿（Desk Reject）与防范机制，同时谷歌学术引用量等传统量化指标将因机器自循环而严重失真。格瓦克斯进一步指出，经济学界长达5至10年的出版滞后与对论文细微打磨纯属资源浪费，庞大的AI产出压力可能彻底击垮传统的期刊门禁系统，迫使金融与经济学转向类似数学与物理学以预印本（Preprint）为主导、交由市场直接检验的发表模式。 生成式AI大幅降低了金融研究中技术实现与模型装配的边际成本，将学者的竞争核心推向概念创新、理论品味与结构化审据。然而，在缺乏客观基准真理且受制于训练语料边界的社会科学中，AI仍难以替代人类对复杂因果与主观效用的裁量；面对机器生成论文对学术期刊承载力与引文指标的冲击，如何重构针对真实智识贡献的评价体系仍是未决议题。

[38:29] Gary Gensler: even if the humans aren't talking, the machines will start to have a sense of cooperation and we've already seen this in high frequency trading. [55:35] Dacheng Xiu: boring now [55:35] Dacheng Xiu: Can machines invent the Efficient Market Hypothesis? [55:35] Dacheng Xiu: the debate has not been settled in Chicago yet. [59:00] Dacheng Xiu: library of experts in your pocket [59:00] Dacheng Xiu: a prior is not a proof. [74:30] Dacheng Xiu: I'm going to follow the existing norm here and disclose that the slides were prepared with the help of GenAI, but my own judgment still matters here... So all errors are mine.

[0:08] All right. Well, welcome to our session [0:09] on genai and finance scholarship. Uh for [0:13] our panel today, we have Harvard's [0:15] Zavier Gave, uh Wharton's Itai [0:18] Goldstein, and Chicago's uh Dachon Xiao. [0:22] Um I'm going to start with a little [0:24] disclaimer which is that uh while we [0:26] will certainly discuss Gen AI uh Gen AI [0:31] in this um session, we are not [0:32] restricting ourselves to that. So uh [0:34] Genai has certainly captured the [0:36] imagination of the public. Um it's gone [0:38] a long way towards confusing the [0:40] language that we use when we talk about [0:42] it and that's something we can discuss a [0:43] little more if people are interested. [0:45] Um, but they're just one of many tools [0:47] that uh are used in conjunction with [0:49] each other and we're not going to be as [0:52] restrictive as limiting ourselves to [0:53] just talking about Gen AI. Um, the [0:56] session today by its nature is also uh [1:00] hopefully speculative uh more [1:02] speculative than a lot of what we do in [1:03] academics. Um, and I hope that when we [1:06] get to the panel that really it won't [1:08] just be us, but that we'll have broad [1:10] participation um especially with that [1:12] speculative nature. um you know none of [1:15] us really know what's happening. So um [1:19] before we get to the interactive panel [1:21] I've asked each of our panelists to talk [1:23] a little bit about their own work um [1:25] which has informed the way they think [1:27] about the topics that we're going to get [1:28] to when we get to the discussion. Um and [1:30] in particular I've t asked them to to [1:33] talk about their own work in a way that [1:34] to the extent they can um that they do [1:37] it at a high level that they emphasize [1:39] intuition. Um and again that they [1:41] speculate both on the potentials and the [1:44] potential limitations [1:46] um in a way that we might be you know [1:49] more speculative than we would do in a [1:50] typical academic talk. All right. So [1:53] more than anything I hope we can have [1:54] some fun. The topic I think leads to [1:56] lively discussion. Um and I hope [1:59] everyone will participate. Okay. So, uh [2:01] we'll start off with Zavier. [2:06] Great. Uh good afternoon. So, um this is [2:11] uh those thoughts are you know very [2:13] speculative like Robert said they're [2:15] inspired by conversations with my [2:17] co-authors in particular Ralph Moto and [2:20] Rob. and I'll get some I'll draw some [2:23] results from two papers asset and [2:25] investor embeddings and upgrading uh [2:28] credit pricing and risk assessment [2:29] through embeddings. Okay. Okay. So uh [2:32] I'll start with some general observation [2:34] to see the landscape outside our little [2:36] world. Okay. And that will be some [2:40] observation useful for the general [2:41] discussion I think. Uh okay. So in AI [2:44] competitions have have been extremely [2:46] useful to uh you know accelerate [2:49] basically research and for instance most [2:53] of you know that but let me just mention [2:54] it nonetheless just in case for image [2:56] recognitions you had this imageet [2:58] competition. So the task you have lots [3:00] of images you need to classify cat dog [3:03] cars etc etc. Participants submit [3:06] programs, new images are given and then [3:09] you see who is more successful at [3:12] classifying the pictures and you could [3:14] see some evolution and one day one year [3:16] there was alexet. It was the first [3:18] successful deep network uh that um you [3:22] know increased a lot the performance of [3:25] that competition uh for protein folding [3:27] in biology. So is you get a protein [3:31] return with you know C H O various [3:33] whatever atoms and you need to predict [3:35] the three-dimensional structure. So [3:37] there's also an annual competition and [3:41] there was a breakthrough alpha from [3:42] Dickman that was a very big deal and [3:44] they got actually a Nobel prize for [3:45] that. So uh one inference what can make [3:48] from that is it would be great to have [3:50] more many more competitions. We have [3:52] essentially known in finance and [3:54] economics organized every quarter every [3:56] year to uh where people can submit the [4:00] program. We can see how how well they [4:02] how well they do and more more thoughts [4:06] will come soon about how to do that. Now [4:10] another observation is that those [4:12] predictions predicting is very useful [4:14] but sadly in some sense or interestingly [4:17] doesn't need you don't need a deep [4:19] understanding for that. So, LLMs, large [4:21] language models, they've been immensely [4:23] useful for text translation, generation, [4:26] etc., but they use exactly zero chian uh [4:31] linguistics. For instance, kind of [4:32] interesting alpha fold protein folding. [4:36] It uses zero shinger's equation zero [4:39] like very very fundamental uh high level [4:42] physics. Maybe the existence [4:45] of atoms, existence of molecules. uh it [4:49] doesn't simulate uh what happens and it [4:52] just learn does use learn geometry from [4:54] existing structures and something about [4:57] uh evolutionary u you know data but very [5:01] little in terms of high theory if you [5:03] want um so likewise you could imagine a [5:07] process by which we'll be able to [5:09] predict better and better not just what [5:11] the market does but maybe what investors [5:13] do and variety of things maybe macro [5:18] using um various AI machine learning [5:22] techniques uh and that would be great [5:25] but it's probably the case that at least [5:28] for a long time theory will not be that [5:30] useful actually for that uh still what's [5:33] nice in terms of the understanding [5:35] dimension is that once we have those [5:37] prediction engines uh trained via via AI [5:40] techniques and uh you know [5:43] knowledge diffusing via competition uh [5:46] they will be useful to constraint [5:48] theories, you know, because your theory [5:50] instead of testing it on a very very [5:52] simple maybe a doc data set, you will [5:54] see how it explains reality as it's been [5:58] encapsulated by the prediction engines. [6:00] Anyway, so that's like a view on how uh [6:03] we might proceed. Uh what kind of papers [6:06] will AI engines write? So I find it [6:09] useful to uh um call on the kakutani [6:12] matrix. So I teach that to my students. [6:15] The law is that Kakuteni was teaching [6:16] that and it propagates. So according to [6:19] him there are four four types of papers. [6:22] So a paper can be easy or hard. Simple [6:25] and complicated. So what's that? Simple [6:27] and easy. That's like an exercise. Okay. [6:31] Uh hard and complicated [6:34] it's almost impossible to do. It's like [6:36] you know the proof of the mass theorem [6:38] or something. So most papers and [6:40] certainly all many many many papers that [6:41] AIS will write it initially are in the [6:44] easy and complicated box which is a [6:47] useful box. Okay you have some macro [6:49] model there some asset pricing model [6:51] then you glue them and there's some [6:52] something interesting going on. Uh you [6:55] know requires skills those skills [6:57] presumably will deprecated via uh by [6:59] AIS. Uh but you don't need to reinvent [7:02] many wheels. You just put together [7:04] various things that are that are there [7:06] and tell them the students try to aim at [7:09] the you know at the oop how do I go back [7:12] here? uh simple and hard and I think [7:16] that AIS will not it will take much more [7:20] time for them to do the the hard things [7:22] meaning some you know really uh [7:24] inventivity etc etc but they will be [7:26] very useful to do the complicated [7:29] assembling of things uh that will be see [7:32] a great s use a great plus for financial [7:35] economics because um we'll be able to [7:39] this way maybe predict better have more [7:41] macro model useful for the Fed and and [7:44] the BIS etc. and it will be useful for [7:47] world welfare and our credibility. We [7:49] can predict and mimic the world better [7:51] use held by those AIS. So let me uh in [7:56] part to talk about these competitions in [7:58] mind uh draw from [8:01] paper we have a coffers called asset and [8:04] investor embeddings. So just many of you [8:06] in this room know what it is but let me [8:08] just uh say what's an embedding is just [8:11] a vector. So um in some typically [8:14] highdimensional space maybe 100 [8:16] dimensional 1,000 dimensional. So if you [8:19] have x a for a being some asset maybe [8:22] apple uh it will be a vector of [8:25] characteristics traditionally that you [8:27] know from like accounting data or maybe [8:29] something we will infer from some of the [8:32] source as I'll explain. And in LLM, [8:35] that's like the some of the, you know, [8:37] key components of one of the key [8:39] components of LLMs. And you have nice [8:42] properties that make sense. If you take [8:44] the X Paris, that would be the vector [8:47] maybe in a 10,000 dimensional space [8:49] representing Paris minus the vector for [8:52] France plus the vector for Spain. And [8:54] per QD is kind of close to the vector [8:56] for M Madrid. So it kind of makes sense. [8:59] It captures the what seems to be going [9:01] on in in some way. Okay. And uh and [9:05] there are a number of papers using using [9:07] embedding embeddings but I think there [9:09] will be uh many more. So just a few [9:12] slides from what we uh do here in our [9:15] paper. We say so we have um you know we [9:18] model demand by investors in a some semi [9:21] behavioral model and um the log value of [9:25] how in how much investor I invest in [9:28] asset a think in dollar terms will will [9:32] be we'll call it lambda lambda i prime [9:35] time xa plus some constant and some [9:38] noise. So um lambda i and x those are [9:42] vectors those are the embeddings okay in [9:44] the simplest model we have fancier model [9:46] later that I won't talk about um so um [9:50] lambda ik is the taste of investor i for [9:53] characteristic k and x is like the [9:56] loading of asset a on charact how much [9:58] it loads in maybe doing a for instance [10:02] and you can imagine some investors want [10:04] to do a bet on AI so they you know u [10:08] they [10:09] bet on this by they have a high taste [10:11] for calcics K that would be AI and we [10:15] can learn the proposition a very simple [10:17] proposition or in principle with enough [10:20] investors and assets we can generically [10:23] we can learn all the relevant [10:24] characteristics from just this holdings [10:27] data by doing basically some sort of PCA [10:30] or something uh fancier and with that [10:33] say for instance with this investor [10:35] embeddings in principle there are lots [10:37] of things you can And we have just [10:38] scratched the surface. You can classify [10:40] investors by uh beyond institutional [10:43] type etc etc by seeing what what is it [10:46] that they like and you can look at maybe [10:48] identify crowded trades do much more [10:50] precise perform much measurement etc etc [10:54] uh and so it's failed to start my timer [10:58] so uh you tell me how much time I have [11:01] you okay so uh okay so how do you [11:04] extract embeddings the simplest is to do [11:07] PCA called represent system And you can [11:10] do transformer models but uh there's no [11:12] time for us to do it now. You get [11:14] interesting results. So I show you how [11:16] you do the plain thing. It's very [11:19] simple. You say I want the model to fit [11:22] optimally. So I minimize the sum of [11:24] squares and also I have a rich penalty [11:27] to have a not too complicated uh model. [11:30] Okay. And uh uh another another type of [11:34] loss function and thinking about loss [11:36] function is very very fundamental for [11:39] you know AI AI friendly research. Uh [11:43] it's mask modeling. So guess guess the [11:47] word. So in linguistics you must have [11:49] seen that a lot of how you train an LLM [11:51] is you predict the word for instance [11:54] please pass me the and pepper. what is [11:57] the you know and the LLM will be trying [12:01] to make the right right guess [12:03] proistically. So part of what we do is [12:05] we train the LLM on our own little [12:08] transformer machine. Uh here's a hedge [12:11] fun for instance arc. We show you all [12:14] the holdings of arc masking by size [12:17] order by size except we mask here the [12:19] fourth one and the machine needs to [12:23] predict the number four. So that gives [12:25] that forces the machine to learn in some [12:28] sense some of the structure. Again we [12:29] just imitate what they do in machine [12:31] learning uh using some financial data. [12:35] So we use faxet type of data [12:38] and other kind of [12:40] jensenetal characteristics and farmer [12:43] French characteristics. We compare that [12:45] to traditional very plain farmer farmer [12:48] French characteristics book to market [12:51] cap beta etc etc and we compare that we [12:55] compare our holdings based embeddings to [12:57] them and also what if you just do did [13:00] the word embedding from AI for say IBM [13:03] what would you get okay so now the key [13:06] thing is suppose we wanted to do a [13:08] competition between not just our model [13:10] but lots of models um what would be [13:13] meaningful benchmark. So we thought [13:15] about three benchmarks. You can figure [13:17] more. One is prediction relative [13:19] valuations in a given quarter. So you [13:22] say uh you regress the log market value [13:25] MA on the book value first the simplest [13:28] predictor and you get a residual MA [13:30] perpendicular. Then you uh try to [13:34] explain what's not explained by book [13:36] value via the embeddings. So you got [13:38] your embeddings drawn from holdings. [13:40] Okay. and you try to see how well do [13:43] they explain valuation. You feed that on [13:45] 80% of a sample in a given quarter and [13:48] then you calculate the R square for the [13:51] rest of the sample. The resting say 20% [13:53] of the sample. What's really important [13:55] here, you don't need any time series [13:57] data with just one fresh quarter of [13:59] data. You can see which model is best. [14:02] This way each each quarter you'll have a [14:04] new round of competition and that's very [14:07] uh you know that's very useful. Um wait [14:10] another one would be explaining co- [14:12] movement. So you calculate the co move [14:14] how much do do those embeddings explain [14:17] the co movement in a given quarter maybe [14:19] using daily data or something like that. [14:21] Okay. And you can feed the R square. [14:24] Another one do you would be this asset [14:26] similarity in managed portfolios like [14:29] guess uh holdings number four number [14:31] eight number 16 that kind of thing. And [14:33] again what something I want to emphasize [14:35] is those benchmarks they're useful to [14:37] discriminate. you just need one quarter [14:39] of data to do the competition as opposed [14:42] to a more traditional like feed the [14:44] farmer French portfolios you know we had [14:46] already tons of profession data mining [14:49] on those portfolios uh and if you have [14:51] one more quarter data it's almost the [14:53] same data set so it's very hard to [14:55] really have you know new uh you know out [14:57] of sample out of sample way to uh [14:59] discriminate between the models. So [15:01] here's something we do we find with our [15:04] you know first pass uh the okay um capen [15:08] beta [15:09] um terrible confirmed it explains about [15:13] 2% of our square okay so again the [15:17] farmer characteristic explain maybe 15% [15:20] okay the just the largest fairly high [15:23] dimensional embeddings that we've used [15:25] from holdings data uh they explain about [15:28] you know 60% of the R squares So we say, [15:31] "Wow, there's a lot of information in [15:34] the uh in the embeddings." Okay. Um [15:38] Okay. Um you can combine characteristic [15:41] embeddings. You do a little more and [15:43] reach something close to 70%. Okay. So [15:45] embedded are very useful, more useful [15:47] than traditional at least from a French [15:49] characteristics. Uh text embeddings from [15:52] open air etc would do miserably because [15:54] they say IBM we don't really know what [15:56] it means was using a investor reveal [16:00] preferences. there's a much more [16:02] semantic richness there. Uh let me skip [16:05] that. Explaining co- movement actually [16:08] then fafrench characteristic do pretty [16:10] well to explain co movement. It's not [16:12] very big. It's 5% are square. Uh and [16:15] embeddings do maybe it's a professional [16:19] mining and alone uh or just growing [16:22] wisdom. uh that's another possibility [16:25] our embeddings per se do as less well [16:27] and but they add still some value to the [16:30] characteristics for French [16:32] characteristics and asset similarity is [16:34] guessing [16:35] guessing some mask assets um we do um [16:39] you know um there's something [16:43] young transformerish model like work [16:46] tovec does actually pretty well much [16:49] better than anything else so we we get [16:51] to see how um those benchmarks can be [16:54] used and how embeddings can be useful. [16:57] Let me skip that and you can uh also get [17:00] insights about you know what are sim as [17:04] uh investors similar in the endic space [17:06] to DFA lots of funds I never heard of or [17:10] similar to AQR etc. So it helps you uh [17:13] you know navigate the space uh better [17:16] and maybe understand better. We also uh [17:19] do the same this was all for stocks. We [17:21] also do the same for bonds in a fixed [17:23] income paper with the same team. And [17:25] then one of the insights is that [17:28] potentially AI could automatize the [17:30] production of bond rating. So I won't [17:33] explain how but potentially you could [17:35] and that could be a very very useful uh [17:38] engineering way of using AI and finance. [17:42] Okay, I'd like to say a little more [17:43] about the advantages of competition in [17:45] finance. So uh again in asset pendings [17:48] we try to discipline ourselves to use [17:50] really uh just one quarter of data will [17:52] be very very useful to uh for new [17:55] competition and in general in the [17:57] profession I invite you uh it's great to [17:59] think about more benchmarks and maybe [18:01] volunteers to organize such competitions [18:04] again they've been crucial in biology in [18:06] computer science to um you know get the [18:09] ball rolling. Why are they useful? So [18:13] it's a useful way to measure and [18:15] stimulate progress and also it's more [18:16] democratic if you want. So suppose you [18:18] had a bright and obscure student from [18:20] Indonesia could have a fantastic way to [18:23] do things and completely win the [18:25] competition where if it's okay [18:27] publishing in the journal of finance [18:29] let's say you there are so many barriers [18:31] to entry in the various mannerisms the [18:33] profession and what you should site and [18:35] not site etc. It's completely you know [18:38] it creates lots of artificial barriers [18:39] to entry whereas or maybe voluntary uh [18:43] but anyway it would remove some of the [18:44] barriers to entry so that's very good [18:47] and there's a an interesting paper by [18:49] Helum Jensen Kelly Peterson uh that also [18:52] proposes competition uh recent paper and [18:55] but they use more time series kind of [18:57] thing and maybe maybe I can we can [18:59] convince them to use cross-sectional uh [19:02] things okay um and what's the advantage [19:06] of having those clear metrics of [19:07] success. Uh you can one you can see you [19:11] know what techniques seem to work [19:13] another you can automatize the [19:15] production of new uh of new algorithms [19:19] and potentially new theories. So in the [19:21] AI it's called reinforcement learning as [19:23] you've heard but reinforcement learning [19:25] with verifiable rewards. So for instance [19:28] you try to uh train the AI to solve math [19:31] problems at the end you can test does it [19:34] is the proof correct or not correct. So [19:36] verify your reward zero or one. So [19:38] likewise you would have those verified [19:40] rewards and so that immediately allows [19:43] you to in principle to build a more and [19:46] more you know good product prediction [19:48] engines that will encapsulate a lot of [19:51] regularities in the world but also once [19:53] you have them you can second round [19:56] almost iterate on par theories. They [19:59] will never do as well as the blackboxy [20:02] prediction engine, but maybe you can [20:04] test them more immediately on the fall [20:07] of reality as captured by this [20:10] prediction engine. Anyway, that's how I [20:11] can see how AI could be useful to do [20:14] even uh to do theory. Uh [20:17] um please to do more uh I think we need [20:21] more engineering applications and [20:22] successes in in uh in economics and in [20:25] finance. We had some fantastic successes [20:27] like portfolio formation, marowitz, [20:30] black sh amazing finding alpha they [20:33] useful at least privately maybe not [20:35] socially uh useful for hon definitely [20:38] now I think we have lots of [20:39] opportunities of really concrete novel [20:41] things to do for MBAs more applied u you [20:45] know real world applied PhDs using AI so [20:48] predicting returns a lot of work done in [20:51] hedge funds some of a great way also [20:53] done in academia often with people [20:55] linked hedge funds like Brian and [20:57] Dashing and people like that. Uh [21:00] predicting macro would be very useful. [21:02] It's a great shame. That's part of why [21:04] we're not that considered and listened [21:06] to is because we don't predict very [21:08] well. So if we had even the AI based [21:10] ways to predict that would be fantastic. [21:12] Like I said, we could potentially [21:14] replace Moody's bond ratings using AI [21:17] some economic thinking by the way and [21:19] embedded and your decision-m tools. [21:22] there are lots of very very concrete [21:23] things uh we can do. What about pure [21:26] like university level research uh so I I [21:29] here I'm going to show you some slides [21:30] given by my co-author uh Sang Minho at [21:33] Colombia which I thought was fantastic. [21:36] So uh so far uh there are at least three [21:38] types of AI papers finding new data [21:41] modeling complex relationships and fair [21:43] benchmark. Let me just flag a few [21:45] things. One is just generating new data [21:48] via AI to be just too time consuming to [21:51] do it. For instance, for you know [21:53] measuring regulation or measuring [21:55] stress, looking at how people look, you [21:58] know, completely stressed and tired and [22:00] uh aging. Uh another one is modeling [22:04] complex relationships. So like clearly [22:06] machine learning is useful actually [22:08] dashing is here. Uh and um we have other [22:12] ways to do it. Another one is a fair [22:14] bench. fair benchmark. So some actually [22:17] are fair benchmarks say what would a [22:18] rational computer do train just with [22:21] data up to 1920 what should the rational [22:24] computer predict? So uh blankie and [22:28] lubixon have a series of papers on that [22:29] thing was very interesting and some [22:32] people like by uh have papers using more [22:35] behavioral agents uh modeled by AI. So [22:38] lots of good things we can uh we can do [22:41] and some of the papers and I guess we do [22:43] that too is you know intellectual [22:45] transfer using kind of straight ideas [22:47] from AI and applying them to uh to [22:50] finance uh still what's the role what's [22:53] the role for researchers you know not [22:55] just doing engineering things but so [22:59] even if you can predict a lot we to [23:01] understand the causal mechanisms so for [23:03] that we need to find instruments so [23:06] traditionally requires a lot of [23:07] ingenuity like I know the co died or the [23:10] senator died and this as a result the [23:12] states lose some power etc historical [23:15] quirks so AI can really help finding [23:17] such historical quirks uh with we have [23:21] something we we like a lot called [23:22] granular instrumental variables it's [23:24] using idiocotic shocks to large firms or [23:27] large hedge funds or large industries [23:29] and see how they propagate uh in the [23:31] network and AI could be very useful also [23:34] to find some uh you know do some of the [23:36] work can find some of these inducotic [23:39] shocks. And something else we could do [23:41] is define the big goals. Uh part of the [23:44] research talent is you know cleverness, [23:46] some cleverness, a lot of technical [23:47] skills, a lot of industriousness and a [23:50] knack for formulating interesting but [23:52] still manageable questions and maybe for [23:55] the latter part we have some advantage [23:58] for a little while longer. So for [24:01] theory, will AI be useful? Again, it's [24:04] useful to look at what's going on in [24:06] math. So now AI can win gold medals at [24:08] the math Olympia. So extremely so short [24:11] but extremely difficult problems. Uh so [24:15] not yet research level ideas but you [24:17] know it's progressing very fast. So [24:18] maybe in five years they will be able to [24:20] do it or 10 years. So now AI research is [24:23] directly has been proven useful say for [24:25] the double descent phenomenon uh which [24:28] you know makes you want to do more very [24:30] high dimensional models in in finance uh [24:33] for cognitive modeling uh for instance [24:35] very nice barbaris gene paper using [24:38] reinforcement learning from AI uh to [24:43] model people in in my own more behavior [24:46] research I wanted to model the mind [24:48] wants to build a simplified model of the [24:50] world and I thought those lasso sparity [24:53] type penalties were very useful or [24:55] people don't like complexity. It's [24:57] useful to see what's going on in AI even [24:59] just to do pure theory and there's lots [25:01] of interesting work doing cognitive [25:03] modeling for behavioral psychology and [25:06] also uh economics by these kind of [25:09] co-authors. Uh now a question and maybe [25:12] we can ask you think what you think [25:14] yourself is uh will human be uh will we [25:18] human be obsolete. Uh [25:21] if AI can do all we do but at some point [25:24] then they can do it a thousand times [25:26] faster and cheaper one will be [25:28] completely obsolete. So that's a that's [25:32] a question. Okay. So a little bit of [25:35] theory. uh let's do a CS production [25:38] function for those metaphysical [25:40] questions. Uh there's capital labor you [25:42] model AI there's infin capital becomes [25:45] infinitely abundant. So what's the labor [25:47] share in in in the whole economy? So [25:50] there's this thing the elasticity of [25:52] substitution row. So if row equals one [25:55] it's coplass if row is less than one [25:58] capital and labor substitute. So as [26:00] capital becomes infinitely abundant the [26:03] labor share goes to zero. That's like [26:04] the doom scenario. But if a labor share [26:08] if capital and complement are capital [26:11] and labor complement is bigger than one [26:14] then the labor share actually goes to [26:15] 100%. [26:17] As as um capital you know AI makes [26:21] capital infinitely abundant. So the [26:23] question then is are there enough [26:25] activities for which human labor is you [26:29] know essential meaning row is bigger [26:31] than one. [26:32] But you know complement to capital [26:35] rather than a substitute [26:37] and we can speculate it's a bit hard [26:40] definitely maybe the human touch for [26:42] teaching [26:44] uh maybe the defining some big goals and [26:47] ideas now AI can't do that maybe in five [26:50] years or 10 years they will be to do it [26:52] or maybe we'll just be like a trusted [26:54] like a like a non general you know [26:58] low-level doctors who don't operate on [27:01] you you know frankly AIs can do better [27:03] than them. I think I understand uh but [27:06] still people you know the grandmother [27:07] will trust them and they look wise and [27:09] all that. So maybe we'll be like that uh [27:11] trusted reassuring interfaces for the [27:14] public uh or will be completely obsolete [27:17] uh altogether in part it depends on this [27:19] elasticity of substitution. So uh uh now [27:24] there will be a consolation which is we [27:26] came to that was re this research life [27:28] because we presumably wanted to [27:30] understand the world and before we [27:32] become completely obsolete there will be [27:34] this explosion of understanding and [27:36] we'll see you know the world the sky [27:38] suddenly becoming lit with understanding [27:41] and that will be also our our end as [27:44] valuable contributors. Anyway, um [27:49] conclusion for the short run more data [27:51] types will come from AI. Uh competitive [27:54] benchmark would be great. They're more [27:56] democratic measure stimulate progress. [27:59] Will be hopefully lots of improvement in [28:00] predictions, understanding the world at [28:02] a detailed level and we can glue [28:05] together existing theories like in the [28:08] easy but complicated part of the [28:10] quakutini matrix. In the medium run [28:14] perhaps we'll use AI to build new [28:16] theories. Again they will be benchmarked [28:18] not just on a few moments but on this [28:20] kind of how they explain the behavior of [28:23] this optimized prediction engines be on [28:26] very large data sets and for the long [28:28] run maybe it will be glorious [28:30] obsolescence for us humans as we pass [28:33] the torch to our faster AI descendants. [28:38] We'll see. Maybe we'll take a poll later [28:40] to see what's how many years you think [28:41] that may happen. Thank you. [28:43] >> Thank you. [28:59] Okay. Uh, thank you, Robert, for [29:02] organizing this panel and asking me to [29:03] be on it. So I'm following your [29:06] instructions here to uh give you an [29:08] overview of my work in this area. And as [29:12] you noted, this is not about generative [29:14] AI but generally AI in financial markets [29:18] and in particular using reinforcement [29:20] learning in financial markets and trying [29:23] to understand how AI algorithms that are [29:26] based on reinforcement learning are [29:28] going to change uh the financial market [29:31] uh environment. So all the work that I'm [29:34] going to describe here is co-authored [29:36] with Winston who is sitting here. Um and [29:40] uh we have uh one uh paper that has been [29:43] out for a couple of years and uh [29:45] presented quite a lot. I will focus on [29:48] on this one. Uh but I will also give you [29:52] uh a couple of nuggets from two uh [29:54] follow-up papers that we have uh that uh [29:58] we just put out uh and uh I think are [30:00] pushing it a little uh further and give [30:03] us some more speculative ideas on how to [30:06] think about AI in financial markets and [30:09] I will keep my uh comments uh to be [30:12] non-technical. So uh this is going to be [30:14] a non-technical uh summary of these uh [30:17] papers. Um so uh the the big uh question [30:22] is how AI uh traders uh are going to [30:26] behave uh when essentially we replace uh [30:30] humans with AI algorithms. Um and uh we [30:35] all know a lot about uh trading and [30:37] information in financial markets. We [30:39] have models, we have empirical evidence. [30:41] Uh once we think about AI algorithms [30:44] that are replacing humans, uh things [30:46] change uh quite a lot. Um the method [30:49] that we are going to use in two of these [30:52] uh papers is essentially an experimental [30:55] uh method where uh we're doing an [30:57] experiment on AI algorithms. So this is [30:59] quite a different type of experiment [31:01] than what you would do on humans but it [31:03] is also uh easier to to implement and I [31:07] think opens the door to a lot of uh uh [31:09] research uh that can be done uh along [31:11] these lines. Um the the first paper [31:14] which is our main paper uh also with [31:17] Yanji in uh Hong Kong is uh AI powered [31:21] trading algorithmic collusion and price [31:23] efficiency where we basically think [31:25] about how AI algorithms are learning in [31:29] different ways to collude and the idea [31:32] of collusion is that they are uh trading [31:35] non-competitively. [31:37] Um in the second paper we take this [31:39] framework and extend it to think more [31:42] generally about uh market stability and [31:45] see whether AI algorithms have the [31:48] capacity to generate bubbles and uh [31:52] benefit from the uh rise and crash in in [31:55] prices. Um and then uh the third paper [31:59] uh is a theoretical exercise that is [32:02] trying to understand uh the interaction [32:05] between humans and AI. So that will be [32:07] the new element here not just [32:09] interaction between AI uh but rather uh [32:13] if you have a framework with AI and [32:15] humans uh what's going to happen so [32:18] that's maybe another take on the [32:19] question that Chavier was asking uh if [32:23] if humans are going to survive of course [32:24] this is more limited in the context of [32:27] of this uh model [32:29] okay so let me uh start with uh the the [32:32] first uh and the main paper and I will [32:35] spend most of the time on this and then [32:37] give you a couple of highlights from the [32:39] other uh follow-up uh papers. Um and as [32:45] I mentioned the way we think about it is [32:47] we set reinforcement learning algorithms [32:51] to trade in financial market. Uh and we [32:55] basically see how they learn to behave, [32:57] what kind of equilibrium uh they end up [33:01] uh with. Um reinforcement learning is a [33:05] basic building block in many of the AI [33:08] tools uh that you see uh that are being [33:11] used. Uh what you see here is the game [33:13] go and alpho which is an example of [33:16] reinforcement learning algorithm [33:18] learning how to beat a human in a very [33:21] complex game. uh the reinforcement [33:24] learning algorithms that we are going to [33:26] work with are going to be simpler than [33:28] that but they have the same uh basic [33:31] principle of reinforcement learning that [33:34] I will explain uh in in a second. [33:38] So um how does reinforcement learning uh [33:42] work? Uh a very quick overview of uh [33:47] what reinforcement learning is uh and [33:50] how it works. So the version of [33:52] reinforcement learning that we are [33:54] working with is known as uh Q-learning. [33:58] Um and when you think about Q-learning, [34:00] imagine that the algorithm has a big [34:03] matrix where on one dimension you have [34:06] the state of the world and on the other [34:09] dimension you have the action and every [34:12] cell in this matrix is essentially what [34:15] is the value that you expect to get if [34:18] you take a given action in a given uh [34:21] state. Okay. Um the reinforcement [34:24] learning algorithms are basically thrown [34:26] into the financial market without any [34:27] prior knowledge but then they try [34:30] different things and they learn over [34:32] time what works uh best and this is how [34:35] they uh evolve. Of course, what what is [34:38] most interesting is not uh when you have [34:41] just one uh reinforcement learning [34:43] algorithm playing against nature uh but [34:46] rather a bunch of them that are [34:48] interacting with each other because then [34:50] uh the action that each one of them [34:52] takes is also affecting the state the [34:53] state of the world for the others. So [34:55] this is generally quite a complex uh [34:58] environment that is very difficult to uh [35:01] predict uh and this is why the [35:03] experiments here are so uh useful. So [35:07] how does the um Q-learning algorithm [35:11] update the different cells? Well, every [35:14] period uh the algorithm is going to pick [35:17] an action uh for a given state and then [35:20] they will have a new observation [35:23] with which they can then update the [35:26] value in this particular cell. Okay. Uh [35:31] how do they update? uh they do uh some [35:34] mixing between the new information they [35:37] just gathered, the new experience they [35:39] had and what was the historical [35:42] knowledge that they had. Okay, so they [35:45] basically do some kind of a weighted [35:47] average between the history and what [35:49] they just uh experienced and then every [35:52] period the algorithm will have to choose [35:55] uh which action uh to take. And the way [35:59] that they do that is with some [36:01] combination between exploration and [36:04] exploitation. Well, exploration means [36:07] that you are just randomly picking an [36:09] action out of the set of available uh [36:13] options and then uh exploitation means [36:16] that you are picking the action that [36:18] currently based on your information is [36:21] going to give you the highest value uh [36:23] for this uh particular uh state. Okay. [36:27] So with this combination of exploration, [36:30] exploitation and then updating using [36:32] both historical information and the [36:35] recent experience uh the Q-learning [36:38] algorithm is evolving and updating the [36:40] matrix and then uh again the interaction [36:43] between uh the different uh algorithms [36:46] is is quite interesting and uh trying to [36:49] understand what kind of equilibrium uh [36:51] they get to. Okay. So why uh do we think [36:56] about it in uh the context of financial [36:59] markets? Well, AI is clearly everywhere [37:03] as we heard and we'll hear uh later in [37:06] this uh session. Um and uh financial [37:10] markets are quite a natural uh [37:13] application. Um of course it's very hard [37:16] to have hard data on which firms are [37:18] using what tools. However, there are [37:21] emerging anecdotes that reinforcement [37:24] learning algorithms are uh increasingly [37:26] being used for example in crypto [37:29] markets, in FX markets uh and and [37:32] others. The question that we address in [37:35] this uh first paper is a question about [37:38] uh collusion and in the broader economic [37:42] context uh the concern about AI [37:44] collusion has come up in different ways. [37:47] uh for example of course gas stations [37:50] that are setting prices based on uh [37:53] reinforcement learning algorithms uh in [37:55] real estate markets. These are different [37:57] types of collusion but in general there [37:59] is always the concern that once you let [38:01] algorithms choose their behavior uh you [38:05] will move away from competition and we [38:07] are trying to understand the extent to [38:09] which this happens in a financial [38:11] market. uh financial markets are quite [38:14] different uh than all these other [38:16] settings. Uh think about the noise [38:18] traders for example that are uh [38:20] constantly there and uh injecting noise [38:23] into the price. Think about the price [38:25] setting mechanism market maker that is [38:28] taking all the information and trying to [38:29] set the price uh in a fair way. Um so [38:34] all these things are making the [38:35] financial market quite a different [38:36] environment and so uh it is uh I think [38:40] very important to try to take uh these [38:43] uh frameworks and apply them in [38:44] financial markets and then see the [38:46] extent to which uh we are going to see [38:49] this kind of collusion. Our definition [38:52] of collusion is what you see there on [38:54] the slide. uh basically autonomous [38:58] self-interested algorithms learn to [39:00] achieve and maintain coordination [39:03] without agreement without communication [39:06] or even uh without uh intention. Okay. [39:10] Um so that basically means that they [39:13] somehow learn over time how they can [39:16] coordinate on a non-competitive behavior [39:19] that benefits them in the in the long [39:22] run. um even though in the short run [39:25] they could have made a profit if they [39:28] deviated and we're going to talk about [39:30] what are the mechanisms through which [39:32] they uh learn to to collude. [39:36] Um I'm not going to get too much into [39:39] that but I will say that uh AI collusion [39:42] is very important very challenging for [39:46] uh regulators uh in general financial [39:49] market regulators are always trying to [39:52] make sure that the market is competitive [39:55] and when they see signs of collusion uh [39:58] they are taking all sorts of enforcement [40:01] actions against that. However, typically [40:05] uh when humans are involved uh [40:07] regulators are going to look for signs [40:09] of actual communication among people uh [40:13] to suggest that there is collusion and [40:16] this kind of uh communication does not [40:18] exist with the AI algorithms. So [40:21] basically they learn how they can act [40:25] less competitively without any [40:27] communication just through this uh [40:29] reinforcement learning. uh and so it [40:32] potentially opens the door to some [40:34] broader questions about regulation and [40:37] how we should think about collusion and [40:39] competition in a market that is uh [40:42] populated by AI uh algorithms. And I [40:46] think those are questions that uh people [40:48] should uh come back to uh in the future [40:51] when thinking about uh new financial [40:53] market uh regulation. [40:56] Um, you know, some of these concerns [40:59] have been raised uh before by uh policy [41:02] makers. Uh, we have here a quote from uh [41:05] the previous SEC chair Gary Gensler [41:08] where he says that financial market [41:10] instability or even a financial crisis [41:12] caused by AI is nearly unavoidable [41:15] without regulation. This is maybe more [41:18] related to the second paper that I'm [41:19] going to mention more briefly in a few [41:21] minutes. But the second part of the [41:23] sentence is directly related to this [41:25] paper where uh he says even if the [41:29] humans aren't talking the machines will [41:31] start to have a sense of cooperation and [41:33] we've already seen this in high [41:34] frequency trading. So I'm not sure if [41:36] we've seen that or not but it's [41:38] certainly a possibility and uh when you [41:41] experiment with that and and you see how [41:44] uh they behave in the lab uh I think it [41:47] gives you a a great uh sense um that [41:51] that this might be happening in in real [41:53] world financial markets and and might uh [41:56] increase uh in frequency uh in in the [41:58] future. I will also say that you know [42:01] collusion is not interesting just for [42:03] the sake of collusion. Uh but rather [42:06] once there is collusion and we see that [42:08] very clearly in our model and in our [42:11] experiments. Once there is collusion um [42:14] market quality is uh affected and we [42:17] basically show that with collusion uh [42:20] liquidity goes down uh price [42:22] informativeness goes down mispricing [42:24] goes up. So those are things that in [42:26] general we should pay attention to. [42:30] Okay. So uh I'm I'm going to be brief [42:32] here. Uh as I said this is a [42:34] non-technical uh summary uh of the of [42:38] the paper. Um and uh I will just say [42:41] that we set up a trade trading [42:44] environment that is influenced by the [42:47] classic Kyle uh 85 model uh but then [42:51] extended along uh several dimensions. Uh [42:55] so generally speaking there will be [42:56] several informed speculators who use uh [43:00] Q-learning to guide their trading [43:02] execution decisions. There will be noise [43:05] traders uh whose trades are not based on [43:08] information. Uh there will be [43:10] sophisticated market makers that trade [43:12] strategically against speculators. [43:15] And there will also be a large group of [43:18] information insensitive investors. Those [43:20] are known in some of the literature as [43:22] preferred habitat. Those are the [43:24] investors who are there in the long run [43:26] not paying attention to short-term [43:28] fluctuations in value and just trade [43:31] based on long-term expected uh value. Um [43:35] what we basically find is that AI [43:39] collusion emerges uh robustly uh through [43:43] two distinct mechanisms. Um one of them [43:47] is uh known as price trigger strategies. [43:51] Um and um this is basically the idea [43:55] that um the uh algorithms uh come to [43:59] believe that if they uh start trading [44:03] competitively and trading competitively [44:05] means trading aggressively then the [44:08] price is going to move uh significantly [44:11] and then once the price moves then uh [44:14] there will be some punishment. Okay. So [44:16] this is the typical notion of collusion [44:18] that is generated by a potential uh [44:21] punishment. Uh and we call this [44:24] artificial intelligence because here it [44:25] seems like the algorithms are learning [44:28] to behave uh in quite an intelligent [44:31] way. Um in another range of parameters [44:35] we basically see that there is collusion [44:37] that is driven by a very different [44:39] mechanism uh more akin to a bias in [44:43] learning where uh the algorithms just [44:46] end up not trading aggressively because [44:50] somehow in their memory uh trading [44:52] aggressively is associated with a very [44:55] bad outcome and then they just don't [44:57] come back to it and uh this is the uh [45:00] learning bias or overp pruning. They [45:03] kind of overp prune uh possible uh [45:07] actions. Um and we call this artificial [45:09] stupidity because basically here they [45:13] end up with a collusive outcome but not [45:15] because of something particularly [45:18] sophisticated. Uh rather this is more of [45:21] a uh bias. Um I'll show you here the [45:25] main uh pattern that we see out of the [45:29] uh experiments and then I will move on [45:31] to talk more briefly about the two other [45:33] papers. Um so what you see on the left [45:36] panel um in the horizontal axis you [45:40] basically see the noise of the market. [45:43] So sigma U is the standard deviation of [45:46] noise uh traders in the market. So the [45:48] higher uh sigma U is uh the more noisy [45:52] the market is. What you see on the [45:55] vertical axis are the profits that our [45:58] speculators are making uh over time uh [46:01] after the system converges. Okay. And [46:05] convergence by the way takes quite a [46:06] while but we report results after uh [46:10] convergence. Um you see two benchmarks [46:13] there. The red line is what we call the [46:16] non-olusive [46:18] Nash equilibrium. Um so this is where [46:22] speculators are acting competitively and [46:25] in this case you see that they make [46:26] quite a low profit. Okay. Uh the black [46:30] line is another benchmark uh which uh we [46:33] think of as the perfect cartel. Uh this [46:36] is where the speculators merge and uh [46:39] make their decisions jointly. And [46:42] obviously when they make their decisions [46:44] jointly uh they are going to uh make [46:47] high profits because then uh they trade [46:49] much less aggressively, much less [46:51] competitively and they maximize their [46:53] joint profits. And in the middle is the [46:57] profit that we observe uh in equilibrium [47:00] after all the rounds of of experiment. [47:03] And what you basically see is when the [47:06] noise is very small, they manage to [47:09] achieve a very high profit. And when we [47:12] look under the hood and see what kind of [47:13] mechanism leads to that, we find [47:16] evidence that is consistent with the [47:18] price trigger uh mechanism. Okay. So [47:21] they manage when the market is not too [47:23] noisy to coordinate on the price as a [47:26] device for collusion, trading less [47:28] aggressively, making higher profits. As [47:32] noise becomes bigger, this becomes more [47:35] and more difficult to sustain. [47:38] Um and this is because once the market [47:40] is very noisy uh then uh the price [47:43] ceases to be a very useful coordination [47:46] device and then you see how their [47:48] profits deteriorate and they do not [47:51] really manage to collude anymore. [47:53] Interestingly, at some point when the [47:56] noise becomes very loud, uh, profits [47:59] spike up again. Uh, and this is where we [48:01] detect the other type of collusion that [48:03] is more kind of a stupidity based [48:05] collusion where because of overp pruning [48:08] and learning bias, they end up uh, [48:10] achieving higher profits. And we go in [48:13] the paper into much more detail on uh [48:16] how these mechanisms evolve and and sort [48:19] of providing more evidence consistent [48:21] with those different uh mechanisms. [48:24] Okay. So this is the the first uh and [48:27] and the main paper uh as I said and [48:29] again the main insight is that uh these [48:32] reinforcement learning algorithms [48:34] essentially learn over time via [48:36] different mechanisms to behave in a way [48:38] that is not competitive that ends up [48:41] generating higher profits in uh [48:44] equilibrium. Uh let me briefly uh I [48:48] think I have a few more minutes right? [48:52] >> Yeah. So let me briefly talk about the [48:54] two follow-ups that that I uh mentioned [48:57] to kind of see where uh we are going [48:59] with it. Uh next. Um so the second one [49:02] which is also uh co-authored with Yanji [49:06] in in Hong Kong is basically extending [49:10] this framework to think about an AI [49:13] algorithm that is doing more than [49:17] reactive learning but is going more into [49:20] planning. What do I mean by this? So you [49:23] can think about the kind of AI collusion [49:25] that I described in the first paper as [49:28] reactive learning in the sense that the [49:31] speculators, the algorithms were getting [49:34] information and deciding based on the [49:36] information how to trade and they came [49:38] to some uh conclusion that it is best [49:43] for them not to trade aggressively and [49:46] this is how they essentially collude. [49:49] What do I mean by AI planning? [49:52] Well, this is where AI algorithms can [49:54] potentially set up a whole course of [49:57] prices that will benefit them over a few [50:00] periods. Okay? So, think about [50:03] algorithms that are starting to trade in [50:06] a way that generates a bubble and then a [50:10] crash. And since they uh know when the [50:13] bubble starts and ends, they sort of [50:15] benefit from the timing uh of it. Okay? [50:18] Okay, so they basically collude to [50:20] inflate prices and then bring them down [50:23] and make a profit on the way. This is a [50:26] more sophisticated version of AI and [50:29] reinforcement learning. And in this new [50:31] paper, what we are trying to do is put [50:33] that in the lab and see how it how it [50:36] works and the extent to which it can [50:38] indeed uh destabilize the the market. [50:42] So [50:43] um [50:45] yeah so I think I basically said that uh [50:49] let me just show you the the main uh the [50:52] main uh result here. Um so if you look [50:56] at the left panel um you basically see [51:00] how uh this kind of strategy of [51:03] inflating prices and then bringing them [51:06] down works over several periods. So at [51:09] time uh zero uh they start receiving [51:13] information about an oncoming uh shock [51:16] in fundamental. They coordinate to trade [51:19] the asset aggressively upwards um and [51:22] then downwards. And on the right panel [51:25] you basically see uh the pattern of [51:27] price exhibiting a very uh steep [51:31] increase and then a sharp uh decline. So [51:34] the the blue line here is uh basically [51:37] the the path of the price under this uh [51:40] more advanced version of of Q-learning. [51:44] Um the red line is what you would get in [51:47] case there was no type of collusion [51:49] between uh the AI algorithms. Uh the [51:53] green line is what you would get just [51:56] based on the fundamental uh alone. and [51:59] and basically you can see that here [52:02] under this version uh they kind of think [52:04] ahead. It's not just reacting to the [52:06] current news but it's also thinking we [52:09] are going to lead to a price increase [52:11] and then a price decrease and uh [52:14] basically we show through some of these [52:15] lab experiments that they learn how to [52:18] do that uh over time. Um so in the last [52:23] couple minutes uh that that I have um I [52:27] I will just say uh in the last paper [52:30] that is uh co-authored also with uh [52:33] Jiang Lee and Lean Young from uh Toronto [52:37] um we we basically tried to ask suppose [52:40] that you have AI and humans together in [52:43] one framework what happens and who uh [52:47] will benefit more. You can't really do [52:49] that in an experimental framework [52:51] because then you will need to bring in [52:54] humans and tie them to the computer for [52:56] quite a long time until things converge. [52:59] Um so we solved that with a theoretical [53:01] framework and making some simplifying [53:03] assumptions about the AI. Our basic take [53:06] on AI versus human is humans are good at [53:10] getting new insights. So they will have [53:13] new signals about the world but they are [53:16] not very good about uh at at [53:18] understanding how the market works. So [53:21] we model humans as having new uh sources [53:25] of information but they are klevel [53:28] thinking. So uh they don't understand [53:31] everything that is going on in [53:32] equilibrium. On the other hand the AI is [53:35] super rational does reinforcement [53:37] learning and understands everything that [53:39] goes on in equilibrium. So that's kind [53:40] of the trade-off uh between uh the two. [53:44] And when we model this and what you see [53:47] here is uh basically the gap between the [53:51] profit that the AI makes and the profit [53:53] that the human makes. Uh and on the [53:56] horizontal axis is how noisy is the [54:00] signal uh of the of the humans. And you [54:04] can see that humans at the end of the [54:06] day end up doing better than AI both [54:11] when they have uh a very precise signal [54:14] and when they have a very noisy signal. [54:17] So when they have a very precise signal [54:19] they basically have an information [54:21] advantage over uh the the AI. when they [54:24] have a very noisy signal their [54:26] information advantage is not that useful [54:29] but then AI doesn't really learn that [54:31] much because remember uh the advantage [54:33] of AI is that it can understand the [54:35] equilibrium better but if AI just [54:38] updates from what humans are doing and [54:39] humans don't have super useful [54:42] information then this advantage kind of [54:44] goes away so AI's ability are kind of [54:47] limited by what uh humans uh bring to [54:50] the table to to begin with and there is [54:53] only a region in the middle where AI is [54:55] doing significantly better. Um so I'll [54:59] stop here. Um I think in general this [55:02] kind of approach helps us understand uh [55:05] what can potentially happen when uh [55:08] humans are replaced with AI algorithms [55:11] employing uh reinforcement learning [55:12] mechanisms in financial markets. And we [55:16] talked about implications for collusion [55:18] for market stability and then also [55:20] thinking a little more about humans [55:22] versus AI and uh what could emerge as a [55:25] market equilibrium. [55:27] Thank you. [55:28] >> Thank you. [55:39] >> All right. It's my great pleasure to be [55:42] invited to this panel along with this uh [55:45] um excellent scholars that I admire. Uh [55:48] so uh Robert asked me to have some fun [55:51] uh in this panel discussion. So instead [55:53] of uh um you know showing uh some of my [55:57] research which I find boring now uh I'm [56:00] going to try to entertain another [56:02] thought experiment with audience. Uh so [56:05] uh the question I'm going to address is [56:08] on can machines invent the efficient [56:11] market hypothesis. All right. So I'm not [56:13] here to legitate uh the EMH hypothesis [56:17] empirically. As far as I know uh the [56:20] debate has not been settled in the [56:22] Chicago yet. So instead um I am trying [56:26] to um you know use this question to [56:29] motivate um our discussion about um the [56:33] usefulness of geni and its uh limitation [56:37] and then hopefully I um the framework [56:39] that I'm going to propose uh can help us [56:42] organize some of our thoughts about geni [56:46] and um uh with that I want to I want to [56:48] say um I want to give a disclosure first [56:50] that um you know the framework I'm going [56:52] to propose is incredible really simple [56:55] there's no math it's completely heristic [56:58] um and uh um you know it does not meant [57:01] to be rigorous [57:02] all right so let's get started so I'm [57:05] going to start with um with a precise [57:07] definition about geni all right in my [57:09] view okay so uh if you think about uh at [57:13] a high level right what is a generative [57:15] model generative model is as simple as a [57:18] probability distribution from which we [57:19] can sample data all right so we have [57:21] been using you know any kind of software [57:23] where to generate random variable from a [57:25] normal distribution and that is a [57:26] generated model. So what distinct GNAI [57:30] is its training medium as well as uh its [57:35] scale. All right. So um so in terms of [57:38] the scale right so um the geni is is use [57:42] using pretty much all the data available [57:45] uh to date we have on the internet and [57:48] um and for that purpose you need a you [57:50] know substantially large model uh to be [57:53] able to have the capacity to remember [57:54] such uh such information uh in addition [57:57] the data are actually uh human produced [58:00] um I would call it um symbolic artifacts [58:03] all right and um that's why the term AI [58:05] geni is pretty uh is very precise [58:08] because the term AI was uh introduced to [58:11] to kind of represent you know [58:13] sophisticated algorithms to mimic human [58:16] being right so you know this is very [58:18] very good um you know um understanding [58:21] or the use the use of this term is is [58:24] pretty uh good here um and and what's [58:26] cool here is that human knowledge right [58:29] so happens to be recorded uh you know or [58:33] u represented uh in the same um you know [58:36] symbolic artifacts right so as a result [58:39] um this geni not only can generate some [58:42] of the data and the text and code is [58:44] actually generating human knowledge [58:46] right in a sense all right so um and [58:49] that is a great deal right so and in [58:52] addition I mean I could say I I I like [58:55] to say that this geni system is you know [58:58] the largest system that we human being [59:00] have ever estimated by far um you know [59:04] uh it captures there's um it captures um [59:07] our um you know uh knowledge accumulated [59:10] over thousands of years. Uh that being [59:12] said, right? So there are here are some [59:14] important caveats, right? So these geni [59:17] system they do not reason in the human [59:20] sense. All right? So they give you some [59:22] kind of approximate reasoning while [59:24] patent compilation. In addition, they're [59:27] not database, right? So there's they're [59:29] just like a lossy and uh structured [59:32] compression of human knowledge and um [59:36] they do not know truth right so they are [59:38] just u generate output that are [59:40] statistically plausible all right um and [59:44] given given the context [59:46] all right so um I am going to I'm going [59:50] I'm going to take a you know framework [59:52] of thinking about geni not as a machine [59:55] to give you answers not as a tool not as [59:58] a um decision maker but as a prior. All [60:02] right. So it's a mechanism to to provide [60:04] you with a very rich prior over language [60:08] over ideas over uh relationship and um [60:12] this is perhaps the first time in [60:14] history right to have a prior right this [60:17] rich and uh s sophisticated and uh [60:21] reusable right for our finance research [60:24] right so we should be really happy with [60:26] that um and think about this as a prior [60:29] so the model output are essentially you [60:31] know conditioning on the context or are [60:34] generated. So they're random in nature, [60:35] right? So they're generated given uh [60:38] conditional on the context, right? [60:40] Condition on the prompt uh and uh the [60:43] prior information. All right? And [60:45] because it's actually a big prior so you [60:47] know and you're using this prior to make [60:49] inference. So you're essentially you [60:51] know have to your inference inherently [60:54] um you know um uh have the same kind of [60:57] strengths and the bias um of the um um [61:00] of of the uh priors. [61:03] All right. So intuitively right using a [61:05] large language model is like having [61:07] access of entire library of uh experts [61:12] in your pockets at once. All right. So [61:14] think about you as a financial economist [61:17] think about having you know Jin Farama, [61:19] Lars Hansen, Richard Thaylor, right? Doc [61:22] Diamond, right? 724, right? So you can [61:25] ask them questions all the time. This is [61:27] such an elevated Chicago experience that [61:30] I've never had, but you know something [61:31] that is in your pockets now. Uh but keep [61:34] in mind that these experts, right, they [61:38] may disagree, right? So in other words, [61:41] you you you're not going to get [61:42] something that is 100% correct, right? A [61:45] prior is not a proof. [61:48] All right. Now, speak of prior, right? [61:51] So why priors matter in finance? So I [61:55] would argue that finance is inherently [61:58] uh you know is is fundamental in a weak [62:00] signal uh environment, right? Think [62:03] about uh measuring you know some [62:05] interesting economic uh economic [62:07] quantities or return portability or [62:09] alphas. These are uh incredibly [62:11] difficult quantities to measure right. [62:13] So how do we you know agree with the you [62:15] know risk premium on everything we we [62:17] would probably not need to do any more [62:19] studies and and also the identification [62:21] of economic mechanisms right some of [62:23] these econisms are really ambiguous and [62:26] hard to distinguish and also to evaluate [62:29] to evaluate counterfactuals. So these [62:31] are really noisy and imperfect uh [62:34] activities. uh now um because of that [62:38] right so if you kind of use some [62:41] research you use some approach that [62:43] completely based on data and then the [62:46] conclusions may be really fragile right [62:49] so um that's why we need some ideas to [62:51] kind of impose uh regularization right [62:54] or structure whether they can be you [62:57] know uh explicit such as you know [63:00] shrinkage such as you know uh using some [63:02] sort of machine learning techniques or [63:05] It could be um something quite implicit [63:08] uh like a theory like a institutional [63:11] details institutional knowledge or it [63:13] could be something like a researcher [63:14] judgment right so uh but they are kind [63:17] of more broadly in the form of prior [63:19] okay so we use priors to regularize a [63:22] problem otherwise just using data we [63:25] will get conclusions that are generally [63:27] very fragile [63:29] now uh to illustrate the point um [63:32] weakness uh in the signals right So I'm [63:35] kind of showing you a histogram of a [63:38] squares collected from top finance and [63:40] top economic journals. All right. And [63:42] you can see that these osquares uh had [63:45] these are hand hand collected osquares. [63:47] They're about like 20,000 osquares uh we [63:49] published in the last year I mean 2022 [63:52] alone. Um and as you can see there is a [63:55] huge spike near zero. But keep in mind [63:58] that these are published o squares. All [64:00] right. And many of them are in sample o [64:03] squares. Okay. A as a result, right? So [64:05] the real R squares, right, we we kind of [64:08] uh you know, deal with are kind of only [64:11] going to be, you know, more to the left [64:12] or closer to to closer to zero. [64:15] Um you know, and and I'm kind of trying [64:18] to use this point to illustrate that the [64:19] signals we're dealing with are actually [64:21] weak. Uh I mean there is actually a [64:24] broader point to make, right? So if you [64:26] think about given kind of the um given [64:30] what we already know, right? any new [64:32] findings right must be based on [64:35] relatively you know must be trying to [64:37] find relatively weak uh signals [64:39] otherwise somebody else would have [64:41] already found it right given current [64:43] technology right so um therefore I think [64:45] this point of uh you know um weakness uh [64:49] you know in terms of signal weakness is [64:51] actually everywhere it's not just about [64:53] arbitrageers alpha it's it's everywhere [64:56] in in social science discoveries as well [64:59] anyway so now now that we know prior uh [65:03] should uh help you and uh you know uh [65:06] and and in particular to deal with this [65:07] weak signals. So let's talk about how [65:09] genai can help finance research right. [65:12] So um broadly speaking you know uh [65:15] Jennai provides you with a with a with a [65:18] prior and this prior can help you [65:20] aggregate information and imposing [65:22] structure as I have indicated now in [65:26] practice. Right. So some of the examples [65:28] the previous uh panelists have shown you [65:30] actually fall into these categories [65:31] right so if you think about it you know [65:33] we have been constructing [65:36] representations [65:37] right so of unstructured uh information [65:40] right so we have been you know turn [65:41] unstructured data into structured data [65:43] that's kind of ways of imposing [65:44] structure using prior using larger [65:46] language models so we have created for [65:48] instance uh embeddings right so news [65:51] embeddings uh we have created uh [65:54] sentiment which is essentially a you [65:56] know embedding as well like [65:58] onedimensional right so um you know [66:01] positive and negative embeddings uh we [66:03] have have topics right which is another [66:05] type of embeddings right so those are [66:07] kind of representations um that we use [66:09] our priors from large language model to [66:11] kind of help us to uh actually uh you [66:14] know uh turn nonstructured data into uh [66:17] we can also use large language models to [66:19] retrieve information right to um to uh [66:22] synthesis information right so to to [66:25] collect uh some particular particular uh [66:27] you know uh say risk disclosure in the [66:29] in the 10k 10q files and things like [66:31] that. Uh speaking of embeddings, we also [66:34] can create a firm embeddings, asset [66:36] embeddings as the literature have shown [66:37] you. But in any case I think um you know [66:40] but but more more broadly right more [66:42] broadly um I think um you know financial [66:46] research involves uh a lot of [66:48] exploratory uh exploratory work uh such [66:51] as combining you know data narratives [66:54] institutional details and prior evidence [66:57] to form hypothesis to select structures [67:01] uh and to refine economic intuitions. I [67:03] think genis are generally helpful uh in [67:06] all these exercises. Um but some of [67:09] these upstream uh you know um process uh [67:13] of research are um you know they are [67:16] central to our design to identification [67:19] to interpretation but they are kind of [67:21] not formal enough right so for us to [67:24] kind of um know uh site them or or [67:27] actually u you know mention them uh in [67:30] in our study yet. All right. So where am [67:34] I now? Let me see. All right. So now [67:36] let's talk about this question, right? [67:37] Can Jenny and I come up with a efficient [67:39] market hypothesis? Well, let's start [67:41] with some simpler ones, right? So could [67:43] AI discover return predability? I think [67:45] a lot of research have shown positive [67:47] evidence that AIs can find alphas and [67:49] can find uh high sharp ratios including [67:51] some of my own. So I would say extremely [67:53] well. Could AI document anomalies? [67:57] Right. I think uh you know a paper by um [68:00] Nory Marox and Billy Coff they have [68:02] actually tried to generate anomalies at [68:06] scale and their exercise was very [68:08] interesting and and quite successful [68:10] right so I'd say yes all right but could [68:13] AI independently invent EMH as an [68:17] equilibrium concept about information [68:20] competition and price formation without [68:23] being told to think in those terms So in [68:27] other words, had Jane FMA had that geni [68:30] tool in 1970s [68:33] well not not Jen not Jane farmer I guess [68:35] but you know had somebody have that tool [68:38] in 1970s what will what be be able to [68:40] you know propose uh efficient market [68:42] hypothesis uh I think under the current [68:46] um paradigm of of Jani I think it's very [68:49] unlikely um you know at a deeper level I [68:52] think EMH is not just a pattern about [68:55] predability [68:56] It's about reframing. It's about [68:58] framing. It's about uh you know [69:00] conceptualizing this price as [69:02] equilibrium outcome. And these are [69:04] things that are new uh and they're not [69:06] in the data of AI training. And um [69:10] they're not uh you know particularly uh [69:13] you know uh score well with the uh with [69:15] the um objective function of geni which [69:18] I'm going to talk more about right now. [69:20] Right. So why this is hard for geni? [69:23] um you know um I think I think this has [69:26] something to do with um the the the um [69:28] the training algorithm um and also um uh [69:32] the availability of data um uh I uh [69:37] let's let's start with another analogy [69:39] if you're not like the EMH example let's [69:42] talk about general relativity right so I [69:44] would also think that general relativity [69:46] is not something that Jenn and I can [69:48] come up with independently um because [69:51] you know in at theory uh it's not like [69:54] based on experimental data right so in [69:56] fact uh you know data only come later [69:59] experiment only come later after theory [70:00] has been proposed to verify it uh and [70:03] the theory essentially says that gravity [70:06] uh is is actually redefined as uh as as [70:10] a space-time cvature right instead of a [70:12] force uh that's a you know brand new [70:15] idea uh and based on uh invarian [70:17] principles and new primitives I don't [70:20] think Jenny is trying to to to to do [70:23] that. Uh another distinction um holds in [70:27] mathematics as uh you know Zavia [70:29] mentioned earlier. So lots of Jani uh [70:32] companies are trying to sell their great [70:35] um uh you know uh benchmarks in terms of [70:37] Olympian math, right? But Olympian math [70:40] is essentially a closed system with uh [70:44] with theorems that you know are right [70:46] and you can try different ways to prove [70:48] it but you know they're right. Right. So [70:50] and and you know the llamas are the [70:52] number of llamas theorems are finite um [70:55] but re research mathematics is [70:58] completely open territory right so you [71:00] don't know whether a conjecture is true [71:01] or not so in this situation it's hard [71:04] for Jenn and I at this stage to actually [71:07] compete at human level um now um to [71:11] summarize right so I think uh if the [71:13] patterns in the data are really weak or [71:16] you don't have enough data then geni [71:18] would suffer [71:19] um if the objective or you know the [71:22] reward or a reward or loss function in a [71:26] reinforcement learning environment uh [71:28] are ambiguous and then AI would be uh it [71:31] would be difficult for AI to actually uh [71:33] to to actually help. Um uh the last part [71:37] I can also um I can also think of is uh [71:41] for instance you know in a reinforcement [71:43] learning scenario right so you have to [71:46] uh prespecify um the action space uh you [71:50] know the state space right so in other [71:53] words um you know if the true solution [71:56] if if if the truth lies outside the [71:58] space that are predefined that would be [72:00] a completely uncharted territories for [72:02] AI for for reinforcement learning [72:04] algorithms and that would not be [72:05] possible for reinforcement learning [72:07] algorithms to actually find out those [72:09] solutions. So, uncharted territories are [72:11] things that I think the current [72:12] algorithms would not be able to uh to [72:14] actually uh deal with. All right. So, uh [72:18] given that I have actually uh set up [72:20] this framework, let's also think about [72:22] what else we can learn from it. Right? [72:24] So, um you know, as I already mentioned, [72:26] each generated output can be regarded as [72:29] a draw from a you know, learned prior. [72:32] So, the prior is learned from the data. [72:33] Right? So it's a big geni model over [72:36] some latent representations like [72:37] embeddings um obtained on the uh and and [72:41] conditioning of the prompt. So this is [72:42] how they understand the output from a [72:44] geni. Um and hallucination now becomes [72:47] pretty easy to understand right. So [72:48] hallucination arises because um it's [72:51] it's essentially give you something that [72:52] is statistically plausible but is but [72:55] but is not uh necessarily true in a [72:57] given context. All right. So um and the [73:01] learned priors will also in inevitably [73:05] um uh inherit biases presented in the [73:08] historical uh data we human being [73:10] generated right so in other words you [73:12] know AI trained with human data will [73:14] mimic humans that's not you know that's [73:16] going to happen um and also uh you know [73:19] confidence um right so reflects this [73:23] within frame uh certainty [73:26] uh you know so in other words if your [73:27] prior is highly heavily concentrated on [73:31] the continuation and then uh you know [73:33] the generated uh you know sentences will [73:36] be you know looks like really confident. [73:38] So this is like you know this is why [73:40] sometimes when we see AI hallucinate it [73:43] it it it generate it it it it looks like [73:46] pretty uh affirmative um but that uh [73:48] that confidence is actually because the [73:51] prior is heavily concentrated on around [73:54] this continuation. Now on the other side [73:57] uh you know sensitivity to prompt uh is [74:00] kind of really about the fact that you [74:02] know um your prompt moved a little bit [74:04] to uh different your prompt move uh move [74:07] the prior to a slightly different um [74:10] region and as a result um you know uh [74:12] the generated response may be may be um [74:15] you know uh quite quite uh sensitive to [74:18] to that but in any case I think um prior [74:21] would be able to aggregate information [74:22] as well but they're not able to revise [74:25] objectives they are not able to discover [74:28] new structure. [74:30] All right. Now speaking of you know if [74:32] you think about priors uh more seriously [74:35] and then you know it limit to the [74:37] following question um which you know I'm [74:40] not sure if this is a good topic to [74:42] discuss here but so it's about research [74:44] norms right so turns out geni has [74:47] already been adopted right in in the [74:49] research process uh including upstream [74:53] exploration as I already discussed as as [74:56] as well as reviewing all right so uh in [74:59] fact you know according to a recent [75:00] science and the nature uh articles and [75:03] news articles and also research articles [75:05] we find the majority of I mean not [75:07] necessarily majority but lots of [75:08] researchers have been using um Jenny I [75:11] tools not only for writing papers but [75:13] also for reviewing papers all right so [75:16] and um on the other hand you know many [75:20] researchers feel uh you might call it AI [75:23] shame or uneasy acknowledging the fact [75:27] that we're using AI um you particularly [75:31] uh when it occurs upstream of formal [75:34] analysis. So uh my view is that I think [75:37] this is more like a reflection of um the [75:41] uh mismatch between existing academic [75:44] norms and um uh how we actually uh how [75:49] research is actually conducted uh when [75:52] externalized the priors uh become [75:54] available. Right? So as as we have said [75:57] we now have externalized the priors and [75:58] typically researchers are you know um [76:01] using priors in their own mind right but [76:04] now you have externalized the prior [76:05] you're using it do you have to knowledge [76:07] it you know it's it's hard question but [76:09] in any case uh I'm going to follow the [76:11] norm the existing norm here and disclose [76:14] that the slides were prepared uh with [76:17] the help of geni um but my own judgment [76:22] uh still matters here I decided which [76:25] things that I'd like to include in my [76:26] slides. So all errors are mine. All [76:29] right. So now let me talk about some [76:32] implic implications uh for financial [76:34] economists. [76:36] All right. So um I think um you know um [76:40] this externalized the prior is really [76:42] providing us with a is a golden [76:44] opportunity um that kind of help us [76:46] level the playing field uh and uh [76:49] raising the baseline u productivity [76:52] across the profession. Right. a lot of [76:53] things that now um you know we can do it [76:56] easily uh such as data analysis uh such [76:59] as polishing or writing things like [77:00] that. Uh so which means that um you know [77:04] uh the comparative advantage is going to [77:07] be shifted away from routine [77:08] implementation. [77:10] All right. So um uh in addition these [77:13] new AI tools also provide us with [77:15] interesting new things we can study new [77:17] topics we can study. We can study human [77:18] AI interactions. we can model you know [77:21] human beliefs better and things like [77:22] that. we can conduct surveys you know um [77:26] but um you know as I have uh mentioned I [77:29] think now um the bar for novelty for [77:33] original ideas uh probably also rise um [77:37] and uh uh you know we we really are [77:40] looking for uh ideas that can complement [77:44] uh AI right by by shaping or challenging [77:47] or reinterpreting the priors implicit uh [77:50] in the AI output um And I would I would [77:54] I would suggest economists to uh use AI [77:57] actively and not be ashamed of it uh [77:59] using it but cautiously um because you [78:02] not only have to um you're still the [78:05] decision maker right you still have to [78:07] judge whether um you know whether you [78:09] agree or disagree with output your [78:11] internal prior still matters but you [78:14] know now nowadays you also have to kind [78:16] of uh you know be responsible for the [78:18] output that you decide to include right [78:19] because you have to oversight you have [78:21] you you have to be uh you carefully [78:23] oversight the AI output, the generated [78:25] AI output. Now, I could end my talk [78:27] here, but you know, so um I I I cannot [78:32] resist uh the idea of the next question. [78:37] So why we have a pan of humans given [78:39] that you all have a prior, right? you [78:42] have the prior of you have experts you [78:44] have you have a vast library of experts [78:46] of all kinds and you can ask better [78:49] questions right than the questions that [78:51] you're going to ask us um so I think [78:55] maybe part of the answer is um you know [78:58] you want to see some kind of judgment [79:00] formed in real time you want to see us [79:02] disagree with each other u you know you [79:05] might want to see some you know [79:06] embarrassment [79:08] um but I think um you know um uh the the [79:13] human touch is still like matters a lot [79:15] in particular when the objectives are [79:17] ambiguous and the uh research questions [79:20] are being formed. Uh well to end my talk [79:24] you know I'm going to ask I'm going to [79:25] show you what I did with um with the [79:27] chatbt. So I'm I'm kind of asking chatbt [79:30] to uh to actually guess what other [79:33] panelists might be thinking about. [79:36] Right. So my panelists are all like uh [79:38] you know their priors are already in [79:40] chatbt. So I think chatbt can do a [79:42] pretty good job you know uh knowing what [79:45] they're doing. And um these are actually [79:48] generated scripts uh in the spirit of [79:52] geni why don't we create uh some digital [79:56] avatars and have them actually can you [80:00] play the video please? Uh have them [80:02] actually discuss [80:04] >> provocative question. Genai summarizes [80:06] papers, writes code, and drafts referee [80:09] reports. Productivity shock or [80:12] redefining what a finance scholar is. I [80:15] think it's a productivity shock, but not [80:16] because AI replaces researchers. Geni [80:19] collapses fixed costs, data cleaning, [80:21] boilerplate code, and spec testing that [80:23] used to take weeks. That shifts the [80:25] constraint. Execution isn't scarce [80:27] anymore. Problem formulation and [80:29] judgment are. In finance research, Jenni [80:32] raises the return to taste. [80:34] I broadly agree with a caveat. [80:36] Productivity shocks aren't neutral. When [80:39] you slash the cost of producing results, [80:41] you flood the field with results. Then [80:44] selection becomes the challenge. If [80:46] everyone runs 10,000 specs, which ones [80:48] matter? Theory, structural thinking must [80:50] discipline the search or we drown in [80:52] correlations. Let me come at it [80:54] differently. In finance, we study how [80:56] information shapes behavior and [80:58] stability. Jedi changes the information [81:00] environment itself. Everyone gets [81:02] instant interpretation and narratives. [81:04] It may amplify hurting, overreaction, [81:06] and coordination failures. So, Geni [81:08] isn't just a tool. It's an object of [81:10] study. [81:11] >> Models should treat agents as delegating [81:13] cognition. [81:14] >> All right. [81:15] >> I think the AI did not quite get [81:17] everyone's accent, but uh you know, with [81:20] that, uh let's have the real panel [81:22] started. [81:32] All All right. So, um I'm gonna ask a [81:35] few questions first and then we can open [81:37] it up to the audience. Um I think for [81:39] this if we can answer kind of in the [81:41] reverse order we presented so that you [81:44] can come to me last and I can tell the [81:46] people if they were right or not. Um but [81:50] uh I was going to start with some big [81:52] questions but kind of with how um Da [81:55] Chang [81:56] ended things I I guess I want to start [81:58] with a little littleer question which I [82:01] you know bothers me a lot which is [82:03] should we be citing AI and will we be [82:05] citing AI in a few years? [82:09] Oh, so all right. So, um I mean I I [82:13] think this is a good question. That's [82:14] why I kind of started this in my slides. [82:17] Um I mean as you can see from my view um [82:21] I think priors are something that we I [82:24] mean I I really think of AI just like a [82:26] externalized the prior um and uh when [82:30] you use a prior in upstream analysis [82:33] exploratory analysis and before you know [82:35] a formal paper is written it's hard to [82:38] tell you know whether somebody uh [82:40] whether it should be given the credit um [82:42] so that that's why I would be you know [82:45] more cautious about giving add it to [82:47] chat GPT or some other gen AI tech. [82:50] Yeah. [82:52] >> Yeah. I I I would say generally uh [82:54] probably not. Um I think uh AI is an [82:59] input that goes into research and not a [83:02] collaborator on on research. So I would [83:04] tend not to site it. [83:06] >> Well, I think we should of course site [83:08] AI. I mean let's take a limiting thought [83:10] experiment. uh suppose you have an AI [83:13] who finds a a new big proof of something [83:18] uh say [83:20] hypothesis something of course it should [83:21] be cited now of course in the that maybe [83:24] that will be in 50 years uh maybe in the [83:27] meantime it will just be some some lab [83:29] that will have a very very good software [83:31] and they will acknowledge the input of [83:33] CH GPT in a very special [83:36] um special software that they want fact [83:40] that at some point if machines can [83:42] produce essentially the whole thing by [83:44] themselves. It will be the machine may [83:46] be helped by you know a few a few few [83:49] humans in the background. Yeah, it [83:51] should be cited. So my view is maybe in [83:55] 50 years it will be writing the proofs [83:57] and citing itself but until then it's a [84:00] tool and you have to take responsibility [84:03] for the output and um I just think it's [84:06] crazy that we you know are asked or [84:08] required to site a tool that we use in [84:11] research production because ultimately [84:13] the work is yours and if you're good at [84:15] using a tool um that's part of your [84:18] research production and but um I I won't [84:22] make any of my students site uh AI and I [84:26] don't think we will be doing it in the [84:27] near future. Um so is it just a tool? I [84:31] mean economics was done very differently [84:34] before kind of connectivity and machine [84:37] readable data. It was before that it was [84:39] done differently before we could run [84:41] regressions easily and had computers. Um [84:44] this is a new tool. Um, but is it [84:47] different or is it is it qualitatively [84:49] different or is it just another tool [84:51] that's going to change things a bit? [84:53] >> This is I guess a a reframing of Zav's [84:55] question as he was going to survey [84:58] people on whether we were going to be [85:00] obsolete in five years or 10 years. [85:01] >> Oh, okay. So, it's a question about [85:03] human eye. So, I think um in my view uh [85:07] human will not be obsolete um for the [85:10] following reasons, right? I mean imagine [85:12] imagine I mean there are sever several [85:13] reasons. So first of all I think about [85:15] the case where um you know um it's [85:19] completely AI and then AI would have to [85:21] train based on their own data right so [85:24] the data generated by AI and the train [85:25] on that and we know that uh training AI [85:29] on this uh generated data is going to [85:31] degrade the performance for obvious [85:33] reasons because the generated data uh [85:36] they're not going to um you know uh [85:39] perfectly reflect information on the [85:40] tail right so as a result um the trend [85:44] data will see that the trend machine [85:46] will see more and more data in the body [85:48] the main body part of the distribution [85:51] and eventually the performance will keep [85:52] degrading and uh and eventually to a [85:54] singular alert so so I think I think you [85:56] know human and AI in general will [85:58] coexist uh homy in a s system where uh [86:02] human can provide data uh at least um [86:06] now in addition um I also think the [86:08] training algorithm um at least the [86:11] current training paradigms cannot handle [86:13] situations like what I discussed for [86:15] instance the objective is ambiguous or [86:18] is completely uncharted areas. So I [86:20] think human are still there uh to [86:22] propose you know what are the good [86:24] questions um what to propo to what to [86:27] write in the prompt and things like [86:29] that. So I I I think we will coexist [86:31] >> and is it similarly important or [86:33] qualitatively different than the other [86:35] big steps we've seen in the last [86:38] decades? [86:39] >> Uh me again. Okay. [86:40] >> Yeah. Just [86:41] >> I mean I I I don't know. Oh, I think um [86:44] I feel like they're equally important [86:46] issues, right? So providing data uh come [86:48] up with good questions um you know come [86:51] up with the evaluation criterion things [86:53] like that I think very important issues [86:55] u as important as training the algorithm [86:57] you know training algorithm yeah [87:01] at this point I think it's a it's a tool [87:04] um and when I try to think about how [87:07] things will evolve in the future I tend [87:09] to believe just like Taoen that it's [87:12] humans are not going to be obsolete uh I [87:14] think there are particular skills [87:17] uh particular angles that humans will [87:19] bring uh to the process that will be [87:21] hard to replace and and one of course is [87:23] creativity originality uh coming up with [87:26] new ideas new frameworks as we heard [87:29] before uh there's also the um human [87:33] interaction I think Shaveier mentioned [87:35] teaching um at the end of the day I [87:37] think a lot of what we do does uh boil [87:39] down to to human interaction and those [87:41] things will be uh difficult to to [87:44] replace uh is is this a um substantially [87:48] different types of innovation or is it [87:50] uh just another innovation? It it's [87:52] certainly very uh significant. I think [87:55] that's hard to to deny. Um but um I [88:00] don't know that it is uh an entirely a [88:03] new class uh uh because you know when [88:05] you think about some of the things that [88:07] we experience the way that data sets [88:09] have been evolving the way that the [88:10] internet came up uh I mean all all these [88:13] things uh have changed uh the process of [88:16] research quite a lot uh and and I think [88:18] um a AI may change it more but I I [88:23] wouldn't put it completely on a class of [88:25] its own at this point. [88:27] Well, I mean, in the short run, of [88:28] course, they're just just a tool, but [88:32] in the long I mean, the long run in the [88:34] medium run, the long run is too too far. [88:36] Um, it's not clear. I mean, again, if if [88:39] uh AI can do not just Olympian math, but [88:43] really top top level research math, then [88:46] math mentions are largely obsolete [88:48] except maybe as, you know, [88:49] intermediaries to explain to the broad [88:51] public. But uh if AIs can make, you [88:55] know, write jokes funnier than any [88:57] humans, then also we're kind of useless. [88:59] Uh if they write, you know, music more [89:02] enchanting and uh um also we're kind of [89:06] useless. Uh if they write novels and [89:09] movies that are much better, movies that [89:12] are pretty poor. Um [89:15] what are humans good for? When is that [89:18] moment? Is it in? So of course we we [89:22] can't do it now but given the [89:26] you know the trend on the log axis is [89:28] going fast. So I don't know maybe in 10 [89:31] years or 20 years definitely we'll be uh [89:35] will I think we'll be obsolete and [89:37] probably 90% I know it's a bit out of [89:40] nowhere. I'm I'm good for you if you [89:42] already have tenure. I'm a bit worried [89:45] about the balance sheet the balance [89:47] sheet of universities giving 10 years to [89:50] newly minted people who are should be [89:52] there for 40 years. I don't know what [89:54] those people will be useful for but [89:56] anyway [89:58] >> including us by the way [90:02] >> I have no idea I don't think anyone does [90:04] but I will take the over on Zavier's [90:06] timing of that. So um good or bad for [90:11] experts? [90:13] >> Good, bad for what? [90:14] >> Experts. experts. [90:17] I think I mean I also kind of uh briefly [90:21] mentioned this I think it's going to [90:22] level the playing field for many right [90:25] so you know experts who were um [90:29] you know uh the if if if their expertise [90:33] was to propose uh you know new economic [90:35] theory new ideas new frame I think that [90:38] kind of expertise will still be valued [90:40] but if their experts is like writing uh [90:44] you know codes you know clean data I [90:46] think that kind expertise will [90:48] depreciate uh in the near future. [90:51] >> Yeah, going back to what I said before, [90:53] if this is a task that cannot be easily [90:56] performed by human, then it's good for [90:58] experts. So, if if you're an expert that [91:00] comes up with original ideas, uh then AI [91:03] helps you. If uh you are an expert that [91:06] uh relies on human interaction, I think [91:08] AI can help you. But but if you always [91:10] sit in the back office and just prepare [91:12] PowerPoint presentations, maybe that's [91:13] not so good for you. [91:16] I think there's a bit of empirical [91:17] evidence on that on just what happens in [91:19] law firms etc that people who already [91:22] have a tenure or partner in the law [91:24] firms they do well and the at the [91:25] entrance at the uh entrance ent [91:29] people uh the equivalent pay has been [91:33] going down. So maybe for a short term [91:36] for a short while the experts you know [91:38] do have their comparative advantage and [91:40] it's eroded more at the at the level of [91:43] entrance [91:44] um you know in the long run um if you [91:48] want to be an expert in finance I don't [91:51] know what or in any field um I do again [91:56] fundamentally there's nothing that makes [91:58] the AI not surpass humans at some point [92:04] So I think mine was a trick question. I [92:06] think it's the answer is yes. Um so I [92:09] think it's well I mean I I have no idea [92:11] about this either but [92:12] >> yes we [92:13] >> yes. [92:16] >> So uh so I I think you know for the true [92:20] outlier experts the very best in the [92:23] field um they're very valuable. AI is [92:26] very valuable to them that it you know [92:28] there are ultimately people who need to [92:30] judge the output of the model and [92:31] understand how they work but I think for [92:34] um you know people who have expertise [92:36] but not the same degree of expertise. We [92:39] need we need fewer experts but the [92:41] experts we need we need I I just think I [92:44] think in some ways it makes um the value [92:46] of being the true experts in any field [92:49] higher and being uh very inframarginal [92:52] much lower. Um but um [92:55] >> I mean then you get into alignment and [92:57] those things. So here we were all [92:58] imagining benevolent truth oriented AIs. [93:02] Of course then if they want to you know [93:04] game humans and take over whatever then [93:06] it's we need you need human experts to [93:10] somehow act as a buffer. would be some [93:13] interesting [93:15] >> principal agent [93:16] >> also again that they are mimicking so [93:18] they're not always right and um you know [93:21] it's hard to judge often if they're [93:23] right or not and fewer people can do [93:25] that than our experts in general. Um [93:29] will the impact on what we do in social [93:31] sciences be different than in other [93:34] fields of academia or science? [93:36] >> Are we somehow different? [93:41] Um [93:43] I mean I don't see much difference [93:45] between what we do and what they do. Um [93:48] except that our stuff probably more [93:51] subjective to some degree uh and which [93:54] means that we need more human judgment [93:57] um and than you know somebody who just [93:59] kind of prove things and uh do [94:01] experiments and uh discover something [94:03] that nature precisely prespecified. So I [94:07] I would I would think you know uh uh for [94:10] for us it's actually uh m our judgment [94:13] matters even more. [94:16] I mean I think we are probably right in [94:18] the middle among other disciplines in [94:20] terms of uh how subjective it is and how [94:22] how much human judgment you need. So [94:25] that would also be the extent of impact [94:28] that we will have. [94:30] uh I I would say will be uh humans will [94:34] remain more important human researchers [94:36] remain more important in the social [94:37] sciences than in uh say in math or or [94:41] physics. Uh why do I say that? Because [94:43] in math just on fundamental grounds. [94:45] Math and physics you just need the sorry [94:48] math you just need the internal logic of [94:50] things and and and you can train on [94:52] proposition you can see you can verify [94:54] them etc without any almost human [94:57] impact. uh physics is more complicated [95:00] because at some point you need to test [95:01] in the real world and there's an issue [95:03] of you know having robots who would I [95:05] know test the materials or things like [95:07] that whereas um say biology is going to [95:11] move very likely much more slowly [95:13] because you can imagine loss of the [95:16] molecules but at some point the reason [95:18] why developing a drug costs I don't know [95:20] $2 billion because you need to bring [95:23] 2,000 people and test their blood and [95:26] their pressure etc every month and you [95:28] need nice nurses to make them walk [95:30] through the hospital. So, it's very very [95:31] human intensive and so we in the social [95:34] sciences we're kind of in between [95:36] physics of mater material physics and [95:38] those very very costly um medical [95:43] experiments where you need to deal with [95:45] lots of humans. So in that sense will uh [95:48] uh inter interfacing with humans is more [95:50] important. So uh it will force AIS to uh [95:55] um concure us much more slowly. [96:00] >> So I I think we're different in that we [96:02] don't have a ground truth to train the [96:04] models to. I mean we're training the [96:06] models to sort of collective wisdom, but [96:10] um these AIs work best and are easiest [96:12] to train when there's a right and wrong [96:14] answer that you know with certainty is [96:17] right and wrong. And and uh we can't do [96:19] that kind of training. So I think it [96:20] makes it much harder. [96:22] Um I I'm happy to open it up to the [96:24] floor too. People people want to get [96:27] involved here. [96:30] No one wants to go to a speaker. [96:33] >> Okay. I'd like to ask a question vote. [96:35] So it is uh who I mean I wish I had [96:37] thought a little more about the question [96:39] made it more more actionable but uh the [96:42] the aspirations of the question is uh in [96:45] how many years will we be obsolete? So [96:48] I'd like to uh say so what fraction of [96:52] the room and I wish somebody could have [96:54] a phone to just to try to ask the AI to [96:57] quantify the fraction of the room but uh [97:00] uh who thinks it will be obsolete in uh [97:05] within the next um um 25 years. [97:14] Okay. in the next 10 years. [97:18] >> Okay, that's that's interesting. Very [97:20] good. In the next 100 years again for [97:24] for research kind of thing. [97:25] >> 100 years. [97:26] >> 100 years. Yeah. [97:29] >> It's hard to tell. [97:30] >> Okay. So, okay. So, it's not the full [97:32] distribution. Who abstains? [97:35] Maybe it's the wisest position. [97:37] >> What about the definition of [97:40] >> No, I agree. There's an issue what [97:42] obsolete means. Exactly. Here I coach it [97:44] on the labor share to be some sort of [97:46] labor share of research papers but then [97:48] issue of publication bias. So I'll think [97:51] more about the question make it [97:53] operational but here my my my reading is [97:55] that for the 25 years there was maybe a [97:58] third of the room who said we'll be [97:59] obsolete within 25 years. So [98:01] >> so let me let me ask you a follow-up [98:03] question. So you say obsolete meaning [98:05] that AI is going to write all the [98:07] research. [98:08] >> My question to you is who is going to [98:10] read that research? Oh no no no no no [98:12] it's not about it will be understanding [98:14] so it will be useful to you know predict [98:17] >> I mean you're saying there will be no [98:19] humans working as [98:20] >> no no it will be no the same way the [98:22] same way you know us informed citizen [98:25] you say oh you know what what created [98:28] the cosmos it's kind of interesting and [98:30] you hear a bit about the big bang etc [98:32] you don't need to know all the [98:33] fundamental physics equations and [98:35] inflation theory and all that but you [98:37] hear the little story so I was with this [98:39] great understanding over there and we'll [98:42] have we'll hear the little stories for [98:44] you know for children or curious [98:47] teenagers and that will be very [98:49] satisfying you know [98:51] >> so do do you know that like five years [98:53] ago uh Jeffrey Hinton has this famous [98:56] you know statement saying radiologists [98:58] are going to be obsolete in five years [98:59] >> right [99:00] >> right what happened is you know we have [99:02] more of them [99:03] >> exactly he should have thought [99:04] >> tasks are [99:05] >> because it's the complimentarity you [99:06] should have thought the complimentary [99:08] the AI photo doing taking photos and the [99:11] radiologist is uh is less than one. So [99:15] they're compliments rather than [99:16] substitutes. If he had learned a bit of [99:18] economics, he wouldn't have made that [99:20] gross mistake. Yeah. Okay. [99:24] So given Zavier's [99:28] uh pessimistic view about tenure in the [99:31] future [99:32] um and the biodal distribution that this [99:34] conference tends to attract of people [99:36] who don't care because they have tenure [99:39] and who those who still are relatively [99:43] young and need to get it. Um I guess is [99:45] there uh [99:47] >> I mean I can I can add something to this [99:49] issue. [99:50] >> Okay. So, so, so let me finish with the [99:51] question and then you can take it a [99:52] different way if you want. But I I'm I'm [99:54] just looking for something more [99:55] constructive because you know there is [99:57] this notion that [99:59] >> technology doesn't replace jobs, it [100:01] replaces functions and some jobs have [100:04] more or less functions that get [100:05] replaced. And the jobs that don't get [100:07] replaced, you end up doing something [100:09] different than what you used to do. Um [100:11] so um I guess the question is in a world [100:14] in which AI can do a lot and replace a [100:17] lot of functionality of our jobs what [100:19] aspects of our jobs are most important [100:22] to develop and concentrate on so that [100:24] you're rewarded in the future for uh [100:28] doing the job you do. So, so what what [100:30] advice would you be giving to young [100:31] researchers who don't yet have tenure on [100:34] what they should be concentrating on and [100:36] what aspects of their jobs they should [100:37] be um sort of promoting and and uh [100:41] emphasizing to be successful, [100:45] right? So, so I I I was about to kind of [100:47] mention something about like tenure and [100:49] other stuff. Um it's kind of related to [100:52] what I said in the slides about AI [100:54] anxiety, right? So given that AI has [100:56] raised um the you know the bar for for [101:00] writing papers and given that many [101:03] people's in this room have their [101:04] productivity increased by using genai [101:06] techniques. So I would expect more [101:08] papers to be written and the bar is [101:10] going to be higher and uh many people in [101:13] the room might also like me about left [101:15] behind. uh you know these are things [101:18] that keep up keep me up in the night and [101:20] and and you know so so I think AI [101:22] anxiety is is a serious issue now to [101:26] battle that I I'm not an expert to do [101:27] that but I do think you know when I [101:30] think about how to train my 9-year-old [101:32] you know I would I would still think you [101:34] know uh analytical skills mathematics [101:36] are kind kind of important things um [101:38] that are hard to be um you know uh [101:42] replaceable by by an AI and if you want [101:44] to invent new AI algorithms you would [101:46] have to understand what's going on. Uh [101:48] on the other hand, if if if the skills [101:50] are something like just uh you know [101:53] labor intensive things and it's going to [101:55] be replaced by AI clearly and you should [101:57] probably try to focus on more uh you [102:00] know to change your jobs or whatever. Um [102:02] and also like jobs related to sentiment [102:04] you know human sentiment human touch I [102:06] think going to be safe for now and [102:08] things like that. Yeah, [102:10] >> I mean in some sense I think the advice [102:12] I would give an assistant professor [102:14] today is not that different from the [102:16] advice I gave 20 years ago. I mean 20 [102:18] years ago I was still an assistant [102:20] professor myself but I think the the [102:22] advice is is fairly uh similar which is [102:25] to think about big ideas to uh try to [102:29] come up with a new research agenda uh [102:33] new model new framework you know [102:36] whatever you do. Um but but basically [102:39] you know we call it a home run. Uh think [102:41] about big ideas, big papers and try to [102:44] develop them. Um and that was always [102:47] true. Uh maybe now in the era of AI it's [102:50] even more true because those things are [102:53] more difficult to replicate with AI. [102:57] >> Yeah. Uh what I mean what I tell the [102:59] students for what it's worth is uh of [103:02] course learn lots of AI. I mean a lot of [103:04] I'm surprised they're not not that the [103:07] fraction of job market papers using [103:09] intensely AI themes is not so not not [103:12] higher. I think it it will be higher. It [103:15] should be higher for like the trilogy by [103:19] Simon who I mentioned earlier with you [103:22] know creating new data finding new [103:24] techniques to massage existing data etc. [103:27] uh and for more theory oriented people [103:31] more in general if you remember that [103:32] kakutani matrix you know try to think uh [103:36] about a few things that are simple and [103:38] hard so what's what's complicated and [103:41] will be just too too too quickly but [103:44] easy using existing well-known things uh [103:48] will be too too easily replaced by AI [103:51] quite fast uh whereas maybe there's room [103:55] for this kind of [103:57] um you know think about a few difficult [104:00] things try to trace a research agenda uh [104:03] one year try to give that as an [104:05] assignment to the PhD students uh try to [104:08] delineate your research agenda for the [104:10] next 10 years they said it was [104:11] extraordinarily [104:13] anxiety producing and they didn't like [104:16] it that much uh uh but some some some [104:20] did I think it's a very good exercise [104:22] actually uh just try to think try to [104:24] create some big picture and and every [104:26] six months try even as a grad student [104:29] try to re find what are the great [104:30] questions you would like to answer and [104:32] that's still are answerable by some [104:35] conceivable tools. uh this kind of uh [104:39] meta thinking a lot of what we reward is [104:41] you know being clever uh you can solve [104:44] math problems given problems homework [104:47] problem very fast but the more meta [104:49] cognitive level which is what will I do [104:52] with my finite res cognitive and time [104:55] resources in the next five years uh Siri [104:59] want to intervene no uh we don't develop [105:02] it enough and so maybe there's a way to [105:04] you know track to develop that in grad [105:06] school or personally [105:09] >> I agree with everything that was said. I [105:11] also think that I mean I can envision a [105:15] world where um the things that are [105:18] already important and under uh [105:20] underinvested on by a lot of our [105:22] profession um become even more important [105:24] which is just the interpersonal aspects [105:26] of it are already so important in [105:29] everything we do and um I think selling [105:32] work and um you know paying attention to [105:36] the theatrical aspect of presenting and [105:39] and doing a good [105:40] on that aspect. I'm not saying that [105:42] should be what wins in the war of ideas, [105:44] but I I can imagine a world where that [105:46] becomes more important because getting [105:49] attention and having people care about [105:50] what you're doing gets harder and and [105:53] doing a good job on that aspect becomes [105:55] more important. [105:56] >> But see, I should have said something [105:57] else. So something else which I [105:59] mentioned in the in the little talk is [106:02] just more engineering applications. You [106:04] have all those new tools. So before we [106:06] say oh it's too sad because we we there [106:09] won't be u uh there will be less room [106:12] for beautiful uh [106:14] purely academic papers but certainly [106:17] there should be lots of great uh [106:19] engineering applications and really need [106:21] that uh you know uh finance has been [106:24] very very useful via maybe portfolio [106:27] formation marcoits and fancier things [106:30] black shores but a lot of the more [106:33] recent theories I'm sorry to have not [106:35] been that uh that useful. Uh and with [106:38] this AI then you can think very product [106:41] you can be productive members of society [106:43] also become extremely wealthy and donate [106:46] to those universities that may be ruined [106:49] uh by their business [106:51] business model later. Um, so I think [106:55] that would be a very productive things [106:56] as a as a PhD student do something that [106:59] based on AI but somehow can help the [107:01] world make the world better as a [107:03] concrete engineering tool. It could be [107:06] predictions, it could be advice, it [107:08] could be somehow automatiz [107:14] um I think that would be um those would [107:17] be great contribution of our social [107:19] sciences. Otherwise, we we look a bit [107:23] strange doing our [107:27] internally internal game that doesn't [107:30] seem to interest a lot of people except [107:32] petrons who want alphas. [107:36] >> So, so there is one uh one one issue [107:38] that I was worried about when we think [107:40] about you know having the competition [107:41] for um say uh you know um predicting [107:46] returns and things like that, right? So [107:49] I mean we we we we we know that we we [107:51] don't have a lot of data and we have [107:53] seen and our algorithms have been [107:56] trained by essentially all the data up [107:57] to now right so to really truly evaluate [108:00] the performance of of a new machine we [108:04] need to wait for a long time [108:06] >> no no that's why that's why the stuff [108:09] with Ralph Moto and Rob uh no we're very [108:12] careful the type of things I mentioned [108:15] it works with one cross-section so now [108:17] it's a limited amount of prediction you [108:19] say given uh given I I tell you lots of [108:23] things about what happened in 80% of the [108:25] stocks in that quarter try to predict in [108:29] some sense what will happen what happens [108:31] will happen in the same quarter in the [108:33] remaining 20% of the stocks okay uh but [108:36] then to be able to do that so it's not [108:39] doesn't predict returns but it means you [108:41] need to understand well enough even [108:43] retrospectively what happened to those [108:45] 80% of the stocks to then can guess if [108:48] you want what what what what did happen [108:52] in the same period the same 20%. So [108:54] that's this is a world in your in your [108:56] slides where you have a pretty higher [108:58] square. Okay. So embedding the 50 order [109:01] of magnitude 50%. Uh and and there are [109:05] lots of uh lots of u [109:08] room for improvement at the same time. [109:10] Okay. And and then that will be a some [109:13] form of understanding. The same way if [109:15] you um [109:18] for a variety of you know image [109:20] recognition etc. If you recognize the [109:23] image it forces you to do some form of [109:25] understanding. Likewise if you can [109:27] understand 80% of the market try to so [109:29] knowing 80% of the market try to you can [109:32] predict what happened in the same [109:34] quarter remaining 20%. Uh it means you [109:37] have some quasha some modeling of the [109:41] forces in the economy etc. I think that [109:43] will be a measurable progress for [109:45] instance [109:45] >> right so I I another kind of tiny kind [109:48] of point that I want to add to that is [109:50] um I mean I think return prediction is [109:53] an interesting exercise no wonder I'm [109:56] trying to do it I mean partially because [109:58] uh you can evaluate the economic uh you [110:00] can evaluate your prediction economic [110:02] terms right so I think your slides you [110:04] also mentioned about um predicting [110:06] macros right predicting other stuff um I [110:09] think that that kind of kind of stopped [110:11] at you know reporting MS es uh which is [110:14] kind of hard for me to really appreciate [110:16] um the the incre the improvements in the [110:19] MSE you know so I wonder like you know [110:22] if if this could be an issue for for [110:25] certain prediction exercises [110:28] >> no I think one productive activity is [110:32] developing interesting benchmarks [110:35] >> uh for that are not two time series [110:38] dependent because any new quarter of [110:40] macro or financial data doesn't give you [110:43] uh that much. I understand that uh so [110:46] Helum Brian Kelly uh Ty Jensen and L [110:50] person they're developing some [110:52] competition and we're in some email [110:54] conversation to add our cross-sectional [110:56] benchmarks and maybe that will be that [110:58] will be uh useful. Again, it's not that [111:00] we don't care we care directly about the [111:03] outcome, but it just forces some [111:06] creating machines that de facto will [111:08] develop some understanding of a of a [111:11] situation, I think. [111:13] >> So, I asked about advice you'd have for [111:16] junior faculty. What about for journals [111:18] and PNT committees? [111:20] >> For what? [111:20] >> Journals and PNT committees. [111:23] >> PT committees. Well, just I I think that [111:25] the you know the way that we can write [111:28] papers now and [111:30] >> the tools uh are going to swamp journals [111:33] and it's going to make citations at [111:36] least the way they're currently measured [111:38] much less informative and [111:40] >> right I I think I think uh I mean in [111:42] terms of Google Scholar citations for [111:44] instance right or SSR dollars for that [111:46] for the similar uh yeah I think that [111:48] that's certainly something that we [111:50] should not think about or or truly focus [111:52] on and if We I mean I I mean in Chicago [111:56] we kind of eval I mean anywhere else [111:58] right I think we evaluate paper's impact [112:00] uh not necessarily by these measures um [112:03] these measures are easily you know uh [112:06] influenced by John AI tools as as you [112:08] correctly mentioned I totally agree with [112:09] you on that yeah [112:11] >> and and journals [112:13] >> journals uh uh I mean um so far I I'm [112:19] I'm the editor for uh for one journal u [112:22] and associated for a couple of journals. [112:25] Um I do see sometimes that we we have [112:28] articles that looks like you know [112:30] generated. Um I mean for that uh I would [112:34] always just desk reject not wasting our [112:35] resources. So I think whenever uh new [112:38] technology arise there's there are you [112:40] know bad actors expected right so if our [112:43] system have not built up a good defense [112:45] system we probably you know have to do [112:47] things uh more aggressively and you know [112:50] yeah so I I I do kind of dri a lot of [112:53] papers u yeah [112:55] so I I think I stepped down from my [112:57] editorial position just in time because [113:02] it wasn't an issue when I was uh still [113:04] an editor or at least it was just [113:06] starting to be an issue. Um but c it is [113:10] certainly a problem if people are going [113:11] to abuse the system and start generating [113:14] papers uh with uh AI and not much human [113:18] involvement and journals got to have [113:20] some system to detect that and obviously [113:24] desk rejected. So I I think it it's not [113:26] it shouldn't be acceptable that someone [113:27] does that and uh it should be desk [113:30] rejected and there should potentially [113:32] maybe even be a penalty associated with [113:34] that. I'm I'm just I don't know as much [113:36] as you probably how easy it is to to [113:38] detect uh when it's completely generated [113:41] by uh by AI but by but but assuming that [113:44] you can fairly easily detect that I [113:46] think uh journals should uh clearly [113:48] protect themselves against that. Um [113:51] regarding uh your question about uh [113:53] citations and and promotions I I mean I [113:56] I agree uh with the view that um you [113:59] know we always think about citations as [114:02] just one uh indicator. I think in [114:04] general you are looking to assess what [114:07] is the overall uh quality and impact of [114:09] of research and you know we look at uh [114:12] how many papers people write. We look at [114:14] where they publish them. We look at how [114:16] many times they're cited but we also [114:18] look at other things like how they're [114:20] evaluated by peers and uh people in the [114:23] committee themselves. They're expected [114:24] to to read and have some uh evaluation [114:28] of of the paper. Uh thinking about [114:30] potential impact if impact is still not [114:32] there. uh thinking uh whether the impact [114:35] that you see right now maybe it's just [114:37] artificial maybe it's not uh actual [114:40] meaningful impact so I think we should [114:42] continue to just treat that as one [114:44] signal I don't know that it's [114:47] necessarily becoming less important than [114:49] it was in the past but as long as it's [114:51] not taking over the whole evaluation [114:53] system I think uh it's okay [114:56] >> uh I think Robert you saying there will [114:59] be many more papers so how does a finite [115:02] uh editorial capacity can handle all [115:05] that. uh so I mean per perhaps this the [115:09] whole journal system we have should die [115:12] and it's just a bit crazy and it should [115:14] become as it is now uh all this time we [115:18] spend in revising to do you know putting [115:21] the paper from perfection 95% to 98 to [115:24] 99 to 99.99 I think is a huge waste of [115:28] time in the broad scheme of things and [115:31] uh maybe we should become more like math [115:33] and physics where you have a little bit [115:35] of a hierarchy of journal but not really [115:38] and people say actually when I talk to [115:40] some friends where I'm in math and [115:41] physics uh they say look uh use [115:44] translate a bit their language we let [115:46] the market decide so it does it doesn't [115:48] matter where the paper is published we [115:51] can see a bit later uh if it's taken off [115:54] uh well or not and in fact it's kind of [115:58] indogenous given the publication lags [116:00] are much faster it's much actually [116:02] easier to see if the idea takes on or [116:04] not uh Um so we have much more blurry uh [116:08] tools vision in economics because often [116:11] it takes five to 10 years to publish a [116:13] paper. Uh so so maybe it will it will be [116:17] sorry it's likely it's going to be [116:19] completely this abundance of new papers [116:21] will be totally unmanageable by uh [116:24] number of existing number of journals [116:27] and the system will crack and that will [116:30] be for the better and we will uh we will [116:32] have all this obsession with the top [116:34] threes and top fives and all that. [116:37] We had an audience question there. [116:39] >> Suggestion is creating a competition to [116:42] create an AI. [116:46] >> Going with your suggestion and Brian [116:49] Kelly's work suggesting a competition [116:52] where we have a journal that uses AI to [116:56] test the papers if they are created by [116:58] AI. [117:00] If not, comes through. [117:04] >> Yeah, it sounds like a very good idea. [117:05] It could be doesn't need to be a [117:06] journal. It could be a much more [117:08] automatic uh you know web web page quote [117:10] unquote service. Yeah. [117:12] >> And using your terminology of being [117:15] obsolete, if AI creates a paper and AI [117:18] accepts it, then we don't need us. I [117:21] don't think that will happen. But [117:26] >> yeah, [117:30] just a a great comment. Yeah, you asked [117:32] a question. you mentioned about experts [117:34] and where will they be so I just wanted [117:36] to share a thought on that so I think [117:38] experts certainly will become more [117:40] effective there's no question about it [117:42] and the gap between the experts and the [117:43] others will widen and the other part you [117:46] you were talking about it earlier today [117:48] of why do we looking at the blockchain [117:51] issue and the DeFi what are better [117:54] outcomes right so in a corporate setting [117:57] where they're making capital allocations [118:00] right um the the risk in decision making [118:06] is lowered when you can do red team blue [118:09] team discussions and AI is extremely [118:11] good at giving you different points of [118:13] view so that the human being can then [118:16] choose which way to go and in an [118:18] international setting you know US [118:19] businesses are known to be more [118:20] competitive um executives make decisions [118:23] with 60% information as opposed to [118:25] Germans who need 110% certainty before [118:28] they make decisions capital allocation [118:30] and business decisions [118:31] in that setting, in the international [118:33] setting, it will level the playing field [118:35] more and make other countries more [118:37] competitive than the US because it'll [118:39] give greater certainty to their [118:40] decisions and they can can make capital [118:42] allocation decisions and board level [118:44] disc decisions earlier at a lower level [118:47] of involvement at least that's so I'd [118:49] like to know if you have any thoughts on [118:50] that. [118:54] >> So maybe I'll just add one more u point [118:57] here u to to clarify one thing. Right. [118:59] So um I I don't know whether it's [119:02] relevant to your question or not but but [119:04] I think uh in in certain settings uh you [119:07] know um human experience uh matters [119:10] quite a lot and u gen at least the [119:14] current system are not trained to uh to [119:17] actually um you know uh to to actually [119:20] put so the current geni systems are [119:23] trained by you know statistical [119:24] associations right um but if you think [119:26] about uh human experience you know [119:29] sometimes you carry something more [119:31] because of your utility loss not because [119:33] of you know certain things happens more. [119:35] So I think in in a in a general decision [119:38] making scenario uh in a human experience [119:40] are quite important and the current geni [119:43] systems are insufficient to actually [119:45] handle very general settings and we [119:47] would need new training algorithms or [119:49] machines designed specifically for a [119:51] setting to deal with those things. [119:56] All right. Well, we're out of time, but [119:57] thank you very much for coming out. And [119:59] uh I at least for one will stick around [120:01] and talk to anyone who wants to talk [120:02] longer. So, [120:06] thank you guys.