【文章标题】：The Emergent Symbolic Structure of Artificial Neural Networks
人工神经网络的涌现符号结构
【文章正文】：
Computer Science > Computation and Language
计算机科学 > 计算与语言
[Submitted on 30 Aug 2026]
[提交于2026年8月30日]
Title:The Emergent Symbolic Structure of Artificial Neural Networks
标题：人工神经网络的涌现符号结构
View PDF HTML (experimental)
查看PDF HTML（实验性）
Abstract:Modern systems in artificial intelligence (AI) somehow excel in domains for which they seem poorly suited. Intelligence has traditionally been modeled as operating over structured combinations of symbols, such as logical formulas. However, the strongest modern AI systems are based on neural networks, which instead represent information in continuous vectors. Vectors seem inadequate for capturing the structure of language, logic, and other cognitive domains, yet neural networks achieve impressive performance in these areas. How do they do it? In this work, we propose a potential answer: Despite appearances, perhaps the internal representations of neural networks implicitly realize symbolic structure. In support of this hypothesis, we show that the vector representations of a variety of neural networks can be closely approximated with symbolic structures: we can replace the network’s entire representation-generating process with a closed-form equation instantiating a symbolic structure, and the network’s behavior remains largely unchanged. This finding holds for both small-scale neural networks trained to manipulate lists as well as large language models (LLMs) operating in four domains that are central in symbolic traditions: arithmetic, logic, computer code, and language. Further, our symbolic approximation allows us to modify an LLM’s behavior in targeted ways via precise interventions on its internal representations, showing that the LLM’s behavior is reliant on the symbolic structures we have identified. This work provides a potential way to reconcile longstanding symbolic conceptions of intelligence with the vector-based nature of modern AI.
摘要：现代人工智能（AI）系统在某些看似不适合的领域中表现出色。传统上，智能被建模为对符号的结构化组合进行操作，例如逻辑公式。然而，最强大的现代AI系统基于神经网络，后者用连续向量表示信息。向量似乎不足以捕捉语言、逻辑和其他认知领域的结构，但神经网络在这些领域取得了令人印象深刻的性能。它们是如何做到的？在这项工作中，我们提出了一个可能的答案：尽管表面上看不出来，但神经网络的内部表示可能隐式地实现了符号结构。为了支持这一假设，我们展示了多种神经网络的向量表示可以用符号结构紧密近似：我们可以用一个实例化符号结构的闭式方程替换网络的整个表示生成过程，而网络的行为基本保持不变。这一发现既适用于训练用于操作列表的小规模神经网络，也适用于在符号传统核心的四个领域（算术、逻辑、计算机代码和语言）中运行的大型语言模型（LLM）。此外，我们的符号近似使我们能够通过对LLM内部表示的精确干预，有针对性地修改其行为，这表明LLM的行为依赖于我们已识别的符号结构。这项工作为调和长期存在的符号智能概念与现代AI基于向量的本质提供了一种可能的途径。
References & Citations
参考文献与引用
Loading…
加载中…
Bibliographic and Citation Tools
文献与引用工具
Bibliographic Explorer (What is the Explorer?)
文献浏览器（什么是浏览器？）
Connected Papers (What is Connected Papers?)
关联论文（什么是关联论文？）
Litmaps (What is Litmaps?)
文献图谱（什么是文献图谱？）
scite Smart Citations (What are Smart Citations?)
scite智能引用（什么是智能引用？）
Code, Data and Media Associated with this Article
与本文相关的代码、数据和媒体
alphaXiv (What is alphaXiv?)
alphaXiv（什么是alphaXiv？）
CatalyzeX Code Finder for Papers (What is CatalyzeX?)
CatalyzeX论文代码查找器（什么是CatalyzeX？）
DagsHub (What is DagsHub?)
DagsHub（什么是DagsHub？）
Gotit.pub (What is GotitPub?)
Gotit.pub（什么是GotitPub？）
Hugging Face (What is Huggingface?)
Hugging Face（什么是Huggingface？）
ScienceCast (What is ScienceCast?)
ScienceCast（什么是ScienceCast？）
Demos
演示
Recommenders and Search Tools
推荐与搜索工具
Influence Flower (What are Influence Flowers?)
影响力花（什么是影响力花？）
CORE Recommender (What is CORE?)
CORE推荐器（什么是CORE？）
arXivLabs: experimental projects with community collaborators
arXivLabs：与社区合作者的实验项目
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
arXivLabs是一个框架，允许合作者直接在我们的网站上开发和分享新的arXiv功能。
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
与arXivLabs合作的个人和组织都接受并认同我们的开放、社区、卓越和用户数据隐私的价值观。arXiv致力于这些价值观，并只与遵守这些价值观的合作伙伴合作。
Have an idea for a project that will add value for arXiv’s community? Learn more about arXivLabs.
有一个能为arXiv社区增添价值的项目想法？了解更多关于arXivLabs的信息。