arXiv · Machine Learning· Md Abdul Kadir (German Research Center for Artificial Intelligence, University of Oldenburg, Oldenburg, Germany), Md Mohasin Hossain (German Research Center for Artificial Intelligence, Saarland University, Saarbr\"ucken, Germany), Omair Shahzad Bhatti (German Research Center for Artificial Intelligence), Abdulrahman Mohamed Selim (German Research Center for Artificial Intelligence, University of Oldenburg, Oldenburg, Germany), Daniel Sonntag (German Research Center for Artificial Intelligence, University of Oldenburg, Oldenburg, Germany)·· 10 小时前AI 评分40
LLM 从哪里检索网络信息？识别起作用的注意力头与神经元
Where Do LLMs Retrieve Network Information? Identifying the Responsible Heads and Neurons
AI 导读
研究者在五个、涵盖三种架构家族的 LLM 中，定位从上下文检索网络信息的注意力头与神经元。每个模型仅用1–9个经因果验证的注意力头，就能构建 held-out 准确率达99.5–100%的检测器；在独立 CAIDA 数据上，头部检测器的 AUC 均≥0.971。单神经元检测器表现因模型而异，在五个模型中的三个上低于随机水平。
来源：arXiv · Machine Learning · arxiv.org