计算机科学
语法性
自然语言处理
语法
人工智能
独立性(概率论)
一般化
信息结构
背景(考古学)
代表(政治)
自然语言
条件独立性
任务(项目管理)
语言学
语法
数学
统计
管理
政治
政治学
法学
经济
生物
数学分析
古生物学
哲学
作者
Raphaël Bailly,Kata Gábor,Laurent Leblond
标识
DOI:10.18653/v1/2023.acl-long.590
摘要
This paper presents an information-theoretical model of syntactic generalization.We study syntactic generalization from the perspective of the capacity to disentangle semantic and structural information, emulating the human capacity to assign a grammaticality judgment to semantically nonsensical sentences.In order to isolate the structure, we propose to represent the probability distribution behind a corpus as the product of the probability of a semantic context and the probability of a structure, the latter being independent of the former.We further elaborate the notion of abstraction as a relaxation of the property of independence.It is based on the measure of structural and contextual information for a given representation.We test abstraction as an optimization objective on the task of inducing syntactic categories from natural language data and show that it significantly outperforms alternative methods.Furthermore, we find that when syntax-unaware optimization objectives succeed in the task, their success is mainly due to an implicit disentanglement process rather than to the model structure.On the other hand, syntactic categories can be deduced in a principled way from the independence between structure and context.
科研通智能强力驱动
Strongly Powered by AbleSci AI