模棱两可
透视图(图形)
计算机科学
分辨率(逻辑)
自然语言处理
歧义消解
人工智能
程序设计语言
电信
全球导航卫星系统应用
全球定位系统
作者
Seung-Hyeon Yang,F.R. Chen,Yiming Yang,Z. G. Zhu
标识
DOI:10.1145/3632971.3632973
摘要
Generative large language models (LLMs) have demonstrated outstanding performance in language understanding and generation tasks. Whether current LLMs can truly understand meaning or not is an open question. Semantic disambiguation is an important indicator to test machine language ability. To this end, three experiments were conducted to test the ambiguity resolution ability. In Experiment 1, the LLMs were asked to make a semantic discrimination on ambiguity phrases. The ChatGPT (May 3 Version) and ChatGPT Plus (May 3 Version) showed a random level performance of discrimination. To make the task easier for the LLMs, ambiguity sentences were presented with context in the Experiment 2. While the LLMs unified the context to fill the position for ambiguity word, the performance of semantic discrimination on these sentences remained at random level. In addition to test the performance of LMMs, prediction is one of the key ability of LLMs. The prediction by unifying the context is one of the key skill for LLMs. Therefore, the Experiment 3 further test the similarity between the human prediction mechanism and the strong generation capabilities of LLMs. Weak similarities were only found under the condition of local constraint features gradually accumulating, specifically, the cosine similarity of predicted sentence semantic vectors and ending word semantic vectors negatively correlated with close probability only. These series of experiments demonstrate that, it is far away to claim that LLMs truly understand human sentences.
科研通智能强力驱动
Strongly Powered by AbleSci AI