Benchmarking Natural Language Understanding: A Psychological and Philosophical Perspective?

2022/02/09 (Wed) 12:00 (JST)

菅原朔 (国立情報学研究所)

[Webサイト]

国立情報学研究所助教。2020年東京大学大学院情報理工学系研究科博士課程修了。2020年より現職。

概要

Machine reading comprehension (MRC) has received considerable attention as a benchmark for natural language understanding. However, the conventional task design of MRC lacks explainability beyond the model interpretation, i.e., reading comprehension by a model cannot be explained in human terms. To this end, this talk provides a theoretical basis for the design of MRC datasets based on psychology as well as psychometrics, and summarizes it in terms of the prerequisites for benchmarking MRC. The talk may also include our recent (a little bit philosophical) discussion on language understanding and its evaluation.

※トークは日本語です。

[動画] [スライド] [論文1] (EACL 2021) [論文2] (ACL 2021)

メーリングリストへの登録: 参加用URLなどNLPコロキウムに関するお知らせを受け取りたい方はメーリングリストへのご登録をお願いします。

メーリングリスト登録フォーム

[トップページへ戻る]