AMR Similarity Metrics from Principles

Different metrics have been proposed to compare Abstract Meaning Representation (AMR) graphs. The canonical S metric (Cai and Knight, ) aligns the variables of two graphs and assesses triple matches. The recent S B metric (Song and Gildea, ) is based on the machine-translation metric B (Papineni et...

Full description

Saved in:
Bibliographic Details
Published in:Transactions of the Association for Computational Linguistics 2020-01, Vol.8, p.522-538
Main Authors: Opitz, Juri, Parcalabescu, Letitia, Frank, Anette
Format: Article
Language:English
Subjects:
Ablation
Asymmetry
Computational linguistics
Criteria
Graphical representations
Graphs
Linguistics
Machine translation
Meaning
Principles
Semantics
Symmetry
Variables
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Different metrics have been proposed to compare Abstract Meaning Representation (AMR) graphs. The canonical S metric (Cai and Knight, ) aligns the variables of two graphs and assesses triple matches. The recent S B metric (Song and Gildea, ) is based on the machine-translation metric B (Papineni et al., ) and increases computational efficiency by ablating the variable-alignment. In this paper, i) we establish criteria that enable researchers to perform a comparing meaning representations like AMR; ii) we undertake a of S and S B where we show that the latter exhibits some undesirable properties. For example, it does not conform to the rule and introduces biases that are hard to control; and iii) we propose a that is more benevolent to only very slight meaning deviations and targets the fulfilment of all established criteria. We assess its suitability and show its advantages over S and S B .
ISSN:2307-387X
2307-387X
DOI:10.1162/tacl_a_00329