KG-EDAS: Evaluating Knowledge Graph Completion Models

Abstract

Current evaluation practices for knowledge graph completion models lack standardization and often fail to capture model capabilities comprehensively. We introduce KG-EDAS (Knowledge Graph Evaluation through Data-Aware Scoring), a meta-metric framework that provides more robust and interpretable evaluation of KGC models. Our framework addresses key limitations in existing evaluation protocols.

Methodology

KG-EDAS combines multiple evaluation dimensions including semantic consistency, structural awareness, and robustness to data distribution shifts. We conduct extensive experiments across different KGC architectures and datasets to validate the framework’s effectiveness. The meta-metric approach provides both aggregate scores and detailed breakdowns of model performance. Current evaluation practices for knowledge graph completion models lack standardization and often fail to capture model capabilities comprehensively. We introduce KG-EDAS (Knowledge Graph Evaluation through Data-Aware Scoring), a meta-metric framework that provides more robust and interpretable evaluation of KGC models. Our framework addresses key limitations in existing evaluation protocols.