- “YallaMorph is a large-scale benchmark for evaluating the ability of Large Language Models (LLMs) to perform controlled Arabic morphological generation. Given an Arabic lemma, part-of-speech (POS), gloss, and a set of target morphosyntactic features, the task is to generate the corresponding fully diacritized Arabic surface form or forms. The benchmark also includes morphologically invalid feature configurations for which no valid form exists….The benchmark contains 663,804 evaluation instances across 4,795 sampled lemmas.”