Daily Papers

Arabic poetry is one of the richest and most culturally rooted forms of expression in the Arabic language, known for its layered meanings, stylistic diversity, and deep historical continuity. Although large language models (LLMs) have demonstrated strong performance across languages and tasks, their ability to understand Arabic poetry remains largely unexplored. In this work, we introduce Fann or Flop, the first benchmark designed to assess the comprehension of Arabic poetry by LLMs in 12 historical eras, covering 14 core poetic genres and a variety of metrical forms, from classical structures to contemporary free verse. The benchmark comprises a curated corpus of poems with explanations that assess semantic understanding, metaphor interpretation, prosodic awareness, and cultural context. We argue that poetic comprehension offers a strong indicator for testing how good the LLM understands classical Arabic through Arabic poetry. Unlike surface-level tasks, this domain demands deeper interpretive reasoning and cultural sensitivity. Our evaluation of state-of-the-art LLMs shows that most models struggle with poetic understanding despite strong results on standard Arabic benchmarks. We release "Fann or Flop" along with the evaluation suite as an open-source resource to enable rigorous evaluation and advancement for Arabic language models. Code is available at: https://github.com/mbzuai-oryx/FannOrFlop.

The rapid proliferation of information in the digital era underscores the importance of accurate historical representation and interpretation. While artificial intelligence has shown promise in various fields, its potential for historical fact-checking and gap-filling remains largely untapped. This study evaluates the performance of three large language models LLMs GPT 3.5, GPT 4, and GoogleBARD in the context of predicting and verifying historical events based on given data. A novel metric, Distance to Reality (DTR), is introduced to assess the models' outputs against established historical facts. The results reveal a substantial potential for AI in historical studies, with GPT 4 demonstrating superior performance. This paper underscores the need for further research into AI's role in enriching our understanding of the past and bridging historical knowledge gaps.

Historical weather records from Europe indicate that the Earth experienced substantial climate variability, which caused, for instance, the Little Ice Age and the global crisis in the period between the 14th and 19th centuries. However, it is still unclear how global this climate variability was because of the scarce meteorological data availability in other regions including East Asia, especially around the 17th century. In this context, Seungjeongwon Ilgi, a daily record of the Royal Secretariat of the Joseon Dynasty of Korea, is a precious source of historical meteorological records for the Korean Peninsula, as it covers 288 years of weather observations made during 1623-1910. We used the digital database of Seungjeongwon Ilgi to construct a machine-readable weather condition dataset. To this end, we extracted valid weather information from the original weather description text and compiled them into predefined weather categories. Additionally, we attempted to improve the usability of the dataset by converting the reported dates in the traditional calendar system to those in the Gregorian calendar. Finally, we outlined the promising implications of this dataset for meteorological and climatological studies, while describing the limitations of the dataset. Overall, future studies focusing on the climate and weather of the past could use this meteorological database for investigating long-term climate variability. Our datasets are publicly available at 10.5281/zenodo.8142701.

After decades of massive digitisation, an unprecedented amount of historical documents is available in digital format, along with their machine-readable texts. While this represents a major step forward with respect to preservation and accessibility, it also opens up new opportunities in terms of content mining and the next fundamental challenge is to develop appropriate technologies to efficiently search, retrieve and explore information from this 'big data of the past'. Among semantic indexing opportunities, the recognition and classification of named entities are in great demand among humanities scholars. Yet, named entity recognition (NER) systems are heavily challenged with diverse, historical and noisy inputs. In this survey, we present the array of challenges posed by historical documents to NER, inventory existing resources, describe the main approaches deployed so far, and identify key priorities for future developments.