Hugging Face's logo
Juho Inkinen's picture
5 9 27

Juho Inkinen

juhoinkinen
JLouisBiz's profile picture holehan's profile picture celsowm's profile picture
·

AI & ML interests

NLP

Recent Activity

posted an update about 18 hours ago
We investigated how the relevance of keyword suggestions produced by Annif has evolved in the publication repositories of https://huggingface.co/NatLibFi. The plot shows the scores averaged over months (dots) and between Annif model updates (horizontal lines): the quality of subject suggestions has essentially remained constant since the third model update. https://doi.org/10.25033/sig.162683 https://github.com/NatLibFi/Annif
liked a dataset 1 day ago
institutional/institutional-books-1.0
liked a Space 12 days ago
birder-project/birder-image-classification
View all activity

Organizations

Blog-explorers's profile picture National Library of Finland's profile picture Social Post Explorers's profile picture

Posts 4

view post
Post
122
We investigated how the relevance of keyword suggestions produced by Annif has evolved in the publication repositories of NatLibFi .

The plot shows the scores averaged over months (dots) and between Annif model updates (horizontal lines): the quality of subject suggestions has essentially remained constant since the third model update.

https://doi.org/10.25033/sig.162683
https://github.com/NatLibFi/Annif
view post
Post
2772
We ( @osma , @MonaLehtinen & me, i.e. the Annif team at the National Library of Finland) recently took part in the LLMs4Subjects challenge at the SemEval-2025 workshop. The task was to use large language models (LLMs) to generate good quality subject indexing for bibliographic records, i.e. titles and abstracts.

We are glad to report that our system performed well; it was ranked

🥇 1st in the category where the full vocabulary was used
🥈 2nd in the smaller vocabulary category
🏅 4th in the qualitative evaluations.

14 participating teams developed their own solutions for generating subject headings and the output of each system was assessed using both quantitative and qualitative evaluations. Research papers about most of the systems are going to be published around the time of the workshop in late July, and many pre-prints are already available.

We applied Annif together with several LLMs that we used to preprocess the data sets: translated the GND vocabulary terms to English, translated bibliographic records into English and German as required, and generated additional synthetic training data. After the preprocessing, we used the traditional machine learning algorithms in Annif as well as the experimental XTransformer algorithm that is based on language models. We also combined the subject suggestions generated using English and German language records in a novel way.

More information can be found in our system description preprint: Annif at SemEval-2025 Task 5: Traditional XMTC augmented by LLMs (2504.19675)

See also the task description preprint: SemEval-2025 Task 5: LLMs4Subjects -- LLM-based Automated Subject Tagging for a National Technical Library's Open-Access Catalog (2504.07199)

The Annif models trained for this task are available here: NatLibFi/Annif-LLMs4Subjects-data
View all Posts

Collections 4

Papers 1