Neurons & Peppers #3

Neurons & Peppers is THE state of the art AI research meetup in Paris!

For this 3rd edition, we will dive into domain-specific modeling and RAG benchmarking !

The LLM Pro Finance Suite: Multilingual Large Language Models for Financial Applications
by Raheel Qader, Head of R&D and Gaëtan Caillaut, AI Researcher – DragonLLM.

Abstract: The financial industry’s growing demand for advanced natural language processing (NLP) capabilities has highlighted the limitations of generalist large language models (LLMs) in handling domain-specific financial tasks. To address this gap, we introduce the LLM Pro Finance Suite, a collection of five instruction-tuned LLMs (ranging from 8B to 70B parameters) specifically designed for financial applications. Our approach focuses on enhancing generalist instruction-tuned models, leveraging their existing strengths in instruction following, reasoning, and toxicity control, while fine-tuning them on a curated, high-quality financial corpus comprising over 50% finance-related data in English, French, and German […]

Check out the full paper on Arxiv
Check out the models on their HuggingFace collections:
– LLM Open Finance  (open-source)
– LLM Pro Finance

---

​ViDoRe V3: A Comprehensive Evaluation of Retrieval Augmented Generation in Complex Real-World Scenarios ​
by Quentin Macé et Antoine Edy, data scientists – ILLUIN Technology.

​Abstract: Retrieval-Augmented Generation (RAG) pipelines must address challenges beyond simple single-document retrieval, such as interpreting visual elements (tables, charts, images), synthesizing information across documents, and providing accurate source grounding. Existing benchmarks fail to capture this complexity, often focusing on textual data, single-document comprehension, or evaluating retrieval and generation in isolation. We introduce ViDoRe v3, a comprehensive multimodal RAG benchmark featuring multi-type queries over visually rich document corpora. It covers 10 datasets across diverse professional domains, comprising ~26,000 document pages paired with 3,099 human-verified queries, each available in 6 languages […]

​Check out the full paper on Arxiv
Test the benchmark yourself on their HuggingFace Space.

---