QLoRA Fine-Tuning Enhances Multimodal Opinion Extraction for STI
A recent paper submitted to arXiv (2608.14152) introduces a framework for multimodal core-opinion extraction tailored for Science and Technology Intelligence (STI). This research tackles the difficulties of extracting opinions from extensive information streams, highlighting the limitations of existing models in filtering noise and ensuring reliable structured outputs in zero-shot multilingual and multimodal contexts. The proposed framework leverages visual evidence to enhance textual assessments, utilizing the VideoLLaMA2 (VL2) and VideoLLaMA2.1 (VL2.1) base models. Fine-tuning was conducted using Quantized Low-Rank Adaptation (QLoRA) on a dataset comprising 2,194 multilingual and multimodal samples. In the Image-Augmented setting, the fine-tuned VL2.1 model produced structured JSON outputs, achieving 64.98% Precision, 42.15% Recall, 51.14% F1-score, and 74.00% accuracy. This research seeks to alleviate information overload and improve extraction focus in STI, providing a more effective method for handling multimodal data.
Key facts
- The paper is available on arXiv with identifier 2608.14152v1.
- The study proposes a multimodal core-opinion extraction framework for Science and Technology Intelligence (STI).
- The framework uses visual evidence as a contextual anchor for textual judgment.
- VideoLLaMA2 (VL2) and VideoLLaMA2.1 (VL2.1) are used as base models.
- Quantized Low-Rank Adaptation (QLoRA) fine-tuning is applied on a curated dataset of 2,194 multilingual and multimodal samples.
- The fine-tuned VL2.1 model achieves 64.98% Precision, 42.15% Recall, 51.14% F1-score, and 74.00% (likely accuracy) in the Image-Augmented setting.
- The research addresses information overload and extraction defocus in STI.
- The model generates structured JSON core-opinion outputs.
Entities
Institutions
- arXiv