Tensorized Clustered LoRA Merging for Multi-Task Interference

Aug 15, 2025·
Zhan Su
,
Fengran Mo
,
Guojun Liang
,
Jinghan Zhang
Bingbing Wen
Bingbing Wen
,
Prayag Tiwari
,
Jian-Yun Nie
· 1 min read
Abstract
We propose a tensorized clustered LoRA merging method to mitigate multi-task interference when adapting large language models to many tasks using low-rank adapters.
Type
Publication
arXiv preprint

We introduce tensorized clustered LoRA merging to address multi-task interference when large language models are adapted to many tasks via low-rank adapters, improving performance and parameter efficiency.

LoRA Multi-Task Learning Parameter-Efficient Tuning
Bingbing Wen
Authors
Bingbing Wen