Tensorized Clustered LoRA Merging for Multi-Task Interference
Aug 15, 2025·,,,
,,·
1 min read
Ziyu Su
Fan Mo
Guancheng Liang
Jing Zhang
Bingbing Wen
Pratiksha Tiwari
Jian-Yun Nie
Abstract
We propose a tensorized clustered LoRA merging method to mitigate multi-task interference when adapting large language models to many tasks using low-rank adapters.
Type
Publication
arXiv preprint
We introduce tensorized clustered LoRA merging to address multi-task interference when large language models are adapted to many tasks via low-rank adapters, improving performance and parameter efficiency.
