FlexRank: Nested Low-Rank Knowledge Decomposition for Adaptive Model Deployment
The growing scale of deep neural networks, encompassing large language models (LLMs) and vision transformers (ViTs), has made training from scratch prohibitively expensive and deployment increasingly costly. These models are often used as computational monoliths with fixed cost, hindering adaptive deployment across different cost budgets. We argue that nested components, ordered by importance, can be extracted from pretrained models and selectively activated within the available computational budget. To this end, our proposed FlexRank method leverages low-rank weight decomposition with nested, importance-based consolidation to extract submodels of increasing capabilities. Our approach enables a “train-once, deploy-everywhere” paradigm offering a graceful trade-off between cost and performance without training from scratch for each budget - advancing practical deployment of large models.
Authors: Riccardo Zaccone, Stefanos Laskaridis, Marco Ciccone, Samuel Horvàth
Published at: International Conference on Machine Learning (ICML’26), Spotlight (top 5.3%)
Overview
Large pretrained models are usually deployed as computational monoliths: one fixed cost, regardless of what the serving budget actually allows. FlexRank challenges this by showing that a single trained model can be decomposed into nested submodels of increasing capability, extracted purely through low-rank weight decomposition with importance-based consolidation.
This gives a “train-once, deploy-everywhere” recipe: practitioners can select a submodel that fits their compute budget at deployment time, without retraining from scratch for every target size. The result is a graceful, continuous cost-performance trade-off, making it easier to serve the same underlying model across heterogeneous deployment constraints.
Links
Reference
@inproceedings{
zaccone2026flexrank,
title={FlexRank: Nested Low-Rank Knowledge Decomposition for Adaptive Model Deployment},
author={Zaccone, Riccardo and Laskaridis, Stefanos and Ciccone, Marco and Horv{\`a}th, Samuel},
booktitle={International Conference on Machine Learning},
year={2026}
}
