Latency-Minimal Decentralized LAM Training with Looped Transformers in Heterogeneous LEO Satellite Constellations

The traditional approach of transmitting sensing data to ground stations for training large AI models (LAMs) is becoming increasingly impractical due to unstable ground-to-satellite links (GSLs) and growing privacy concerns. Fortunately, the rapid expansion of Low Earth Orbit (LEO) satellite constellations offers a new solution. The increasing number of LEO satellites constitutes a distributed computing network, enabling on-orbit computation without the need for data transmission to ground stations. However, the limited computational resources on satellites result in substantial latency when training edge LAMs directly on a single satellite, and the memory capacity is insufficient to accommodate large models for training. To address these challenges, we propose a latency-optimized decentralized training framework that enables collaborative in-orbit LAM pretraining across heterogeneous LEO satellites without the need for downloading data to the ground. In the proposed architecture, we adopt a looped Transformer architecture that reduces memory usage through cross-layer parameter sharing. To improve training efficiency, we introduce a hybrid parallel training strategy that combines intra-group pipeline parallelism and inter-group data parallelism through satellite grouping. Furthermore, to accommodate heterogeneous satellite capabilities, we formulate an optimization problem for workload balancing and develop a dynamic programming-based strategy to minimize overall system latency. Extensive experiments on a heterogeneous satellite testbed demonstrate that our method significantly outperforms existing baselines in both training speed and memory efficiency.