AIOZ AI Research: Lightweight Temporal Transformer Decomposition for Federated Autonomous Driving

AIOZ AI Research: Lightweight Temporal Transformer Decomposition for Federated Autonomous Driving

We are delighted to announce that our AI research paper "Lightweight Temporal Transformer Decomposition for Federated Autonomous Driving" has been accepted and will be published in November 2025 as the top-ranking submission at the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) 2025.

This research introduces Lightweight Temporal Transformer Decomposition, a novel method that enables efficient temporal modeling in federated autonomous driving without compromising privacy or edge performance.

Let's dive into the key aspects of this groundbreaking research below:

The Challenges in Autonomous Driving and the Need for Federated Learning

Autonomous driving requires reliable navigation through ever-changing environments, necessitating systems that can anticipate hazards and adapt quickly.

Traditional vision models, which rely on single-frame inputs, struggle significantly with motion prediction. Centralized data collection also complicates matters by raising privacy concerns due to the sensitivity of vehicle data.

Federated learning (FL) offers a viable solution by facilitating decentralized training across vehicles while keeping data local. However, multiple challenges persist, such as:

=> Processing temporal sequences

=> Managing computational loads on edge devices

=> Ensuring convergence across heterogeneous data

Our proposed solution, Lightweight Temporal Transformer Decomposition (LTTD), addresses these challenges by integrating temporal data—image frame sequences and steering signals—into a federated learning framework.

Key Innovations: Lightweight Temporal Transformer Decomposition

LTTD is a novel approach tailored for federated autonomous driving, which leverages temporal data and a lightweight design.

  • Federated Learning Setup: Enables decentralized model training across multiple vehicles while ensuring data remains local to preserve privacy.
  • Temporal Data Advantage: Leverages past frames and steering sequences to improve robustness in dynamic scenarios.
  • Lightweight Design: Decomposes large attention maps into smaller, computationally efficient matrices.
  • Real-Time Feasibility: Optimized for use on resource-limited edge devices.

This approach achieves state-of-the-art performance by striking a balance between privacy, efficiency, and accuracy. Its recognition as an oral presentation at the prestigious IROS 2025 conference underscores its significance.

Methodology: How LTTD Works

Building a robust autonomous driving model in a federated learning environment requires efficient handling of temporal data without overwhelming limited edge resources.

Our method achieves this through a combination of innovative techniques tailored for federated autonomous driving.

Key Methodological Innovations:

  • Federated Network Setup: A network of autonomous vehicles collaboratively trains a global driving policy by aggregating local model weights from each vehicle silo. Each vehicle minimizes a regression loss using local data, which ensures privacy.
  • Unitary Attention Decoupling: Applies a unitary attention mechanism to represent input triplets (e.g., image channels from multiple frames) as joint representations. This reduces attention tensor size by minimizing redundant linear combinations.
  • Tensor Factorization: Breaks down large attention maps into smaller, low-rank matrices using tensor factorization, which reduces computational complexity while preserving critical temporal information.
  • Efficient Training: The lightweight model design enables training on resource-constrained devices, ensuring convergence in federated settings and supporting real-time predictions.

This approach allows the integration of temporal data (past frames and steering sequences) into federated learning, improving prediction accuracy for dynamic driving conditions.

The lightweight architecture ensures feasibility for edge devices, making it practical for real-world autonomous vehicles.

Experimental Validation and Proven Performance

Validating the effectiveness of our Lightweight Temporal Transformer Decomposition method is critical for its adoption in real-world autonomous driving systems.

We conducted extensive experiments to demonstrate its superiority in federated learning environments.

Experimental Highlights:

  • Dataset Evaluation: Tested across three diverse datasets, LTTD outperforms other recent federated autonomous driving methods in both steering prediction and hazard anticipation.
  • Real-Time Performance: The lightweight design ensures efficient computation, enabling real-time predictions on resource-constrained edge devices.
  • Robustness to Temporal Data: Incorporating sequences of image frames and steering inputs (e.g., 5-30 frames) enables our model to effectively capture motion patterns and dynamic scenarios. Performance slightly degrades with excessive frame inputs due to approximation limitations.
  • Real Robot Experiments: Visualizations (Fig. 1) from real-world robot deployments confirm the model's smooth integration into edge systems while maintaining high performance.

Our approach sets a new benchmark for federated autonomous driving, surpassing state-of-the-art methods in both accuracy and efficiency.

Future improvements may involve addressing limitations such as the reduced expressiveness of rank-1 tensor approximations through adaptive decomposition parameters or regularization techniques.

Fig. 1

Why It Matters: Broader Impact of AIOZ AI Research

The acceptance of Lightweight Temporal Transformer Decomposition at IROS 2025 highlights its technical merit and aligns with the broader goals of the AIOZ DePIN ecosystem, which includes over 300,000 devices.

By introducing temporal data into federated learning, LTTD significantly enhances performance for real-world autonomous driving tasks, while its lightweight architecture ensures compatibility with edge deployment.

Although challenges like non-IID data scaling persist, our established track record in AI research suggests continued advancements.

Conclusion

AIOZ Network’s latest AI research paper, "Lightweight Temporal Transformer Decomposition for Federated Autonomous Driving", marks a pivotal leap forward in AI for robotics.

By merging transformer efficiency with the privacy advantages of federated learning, LTTD brings us closer to safe, scalable, real-world autonomous driving.

Explore the full project through the GitHub repository and project page.

As IROS 2025 approaches, AIOZ DePIN solutions and collaborations signal a decentralized AI evolution.

Paper Link: https://arxiv.org/abs/2506.23523

Github Page: https://aioz-ai.github.io/IROS2025_LTFed_github-page/

Source Code: https://github.com/aioz-ai/IROS2025_LTFed/tree/master?tab=readme-ov-file

About the AIOZ Network

AIOZ Network is a DePIN for Web3 AI, Storage, and Streaming.

Powered by a global community of AIOZ DePINs, AIOZ rewards you for sharing your computational resources for storing, transcoding, and streaming digital media content and powering decentralized AI computation.

Find Us

AIOZ All Links | Website | X | Telegram