Loading...

← Back to Projects
AI/MLComputer VisionResearch⭐ Featured

CPF-Net: Continuous Perturbation Fusion Network for Weather-Robust LiDAR Segmentation

A continuous perturbation fusion network for weather-robust LiDAR segmentation, achieving SOTA performance on the SemanticKITTI -> SemanticSTF dataset.

📍 Challenge

LiDAR-based semantic segmentation is crucial for autonomous driving. However, LiDAR segmentation models trained on clear weather fail dramatically under fog, rain, and snow: performance drops by 30% on average. Existing methods treat weather as discrete cases, failing to capture the continuous nature of weather distortion.

🔧 Methods

I developed CPF-Net, which treats weather as a continuous perturbation process and uses stable geometric structure to guide learning under corruption. The key insight: geometric information remains stable while appearance (reflectance) degrades under weather shifts. Three Core Components of Continuous Perturbation Fields Network (CPF-Net): 1. Continuous Perturbation Fields (CPF) Simulate spatially-varying weather distortions during training, modeling weather as a continuous process rather than discrete cases. 2. Cross-Attention Correction (CA) At each encoder layer, use stable geometric cues to correct corrupted reflectance features through cross-attention. 3. Adaptive Fusion (AF) Dynamically adjust the contribution of geometry and reflectance based on estimated local distortion strength.

📸 Figures

CPF-Net
Figure 1. Overall Pipeline of CPF-Net. (a) CPF generates spatially-varying weather perturbations during training. (b) CA uses geometric features to correct corrupted reflectance features. (c) AF dynamically adjusts the contribution of geometry and reflectance based on estimated local distortion strength.

📊 Results

Achieved state-of-the-art performance on SemanticKITTI → SemanticSTF benchmark:
  • 43.6% mIoU on SemanticSTF benchmark (+1.1% over prior state-of-the-art)
  • Consistent improvements across all weather conditions (fog, rain, snow)
  • Demonstrated that continuous modeling + geometric guidance enables robust representations under domain shift
CPF-Net Results
Figure 2. Quantitative results on SemanticSTF benchmark. (a) Overall performance comparison with state-of-the-art methods. (b) Performance under different weather conditions.

🛠 Tech Stack

PyTorchCUDAPointTransformer v3FlashAttention

Other Projects

MOSAIC: Exploiting Compositional Blindness in Multimodal Alignment

A novel multimodal jailbreak framework that targets compositional blindness in Vision-Language Models. MOSAIC rewrites harmful requests into Action–Object–State triplets, renders them as stylized visual proxies, and induces state-transition reasoning to bypass safety guardrails. Published at NTU ML 2025 Fall Mini-Conference (Oral).

PEFT-STVG: Parameter-Efficient Fine-Tuning for Spatio-Temporal Video Grounding

Spatio-Temporal Video Grounding (STVG) localizes objects in video frames that match natural language queries across time. While effective, existing methods require full model fine-tuning, creating significant computational bottlenecks that limit scalability and accessibility.