Multimodal 3D/4D world modeling for autonomous driving, with broader relevance to embodied systems
B.Eng. candidate in Automation at Huazhong University of Science and Technology
I study how camera video and LiDAR observations can be organized into persistent 3D/4D scene representations for autonomous driving, and how those representations can support future scene prediction. My broader interests include embodied systems and the use of large visual and multimodal models as representation and reasoning tools.
My recent work spans multiview sensor geometry, multimodal 3D perception, causal temporal LiDAR modeling, tiny-object video perception, and autonomous-driving software.
In a sequence-disjoint fitting-stage comparison across eight designated STU sequences, a causal ten-frame LiDAR model improved macro AP by 3.41 percentage points over a matched current-frame model. This work motivates my current questions around persistent temporal state, cross-frame association, and future scene prediction from multimodal streams.
A strictly causal stream from a real STU sequence. Data: STU, Nekrasov et al., CVPR 2025.
Jinze Gao, Jie Ma · manuscript
I studied how stress cases mined from one 3D detector transfer to others using BEVFusion, SparseFusion, and DeepInteraction on nuScenes. The study covers 2,007 keyframes under 12 signed single-axis calibration perturbations and a log-disjoint evaluation on 725 keyframes from nine sequences. Within the evaluated two-source pools, candidate-pool limitation accounted for 87.6–96.3% of mean total regret, separating the target-side utility of source-mined hard cases from the quality of the selector applied afterward.
I investigated inter-camera calibration for a multi-fisheye vehicle rig using Dongfeng real-vehicle recordings. In a separate controlled four-camera synthetic benchmark with 3% corrupted correspondences, explicit fisheye projection, observability checks, and robust joint SE(3) refinement reduced median reprojection error from 7.469 px to 0.411 px.
CalibDecisionAudit extends this work to retrospective camera–LiDAR calibration-decision evaluation. It combines leakage checks, scene-grouped out-of-fold validation, scene-level uncertainty, and reusable SE(3), visibility, and oriented 3D matching utilities.
TinyDroneLab is an RGB tiny-drone detection and tracking pipeline with resolution-aware routing, overlapping tiled inference, full-frame projection, confidence-aware association, and track-conditioned local recovery.
At frame 3,890 of a low-resolution sequence, the adaptive route localizes the annotated drone while three earlier routes return no target. Selected source-study panels are reproduced with permission.
The public repository isolates the executable perception core and includes a deterministic implementation regression with frame-level outputs for recomputation.
-
CXRShift studies cross-source chest X-ray classification across three backbones and three seeds. Adding labeled RSNA training and validation data improved DenseNet121 over the Kermany-only ERM baseline by 12.22 percentage points on a fixed exploratory 442-image RSNA test partition; this is known-source adaptation.
-
At PhiGent Robotics, I validated production C++ modules for hybrid A* parking planning and control. Boundary, exception, and parameterized Parasoft C/C++test cases expanded executable code under test from approximately 5,000 to 25,000 lines and reached near-100% line coverage.