DexVT-WAM: Exploring Foundation Models for Dexterous Manipulation
A foundation-model pipeline for dexterous manipulation, from egocentric data to sim-and-real evaluation.
DexVT-WAM is an end-to-end dexterous manipulation project spanning hardware integration, egocentric data processing, foundation-model development, and evaluation in simulation and on real robots.
Project scope
01 · Dexterous platform
Built an integrated manipulation platform covering robot and hand control, perception, calibration, safety, and synchronized data collection.
02 · Ego-data pipeline
Developed a repeatable cleaning workflow for episode segmentation, temporal alignment, invalid-frame filtering, quality checks, and training-ready metadata.
03 · Model development
Developed task representations, training and evaluation loops, and adaptation interfaces for a dexterous manipulation foundation model.
04 · Sim and real evaluation
Established consistent simulation and physical-robot protocols to measure task success, generalization, robustness.
System workflow
The project connects the full development loop: platform setup and data capture, ego-data cleaning, model training and iteration, then simulation and real-world evaluation. Results from both evaluation tracks feed back into data quality controls and the next model iteration.