AutoPhyX: Automatic Text-Condition Physics Property Generation

Why text matters

Resolving visual ambiguity with language.

Objects with nearly identical appearance can have very different physical properties: fresh fruit and a rigid plastic replica may share the same shape and texture while responding differently to force. Methods that rely only on visual cues cannot reliably distinguish these material states.

AutoPhyX introduces text as a semantic constraint and fuses it with spatial 3D features. In one forward pass, the model predicts a dense voxel field of Young's modulus, Poisson's ratio, and density that can be transferred to meshes, point clouds, Gaussian Splatting, and NeRF for MPM or FEM simulation.

The same bonsai and box exhibit different physical behavior under different text descriptions. — Text descriptions disambiguate physically different states under identical visual appearance and boundary conditions.

Text-controlled dynamics

Demo

One object. Different physics. Select a scene to compare simulations under the same setup with different text-conditioned properties.

Condition A

Stiff stems

The potted plant stays upright and shows only minor shifts in the wind.

Condition B

Flexible stems

The plant undergoes pronounced flexible bending and sways deeply to the side.

Method

Language modulates a physics-ready 3D field.

01

Extract

Render multi-view OpenCLIP features into a spatially coherent 3D voxel field.

02

Modulate

Apply text-conditioned FiLM transformations across multiple 3D U-Net scales.

03

Predict

Regress dense fields for Young's modulus, Poisson's ratio, and density.

04

Simulate

Transfer the voxel properties to GS, mesh, or point representations for MPM and FEM.

Volumetric grounding

From visible surfaces to a complete 3D field.

AutoPhyX keeps semantic structure intact while lifting multi-view features into a physics-ready volume. A second pass fills the interior that cameras cannot observe.

A bonsai object compared with direct VoMP feature extraction and AutoPhyX visibility-aware extraction. — 01

Visibility-aware extraction

AutoPhyX extracts dense 2D OpenCLIP features from multi-view renders, then lifts them into a continuous 3D feature field through differentiable NeRF-style volume rendering.

Unlike VoMP-style direct averaging, the accumulated transmittance along each camera ray suppresses samples behind visible surfaces. This prevents foreground leaves from contaminating an occluded trunk and preserves spatial and semantic consistency.

Two ray-casting conditions used to classify and fill interior voxels inside a shape. — 02

Interior completion

Surface features alone leave the object hollow, while simulation requires Young's modulus, Poisson's ratio, and density throughout the interior. AutoPhyX begins with six-direction boundary checks to find coarse enclosed candidates.

It then applies the even-odd rule: rays cast to infinity must intersect the surface an odd number of times for a voxel to be considered inside. Accepted voxels receive semantic features through nearest-neighbor interpolation from the surface, producing a solid, physics-ready volume.

Text2Physics

Part-aware supervision for text-conditioned physics.

Text2Physics contains 1,700 3D assets, each paired with eight text-and-property annotations. The same geometry can therefore represent multiple plausible physical interpretations instead of one fixed material assignment.

Every annotation is grounded to semantic 3D parts before Young's modulus, Poisson's ratio, and density are assigned from engineering references.

Browse the available Text2Physics dataset

Text2Physics data generation from engineering databases through semantic part labels to dense ground-truth physical properties. — The generation pipeline combines engineering references, OpenCLIP part labels, and VLM-guided property assignment to produce dense, part-aware supervision.

01
Segment

Render 15 upper-hemisphere views, generate five part-keyword sets, and select the most coherent OpenCLIP voxel result from five segmentation candidates.
02
Describe

Randomly select a subset of parts and generate diverse, physically plausible language conditions so one asset can express multiple material interpretations.
03
Assign

Query MatWeb, Wikipedia, and The Engineering Toolbox for valid ranges, then assign Young's modulus, Poisson's ratio, and density consistently with each description.

Quantitative results

Higher fidelity and lower physical error.

AutoPhyX with OpenCLIP achieves the strongest reported performance across rendering and physical-property metrics on Text2Physics.

Highlighted row marks the best result in every reported metric. Up and down indicate the preferred direction.
Method	PSNR up	SSIM up	LPIPS down	Avg. physics error down
NeRF2Physics	18.554	0.889	0.245	0.841
Gemini (avg.)	21.124	0.869	0.205	0.218
Pixie (CLIP)	23.421	0.908	0.092	0.078
AutoPhyX (CLIP)	25.105	0.916	0.091	0.069
AutoPhyX (OpenCLIP)	26.012	0.921	0.089	0.059

On the real-world ABO-500 benchmark, AutoPhyX lowers average displacement error to 9.433, compared with 11.826 for NeRF2Physics and 12.565 for Pixie. Feed-forward inference takes 2 seconds; optimization-based baselines require minutes to hours.

Citation

Cite this work.

AutoPhyX is currently under anonymous review for ECCV 2026. Author names will be added after the review period.

BibTeX

@inproceedings{anonymous2026autophyx,
  title     = {AutoPhyX: Automatic Text-Condition Physics Property Generation},
  author    = {Anonymous},
  booktitle = {ECCV},
  year      = {2026}
}

AutoPhyX