3DSceneBench

3DSceneBench

Yani Zhang1* Xinyu Wang1* Hanwei Wu1* Jindu Wang1 Siyuan Luo1 Jason Pho1 Zonglin Di2 Yiyou Sun2 Yi Zhu1†

1Prana Labs2University of California, Berkeley

*Equal Contribution†Corresponding Author

Can coding agents build functional and versatile 3D worlds?

Can coding agents build one simulated world that supports a scene's many uses? 3DSceneBench covers 49 whole-room scenes in seven categories, with up to 3,002 objects per scene, 471 tasks and 36,871 per-object tests. Each world is scored for appearance, geometry and functionality.

or go straight to the leaderboard

Scenes

How it works

1

See

The agent gets eight pictures of a scene with their camera poses, up to 16 close-ups, and the scene's tasks in plain words. Weights and friction are never given.

2

Build

In a closed container, with up to 180 minutes or 10M output tokens, it writes one MuJoCo scene. Every object is a named body; every door and drawer is a real joint.

0Appearance
0Geometry
0Functionality

3

Score

Verifiers load the world, render it, measure it, push its objects, open its doors and let a robot work in it, then do the same in the reference room. The dials show the best score any agent reaches.

Built to be used, not just seen.

A picture can't tell you if it carries.

Reference world

Pushed once, the car rolls across the rug.

Submitted world

Same push. The car moves 4.7 cm and stops.

Same scene, same camera, same test, both recorded in the simulator. The submitted world is drawn in plain colours, without its textures.

Results

Reference world

Submitted world

Same scene, same camera, same test, both recorded in the simulator. Marks are from each build's grade report. The submitted world is drawn in plain colours, without its textures.

Leaderboard

Bring your agent.

3DSceneBench mini is out: seven of the 49 scenes, one from each category, each with its reference room inside the task, so a run ends with the full scores on your own machine. The seven are the Shared Office, Pipe Maintenance Room, Outdoor Gear Store, Rehabilitation Center, Chemistry Lab, Bakery and Home Office. The other 42 scenes are coming soon. Getting started, step by step →

First

Get the code and the benchmark

git clone https://github.com/PranaLabs/3DSceneBench
cd 3DSceneBench
uv tool install harbor==0.23.0
harbor auth login
harbor download pranalabs-ai-team/3dscenebench-mini@v1.0.0

Then build the grader once and check your setup for free, as the guide shows. The benchmark scores a world and does not mind how the world was made, so there are two ways to produce one.

Way 1

Your agent builds the worlds

For testing a coding agent or a model. Harbor starts your agent inside each scene's container, the agent builds the world there, and the grader scores it. Claude Code, Codex and OpenCode are supported.

harbor run -p 3dscenebench-mini -a claude-code \
  -m anthropic/<model> \
  --ak disallowed_tools=WebFetch,WebSearch -n 1

Way 2

You bring worlds your own pipeline built

For testing a method you already have, such as a reconstruction pipeline or a scene generator. Build the worlds anywhere, with no limits, and hand in the files. Harbor only grades them.

PYTHONPATH=$PWD harbor run -p 3dscenebench-mini \
  -a scenebench_harbor.submit_files:SubmitFiles \
  --ak folder=$PWD/my_copies -n 1
Limits per scene, way 1
3 hours or 10M output tokens, no internet
Machine
Linux or macOS with Docker. No GPU needed
Memory for Docker
16 GB for the small scenes, 48 GB for all seven
Licence
Apache-2.0

For more information about adding your run to the leaderboard, see Get on the leaderboard on GitHub.