diffusion-planner-p150 on Tenstorrent p150

Derived from AutowareFoundation/diffusion_planner (tag v5.0), the Diffusion Planner network that TIER IV trained for Autoware's autoware_diffusion_planner trajectory planner, with unmodified weights. Diffusion Planner: Zheng et al., arXiv:2501.15564. See the original model card for training details.

Diffusion Planner v5.0, a 14.55M-parameter diffusion-transformer planner: six MLP-Mixer entity encoders and a 6-block transformer fusion encoder over 564 scene tokens, a 3-block DiT decoder with adaLN (self-attention over the ego and 320 agents, cross-attention to the scene) evaluated 11 times by a 10-step DPM-Solver++(2M) loop, and a turn-indicator head; the whole plan runs on the device. The 15 planner tensors of the Autoware node go in (ego and neighbour histories, lanes with speed limits, route lanes, polygons, line strings, goal pose, ego shape, turn-indicator history, in the ego frame). Out come an 8 s ego trajectory (80 poses at 10 Hz with velocity and acceleration), an 8 s predicted path for every valid neighbour and a turn-indicator command, with the pre- and post-processing of the autoware_diffusion_planner node (input normalization, agent ordering, velocity smoothing, force stop, turn-indicator decision).

Runs on Tenstorrent p150 (mesh P150).

Packaged and published with tt-model-manager 0.1.0 (manifest schema 5.1).

At a glance

Hardware	p150
License	apache-2.0

Intended use

Ego trajectory planning and neighbour motion prediction for Autoware-style driving stacks: research, prototyping and offline evaluation of recorded or simulated scenes, with the outputs of one autoware_diffusion_planner node. The caller builds the planner tensors from its own ROS messages and Lanelet2 map.

Prerequisites

• The Tenstorrent CLI, tt: uv tool install tenstorrent (or use tt-model alone, see the Quickstart)
• Docker
• Tenstorrent hardware: p150

Quickstart

tt serve changh95/diffusion-planner-p150

tt serve downloads the Docker image and the AutowareFoundation/diffusion_planner at 423efde67f5414734da43a7ad856c17ceb8b51aa weights (into your HF cache; they are not in the image), then starts the model's own HTTP server on port 20000 (or the next free port, if that one is busy). The first start compiles kernels for your device, which takes several minutes; the server is ready when it logs Application startup complete.

Without tt-cli: tt-model serve changh95/diffusion-planner-p150.

Send the sample scene shipped in this repo (a scripted scene on Autoware's Kashiwanoha map, 88 neighbours) to /predict, on the port tt serve reported:

curl -sLO https://huggingface.co/changh95/diffusion-planner-p150/resolve/main/code/tt_diffusion_planner/samples/kashiwanoha_dense.npz
printf '{"inputs": {"format": "npz", "data": "%s"}}' "$(base64 -w0 kashiwanoha_dense.npz)" > req.json
curl -s localhost:20000/predict -H 'Content-Type: application/json' -d @req.json

code/tt_diffusion_planner/server/client.py in this repo (standard library only) builds the same request from any .npz of the 15 tensors, with per-request params.

Capabilities

This package exposes the model's own HTTP server on port 20000, not an OpenAI-compatible chat API — its request and response shapes are the model's own. See the author's notes for the payload it expects.

POST /predict takes one planning instant (the 15 planner tensors) as JSON and returns the ego plan, the neighbour predictions and the turn-indicator command. GET /health and GET /info report the server and device state (/info lists the 15 tensors with their shapes); GET /v1/models is a stub, not a chat API.

request field	meaning
inputs	{"format": "npz", "data": "<base64 .npz>"} or {"format": "json", "arrays": {name: nested list}} with exactly the 15 raw tensors of the node's create_input_data(), float32, ego (base_link) frame, before normalization: sampled_trajectories [1,321,81,4], ego_agent_past [1,31,4], ego_current_state [1,10], neighbor_agents_past [1,320,31,11], static_objects [1,5,10], lanes [1,140,20,33], lanes_speed_limit [1,140,1], route_lanes [1,25,20,33], route_lanes_speed_limit [1,25,1], polygons [1,10,40,3], line_strings [1,60,20,4], goal_pose [1,4], ego_shape [1,3], turn_indicators [1,31], delay [1,1]
params (optional)	velocity_smoothing_window (default 8), stopping_threshold (0.3 m/s), turn_indicator_keep_offset (-1.25), return_denoising_steps (false)
output_format (optional)	json (default) or npz (adds the poses as a lossless base64 npz)

Response for the sample scene (the trajectory cut to 3 of 80 rows, the predicted-agents array elided):

{"model": "diffusion-planner-p150", "frame_id": "base_link", "num_poses": 80,
 "columns": ["x", "y", "yaw", "cos", "sin", "velocity", "acceleration"],
 "trajectory": [[0.3644, 0.01, -0.0169, 0.9964, -0.0169, 3.8033, -0.0392], [0.7626, -0.0021, -0.0412, 0.9968, -0.0412, 3.7994, -0.5496], [1.1517, -0.0281, -0.0674, 0.9942, -0.0672, 3.7444, -0.4683], "... 77 more rows"],
 "turn_indicator": {"command": 1, "command_name": "DISABLE", "keep_selected": true, "held": false, "logits": [-15.96, -5.12, -5.235, -0.764, 5.502], "probabilities": [1.7e-09, 8.4e-05, 7.5e-05, 0.0066, 0.9932]},
 "predicted_agents": {"format": "npz", "key": "predicted_agents", "dtype": "float32", "shape": [88, 80, 5], "data": "<base64 npz>"},
 "meta": {"predicted_agent_columns": ["x", "y", "yaw", "cos", "sin"], "force_stop": false,
          "valid_counts": {"ego": 1, "neighbor": 88, "static": 0, "lane": 123, "route": 17, "polygon": 0, "line_string": 60, "goal": 1, "ego_shape": 1, "turn": 1}},
 "timing_ms": {"preprocess": 2.4, "device": 22.8, "postprocess": 1.2, "total": 36.9, "decode": 8.4, "model_call": 26.7}}

• trajectory: 80 poses at 0.1-8.0 s in base_link (the ego frame of the inputs), metres, radians, m/s and m/s², post-processed like the node's ~/output/trajectory: velocity from consecutive poses, a forward moving average over velocity_smoothing_window poses, force stop (poses frozen once the smoothed speed falls below stopping_threshold), acceleration by finite difference. yaw is what tf2::getYaw reads from the node's quaternion; cos / sin are the raw network outputs.
• turn_indicator.command: 0 NO_COMMAND, 1 DISABLE, 2 ENABLE_LEFT, 3 ENABLE_RIGHT.
• predicted_agents: one 80-point path (x, y, yaw, cos, sin) in base_link per non-empty neighbour row, in input order (meta.predicted_agent_rows).
• A Python API with the same outputs (tt_diffusion_planner.DiffusionPlanner) is described in code/PYTHON.md, including what the caller keeps between plans; it needs a tt-metal build with the two patches in this repo's patches/ directory.

Example output

Plans computed by this package for 33 consecutive planning instants of nuScenes scene-0061 (Singapore: following a van, then a left turn; 2 Hz, shown at 7 frames/s), drawn from above with the lanes, the neighbours' predicted paths and the ego plan. Also as MP4. Non-commercial use only: rendered from the nuScenes dataset, © Motional AD Inc., CC BY-NC-SA 4.0 and the nuScenes Terms of Use; Motional does not endorse this work. Attributions for every file in media/: media/ATTRIBUTION.md.

Expected performance

Agreement with the fp32 CPU reference of the same network (same weights, same pre- and post-processing, same initial solver state); gates: ego plan max ≤ 1.0 m and mean ≤ 0.3 m over the 8 s, turn command equal, neighbour paths' median per-agent max ≤ 1.5 m:

eval	this package	fp32 CPU reference	scenes
Ego plan difference (max / mean over 8 s), sample scene kashiwanoha_dense	2.0 cm / 0.8 cm, turn command equal	–	1
Ego plan difference, sample scene straight_road	2.3 cm / 0.9 cm, turn command equal	–	1
Ego plan difference, gate set: worst max / worst mean (median of the means)	0.346 m / 0.154 m (1.3 cm)	–	99
Turn command equal, gate set	99 of 99	–	99
Neighbour paths, median per-agent max difference, worst scene of the gate set	0.094 m	–	99
Held-out scenes within every gate: worst ego max / worst mean	50 of 50: 0.507 m / 0.174 m	–	50
Module output PCC, sample scenes: encoder categories / scene encoding / teacher-forced decoder steps (minimum)	0.999948 / 0.999976 / 0.9999995	–	2
Open-loop distance to the logged nuScenes drive at 8 s, ADE / FDE	4.75 m / 11.87 m	4.74 m / 11.87 m	92
Turn command equal to the logged blinker, nuScenes	88 of 92	88 of 92	92

Gate set: the 2 shipped sample scenes, 5 scripted research scenes and 92 planning instants converted from nuScenes v1.0-mini (Singapore and Boston). Held-out scenes, never used to choose the device numerics: the 33 consecutive instants of nuScenes scene-0061, 4 full-capacity and noisy-initial-state scenes and 13 scenes at every agent-bucket boundary. The open-loop rows compare with one human driver, not a planning benchmark; the paper's nuPlan closed-loop benchmark was not run, and the weights repo publishes no dataset-level metric to compare with.

speed (batch 1, warm)	input	median	throughput
Device time per plan, back-to-back trace replays, up to 31 neighbours	straight_road sample	17.44 ms	57.3 plans/s
Device time per plan, 32-63 neighbours	nuScenes scene-0103 instant	19.25 ms	51.9 plans/s
Device time per plan, 64-95 neighbours	kashiwanoha_dense sample (88 neighbours)	20.00 ms	50.0 plans/s
Device time per plan, 96-127 / 128-191 / 192-320 neighbours	synthetic scenes	21.11 / 22.30 / 26.65 ms	47.4 / 44.8 / 37.5 plans/s
Python model() call (pre-processing, device, post-processing)	kashiwanoha_dense sample	26.7 ms (p99 27.5)	37.4 plans/s
Served POST /predict, server timing_ms.total	kashiwanoha_dense sample (152 kB request)	37.2 ms	–
Served POST /predict, client round trip on loopback	kashiwanoha_dense sample	41.6 ms	–

Measured with this package's code on one p150 in an 8-vCPU AMD EPYC-Rome host: code/scripts/bench.py (100 warm iterations per stage; device time over back-to-back trace replays) and 50 served requests of the sample scene. The device time depends on the number of neighbours: each plan runs the trace of the smallest agent bucket that holds the scene, with outputs equal to the full-capacity trace. The host stages (pre-processing, tensor packing, transfers, post-processing) take about 7 ms of the model() call, and request decoding 8.6 ms of the served time.

Limitations

• Do not use it to drive or control a vehicle, in closed loop or as any safety function: it is not a certified Autoware component and has no functional-safety qualification.
• Do not use its outputs as a Diffusion Planner benchmark: the weights are TIER IV's, trained on synthetic and real driving data that is not published. On nuScenes, a domain they were not trained on, the plans are plausible but conservative (moving plans about 17 % shorter than the logged drive), on CPU and p150 alike.
• Not a ROS 2 node: it serves HTTP and Python only. Building the 15 planner tensors from ROS messages and the Lanelet2 map is the caller's. The API is stateless: the node's turn-indicator hold window, the agent and ego histories and the initial-solver-state prefix between plans are the caller's, and the node's guidance services (start, stop, centerline guidance) are not available.
• Batch 1, one plan per request; requests are served one at a time. The v5.0 capacities (320 neighbours, 140 lanes, 25 route lanes, 10 polygons, 60 line strings) and 10 solver steps are fixed; the map entities are always computed at full capacity.
• One p150 only. The image runs device dispatch on the chip's Ethernet cores, so multi-chip meshes are not supported.
• Differences from the deployed node: the output stays in base_link (the node transforms it to map with the ego pose), the turn command is decided without the node's 1 s hold window, and the delay input is accepted and ignored (the node's multi-step mode does not read it). Outputs were compared with the fp32 CPU reference, not with TensorRT.
• Device numerics (fp32 residual streams, LayerNorm, attention and solver state; split bf16 + fp32 matmul operands; bf16 weights in the encoder MLPs) move the plan slightly from the fp32 reference, by a few cm on most scenes and up to 0.35 m (gate set) or 0.51 m (a synthetic scene with 192 neighbours) at the worst; the effect differs from scene to scene. Neighbour paths are gated only by their median, so single agents can differ more: on the kashiwanoha_dense sample the per-agent max difference is 8.6 cm median, 0.23 m at the 90th percentile and 3.72 m for the worst of 88 agents.

Licensing

• Weights: AutowareFoundation/diffusion_planner at tag v5.0 (commit 423efde67f5414734da43a7ad856c17ceb8b51aa), Apache-2.0 per its model card. Downloaded by tt serve at that commit, not redistributed here. The upstream card states that TIER IV trained the models on its synthetic and real driving data; the dataset composition is not published.
• Pre- and post-processing ported from autoware_universe planning/autoware_diffusion_planner (Apache-2.0).
• Port and serving code (code/) and the tt-metal patches in patches/: Apache-2.0.
• Sample scenes code/tt_diffusion_planner/samples/: kashiwanoha_dense.npz is derived from the Apache-2.0 Lanelet2 map AutowareFoundation/map-carla-kashiwanoha (0.2.0) with a scripted ego, route and agents; straight_road.npz is generated by this repo's code. Both Apache-2.0.
• Demo media in media/ are renders, not part of the package. The demo video and media/*_NC.* come from nuScenes: non-commercial use only, CC BY-NC-SA 4.0, © Motional AD Inc.; Motional does not endorse this work. The renders of the sample scenes: Apache-2.0. Full attributions: media/ATTRIBUTION.md.

Related packages

• The Autoware collection: the other Autoware perception and planning models packaged for the p150.

Feedback

Questions or problems with this package: open a discussion at https://huggingface.co/changh95/diffusion-planner-p150/discussions — that is what reaches its author. A problem with the tt tooling itself: tt report issue (collects your environment and opens a prefilled issue against tenstorrent/tt-cli). Product feedback: support@tenstorrent.com.

Provenance

The exact sources the image was built from — code/ in this repo is byte-identical to the model code inside the image:

component	built from
tt-metal	44d66500520fda9f2c7060c0f6b41ec48f7ab37e (dirty tree — the image includes uncommitted changes)
code/ digest	5da22b97bbf89b03 (sha256, first 16 hex digits)
built	2026-10-16T06:42:04+00:00 by tt-model 0.1.0