YAML Metadata Warning:empty or missing yaml metadata in repo card
Check out the documentation for more information.
pi0.5 Inference Server
pi05-server runs the pi0.5 vision-language-action model on the board's NPU and serves it over HTTP.
Your robot sends camera images, its current state and a task in plain language. The server sends
back a chunk of 50 future actions.
The model runs with W8A8 quantization: 8-bit weights and 8-bit activations for the language model and the action expert. This is the default and the recommended configuration.
How it works
YOUR SIDE (robot / PC) THE BOARD (aarch64 + NPU)
ββββββββββββββββββββββββββββββ βββββββββββββββββββββββββββββββββββββββββ
$ ./pi05-server ... --port 8080
At startup (once):
load the W8A8 weights
deploy the model on the NPU
Then the server waits for requests.
3 camera frames (JPEG/PNG)β
robot state βββΌββ POST /v1/actions βββΊ 1. decode the images
task ("pick up the cup") ββ (JSON) 2. vision encoder (NPU)
3. language model prefill (NPU, W8A8)
4. action expert,
10 denoising steps (NPU, W8A8)
β
ββββββββββββββββ 50 x 32 actions (JSON) ββββββββββββββββββββββ
unnormalize the actions,
execute them on the robot,
send the next observation ...
The model is loaded and deployed once, when the server starts. A request pays only for the inference itself (vision encoding, prefill and action sampling), so the server stays running between requests.
The NPU runs one inference at a time. When several requests arrive together, they are queued and served in order.
Files
| File | Description |
|---|---|
pi05-server |
the server executable |
model.safetensors |
pi0.5 checkpoint (vision encoder and embeddings) |
pi05_llm_w8a8.gguf |
W8A8 language model |
pi05_ae_w8a8.gguf |
W8A8 action expert |
model.safetensors is the G1-finetuned pi0.5 checkpoint
niravpanchalmerai/g1_pi05_v6:
hf download niravpanchalmerai/g1_pi05_v6 model.safetensors --local-dir g1_pi05_v6
Starting the server
On the board:
sh pi05-server-0.1.0-Linux.sh --skip-license --exclude-subdir
./pi05-server -w model.safetensors \
-w-llm-w8a8 pi05_llm_w8a8.gguf \
-w-ae-w8a8 pi05_ae_w8a8.gguf \
--host 0.0.0.0 --port 8080
| Option | Description | Default |
|---|---|---|
-w FILE |
pi0.5 model.safetensors |
required |
-w-llm-w8a8 FILE |
W8A8 language model | pi05_llm_w8a8.gguf |
-w-ae-w8a8 FILE |
W8A8 action expert | pi05_ae_w8a8.gguf |
--host ADDR |
address to listen on | 0.0.0.0 |
--port N |
port to listen on | 8080 |
--npu-idx N |
NPU cluster, 0 or 1 | 0 |
The server is ready when it prints that it is listening. You can also poll GET /health.
API
POST /v1/actions: get the next action chunk
Request (Content-Type: application/json)
{
"prompt": "pick up the red cup and place it in the bowl",
"state": [0.12, -0.40, 0.05, 0.88, -0.10, 0.33, 1.00],
"images": [
"/9j/4AAQSkZJRgABAQAAAQABAAD...",
"/9j/4AAQSkZJRgABAQAAAQABAAD...",
"/9j/4AAQSkZJRgABAQAAAQABAAD..."
],
"seed": 0
}
| Field | Type | Required | Description |
|---|---|---|---|
prompt |
string | yes | The task in plain language. |
state |
array of numbers | no | Current robot state, normalized to [-1, 1], up to 32 values. Missing values are treated as 0. |
images |
array of strings | yes | Exactly 3 camera images, base64-encoded JPEG or PNG. Images of any size are accepted and resized to 224Γ224 on the board. |
seed |
integer | no | Seed of the initial noise for the action sampler. Leave it out for a fresh random seed on each request. Set it for reproducible results. |
noise |
array of 50 arrays of 32 numbers | no | The initial noise itself, drawn from N(0, 1), shape [50, 32]. Use it only for validation (see below). When given, seed is ignored. |
Camera order matters. Send the 3 cameras in the same order that was used when the model was trained (for example: base camera, left wrist, right wrist).
Validating against a reference (optional). You do not need noise in normal operation. Both
seed and noise are optional, and requests without them work as usual. seed makes the server's
results reproducible, but the same seed does not give the same noise as NumPy or PyTorch, because
the random number generators differ. To compare the server's actions with your own reference run
of pi0.5 (for example a PyTorch/openpi run in fp32), generate the noise once, pass it to your
reference as the sampler's initial noise and send the same values in noise. Both sides then start
from identical noise, so any remaining difference comes from the model execution (W8A8 on the NPU).
noise = np.random.default_rng(0).standard_normal((50, 32)).astype(np.float32)
# reference: your pi0.5 sampler, starting from `noise`
# server: requests.post(..., json={..., "noise": noise.tolist()})
Response (200 OK)
{
"actions": [
[ 0.0213, -0.1180, 0.0457, 0.3021, -0.0042, 0.1175, 0.9810, 0.0, ... ],
[ 0.0251, -0.1123, 0.0490, 0.2987, -0.0051, 0.1150, 0.9802, 0.0, ... ],
...
],
"shape": [50, 32],
"seed": 0,
"timing": {
"encode_s": 0.000,
"prefill_s": 0.000,
"sample_s": 0.000,
"total_s": 0.000
}
}
| Field | Description |
|---|---|
actions |
50 rows (time steps) Γ 32 values, normalized. |
shape |
always [50, 32] |
seed |
the seed the initial noise came from: yours, or the random one the server picked. Send it back as seed to reproduce the result. Not present when you sent noise. |
timing |
time spent on the NPU for this request, in seconds |
Using the actions
- The model always outputs 32 values per step. Use only the first N, where N is your robot's action dimension (for example 7 for a 6-DoF arm with a gripper). Ignore the rest.
- The actions are normalized. Unnormalize them with the same statistics that were used to normalize the state and actions in training.
- A common pattern is to execute the first part of the chunk, then send a new observation, rather than executing all 50 steps open-loop.
Errors
Invalid requests get 400 Bad Request with a JSON body:
{ "error": { "message": "images: 2 given, 3 required", "type": "invalid_request_error" } }
Typical causes: missing prompt or images, not exactly 3 images, an image that cannot be decoded,
more than 32 state values, or noise that is not 50 Γ 32 numbers. A failure on the board itself returns 500.
GET /health
{ "status": "ok" }
GET /v1/info
{
"model": "pi0.5",
"precision": "w8a8",
"images": 3,
"image_size": [224, 224],
"state_dim": 32,
"action_horizon": 50,
"action_dim": 32,
"denoising_steps": 10
}
Examples
Test from the PC (through an ssh tunnel to the board):
ssh -J <user>@<labhost> -N -L 8080:localhost:8080 root@10.0.0.2
curl -s localhost:8080/health
curl -s localhost:8080/v1/info
python3 tools/pi05_client.py --server http://localhost:8080 \
test_data/pi05-inputs/cam0_cam_left_high.png test_data/pi05-inputs/cam1_cam_left_wrist.png \
test_data/pi05-inputs/cam2_cam_right_wrist.png \
--prompt-file test_data/pi05-inputs/prompt.txt \
--state-file test_data/pi05-inputs/state.txt \
--noise test_data/pi05-inputs/noise.f32
--noise sends the initial noise from a raw f32 [50][32] file (the format of
pi05-runner --noise), so the server's actions can be compared with the runner's or a
reference run from the same noise. Leave it out and pass --seed N for a seeded random
noise instead.
- Downloads last month
- 19
We're not able to determine the quantization variants.