Fix chat_template crash when tool-call arguments are an OpenAI-style JSON string

#28
by SamJoshua - opened

chat_template.jinja crashes when an assistant tool call's arguments are a JSON string (the OpenAI Chat Completions shape that many harnesses, traces, and datasets replay into apply_chat_template). Dict arguments already render correctly.

Repro (transformers 5.22.2, tokenizer only)

from transformers import AutoTokenizer

tok = AutoTokenizer.from_pretrained("tencent/Hy3")
tools = [{"type": "function", "function": {
    "name": "get_weather", "description": "Get weather",
    "parameters": {"type": "object",
                   "properties": {"city": {"type": "string"}, "metric": {"type": "boolean"}},
                   "required": ["city"]}}}]

def history(args):
    return [
        {"role": "user", "content": "Weather in Paris?"},
        {"role": "assistant", "content": "", "tool_calls": [
            {"id": "c1", "type": "function",
             "function": {"name": "get_weather", "arguments": args}}]},
        {"role": "tool", "tool_call_id": "c1", "name": "get_weather", "content": '{"temp": 21}'},
    ]

tok.apply_chat_template(history({"city": "Paris", "metric": True}), tools=tools, tokenize=False)
# OK:
# <tool_call:opensource>get_weather<tool_sep:opensource>
# <arg_key:opensource>city</arg_key:opensource>
# <arg_value:opensource>Paris</arg_value:opensource>
# ...

tok.apply_chat_template(history('{"city": "Paris", "metric": true}'), tools=tools, tokenize=False)
# UndefinedError: 'str object' has no attribute 'items'

The template does {% set arguments = tool['function']['arguments'] %} then {% for key, value in arguments.items() %} with no mapping/string branch. I checked discussions on this repo and did not find an existing report of this crash.

What changed

Before the existing <arg_key> / <arg_value> loop, string arguments are parsed into a mapping so the loop still emits the trained tags. A raw-string dump would skip those tags and break tool-call replay. transformers 5.22.2's chat-template environment has tojson but no fromjson filter, so the parse lives in the template (dict arguments pass through unchanged).

Tests

On current main (a960ebc3da), with this template substituted:

Case Current This PR
Dict arguments ({"city": "Paris", "metric": True}) renders unchanged (byte-identical)
OpenAI-style string arguments ('{"city": "Paris", "metric": true}') UndefinedError: 'str object' has no attribute 'items' renders, byte-identical to the dict case
Empty dict / empty object string ({} / '{}') dict OK; string crashes both render, identical
Nested dict / nested JSON string dict OK; string crashes both render, identical
Unicode dict / unicode JSON string (París, 東京の天気) dict OK; string crashes both render, identical
Multiple dict tool calls renders unchanged (byte-identical)

This does not change tool-name escaping.

Verified with transformers 5.22.2 via tokenizer.apply_chat_template (no weights).

Ready to merge
This branch is ready to get merged automatically.

Sign up or log in to comment