Instructions to use tencent/Hy3 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use tencent/Hy3 with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="tencent/Hy3") messages = [ {"role": "user", "content": "Who are you?"}, ] pipe(messages)# pip install -U transformers accelerate # Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("tencent/Hy3") model = AutoModelForCausalLM.from_pretrained("tencent/Hy3", device_map="auto") messages = [ {"role": "user", "content": "Who are you?"}, ] inputs = tokenizer.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=256) print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Inference
- HuggingChat
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use tencent/Hy3 with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "tencent/Hy3" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "tencent/Hy3", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/tencent/Hy3
- SGLang
How to use tencent/Hy3 with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "tencent/Hy3" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "tencent/Hy3", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "tencent/Hy3" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "tencent/Hy3", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }' - Docker Model Runner
How to use tencent/Hy3 with Docker Model Runner:
docker model run hf.co/tencent/Hy3
Fix chat_template crash when tool-call arguments are an OpenAI-style JSON string
chat_template.jinja crashes when an assistant tool call's arguments are a JSON string (the OpenAI Chat Completions shape that many harnesses, traces, and datasets replay into apply_chat_template). Dict arguments already render correctly.
Repro (transformers 5.22.2, tokenizer only)
from transformers import AutoTokenizer
tok = AutoTokenizer.from_pretrained("tencent/Hy3")
tools = [{"type": "function", "function": {
"name": "get_weather", "description": "Get weather",
"parameters": {"type": "object",
"properties": {"city": {"type": "string"}, "metric": {"type": "boolean"}},
"required": ["city"]}}}]
def history(args):
return [
{"role": "user", "content": "Weather in Paris?"},
{"role": "assistant", "content": "", "tool_calls": [
{"id": "c1", "type": "function",
"function": {"name": "get_weather", "arguments": args}}]},
{"role": "tool", "tool_call_id": "c1", "name": "get_weather", "content": '{"temp": 21}'},
]
tok.apply_chat_template(history({"city": "Paris", "metric": True}), tools=tools, tokenize=False)
# OK:
# <tool_call:opensource>get_weather<tool_sep:opensource>
# <arg_key:opensource>city</arg_key:opensource>
# <arg_value:opensource>Paris</arg_value:opensource>
# ...
tok.apply_chat_template(history('{"city": "Paris", "metric": true}'), tools=tools, tokenize=False)
# UndefinedError: 'str object' has no attribute 'items'
The template does {% set arguments = tool['function']['arguments'] %} then {% for key, value in arguments.items() %} with no mapping/string branch. I checked discussions on this repo and did not find an existing report of this crash.
What changed
Before the existing <arg_key> / <arg_value> loop, string arguments are parsed into a mapping so the loop still emits the trained tags. A raw-string dump would skip those tags and break tool-call replay. transformers 5.22.2's chat-template environment has tojson but no fromjson filter, so the parse lives in the template (dict arguments pass through unchanged).
Tests
On current main (a960ebc3da), with this template substituted:
| Case | Current | This PR |
|---|---|---|
Dict arguments ({"city": "Paris", "metric": True}) |
renders | unchanged (byte-identical) |
OpenAI-style string arguments ('{"city": "Paris", "metric": true}') |
UndefinedError: 'str object' has no attribute 'items' |
renders, byte-identical to the dict case |
Empty dict / empty object string ({} / '{}') |
dict OK; string crashes | both render, identical |
| Nested dict / nested JSON string | dict OK; string crashes | both render, identical |
Unicode dict / unicode JSON string (París, 東京の天気) |
dict OK; string crashes | both render, identical |
| Multiple dict tool calls | renders | unchanged (byte-identical) |
This does not change tool-name escaping.
Verified with transformers 5.22.2 via tokenizer.apply_chat_template (no weights).