Metadata-Version: 2.4
Name: flint-sglang
Version: 0.1.4
Summary: SGLang model support for Flint (tested with sglang 0.5.16 and transformers 5.12.1)
License-Expression: Apache-2.0
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
License-File: NOTICE
Dynamic: license-file

# flint-sglang

SGLang model support for Flint checkpoints, tested with SGLang 0.5.16 and transformers 5.12.1.
Installing the wheel registers:

- the `flint` model type with `transformers.AutoConfig`, through an `sglang.srt.plugins` entry point
  (which also sets `SGLANG_EXTERNAL_MODEL_PACKAGE=flint_sglang` and `FLASHINFER_USE_CUDA_NORM=1`);
- the `FlintForCausalLM` architecture for SGLang;
- the `flint` tool-call parser (`--tool-call-parser flint`): parses Flint's tool calls and constrains
  what Flint writes after its reasoning in turns with `tools` to Flint's tool-call format with an
  XGrammar structural tag (one call when the request sets `parallel_tool_calls` to false);
- the `flint` reasoning parser (`--reasoning-parser flint`): `<think>` parsing that never assumes a turn
  starts in reasoning (Flint decides per turn; its template has no thinking switch) and keeps a
  `<tool_call>` written inside the reasoning as reasoning;
- the `flint` grammar backend (`--grammar-backend flint`): XGrammar, with Flint's optional `<think>` block
  before every constraint: `response_format`, `regex`, `ebnf`, structural tags and the tool-call grammar.
  Flint refuses to start without it. It needs `--reasoning-parser flint`, which keeps that block out of
  `content`, and the plugin refuses that combination at launch, as well as `--enable-strict-thinking` and
  `--constrained-json-disable-any-whitespace` with this backend.

The supported install is `install.sh` (or the Dockerfile that runs it) in an
`lmsysorg/sglang:v0.5.16-cu130` environment: it also pins the kernel libraries Flint was validated
with and applies the FA4 learned-sink fix required by `--attention-backend fa4`. The wheel alone is
not enough; the model refuses to start on an unpatched FA4 kernel.
