ccore commited on
Commit
196c03f
·
verified ·
1 Parent(s): 1bf0f95

Add optional vision projector and image-guided tree example

Browse files

Add the original Qwen3.5-4B BF16 vision projector, tree reference and generated scene screenshots, and llama.cpp usage instructions. Preserve all existing model weights and other release files.

.gitattributes CHANGED
@@ -49,3 +49,6 @@ images/dif.png filter=lfs diff=lfs merge=lfs -text
49
  images/roma3.png filter=lfs diff=lfs merge=lfs -text
50
  tokenizer.json filter=lfs diff=lfs merge=lfs -text
51
  Qwen3.5-4B-3D-Coder_Technical_Report.pdf filter=lfs diff=lfs merge=lfs -text
 
 
 
 
49
  images/roma3.png filter=lfs diff=lfs merge=lfs -text
50
  tokenizer.json filter=lfs diff=lfs merge=lfs -text
51
  Qwen3.5-4B-3D-Coder_Technical_Report.pdf filter=lfs diff=lfs merge=lfs -text
52
+ vision-tree-reference.png filter=lfs diff=lfs merge=lfs -text
53
+ vision-tree-example.png filter=lfs diff=lfs merge=lfs -text
54
+ mmproj-Qwen3.5-4B-BF16.gguf filter=lfs diff=lfs merge=lfs -text
README.md CHANGED
@@ -22,6 +22,48 @@ This is an early preview of a 4B model I am still training, with a particular fo
22
 
23
  This is an **independent fine-tune** by [CCORE](https://huggingface.co/ccore), based on [Qwen/Qwen3.5-4B](https://huggingface.co/Qwen/Qwen3.5-4B). **It is not an official Qwen release.**
24
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
25
  ## Technical report — now available
26
 
27
  The **technical report for this training experiment is now available**.
 
22
 
23
  This is an **independent fine-tune** by [CCORE](https://huggingface.co/ccore), based on [Qwen/Qwen3.5-4B](https://huggingface.co/Qwen/Qwen3.5-4B). **It is not an official Qwen release.**
24
 
25
+ ## Optional vision support — October 30, 2026
26
+
27
+ The **Q4_K_M and Q8_0 GGUF versions can also accept image inputs** when loaded with the original Qwen3.5-4B vision encoder/projector. You can attach a reference image and ask the model to generate a Three.js scene from it.
28
+
29
+ [**Download the optional BF16 vision projector**](https://huggingface.co/ccore/Qwen3.5-4B-3D-Coder/resolve/main/mmproj-Qwen3.5-4B-BF16.gguf?download=true) — approximately **644 MiB**. Download it once; the same file is used with either GGUF quantization.
30
+
31
+ ### Image-guided Three.js example
32
+
33
+ With a reference image attached and the Three.js system prompt below, the user prompt was simply:
34
+
35
+ ```text
36
+ create this scene using voxels 3d
37
+ ```
38
+
39
+ <table>
40
+ <thead><tr><th>Input reference</th><th>Generated Three.js scene</th></tr></thead>
41
+ <tbody><tr>
42
+ <td width="40%" align="center"><a href="https://huggingface.co/ccore/Qwen3.5-4B-3D-Coder/blob/main/vision-tree-reference.png"><img src="vision-tree-reference.png" alt="Input reference: an orange voxel tree on a floating island" width="404" /></a></td>
43
+ <td width="60%" align="center"><a href="https://huggingface.co/ccore/Qwen3.5-4B-3D-Coder/blob/main/vision-tree-example.png"><img src="vision-tree-example.png" alt="Generated Three.js scene: Tree Island" width="1000" /></a></td>
44
+ </tr></tbody>
45
+ </table>
46
+
47
+ The reference image was supplied in [community discussion #9](https://huggingface.co/ccore/Qwen3.5-4B-3D-Coder/discussions/9); credit for the reference belongs to its original creator. The right-hand image is a screenshot of the generated scene. This is a qualitative example of image-guided generation; the output interprets the reference and does not reproduce every detail.
48
+
49
+ ### Load the GGUF with vision in llama.cpp
50
+
51
+ Download a model GGUF and the projector, then load both files explicitly:
52
+
53
+ ```bash
54
+ llama-server \
55
+ -m Qwen3.5-4B-3D-Coder-Q4_K_M.gguf \
56
+ --mmproj mmproj-Qwen3.5-4B-BF16.gguf \
57
+ --n-gpu-layers all --flash-attn on \
58
+ --ctx-size 65536 --parallel 1
59
+ ```
60
+
61
+ Open the server's WebUI, attach an image, and send your prompt. The setup was tested locally with **llama.cpp b11889**. For Q8_0, change the model filename to `Qwen3.5-4B-3D-Coder-Q8_0.gguf`; `--cache-type-k q8_0 --cache-type-v q8_0` can reduce KV-cache memory use and was tested locally with vision at a configured 64K context. These were short tests, not full-context stress tests.
62
+
63
+ The projector uses additional memory when loaded, and image tokens share the context window with the prompt and generated code. Image support in other frontends depends on their multimodal implementation.
64
+
65
+ **Projector provenance:** this is the unmodified BF16 projector from [lmstudio-community/Qwen3.5-4B-GGUF](https://huggingface.co/lmstudio-community/Qwen3.5-4B-GGUF/tree/f9f88ac3e234be915e23811a6d28ea287bdb927e), based on [Qwen/Qwen3.5-4B](https://huggingface.co/Qwen/Qwen3.5-4B), released under Apache 2.0. It is an optional companion file for the existing GGUF weights.
66
+
67
  ## Technical report — now available
68
 
69
  The **technical report for this training experiment is now available**.
mmproj-Qwen3.5-4B-BF16.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ae08d9d7eceb8f2d0672d61b5e6aa78b611f2942b55ed71d21414980cc454b91
3
+ size 675568768
vision-tree-example.png ADDED

Git LFS Details

  • SHA256: 53b99d0cd3a68aeb7e18bedf54d1950fcfe7b7de4f61bf5e96848e82feb01069
  • Pointer size: 131 Bytes
  • Size of remote file: 280 kB
vision-tree-reference.png ADDED

Git LFS Details

  • SHA256: b4f68f83fec288574cd977fa016a7618f03d77196c9bb316e4e555f044879a84
  • Pointer size: 131 Bytes
  • Size of remote file: 162 kB