Hoglet-33/3.33BT-Pre-Training-Mix-Three-Of-Three
Viewer • Updated • 3.9M • 175
🤔 Thinking about the next gen Pebble
Ash
Hoglet-33
AI & ML interests
Open source AI, datasets, parameter efficiency, SLMs, AI for the betterment of humanity. Contact at ash@basicallyai.co
Recent Activity
repliedto their post 1 day ago
Today, we planned to release Pebble-50M and Pebble-50M-Chat to the world. Unfortunately, due to a few issues, that didn't go quite as planned.
What happened:
- Some data and benchmark results were lost or corrupted
- The models performed worse on benchmarks than our other Pebble models
Despite that, you can still find both models here:
Pebble-50M-beta: https://huggingface.co/basically-experimental/Pebble-50M-beta
Pebble-50M-Chat-beta: https://huggingface.co/basically-experimental/Pebble-50M-Chat-beta
There are still some interesting improvements in these models:
- Compatible with non-CUDA devices
- Vocabulary increased to 16K tokens
- Context length increased to 16K tokens
For now, there won't be any more Pebble releases for a while. We're going to take some time to experiment with other approaches and hopefully make the next generation a monumental leap over this one.
Follow for updates:
@Hoglet-33
https://huggingface.co/basically-ai
https://huggingface.co/basically-experimental updated a Space 1 day ago
basically-experimental/README reacted to theirpost with 😔 1 day ago
Today, we planned to release Pebble-50M and Pebble-50M-Chat to the world. Unfortunately, due to a few issues, that didn't go quite as planned.
What happened:
- Some data and benchmark results were lost or corrupted
- The models performed worse on benchmarks than our other Pebble models
Despite that, you can still find both models here:
Pebble-50M-beta: https://huggingface.co/basically-experimental/Pebble-50M-beta
Pebble-50M-Chat-beta: https://huggingface.co/basically-experimental/Pebble-50M-Chat-beta
There are still some interesting improvements in these models:
- Compatible with non-CUDA devices
- Vocabulary increased to 16K tokens
- Context length increased to 16K tokens
For now, there won't be any more Pebble releases for a while. We're going to take some time to experiment with other approaches and hopefully make the next generation a monumental leap over this one.
Follow for updates:
@Hoglet-33
https://huggingface.co/basically-ai
https://huggingface.co/basically-experimentalOrganizations
