Bc-AI/logs
Updated
🏗️ Building on HF
Boning Cui
Bc-AI
AI & ML interests
He/Him. I like LLM's and VLM's. I work with my other friends to make stuff. We are in year 7 and we are enthusiastic about AI. We are based in Australia 🇦🇺 (QLD Btw)
Recent Activity
published a model about 1 hour ago
OpenModel-Labs/Galaxy-1-Looped-Checkpoints reacted to kutomanovhennadii's post with 🔥 about 2 hours ago
How much load can a language model handle?
“Summarize a Wikipedia article in at least 300 words. Use no commas. Split your answer into three titled sections.”
This is the kind of task used in IFEval, a benchmark of how well models follow multiple requirements at once.
Suppose the model breaks one requirement. Did it lose track of an instruction? Did the no-comma rule make the summary harder to write? Or was summarizing the article difficult to begin with? A program can tell us what failed, but not separate instruction load from task difficulty and interacting constraints.
There is another approach: synthetic tests where a model retrieves a key's value from a long list. Those are easier to control, but they test information retrieval rather than the execution of multiple instructions.
So how can we measure the load itself?
That's the question behind my new article on the IEC instrument. Instead of an essay, the model gets meaningless text made of random words. Instead of complex constraints, it gets simple replacement rules such as `lake -> stone`. Its job is to rewrite the text while applying every rule.
The simplicity serves a purpose. Each rule applies to its own word, so we can add instructions without changing the operation. We can also set how many different words the text has and how often each repeats. And every application has a known correct outcome.
A single response thus produces many individually verifiable trials, not just one overall score. This lets us study how a model's behavior degrades under different kinds of load.
The article walks through the instrument, an actual prompt and GPT-4o mini response, and how to reproduce the trials. Code and data are public.
Article: https://huggingface.co/blog/kutomanovhennadii/how-to-measure-a-language-models-behavior-under-lo
Data: https://huggingface.co/datasets/kutomanovhennadii/iec-gpt-4o-mini-2024-07-18
Originally written in Russian. English translation assisted by AI.
reacted to iliya-radulov's post with 🔥 about 2 hours ago
Field Notes from a Low-Cost Robot Learning Lab: Hardware Churn, Software Whiplash, and What Actually Went Wrong
An informal companion piece documenting the hardware incidents, framework churn, and debugging dead-ends from roughly seven weeks of work. It is written for people building similar setups on a hobbyist budget and is candid about the time lost to causes like loose power cables and swapped ports.
Zenodo: https://zenodo.org/records/23259698
About me: I am an independent researcher. These works are part of a single-operator project on low-cost robot learning. My ORCID is 0000-0001-8943-5083. For questions or collaboration, feel free to reach out via the HF community.
Organizations
Bc-AI/T1-Mini-Preview-Q2_K-GGUF
Image-Text-to-Text • 4B • Updated • 19
Bc-AI/T1-Mini-Preview
Image-Text-to-Text • 4B • Updated • 17 • 1
Bc-AI/Code-thing
Updated
Bc-AI/joint-flow-3b
Updated • 1
Bc-AI/CodVa-1-session-001
Updated
Bc-AI/nova-1-standard-checkpoints
Updated
Bc-AI/vacinova-1-checkpoints
Updated
Bc-AI/nova-1-large-checkpoints
Updated
Bc-AI/nova-1-phase2-checkpoints
Updated
Bc-AI/suprava-1
Updated
Bc-AI/codva-10b
Updated • 1
Bc-AI/codva-exp-7b
Updated
Bc-AI/nova-chat-ui
Updated
Bc-AI/sam-x-chat-instruct
Updated • 1