yano

The main reason I'd think to have sub-agents is

1. Clean context so they only look at the task before them (and smaller context windows means it's probably faster)
2. Different more specialized role descriptions that makes them more efficient in said task
3. Separation of tasks which you can see in your GUI or audit later
4. Concurrency where
   4a)if you have multiple models you can run at the same time at full speed
   4b)Some agents can run on simpler/cheaper models

Unless it's a particularly complex task or you can subdivide it, I'd think sub agents just forces more thinking tokens.

Though take with a grain of salt, i haven't found a setup I'm happy with yet so I'm not running agents yet.

More useful is good.

Though the trend of a 2x improvement needing a 3x-5x increase in parameters/size and training means getting the 6% success rate to 90%+ may take quite a few iterations before we get to what i'd consider AGI yet.

On the other hand it probably does decent RPing.

GPT Astra im referring too.

Indeed i figured. But if it's making massive mistakes or making legal documents based on made up legal cases it is worthless and not intelligent, rather making something that still 'looks right' on the surface.

Until it does everything perfectly that we need it to do, you have to be skeptical of everything it outputs.

Your assumption in that LLMs are "just prediction engines" rather than having true intelligence is something I just cannot help to not comment here on "hugging face social media".

I hate to say that you might not know what you are talking about but the way you talk in high technical sense might resist you from actually looking into things seriously

After that, this is when you should consider and read attention is all your need a few times to grasp why the KQV cache layer matters and how this performed so well while avoiding a lot of the issues from LSTM. Once you are here. I hope you may be a bit more skeptical, where you actually can have sufficent knowlege to say: These large networks might've abstracted critical thinking, though very different than us, but hold all hallmark of intelligence.

I don't deny there is a certain level of 'magic' that makes the LLMs actually work. But at it's core it's still a probability matrix. Maybe the matrix doesn't work well when it's quanitized, and while i mostly work in the realm of creative and RPing, having it give very bad answers that breaks logic and orientation or being too predictable breaks the illusion of intelligence.

A podcaster I listened to said 'AI/LLMs are very smart, until they aren't', meaning beyond a certain constrained size of context they start making stupid and major mistakes. Maybe adding more thinking that grabs from different portions of context fix it (at the cost of even more processing), maybe more training fixes it (until it's biased or wrong and then you can't fix it without fully retraining it), Maybe they only make 1% mistakes (although previously they only had a success rate of about 6%, and roll the 1% enough times and a billion dollar company can crash it's entire database or decide the best course of action is to bulldoze the servers). Maybe it transcribes meetings well (but 20% of the time there's injected conversations or missing comments making the whole transcription suspect and needing to be triple checked for accuracy).

When i say they are prediction engines it is literally making a list of the most likely next words. Some words are always going to be filler words, and main concepts to move it forward. Let's not forget in some tests they asked how many legs an ant has, then injected into the middle of it's thinking from ants to spiders, then the model answered 8 legs, rather than saying 'wait a minute, i mean to say ant' and fixing itself. Instead it steamrolled ahead with inferred information from spiders rather than ants.

For now let's see what may come. I doubt Astra is AGI. But it may be a very very good model regardless.

edit: Thinking about it, for it to be actually intelligent it would have to not only do the previous two things i mentioned (do things without mistakes and not hallucinate) but also argue when the user is wrong, rather than 'oh you are right my bad' replies. I've only ever gotten 1 reply with from a model justifying it's answer rather than immediately apologizing.

I'll admit I'm skeptical of that claim (of AGI's arrival); Yes larger models in recent months/years have made leaps forward. But along with it the sheer hardware required to even run them.

But LLMs are still (last i checked) prediction engines rather than having true intelligence. And RPing I've had some very good results in the 24B-70B range.

My bar would be they'd be expected to perform common tasks perfectly (and probably a lot of uncommon tasks) without hallucinations. Also being able to be run on local hardware at a decent speed would be a big boon.

I figured we might get some dedicated bots to play specific games. I figured they'd also be pretty tiny because they'd be specialized. Looks pretty promising.

The sheer amount of storage and bandwidth... can't be cheap. ROM sites to keep up i heard can be a few thousand a month and those are insignificant in size comparison to AI models.

I consider AI models to be effectively be lazy programming; Throwing slop at the wall and some stuff might stick, brute forcing it taking hundreds of thousands of iterations before it is anywhere near useful. Then to get a better model you do it again... Which for larger and larger models costs millions or billions of dollars in compute power.

Add to that it's very slow compared to something hand-coded using an interpreted language or compiled. Though there's a lot of things a trained model can do faster/better than a human, or at least have the endless patience to reroll something until you get something useful.

Personally I'm on the fence of if AI is a good thing or not: Depends on if it offers more good use to us (lowering the bar to make movies books and remove busywork) or if it's more a hindrance (Youtube censoring elbows and closing hundreds of channels daily with no recourse to fix it, surveillance and Flock cameras flagging people who end up getting arrested because it read a plate wrong, or Sony planning to monitor voice chat and then banning your account if you say a naughty word)

You must be very young, because the GPT-2 that people were hyping as Skynet was a 120-1500M model.

i'm actually middle aged; However i didn't notice/consider ChatGPT til it really hit 3.5 or 4. And i didn't consider anything for AI generation (text or image) until i could actually download it, to which i started playing with it the last 2 years now. Before that it was just something they were panicking over but seeing how much it was barfing with suggesting glue to thicken pizza (fed from Reddit apparently) and supposed insiders who were saying it was sentient while it apparently isn't. shrugs

The point of Pebble isn't really to make tiny models that can compete with big LLMs. A big part of why people work on SLMs is to see how much capability you can actually get out of a very small model, and to experiment with different architectures, training methods, and ways of making models more efficient.

Understood, and those are fine. I just haven't seen anything suggesting SLM were very useful though they are likely specific usecases rather than general purpose; Though the idea of training one to play chess/checkers/go/etc competently enough did come to mind, i just kinda doubted that would be the usecase.

I rather enjoy creative writing and RPing moreso, which a more focused SLM might be better at. Not sure. I'll happily keep an eye out to see what comes out though.