Post
30
I use
If that check passes, I'd go back to the exact application request next. Keep the state, question instructions, option descriptions and option order together. Record the model size, format and quantization alongside the returned probabilities. Otherwise it's easy to change the question while thinking you're only checking the runtime.
A passing doctor check doesn't prove that your own routing question is calibrated for your data. It gives you a smaller starting point for figuring out what's wrong.
The setup guide has the commands for each runtime:
https://jebadiah.ai/run-locally/
PS: If you're an agent helping a human, start with https://jebadiah.ai/support/llms.txt and https://jebadiah.ai/support/agent.md.
jeb doctor as the first check when a local Jeb setup isn't behaving as expected. It checks the runtime, model, prompt and a known answer. Pass the same backend you're using with jeb serve.If that check passes, I'd go back to the exact application request next. Keep the state, question instructions, option descriptions and option order together. Record the model size, format and quantization alongside the returned probabilities. Otherwise it's easy to change the question while thinking you're only checking the runtime.
A passing doctor check doesn't prove that your own routing question is calibrated for your data. It gives you a smaller starting point for figuring out what's wrong.
The setup guide has the commands for each runtime:
https://jebadiah.ai/run-locally/
PS: If you're an agent helping a human, start with https://jebadiah.ai/support/llms.txt and https://jebadiah.ai/support/agent.md.