Instructions to use FreedomIntelligence/ALLaVA-3B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use FreedomIntelligence/ALLaVA-3B with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="FreedomIntelligence/ALLaVA-3B", trust_remote_code=True)# pip install -U transformers accelerate # Load model directly from transformers import AutoModelForCausalLM model = AutoModelForCausalLM.from_pretrained("FreedomIntelligence/ALLaVA-3B", trust_remote_code=True, device_map="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use FreedomIntelligence/ALLaVA-3B with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "FreedomIntelligence/ALLaVA-3B" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "FreedomIntelligence/ALLaVA-3B", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker
docker model run hf.co/FreedomIntelligence/ALLaVA-3B
- SGLang
How to use FreedomIntelligence/ALLaVA-3B with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "FreedomIntelligence/ALLaVA-3B" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "FreedomIntelligence/ALLaVA-3B", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "FreedomIntelligence/ALLaVA-3B" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "FreedomIntelligence/ALLaVA-3B", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }' - Docker Model Runner
How to use FreedomIntelligence/ALLaVA-3B with Docker Model Runner:
docker model run hf.co/FreedomIntelligence/ALLaVA-3B
Commit History
-
Update README.md
Verified signatureab6266a -
Upload training_datasets_by_stage.jpg
Verified signature3d1bc7a -
Update README.md
Verified signature09705e0
-
Update README.md
Verified signature31438a3 -
Update README.md
Verified signature1b2b1ff -
Update README.md
Verified signature4003a9b -
Update README.md
Verified signature80b3dac -
upload model.safetensors.index.json
Verified signature29e61bf -
upload merges.txt
Verified signature28880e4 -
upload configuration_phi.py
Verified signatureedaab35 -
upload tokenizer.json
Verified signature4c3a94b -
upload modeling_phi.py
Verified signature6ff0f13 -
upload added_tokens.json
Verified signature0777182 -
upload vocab.json
Verified signature9547e98 -
upload llava_arch.py
Verified signaturefa9a6b9 -
upload config.json
Verified signature2e82300 -
upload tokenizer_config.json
Verified signature1a113eb -
upload trainer_state.json
Verified signature0ff3693 -
upload training_args.bin
Verified signaturef0dea82 -
upload generation_config.json
Verified signaturef48beb0 -
upload special_tokens_map.json
Verified signatureb6d06fc -
upload generation_utils.py
Verified signaturef073d79 -
upload modeling_llava_phi.py
Verified signature42bd287 -
Delete constants.py with huggingface_hub
Verified signature570e8c5 -
upload model.safetensors.index.json
Verified signature1bbfd18 -
upload merges.txt
Verified signaturedc32a15 -
upload configuration_phi.py
Verified signature82bfac7 -
upload tokenizer.json
Verified signatureb440d28 -
upload modeling_phi.py
Verified signaturea411f3f -
upload added_tokens.json
Verified signature7b7dc8c -
upload vocab.json
Verified signatureb18cb5b -
upload llava_arch.py
Verified signature3770dba -
upload config.json
Verified signaturece27692 -
upload tokenizer_config.json
Verified signature7a91fcf -
upload trainer_state.json
Verified signature448fbcc -
upload training_args.bin
Verified signature11425c9 -
upload constants.py
Verified signaturec0eb2ae -
upload generation_config.json
Verified signature0207deb -
upload special_tokens_map.json
Verified signature384e473 -
upload generation_utils.py
Verified signature16c3fd6 -
upload modeling_llava_phi.py
Verified signature971fc91
-
upload model-00001-of-00002.safetensors
Verified signaturef185b0e -
upload model.safetensors.index.json
Verified signatured4b32e7 -
upload merges.txt
Verified signature54dfad2 -
upload tokenizer.json
Verified signature8eac67e -
upload model-00002-of-00002.safetensors
Verified signature8846580 -
upload added_tokens.json
Verified signature7635984 -
upload vocab.json
Verified signatured429507 -
upload config.json
Verified signaturecceba91 -
upload tokenizer_config.json
Verified signature6ffba71