Guid/gromov_GRIT

Gromov loss checkpoints trained on GRIT

Checkpoints for github.com/fbyrman/gromov: hyperbolic (Lorentz) vision-language models trained with the Gromov loss, and three hyperbolic baselines trained with the same data and schedule.

• Data: GRIT, ~13.4M image-caption pairs with grounded boxes. One box crop and its phrase are sampled per image per step.
• Training: 200,000 iterations at total_batch_size=512, AdamW (lr 5e-4, weight decay 0.2), 4,000-step linear warmup and cosine decay, mixed precision. Image and text encoders are trained from scratch.

Each checkpoint is stored as {"model", "iteration"} in fp32 with no optimizer state. manifest.json records each file's sha256. config.yaml is the full training config of the run. Pass it as --train-config.

Gromov loss models

checkpoint	model	supervision
grit_power_inter_k3_vit_s/model.pth	PowerGromovInter, power_k=3	captions
grit_power_inter_k3_vit_b/model.pth	PowerGromovInter, power_k=3	captions
grit_power_all_vit_s/model.pth	PowerGromovAll, power_k_inter=2, power_k_intra=3	captions and boxes
grit_power_all_vit_b/model.pth	PowerGromovAll, power_k_inter=2, power_k_intra=3	captions and boxes

Baselines

Hyperbolic baselines as implemented in the GitHub repo (gvlm/models.py). Only the dataset and the number of iterations differ from the repo's configs.

checkpoint	model	supervision
grit_hycoclip_vit_s/model.pth	HyCoCLIP (entailment cones)	captions and boxes
grit_hycoclip_vit_b/model.pth	HyCoCLIP (entailment cones)	captions and boxes
grit_phyclip_vit_s/model.pth	PHyCLIP (product of 64 hyperbolic spaces of dimension 8)	captions and boxes
grit_phyclip_vit_b/model.pth	PHyCLIP (product of 64 hyperbolic spaces of dimension 8)	captions and boxes
grit_argent_vit_s/model.pth	ARGENT (angle-based entailment loss with a regulariser)	captions and boxes
grit_argent_vit_b/model.pth	ARGENT (angle-based entailment loss with a regulariser)	captions and boxes

PHyCLIP supports only feature-level evaluation (configs/eval_zero_shot_classification.py, eval_zero_shot_retrieval.py, eval_hierarchical_metrics.py). The _score_level configs need a single hyperboloid and raise on it.

Use

From a clone of the GitHub repo:

hf download Guid/gromov_GRIT grit_power_all_vit_b/model.pth grit_power_all_vit_b/config.yaml \
    --local-dir checkpoints
python scripts/evaluate.py --config configs/eval_zero_shot_classification_score_level.py \
    --checkpoint-path checkpoints/grit_power_all_vit_b/model.pth \
    --train-config checkpoints/grit_power_all_vit_b/config.yaml \
    --results-file evaluate_score_level.json

The same works for any checkpoint above: change the folder name, and use configs/eval_zero_shot_classification.py for PHyCLIP.