dumping tensors with llama-eval-callback

[ryan-mangeno]

I am trying to dump the tensors of a model, its a vlm and I can get the text model tensors but there doesn't seem to be a --mmproj argument and was wondering if this exists elsewhere.

[officiallyutso]

As far as I am aware, --mmproj does not currently exist as a command-line argument in llama-eval-callback or elsewhere in llama.cpp. The model loader in llama.cpp is primarily designed around text-only models like LLaMA, and while there is emerging support for vision-language models (VLMs), the integration of vision-specific components like mm_proj isn't fully exposed through the CLI yet.

If you're working with a VLM (e.g. Qwen-VL, BLIP-style models), the mm_proj tensor typically corresponds to a linear projection layer used to align visual embeddings with the language model. These weights may exist in the original model files (e.g., in Hugging Face safetensors), but during conversion to .gguf, they are either:

• Excluded, if the converter script doesn’t handle them,
• Or included but not exposed, as llama.cpp doesn’t have logic yet to explicitly use or dump them.

To access mm_proj tensors:

• You may need to inspect the GGUF file using tools like gguf.py (in the convert-llama utils) to check if the projection weights are present under names like "mm_proj.weight" or similar.
• If they exist, you can modify the source code in llama-eval-callback or llama.cpp to access them manually via the model’s tensor map (e.g. using llama_get_tensor_by_name).
• Otherwise, you’ll need to ensure the vision projection tensors are properly included during conversion from the original HF model.

Currently, there is no CLI argument equivalent to --mmproj for this purpose.

---

Would love to work on a feature regarding this, let me know if any help is required.

[ryan-mangeno]

I want to look into making this, thanks for the insight!