[Performance]: Option for disabling model info collection in subprocess

[ibl-g]

Proposal to improve performance

By default vLLM collects model support info in a single sub process per model
(added in in #9233). Specifically, this
https://github.com/vllm-project/vllm/blob/main/vllm/model_executor/models/registry.py#L336
_run_in_subprocess call.

This adds ~4s when running against local ssd and can easily be double or more
against a network filesystem in some environments. Collecting the info
in-process does not seem to have adverse effects, at least based on my limited
manual testing, but I lack context on why this was done in the first place.

Can we make this behaviour configurable via a boolean flag or env var? That way
users could opt out.

collect_model_info_via_subprocess = True

Something like

if self.model_config.collect_model_info_via_subprocess:
    return _run_in_subprocess(
        lambda: _ModelInfo.from_model_cls(self.load_model_cls()))
return _ModelInfo.from_model_cls(self.load_model_cls())

Show the latency in "inspect-model" span based on my local wip otel tracing of start up

CC @DarkLight1337

Report of performance regression

No response

Misc discussion on performance

No response

Your current environment (if you think it is necessary)

Before submitting a new issue...

• Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.

[DarkLight1337]

Some vLLM features (e.g. multi-modality and pooling model support) depend on model inspection.

A number of models will raise CUDA reinitialization error if the check isn't being done in a subprocess so definitely we can't disable this by default. I'm fine with making this optional though.

[chaunceyjiang]

If model inspection is not performed automatically, then when the task is set to "auto", it's impossible to obtain model-related information. However, when the task is explicitly specified, it seems that automatic detection doesn't occur, and the subprocess is likely skipped.

[ibl-g]

@DarkLight1337 thank you for the context!

@chaunceyjiang I tried with --task=generate --model=google/gemma-2-2b-it and still observed model inspection in subprocess. Was that what you were referring to?

[njhill]

Could we attempt to do the model inspection inline first, but monkey-patching cuda so that it won't attempt to initialize and instead will raise a specific error which we can catch and then fall back to the existing subprocess check?

[github-actions]

This issue has been automatically marked as stale because it has not had any activity within 90 days. It will be automatically closed if no further activity occurs within 30 days. Leave a comment if you feel this issue should remain open. Thank you!

[github-actions]

This issue has been automatically closed due to inactivity. Please feel free to reopen if you feel it is still relevant. Thank you!

[DarkLight1337]

Closing as completed by #23558