Bump kv-cache-manager to v0.4.0-rc2

[pierDipi]

Bump + fix breaking changes

[vMaroon]

There will be a v0.4.0 release sometime today - this PR can import it.

[nirrozenbaum]

/hold until kv-cache manager v0.4 is released.

[pierDipi]

yeah, it's WIP for that reason, is this a known flake?

  STEP: Waiting for pods of qwen-qwen2-5-1-5b-instruct-vllm-sim to be ready @ 11/20/25 08:59:30.392
panic: test timed out after 10m0s
	running tests:
		TestEndToEnd (10m0s)

[vMaroon]

https://github.com/llm-d/llm-d-kv-cache-manager/releases/tag/v0.4.0-rc1 was released, it requires some changes in addition to the version bump. I tagged you in the relevant PR, this one is mandatory: https://github.com/llm-d/llm-d-kv-cache-manager/pull/150/files#r2547769214, 2nd comment is optional.

Do you want to handle? An alternative can be merging this and handling separately.

yeah, it's WIP for that reason, is this a known flake?

  STEP: Waiting for pods of qwen-qwen2-5-1-5b-instruct-vllm-sim to be ready @ 11/20/25 08:59:30.392
panic: test timed out after 10m0s
	running tests:
		TestEndToEnd (10m0s)

This is not common - weird.

[pierDipi]

It should be fixed now

[pierDipi]

/cc @vMaroon @nirrozenbaum

[nirrozenbaum]

/hold cancel
/cc @vMaroon

[nirrozenbaum]

these lines look like internal implementation details of kvcache manager code.
can we move this to kvcache repo?
I'd expect the call indexerConfig, err := kvcache.NewDefaultConfig() in L45 to initialize that internally.

[vMaroon]

The kvcache library defines a configuration with default values. This code piece is that of a user, permitting configuration through the env-var HF_TOKEN. It is arguable that this is a special env-var that is widely accepted but generally in the kvcache library we attempt to contain all configuration in the referenced structure, leaving customized UX to the users.

[nirrozenbaum]

right. it's a user defined env var (with the user's token).
HF_TOKEN as env var is very acceptable.

the question I was trying to answer is why do we read that env var here and not in kvcache code.
this part looks very not natural, the scorer factory function writes to an internal config of the kvcache indexer parameters.
I was expecting to have the os.getenv call inside kvcache.NewDefaultConfig().

[pierDipi]

Since this discussion is applicable to the current setup, should we discuss it separately?

[pierDipi]

I agree that kv cache manager library can do this since the use can still opt-out from the default env-var injection by not using the NewDefaultConfig function if they want to have different defaults

[vMaroon]

Let's track this in a new issue, I think this should not be a blocker.

[nirrozenbaum]

not a blocker 👍

[vMaroon]

/lgtm
/approve