fix: enforce allowed_models during inference requests #4197
+126
−4
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
The `allowed_models` configuration was only filtering the model list endpoint but not enforcing restrictions during actual inference requests. This allowed users to bypass the restriction by directly requesting models not in the allowed list, potentially accessing expensive models when only cheaper ones were intended. This change adds validation to all inference methods (`openai_chat_completion`, `openai_completion`, `openai_embeddings`) to reject requests for disallowed models with a clear error message. **Implementation:** - Added `_validate_model_allowed()` helper method that checks if a model is in the `allowed_models` list - Called validation in all three inference methods before making API requests - Validation occurs after resolving the provider model ID to ensure consistency **Test Plan:** - Added unit tests verifying all inference methods respect `allowed_models` - Tests cover allowed models (success), disallowed models (rejection), and no restrictions (None allows all, empty list blocks all) - All existing tests continue to pass Fixes GHSA-5rjj-4jp6-fw39
ashwinb
requested review from
bbrowning,
ehhuang,
franciscojavierarceo,
hardikjshah,
leseb,
mattf,
raghotham,
reluctantfuturist,
slekkala1 and
yanxi0830
as code owners
meta-cla
bot
added
the
CLA Signed
Contributor
Author
|
cc @derekhiggins this is different than your proposed patch btw because it hooks in inside |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The
allowed_modelsconfiguration was only being applied when listing models via the/v1/modelsendpoint, but the actual inference requests weren't checking this restriction. This meant users could directly request any model the provider supports by specifying it in their inference call, completely bypassing the intended cost controls.The fix adds validation to all three inference methods (chat completions, completions, and embeddings) that checks the requested model against the allowed_models list before making the provider API call.
Test plan
Added unit tests