Skip to content

Conversation

@yokofly
Copy link
Collaborator

@yokofly yokofly commented Nov 18, 2025

Save named collection to metastore

Add named configuration support in external table/stream configuration. The 'config_file' setting could be a named collection. An example is as below, external table does not find a file 'minio1' so it try to read it from named collection.
CREATE NAMED COLLECTION minio1 AS
    endpoint='http://minio:9000',
    access_key_id='minioroot',
    secret_access_key='minioroot';

CREATE EXTERNAL TABLE minio_ext (key string, value int32)
SETTINGS
    type='s3',
    named_collection='minio1',
    bucket='smoketest-0048-named-collection',
    write_to='00_basic/data.json';

Copy link

@chatgpt-codex-connector chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +52 to +56
NamedCollectionPtr NamedCollectionFactory::get(const std::string & collection_name) const
{
std::erase_if(
loaded_named_collections,
[&](const auto & value) { return value.second->getSourceId() == id; });
/// proton: starts
return metadata_storage->get(collection_name);
/// proton: ends

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Initialize metadata storage before named collection access

In NamedCollectionFactory::get the code dereferences metadata_storage directly (lines 52‑56), but the pointer is only initialized in loadIfNot (lines 138‑154) and the public paths such as InterpreterCreateNamedCollectionQuery::execute call createFromSQL without invoking loadIfNot. On a fresh server the first CREATE/ALTER/DROP NAMED COLLECTION or any config-based lookup will therefore hit this method with a null metadata_storage and crash. The factory methods should ensure metadata_storage is initialized (e.g., call loadIfNot) before use.

Useful? React with 👍 / 👎.

@yokofly yokofly merged commit 0c8b35a into develop Nov 18, 2025
20 checks passed
@yokofly yokofly deleted the store-named-collection-in-metastore branch November 18, 2025 06:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants