Skip to content

Commit 9555415

Browse files
api-clients-generation-pipeline[bot]ci.datadog-api-spec
andauthored
Flatten file_metadata response schema to avoid OneOf unmarshaling issues (#2955)
Co-authored-by: ci.datadog-api-spec <packages@datadoghq.com>
1 parent 1b2d5ed commit 9555415

File tree

9 files changed

+198
-274
lines changed

9 files changed

+198
-274
lines changed

.generator/schemas/v2/openapi.yaml

Lines changed: 79 additions & 52 deletions
Original file line numberDiff line numberDiff line change
@@ -13142,30 +13142,6 @@ components:
1314213142
CreateTableRequest:
1314313143
description: Request body for creating a new reference table from a local file
1314413144
or cloud storage.
13145-
example:
13146-
data:
13147-
attributes:
13148-
description: this is a cloud table generated via a cloud bucket sync
13149-
file_metadata:
13150-
access_details:
13151-
aws_detail:
13152-
aws_account_id: test-account-id
13153-
aws_bucket_name: test-bucket
13154-
file_path: test_rt.csv
13155-
sync_enabled: true
13156-
schema:
13157-
fields:
13158-
- name: name
13159-
type: STRING
13160-
- name: account_id
13161-
type: STRING
13162-
primary_keys:
13163-
- account_id
13164-
source: S3
13165-
table_name: test_reference_table
13166-
tags:
13167-
- test_tag
13168-
type: reference_table
1316913145
properties:
1317013146
data:
1317113147
$ref: '#/components/schemas/CreateTableRequestData'
@@ -13387,6 +13363,17 @@ components:
1338713363
CreateUploadRequest:
1338813364
description: Request to create an upload for a file to be ingested into a reference
1338913365
table.
13366+
example:
13367+
data:
13368+
attributes:
13369+
headers:
13370+
- product_id
13371+
- product_name
13372+
- price
13373+
part_count: 3
13374+
part_size: 10000000
13375+
table_name: my_products_table
13376+
type: upload
1339013377
properties:
1339113378
data:
1339213379
$ref: '#/components/schemas/CreateUploadRequestData'
@@ -51089,32 +51076,35 @@ components:
5108951076
type: string
5109051077
type: object
5109151078
TableResultV2DataAttributesFileMetadata:
51092-
description: Metadata specifying where and how to access the reference table's
51093-
data file.
51094-
oneOf:
51095-
- $ref: '#/components/schemas/TableResultV2DataAttributesFileMetadataCloudStorage'
51096-
- $ref: '#/components/schemas/TableResultV2DataAttributesFileMetadataLocalFile'
51097-
TableResultV2DataAttributesFileMetadataCloudStorage:
5109851079
additionalProperties: false
51099-
description: File metadata for reference tables created by cloud storage.
51080+
description: "Metadata specifying where and how to access the reference table's
51081+
data file.\n\nFor cloud storage tables (S3/GCS/Azure):\n - sync_enabled and
51082+
access_details will always be present\n - error fields (error_message, error_row_count,
51083+
error_type) are present only when errors occur\n\nFor local file tables:\n
51084+
\ - error fields (error_message, error_row_count) are present only when errors
51085+
occur\n - sync_enabled, access_details are never present"
5110051086
properties:
5110151087
access_details:
5110251088
$ref: '#/components/schemas/TableResultV2DataAttributesFileMetadataOneOfAccessDetails'
51089+
description: Cloud storage access configuration. Only present for cloud
51090+
storage sources (S3, GCS, Azure).
5110351091
error_message:
51104-
description: The error message returned from the sync.
51092+
description: The error message returned from the last operation (sync for
51093+
cloud storage, upload for local file).
5110551094
type: string
5110651095
error_row_count:
51107-
description: The number of rows that failed to sync.
51096+
description: The number of rows that failed to process.
5110851097
format: int64
5110951098
type: integer
5111051099
error_type:
5111151100
$ref: '#/components/schemas/TableResultV2DataAttributesFileMetadataCloudStorageErrorType'
51101+
description: The type of error that occurred during file processing. Only
51102+
applicable for cloud storage sources.
5111251103
sync_enabled:
51113-
description: Whether this table is synced automatically.
51104+
description: Whether this table is synced automatically from cloud storage.
51105+
Only applicable for cloud storage sources.
5111451106
type: boolean
51115-
required:
51116-
- access_details
51117-
title: CloudFileMetadataV2
51107+
title: FileMetadataV2
5111851108
type: object
5111951109
TableResultV2DataAttributesFileMetadataCloudStorageErrorType:
5112051110
description: The type of error that occurred during file processing. This field
@@ -51141,21 +51131,6 @@ components:
5114151131
- STATE_ERROR
5114251132
- OPERATION_ERROR
5114351133
- SYSTEM_ERROR
51144-
TableResultV2DataAttributesFileMetadataLocalFile:
51145-
additionalProperties: false
51146-
description: File metadata for reference tables created by upload. Note that
51147-
upload_id is only returned in the immediate create/replace response and is
51148-
not available in subsequent GET requests.
51149-
properties:
51150-
error_message:
51151-
description: The error message returned from the creation/update.
51152-
type: string
51153-
error_row_count:
51154-
description: The number of rows that failed to create/update.
51155-
format: int64
51156-
type: integer
51157-
title: LocalFileMetadataV2
51158-
type: object
5115951134
TableResultV2DataAttributesFileMetadataOneOfAccessDetails:
5116051135
description: Cloud storage access configuration for the reference table data
5116151136
file.
@@ -73511,6 +73486,58 @@ paths:
7351173486
requestBody:
7351273487
content:
7351373488
application/json:
73489+
examples:
73490+
cloud_storage:
73491+
summary: Create table from cloud storage (S3)
73492+
value:
73493+
data:
73494+
attributes:
73495+
description: Customer reference data synced from S3
73496+
file_metadata:
73497+
access_details:
73498+
aws_detail:
73499+
aws_account_id: '924305315327'
73500+
aws_bucket_name: my-data-bucket
73501+
file_path: customers.csv
73502+
sync_enabled: true
73503+
schema:
73504+
fields:
73505+
- name: customer_id
73506+
type: STRING
73507+
- name: customer_name
73508+
type: STRING
73509+
- name: email
73510+
type: STRING
73511+
primary_keys:
73512+
- customer_id
73513+
source: S3
73514+
table_name: customer_reference_data
73515+
tags:
73516+
- team:data-platform
73517+
type: reference_table
73518+
local_file:
73519+
summary: Create table from local file upload
73520+
value:
73521+
data:
73522+
attributes:
73523+
description: Product catalog uploaded via local file
73524+
file_metadata:
73525+
upload_id: 00000000-0000-0000-0000-000000000000
73526+
schema:
73527+
fields:
73528+
- name: product_id
73529+
type: STRING
73530+
- name: product_name
73531+
type: STRING
73532+
- name: price
73533+
type: DOUBLE
73534+
primary_keys:
73535+
- product_id
73536+
source: LOCAL_FILE
73537+
table_name: product_catalog
73538+
tags:
73539+
- team:ecommerce
73540+
type: reference_table
7351473541
schema:
7351573542
$ref: '#/components/schemas/CreateTableRequest'
7351673543
required: true

docs/datadog_api_client.v2.model.rst

Lines changed: 0 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -22537,27 +22537,13 @@ datadog\_api\_client.v2.model.table\_result\_v2\_data\_attributes\_file\_metadat
2253722537
:members:
2253822538
:show-inheritance:
2253922539

22540-
datadog\_api\_client.v2.model.table\_result\_v2\_data\_attributes\_file\_metadata\_cloud\_storage module
22541-
--------------------------------------------------------------------------------------------------------
22542-
22543-
.. automodule:: datadog_api_client.v2.model.table_result_v2_data_attributes_file_metadata_cloud_storage
22544-
:members:
22545-
:show-inheritance:
22546-
2254722540
datadog\_api\_client.v2.model.table\_result\_v2\_data\_attributes\_file\_metadata\_cloud\_storage\_error\_type module
2254822541
---------------------------------------------------------------------------------------------------------------------
2254922542

2255022543
.. automodule:: datadog_api_client.v2.model.table_result_v2_data_attributes_file_metadata_cloud_storage_error_type
2255122544
:members:
2255222545
:show-inheritance:
2255322546

22554-
datadog\_api\_client.v2.model.table\_result\_v2\_data\_attributes\_file\_metadata\_local\_file module
22555-
-----------------------------------------------------------------------------------------------------
22556-
22557-
.. automodule:: datadog_api_client.v2.model.table_result_v2_data_attributes_file_metadata_local_file
22558-
:members:
22559-
:show-inheritance:
22560-
2256122547
datadog\_api\_client.v2.model.table\_result\_v2\_data\_attributes\_file\_metadata\_one\_of\_access\_details module
2256222548
------------------------------------------------------------------------------------------------------------------
2256322549

examples/v2/reference-tables/CreateReferenceTable.py

Lines changed: 29 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -16,6 +16,12 @@
1616
from datadog_api_client.v2.model.create_table_request_data_attributes_file_metadata_one_of_access_details_aws_detail import (
1717
CreateTableRequestDataAttributesFileMetadataOneOfAccessDetailsAwsDetail,
1818
)
19+
from datadog_api_client.v2.model.create_table_request_data_attributes_file_metadata_one_of_access_details_azure_detail import (
20+
CreateTableRequestDataAttributesFileMetadataOneOfAccessDetailsAzureDetail,
21+
)
22+
from datadog_api_client.v2.model.create_table_request_data_attributes_file_metadata_one_of_access_details_gcp_detail import (
23+
CreateTableRequestDataAttributesFileMetadataOneOfAccessDetailsGcpDetail,
24+
)
1925
from datadog_api_client.v2.model.create_table_request_data_attributes_schema import (
2026
CreateTableRequestDataAttributesSchema,
2127
)
@@ -29,36 +35,45 @@
2935
body = CreateTableRequest(
3036
data=CreateTableRequestData(
3137
attributes=CreateTableRequestDataAttributes(
32-
description="this is a cloud table generated via a cloud bucket sync",
3338
file_metadata=CreateTableRequestDataAttributesFileMetadataCloudStorage(
3439
access_details=CreateTableRequestDataAttributesFileMetadataOneOfAccessDetails(
3540
aws_detail=CreateTableRequestDataAttributesFileMetadataOneOfAccessDetailsAwsDetail(
36-
aws_account_id="test-account-id",
37-
aws_bucket_name="test-bucket",
38-
file_path="test_rt.csv",
41+
aws_account_id="123456789000",
42+
aws_bucket_name="example-data-bucket",
43+
file_path="reference-tables/users.csv",
44+
),
45+
azure_detail=CreateTableRequestDataAttributesFileMetadataOneOfAccessDetailsAzureDetail(
46+
azure_client_id="aaaaaaaa-1111-2222-3333-bbbbbbbbbbbb",
47+
azure_container_name="reference-data",
48+
azure_storage_account_name="examplestorageaccount",
49+
azure_tenant_id="cccccccc-4444-5555-6666-dddddddddddd",
50+
file_path="tables/users.csv",
51+
),
52+
gcp_detail=CreateTableRequestDataAttributesFileMetadataOneOfAccessDetailsGcpDetail(
53+
file_path="data/reference_tables/users.csv",
54+
gcp_bucket_name="example-data-bucket",
55+
gcp_project_id="example-gcp-project-12345",
56+
gcp_service_account_email="example-service@example-gcp-project-12345.iam.gserviceaccount.com",
3957
),
4058
),
41-
sync_enabled=True,
59+
sync_enabled=False,
4260
),
4361
schema=CreateTableRequestDataAttributesSchema(
4462
fields=[
4563
CreateTableRequestDataAttributesSchemaFieldsItems(
46-
name="name",
47-
type=ReferenceTableSchemaFieldType.STRING,
48-
),
49-
CreateTableRequestDataAttributesSchemaFieldsItems(
50-
name="account_id",
64+
name="field_1",
5165
type=ReferenceTableSchemaFieldType.STRING,
5266
),
5367
],
5468
primary_keys=[
55-
"account_id",
69+
"field_1",
5670
],
5771
),
58-
source=ReferenceTableCreateSourceType.S3,
59-
table_name="test_reference_table",
72+
source=ReferenceTableCreateSourceType.LOCAL_FILE,
73+
table_name="table_1",
6074
tags=[
61-
"test_tag",
75+
"tag_1",
76+
"tag_2",
6277
],
6378
),
6479
type=CreateTableRequestDataType.REFERENCE_TABLE,

src/datadog_api_client/v2/model/table_result_v2_data_attributes.py

Lines changed: 11 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -19,12 +19,6 @@
1919
)
2020
from datadog_api_client.v2.model.table_result_v2_data_attributes_schema import TableResultV2DataAttributesSchema
2121
from datadog_api_client.v2.model.reference_table_source_type import ReferenceTableSourceType
22-
from datadog_api_client.v2.model.table_result_v2_data_attributes_file_metadata_cloud_storage import (
23-
TableResultV2DataAttributesFileMetadataCloudStorage,
24-
)
25-
from datadog_api_client.v2.model.table_result_v2_data_attributes_file_metadata_local_file import (
26-
TableResultV2DataAttributesFileMetadataLocalFile,
27-
)
2822

2923

3024
class TableResultV2DataAttributes(ModelNormal):
@@ -68,12 +62,7 @@ def __init__(
6862
self_,
6963
created_by: Union[str, UnsetType] = unset,
7064
description: Union[str, UnsetType] = unset,
71-
file_metadata: Union[
72-
TableResultV2DataAttributesFileMetadata,
73-
TableResultV2DataAttributesFileMetadataCloudStorage,
74-
TableResultV2DataAttributesFileMetadataLocalFile,
75-
UnsetType,
76-
] = unset,
65+
file_metadata: Union[TableResultV2DataAttributesFileMetadata, UnsetType] = unset,
7766
last_updated_by: Union[str, UnsetType] = unset,
7867
row_count: Union[int, UnsetType] = unset,
7968
schema: Union[TableResultV2DataAttributesSchema, UnsetType] = unset,
@@ -94,6 +83,16 @@ def __init__(
9483
:type description: str, optional
9584
9685
:param file_metadata: Metadata specifying where and how to access the reference table's data file.
86+
87+
For cloud storage tables (S3/GCS/Azure):
88+
89+
* sync_enabled and access_details will always be present
90+
* error fields (error_message, error_row_count, error_type) are present only when errors occur
91+
92+
For local file tables:
93+
94+
* error fields (error_message, error_row_count) are present only when errors occur
95+
* sync_enabled, access_details are never present
9796
:type file_metadata: TableResultV2DataAttributesFileMetadata, optional
9897
9998
:param last_updated_by: UUID of the user who last updated the reference table.

0 commit comments

Comments
 (0)