Skip to content

Commit f0059f1

Browse files
anasatirbasaalextwoods
anasatirbasa
and
alextwoods
authored
Support to flatten a Map into top level attributes of the object (#6102)
Co-authored-by: Alex Woods <alexwoo@amazon.com>
1 parent a2c7f8e commit f0059f1

File tree

17 files changed

+1395
-48
lines changed

17 files changed

+1395
-48
lines changed

‎.changes/next-release/feature-AmazonDynamoDBEnhancedClient-22723fc.json‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
{
2+
"type": "feature",
3+
"category": "Amazon DynamoDB Enhanced Client",
4+
"contributor": "",
5+
"description": "Add support for @DynamoDbFlatten to flatten a Map<String, String> to top level attributes of an object"
6+
}

‎services-custom/dynamodb-enhanced/README.md‎

Lines changed: 55 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -613,7 +613,7 @@ private static final StaticTableSchema<Customer> CUSTOMER_TABLE_SCHEMA =
613613
```
614614
#### Using composition
615615

616-
Using composition, the @DynamoDbFlatten annotation flat maps the composite class:
616+
Using composition, the @DynamoDbFlatten annotation flat maps composite classes and Map attributes:
617617
```java
618618
@DynamoDbBean
619619
public class Customer {
@@ -640,10 +640,40 @@ public class GenericRecord {
640640
public void setCreatedDate(String createdDate) { this.createdDate = createdDate;}
641641
}
642642
```
643-
You can flatten as many different eligible classes as you like using the flatten annotation.
643+
644+
You can also apply the @DynamoDbFlatten annotation to flatten a Map into top-level attributes:
645+
```java
646+
@DynamoDbBean
647+
public class Customer {
648+
private String name;
649+
private String city;
650+
private String address;
651+
private Map<String, String> detailsMap;
652+
653+
public String getName() { return this.name; }
654+
public void setName(String name) { this.name = name;}
655+
public String getCity() { return this.city; }
656+
public void setCity(String city) { this.city = city;}
657+
public String getAddress() { return this.address; }
658+
public void setAddress(String address) { this.address = address;}
659+
660+
@DynamoDbFlatten
661+
public Map<String, String> getDetailsMap() { return this.detailsMap; }
662+
public void setDetailsMap(Map<String, String> detailsMap) { this.detailsMap = detailsMap;}
663+
}
664+
```
665+
666+
**Object Flattening**: You can flatten as many different eligible classes as you like using the flatten annotation.
644667
The only constraints are that attributes must not have the same name when they are being rolled
645668
together, and there must never be more than one partition key, sort key or table name.
646669

670+
**Map Flattening**:
671+
- A record can contain at most one `@DynamoDbFlatten` on a Map property. This limit applies across the entire class hierarchy, including any composed or flattened classes.
672+
- The flattened map must use `String` as both the key and value type (`Map<String, String>`). Other key or value types are not supported.
673+
- Attribute names generated from map keys must not conflict with existing attributes on the record. If a conflict is detected, an exception will be thrown.
674+
- If more than one flattened map is present, an exception will be thrown during schema creation.
675+
- Other annotations like `@DynamoDbUpdateBehavior` are not supported on flattened maps, consistent with the existing object flattening behavior.
676+
647677
Flat map composite classes using StaticTableSchema:
648678

649679
```java
@@ -685,3 +715,26 @@ private static final StaticTableSchema<Customer> CUSTOMER_TABLE_SCHEMA =
685715
```
686716
Just as for annotations, you can flatten as many different eligible classes as you like using the
687717
builder pattern.
718+
719+
For map flattening using StaticTableSchema:
720+
721+
```java
722+
@Data
723+
public class Customer {
724+
private String name;
725+
private String city;
726+
private String address;
727+
private Map<String, String> detailsMap;
728+
//getters and setters for all attributes
729+
}
730+
731+
private static final StaticTableSchema<Customer> CUSTOMER_TABLE_SCHEMA =
732+
StaticTableSchema.builder(Customer.class)
733+
.newItemSupplier(Customer::new)
734+
.addAttribute(String.class, a -> a.name("name")
735+
.getter(Customer::getName)
736+
.setter(Customer::setName))
737+
// Because we are flattening a Map object, we supply a getter and setter so the mapper knows how to access it
738+
.flattenMap(Customer::getDetailsMap, Customer::setDetailsMap)
739+
.build();
740+
```

‎services-custom/dynamodb-enhanced/src/it/java/software/amazon/awssdk/enhanced/dynamodb/ScanQueryWithFlattenMapIntegrationTest.java‎

Lines changed: 170 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,170 @@
1+
/*
2+
* Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
3+
*
4+
* Licensed under the Apache License, Version 2.0 (the "License").
5+
* You may not use this file except in compliance with the License.
6+
* A copy of the License is located at
7+
*
8+
* http://aws.amazon.com/apache2.0
9+
*
10+
* or in the "license" file accompanying this file. This file is distributed
11+
* on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either
12+
* express or implied. See the License for the specific language governing
13+
* permissions and limitations under the License.
14+
*/
15+
16+
package software.amazon.awssdk.enhanced.dynamodb;
17+
18+
import static org.hamcrest.MatcherAssert.assertThat;
19+
import static org.hamcrest.Matchers.equalTo;
20+
import static org.hamcrest.Matchers.is;
21+
import static org.hamcrest.Matchers.nullValue;
22+
import static software.amazon.awssdk.enhanced.dynamodb.internal.AttributeValues.numberValue;
23+
import static software.amazon.awssdk.enhanced.dynamodb.internal.AttributeValues.stringValue;
24+
import static software.amazon.awssdk.enhanced.dynamodb.mapper.StaticAttributeTags.primaryPartitionKey;
25+
import static software.amazon.awssdk.enhanced.dynamodb.mapper.StaticAttributeTags.primarySortKey;
26+
import static software.amazon.awssdk.enhanced.dynamodb.mapper.StaticAttributeTags.secondaryPartitionKey;
27+
import static software.amazon.awssdk.enhanced.dynamodb.mapper.StaticAttributeTags.secondarySortKey;
28+
import static software.amazon.awssdk.enhanced.dynamodb.model.QueryConditional.sortBetween;
29+
30+
import java.util.Collections;
31+
import java.util.HashMap;
32+
import java.util.Iterator;
33+
import java.util.List;
34+
import java.util.Map;
35+
import java.util.stream.Collectors;
36+
import java.util.stream.IntStream;
37+
import org.junit.AfterClass;
38+
import org.junit.BeforeClass;
39+
import org.junit.Test;
40+
import software.amazon.awssdk.enhanced.dynamodb.mapper.StaticTableSchema;
41+
import software.amazon.awssdk.enhanced.dynamodb.model.Page;
42+
import software.amazon.awssdk.enhanced.dynamodb.model.QueryEnhancedRequest;
43+
import software.amazon.awssdk.enhanced.dynamodb.model.Record;
44+
import software.amazon.awssdk.services.dynamodb.DynamoDbClient;
45+
import software.amazon.awssdk.services.dynamodb.model.AttributeValue;
46+
47+
public class ScanQueryWithFlattenMapIntegrationTest extends DynamoDbEnhancedIntegrationTestBase {
48+
49+
private static final String TABLE_NAME = createTestTableName();
50+
51+
private static final TableSchema<Record> RECORD_WITH_FLATTEN_MAP_TABLE_SCHEMA =
52+
StaticTableSchema.builder(Record.class)
53+
.newItemSupplier(Record::new)
54+
.addAttribute(String.class, a -> a.name("id")
55+
.getter(Record::getId)
56+
.setter(Record::setId)
57+
.tags(primaryPartitionKey(), secondaryPartitionKey("index1")))
58+
.addAttribute(Integer.class, a -> a.name("sort")
59+
.getter(Record::getSort)
60+
.setter(Record::setSort)
61+
.tags(primarySortKey(), secondarySortKey("index1")))
62+
.addAttribute(Integer.class, a -> a.name("value")
63+
.getter(Record::getValue)
64+
.setter(Record::setValue))
65+
.addAttribute(String.class, a -> a.name("gsi_id")
66+
.getter(Record::getGsiId)
67+
.setter(Record::setGsiId)
68+
.tags(secondaryPartitionKey("gsi_keys_only")))
69+
.addAttribute(Integer.class, a -> a.name("gsi_sort")
70+
.getter(Record::getGsiSort)
71+
.setter(Record::setGsiSort)
72+
.tags(secondarySortKey("gsi_keys_only")))
73+
.addAttribute(String.class, a -> a.name("stringAttribute")
74+
.getter(Record::getStringAttribute)
75+
.setter(Record::setStringAttribute))
76+
.flatten("attributesMap",
77+
Record::getAttributesMap,
78+
Record::setAttributesMap)
79+
.build();
80+
81+
private static final List<Record> RECORDS_WITH_FLATTEN_MAP =
82+
IntStream.range(0, 9)
83+
.mapToObj(i -> new Record()
84+
.setId("id-value")
85+
.setSort(i)
86+
.setValue(i)
87+
.setStringAttribute(getStringAttrValue(10 * 1024))
88+
.setGsiId("gsi-id-value")
89+
.setGsiSort(i)
90+
.setAttributesMap(new HashMap<String, String>() {{
91+
put("mapAttribute1", "mapValue1");
92+
put("mapAttribute2", "mapValue2");
93+
put("mapAttribute3", "mapValue3");
94+
}}))
95+
.collect(Collectors.toList());
96+
97+
private static DynamoDbClient dynamoDbClient;
98+
private static DynamoDbEnhancedClient enhancedClient;
99+
private static DynamoDbTable<Record> mappedTable;
100+
101+
@BeforeClass
102+
public static void setup() {
103+
dynamoDbClient = createDynamoDbClient();
104+
enhancedClient = DynamoDbEnhancedClient.builder().dynamoDbClient(dynamoDbClient).build();
105+
mappedTable = enhancedClient.table(TABLE_NAME, RECORD_WITH_FLATTEN_MAP_TABLE_SCHEMA);
106+
mappedTable.createTable();
107+
dynamoDbClient.waiter().waitUntilTableExists(r -> r.tableName(TABLE_NAME));
108+
}
109+
110+
@AfterClass
111+
public static void teardown() {
112+
try {
113+
dynamoDbClient.deleteTable(r -> r.tableName(TABLE_NAME));
114+
} finally {
115+
dynamoDbClient.close();
116+
}
117+
}
118+
119+
private void insertRecords() {
120+
RECORDS_WITH_FLATTEN_MAP.forEach(record -> mappedTable.putItem(r -> r.item(record)));
121+
}
122+
123+
@Test
124+
public void queryWithFlattenMapRecord_correctlyRetrievesProjectedAttributes() {
125+
insertRecords();
126+
127+
Iterator<Page<Record>> results =
128+
mappedTable.query(QueryEnhancedRequest.builder()
129+
.queryConditional(sortBetween(k -> k.partitionValue("id-value").sortValue(2),
130+
k -> k.partitionValue("id-value").sortValue(6)))
131+
.attributesToProject("mapAttribute1", "mapAttribute2")
132+
.limit(3)
133+
.build())
134+
.iterator();
135+
136+
Page<Record> page1 = results.next();
137+
assertThat(results.hasNext(), is(true));
138+
Page<Record> page2 = results.next();
139+
assertThat(results.hasNext(), is(false));
140+
141+
Map<String, String> expectedAttributesMap = new HashMap<>();
142+
expectedAttributesMap.put("mapAttribute1", "mapValue1");
143+
expectedAttributesMap.put("mapAttribute2", "mapValue2");
144+
145+
List<Record> page1Items = page1.items();
146+
assertThat(page1Items.size(), is(3));
147+
assertThat(page1Items.get(0).getAttributesMap(), is(expectedAttributesMap));
148+
assertThat(page1Items.get(1).getAttributesMap(), is(expectedAttributesMap));
149+
assertThat(page1Items.get(2).getAttributesMap(), is(expectedAttributesMap));
150+
assertThat(page1.consumedCapacity(), is(nullValue()));
151+
assertThat(page1.lastEvaluatedKey(), is(getKeyMap(4)));
152+
assertThat(page1.count(), equalTo(3));
153+
assertThat(page1.scannedCount(), equalTo(3));
154+
155+
List<Record> page2Items = page2.items();
156+
assertThat(page2Items.size(), is(2));
157+
assertThat(page2Items.get(0).getAttributesMap(), is(expectedAttributesMap));
158+
assertThat(page2Items.get(1).getAttributesMap(), is(expectedAttributesMap));
159+
assertThat(page2.lastEvaluatedKey(), is(nullValue()));
160+
assertThat(page2.count(), equalTo(2));
161+
assertThat(page2.scannedCount(), equalTo(2));
162+
}
163+
164+
private Map<String, AttributeValue> getKeyMap(int sort) {
165+
Map<String, AttributeValue> result = new HashMap<>();
166+
result.put("id", stringValue(RECORDS.get(sort).getId()));
167+
result.put("sort", numberValue(RECORDS.get(sort).getSort()));
168+
return Collections.unmodifiableMap(result);
169+
}
170+
}

‎services-custom/dynamodb-enhanced/src/it/java/software/amazon/awssdk/enhanced/dynamodb/model/Record.java‎

Lines changed: 15 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -15,6 +15,7 @@
1515

1616
package software.amazon.awssdk.enhanced.dynamodb.model;
1717

18+
import java.util.Map;
1819
import java.util.Objects;
1920

2021
public class Record {
@@ -27,6 +28,8 @@ public class Record {
2728

2829
private String stringAttribute;
2930

31+
private Map<String, String> attributesMap;
32+
3033
public String getId() {
3134
return id;
3235
}
@@ -81,6 +84,15 @@ public Record setStringAttribute(String stringAttribute) {
8184
return this;
8285
}
8386

87+
public Map<String, String> getAttributesMap() {
88+
return attributesMap;
89+
}
90+
91+
public Record setAttributesMap(Map<String, String> attributesMap) {
92+
this.attributesMap = attributesMap;
93+
return this;
94+
}
95+
8496
@Override
8597
public boolean equals(Object o) {
8698
if (this == o) return true;
@@ -91,11 +103,12 @@ public boolean equals(Object o) {
91103
Objects.equals(value, record.value) &&
92104
Objects.equals(gsiId, record.gsiId) &&
93105
Objects.equals(stringAttribute, record.stringAttribute) &&
94-
Objects.equals(gsiSort, record.gsiSort);
106+
Objects.equals(gsiSort, record.gsiSort) &&
107+
Objects.equals(attributesMap, record.attributesMap);
95108
}
96109

97110
@Override
98111
public int hashCode() {
99-
return Objects.hash(id, sort, value, gsiId, gsiSort, stringAttribute);
112+
return Objects.hash(id, sort, value, gsiId, gsiSort, stringAttribute, attributesMap);
100113
}
101114
}

0 commit comments

Comments
 (0)