Merge branch 'develop' v0.3.8

Author: Bob Strahan

1751146101381_classification_state.json

@@ -7,8 +7,61 @@ SPDX-License-Identifier: MIT-0
 
 ### Added
 
+## [0.3.8]
+
+### Added
+
+- **Lending Package Configuration Support for Pattern-2**
+  - Added new `lending-package-sample` configuration to Pattern-2, providing comprehensive support for lending and financial document processing workflows
+  - New default configuration for Pattern-2 stack deployments, optimized for loan applications, mortgage processing, and financial verification documents
+  - Previous `rvl-cdip-sample` configuration remains available by selecting `rvl-cdip-package-sample` for the `Pattern2Configuration` parameter when deploying or updating stacks
+
+- **Text Confidence View for Document Pages**
+  - Added support for displaying OCR text confidence data through new `TextConfidenceUri` field
+  - New "Text Confidence View" option in the UI pages panel alongside existing Markdown and Text views
+  - Fixed issues with view persistence - Text Confidence View button now always visible with appropriate messaging when content unavailable
+  - Fixed view toggle behavior - switching between views no longer closes the viewer window
+  - Reordered view buttons to: Markdown View, Text Confidence View, Text View for better user experience
+
+- **Enhanced OCR DPI Configuration for PDF files**
+  - DPI for PDF image conversion is now configurable in the configuration editor under OCR image processing settings
+  - Default DPI improved from 96 to 150 DPI for better default quality and OCR accuracy
+  - Configurable through Web UI without requiring code changes or redeployment
+
+### Changed
+
+- **Converted text confidence data format from JSON to markdown table for improved readability and reduced token usage**
+  - Removed unnecessary "page_count" field
+  - Changed "text_blocks" array to "text" field containing a markdown table with Text and Confidence columns
+  - Reduces prompt size for assessment service while improving UI readability
+  - OCR confidence values now rounded to 1 decimal point (e.g., 99.1, 87.3) for cleaner display
+  - Markdown table headers now explicitly left-aligned using `|:-----|:-----------|` format for consistent appearance
+
+- **Simplified OCR Service Initialization**
+  - OCR service now accepts a single `config` dictionary parameter for cleaner, more consistent API
+  - Aligned with classification service pattern for better consistency across IDP services
+  - Backward compatibility maintained - old parameter pattern still supported with deprecation warning
+  - Updated all lambda functions and notebooks to use new simplified pattern
+- Removed fixed image target_height and target_width from default configurations, so images are processed in original resolution by default.
+
+- **Updated Default Configuration for Pattern1 and Pattern2**
+  - Changed default configuration for new stacks from "default" to "lending-package-sample" for both Pattern1 and Pattern2
+  - Maintains backward compatibility for stack updates by keeping the parameter value "default" mapped to the rvl-cdip-sample for pattern-2.
+
+- **Reduce assessment step costs**
+  - Default model for granular assessment is now `us.amazon.nova-lite-v1:0` - experimentation recommended
+  - Improved placement of <<CACHEPOINT>> tags in assessment prompt to improve utilization of prompt caching
+
 ### Fixed
 
+- **Fixed Image Resizing Behavior for High-Resolution Documents**
+  - Fixed issue where empty strings in image configuration were incorrectly resizing images to default 951x1268 pixels instead of preserving original resolution
+  - Empty strings (`""`) in `target_width` and `target_height` configuration now preserve original document resolution for maximum processing accuracy
+- Fixed issue where PNG files were being unnecessarily converted to JPEG format and resized to lower resolution with lost quality
+- Fixed issue where PNG and JPG image files were not rendering inline in the Document Details page
+- Fixed issue where PDF files were being downloaded instead of displayed inline
+- Fixed pricing data for cacheWrite tokens for Amazon Nova models to resolve innacurate cost estimation in UI.
+
 
 ## [0.3.7]
 

CHANGELOG.md

@@ -73,8 +73,8 @@ After deployment, you can quickly process a document and view results:
    - **Via S3**: Upload directly to the S3 input bucket (find the bucket URL in CloudFormation stack Outputs)
 
 2. **Use Sample Documents**:
-   - For Pattern 1 (BDA): Use [samples/lending_package.pdf](./samples/lending_package.pdf)
-   - For Patterns 2 and 3: Use [samples/rvl_cdip_package.pdf](./samples/rvl_cdip_package.pdf)
+   - For Patterns 1 (BDA) and Pattern 2: Use [samples/lending_package.pdf](./samples/lending_package.pdf)
+   - For Pattern 3 (UDOP): Use [samples/rvl_cdip_package.pdf](./samples/rvl_cdip_package.pdf)
 
 3. **Monitor Processing**:
    - **Via Web UI**: Track document status on the dashboard
@@ -105,8 +105,8 @@ To update an existing GenAIIDP stack to a new version:
 7. For detailed instructions, see the [Deployment Guide](./docs/deployment.md#updating-an-existing-stack)
 
 For testing, use these sample files:
-- Pattern-1 BDA default project: `samples/lending_package.pdf`
-- Patterns 2 and 3 default configurations: `samples/rvl_cdip_package.pdf`
+   - For Patterns 1 (BDA) and Pattern 2: Use [samples/lending_package.pdf](./samples/lending_package.pdf)
+   - For Pattern 3 (UDOP): Use [samples/rvl_cdip_package.pdf](./samples/rvl_cdip_package.pdf)
 
 For detailed deployment and testing instructions, see the [Deployment Guide](./docs/deployment.md).
 

README.md

@@ -1 +1 @@
-0.3.7
+0.3.8

VERSION

@@ -27,4 +27,4 @@ See the main [README.md](../README.md) for more detailed instructions on creatin
 
 ## Available Configurations
 
-Currently, only the default configuration is available for Pattern 1. Contributions are welcome!
+Currently, only the default lending-package-sample configuration is available for Pattern 1. Contributions are welcome!

config_library/pattern-1/README.md

@@ -1,7 +1,7 @@
 Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
 SPDX-License-Identifier: MIT-0
 
-# Default Configuration
+# Default Configuration (lending-package-sample)
 
 This directory contains the default configuration for the GenAI IDP Accelerator. This configuration serves as the baseline for all document processing tasks and can be used as a starting point for creating custom configurations.
 

config_library/pattern-1/default/README.md renamed to config_library/pattern-1/lending-package-sample/README.md

@@ -121,7 +121,7 @@ pricing:
       - name: cacheReadInputTokens
         price: '1.5E-8'
       - name: cacheWriteInputTokens
-        price: '0'
+        price: '6.0E-8'
   - name: bedrock/us.amazon.nova-pro-v1:0
     units:
       - name: inputTokens
@@ -131,7 +131,7 @@ pricing:
       - name: cacheReadInputTokens
         price: '2.0E-7'
       - name: cacheWriteInputTokens
-        price: '0'
+        price: '8.0E-7'
   - name: bedrock/us.amazon.nova-premier-v1:0
     units:
       - name: inputTokens

config_library/pattern-1/default/config.yaml renamed to config_library/pattern-1/lending-package-sample/config.yaml

@@ -10,8 +10,9 @@ ocr:
   features:
     - name: LAYOUT
   image:
-    target_width: '951'
-    target_height: '1268'
+    dpi: '150'
+    target_width: ''
+    target_height: ''
 classes:
   - name: Bank Statement
     description: Monthly bank account statement
@@ -68,8 +69,8 @@ classes:
         attributeType: list
 classification:
   image:
-    target_height: '1268'
-    target_width: '951'
+    target_height: ''
+    target_width: ''
   top_p: '0.1'
   max_tokens: '4096'
   top_k: '5'
@@ -210,8 +211,8 @@ classification:
   classificationMethod: textbasedHolisticClassification
 extraction:
   image:
-    target_height: '1268'
-    target_width: '951'
+    target_height: ''
+    target_width: ''
   top_p: '0.1'
   max_tokens: '10000'
   top_k: '5'
@@ -368,19 +369,19 @@ summarization:
     You are a document summarization expert who can analyze and summarize documents from various domains including medical, financial, legal, and general business documents. Your task is to create a summary that captures the key information, main points, and important details from the document. Your output must be in valid JSON format. \nSummarization Style: Balanced\\nCreate a balanced summary that provides a moderate level of detail. Include the main points and key supporting information, while maintaining the document's overall structure. Aim for a comprehensive yet concise summary.\n Your output MUST be in valid JSON format with markdown content. You MUST strictly adhere to the output format specified in the instructions.
 assessment:
   image:
-    target_height: '1268'
-    target_width: '951'
+    target_height: ''
+    target_width: ''
   granular:
     enabled: true
     max_workers: "20"
     simple_batch_size: "3"
     list_batch_size: "1"
-  default_confidence_threshold: '0.9'
+  default_confidence_threshold: '0.8'
   top_p: '0.1'
   max_tokens: '10000'
   top_k: '5'
   temperature: '0.0'
-  model: us.anthropic.claude-3-7-sonnet-20250219-v1:0
+  model: us.amazon.nova-pro-v1:0
   system_prompt: >-
     You are a document analysis assessment expert. Your task is to evaluate the confidence of extraction results by analyzing the source document evidence. Respond only with JSON containing confidence scores for each extracted attribute.
   task_prompt: >-
@@ -578,7 +579,7 @@ pricing:
       - name: cacheReadInputTokens
         price: '1.5E-8'
       - name: cacheWriteInputTokens
-        price: '0'
+        price: '6.0E-8'
   - name: bedrock/us.amazon.nova-pro-v1:0
     units:
       - name: inputTokens
@@ -588,7 +589,7 @@ pricing:
       - name: cacheReadInputTokens
         price: '2.0E-7'
       - name: cacheWriteInputTokens
-        price: '0'
+        price: '8.0E-7'
   - name: bedrock/us.amazon.nova-premier-v1:0
     units:
       - name: inputTokens