Add troubleshooting section and resolution for Elasticsearch/Elastic Cloud use with Migration Assistant capture proxy and replayer

Signed-off-by: Andre Kurait <akurait@amazon.com>

Author: AndreKurait

_migration-assistant/migration-phases/replay-captured-traffic.md

@@ -323,4 +323,34 @@ The following metrics are also reported:
 
 Metrics and dashboards pushed to CloudWatch may experience a visibility lag of around 5 minutes. CloudWatch also retains higher-resolution data for a shorter period than lower-resolution data. For more information, see [Amazon CloudWatch concepts](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/cloudwatch_concepts.html).
 
+## Troubleshooting
+
+### Elasticsearch content type and accept header compatibility
+
+Newer Elasticsearch clients (version 7.11 and later, including all 8.x versions) use Elasticsearch-specific media types in both `Content-Type` and `Accept` headers. These clients may send headers such as:
+
+- `Content-Type: application/vnd.elasticsearch+json;compatible-with=8`
+- `Accept: application/vnd.elasticsearch+json;compatible-with=8`
+
+When migrating to OpenSearch or another service that does not support these Elasticsearch-specific media types, requests from these clients may fail or be rejected by the target cluster.
+
+**Important**: If you are using Elasticsearch clients version 7.11 or later and migrating to OpenSearch or a service that does not recognize `application/vnd.elasticsearch+json` media types, you need to apply a transformation to convert both the `Content-Type` and `Accept` headers to the standard `application/json` format.
+{: .note}
+
+To resolve this issue, configure Traffic Replayer with a transformation that converts the Elasticsearch-specific media types to the standard `application/json` format. Add the following transformation to your Traffic Replayer configuration using the `--transformer-config` option:
+
+```json
+[
+  {
+    "JsonJSTransformerProvider": {
+      "initializationScript": "const NEW_CONTENT_TYPE = \"application/json\";\nconst ELASTIC_CONTENT_TYPE = \"application/vnd.elasticsearch+json\";\n\nfunction transform(request, context) {\n  let headers = request.get(\"headers\");\n  if (headers) {\n    let contentType = headers.get(\"Content-Type\");\n    if (Array.isArray(contentType)) {\n      headers.set(\"Content-Type\", contentType.map(v => v.includes(ELASTIC_CONTENT_TYPE) ? NEW_CONTENT_TYPE : v));\n    } else if (typeof contentType === \"string\") {\n      if (contentType.includes(ELASTIC_CONTENT_TYPE)) {\n        headers.set(\"Content-Type\", NEW_CONTENT_TYPE);\n      }\n    }\n    let accept = headers.get(\"Accept\");\n    if (Array.isArray(accept)) {\n      headers.set(\"Accept\", accept.map(v => v.includes(ELASTIC_CONTENT_TYPE) ? NEW_CONTENT_TYPE : v));\n    } else if (typeof accept === \"string\") {\n      if (accept.includes(ELASTIC_CONTENT_TYPE)) {\n        headers.set(\"Accept\", NEW_CONTENT_TYPE);\n      }\n    }\n  }\n  return request;\n}\n\nfunction main(context) {\n  return (request) => {\n    if (Array.isArray(request)) {\n      return request.flat().map(item => transform(item, context));\n    }\n    return transform(request, context);\n  };\n}\n(() => main)();",
+      "bindingsObject": "{}"
+    }
+  }
+]
+```
+{% include copy.html %}
+
+This transformation script automatically detects Elasticsearch-specific media types in both `Content-Type` and `Accept` headers (including those with version parameters like `compatible-with=8`) and replaces them with the standard `application/json` format, ensuring compatibility with OpenSearch and other services that do not support the Elasticsearch-specific media types.
+
 {% include migration-phase-navigation.html %}

_migration-assistant/migration-phases/reroute-source-to-proxy.md

@@ -63,29 +63,48 @@ Note the records in the logging topic.
 
 After a short period, re-execute the same command again and compare the increased number of records against the expected HTTP requests.
 
-## Backfilling documents to the source cluster
+## Troubleshooting
 
-From your source cluster snapshot, you can begin backfilling documents into the target cluster. Once you have started this process, a fleet of workers will spin up to read the snapshot and reindex documents into the target cluster. This fleet of workers can be scaled to increase the speed at which documents are reindexed into the target cluster.
+### Host header routing configuration
 
-### Checking the starting state of the clusters
+Some systems, such as Elastic Cloud and other hosted Elasticsearch services, use the Host header for routing traffic to the appropriate cluster. When using the Capture Proxy with these systems, you need to configure the proxy to set the Host header to match your source cluster's domain.
 
-You can check the indexes and document counts of the source and target clusters by running the `cat-indices` command. This can be used to monitor the difference between the source and target for any migration scenario. Check the indexes of both clusters using the following command:
+**Important**: This configuration is required for Elastic Cloud deployments and any system that uses Host header-based routing. Without this setting, requests will fail with an error response like `{"ok":false,"message":"Unknown resource."}` on Elastic Cloud or be misrouted on other systems.
+{: .note}
+
+To configure the Host header, add the `captureProxyExtraArgs` parameter to your `cdk.context.json` file:
+
+```json
+{
+  "captureProxyExtraArgs": "--setHeader Host <domain-host-without-protocol>"
+}
+```
+{% include copy.html %}
 
-```shell
-console clusters cat-indices
+For example, if your Elastic Cloud domain is `https://my-cluster.es.us-east-1.aws.found.io`, you would configure:
+
+```json
+{
+  "captureProxyExtraArgs": "--setHeader Host my-cluster.es.us-east-1.aws.found.io"
+}
 ```
 {% include copy.html %}
 
-You should receive the following response:
+**Tip**: The Host header value should include only the domain name without the protocol (https://) or port number.
+{: .tip}
+
+#### Validating the configuration
 
-```shell
-SOURCE CLUSTER
-health status index       uuid                   pri rep docs.count docs.deleted store.size pri.store.size
-green  open   my-index WJPVdHNyQ1KMKol84Cy72Q   1   0          8            0     44.7kb         44.7kb
+Before routing production traffic through the Capture Proxy, validate that the proxy is correctly configured by sending test requests directly to it. You can use `curl` to verify the connection:
 
-TARGET CLUSTER
-health status index                        uuid                   pri rep docs.count docs.deleted store.size pri.store.size
-green  open   .opendistro_security         N3uy88FGT9eAO7FTbLqqqA   1   0         10            0     78.3kb         78.3kb
+```bash
+curl -k https://<capture-proxy-endpoint>:9200/
 ```
+{% include copy.html %}
+
+If the Host header configuration is correct, you should receive a successful or authentication failure response from your source cluster. If you receive an error like `{"ok":false,"message":"Unknown resource."}`, verify that:
+- The `captureProxyExtraArgs` parameter is correctly set in your `cdk.context.json`
+- The Host header value matches your source cluster's domain exactly
+- You have redeployed the Capture Proxy service after making configuration changes
 
 {% include migration-phase-navigation.html %}