CLOUDP-317663: Archive snippets outputs instead of comitting (#478)

# Summary

With this PR we will no longer commit snippets outputs back to the repo
and just archive them to s3 instead.

Additionally, `scripts/dev/update_docs_snippets.sh` scripts was updated.
But automated snippets updates from CI runs are not part of this PR.

**When reviewing collapse changes to tests/output directory** - all
those files (previously comitted outputs) were just deleted.
<img width="500" alt="image"
src="https://github.com/user-attachments/assets/ec5bb172-425c-4545-996c-7a01c60a9a0b"
/>


## Proof of Work

**Archiving outputs to s3**
[manual evg
run](https://spruce.mongodb.com/version/69038706b0fce50007f25a9d/tasks?sorts=STATUS%3AASC%3BBASE_STATUS%3ADESC)

**locally executed update script**
```
$ MCK_DIR=$(pwd) MCK_BRANCH=lsierant/archive-snippets-outputs DOCS_DIR=~/mdb/docs-mongodb-internal OUTPUTS_VERSION_ID=69038706b0fce50007f25a9d scripts/dev/update_docs_snippets.sh
```

```
~/mdb/lsierant ~/mdb/lsierant/lsierant_archive-snippets-outputs
~/mdb/docs-mongodb-internal ~/mdb/lsierant ~/mdb/lsierant/lsierant_archive-snippets-outputs
Switched to branch 'main'
Your branch is up to date with 'origin/main'.
HEAD is now at 308db2bd35 DOCSP-54633 MongoDB Search product name change-main-backport (15252) (#15805)
fatal: a branch named 'MCK-snippets-update-69038706b0fce50007f25a9d' already exists
Switched to branch 'MCK-snippets-update-69038706b0fce50007f25a9d'
~/mdb/lsierant ~/mdb/lsierant/lsierant_archive-snippets-outputs
Downloading snippets outputs from s3 to /var/folders/_l/xlckyyss4nsf1hd_0wf1pt3r0000gp/T/tmp.f9z7UeJ2Te
download: s3://operator-e2e-artifacts/snippets_outputs/69038706b0fce50007f25a9d/snippets_outputs_69038706b0fce50007f25a9d_test_kind_search_community_snippets.tgz to ../../../../var/folders/_l/xlckyyss4nsf1hd_0wf1pt3r0000gp/T/tmp.f9z7UeJ2Te/69038706b0fce50007f25a9d/snippets_outputs_69038706b0fce50007f25a9d_test_kind_search_community_snippets.tgz
download: s3://operator-e2e-artifacts/snippets_outputs/69038706b0fce50007f25a9d/snippets_outputs_69038706b0fce50007f25a9d_test_kind_search_external_mongod_snippets.tgz to ../../../../var/folders/_l/xlckyyss4nsf1hd_0wf1pt3r0000gp/T/tmp.f9z7UeJ2Te/69038706b0fce50007f25a9d/snippets_outputs_69038706b0fce50007f25a9d_test_kind_search_external_mongod_snippets.tgz
download: s3://operator-e2e-artifacts/snippets_outputs/69038706b0fce50007f25a9d/snippets_outputs_69038706b0fce50007f25a9d_test_kind_search_enterprise_snippets.tgz to ../../../../var/folders/_l/xlckyyss4nsf1hd_0wf1pt3r0000gp/T/tmp.f9z7UeJ2Te/69038706b0fce50007f25a9d/snippets_outputs_69038706b0fce50007f25a9d_test_kind_search_enterprise_snippets.tgz
x scripts/code_snippets/tests/outputs/test_kind_search_community_snippets/
x scripts/code_snippets/tests/outputs/test_kind_search_community_snippets/01_0090_helm_add_mogodb_repo.out
x scripts/code_snippets/tests/outputs/test_kind_search_community_snippets/01_0100_install_operator.out
x scripts/code_snippets/tests/outputs/test_kind_search_community_snippets/01_0110_wait_for_operator_deployment.out
x scripts/code_snippets/tests/outputs/test_kind_search_community_snippets/01_0315_wait_for_community_resource.out
x scripts/code_snippets/tests/outputs/test_kind_search_community_snippets/01_0330_wait_for_community_resource.out
x scripts/code_snippets/tests/outputs/test_kind_search_community_snippets/01_0335_show_running_pods.out
x scripts/code_snippets/tests/outputs/test_kind_search_community_snippets/03_0420_import_movies_mflix_database.out
x scripts/code_snippets/tests/outputs/test_kind_search_community_snippets/03_0440_wait_for_search_index_ready.out
x scripts/code_snippets/tests/outputs/test_kind_search_community_snippets/03_0444_list_search_indexes.out
x scripts/code_snippets/tests/outputs/test_kind_search_community_snippets/03_0445_list_vector_search_indexes.out
x scripts/code_snippets/tests/outputs/test_kind_search_community_snippets/03_0450_execute_search_query.out
x scripts/code_snippets/tests/outputs/test_kind_search_community_snippets/03_0455_execute_vector_search_query.out
x scripts/code_snippets/tests/outputs/test_kind_search_enterprise_snippets/
x scripts/code_snippets/tests/outputs/test_kind_search_enterprise_snippets/02_0090_helm_add_mogodb_repo.out
x scripts/code_snippets/tests/outputs/test_kind_search_enterprise_snippets/02_0100_install_operator.out
x scripts/code_snippets/tests/outputs/test_kind_search_enterprise_snippets/02_0310_wait_for_database_resource.out
x scripts/code_snippets/tests/outputs/test_kind_search_enterprise_snippets/02_0335_show_running_pods.out
x scripts/code_snippets/tests/outputs/test_kind_search_enterprise_snippets/03_0420_import_movies_mflix_database.out
x scripts/code_snippets/tests/outputs/test_kind_search_enterprise_snippets/03_0440_wait_for_search_index_ready.out
x scripts/code_snippets/tests/outputs/test_kind_search_enterprise_snippets/03_0444_list_search_indexes.out
x scripts/code_snippets/tests/outputs/test_kind_search_enterprise_snippets/03_0445_list_vector_search_indexes.out
x scripts/code_snippets/tests/outputs/test_kind_search_enterprise_snippets/03_0450_execute_search_query.out
x scripts/code_snippets/tests/outputs/test_kind_search_enterprise_snippets/03_0455_execute_vector_search_query.out
x scripts/code_snippets/tests/outputs/test_kind_search_external_mongod_snippets/
x scripts/code_snippets/tests/outputs/test_kind_search_external_mongod_snippets/04_0090_helm_add_mogodb_repo.out
x scripts/code_snippets/tests/outputs/test_kind_search_external_mongod_snippets/04_0100_install_operator.out
x scripts/code_snippets/tests/outputs/test_kind_search_external_mongod_snippets/04_0315_wait_for_community_resource.out
x scripts/code_snippets/tests/outputs/test_kind_search_external_mongod_snippets/04_0330_wait_for_community_resource.out
x scripts/code_snippets/tests/outputs/test_kind_search_external_mongod_snippets/04_0335_show_running_pods.out
x scripts/code_snippets/tests/outputs/test_kind_search_external_mongod_snippets/03_0420_import_movies_mflix_database.out
x scripts/code_snippets/tests/outputs/test_kind_search_external_mongod_snippets/03_0440_wait_for_search_index_ready.out
x scripts/code_snippets/tests/outputs/test_kind_search_external_mongod_snippets/03_0444_list_search_indexes.out
x scripts/code_snippets/tests/outputs/test_kind_search_external_mongod_snippets/03_0445_list_vector_search_indexes.out
x scripts/code_snippets/tests/outputs/test_kind_search_external_mongod_snippets/03_0450_execute_search_query.out
x scripts/code_snippets/tests/outputs/test_kind_search_external_mongod_snippets/03_0455_execute_vector_search_query.out
Replacing outputs for test: /var/folders/_l/xlckyyss4nsf1hd_0wf1pt3r0000gp/T/tmp.f9z7UeJ2Te/69038706b0fce50007f25a9d/scripts/code_snippets/tests/outputs/test_kind_search_community_snippets
Replacing outputs for test: /var/folders/_l/xlckyyss4nsf1hd_0wf1pt3r0000gp/T/tmp.f9z7UeJ2Te/69038706b0fce50007f25a9d/scripts/code_snippets/tests/outputs/test_kind_search_enterprise_snippets
Replacing outputs for test: /var/folders/_l/xlckyyss4nsf1hd_0wf1pt3r0000gp/T/tmp.f9z7UeJ2Te/69038706b0fce50007f25a9d/scripts/code_snippets/tests/outputs/test_kind_search_external_mongod_snippets
/var/folders/_l/xlckyyss4nsf1hd_0wf1pt3r0000gp/T/tmp.f9z7UeJ2Te/69038706b0fce50007f25a9d/scripts/code_snippets/tests/outputs
/var/folders/_l/xlckyyss4nsf1hd_0wf1pt3r0000gp/T/tmp.f9z7UeJ2Te/69038706b0fce50007f25a9d/scripts/code_snippets/tests/outputs
├── test_kind_search_community_snippets
│   ├── 01_0090_helm_add_mogodb_repo.out
│   ├── 01_0100_install_operator.out
│   ├── 01_0110_wait_for_operator_deployment.out
│   ├── 01_0315_wait_for_community_resource.out
│   ├── 01_0330_wait_for_community_resource.out
│   ├── 01_0335_show_running_pods.out
│   ├── 03_0420_import_movies_mflix_database.out
│   ├── 03_0440_wait_for_search_index_ready.out
│   ├── 03_0444_list_search_indexes.out
│   ├── 03_0445_list_vector_search_indexes.out
│   ├── 03_0450_execute_search_query.out
│   └── 03_0455_execute_vector_search_query.out
├── test_kind_search_enterprise_snippets
│   ├── 02_0090_helm_add_mogodb_repo.out
│   ├── 02_0100_install_operator.out
│   ├── 02_0310_wait_for_database_resource.out
│   ├── 02_0335_show_running_pods.out
│   ├── 03_0420_import_movies_mflix_database.out
│   ├── 03_0440_wait_for_search_index_ready.out
│   ├── 03_0444_list_search_indexes.out
│   ├── 03_0445_list_vector_search_indexes.out
│   ├── 03_0450_execute_search_query.out
│   └── 03_0455_execute_vector_search_query.out
└── test_kind_search_external_mongod_snippets
    ├── 03_0420_import_movies_mflix_database.out
    ├── 03_0440_wait_for_search_index_ready.out
    ├── 03_0444_list_search_indexes.out
    ├── 03_0445_list_vector_search_indexes.out
    ├── 03_0450_execute_search_query.out
    ├── 03_0455_execute_vector_search_query.out
    ├── 04_0090_helm_add_mogodb_repo.out
    ├── 04_0100_install_operator.out
    ├── 04_0315_wait_for_community_resource.out
    ├── 04_0330_wait_for_community_resource.out
    └── 04_0335_show_running_pods.out

4 directories, 33 files
~/mdb/docs-mongodb-internal /var/folders/_l/xlckyyss4nsf1hd_0wf1pt3r0000gp/T/tmp.f9z7UeJ2Te/69038706b0fce50007f25a9d ~/mdb/lsierant/lsierant_archive-snippets-outputs
[MCK-snippets-update-69038706b0fce50007f25a9d 5198ea9981] Update sample files from MCK
 18 files changed, 78 insertions(+), 623 deletions(-)
 delete mode 100644 content/kubernetes/upcoming/source/includes/code-examples/search/04-search-external-mongod/README.md
Enumerating objects: 57, done.
Counting objects: 100% (57/57), done.
Delta compression using up to 16 threads
Compressing objects: 100% (34/34), done.
Writing objects: 100% (34/34), 4.72 KiB | 4.72 MiB/s, done.
Total 34 (delta 30), reused 0 (delta 0), pack-reused 0
remote: Resolving deltas: 100% (30/30), completed with 21 local objects.
remote:
remote: Create a pull request for 'MCK-snippets-update-69038706b0fce50007f25a9d' on GitHub by visiting:
remote:      https://github.com/10gen/docs-mongodb-internal/pull/new/MCK-snippets-update-69038706b0fce50007f25a9d
remote:
remote: GitHub found 264 vulnerabilities on 10gen/docs-mongodb-internal's default branch (25 critical, 106 high, 86 moderate, 47 low). To find out more, visit:
remote:      https://github.com/10gen/docs-mongodb-internal/security/dependabot
remote:
To github.com:10gen/docs-mongodb-internal.git
 * [new branch]            MCK-snippets-update-69038706b0fce50007f25a9d -> MCK-snippets-update-69038706b0fce50007f25a9d
```

Branch was automatically pushed with all gathered snippets and outputs: 

https://github.com/10gen/docs-mongodb-internal/compare/MCK-snippets-update-69038706b0fce50007f25a9d?expand=1

Author: lsierant

.evergreen-functions.yml

@@ -49,6 +49,7 @@ variables:
       - READINESS_PROBE_VERSION
       - VERSION_UPGRADE_HOOK_VERSION
       - BUILD_SCENARIO
+      - MDB_BASH_DEBUG
 
 functions:
 
@@ -769,18 +770,14 @@ functions:
   #
   # Code snippet test automation
   #
-
-  sample_commit_output:
-    - command: github.generate_token
-      params:
-        expansion_name: GH_TOKEN
+  archive_snippets_output:
     - command: subprocess.exec
       params:
         include_expansions_in_env:
           - GH_TOKEN
-          - code_snippets_commit_output
+          - MDB_BASH_DEBUG
         working_dir: src/github.com/mongodb/mongodb-kubernetes
-        binary: scripts/code_snippets/sample_commit_output.sh
+        binary: scripts/code_snippets/archive_snippets_output.sh
 
   # it executes a script by convention: ./scripts/code_snippets/tests/${task_name}
   test_code_snippets:
@@ -793,6 +790,7 @@ functions:
           - code_snippets_teardown
           - code_snippets_reset
           - task_name
+          - MDB_BASH_DEBUG
         script: |
           ./scripts/code_snippets/tests/${task_name}
 

.evergreen-snippets.yml

@@ -69,33 +69,19 @@ functions:
         permissions: private
         visibility: signed
         content_type: text/plain
-    - command: s3.put
-      params:
-        aws_key: ${enterprise_aws_access_key_id}
-        aws_secret: ${enterprise_aws_secret_access_key}
-        local_files_include_filter:
-          - src/github.com/mongodb/mongodb-kubernetes/public/architectures/**/*.out
-          - src/github.com/mongodb/mongodb-kubernetes/docs/**/*.out
-        preserve_path: true
-        remote_file: logs/${task_id}/${execution}/
-        bucket: operator-e2e-artifacts
-        permissions: private
-        visibility: signed
-        content_type: text/plain
 
   upload_code_snippets_outputs:
     - command: s3.put
       params:
         aws_key: ${enterprise_aws_access_key_id}
         aws_secret: ${enterprise_aws_secret_access_key}
         local_files_include_filter:
-          - snippets_outputs.tgz
-        remote_file: logs/${task_id}/${execution}/
+          - snippets_outputs*.tgz
+        remote_file: snippets_outputs/${version_id}
         bucket: operator-e2e-artifacts
         permissions: private
         visibility: signed
         content_type: ${content_type|application/x-gzip}
-        display_name: "Snippets Outputs"
 
 tasks:
   # Code snippets tasks
@@ -106,31 +92,31 @@ tasks:
     tags: [ "code_snippets" ]
     commands:
       - func: test_code_snippets
-      - func: sample_commit_output
+      - func: archive_snippets_output
 
   - name: test_gke_multi_cluster_no_mesh_snippets.sh
     tags: [ "code_snippets" ]
     commands:
       - func: test_code_snippets
-      - func: sample_commit_output
+      - func: archive_snippets_output
 
   - name: test_kind_search_community_snippets.sh
     tags: [ "code_snippets", "patch-run" ]
     commands:
       - func: test_code_snippets
-      - func: sample_commit_output
+      - func: archive_snippets_output
 
   - name: test_kind_search_enterprise_snippets.sh
     tags: [ "code_snippets", "patch-run" ]
     commands:
       - func: test_code_snippets
-      - func: sample_commit_output
+      - func: archive_snippets_output
 
   - name: test_kind_search_external_mongod_snippets.sh
     tags: [ "code_snippets", "patch-run" ]
     commands:
       - func: test_code_snippets
-      - func: sample_commit_output
+      - func: archive_snippets_output
 
 task_groups:
   - name: gke_code_snippets_task_group

.evergreen-tasks.yml

@@ -66,25 +66,25 @@ tasks:
     tags: [ "code_snippets" ]
     commands:
       - func: test_code_snippets
-      - func: sample_commit_output
+      - func: archive_snippets_output
 
   - name: task_gke_multi_cluster_no_mesh_snippets
     tags: [ "code_snippets" ]
     commands:
       - func: test_code_snippets
-      - func: sample_commit_output
+      - func: archive_snippets_output
 
   - name: task_kind_search_community_snippets
     tags: [ "code_snippets", "patch-run" ]
     commands:
       - func: test_code_snippets
-      - func: sample_commit_output
+      - func: archive_snippets_output
 
   - name: task_kind_search_enterprise_snippets
     tags: [ "code_snippets", "patch-run" ]
     commands:
       - func: test_code_snippets
-      - func: sample_commit_output
+      - func: archive_snippets_output
 
 ## Below are only e2e runs for .evergreen.yml ##
 

.evergreen.yml

@@ -191,9 +191,9 @@ parameters:
     value: "false"
     description: set this to true if you would like to delete the resources created in the code snippet tests, but keep the clusters
 
-  - key: code_snippets_commit_output
-    value: "false"
-    description: set this to true if you would like the pipeline to automatically push a branch with updated snippets outputs
+  - key: MDB_BASH_DEBUG
+    value: "0"
+    description: set this to 1 if you want shell scripts to enable set -x
 
 # Triggered manually or by PCT.
 patch_aliases:

scripts/code_snippets/archive_snippets_output.sh

@@ -0,0 +1,25 @@
+#!/usr/bin/env bash
+
+set -eou pipefail
+test "${MDB_BASH_DEBUG:-0}" -eq 1 && set -x
+
+source scripts/dev/set_env_context.sh
+
+# task_name is available from evergreen expansions and its set to the snippets test name (see tasks in .evergreen-snippets.yml)
+test_name=${task_name:?}
+test_name=${test_name%.sh}
+patch_id=${version_id:?}
+
+# to run it locally:
+# $ test_name=test_kind_search_enterprise_snippets.sh archive_snippets_output.sh
+
+output_dir="scripts/code_snippets/tests/outputs/${test_name}"
+if [[ ! -d "${output_dir}" ]]; then
+  echo "Output dir is missing: ${output_dir}"
+  exit 1
+fi
+
+file_name="snippets_outputs_${patch_id}_${test_name}.tgz"
+tar -cvzf "${file_name}" "${output_dir}"
+
+echo "Collected snippets outputs from ${test_name} into ${file_name}"