|
1 | | -# elasticsearch-curator-aws-lambda |
| 1 | +# Elasticsearch Curator Lambda |
| 2 | + |
| 3 | +## Overview |
2 | 4 | This repository contains the manifest for the elasticsearch curator lamdba. |
| 5 | + |
| 6 | +## Details |
| 7 | + |
| 8 | +1. Build and package this project by following these commands, so that it deployed on lambda: |
| 9 | + |
| 10 | +```bash |
| 11 | + |
| 12 | +sudo pip3 install virtualenv |
| 13 | + |
| 14 | +# i have used python3.7, other versions can also be used that are supported by lambda |
| 15 | +virtualenv -p python3.7 v-env |
| 16 | + |
| 17 | +# activate the virtual environment |
| 18 | +source v-env/bin/activate |
| 19 | + |
| 20 | + |
| 21 | +# install packages |
| 22 | +pip3 install -r requirements.txt |
| 23 | + |
| 24 | +# deactivate the virtual environment |
| 25 | +deactivate |
| 26 | + |
| 27 | +# move inside virtual environment's packages section |
| 28 | +cd v-env/lib/<python-version>/site-packages |
| 29 | + |
| 30 | +# archive the library contents |
| 31 | +zip -r9 ${OLDPWD}/elasticsearch-curator-lambda.zip . |
| 32 | + |
| 33 | +# move back to the directory |
| 34 | +cd $OLDPWD |
| 35 | + |
| 36 | +# add the lambda function code to the archive |
| 37 | +zip -g elasticsearch-curator-lambda.zip lambda_function.py |
| 38 | + |
| 39 | +``` |
| 40 | + |
| 41 | +2. Upload the archive save it, there are two ways to do it |
| 42 | + |
| 43 | + 1. direct upload(not recommended) |
| 44 | + 2. upload to s3 and specify the path in the lambda. |
| 45 | + 3. [zappa](https://github.com/Miserlou/Zappa) |
| 46 | + |
| 47 | +3. Following environment variables can be configured on AWS Lambda: |
| 48 | + |
| 49 | +| Environment Variable | Description | Default Value | |
| 50 | +|---|---|---| |
| 51 | +| ES_HOST | Elasticsearch URL without `https//` | localhost | |
| 52 | +| ES_RESION | Elasticsearch cluster reason | us-west-1 | |
| 53 | +| ES_INDICES_DATA_RETENTION_DAYS_THRESHOLD | Data retention period in days. | 90 | |
| 54 | +| ES_INDICES_PREFIX_TO_BE_DELETED | Indices prefix, it will be used as **`first`** filter, it will filter all the indices that have this prefix in their name. The remaining indices name will be filtered based on the age filter ES_INDICES_DATE_FORMAT. The reason to do these two filters is that we don't want to delete | logs | |
| 55 | +| ES_INDICES_DATE_FORMAT | Date format in the indices name. Indices will be filtered if their date is older than ES_INDICES_DATA_RETENTION_DAYS_THRESHOLD | %Y.%m.%d | |
0 commit comments