Skip to content

Commit 3f7c13b

Browse files
liudmylaruspavlusievajdesantomrDzurbqiuosier
authored
Setup unit tests (#118)
Co-authored-by: Oleksandra Pavlusieva <oleksandra.pavlusieva@gmail.com> Co-authored-by: John DeSanto <202220+jdesanto@users.noreply.github.com> Co-authored-by: Dmitrii Cherkasov <dmitrii.cherkasov@oracle.com> Co-authored-by: Qiu Qin <qiu.qin@oracle.com> Co-authored-by: Ziqun Ye <ziqun.ye@oracle.com> Co-authored-by: Lu Peng <bolu.peng@oracle.com> Co-authored-by: Lu Peng <118394507+lu-ohai@users.noreply.github.com> Co-authored-by: MING KANG <ming.kang@oracle.com> Co-authored-by: Daren Race <daren.race@gmail.com> Co-authored-by: Mayoor Rao <mayoor.rao@oracle.com> Co-authored-by: Mayoor Rao <mayoor.rao@gmail.com>
1 parent 31b04e8 commit 3f7c13b

File tree

237 files changed

+40584
-70
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

237 files changed

+40584
-70
lines changed

‎.coveragerc‎

Lines changed: 45 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,45 @@
1+
[paths]
2+
source =
3+
ads/
4+
5+
[run]
6+
branch = true
7+
parallel = true
8+
source =
9+
ads
10+
omit =
11+
docs/*
12+
tests/*
13+
ads/automl/*
14+
ads/common/artifact/*
15+
ads/environment/*
16+
ads/experiments/*
17+
ads/templates/*
18+
*/constants.py
19+
*/_imports.py
20+
*/__init__.py
21+
*/__main__.py
22+
*/.*
23+
24+
[report]
25+
exclude_lines =
26+
pragma: no cover
27+
def __repr__
28+
if __name__ == .__main__.:
29+
omit =
30+
ads/automl/*
31+
ads/common/artifact/*
32+
ads/environment/*
33+
ads/experiments/*
34+
ads/templates/*
35+
*/constants.py
36+
*/_imports.py
37+
*/__init__.py
38+
*/__main__.py
39+
__init.py__
40+
show_missing = true
41+
skip_empty = true
42+
precision = 2
43+
44+
[html]
45+
directory = htmlcov

‎.github/workflows/run-unittests-default_setup.yml‎

Lines changed: 48 additions & 26 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,18 @@
1-
name: Unit Tests unitary/default_setup/*
1+
name: tests/unitary/default_setup/**
22

33
on:
4-
workflow_dispatch:
4+
push:
5+
branches:
6+
- main
7+
- 'release/**'
8+
- develop
9+
paths:
10+
- 'ads/**'
11+
- setup.py
12+
- '**requirements.txt'
13+
- '!docs/**'
14+
15+
pull_request:
516

617
# Cancel in progress workflows on pull_requests.
718
# https://docs.github.com/en/actions/using-jobs/using-concurrency#example-using-a-fallback-value
@@ -12,6 +23,10 @@ concurrency:
1223
permissions:
1324
contents: read
1425

26+
# hack for https://github.com/actions/cache/issues/810#issuecomment-1222550359
27+
env:
28+
SEGMENT_DOWNLOAD_TIMEOUT_MINS: 5
29+
1530
jobs:
1631
test:
1732
name: python ${{ matrix.python-version }}, tests/unitary/default_setup
@@ -38,28 +53,35 @@ jobs:
3853
with:
3954
python-version: ${{ matrix.python-version }}
4055

41-
# - name: "Test config setup"
42-
# shell: bash
43-
# env:
44-
# ADS_OCI_CONFIG_REALLIKE: ${{ secrets.ADS_OCI_CONFIG_REALLIKE }}
45-
# ADS_OCI_PRIVATE_KEY_REALLIKE: ${{ secrets.ADS_OCI_PRIVATE_KEY_REALLIKE }}
46-
# run: |
47-
# set -x # print commands that are executed
48-
# mkdir -p /home/runner/.oci
49-
# ls -lha
50-
# echo "$ADS_OCI_CONFIG_REALLIKE" > "/home/runner/.oci/config"
51-
# echo "$ADS_OCI_PRIVATE_KEY_REALLIKE" > "/home/runner/.oci/oci_ads_user.pem"
52-
# echo "key_file=/home/runner/.oci/oci_ads_user.pem" >> "/home/runner/.oci/config"
56+
- name: "Test config setup"
57+
shell: bash
58+
env:
59+
HOME_RUNNER_DIR: /home/runner
60+
run: |
61+
set -x # print commands that are executed
62+
mkdir -p "$HOME_RUNNER_DIR"/.oci
63+
openssl genrsa -out $HOME_RUNNER_DIR/.oci/oci_ads_user.pem 2048
64+
cat <<EOT >> "$HOME_RUNNER_DIR/.oci/config"
65+
[DEFAULT]
66+
user=ocid1.user.oc1..xxx
67+
fingerprint=00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00
68+
tenancy=ocid1.tenancy.oc1..xxx
69+
region=test_region
70+
key_file=$HOME_RUNNER_DIR/.oci/oci_ads_user.pem
71+
EOT
72+
ls -lha "$HOME_RUNNER_DIR"/.oci
73+
echo "Test config file:"
74+
cat $HOME_RUNNER_DIR/.oci/config
5375
54-
# - name: "Run default_setup tests folder ONLY with minimum ADS dependencies"
55-
# timeout-minutes: 15
56-
# shell: bash
57-
# env:
58-
# NB_SESSION_COMPARTMENT_OCID: ocid1.compartment.oc1.<unique_ocid>
59-
# NoDependency: True
60-
# run: |
61-
# set -x # print commands that are executed
62-
# $CONDA/bin/conda init
63-
# source /home/runner/.bashrc
64-
# pip install -r test-requirements.txt
65-
# python -m pytest -v -p no:warnings --durations=5 tests/unitary/default_setup
76+
- name: "Run default_setup tests folder ONLY with minimum ADS dependencies"
77+
timeout-minutes: 15
78+
shell: bash
79+
env:
80+
NB_SESSION_COMPARTMENT_OCID: ocid1.compartment.oc1.<unique_ocid>
81+
NoDependency: True
82+
run: |
83+
set -x # print commands that are executed
84+
$CONDA/bin/conda init
85+
source /home/runner/.bashrc
86+
pip install -r test-requirements.txt
87+
python -m pytest -v -p no:warnings --durations=5 tests/unitary/default_setup

‎.github/workflows/run-unittests.yml‎

Lines changed: 59 additions & 43 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,18 @@
1-
name: Unit Tests unitary
1+
name: tests/unitary/**
22

33
on:
4-
workflow_dispatch:
4+
push:
5+
branches:
6+
- main
7+
- 'release/**'
8+
- develop
9+
paths:
10+
- 'ads/**'
11+
- setup.py
12+
- '**requirements.txt'
13+
- '!docs/**'
14+
15+
pull_request:
516

617
# Cancel in progress workflows on pull_requests.
718
# https://docs.github.com/en/actions/using-jobs/using-concurrency#example-using-a-fallback-value
@@ -12,7 +23,6 @@ concurrency:
1223
permissions:
1324
contents: read
1425

15-
1626
# hack for https://github.com/actions/cache/issues/810#issuecomment-1222550359
1727
env:
1828
SEGMENT_DOWNLOAD_TIMEOUT_MINS: 5
@@ -43,52 +53,58 @@ jobs:
4353
key: ${{ runner.os }}-pip-${{ hashFiles('**/dev-requirements.txt') }}
4454
restore-keys: |
4555
${{ runner.os }}-pip-
46-
4756
- uses: actions/setup-python@v4
4857
with:
4958
python-version: ${{ matrix.python-version }}
5059

51-
# - name: "Test config setup"
52-
# shell: bash
53-
# env:
54-
# ADS_OCI_CONFIG_REALLIKE: ${{ secrets.ADS_OCI_CONFIG_REALLIKE }}
55-
# ADS_OCI_PRIVATE_KEY_REALLIKE: ${{ secrets.ADS_OCI_PRIVATE_KEY_REALLIKE }}
56-
# run: |
57-
# set -x # print commands that are executed
58-
# mkdir -p /home/runner/.oci
59-
# ls -lha
60-
# echo "$ADS_OCI_CONFIG_REALLIKE" > "/home/runner/.oci/config"
61-
# echo "$ADS_OCI_PRIVATE_KEY_REALLIKE" > "/home/runner/.oci/oci_ads_user.pem"
62-
# echo "key_file=/home/runner/.oci/oci_ads_user.pem" >> "/home/runner/.oci/config"
60+
- name: "Test config setup"
61+
shell: bash
62+
env:
63+
HOME_RUNNER_DIR: /home/runner
64+
run: |
65+
set -x # print commands that are executed
66+
mkdir -p "$HOME_RUNNER_DIR"/.oci
67+
openssl genrsa -out $HOME_RUNNER_DIR/.oci/oci_ads_user.pem 2048
68+
cat <<EOT >> "$HOME_RUNNER_DIR/.oci/config"
69+
[DEFAULT]
70+
user=ocid1.user.oc1..xxx
71+
fingerprint=00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00
72+
tenancy=ocid1.tenancy.oc1..xxx
73+
region=test_region
74+
key_file=$HOME_RUNNER_DIR/.oci/oci_ads_user.pem
75+
EOT
76+
ls -lha "$HOME_RUNNER_DIR"/.oci
77+
echo "Test config file:"
78+
cat $HOME_RUNNER_DIR/.oci/config
6379
64-
# - name: "Test env setup"
65-
# timeout-minutes: 20
66-
# shell: bash
67-
# run: |
68-
# set -x # print commands that are executed
69-
# sudo apt-get install libkrb5-dev graphviz
70-
# $CONDA/bin/conda init
71-
# source /home/runner/.bashrc
72-
# pip install setuptools
73-
# pip install -r dev-requirements.txt
80+
- name: "Test env setup"
81+
timeout-minutes: 20
82+
shell: bash
83+
run: |
84+
set -x # print commands that are executed
85+
sudo apt-get install libkrb5-dev graphviz
86+
$CONDA/bin/conda init
87+
source /home/runner/.bashrc
88+
pip install setuptools
89+
pip install -r dev-requirements.txt
7490
75-
# - name: "Run unitary tests folder with maximum ADS dependencies"
76-
# timeout-minutes: 60
77-
# shell: bash
78-
# env:
79-
# NB_SESSION_COMPARTMENT_OCID: ocid1.compartment.oc1.<unique_ocid>
80-
# CONDA_PREFIX: /usr/share/miniconda
81-
# run: |
82-
# python -m pytest -v -p no:warnings --durations=5 \
83-
# ${{ matrix.test-path }} \
84-
# --ignore "${{ matrix.ignore-path }}" \
85-
# --cov --cov-append --cov-report=html
91+
- name: "Run unitary tests folder with maximum ADS dependencies"
92+
timeout-minutes: 60
93+
shell: bash
94+
env:
95+
NB_SESSION_COMPARTMENT_OCID: ocid1.compartment.oc1.<unique_ocid>
96+
CONDA_PREFIX: /usr/share/miniconda
97+
run: |
98+
python -m pytest -v -p no:warnings --durations=5 \
99+
${{ matrix.test-path }} \
100+
--ignore "${{ matrix.ignore-path }}" \
101+
--cov --cov-append --cov-report=html
86102
87103
# Uploading test artifacts
88104
# https://docs.github.com/en/actions/using-workflows/storing-workflow-data-as-artifacts#uploading-build-and-test-artifacts
89-
# - name: "Save html coverage report"
90-
# uses: actions/upload-artifact@v3
91-
# with:
92-
# name: code-coverage-report
93-
# path: htmlcov/
94-
# retention-days: 10
105+
- name: "Save html coverage report"
106+
uses: actions/upload-artifact@v3
107+
with:
108+
name: code-coverage-report
109+
path: htmlcov/
110+
retention-days: 10

‎.gitignore‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -145,8 +145,9 @@ docs/ads-latest.zip
145145
# html reports of covered code by pytests and
146146
# execution data collected by coverage.py
147147
*coverage_html_report*
148+
*htmlcov*
148149
.coverage
149-
.coverage*
150+
.coverage_*
150151
**.sqlite
151152

152153
extensions/**/node_modules/

‎dev-requirements.txt‎

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,16 @@
1+
-r test-requirements.txt
2+
-e ".[opctl]"
3+
-e ".[all-optional]"
4+
pyspark
5+
arff
6+
category_encoders
7+
pdfplumber
8+
dask
9+
nltk
10+
py4j
11+
tables
12+
xlrd>=1.2.0
13+
lxml
14+
fastparquet
15+
imbalanced-learn
16+
pyarrow

‎test-requirements.txt‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
-e .
2+
coverage
3+
faker
4+
mock
5+
pip
6+
pytest
7+
pytest-codecov

‎tests/__init__.py‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
#!/usr/bin/env python
2+
3+
# Copyright (c) 2023 Oracle and/or its affiliates.
4+
# Licensed under the Universal Permissive License v 1.0 as shown at https://oss.oracle.com/licenses/upl/

‎tests/conftest.py‎

Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,27 @@
1+
#!/usr/bin/env python
2+
3+
# Copyright (c) 2021, 2023 Oracle and/or its affiliates.
4+
# Licensed under the Universal Permissive License v 1.0 as shown at https://oss.oracle.com/licenses/upl/
5+
6+
import pytest
7+
8+
9+
def pytest_addoption(parser):
10+
parser.addoption(
11+
"--opctl",
12+
action="store_true",
13+
default=False,
14+
help="run opctl tests",
15+
)
16+
17+
18+
def pytest_configure(config):
19+
config.addinivalue_line("markers", "opctl: mark opctl tests")
20+
21+
22+
def pytest_collection_modifyitems(config, items):
23+
if not config.getoption("--opctl"):
24+
skip_opctl = pytest.mark.skip(reason="need --opctl option to run")
25+
for item in items:
26+
if "opctl" in item.keywords:
27+
item.add_marker(skip_opctl)

‎tests/unitary/__init__.py‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
#!/usr/bin/env python
2+
# -*- coding: utf-8 -*-
3+
4+
# Copyright (c) 2023 Oracle and/or its affiliates.
5+
# Licensed under the Universal Permissive License v 1.0 as shown at https://oss.oracle.com/licenses/upl/

‎tests/unitary/default_setup/__init__.py‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
#!/usr/bin/env python
2+
3+
# Copyright (c) 2023 Oracle and/or its affiliates.
4+
# Licensed under the Universal Permissive License v 1.0 as shown at https://oss.oracle.com/licenses/upl/

0 commit comments

Comments
 (0)