Skip to content

Commit cd28626

Browse files
author
Hao, Ruomeng
committed
Add perf testing
1. Scripts 2. Manifest 3. Document Signed-off-by: Hao, Ruomeng <ruomeng.hao@intel.com>
1 parent 36d01b4 commit cd28626

File tree

8 files changed

+553
-79
lines changed

8 files changed

+553
-79
lines changed

‎container/ccnp-example/Dockerfile‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -36,6 +36,7 @@ RUN chown $USER:$GROUP /run/ccnp
3636

3737
COPY test ./
3838
COPY sdk/python3/example/py_sdk_example.py ./
39+
COPY test/perf/py_perf.py ./
3940
COPY --from=python-builder cc-trusted-api/common/python/dist/cctrusted_base*.whl ./
4041
COPY --from=python-builder ccnp-sdk/dist/ccnp*.whl ./
4142

‎deployment/kubernetes/manifests/ccnp-example-perf-deployment.yaml‎

Lines changed: 26 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,26 @@
1+
apiVersion: apps/v1
2+
kind: Deployment
3+
metadata:
4+
name: ccnp-perf-example-PLACEHOLDER
5+
namespace: perf
6+
spec:
7+
selector:
8+
matchLabels:
9+
app: ccnp-perf-example-PLACEHOLDER
10+
template:
11+
metadata:
12+
labels:
13+
app: ccnp-perf-example-PLACEHOLDER
14+
annotations:
15+
"ccnp.cc-api/require": "true"
16+
spec:
17+
containers:
18+
- name: ccnp-perf-example-PLACEHOLDER
19+
image: "docker.io/library/ccnp-example:latest"
20+
imagePullPolicy: Always
21+
resources:
22+
limits:
23+
memory: "128Mi"
24+
cpu: "100m"
25+
nodeSelector:
26+
"feature.node.kubernetes.io/cpu-security.tdx.protected": "true"

‎test/perf/README.md‎

Lines changed: 26 additions & 79 deletions
Original file line numberDiff line numberDiff line change
@@ -1,102 +1,49 @@
1-
# Performance Test
1+
# CCNP Performance Test
22

3-
We have these KPIs for performance test.
4-
5-
| KPI​ | HIB/LIB​ | Unit​ | Comment​ |
6-
| ------------------------------------------- | -------- | ----- | ----------------------------------------------------- |
7-
| CCNP service get measurement throughput​ | HIB​ | ops​ | Service Throughput​ |
8-
| CCNP service get measurement response time​ | LIB​ | ms​ | Service Response time​ |
9-
| CCNP service get eventlog throughput​ | HIB​ | ops​ | Service Throughput​ |
10-
| CCNP service get eventlog response time​ | LIB​ | ms​ | Service Response time​ |
11-
| CCNP service get quote throughput​ | HIB​ | ops​ | Service Throughput​ |
12-
| CCNP service get quote response time​ | LIB​ | ms​ | Service Response time​ |
13-
| CCNP initialization time​ | LIB​ | s​ | CCNP device plugin, DaemonSet and service readiness.​ |
14-
15-
*Note: we use the CCNP SDK to access the CCNP service because it's convenient to prepare the request data (e.g. container ID, etc.)​
3+
CCNP performance tests focus on the latency of calling CCNP SDK key APIs: `get_cc_eventlog`, `get_cc_report` and `get_cc_measurement`.
4+
It will simulate requests from multiple pods in parallel and calculate average time of all the requests.
165

176
Below are the steps for you to build and run the performance test.
187

198
## Prerequisites
209

21-
To run the test, you need a K8S cluster with CCNP enabled (CCNP Device Plugin and CCNP Service deployed and ready).
22-
23-
## Build
24-
25-
```bash
26-
# Make sure you are on the repo's top dir
27-
cd <the-dir-of-confidential-cloud-native-primitives>
10+
Please make sure you have CCNP deployed in a K8S cluster, and ccnp-example image has been built.
11+
Please refer to [here](../../deployment/README.md) for image building and CCNP deployment.
2812

29-
# Run doker build
30-
docker build --build-arg http_proxy=$http_proxy --build-arg https_proxy=$https_proxy --build-arg no_proxy=$no_proxy -t ccnp-perf:latest -f container/ccnp-perf/Dockerfile .
13+
## Run Tests
3114

32-
# View build result
33-
docker image ls | grep ccnp-perf
15+
### Deploy pods for performance testing
3416

35-
# Save the docker image for later use
36-
docker save ccnp-perf:latest > ccnp-perf_latest.tar
17+
```bash
18+
# Deploy ccnp-example pods
19+
$ sudo ./deploy-perf.sh -r <ccnp-example image registry> -g <ccnp-example image tag> -n <number of pods>
3720
```
3821

39-
## Deploy
22+
### Run Tests
23+
24+
The script will run tests in parallel. The log will be saved in files with prefix `perf_output` under current directory.
4025

4126
```bash
42-
# Load the docker image for K8S using containerd.
43-
# You need to run this on the node where you want to deploy the ccnp-perf test
44-
ctr -n=k8s.io image import ccnp-perf_latest.tar
27+
# Test for get event log
28+
$ sudo ./perf-para.sh -n <number of pods> -e
4529

46-
# Make sure you are on the repo's top dir
47-
cd <the-dir-of-confidential-cloud-native-primitives>
30+
# Test for get measurement
31+
$ sudo ./perf-para.sh -n <number of pods> -m
4832

49-
# Deploy ccnp-perf test
50-
kubectl apply -f deployment/kubernetes/manifests/ccnp-perf-deployment.yaml
33+
# Test for get quote
34+
$ sudo ./perf-para.sh -n <number of pods> -r
5135
```
5236

53-
## Test
37+
Run below script to calculate average time of a request.
5438

5539
```bash
56-
# Get the pod name of ccnp-perf
57-
kubectl get pod | grep ccnp-perf
58-
59-
# Run all perf test on the specified pod name got from above command
60-
kubectl exec -ti <ccnp-perf-pod-name> -- python3 -m pytest --log-cli-level=INFO --verbose ccnp_perf.py
40+
$ sudo ./average.sh -f perf_output_quote
6141
```
6242

63-
Sample test output looks like this:
43+
### Clear
44+
45+
Run below command to delete the pods for performance testing.
6446

6547
```bash
66-
root@ccnp-perf-0:~/ccnp/confidential-cloud-native-primitives# kubectl exec -ti ccnp-perf-7f8798bf85-8s6zg -- python3 -m pytest --log-cli-level=INFO --verbose
67-
ccnp_perf.py
68-
==================================================================== test session starts ====================================================================
69-
platform linux -- Python 3.12.2, pytest-8.1.1, pluggy-1.4.0 -- /usr/local/bin/python3
70-
cachedir: .pytest_cache
71-
rootdir: /run/ccnp
72-
collected 7 items
73-
74-
ccnp_perf.py::test_svc_get_cc_measurement_throughput
75-
----------------------------------------------------------------------- live log call -----------------------------------------------------------------------
76-
INFO ccnp_perf:ccnp_perf.py:191 Perf test average throughput is: 70.75 ops (operations per second)
77-
PASSED [ 14%]
78-
ccnp_perf.py::test_svc_get_cc_measurement_response
79-
----------------------------------------------------------------------- live log call -----------------------------------------------------------------------
80-
INFO ccnp_perf:ccnp_perf.py:213 Perf test average response time is: 25.89662575 ms (milliseconds)
81-
PASSED [ 28%]
82-
ccnp_perf.py::test_svc_get_cc_eventlog_throughput
83-
----------------------------------------------------------------------- live log call -----------------------------------------------------------------------
84-
INFO ccnp_perf:ccnp_perf.py:191 Perf test average throughput is: 57.8 ops (operations per second)
85-
PASSED [ 42%]
86-
ccnp_perf.py::test_svc_get_cc_eventlog_response
87-
----------------------------------------------------------------------- live log call -----------------------------------------------------------------------
88-
INFO ccnp_perf:ccnp_perf.py:213 Perf test average response time is: 76.130223 ms (milliseconds)
89-
PASSED [ 57%]
90-
ccnp_perf.py::test_svc_get_cc_report_throughput
91-
----------------------------------------------------------------------- live log call -----------------------------------------------------------------------
92-
INFO ccnp_perf:ccnp_perf.py:191 Perf test average throughput is: 54.9 ops (operations per second)
93-
PASSED [ 71%]
94-
ccnp_perf.py::test_svc_get_cc_report_response
95-
----------------------------------------------------------------------- live log call -----------------------------------------------------------------------
96-
INFO ccnp_perf:ccnp_perf.py:213 Perf test average response time is: 29.38618825 ms (milliseconds)
97-
PASSED [ 85%]
98-
ccnp_perf.py::test_ccnp_init PASSED [100%]
99-
100-
=============================================================== 7 passed in 66.95s (0:01:06) ================================================================
101-
root@ccnp-perf-0:~/ccnp/confidential-cloud-native-primitives#
102-
```
48+
$ sudo ./deploy-per.sh -n <number of pods> -d
49+
```

‎test/perf/average.sh‎

Lines changed: 73 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,73 @@
1+
#!/bin/bash
2+
# Input filename keyword and keyword for grep. Find the files in current dir, search the numbers needed, calculate sum and average
3+
4+
grep_keyword="total command "
5+
6+
usage() {
7+
cat << EOM
8+
usage: $(basename "$0") [OPTION]...
9+
-f Input file name keyword
10+
-k Keyword to be searched in the input files
11+
EOM
12+
exit 1
13+
}
14+
15+
process_args() {
16+
while getopts ":f:k:h" option; do
17+
case "${option}" in
18+
f) filename_keyword=${OPTARG};;
19+
k) grep_keyword=${OPTARG};;
20+
h) usage;;
21+
*) echo "Invalid option: -${OPTARG}" >&2
22+
usage
23+
;;
24+
esac
25+
done
26+
}
27+
28+
29+
# Find all files containing the filename keyword
30+
files=$(find . -type f -name "*${filename_keyword}*"| cut -d/ -f2)
31+
32+
total_sum=0
33+
total_count=0
34+
total_aver=0
35+
36+
# Loop through the found files
37+
for file in $files; do
38+
sum=0
39+
count=0
40+
41+
tag=$(echo "$file" | cut -d- -f2)
42+
tmpfile="tmp_output_"$tag
43+
touch "$tmpfile"
44+
45+
# Grep the keyword and extract the number. The number is the 2nd last word in the line.
46+
grep -i "${grep_keyword}" "$file" | awk '{print $(NF-1)}' > "$tmpfile"
47+
48+
while IFS= read -r line; do
49+
# Add the number to sum
50+
sum=$(echo "$sum + $line" | bc)
51+
# Increment the count
52+
count=$((count + 1))
53+
done < "$tmpfile"
54+
55+
# Calculate the average
56+
if [ $count -ne 0 ]; then
57+
average=$(echo "scale=6; $sum / $count" | bc)
58+
echo "Output file: $file"
59+
echo "Number file: $tmpfile"
60+
echo "Count: $count"
61+
echo "Sum: $sum"
62+
echo "Average: $average"
63+
else
64+
echo "The file is empty"
65+
fi
66+
total_sum=$(echo "$sum + $total_sum" | bc)
67+
total_count=$(echo "$count + $total_count" | bc)
68+
done
69+
70+
echo "Total Sum: $total_sum"
71+
echo "Total count: $total_count"
72+
total_aver=$(echo "scale=6; $total_sum / $total_count" | bc)
73+
echo "Total average: $total_aver"

‎test/perf/ccnp-pytest.md‎

Lines changed: 102 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,102 @@
1+
# Performance Test
2+
3+
We have these KPIs for performance test.
4+
5+
| KPI​ | HIB/LIB​ | Unit​ | Comment​ |
6+
| ------------------------------------------- | -------- | ----- | ----------------------------------------------------- |
7+
| CCNP service get measurement throughput​ | HIB​ | ops​ | Service Throughput​ |
8+
| CCNP service get measurement response time​ | LIB​ | ms​ | Service Response time​ |
9+
| CCNP service get eventlog throughput​ | HIB​ | ops​ | Service Throughput​ |
10+
| CCNP service get eventlog response time​ | LIB​ | ms​ | Service Response time​ |
11+
| CCNP service get quote throughput​ | HIB​ | ops​ | Service Throughput​ |
12+
| CCNP service get quote response time​ | LIB​ | ms​ | Service Response time​ |
13+
| CCNP initialization time​ | LIB​ | s​ | CCNP device plugin, DaemonSet and service readiness.​ |
14+
15+
*Note: we use the CCNP SDK to access the CCNP service because it's convenient to prepare the request data (e.g. container ID, etc.)​
16+
17+
Below are the steps for you to build and run the performance test.
18+
19+
## Prerequisites
20+
21+
To run the test, you need a K8S cluster with CCNP enabled (CCNP Device Plugin and CCNP Service deployed and ready).
22+
23+
## Build
24+
25+
```bash
26+
# Make sure you are on the repo's top dir
27+
cd <the-dir-of-confidential-cloud-native-primitives>
28+
29+
# Run doker build
30+
docker build --build-arg http_proxy=$http_proxy --build-arg https_proxy=$https_proxy --build-arg no_proxy=$no_proxy -t ccnp-perf:latest -f container/ccnp-perf/Dockerfile .
31+
32+
# View build result
33+
docker image ls | grep ccnp-perf
34+
35+
# Save the docker image for later use
36+
docker save ccnp-perf:latest > ccnp-perf_latest.tar
37+
```
38+
39+
## Deploy
40+
41+
```bash
42+
# Load the docker image for K8S using containerd.
43+
# You need to run this on the node where you want to deploy the ccnp-perf test
44+
ctr -n=k8s.io image import ccnp-perf_latest.tar
45+
46+
# Make sure you are on the repo's top dir
47+
cd <the-dir-of-confidential-cloud-native-primitives>
48+
49+
# Deploy ccnp-perf test
50+
kubectl apply -f deployment/kubernetes/manifests/ccnp-perf-deployment.yaml
51+
```
52+
53+
## Test
54+
55+
```bash
56+
# Get the pod name of ccnp-perf
57+
kubectl get pod | grep ccnp-perf
58+
59+
# Run all perf test on the specified pod name got from above command
60+
kubectl exec -ti <ccnp-perf-pod-name> -- python3 -m pytest --log-cli-level=INFO --verbose ccnp_perf.py
61+
```
62+
63+
Sample test output looks like this:
64+
65+
```bash
66+
root@ccnp-perf-0:~/ccnp/confidential-cloud-native-primitives# kubectl exec -ti ccnp-perf-7f8798bf85-8s6zg -- python3 -m pytest --log-cli-level=INFO --verbose
67+
ccnp_perf.py
68+
==================================================================== test session starts ====================================================================
69+
platform linux -- Python 3.12.2, pytest-8.1.1, pluggy-1.4.0 -- /usr/local/bin/python3
70+
cachedir: .pytest_cache
71+
rootdir: /run/ccnp
72+
collected 7 items
73+
74+
ccnp_perf.py::test_svc_get_cc_measurement_throughput
75+
----------------------------------------------------------------------- live log call -----------------------------------------------------------------------
76+
INFO ccnp_perf:ccnp_perf.py:191 Perf test average throughput is: 70.75 ops (operations per second)
77+
PASSED [ 14%]
78+
ccnp_perf.py::test_svc_get_cc_measurement_response
79+
----------------------------------------------------------------------- live log call -----------------------------------------------------------------------
80+
INFO ccnp_perf:ccnp_perf.py:213 Perf test average response time is: 25.89662575 ms (milliseconds)
81+
PASSED [ 28%]
82+
ccnp_perf.py::test_svc_get_cc_eventlog_throughput
83+
----------------------------------------------------------------------- live log call -----------------------------------------------------------------------
84+
INFO ccnp_perf:ccnp_perf.py:191 Perf test average throughput is: 57.8 ops (operations per second)
85+
PASSED [ 42%]
86+
ccnp_perf.py::test_svc_get_cc_eventlog_response
87+
----------------------------------------------------------------------- live log call -----------------------------------------------------------------------
88+
INFO ccnp_perf:ccnp_perf.py:213 Perf test average response time is: 76.130223 ms (milliseconds)
89+
PASSED [ 57%]
90+
ccnp_perf.py::test_svc_get_cc_report_throughput
91+
----------------------------------------------------------------------- live log call -----------------------------------------------------------------------
92+
INFO ccnp_perf:ccnp_perf.py:191 Perf test average throughput is: 54.9 ops (operations per second)
93+
PASSED [ 71%]
94+
ccnp_perf.py::test_svc_get_cc_report_response
95+
----------------------------------------------------------------------- live log call -----------------------------------------------------------------------
96+
INFO ccnp_perf:ccnp_perf.py:213 Perf test average response time is: 29.38618825 ms (milliseconds)
97+
PASSED [ 85%]
98+
ccnp_perf.py::test_ccnp_init PASSED [100%]
99+
100+
=============================================================== 7 passed in 66.95s (0:01:06) ================================================================
101+
root@ccnp-perf-0:~/ccnp/confidential-cloud-native-primitives#
102+
```

0 commit comments

Comments
 (0)