Skip to content

Commit bb378ce

Browse files
More ui videos (#19)
1 parent 64cf8e4 commit bb378ce

File tree

2 files changed

+74
-177
lines changed

2 files changed

+74
-177
lines changed

docs/backend/spark-performance.md

Lines changed: 6 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -14,7 +14,7 @@ All tests run here were using [Bullet-Spark 0.1.2](https://github.com/bullet-db/
1414

1515
* [jq](https://stedolan.github.io/jq/) - a nice tool to parse Bullet JSON responses
1616
* curl, bash and python - for running and analyzing Bullet queries
17-
* [Apache JMeter](https://jmeter.apache.org/) - a load testing tool to send queries to the server simultaneously
17+
* [Apache JMeter](https://jmeter.apache.org/) - a load testing tool to send multiple queries to the server simultaneously
1818

1919
### Cluster
2020

@@ -112,11 +112,11 @@ Command line:
112112

113113
## Test 1: Latency of Bullet Spark
114114

115-
This test was done on the smaller data. We used a [RAW query without any filtering](../ws/examples/#simplest-query) to measure the latency added by Bullet Spark. This is not the end-to-end latency for a query. It is the latency from receiving the query to finishing the query, not includes the time spent in Kafka. We ran this query 100 times.
115+
This test was done on the smaller data. We used a [RAW query without any filtering](../ws/examples/#simplest-query) to measure the latency added by Bullet Spark. This is not the end-to-end latency for a query. It is the latency from receiving the query to finishing the query, not including the time spent in Kafka. We ran this query 100 times.
116116

117117
### Result
118118

119-
This graph shows the latency of each attempts:
119+
This graph shows the latency of each attempt:
120120

121121
![Bullet Spark Latency](../img/spark-perf-latency.png)
122122

@@ -126,7 +126,7 @@ The average latency was 1173 ms. This result shows that this is the fastest Bull
126126

127127
## Test 2: Scalability for smaller data
128128

129-
This test was done on the smaller data. We want to measure how many of queries we can have running simultaneously on Bullet Spark. We ran 400, 800, 1500 and 1100 queries each for 10 minutes.
129+
This test was done on the smaller data. We want to measure how many queries we can have running simultaneously on Bullet Spark. We ran 400, 800, 1500 and 1100 queries each for 10 minutes.
130130

131131
### Result
132132

@@ -168,7 +168,7 @@ This test was done on the smaller data. We want to measure how many of queries w
168168

169169
### Conclusion
170170

171-
The average processing time for each batch was 1 second 143 ms which was below the batch duration 2 seconds. For average, 1 CPU core and 3GB memory were used in this experiment. CPU and memory usages go slowly up while queries number goes up but they are still within resource limits. We can easily run up to 1500 RAW queries simultaneously in this test.
171+
The average processing time for each batch was 1 second 143 ms which was below the batch duration 2 seconds. On average, 1 CPU core and 3GB memory were used in this experiment. CPU and memory usages go slowly up while queries number goes up but they are still within resource limits. We can easily run up to 1500 RAW queries simultaneously in this test.
172172

173173
## Test 3: Scalability for larger data
174174

@@ -202,4 +202,4 @@ This test was done on the larger data. We ran 100, 400, 800 and 600 queries each
202202

203203
### Conclusion
204204

205-
The average processing time for each batch was 3 second 97 ms which was below the batch duration 5 seconds. For average, 1.2 CPU core and average 5GB memory were used in this experiment. But with queries number goes up, some of the executors memory usage were up to 8-10GB which is close to our resource limits. With more queries running, OOM may happen. So in this experiment, we can only afford up to 800 queries simultaneously.
205+
The average processing time for each batch was 3 seconds 97 ms which was below the batch duration 5 seconds. On average, 1.2 CPU core and average 5GB memory were used in this experiment. But with queries number goes up, some of the executors memory usage were up to 8-10GB which is close to our resource limits. With more queries running, OOM may happen. So in this experiment, we can only afford up to 800 queries simultaneously.

0 commit comments

Comments
 (0)