Skip to content

Commit fb1d526

Browse files
cschreibcschreib
committed
Added speed benchmark
1 parent d7f146d commit fb1d526

File tree

4 files changed

+264
-1
lines changed

4 files changed

+264
-1
lines changed

‎README.md‎

Lines changed: 29 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -80,7 +80,7 @@ The following limitations are features that were not implemented simply because
8080

8181
## Comparison spreadsheet
8282

83-
In this comparison spreadsheet, the raw pointer `T*` is assumed to never be owning, and used only to observe an existing object (which may or may not have been deleted). The stack and heap sizes were measured with gcc and libstdc++.
83+
In this comparison spreadsheet, the raw pointer `T*` is assumed to never be owning, and used only to observe an existing object (which may or may not have been deleted). The stack and heap sizes were measured with gcc 9.3.0 and libstdc++.
8484

8585
Labels:
8686
- raw: `T*`
@@ -120,6 +120,34 @@ Notes:
120120
- (5) 2 by default, or 1 if using `std::make_shared()`.
121121

122122

123+
## Speed benchmarks
124+
125+
Labels are the same as in the comparison spreadsheet. The speed benchmarks were compiled with gcc 9.3.0 and libstdc++, with all optimizations turned on (except LTO), and run on a linux (5.1.0-89) machine with a Ryzen 5 2600 CPU. Speed is measured relative to `std::unique_ptr<T>` used as owner pointer, and `T*` used as observer pointer.
126+
127+
You can run the benchmarks yourself, they are located in `tests/speed_benchmark.cpp`. The benchmark executable runs tests for three object types: `int`, `std::string`, and `std::array<int,65'536>`, to simulate objects of various allocation cost. The timings below are reported for `int`, which should be most relevant to highlight the overhead from the pointer itself. In real life scenarios, the actual measured overhead will be substantially lower, as actual business logic is likely to dominate the time budget.
128+
129+
| Pointer | raw/unique | weak/shared | observer/obs_unique | observer/obs_sealed |
130+
|--------------------------|------------|-------------|---------------------|---------------------|
131+
| Create owner empty | 1 | 0.72 | 0.83 | 1 |
132+
| Create owner | 1 | 2.58 | 1.85 | N/A |
133+
| Create owner factory | 1 | 1.40 | 1.80 | 1.13 |
134+
| Dereference owner | 1 | 1 | 1 | 1 |
135+
| Create observer empty | 1 | 1 | 0.71 | 0.71 |
136+
| Create observer | 1 | 2.14 | 2.14 | 2.04 |
137+
| Create observer copy | 1 | 3.00 | 3.00 | 3.00 |
138+
| Dereference observer | 1 | 3.50 | 0.75 | 0.75 |
139+
140+
Detail of the benchmarks:
141+
- Create owner empty: default-construct an owner pointer (contains nullptr).
142+
- Create owner: construct an owner pointer by taking ownership of an object (for `oup::observer_sealed_ptr`, this is using `oup::make_observable_sealed()`).
143+
- Create owner factory: construct an owner pointer using `std::make_*` or `oup::make_*` factory functions.
144+
- Dereference owner: get a reference to the underlying owned object from an owner pointer.
145+
- Create observer empty: default-construct an observer pointer (contains nullptr).
146+
- Create observer: construct an observer pointer from an owner pointer.
147+
- Create observer copy: construct a new observer pointer from another observer pointer.
148+
- Dereference observer: get a reference to the underlying object from an observer pointer.
149+
150+
123151
## Notes
124152

125153
### Alternative implementation

‎tests/CMakeLists.txt‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -77,3 +77,8 @@ message(STATUS "Running compile-time tests ended.")
7777

7878
add_executable(oup_size_benchmark ${PROJECT_SOURCE_DIR}/tests/size_benchmark.cpp)
7979
target_link_libraries(oup_size_benchmark PRIVATE oup::oup)
80+
81+
add_executable(oup_speed_benchmark
82+
${PROJECT_SOURCE_DIR}/tests/speed_benchmark.cpp
83+
${PROJECT_SOURCE_DIR}/tests/speed_benchmark_utility.cpp)
84+
target_link_libraries(oup_speed_benchmark PRIVATE oup::oup)

‎tests/speed_benchmark.cpp‎

Lines changed: 191 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,191 @@
1+
#include <memory>
2+
#include <iostream>
3+
#include <chrono>
4+
#include <array>
5+
#include <string>
6+
#include <oup/observable_unique_ptr.hpp>
7+
8+
// External functions, the compiler cannot see through. Prevents optimisations.
9+
template<typename T>
10+
void use_object(T&) noexcept;
11+
12+
template<typename T>
13+
struct pointer_traits;
14+
15+
template<typename T>
16+
struct pointer_traits<std::unique_ptr<T>> {
17+
using element_type = T;
18+
using ptr_type = std::unique_ptr<T>;
19+
using weak_type = T*;
20+
21+
static ptr_type make_ptr() noexcept { return ptr_type(new element_type); }
22+
static ptr_type make_ptr_factory() noexcept { return std::make_unique<element_type>(); }
23+
static weak_type make_weak(ptr_type& p) noexcept { return p.get(); }
24+
template<typename F>
25+
static void deref_weak(weak_type& p, F&& func) noexcept { return func(*p); }
26+
};
27+
28+
template<typename T>
29+
struct pointer_traits<std::shared_ptr<T>> {
30+
using element_type = T;
31+
using ptr_type = std::shared_ptr<T>;
32+
using weak_type = std::weak_ptr<T>;
33+
34+
static ptr_type make_ptr() noexcept { return ptr_type(new element_type); }
35+
static ptr_type make_ptr_factory() noexcept { return std::make_shared<element_type>(); }
36+
static weak_type make_weak(ptr_type& p) noexcept { return weak_type(p); }
37+
template<typename F>
38+
static void deref_weak(weak_type& p, F&& func) noexcept { if (auto s = p.lock()) func(*s); }
39+
};
40+
41+
template<typename T>
42+
struct pointer_traits<oup::observable_unique_ptr<T>> {
43+
using element_type = T;
44+
using ptr_type = oup::observable_unique_ptr<T>;
45+
using weak_type = oup::observer_ptr<T>;
46+
47+
static ptr_type make_ptr() noexcept { return ptr_type(new element_type); }
48+
static ptr_type make_ptr_factory() noexcept { return oup::make_observable_unique<element_type>(); }
49+
static weak_type make_weak(ptr_type& p) noexcept { return weak_type(p); }
50+
template<typename F>
51+
static void deref_weak(weak_type& p, F&& func) noexcept { return func(*p); }
52+
};
53+
54+
template<typename T>
55+
struct pointer_traits<oup::observable_sealed_ptr<T>> {
56+
using element_type = T;
57+
using ptr_type = oup::observable_sealed_ptr<T>;
58+
using weak_type = oup::observer_ptr<T>;
59+
60+
static ptr_type make_ptr() noexcept { return oup::make_observable_sealed<element_type>(); }
61+
static ptr_type make_ptr_factory() noexcept { return oup::make_observable_sealed<element_type>(); }
62+
static weak_type make_weak(ptr_type& p) noexcept { return weak_type(p); }
63+
template<typename F>
64+
static void deref_weak(weak_type& p, F&& func) noexcept { return func(*p); }
65+
};
66+
67+
template<typename T>
68+
struct benchmark {
69+
using traits = pointer_traits<T>;
70+
using element_type = typename traits::element_type;
71+
using owner_type = typename traits::ptr_type;
72+
using weak_type = typename traits::weak_type;
73+
74+
owner_type owner;
75+
weak_type weak;
76+
77+
benchmark() : owner(traits::make_ptr()), weak(traits::make_weak(owner)) {}
78+
79+
void construct_destruct_owner_empty() {
80+
auto p = owner_type{};
81+
use_object(p);
82+
}
83+
84+
void construct_destruct_owner() {
85+
auto p = traits::make_ptr();
86+
use_object(p);
87+
}
88+
89+
void construct_destruct_owner_factory() {
90+
auto p = traits::make_ptr_factory();
91+
use_object(p);
92+
}
93+
94+
void construct_destruct_weak_empty() {
95+
auto p = weak_type{};
96+
use_object(p);
97+
}
98+
99+
void construct_destruct_weak() {
100+
auto wp = traits::make_weak(owner);
101+
use_object(wp);
102+
}
103+
104+
void dereference_owner() {
105+
use_object(*owner);
106+
}
107+
108+
void dereference_weak() {
109+
traits::deref_weak(weak, [](auto& o) { use_object(o); });
110+
}
111+
void construct_destruct_weak_copy() {
112+
auto wp = weak;
113+
use_object(wp);
114+
}
115+
};
116+
117+
using timer = std::chrono::high_resolution_clock;
118+
119+
template<typename B, typename F>
120+
double run_benchmark_for(F&& func) {
121+
B bench{};
122+
123+
auto prev = timer::now();
124+
double elapsed = 0.0;
125+
double count = 0.0;
126+
constexpr std::size_t num_iter = 10'000;
127+
128+
while (elapsed < 1.0) {
129+
for (std::size_t i = 0; i < num_iter; ++i) {
130+
func(bench);
131+
}
132+
133+
auto now = timer::now();
134+
elapsed += std::chrono::duration_cast<std::chrono::duration<double>>(now - prev).count();
135+
count += static_cast<double>(num_iter);
136+
std::swap(now, prev);
137+
}
138+
139+
return elapsed/count;
140+
}
141+
142+
template<typename B, typename F>
143+
auto run_benchmark(F&& func) {
144+
using ref_type = benchmark<std::unique_ptr<typename B::element_type>>;
145+
double result = run_benchmark_for<B>(func);
146+
double result_ref = run_benchmark_for<ref_type>(func);
147+
return std::make_pair(result, result/result_ref);
148+
}
149+
150+
template<typename T>
151+
void do_benchmarks_for_ptr(const char* type_name, const char* ptr_name) {
152+
using B = benchmark<T>;
153+
154+
auto construct_destruct_owner_empty = run_benchmark<B>([](auto& b) { return b.construct_destruct_owner_empty(); });
155+
auto construct_destruct_owner = run_benchmark<B>([](auto& b) { return b.construct_destruct_owner(); });
156+
auto construct_destruct_owner_factory = run_benchmark<B>([](auto& b) { return b.construct_destruct_owner_factory(); });
157+
auto dereference_owner = run_benchmark<B>([](auto& b) { return b.dereference_owner(); });
158+
auto construct_destruct_weak_empty = run_benchmark<B>([](auto& b) { return b.construct_destruct_weak_empty(); });
159+
auto construct_destruct_weak = run_benchmark<B>([](auto& b) { return b.construct_destruct_weak(); });
160+
auto construct_destruct_weak_copy = run_benchmark<B>([](auto& b) { return b.construct_destruct_weak_copy(); });
161+
auto dereference_weak = run_benchmark<B>([](auto& b) { return b.dereference_weak(); });
162+
163+
std::cout << ptr_name << "<" << type_name << ">:" << std::endl;
164+
#define report(which) std::cout << " - " << #which << ": " << which.first*1e6 << "us (x" << which.second << ")" << std::endl
165+
166+
report(construct_destruct_owner_empty);
167+
report(construct_destruct_owner);
168+
report(construct_destruct_owner_factory);
169+
report(dereference_owner);
170+
report(construct_destruct_weak_empty);
171+
report(construct_destruct_weak);
172+
report(construct_destruct_weak_copy);
173+
report(dereference_weak);
174+
175+
#undef report
176+
std::cout << std::endl;
177+
}
178+
179+
template<typename T>
180+
void do_benchmarks(const char* type_name) {
181+
do_benchmarks_for_ptr<std::shared_ptr<T>>(type_name, "shared_ptr");
182+
do_benchmarks_for_ptr<oup::observable_unique_ptr<T>>(type_name, "observable_unique_ptr");
183+
do_benchmarks_for_ptr<oup::observable_sealed_ptr<T>>(type_name, "observable_sealed_ptr");
184+
}
185+
186+
int main() {
187+
do_benchmarks<int>("int");
188+
do_benchmarks<std::string>("string");
189+
do_benchmarks<std::array<int,65'536>>("big_array");
190+
return 0;
191+
}

‎tests/speed_benchmark_utility.cpp‎

Lines changed: 39 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,39 @@
1+
#include <oup/observable_unique_ptr.hpp>
2+
#include <array>
3+
#include <string>
4+
#include <memory>
5+
6+
template<typename T>
7+
void use_object(T&) noexcept {}
8+
9+
template void use_object<int>(int&) noexcept;
10+
template void use_object<std::string>(std::string&) noexcept;
11+
template void use_object<std::array<int,65'536>>(std::array<int,65'536>&) noexcept;
12+
13+
template void use_object<int*>(int*&) noexcept;
14+
template void use_object<std::string*>(std::string*&) noexcept;
15+
template void use_object<std::array<int,65'536>*>(std::array<int,65'536>*&) noexcept;
16+
17+
template void use_object<std::unique_ptr<int>>(std::unique_ptr<int>&) noexcept;
18+
template void use_object<std::unique_ptr<std::string>>(std::unique_ptr<std::string>&) noexcept;
19+
template void use_object<std::unique_ptr<std::array<int,65'536>>>(std::unique_ptr<std::array<int,65'536>>&) noexcept;
20+
21+
template void use_object<std::shared_ptr<int>>(std::shared_ptr<int>&) noexcept;
22+
template void use_object<std::shared_ptr<std::string>>(std::shared_ptr<std::string>&) noexcept;
23+
template void use_object<std::shared_ptr<std::array<int,65'536>>>(std::shared_ptr<std::array<int,65'536>>&) noexcept;
24+
25+
template void use_object<std::weak_ptr<int>>(std::weak_ptr<int>&) noexcept;
26+
template void use_object<std::weak_ptr<std::string>>(std::weak_ptr<std::string>&) noexcept;
27+
template void use_object<std::weak_ptr<std::array<int,65'536>>>(std::weak_ptr<std::array<int,65'536>>&) noexcept;
28+
29+
template void use_object<oup::observable_unique_ptr<int>>(oup::observable_unique_ptr<int>&) noexcept;
30+
template void use_object<oup::observable_unique_ptr<std::string>>(oup::observable_unique_ptr<std::string>&) noexcept;
31+
template void use_object<oup::observable_unique_ptr<std::array<int,65'536>>>(oup::observable_unique_ptr<std::array<int,65'536>>&) noexcept;
32+
33+
template void use_object<oup::observable_sealed_ptr<int>>(oup::observable_sealed_ptr<int>&) noexcept;
34+
template void use_object<oup::observable_sealed_ptr<std::string>>(oup::observable_sealed_ptr<std::string>&) noexcept;
35+
template void use_object<oup::observable_sealed_ptr<std::array<int,65'536>>>(oup::observable_sealed_ptr<std::array<int,65'536>>&) noexcept;
36+
37+
template void use_object<oup::observer_ptr<int>>(oup::observer_ptr<int>&) noexcept;
38+
template void use_object<oup::observer_ptr<std::string>>(oup::observer_ptr<std::string>&) noexcept;
39+
template void use_object<oup::observer_ptr<std::array<int,65'536>>>(oup::observer_ptr<std::array<int,65'536>>&) noexcept;

0 commit comments

Comments
 (0)