Adding different types of parallelism to the elementwise layer #222
Conversation
include/layers/EWLayer.hpp
Outdated
| EWLayerImpl() = delete; | ||
| EWLayerImpl(const Shape& shape, std::string function, float alpha = 0.0F, | ||
| float beta = 0.0F); | ||
| float beta = 0.0F, int type_parall = 0); |
There was a problem hiding this comment.
Use a strongly-typed backend enum instead of int for readability and safety.
enum class ParBackend { Seq = 0, Threads = 1, TBB = 2, OMP = 3 };
There was a problem hiding this comment.
Propagate ParBackend through API instead of raw int
include/layers/EWLayer.hpp
Outdated
| int available_threads = -1; | ||
| if (type_parall_ == 0) available_threads = 1; | ||
| if (type_parall_ == 1) | ||
| available_threads = std::thread::hardware_concurrency(); | ||
| if (type_parall_ == 2) | ||
| available_threads = oneapi::tbb::info::default_concurrency(); | ||
| if (type_parall_ == 3) available_threads = omp_get_max_threads(); |
There was a problem hiding this comment.
Please wrap common function for getting thread number
include/layers/Layer.hpp
Outdated
| @@ -1,5 +1,11 @@ | |||
| #pragma once | |||
| #include <omp.h> | |||
There was a problem hiding this comment.
Guard the OpenMP/TBB includes and add ; otherwise non-OpenMP builds fail.
#ifdef HAS_OPENMP
#include <omp.h>
#endif
#include <thread>
#ifdef HAS_TBB
#include <oneapi/tbb/blocked_range.h>
#include <oneapi/tbb/parallel_for.h>
#include <oneapi/tbb/info.h>
#endif
include/layers/EWLayer.hpp
Outdated
| EWLayerImpl() = delete; | ||
| EWLayerImpl(const Shape& shape, std::string function, float alpha = 0.0F, | ||
| float beta = 0.0F); | ||
| float beta = 0.0F, int type_parall = 0); |
There was a problem hiding this comment.
Propagate ParBackend through API instead of raw int
include/layers/Layer.hpp
Outdated
| }; | ||
|
|
||
| template <typename Func> | ||
| inline void parallel_for(int count, Func func, int mode = 0) { |
There was a problem hiding this comment.
| inline void parallel_for(int count, Func func, int mode = 0) { | |
| inline void parallel_for(int count, Func func, int mode = 0) { | |
| if (count <= 0) return; |
include/layers/Layer.hpp
Outdated
| @@ -1,5 +1,11 @@ | |||
| #pragma once | |||
| #include <omp.h> | |||
There was a problem hiding this comment.
Move all backend headers (OpenMP/TBB/Threads) and implementation details into a small parallel module. Expose a single, inline header API so call sites incur no extra call/indirection.
- include/parallel/parallel.hpp (inline API)
- include/parallel/backends.hpp (backend helpers; guarded includes)
- No <omp.h>/TBB headers leaking into layer headers.
Example:
// include/parallel/parallel.hpp
#pragma once
#include <cstddef>
enum class ParBackend { Auto, Seq, Threads, TBB, OMP };
struct ParOptions {
ParBackend backend = ParBackend::Auto;
int max_threads = 0; // 0 = runtime default
std::size_t min_parallel_n = 4096; // small tasks stay sequential
std::size_t grain = 1024; // backend-specific chunk hint
};
// Header-only: one branch + inlined backend
template <class F>
inline void parallel_for(std::size_t n, F&& f, const ParOptions& opt) {
if (n == 0) return;
const ParBackend b = select_backend(opt, n); // inline, cheap
switch (b) {
case ParBackend::Seq: return impl_seq(n, f);
case ParBackend::Threads: return impl_threads(n, f, opt);
case ParBackend::TBB: return impl_tbb(n, f, opt);
case ParBackend::OMP: return impl_omp(n, f, opt);
case ParBackend::Auto: return impl_seq(n, f); // unreachable
}
}
include/layers/Layer.hpp
Outdated
| @@ -1,5 +1,11 @@ | |||
| #pragma once | |||
| #include <omp.h> | |||
|
|
|||
There was a problem hiding this comment.
Avoid re-evaluating “Auto” logic every call. Resolve once (feature flags + environment + problem size) and cache in the layer/context.
// Called once per layer or first use
inline ParBackend resolve_auto_once(const ParOptions& opt, std::size_t n) noexcept {
#if defined(HAS_OMP)
if (n >= opt.min_parallel_n) return ParBackend::OMP;
#elif defined(HAS_TBB)
if (n >= opt.min_parallel_n) return ParBackend::TBB;
#elif defined(HAS_THREADS)
if (n >= opt.min_parallel_n) return ParBackend::Threads;
#endif
return ParBackend::Seq;
}
inline ParBackend select_backend(const ParOptions& opt, std::size_t n) noexcept {
if (opt.backend != ParBackend::Auto) return opt.backend;
static ParBackend cached = resolve_auto_once(opt, n); // or store in the layer
return cached;
}
aobolensk
left a comment
aobolensk
left a comment
There was a problem hiding this comment.
I actually think we can leave remaining solution basically as is. The problem with OpenMP slowdown is actually reproducible, but I suggest to focus on parallel_for itself. Anyway, this effect is not that visible on matrix multiplication workloads. For further investigation we will take a look at the compilation details (which code it has been lowered to). For now we can proceed as is
No description provided.