Merge pull request #60 from AerialMantis/d0796r2-final-changes

P0796r2: Final changes

Author: AerialMantis

affinity/cpp-20/d0796r2.md

@@ -16,9 +16,9 @@
 
 ### P0796r2 (RAP)
 
-* Introduced a free function for retrieving the execution resource underlying the current thread of execution.
+* Introduce a free function for retrieving the execution resource underlying the current thread of execution.
 * Introduce `this_thread::bind` & `this_thread::unbind` for binding and unbinding a thread of execution to an execution resource.
-* Introduce high-level interface for execution binding via executor properties.
+* Introduce `bulk_execution_affinity` executor properties for specifying affinity binding patterns on bulk execution functions.
 
 ### P0796r1 (JAX)
 
@@ -34,7 +34,9 @@
 
 # Abstract
 
-This paper provides an initial meta-framework for the drives toward memory affinity for C++.  It accounts for feedback from the Toronto 2017 SG1 meeting on Data Movement in C++ [[1]][p0687r0] that we should define affinity for C++ first, before considering inaccessible memory as a solution to the separate memory problem towards supporting heterogeneous and distributed computing.
+This paper provides an initial meta-framework for the drives toward an execution and memory affinity model for C++.  It accounts for feedback from the Toronto 2017 SG1 meeting on Data Movement in C++ [[1]][p0687r0] that we should define affinity for C++ first, before considering inaccessible memory as a solution to the separate memory problem towards supporting heterogeneous and distributed computing.
+
+This paper is split into two main parts; firstly a series of executor properties which can be used to apply affinity requirements to bulk execution functions, and secondly an interface for discovering the execution resources within the system topology and querying relative affinity of execution resources.
 
 # Motivation
 
@@ -103,7 +105,7 @@ Some systems give additional user control through explicit binding of threads to
 In this paper we describe the problem space of affinity for C++, the various challenges which need to be addressed in defining a partitioning and affinity interface for C++, and some suggested solutions.  These include:
 
 * How to represent, identify and navigate the topology of execution resources available within a heterogeneous or distributed system.
-* How to query and measure the relative affininty between different execution resources within a system.
+* How to query and measure the relative affinity between different execution resources within a system.
 * How to bind execution and allocation particular execution resource(s).
 * What kind of and level of interface(s) should be provided by C++ for affinity.
 
@@ -120,7 +122,7 @@ The first task in allowing C++ applications to leverage memory locality is to pr
 
 The capability of querying underlying *execution resources* of a given *system* is particularly important towards supporting affinity control in C++. The current proposal for executors [[22]][p0443r4] leaves the *execution resource* largely unspecified. This is intentional: *execution resources* will vary greatly between one implementation and another, and it is out of the scope of the current executors proposal to define those. There is current work [[23]][p0737r0] on extending the executors proposal to describe a typical interface for an *execution context*. In this paper a typical *execution context* is defined with an interface for construction and comparison, and for retrieving an *executor*, waiting on submitted work to complete and querying the underlying *execution resource*. Extending the executors interface to provide topology information can serve as a basis for providing a unified interface to expose affinity. This interface cannot mandate a specific architectural definition, and must be generic enough that future architectural evolutions can still be expressed.
 
-Two important considerations when defining a unified interface for querying the *resource topology* of a *system*, are (a) what level of abstraction such an interface should have, and (b) at what granularity it should describe the topology's *execution resources*. As both the level of abstraction of an *execution resource* and the granularity that it is described in will vary greatly from one implementation to another, it’s important for the interface to be generic enough to support any level of abstraction. To achieve this we propose a generic hierarchical structure of *execution resources*, each *execution resource* being composed of other *execution resources* recursively. Each *execution resource* within this hierarchy can be used to place memory (i.e., allocate memory within the *execution resource’s* memory region), place execution (i.e. bind an execution to an *execution resource’s execution agents*), or both.
+Two important considerations when defining a unified interface for querying the *resource topology* of a *system*, are (a) what level of abstraction such an interface should have, and (b) at what granularity it should describe the typology's *execution resources*. As both the level of abstraction of an *execution resource* and the granularity that it is described in will vary greatly from one implementation to another, it’s important for the interface to be generic enough to support any level of abstraction. To achieve this we propose a generic hierarchical structure of *execution resources*, each *execution resource* being composed of other *execution resources* recursively. Each *execution resource* within this hierarchy can be used to place memory (i.e., allocate memory within the *execution resource’s* memory region), place execution (i.e. bind an execution to an *execution resource’s execution agents*), or both.
 
 For example, a NUMA system will likely have a hierarchy of nodes, each capable of placing memory and placing agents.  A system with both CPUs and GPUs (programmable graphics processing units) may have GPU local memory regions capable of placing memory, but not capable of placing agents.
 
@@ -178,13 +180,11 @@ In this paper we propose an interface for querying and representing the executio
 
 ### Interface granularity
 
-In this paper we propose both a low-level interface and a high-level interface:
-* The low-level interface consists of mechanisms for discovering detailed information about a system's topology and affinity properties which can be utilized to hand optimise parallel applications and libraries for the best performance. The low-level interface has high granularity and is aimed at users who have a high knowledge of the system architecture.
-* The high-level interface consists of policies which describe desired behavior when using parallel algorithms or libraries. The high-level interface has low granularity and is aimed at users who may have little or no knowledge of the system architecture.
-
-## High-level interface
+In this paper is split into two main parts:
+* A series of executor properties describe desired behavior when using parallel algorithms or libraries. These properties provide a low granularity and is aimed at users who may have little or no knowledge of the system architecture.
+* A series of execution resource topology mechanisms for discovering detailed information about the system's topology and affinity properties which can be used to hand optimise parallel applications and libraries for the best performance. These mechanisms provide a high granularity and is aimed at users who have a high knowledge of the system architecture.
 
-The high-level interface is a policy-based design which utilizes the executor property mechanism to provide additional affinity based requirements on executors.
+## Executor properties
 
 ### Bulk execution affinity
 
@@ -207,7 +207,7 @@ Below *(Listing 2)* is an example of executing a parallel task over 8 threads us
 ```
 *Listing 2: Example of using the bulk_execution_affinity property*
 
-## Low-level interface
+## Execution resource topology
 
 ### Execution resources
 
@@ -233,10 +233,6 @@ for (auto res : execution::this_system::get_resources()) {
 ```
 *Listing 3: Example of querying all the system level execution resources*
 
-### Current resource
-
-The `execution_resource` which underlies the current thread of execution can be queried through `this_thread::get_resource`.
-
 ### Querying relative affinity
 
 The `affinity_query` class template provides an abstraction for a relative affinity value between two `execution_resource`s. This value depends on a particular `affinity_operation` and `affinity_metric`. As a result, the `affinity_query` is templated on `affinity_operation` and `affinity_metric`, and is constructed from two `execution_resource`s. An `affinity_query` is not meant to be meaningful on its own. Instead, users are meant to compare two queries with comparison operators, in order to get a relative magnitude of affinity. If necessary, the value of an `affinity_query` can also be queried through `native_affinity`, though the return value of this is implementation defined.
@@ -301,6 +297,10 @@ Only developers that care about resource placement need to care about obtaining
 
 If a particular policy or algorithm requires to access placement information, the resources associated with the passed executor can be retrieved via the link to the `execution_context`.
 
+### Current resource
+
+The `execution_resource` which underlies the current thread of execution can be queried through `this_thread::get_resource`.
+
 ### Binding to execution
 
 A *thread of execution* can be requested to bind to a particular `execution_resource` for a particular *execution agent* by calling `this_thread::bind` if that `execution_resource` is able to place agents. If the current *thread of execution* is successfully bound to the specified `execution_resource` it will return `true` otherwise it will return `false`. If the *thread of execution* is successfully bound to the specified `execution_resource` then `execution_resource` returned by `this_thread::get_resource` must be equal to the `execution_resource` provided to `this_thread::bind`. Subsequently a *thread of execution* can be unbound by calling `this_thread::unbind`.
@@ -420,9 +420,9 @@ A *thread of execution* can be requested to bind to a particular `execution_reso
 
 ## Bulk execution affinity properties
 
-The bulk_execution_affinity_t property describes what guarantees executors provide about the binding of *execution agent*s to the underlying *execution resource*s.
+The `bulk_execution_affinity_t` property describes what guarantees executors provide about the binding of *execution agent*s to the underlying *execution resource*s.
 
-bulk_execution_affinity_t provides nested property types and objects as described below.
+bulk_execution_affinity_t provides nested property types and objects as described below. These properties are behavioral properties as described in [[22]][p0443r4] so must adhere to the requirements of behavioral properties and the requirements described below.
 
 | Nested Property Type | Nested Property Name | Requirements |
 |----------------------|----------------------|--------------|
@@ -629,6 +629,21 @@ The free function `this_thread::get_resource` is provided for retrieving the `ex
 
 # Future Work
 
+## How should we define the execution context?
+
+This paper currently defines the execution context as a concrete type which provides the essential interface requires to be constructed from an `execution_resource` and to provide an affinity-based `allocator` or `pmr::memory_resource` and `executor`.
+
+However going forward there are a few different directions the execution context could take:
+* A) The execution context could be **the** standard execution context type, which can be used polymorphically in place of any concrete execution context type in a similar way to the polymorphic executor [[22]][p0443r4]. This approach allows it to interoperate well with any concrete execution context type, however it may be very difficult to define exactly what this type should look like as the different kinds of execution contexts are still being developed and all the different requirements are still to be fully understood.
+* B) The execution context could be a concrete executor type itself, used solely for the purpose of being constructed from and managing a set of `execution_resource`s. This approach would allow the execution context to be tailored specific for it's original purpose, however it would be more difficult to support interoperability with other concrete execution context types.
+* C) The execution context could be simply a concept, similar to `OnewayExecutor` or `BulkExecutor`, for executors, where it requires the execution context type to provide the required interface for managing *execution_resource*s. This approach would allow for any concrete execution context type to support necessary interface for managing execution resources by simply implementing the requirements of the concept, and would avoid defining any concrete or generic execution context type.
+
+| Straw Poll |
+|------------|
+| Should the execution context be a generic polymorphic execution context, as described above in option A? |
+| Should the execution context be a concrete type specifically for the purpose of managing execution resources, as described above in option B? |
+| Should the execution context be a concept, as described above in option C? |
+
 ## Who should have control over bulk execution affinity?
 
 This paper currently proposes the `bulk_execution_affinity_t` properties and it's nested properties for allowing an *executor* to make guarantees as to how *execution agent*s are bound to the underlying *execution resource*s. However providing control at this level may lead to *execution agent*s being bound to *execution resource*s within a critical path. A possible solution to this is to allow the *execution context* to be configured with `bulk_execution_affinity_t` nested properties, either instead of the *executor* property or in addition. This would allow the binding of *threads of execution* to be performed at the time of the *execution context* creation.
@@ -649,7 +664,7 @@ With the ability to place memory with affinity comes the ability to define algor
 
 ## Level of abstraction
 
-The current proposal provides an interface for querying whether an `execution_resource` can allocate and/or execute work, it can provide the concurrency it supports and it can provide a name. We also provide the `affinity_query` structure for querying the relative affinity metrics between two `execution_resource`s. However, this may not be enough information for users to take full advantage of the system. For example, they may also want to know what kind of memory is available or the properties by which work is executed. We decided that attempting to enumerate the various hardware components would not be ideal, as that would make it harder for implementers to support new hardware. We think a better approach would be to parameterize the additional properties of hardware such that hardware queries could be much more generic.
+The current proposal provides an interface for querying whether an `execution_resource` can allocate and/or execute work, it can provide the concurrency it supports and it can provide a name. We also provide the `affinity_query` structure for querying the relative affinity metrics between two `execution_resource`s. However, this may not be enough information for users to take full advantage of the system. For example, they may also want to know what kind of memory is available or the properties by which work is executed. We decided that attempting to enumerate the various hardware components would not be ideal, as that would make it harder for implementors to support new hardware. We think a better approach would be to parameterize the additional properties of hardware such that hardware queries could be much more generic.
 
 We may wish to mirror the design of the executors proposal and have a generic query interface using properties for querying information about an `execution_resource`. We expect that an implementation may provide additional nonstandard, implementation-specific queries.