Pass custom_data through assemblers to kernel functions. #4013

sclaus2 · 2025-12-01T14:16:43Z

Add void* custom_data member to integral_data struct and pass it to assembly functions (matrix, vector, scalar, lifting). Custom_data defaults back to nullptr if no custom_data is set explicitly (with set_custom_data()). The required changes are minimal

Form.h: Add custom_data to integral_data, add accessors
assemble_matrix_impl.h: Pass custom_data through all functions
assemble_vector_impl.h: Pass custom_data through all functions
assemble_scalar_impl.h: Pass custom_data through all functions
Python bindings: Expose set_custom_data() and custom_data()
Add test_custom_data.py with comprehensive tests

This enables integration kernels to receive additional per-cell user data which is essential for example for runtime quadrature rules. With this extensions users are no longer required to completely rewrite all assembly routines for customised C-kernels.

For an example library that relies on custom_data and its integration into the assembler see

https://github.com/sclaus2/runintgen

jhale · 2025-12-01T15:47:57Z

Instead of void* and nullptr it would probably be better to use std::optional<void*>.

jhale · 2025-12-01T15:49:27Z

I say probably because it's a performance critical part of the code - might need a bit of thought.

sclaus2 · 2025-12-01T16:33:43Z

The void* approach should have minimal overhead because:

The kernel function pointer signature hasn't changed, it is
void (kernel)(T, const T*, const T*, const U*, const int*, const uint8_t*, void*)
The call site simply passes the pointer (or nullptr)
kernel(..., custom_data); // Same cost whether nullptr or valid pointer, before: null_ptr

Potential overhead is:

Storing the pointer in integral_data struct (+8 bytes per integral)
Passing the pointer to kernel (already in the signature)

I can run some performance benchmarks of this branch vs 0.10 release. If you have any existing performance benchmarks that you trust let me know.

jhale · 2025-12-01T17:53:09Z

I didn't explain myself very well. In modern C++ it's canonical to use std::optional rather than a pointer and nullptr to represent a value or absence of value. This also tends to wrap nicely into Python.

For example:

std::optional<void*> opt;
void* raw = opt.value_or(nullptr);
// pass raw up to C kernel

schnellerhase · 2025-12-01T21:27:57Z

How about std::any? It is also type agnostic, but type safe, and would get rid of the undesirable void* in the source. For the kernel invocation it can be cast down to void*.

sclaus2 · 2025-12-01T21:33:56Z

I have changed it to std::optional<void*> for now. Open for suggestions.

Replace void* with nullptr pattern with std::optional<void*> for custom_data in the Form class and assembly functions. This is the canonical modern C++ approach for representing optional values. Changes: - Form.h: integral_data::custom_data now uses std::optional<void*> - Form.h: custom_data() returns std::optional<void*> - Form.h: set_custom_data() accepts std::optional<void*> - assemble_*_impl.h: Function parameters use std::optional<void*> - assemble_*_impl.h: Kernel calls use .value_or(nullptr) - Python bindings: Handle std::optional with None support - Test: Update to expect None instead of 0 for unset custom_data Benefits: - Clearer intent: "optional value" vs "magic nullptr" - Type safety: Distinguishes "no value" from "null pointer value" - Pythonic: Returns None instead of 0 when not set - Zero-cost: .value_or(nullptr) compiles to same code

jhale

Looks pretty good - a few small changes to make.

jhale · 2025-12-02T15:23:20Z

python/dolfinx/wrappers/fem.cpp

+          nb::arg("type"), nb::arg("id"), nb::arg("kernel_idx"),
+          nb::arg("data").none(),
+          "Set custom data pointer for an integral. The data pointer is "
+          "passed to the kernel. Pass None to clear.")


I think the last part "Pass None to clear" is unnecessary. Maybe instead explain that None means no data.

jhale · 2025-12-02T15:24:29Z

python/test/unit/fem/test_custom_data.py

+
+
+# Helper intrinsic to cast void* to a typed pointer for custom_data
+@numba.extending.intrinsic


Do we already have this intrinsic in FFCx?

jhale · 2025-12-02T15:24:58Z

python/test/unit/fem/test_custom_data.py

+        b = numba.carray(b_, (3), dtype=dtype)
+        coordinate_dofs = numba.carray(coords_, (3, 3), dtype=xdtype)
+
+        # Cast void* to float64* and read the scale value


jhale · 2025-12-02T15:26:09Z

python/test/unit/fem/test_custom_data.py

+        coordinate_dofs = numba.carray(coords_, (3, 3), dtype=xdtype)
+
+        # Cast void* to float64* and read the scale value
+        typed_ptr = voidptr_to_float64_ptr(custom_data)


Do we have any more complex examples? For example, an array of float64s index by the current cell. This is what @chrisrichardson mentioned before "a kernel might need to know which cell it's running on".

I saw the complex examples/tests below with a length-two region of memory, but they don't quite hit the point just made.

jhale · 2025-12-02T15:30:36Z

cpp/dolfinx/fem/Form.h

+  }
+
+  /// @brief Set the custom data pointer for an integral.
+  ///


I would like to add a warning here:

"void-pointers are inherently unsafe and cannot be type or bounds checked. Incorrect usage of this feature can and will lead to undefined behaviour and crashes."

jhale · 2025-12-02T15:36:43Z

python/dolfinx/wrappers/fem.cpp

+          [](dolfinx::fem::Form<T, U>& self, dolfinx::fem::IntegralType type,
+             int id, int kernel_idx, std::optional<std::uintptr_t> data)
+          {
+            self.set_custom_data(type, id, kernel_idx,


set_custom_data doesn't quite fit the design of the rest of the class - the custom data pointer should be passed up (or not) at creation time, and then it is immutable (at least, the pointer is - the user could modify the pointed memory of course).

jhale · 2025-12-02T15:40:27Z

python/dolfinx/wrappers/fem.cpp

+          "custom_data",
+          [](const dolfinx::fem::Form<T, U>& self,
+             dolfinx::fem::IntegralType type, int id,
+             int kernel_idx) -> std::optional<std::uintptr_t>


Why are we returning std::uintptr_t and not void*? If there is a good reason there needs to be a comment.

jhale · 2025-12-02T15:43:46Z

python/test/unit/fem/test_custom_data.py

+
+    # Verify the assembly used our custom values
+    # The offset should contribute to each DOF
+    assert la.norm(b) > 0


I suspect this assert always comes up true. Can you do something more precise?

jhale · 2025-12-02T15:49:30Z

How about std::any? It is also type agnostic, but type safe, and would get rid of the undesirable void* in the source. For the kernel invocation it can be cast down to void*.

I don't think nanobind supports std::any.

I'm not sure about the argument regardingvoid* being undesirable. This feature is deliberately type and memory-unsafe, and this clearly signals that the user needs to be very, very careful. I've asked @sclaus2 to add something in the docstring to that effect.

Pass custom_data through assemblers to kernel functions.

fa84113

sclaus2 requested review from chrisrichardson, jhale and jorgensd December 1, 2025 14:17

sclaus2 force-pushed the sclaus2/pass_custom_data_to_assembler branch from 648f439 to cf52114 Compare December 1, 2025 21:36

sclaus2 force-pushed the sclaus2/pass_custom_data_to_assembler branch from cf52114 to 8101685 Compare December 2, 2025 08:25

jhale requested changes Dec 2, 2025

View reviewed changes



		# Helper intrinsic to cast void* to a typed pointer for custom_data
		@numba.extending.intrinsic

Uh oh!

Pass custom_data through assemblers to kernel functions. #4013

Are you sure you want to change the base?

Pass custom_data through assemblers to kernel functions. #4013

Conversation

sclaus2 commented Dec 1, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jhale commented Dec 1, 2025

Uh oh!

jhale commented Dec 1, 2025

Uh oh!

sclaus2 commented Dec 1, 2025

Uh oh!

jhale commented Dec 1, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

schnellerhase commented Dec 1, 2025

Uh oh!

sclaus2 commented Dec 1, 2025

Uh oh!

jhale left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jhale commented Dec 2, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

sclaus2 commented Dec 1, 2025 •

edited

Loading

jhale commented Dec 1, 2025 •

edited

Loading