Internal change: Replace proto_casters.h with a temporary fake/stub (forwarding to native_proto_caster.h).

PiperOrigin-RevId: 458227296

Author: rwgk

README.md

@@ -41,31 +41,58 @@ PYBIND11_MODULE(my_module, m) {
 ```
 
 
-## Wrapped C++ vs Python Native Types
-
-When passing protos between C++ and python, these bindings will convert the
-protocol buffer into the native python type, which usually involves protocol
-buffer serialization. The objects returned to python are the native python
-protocol buffer objects, and so `isinstance()` and normal protocol buffer
-operations and methods will behave as expected.
+## C++ Native vs Python Native Types
+
+When passing protos between C++ and Python, the `native_proto_caster.h`
+bindings will convert protobuf objects to the native type on either side.
+
+While C++ has only one native type, Python has two native types
+(https://rules-proto-grpc.com/en/latest/lang/python.html):
+
+* `--define=use_fast_cpp_protos=false` (aka `use_pure_python_protos`)
+* `--define=use_fast_cpp_protos=true`
+
+With `use_pure_python_protos`, protobuf objects passed between C++ and Python
+are always serialized/deserialized between the native C++ type and the pure
+Python native type. This is very safe but also slow.
+
+With `use_fast_cpp_protos`, the native Python type is internally backed by
+the native C++ type, which unlocks various performance benefits, even when
+only used from Python. When passing protobuf objects between Python and C++,
+in certain situations the serialization/deserialization overhead can be
+avoided, but not universally. Fundamentally, sharing C++ native protobuf
+objects between C++ and Python is unsafe because C++ assumes that it has
+exclusive ownership and may manipulate references in a way that undermines
+Python's much safer ownership semantics. Because of this, sharing mutable
+references or pointers between C++ and Python is not allowed. However, when
+passing a Python protobuf object to C++, the bindings may share the underlying
+C++ native protobuf object with C++ when passed by `const &` or `const *`.
+
+### Protobuf Extensions
+
+With `use_fast_cpp_protos`, any `py_proto_library` becomes implicitly dependent
+on the corresponding `cc_proto_library`, but this is currently not handled
+automatically, nor clearly diagnosed or formally enforced. In the absence of
+protobuf extensions
+usually there is no problem: Python code tends to depend organically on
+a given `py_proto_library`, and pybind11-wrapped C++ code tends to depend
+organically on the corresponding `cc_proto_library`. However, when extensions
+are involved, a well-known pitfall is to accidentally omit the corresponding
+`cc_proto_library`. Currently this needs to be kept in mind as a pitfall
+(sorry), but the usual best-practice unit testing is likely to catch such
+situations. Once discovered, the fix is easy: add the `cc_proto_library`
+to the `deps` of the relevant `pybind_library` or `pybind_extension`.
+
+### Enumerations
 
 Enumerations are passed and returned as integers. You may use the enum values
 from the native python proto module to set and check the enum values used
 by a bound proto enum (see `tests/proto_enum_test.py` for an example).
 
-Fundamentally sharing protocol buffer types between C++ and python runtimes
-is unsafe because C++ assumes that it has full ownership of a protocol buffer
-message, and may manipulate references in a way that undermines python
-ownership semantics. Because of this sharing mutable references or pointers
-between C++ and python is not allowed. However when using the python fast
-proto implementation, the bindings may share an underlying protocol buffer
-with C++ when passed by `const &` into a C++ function.
+### In / Out Parameters
 
 In cases where a protocol buffer is used as an in/out parameter in C++,
-additional logic will be required in the wrapper.
-
-
-### In / Out example
+additional logic will be required in the wrapper. For example:
 
 ```cpp
 #include <pybind11/pybind11.h>
@@ -87,39 +114,15 @@ PYBIND11_MODULE(my_module, m) {
 ```
 
 
-## Integration with the Old Way
-
-Due to the nature of pybind11, extension modules built using `pybind11_protobuf/native_proto_caster.h`
-cannot interoperate with the older bindings (those modules built using `pybind11_protobuf/proto_casters.h`)
-as that may generate duplicate pybind11 specializations for a given protocol buffer
-which would violate C++ ODR rules in an undetectable way. In order to workaround
-that failure mode a nearly-transparent wrapper is also provided.
-
-To use the wrappers:
-
-1. Include the header file `pybind11_protobuf/wrapped_proto_caster.h`
-   in the .cc file with your bindings.
-1. Call `pybind11_protobuf::ImportWrappedProtoCasters();` in your `PYBIND11_MODULE` definition.
-1. Wrap the C++ code with `pybind11_protobuf::WithWrappedProtos`.
-
-### Wrapped Example
-
-```cpp
-#include <pybind11/pybind11.h>
-
-#include "path/to/my/my_message.proto.h"
-#include "pybind11_protobuf/wrapped_proto_caster.h"
-
-// In real use, these 2 functions would probably be defined in a python-agnostic library.
-MyMessage ReturnMyMessage() { ... }
-void TakeMyMessage(const MyMessage& in) { ... }
-
-PYBIND11_MODULE(my_module, m) {
-  pybind11_protobuf::ImportWrappedProtoCasters();
+### `pybind11_protobuf/wrapped_proto_caster.h`
 
-  using pybind11_protobuf::WithWrappedProtos;
+TL;DR: Ignore `wrapped_proto_caster.h` if you can, this header was added as
+a migration aid before the removal of `proto_casters.h`.
 
-  m.def("return_my_message", WithWrappedProtos(&ReturnMyMessage));
-  m.def("take_my_message", WithWrappedProtos(&TakeMyMessage), pybind11::arg("in"));
-}
-```
+Historic background: Due to the nature of pybind11, extension modules
+built using `native_proto_caster.h` could not interoperate with the
+older `proto_casters.h` bindings, as that would have led to C++ ODR
+violations. `wrapped_proto_caster.h` is a nearly-transparent wrapper for
+`native_proto_caster.h` to work around the ODR issue. With the migration to
+`native_proto_caster.h` now completed, `wrapped_proto_caster.h` is obsolete
+and will be removed in the future.

pybind11_protobuf/BUILD

@@ -22,10 +22,6 @@ pybind_library(
     name = "native_proto_caster",
     srcs = ["native_proto_caster.cc"],
     hdrs = ["native_proto_caster.h"],
-    defines = [
-        # Force a build error when build variants are mixed.
-        "PYBIND11_PROTOBUF_MODE=native_protos",
-    ],
     visibility = [
         "//visibility:public",
     ],

pybind11_protobuf/proto_casters.h

@@ -0,0 +1,30 @@
+// Minimal fake/stub to be used as a means to globally switch all dependents
+// from proto_casters.h to native_proto_caster.h (b/229395875).
+
+#ifndef PYBIND11_PROTOBUF_PROTO_CASTERS_H_
+#define PYBIND11_PROTOBUF_PROTO_CASTERS_H_
+
+// Satisfy existing transitive dependencies. To be cleaned up along with
+// replacing the proto_casters.h includes.
+#include <pybind11/functional.h>
+#include <pybind11/stl.h>
+
+// Pull in the new caster.
+#include "pybind11_protobuf/native_proto_caster.h"
+
+namespace pybind11 {
+namespace google {
+
+inline void ImportProtoModule() {
+  pybind11_protobuf::ImportNativeProtoCasters();
+}
+
+template <typename ProtoType>
+void RegisterProtoMessageType(module_) {
+  // NOOP
+}
+
+}  // namespace google
+}  // namespace pybind11
+
+#endif  // PYBIND11_PROTOBUF_PROTO_CASTERS_H_