Easier field definitions

`attr()` and `child()` are no longer required, and are inferred. The type resolving has been improved, I think it's more robust now, although incorrectly defined attributes will be harder to debug.

Author: tobywf

.pylintrc

@@ -22,7 +22,7 @@ disable=
 
 [BASIC]
 
-good-names=_,e,el,ex,f,tp,v
+good-names=_,e,el,ex,f,tp,k,v,ns
 
 [FORMAT]
 

README.md

@@ -22,30 +22,29 @@ Requires Python 3.7 or higher.
 ```python
 from lxml import etree
 from typing import List
-from xml_dataclasses import xml_dataclass, attr, child, load, dump
+from xml_dataclasses import xml_dataclass, rename, load, dump
 
 CONTAINER_NS = "urn:oasis:names:tc:opendocument:xmlns:container"
 
 @xml_dataclass
 class RootFile:
     __ns__ = CONTAINER_NS
-    full_path: str = attr(rename="full-path")
-    media_type: str = attr(rename="media-type")
+    full_path: str = rename(name="full-path")
+    media_type: str = rename(name="media-type")
 
 
 @xml_dataclass
 class RootFiles:
     __ns__ = CONTAINER_NS
-    rootfile: List[RootFile] = child()
+    rootfile: List[RootFile]
 
 
 @xml_dataclass
 class Container:
     __ns__ = CONTAINER_NS
-    version: str = attr()
-    rootfiles: RootFiles = child()
+    version: str
+    rootfiles: RootFiles
     # WARNING: this is an incomplete implementation of an OPF container
-    # (it's missing links)
 
 
 if __name__ == "__main__":
@@ -62,10 +61,61 @@ if __name__ == "__main__":
 * Convert XML documents to well-defined dataclasses, which should work with IDE auto-completion
 * Loading and dumping of attributes, child elements, and text content
 * Required and optional attributes and child elements
-* Lists of child elements are supported
+* Lists of child elements are supported, as are unions and lists or unions
 * Inheritance does work, but has the same limitations as dataclasses. Inheriting from base classes with required fields and declaring optional fields doesn't work due to field order. This isn't recommended
 * Namespace support is decent as long as correctly declared. I've tried on several real-world examples, although they were known to be valid. `lxml` does a great job at expanding namespace information when loading and simplifying it when saving
-* Union child types are supported. When loading XML, they are attempted to be parsed in order
+
+## Patterns
+
+### Defining attributes
+
+Attributes can be either `str` or `Optional[str]`. Using any other type won't work. Attributes can be renamed or have their namespace modified via the `rename` function. It can be used either on its own, or with an existing field definition:
+
+```python
+@xml_dataclass
+class Foo:
+    __ns__ = None
+    required: str
+    optional: Optional[str] = None
+    renamed_with_default: str = rename(default=None, name="renamed-with-default")
+    namespaced: str = rename(ns="http://www.w3.org/XML/1998/namespace")
+    existing_field: str = rename(field(...), name="existing-field")
+```
+
+I would like to add support for validation in future, which might also make it easier to support other types. For now, you can work around this limitation with properties that do the conversion.
+
+### Defining text
+
+Like attributes, text can be either `str` or `Optional[str]`. You must declare text content with the `text` function. Similar to `rename`, this function can use an existing field definition, or take the `default` argument. Text cannot be renamed or namespaced. Every class can only have one field defining text content. If a class has text content, it cannot have any children.
+
+```python
+@xml_dataclass
+class Foo:
+    __ns__ = None
+    value: str = text()
+
+@xml_dataclass
+class Foo:
+    __ns__ = None
+    content: Optional[str] = text(default=None)
+
+@xml_dataclass
+class Foo:
+    __ns__ = None
+    uuid: str = text(field(default_factory=lambda: str(uuid4())))
+```
+
+### Defining children/child elements
+
+Children must ultimately be other XML dataclasses. However, they can also be `Optional`, `List`, and `Union` types:
+
+* `Optional` must be at the top level. Valid: `Optional[List[XmlDataclass]]`. Invalid: `List[Optional[XmlDataclass]]`
+* Next, `List` should be defined (if multiple child elements are allowed). Valid: `List[Union[XmlDataclass1, XmlDataclass2]]`. Invalid: `Union[List[XmlDataclass1], XmlDataclass2]`
+* Finally, if `Optional` or `List` were used, a union type should be the inner-most (again, if needed)
+
+Children can be renamed via the `rename` function, however attempting to set a namespace is invalid, since the namespace is provided by the child type's XML dataclass. Also, unions of XML dataclasses must have the same namespace (you can use different fields if they have different namespaces).
+
+If a class has children, it cannot have text content.
 
 ## Gotchas
 
@@ -79,19 +129,24 @@ parser = etree.XMLParser(remove_blank_text=True)
 
 By default, `lxml` preserves whitespace. This can cause a problem when checking if elements have no text. The library does attempt to strip these; literally via Python's `strip()`. But `lxml` is likely faster and more robust.
 
-## Limitations and Assumptions
+### Optional vs required
+
+On dataclasses, optional fields also usually have a default value to be useful. But this isn't required; `Optional` is just a type hint to say `None` is allowed.
+
+For XML dataclasses, on loading/deserialisation, whether or not a field is required is determined by if it has a `default`/`default_factory` defined. If so, and it's missing, that default is used. Otherwise, an error is raised.
+
+For dumping/serialisation, the default isn't considered. Instead, if a value is marked as `Optional` and the value is `None`, it isn't written.
+
+This makes sense in many cases, but possibly not every case.
+
+### Other limitations and Assumptions
 
 Most of these limitations/assumptions are enforced. They may make this project unsuitable for your use-case.
 
-* All attributes are strings, no extra validation is performed. I would like to add support for validation in future, which might also make it easier to support other types
-* Elements can either have child elements or text content, not both
-* Child elements are other XML dataclasses
-* Text content is a string
 * It isn't possible to pass any parameters to the wrapped `@dataclass` decorator
-* Some properties of dataclass `field`s are not exposed: `default_factory`, `repr`, `hash`, `init`, `compare`. For most, it is because I don't understand the implications fully or how that would be useful for XML. `default_factory` is hard only because of [the overloaded type signatures](https://github.com/python/typeshed/blob/master/stdlib/3.7/dataclasses.pyi), and getting that to work with `mypy`
+* Setting the `init` parameter of a dataclass' `field` will lead to bad things happening, this isn't supported
 * Deserialisation is strict; missing required attributes and child elements will cause an error. I want this to be the default behaviour, but it should be straightforward to add a parameter to `load` for lenient operation
 * Dataclasses must be written by hand, no tools are provided to generate these from, DTDs, XML schema definitions, or RELAX NG schemas
-* Union types must have the same element/tag name and namespace. Otherwise, two different dataclass attributes (XML child fields) may be used
 
 ## Development
 

functional/container_test.py

@@ -1,9 +1,10 @@
 from pathlib import Path
 from typing import List
+
 import pytest
 from lxml import etree
 
-from xml_dataclasses import attr, child, dump, load, xml_dataclass
+from xml_dataclasses import dump, load, rename, xml_dataclass
 
 from .utils import lmxl_dump
 
@@ -15,21 +16,21 @@
 @xml_dataclass
 class RootFile:
     __ns__ = CONTAINER_NS
-    full_path: str = attr(rename="full-path")
-    media_type: str = attr(rename="media-type")
+    full_path: str = rename(name="full-path")
+    media_type: str = rename(name="media-type")
 
 
 @xml_dataclass
 class RootFiles:
     __ns__ = CONTAINER_NS
-    rootfile: List[RootFile] = child()
+    rootfile: List[RootFile]
 
 
 @xml_dataclass
 class Container:
     __ns__ = CONTAINER_NS
-    version: str = attr()
-    rootfiles: RootFiles = child()
+    version: str
+    rootfiles: RootFiles
     # WARNING: this is an incomplete implementation of an OPF container
     # (it's missing links)
 

functional/package_test.py

@@ -4,7 +4,7 @@
 
 from lxml import etree
 
-from xml_dataclasses import attr, child, dump, load, text, xml_dataclass
+from xml_dataclasses import dump, load, rename, text, xml_dataclass
 
 from .utils import lmxl_dump
 
@@ -29,73 +29,73 @@ def to_dict(cls, default_ns: Optional[str] = None) -> Mapping[Optional[str], str
 class DublinCoreMd:
     __ns__ = NsMap.dc.value
     value: str = text()
-    id: Optional[str] = attr(default=None)
+    id: Optional[str] = None
 
 
 @xml_dataclass
 class MdMeta3:
     __ns__ = NsMap.opf.value
 
-    property: str = attr()
+    property: str
     value: str = text()
 
 
 @xml_dataclass
 class MdMeta2:
     __ns__ = NsMap.opf.value
-    content: str = attr()
-    name: str = attr()
+    content: str
+    name: str
 
 
 @xml_dataclass
 class Metadata3:
     __ns__ = NsMap.opf.value
 
-    identifier: List[DublinCoreMd] = child()
-    title: List[DublinCoreMd] = child()
-    language: List[DublinCoreMd] = child()
-    meta: List[Union[MdMeta3, MdMeta2]] = child()
+    identifier: List[DublinCoreMd]
+    title: List[DublinCoreMd]
+    language: List[DublinCoreMd]
+    meta: List[Union[MdMeta3, MdMeta2]]
 
 
 @xml_dataclass
 class Item3:
     __ns__ = NsMap.opf.value
-    id: str = attr()
-    href: str = attr()
-    media_type: str = attr(rename="media-type")
+    id: str
+    href: str
+    media_type: str = rename(name="media-type")
 
 
 @xml_dataclass
 class Manifest3:
     __ns__ = NsMap.opf.value
-    item: List[Item3] = child()
+    item: List[Item3]
 
 
 @xml_dataclass
 class ItemRef3:
     __ns__ = NsMap.opf.value
-    idref: str = attr()
-    properties: Optional[str] = attr(default=None)
+    idref: str
+    properties: Optional[str] = None
 
 
 @xml_dataclass
 class Spine3:
     __ns__ = NsMap.opf.value
-    itemref: List[ItemRef3] = child()
-    toc: Optional[str] = attr(default=None)
+    itemref: List[ItemRef3]
+    toc: Optional[str] = None
 
 
 @xml_dataclass
 class Package3:
     __ns__ = NsMap.opf.value
-    version: str = attr()
-    unique_identifier: str = attr(rename="unique-identifier")
-    metadata: Metadata3 = child()
-    manifest: Manifest3 = child()
-    spine: Spine3 = child()
-    id: Optional[str] = attr(default=None)
-    lang: Optional[str] = attr(default=None, namespace=NsMap.xml.value)
-    dir: Optional[str] = attr(default=None)
+    version: str
+    unique_identifier: str = rename(name="unique-identifier")
+    metadata: Metadata3
+    manifest: Manifest3
+    spine: Spine3
+    id: Optional[str] = None
+    lang: Optional[str] = rename(default=None, ns=NsMap.xml.value)
+    dir: Optional[str] = None
 
 
 def test_functional_package():

lint

@@ -16,7 +16,7 @@ black $BLACK_CHECK src/ tests/ functional/
 mypy src/xml_dataclasses/ --strict
 pylint src/
 # always output coverage report
-if pytest tests/ --cov=xml_dataclasses --random-order $PYTEST_DEBUG; then
+if pytest tests/ --cov=xml_dataclasses --cov-report term --cov-report html --random-order $PYTEST_DEBUG; then
   coverage html
 else
   coverage html

pyproject.toml

@@ -20,7 +20,6 @@ classifiers = [
 [tool.poetry.dependencies]
 python = "^3.7"
 lxml = "^4.5.0"
-typing_inspect = "^0.5.0"
 
 [tool.poetry.dev-dependencies]
 pytest = "^5.3.5"

src/xml_dataclasses/__init__.py

@@ -2,5 +2,6 @@
 
 logging.getLogger(__name__).addHandler(logging.NullHandler())
 
-from .structs import attr, child, text, xml_dataclass  # isort:skip
+from .modifiers import rename, text  # isort:skip
+from .resolve_types import xml_dataclass  # isort:skip
 from .serde import dump, load  # isort:skip

src/xml_dataclasses/exceptions.py

@@ -0,0 +1,32 @@
+class XmlDataclassError(Exception):
+    pass
+
+
+class XmlDataclassInternalError(XmlDataclassError):
+    pass
+
+
+class XmlDataclassNoNamespaceError(XmlDataclassError):
+    MESSAGE = "XML dataclass without namespace"
+
+    def __init__(self) -> None:
+        super().__init__(self.MESSAGE)
+
+
+class XmlDataclassModelError(XmlDataclassError):
+    pass
+
+
+class XmlDataclassContentsError(XmlDataclassModelError):
+    MESSAGE = "XML dataclass with text-only content has children declared"
+
+    def __init__(self) -> None:
+        super().__init__(self.MESSAGE)
+
+
+class XmlDataclassDuplicateFieldError(XmlDataclassModelError):
+    pass
+
+
+class XmlTypeError(XmlDataclassModelError):
+    pass

src/xml_dataclasses/modifiers.py

@@ -0,0 +1,42 @@
+# pylint: disable=unsubscriptable-object
+# unsubscriptable-object clashes with type hints
+from __future__ import annotations
+
+from dataclasses import _MISSING_TYPE, MISSING, Field, field
+from typing import TYPE_CHECKING, Optional, TypeVar, Union, cast
+
+_T = TypeVar("_T")
+
+
+def make_field(default: Union[_T, _MISSING_TYPE]) -> Field[_T]:
+    if TYPE_CHECKING:  # pragma: no cover
+        return cast(Field[_T], field(default=default))
+    return field(default=default)
+
+
+def rename(
+    f: Optional[Field[_T]] = None,
+    default: Union[_T, _MISSING_TYPE] = MISSING,
+    name: Optional[str] = None,
+    ns: Optional[str] = None,
+) -> Field[_T]:
+    if f is None:
+        f = make_field(default=default)
+    metadata = dict(f.metadata)
+    if name:
+        metadata["xml:name"] = name
+    if ns:
+        metadata["xml:ns"] = ns
+    f.metadata = metadata
+    return f
+
+
+def text(
+    f: Optional[Field[_T]] = None, default: Union[_T, _MISSING_TYPE] = MISSING
+) -> Field[_T]:
+    if f is None:
+        f = make_field(default=default)
+    metadata = dict(f.metadata)
+    metadata["xml:text"] = True
+    f.metadata = metadata
+    return f