fix: issue 321 (#323)

* fix: adding additional blank line before inner function

* fix: removing blank line between two preceding comment blocks

* fix: mangling long description when splitting multi-sentence summary

* fix: not detecting string constants

* feat: add function to find inline reST markup

* test: add tests for function to find inline reST markup

* fix: fix tests that failed after updating regex's

As docformatter becomes more strict in identifying patterns and handling the various patterns that are identified, some of the older tests are failing.  These tests fail because the patterns in the test strings are not strictly reST, Sphinx, etc.

* doc: add section discussing text patterns to usage docs

* doc: add assistance section to the README

* fix: use sys module to get default file encoding

* fix: explicitly ignore utf_16 and utf_32 encoding

Author: weibullguy

README.rst

@@ -199,6 +199,17 @@ Do you use *docformatter*?  What style docstrings do you use?  Add some badges t
     .. image:: https://img.shields.io/badge/%20style-google-3666d6.svg
         :target: https://google.github.io/styleguide/pyguide.html#s3.8-comments-and-docstrings
 
+Assistance
+==========
+``docformatter`` has an IRC channel on `Libera.Chat`_ in the `#docformatter`_ room.
+ .. _`Libera.Chat`: https://libera.chat
+ .. _`#docformatter`: https://web.libera.chat/#docformatter
+
+There is no ``docformatter`` channel on the Python Code Quality Discord server, but
+you can ask for help in the `# general`_ channel.
+
+ .. _`# general`: https://discord.com/channels/825463413634891776/934197425357336596
+
 Issues
 ======
 

docs/source/conf.py

@@ -3,11 +3,11 @@
 #
 # For the full list of built-in configuration values, see the documentation:
 # https://www.sphinx-doc.org/en/master/usage/configuration.html
-
 # -- Project information -----------------------------------------------------
 # https://www.sphinx-doc.org/en/master/usage/configuration.html#project-information
 """Configuration file for the Sphinx documentation builder."""
 
+
 project = "docformatter"
 copyright = "2022-2023, Steven Myint"
 author = "Steven Myint"

docs/source/usage.rst

@@ -151,3 +151,52 @@ Use with GitHub Actions
 action.
 
 .. _`python-lint-plus`: https://github.com/marketplace/actions/python-code-style-quality-and-lint
+
+Dostring Text Patterns
+======================
+
+``docformatter`` began as a simple tool to format docstrings to follow PEP257.  It
+was originally a single Python script of 118 lines containing seven functions.
+That's no longer the case as an inspection of the codebase will show.  Over time,
+``docformatter`` has grown to include a number of features that have been requested
+by its most fantastic user base.
+
+In the early days, ``docformatter`` only formatted simple docstrings.  "Complex" text
+patterns like lists, parameter descriptions, and reStructuredText (reST) sections
+caused ``docformatter`` to simply skip formatting the docstring.  As feature requests
+have been and will be incorporated, ``docformatter`` has gained the ability to
+recognize and format more complex text patterns.
+
+As a result, it is necessary for the user to properly format their docstrings to
+follow the patterns documented in the various specifications.  These specifications
+would include:
+
+- PEP 257 - Docstring Conventions
+    https://www.python.org/dev/peps/pep-0257/
+- reStructuredText (reST) Markup Specification
+    https://docutils.sourceforge.io/docs/ref/rst/restructuredtext.html
+- Sphinx Documentation Style
+    https://www.sphinx-doc.org/en/master/usage/restructuredtext/basics.html
+- Epydoc Documentation Style
+    http://epydoc.sourceforge.net/manual-fields.html
+
+Any docstring that does not follow these specifications may not be formatted properly
+as these patterns may be recognized by ``docformatter`` as simple text that needs to
+formatted.  For example, if a user writes a docstring that contains a list but does not
+format the list according to reST specifications, ``docformatter`` may not recognize
+the list and may format the list items as simple text.  This could result in a
+list that is not properly indented or wrapped.
+
+The user is encouraged to read and follow these specifications when writing
+docstrings to ensure that ``docformatter`` can properly format them.  Issues reported
+to the ``docformatter`` project that are the result of docstrings not following these
+specifications will be closed as ``S:wontfix`` with a request for the user to update
+their docstrings to follow the specifications.
+
+Additionally, as ``docformatter`` continues to add support for more text patterns (e.g.,
+Numpy or Google style docstrings), new releases may result in significant docstring
+formatting changes in your code base.  While we hate to see this happen to our users,
+it is the result of our desire to make ``docformatter`` the best tool it can be for
+formatting docstrings and the best way to achieve that is to strigently comply with
+the various specifications.  We appreciate your understanding and patience as we
+continue to improve ``docformatter``.

src/docformatter/classify.py

@@ -28,6 +28,7 @@
 
 
 # Standard Library Imports
+import re
 import sys
 import tokenize
 from tokenize import TokenInfo
@@ -289,9 +290,9 @@ def is_definition_line(token: tokenize.TokenInfo) -> bool:
         True if the token is a definition line, False otherwise.
     """
     if token.type == tokenize.NAME and (
-        token.line.strip().startswith("def ")
-        or token.line.strip().startswith("async ")
-        or token.line.strip().startswith("class ")
+        token.line.startswith("def ")
+        or token.line.startswith("async ")
+        or token.line.startswith("class ")
     ):
         return True
 
@@ -407,14 +408,7 @@ def is_nested_definition_line(token: tokenize.TokenInfo) -> bool:
     bool
         True if the token is a nested definition line, False otherwise.
     """
-    if token.type == tokenize.NAME and (
-        token.line.startswith("    def ")
-        or token.line.startswith("    async ")
-        or token.line.startswith("    class ")
-    ):
-        return True
-
-    return False
+    return re.match(r"^ {4,}(async|class|def) ", token.line) is not None
 
 
 def is_newline_continuation(

src/docformatter/constants.py

@@ -76,9 +76,12 @@
 OPTION_REGEX = r"^ {0,}-{1,2}[\S ]+ \w+"
 """Regular expression to use for finding option lists."""
 
-REST_REGEX = r"((\.{2}|`{2}) ?[\w.~-]+(:{2}|`{2})?[\w ]*?|`[\w.~]+`)"
+REST_DIRECTIVE_REGEX = r"^( {0,}\.\. .+?:{1,2}.*\n(?:[ \t]{1,}.*\n|\n)*)"
 """Regular expression to use for finding reST directives."""
 
+REST_INLINE_REGEX = r"(?<!^\.{2} )(:)*([*]{1,2}|_?[`]{1,2}|[|\[])[\w <>:.-]+([*]{1,2}|[`]{1,2}_?|[|]|[\]]_?)"  # noqa: E501
+"""Regular expression to use for finding inline reST markup."""
+
 REST_SECTION_REGEX = (
     r"(^ *[#\*=\-^\'\"\+_\~`\.\:]+\n)?[\w ]+\n *[#\*=\-^\'\"\+_\~`\.\:]+"
 )
@@ -153,7 +156,8 @@
 )
 """The URL patterns to look for when finding links.
 
-Based on the table at <https://en.wikipedia.org/wiki/List_of_URI_schemes>
+Based on the table at
+<https://en.wikipedia.org/wiki/List_of_URI_schemes>
 """
 
 # This is the regex used to find URL links:

src/docformatter/encode.py

@@ -47,7 +47,7 @@ class Encoder:
     CRLF = "\r\n"
 
     # Default encoding to use if the file encoding cannot be detected
-    DEFAULT_ENCODING = "latin-1"
+    DEFAULT_ENCODING = sys.getdefaultencoding()
 
     def __init__(self):
         """Initialize an Encoder instance."""
@@ -64,9 +64,15 @@ def do_detect_encoding(self, filename) -> None:
         """
         try:
             detection_result = from_path(filename).best()
-            self.encoding = (
-                detection_result.encoding if detection_result else self.DEFAULT_ENCODING
-            )
+            if detection_result and detection_result.encoding in ["utf_16", "utf_32"]:
+                # Treat undetectable/binary encodings as failure
+                self.encoding = self.DEFAULT_ENCODING
+            else:
+                self.encoding = (
+                    detection_result.encoding
+                    if detection_result
+                    else self.DEFAULT_ENCODING
+                )
 
             # Check for correctness of encoding.
             with self.do_open_with_encoding(filename) as check_file: