v3 adds mypy (#187)

* Adds lib for types

* Fixes type errors in format.py

* Adds query_params_suffix to fix type error

* Fixes unset type

* Fixes type errors in schema_configuration

* Adds needed type info in validation.py

* Fixes more type errors in validation.py

* Adds U+T binding to fix type errors

* Improves return type of _get_new_instance_without_conversion

* Changes some typing.Type to type

* Adds validate_base_ in Schema

* Uses Schema.validate_base in api_client

* Removes clashing validate overloads from schemas classes

* Adds more missing types in schema.py

* Adds needed type hint

* Fixes some api_client types

* Fixes type error in update_params_for_auth

* Fixes type errors with vars write_file and new_file

* FIxes type error with serialize methods for header classes

* Fixes type error with None detection

* Type error another none fix

* Fixes type error by using literal for key_prefix in get_security_requirement_object

* Fixes 2 type errors by using literals in get_server_url

* Adds another validate_base signature to fix type errors in dictschema validate invocations

* Improves type hints for DictSchema

* Fixes type error with tuple detection

* Fixes broken python tests by having composed dict schemas use the Schema base class

* Eliminates incompatible sequence overloads

* Fixes 1 type error

* Fixes another type error

* Fixes tuple type error

* Fixes type error for path_to_schemas

* Updates path_to_schemas to store one schema

* Adds _instances type

* Fixes more type errors

* Fixes another type error

* Removes unneeded overload

* Detects and handles addProps=False and only one property

* Removes one sequence usage in template

* Removes another sequence usage

* Relaces last sequence in templates

* Fixes type errors by fixing get_security_requirement_object literal input

* Adds exception raising for statuses in operation handling code, reduces type errors

* Fixes assertion error, adds storage of error status codes for operation

* Adds error status codes to python

* Fixes operation return type

* Fixes get_server_url invocation, adds literals

* Renames all ApiResponse classes to ApiResponse, uses them in operation response

* Fixes type of headers_schema

* Removes dataclass from OpenApiResponse

* Removes unneeded dataclasses from OpenApiResponse and classes it uses

* Fixes skip_deserialization overload

* Prevents parameter collisions

* Adds array output class tuple generic type

* Removes typeddict from responses

* Adds missing elipses for tuple output type

* Adds validate_base sequence back in

* Removes getitem from list schemas

* Removes typeddict from header classes

* Changes order to int float

* Do not generate validate with number schema + enum

* Controls when composed schema validate methods are written, fixes type errors

* Adds type(None) to types definition, fixes type errors

* Turns off getitem, adds optional props

* Writes property methods

* Changes additional_properties to get_additional_property

* Adds get_property method implementation

* Changes template mode to get_property

* Changes property to overloads

* Has overloads use elipses

* Handles the case where there is one required or optional property

* Adds get_additional_property casting

* Only cast individual property outputs

* Ignores pyright reportWildcardImportFromLibrary

* Fixes array self ref type hint

* Makes T param bound to sequences, fixes type errors

* Adds required and optional keys

* Adds unset return for optional property values when there is only one property

* Changes name variable to key

* Removes unused templates

* Adds required property returns in get_property

* Fixes get_property implementation

* Improves get_property implementation

* Changes method name to get_additional_property_

* Adds @Property back

* Convers ref properties back into getters

* Converts get_property back into get_x methods

* Fixes python tests

* Adds unset returning in get_additional_property_

* Adds unset output for many cases

* Adds unset types on all except for array and object models

* Adds array unset return value

* Adds unset types for optional map

* Writes properties with unmodifed names, only writes them if they are valid

* Fixes python tests

* Adds mypy to petstore test

* Fixes type error

* FIxes last mypy errors

* Adds py.typed, improves server imports

* Adds __all__ definitions

* Fixes unset type setting on optional returns

* Fixes additiona prperties return type

* Adds unset case for binary output

* Fixes validate return types

* Adds mapValueSchema

* Refines types of SchemaDict values, supresses mypy response body type error

* Adds get_response method in OpenApiResponse classes

* Fixes java test

* Samples regenerated

* Fixes unit test sample, samples regenerated

Author: spacether

docs/generators/python.md

@@ -35,11 +35,11 @@ These options may be applied as additional-properties (cli) or configOptions (pl
 | Type/Alias | Instantiated By |
 | ---------- | --------------- |
 |array|tuple|
-|boolean|schemas.BoolClass|
-|integer|decimal.Decimal|
-|null|schemas.NoneClass|
-|number|decimal.Decimal|
-|object|frozendict.frozendict|
+|boolean|bool|
+|integer|int|
+|null|None|
+|number|typing.Union[float, int]|
+|object|immutabledict.immutabledict|
 |string|str|
 
 
@@ -85,9 +85,9 @@ These options may be applied as additional-properties (cli) or configOptions (pl
 <li>float</li>
 <li>for</li>
 <li>from</li>
-<li>frozendict</li>
 <li>global</li>
 <li>if</li>
+<li>immutabledict</li>
 <li>import</li>
 <li>in</li>
 <li>int</li>

modules/openapi-json-schema-generator/src/main/java/org/openapijsonschematools/codegen/DefaultCodegen.java

@@ -2338,6 +2338,20 @@ public CodegenSchema fromSchema(Schema p, String sourceJsonPath, String currentJ
             }
         }
         // end of properties that need to be ordered to set correct camelCase jsonPathPieces
+        CodegenSchema additionalProperties = property.additionalProperties;
+        LinkedHashMapWithContext<CodegenKey, CodegenSchema> properties = property.properties;
+        if (additionalProperties != null || properties != null) {
+            CodegenSchema mapValueSchema = new CodegenSchema();
+            if (additionalProperties != null) {
+                mapValueSchema = mapValueSchema.add(additionalProperties);
+            }
+            if (properties != null) {
+                for (CodegenSchema prop: properties.values()) {
+                    mapValueSchema = prop.add(mapValueSchema);
+                }
+            }
+            property.mapValueSchema = mapValueSchema;
+        }
 
         if (currentJsonPath != null) {
             String[] pathPieces = currentJsonPath.split("/");
@@ -2514,7 +2528,10 @@ public CodegenOperation fromOperation(Operation operation, String jsonPath) {
         TreeMap<String, CodegenResponse> nonDefaultResponses = null;
         TreeMap<Integer, CodegenResponse> wildcardCodeResponses = null;
         TreeMap<Integer, CodegenResponse> statusCodeResponses = null;
-        boolean hasErrorResponseObject = false;
+        LinkedHashSet<String> errorStatusCodes = null;
+        LinkedHashSet<Integer> errorWildcardStatusCodes = null;
+        LinkedHashSet<String> nonErrorStatusCodes = null;
+        LinkedHashSet<Integer> nonErrorWildcardStatusCodes = null;
         if (operation.getResponses() != null && !operation.getResponses().isEmpty()) {
             responses = new TreeMap<>();
             for (Map.Entry<String, ApiResponse> operationGetResponsesEntry : operation.getResponses().entrySet()) {
@@ -2545,13 +2562,17 @@ public CodegenOperation fromOperation(Operation operation, String jsonPath) {
                     }
                     int firstNumber = Integer.parseInt(key.substring(0, 1));
                     wildcardCodeResponses.put(firstNumber, r);
-                    if (firstNumber > 3 && r.content != null) {
-                        for (CodegenMediaType cm: r.content.values()) {
-                            if (cm.schema != null) {
-                                hasErrorResponseObject = true;
-                                break;
-                            }
+                    if (firstNumber > 3) {
+                        // store error response code whether on not it has a body
+                        if (errorWildcardStatusCodes == null) {
+                            errorWildcardStatusCodes = new LinkedHashSet<>();
                         }
+                        errorWildcardStatusCodes.add(firstNumber);
+                    } else {
+                        if (nonErrorWildcardStatusCodes == null) {
+                            nonErrorWildcardStatusCodes = new LinkedHashSet<>();
+                        }
+                        nonErrorWildcardStatusCodes.add(firstNumber);
                     }
                     continue;
                 }
@@ -2560,13 +2581,17 @@ public CodegenOperation fromOperation(Operation operation, String jsonPath) {
                 }
                 int statusCode = Integer.parseInt(key);
                 statusCodeResponses.put(statusCode, r);
-                if (statusCode > 299 && r.content != null) {
-                    for (CodegenMediaType cm: r.content.values()) {
-                        if (cm.schema != null) {
-                            hasErrorResponseObject = true;
-                            break;
-                        }
+                if (statusCode > 399) {
+                    // store error response code whether on not it has a body
+                    if (errorStatusCodes == null) {
+                        errorStatusCodes = new LinkedHashSet<>();
+                    }
+                    errorStatusCodes.add(key);
+                } else {
+                    if (nonErrorStatusCodes == null) {
+                        nonErrorStatusCodes = new LinkedHashSet<>();
                     }
+                    nonErrorStatusCodes.add(key);
                 }
             }
 
@@ -2729,7 +2754,10 @@ else if (one.required)
 
         return new CodegenOperation(
                 deprecated,
-                hasErrorResponseObject,
+                nonErrorStatusCodes,
+                nonErrorWildcardStatusCodes,
+                errorStatusCodes,
+                errorWildcardStatusCodes,
                 summary,
                 unescapedDescription,
                 description,

modules/openapi-json-schema-generator/src/main/java/org/openapijsonschematools/codegen/DefaultGenerator.java

@@ -455,6 +455,7 @@ private void generatePathItem(List<File> files, CodegenKey pathKey, CodegenPathI
                 endpointMap.put("pathItem", pathItem);
                 endpointMap.put("httpMethod", httpMethod);
                 endpointMap.put("security", security);
+                endpointMap.put("path", pathKey);
                 generateXs(files, operationJsonPath, CodegenConstants.JSON_PATH_LOCATION_TYPE.OPERATION, CodegenConstants.APIS, endpointMap, true);
 
                 // operation docs

modules/openapi-json-schema-generator/src/main/java/org/openapijsonschematools/codegen/languages/PythonClientCodegen.java

@@ -675,6 +675,7 @@ public void processOpts() {
         supportingFiles.add(new SupportingFile("migration_2_0_0.hbs", "", "migration_2_0_0.md"));
         supportingFiles.add(new SupportingFile("migration_other_python_generators.hbs", "", "migration_other_python_generators.md"));
         supportingFiles.add(new SupportingFile("__init__package.hbs", packagePath(), "__init__.py"));
+        supportingFiles.add(new SupportingFile("__init__.hbs", packagePath(), "py.typed"));
 
         if (!generateSourceCodeOnly) {
             supportingFiles.add(new SupportingFile("tox.hbs", "", "tox.ini"));
@@ -727,6 +728,7 @@ public void processOpts() {
         supportingFiles.add(new SupportingFile("schemas/schema.hbs", packagePath() + File.separator + "schemas", "schema.py"));
         supportingFiles.add(new SupportingFile("schemas/schemas.hbs", packagePath() + File.separator + "schemas", "schemas.py"));
         supportingFiles.add(new SupportingFile("schemas/format.hbs", packagePath() + File.separator + "schemas", "format.py"));
+        supportingFiles.add(new SupportingFile("schemas/original_immutabledict.hbs", packagePath() + File.separator + "schemas", "original_immutabledict.py"));
         supportingFiles.add(new SupportingFile("security_schemes.hbs", packagePath(), "security_schemes.py"));
         supportingFiles.add(new SupportingFile("server.hbs", packagePath(), "server.py"));
 

modules/openapi-json-schema-generator/src/main/java/org/openapijsonschematools/codegen/model/CodegenOperation.java

@@ -25,7 +25,10 @@
 
 public class CodegenOperation {
     public final Boolean deprecated;
-    public final boolean hasErrorResponseObject; // if 4xx, 5xx responses have at least one error object defined
+    public final LinkedHashSet<String> nonErrorStatusCodes; // values like 201
+    public final LinkedHashSet<Integer> nonErrorWildcardStatusCodes; // values like 2 for @2XX
+    public final LinkedHashSet<String> errorStatusCodes; // values like 401
+    public final LinkedHashSet<Integer> errorWildcardStatusCodes; // values like 4 for 4XX
     public final String summary, unescapedDescription, description;
     public final LinkedHashSet<String> produces;
     public final List<CodegenServer> servers;
@@ -55,9 +58,12 @@ public class CodegenOperation {
     public final CodegenKey operationId;
     public final CodegenKey jsonPathPiece;
 
-    public CodegenOperation(Boolean deprecated, boolean hasErrorResponseObject, String summary, String unescapedDescription, String description, LinkedHashSet<String> produces, List<CodegenServer> servers, CodegenRequestBody requestBody, List<CodegenSchema> requestBodySchemas, List<CodegenParameter> allParams, List<CodegenParameter> pathParams, CodegenSchema pathParameters, List<CodegenParameter> queryParams, CodegenSchema queryParameters, List<CodegenParameter> headerParams, CodegenSchema headerParameters, List<CodegenParameter> cookieParams, CodegenSchema cookieParameters, boolean hasRequiredParamOrBody, boolean hasOptionalParamOrBody, List<HashMap<String, CodegenSecurityRequirementValue>> security, Map<String, CodegenTag> tags, TreeMap<String, CodegenResponse> responses, TreeMap<Integer, CodegenResponse> statusCodeResponses, TreeMap<Integer, CodegenResponse> wildcardCodeResponses, TreeMap<String, CodegenResponse> nonDefaultResponses, CodegenResponse defaultResponse, List<CodegenCallback> callbacks, ExternalDocumentation externalDocs, Map<String, Object> vendorExtensions, CodegenKey operationId, CodegenKey jsonPathPiece) {
+    public CodegenOperation(Boolean deprecated, LinkedHashSet<String> nonErrorStatusCodes, LinkedHashSet<Integer> nonErrorWildcardStatusCodes, LinkedHashSet<String> errorStatusCodes, LinkedHashSet<Integer> errorWildcardStatusCodes, String summary, String unescapedDescription, String description, LinkedHashSet<String> produces, List<CodegenServer> servers, CodegenRequestBody requestBody, List<CodegenSchema> requestBodySchemas, List<CodegenParameter> allParams, List<CodegenParameter> pathParams, CodegenSchema pathParameters, List<CodegenParameter> queryParams, CodegenSchema queryParameters, List<CodegenParameter> headerParams, CodegenSchema headerParameters, List<CodegenParameter> cookieParams, CodegenSchema cookieParameters, boolean hasRequiredParamOrBody, boolean hasOptionalParamOrBody, List<HashMap<String, CodegenSecurityRequirementValue>> security, Map<String, CodegenTag> tags, TreeMap<String, CodegenResponse> responses, TreeMap<Integer, CodegenResponse> statusCodeResponses, TreeMap<Integer, CodegenResponse> wildcardCodeResponses, TreeMap<String, CodegenResponse> nonDefaultResponses, CodegenResponse defaultResponse, List<CodegenCallback> callbacks, ExternalDocumentation externalDocs, Map<String, Object> vendorExtensions, CodegenKey operationId, CodegenKey jsonPathPiece) {
         this.deprecated = deprecated;
-        this.hasErrorResponseObject = hasErrorResponseObject;
+        this.nonErrorStatusCodes = nonErrorStatusCodes;
+        this.nonErrorWildcardStatusCodes = nonErrorWildcardStatusCodes;
+        this.errorStatusCodes = errorStatusCodes;
+        this.errorWildcardStatusCodes = errorWildcardStatusCodes;
         this.summary = summary;
         this.unescapedDescription = unescapedDescription;
         this.description = description;

modules/openapi-json-schema-generator/src/main/java/org/openapijsonschematools/codegen/model/CodegenResponse.java

@@ -31,6 +31,17 @@ public class CodegenResponse {
     public final TreeSet<String> imports;
     public final boolean componentModule;
 
+    public CodegenResponse getSelfOrDeepestRef() {
+        if (refInfo == null) {
+            return this;
+        }
+        CodegenResponse refObject = refInfo.ref;
+        while (refObject.refInfo != null) {
+            refObject = refObject.refInfo.ref;
+        }
+        return refObject;
+    }
+
     public CodegenResponse(CodegenKey jsonPathPiece, Map<String, CodegenHeader> headers, CodegenSchema headersObjectSchema, String description, Map<String, Object> vendorExtensions, LinkedHashMap<CodegenKey, CodegenMediaType> content, CodegenRefInfo<CodegenResponse> refInfo, TreeSet<String> imports, boolean componentModule) {
         this.jsonPathPiece = jsonPathPiece;
         this.headers = headers;

modules/openapi-json-schema-generator/src/main/java/org/openapijsonschematools/codegen/model/CodegenSchema.java

@@ -81,6 +81,7 @@ public class CodegenSchema {
     public boolean isBooleanSchemaFalse;  // supports boolean schemas
 
     // Extra needed fields
+    public CodegenSchema mapValueSchema;
     public boolean componentModule;
     public TreeSet<String> imports;
     public CodegenKey jsonPathPiece;
@@ -97,6 +98,7 @@ public class CodegenSchema {
      * used in getAllSchemas to write type definitions for allOfType/anyOfType/oneOfType/propertiesType
      */
     public String instanceType;
+    // used to store the expanded schemas that define a codegenschema in code file
     private ArrayList<CodegenSchema> allSchemas = null;
 
     public boolean hasValidation() {
@@ -134,6 +136,32 @@ public CodegenSchema getSelfOrDeepestRef() {
         return refObject;
     }
 
+    public CodegenSchema add(CodegenSchema other) {
+        CodegenSchema newSchema = new CodegenSchema();
+        if (other == null) {
+            newSchema.types = types;
+            return newSchema;
+        }
+        if (refInfo != null || oneOf != null || anyOf != null || allOf != null || not != null) {
+            return null;
+        }
+        if (other.refInfo != null || other.oneOf != null || other.anyOf != null || other.allOf != null || other.not != null) {
+            return null;
+        }
+        if (types == null) {
+            newSchema.types = other.types;
+            return newSchema;
+        }
+        if (other.types == null) {
+            newSchema.types = types;
+            return newSchema;
+        }
+        LinkedHashSet<String> interSectionTypes = new LinkedHashSet<>(types);
+        interSectionTypes.retainAll(other.types);
+        newSchema.types = interSectionTypes;
+        return newSchema;
+    }
+
     public boolean hasAnyRefs() {
         // todo cache this, also pass in sourceJsonPath because one is looking for external refs
         if (refInfo != null) {
@@ -329,6 +357,8 @@ private void getAllSchemas(ArrayList<CodegenSchema> schemasBeforeImports, ArrayL
             mapOut.requiredProperties = requiredProperties;
             mapOut.additionalProperties = additionalProperties;
             mapOut.mapOutputJsonPathPiece = mapOutputJsonPathPiece;
+            mapOut.properties = properties;
+            mapOut.mapValueSchema = mapValueSchema;
             // inputs needed for Schema validate invocation in new method
             mapOut.mapInputJsonPathPiece = mapInputJsonPathPiece;
             mapOut.jsonPathPiece = jsonPathPiece;
@@ -358,6 +388,8 @@ private void getAllSchemas(ArrayList<CodegenSchema> schemasBeforeImports, ArrayL
             mapOut.optionalProperties = optionalProperties;
             mapOut.additionalProperties = additionalProperties;
             mapOut.mapOutputJsonPathPiece = mapOutputJsonPathPiece;
+            mapOut.properties = properties;
+            mapOut.mapValueSchema = mapValueSchema;
             // inputs needed for Schema validate invocation in new method
             mapOut.mapInputJsonPathPiece = mapInputJsonPathPiece;
             mapOut.jsonPathPiece = jsonPathPiece;
@@ -405,8 +437,10 @@ private void getAllSchemas(ArrayList<CodegenSchema> schemasBeforeImports, ArrayL
             mapOut.instanceType = "propertiesOutputType";
             mapOut.optionalProperties = optionalProperties;
             mapOut.requiredProperties = requiredProperties;
+            mapOut.properties = properties;
             mapOut.additionalProperties = additionalProperties;
             mapOut.mapOutputJsonPathPiece = mapOutputJsonPathPiece;
+            mapOut.mapValueSchema = mapValueSchema;
             // inputs needed for Schema validate invocation in new method
             mapOut.mapInputJsonPathPiece = mapInputJsonPathPiece;
             mapOut.jsonPathPiece = jsonPathPiece;

modules/openapi-json-schema-generator/src/main/resources/python/_helper_required_libraries.hbs

@@ -12,5 +12,7 @@
 {{#if tornado}}
 {{#if quoted}}"{{/if}}tornado >= 4.2{{#if quoted}}",{{/if}}
 {{/if}}
+{{#if quoted}}"{{/if}}types-python-dateutil{{#if quoted}}",{{/if}}
+{{#if quoted}}"{{/if}}types-urllib3{{#if quoted}}",{{/if}}
 {{#if quoted}}"{{/if}}typing_extensions ~= 4.5.0{{#if quoted}}",{{/if}}
 {{#if quoted}}"{{/if}}urllib3 ~= 2.0.a3{{#if quoted}}",{{/if}}