Skip to content

Re-enable all Schemathesis functions and make testing deterministic #951

Description

@juhoinkinen

In PR #879 some functions of Schemathesis testing were disabled:

schema.config.checks.positive_data_acceptance.enabled = False
schema.config.generation.allow_extra_parameters = False

If these are enabled, two tests fail:

________________________________________________ test_openapi_fuzzy[POST /projects/{project_id}/suggest] ________________________________________________
+ Exception Group Traceback (most recent call last):
  |   File "/home/jmminkin/git/Annif/.venv/lib/python3.11/site-packages/hypothesis/core.py", line 2287, in wrapped_test
  |     raise the_error_hypothesis_found
  | schemathesis.core.failures.FailureGroup: Schemathesis found 1 distinct failure
  | 
  | - API rejected schema-compliant request
  | 
  |     Valid data should have been accepted
  |     Expected: 2xx, 401, 403, 404, 409, 5xx
  |     Hint: The request body contains 1 additional property not defined in the schema (``). The server likely rejects unexpected fields. Add `additionalProperties: false` to your schema to prevent this.
  | 
  | [400] Bad Request:
  | 
  |     `{"type": "about:blank", "title": "Bad Request", "detail": "language \"d\ud9d9\uddbc\" not supported by vocabulary", "status": 400}`
  | 
  | Reproduce with:
  | 
  |     curl -X POST -H 'Content-Type: application/x-www-form-urlencoded' -d 'text=d%C2%9F%C2%AD%C3%A6%1B&=%C3%8F%0A&limit=3200206&threshold=1.2279522352668104e-107&language=d%F2%86%96%BC' http://localhost/v1/projects/dummy-en/suggest
  | 
  |  (1 sub-exception)
  +-+---------------- 1 ----------------
    | schemathesis.openapi.checks.RejectedPositiveData: API rejected schema-compliant request
    | 
    | Valid data should have been accepted
    | Expected: 2xx, 401, 403, 404, 409, 5xx
    | Hint: The request body contains 1 additional property not defined in the schema (``). The server likely rejects unexpected fields. Add `additionalProperties: false` to your schema to prevent this.
    +------------------------------------

and

_____________________________________________ test_openapi_fuzzy[POST /projects/{project_id}/suggest-batch] _____________________________________________
+ Exception Group Traceback (most recent call last):
  |   File "/home/jmminkin/git/Annif/tests/test_openapi.py", line 77, in test_openapi_fuzzy
  |     case.call_and_validate()
  |   File "/home/jmminkin/git/Annif/.venv/lib/python3.11/site-packages/schemathesis/generation/case.py", line 601, in call_and_validate
  |     self.validate_response(
  |   File "/home/jmminkin/git/Annif/.venv/lib/python3.11/site-packages/schemathesis/generation/case.py", line 560, in validate_response
  |     raise FailureGroup(_failures, message) from None
  | schemathesis.core.failures.FailureGroup: Schemathesis found 1 distinct failure
  | 
  | - API accepted schema-violating request
  | 
  |     Invalid data should have been rejected
  |     Expected: 400, 401, 403, 404, 405, 406, 409, 422, 428, 5xx
  |     Invalid component: in query - object with unexpected properties
  | 
  | [200] OK:
  | 
  |     `[]`
  | 
  | Reproduce with:
  | 
  |     curl -X POST -H 'Content-Type: application/json' -d '{"documents": []}' 'http://localhost/v1/projects/dummy-fi/suggest-batch?threshold=0&x-schemathesis-unknown-property=42'
  | 
  |  (1 sub-exception)
  +-+---------------- 1 ----------------
    | schemathesis.core.failures.AcceptedNegativeData: API accepted schema-violating request
    | 
    | Invalid data should have been rejected
    | Expected: 400, 401, 403, 404, 405, 406, 409, 422, 428, 5xx
    | Invalid component: in query - object with unexpected properties
    +------------------------------------

I think it should be possible to restrict the generation of languages to resolve the first failure (the error response "not supported by vocabulary" is correct here, it just is not possible to be expressed in OpenAPI spec?).

For the second failure, I speculated on an resolution:

In addition to upgrading to Schemathesis 4, this PR currently enables strict_validation in Connexion "to disallow any extra parameters that are not defined in your specification". I think this would be a good feature, because then typos in API requests parameters are noticed at once, not after debugging (with strict validation e.g. using theshold instead of threshold gives 400 Bad request error, without it the extra, nonfunctional parameter is just silently ignored).

However, Annif 1.4 introduced metadata_* parameters to suggest/, suggest_batch/ and learn/ API methods (#867), and to return a 400 error for requests with parameters not starting with metadata_ I made this: Add Custom request body validator.

Originally posted by @juhoinkinen in #879 (comment)

Also, on some runs a third test has failed (can't remember of find again which or reproduce); to not make this happen surprisingly, Schemathesis could be made generate the test cases determistically: schemathesis/schemathesis#3531 (comment)

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions