Contract tests
Generate Postman contract tests from OpenAPI that check status codes, content type, JSON body, JSON schema, headers and response time.
Contract tests verify that your API's responses match what the OpenAPI document promises: status code, content type, JSON body, JSON schema, required headers and response time. Portman generates them from the spec and injects them into each targeted Postman request, configured under tests.contractTests.
Example
{ "version": 1.0, "tests": { "contractTests": [ { "openApiOperation": "*::/crm/*", "statusSuccess": { "enabled": true } }, { "openApiOperation": "*::/crm/*", "excludeForOperations": ["leadsAdd", "GET::/crm/leads/{id}"], "responseTime": { "enabled": true, "maxMs": 300 } }, { "openApiOperation": "*::/crm/*", "contentType": { "enabled": true } }, { "openApiOperation": "*::/crm/*", "jsonBody": { "enabled": true } }, { "openApiOperation": "*::/crm/*", "schemaValidation": { "enabled": true } }, { "openApiOperation": "*::/crm/*", "headersPresent": { "enabled": true } } ] }, "globals": { "stripResponseExamples": true }}You can enable several checks in one entry or split them across entries, as above.
Targeting
| Property | Description |
|---|---|
openApiOperationId | OpenAPI operationId, for example leadsAll. |
openApiOperationIds | Array of operationIds. |
openApiOperation | Method and path, for example GET::/crm/leads. Supports wildcards. |
excludeForOperations | Optional. Operations to skip. |
openApiRequest | Optional. The request body content type to use, supports wildcards like application/*. Defaults to the spec's request content type, preferring application/json. |
openApiResponse | Optional. The response code or code::content-type to test against, supports wildcards like text/*. Examples: "404", "200::text/plain", "200::text/*". Defaults to the first response in the spec. |
Contract tests are attached to the main request Portman generates for an operation. When an operation lists multiple request or response content types, only one set is used for that request; choose it with openApiRequest and openApiResponse. To create a request for every content type, use variation tests.
Checks
| Check | Description |
|---|---|
statusSuccess | The response status is 2xx. |
statusCode | The response status matches a specific code. |
responseTime | The response arrives within maxMs milliseconds. |
contentType | The Content-Type header matches the content type in the spec. |
jsonBody | The response body is JSON as defined in the spec. |
schemaValidation | The response body matches the JSON schema in the spec. The schema is inlined in the Postman test. |
headersPresent | The required response headers from the spec are present. |
statusCode
{ "openApiOperation": "*::/crm/*", "statusCode": { "enabled": true, "code": 400 }}Generates:
pm.test('[GET]::/crm/leads/:id - Response status code time is 400', function () { pm.expect(pm.response.code).to.equal(400);})responseTime
{ "openApiOperation": "*::/crm/*", "responseTime": { "enabled": true, "maxMs": 300 }}schemaValidation with additionalProperties
{ "openApiOperation": "*::/crm/*", "schemaValidation": { "enabled": true, "additionalProperties": false }}additionalProperties sets all additionalProperties in the expected schema to false (strict) or true (fluid). With false, the test fails when the response contains properties that are not documented, which can point to undocumented fields or an API returning more data than intended.
If schema validation fails on formats the validator does not know, add them with --extraUnknownFormats. See CLI options.
Example generated tests
For GET /crm/leads/{id}, Portman generates tests like:
// Validate if response matches JSON schemapm.test("[GET]::/crm/leads/:id - Schema is valid", function() { pm.response.to.have.jsonSchema(schema,{unknownFormats: ["int32", "int64", "float", "double"]});}); // Validate if response header is presentpm.test("[GET]::/crm/leads/:id - Response header Content-type is present", function () { pm.response.to.have.header("Content-type");});Bundling contract tests
By default, requests live in folders based on the OpenAPI tags, with tests injected directly. With --bundleContractTests, all requests with contract tests are placed in a separate folder, so you can run only those with the Newman --folder option.