Running tests with Newman

Portman can run the collection it just generated with Newman, the Postman command-line runner, so a single command converts your spec and executes the contract tests against your API.

Run Newman

Use --runNewman (alias -n) and usually --baseUrl (alias -b) to point the requests at the server you want to test:

portman -u https://specs.apideck.com/crm.yml -b http://localhost:3050 -n true

Iteration data

Pass a data file for Newman iterations with --newmanIterationData (alias -d):

portman -u https://specs.apideck.com/crm.yml -b http://localhost:3050 -n true -d ./tmp/newman/data/crm.json

Values from the data file can be read in scripts with pm.iterationData.get(...), for example in a collection pre-request script:

"collectionPreRequestScripts": [  "pm.collectionVariables.set('status', pm.iterationData.get('status') || 'open')"]

Newman options

All Newman configuration options can be passed through Portman in three ways.

Options file

portman -u https://specs.apideck.com/crm.yml -c ./tmp/crm/portman-config.json --runNewman --newmanOptionsFile ./tmp/crm/newman-options.json

newman-options.json:

{  "environment": "./tmp/crm/postman-dev.env.json",  "iteration-count": 5,  "ignore-redirects": true,  "insecure": true}

Inline JSON

portman -u https://specs.apideck.com/crm.yml -c ./tmp/crm/portman-config.json --runNewman --newmanRunOptions '{"environment":"./tmp/crm/postman-dev.env.json","iteration-count": 5}'

CLI options file

Include the options as an object under newmanRunOptions in your CLI options file:

{  "local": "./examples/cli-options/crm.openapi.yml",  "baseUrl": "http://localhost:3050",  "portmanConfigFile": "./examples/cli-options/portman-config.crm.json",  "runNewman": true,  "newmanRunOptions": {    "environment": "./tmp/crm/postman-dev.env.json",    "iteration-count": 5,    "ignore-redirects": true,    "insecure": true  }}

Redirects

Newman is set to ignore redirects so you can test redirect response codes. If you run the collection inside the Postman app, set Postman the same way, or your redirect tests will fail: Postman, Preferences, Automatically follow redirects, OFF.

Running only the contract tests

With --bundleContractTests, Portman puts all requests with contract tests in a separate folder. You can then run only that folder with the Newman --folder option.

Run an existing collection

--localPostman uses a local Postman collection and skips the OpenAPI conversion. See CLI options.