Validation Rules

Testing web applications and APIs is not as simple as just sending requests. You’ll also need to make sure the responses come back as expected every time.

Speedway automatically detects many types of errors. Anytime your application (or the server it is running on) returns an HTTP 4xx or 5xx response code, Speedway will automatically recognize it as an error. Network errors, like socket timeouts or closed connections, are also automatically detected.

Errors that are unique to your application, however, might not be automatically recognized as errors. Your application might show an error message to the user, or it might return an altogether different page than your script was expecting.

You can check for these more subtle errors with validation rules.

When to Use Validation

Examples of when you might need to use custom validation rules include:

• Making certain a key page loads in less than 3.0 seconds.
• Making certain the response includes the word “Welcome” when you submit the registration form.
• Making certain the words “User not found” do not appear in the response when you log in.
• Making certain a specific response is always more than 15kb and less than 25kb.

When you’re testing web applications or APIs, it’s a good idea to add validation rules throughout your script to eliminate uncertainty about the responses. Otherwise, certain types of errors might go undetected.

Creating a Validator

You can add a validation rule to an HTTP step, by clicking on the Add… button.

There are five types of validation rules available for any HTTP step:

• Content Validator
• Response Time Validator
• Response Size Validator
• Response Status Validator
• JavaScript Validator

You can add multiple validators of each type to a single HTTP step, if you need to.

Response Time Validators

This type of validator enforces a maximum response time for the primary request, or the entire page (primary request plus resources).

For example, you might have a performance requirement that a certain key page must always load within a certain time, like 2.5 seconds. A response time validation rule is one way to enforce this.

To add this validation rule to a step, select Validate response time from the Add… menu. Enter the maximum allowable response time in seconds. You can also check or uncheck Including resources, depending on whether you want to validate only the initial request’s response time, or the full page including resources.

Response Size Validators

This type of validator checks that the response body is within a certain size range.

Often, if an error happens on the server side, the response may be incomplete or show a short error page. By checking that the response body falls within an expected size range, you can be more confident that the response is being returned correctly.

To add a response size validator to an HTTP step, select Validate response size from the step context menu. Enter a value for the minimum and/or maximum size, in bytes. The validator will then throw an error if the response falls outside the expected size range.

Content Validators

A content validation rule can validate that the response body contains or does not contain a specified string.

To add a content validator to an HTTP step, select Validate content from the step context menu. Enter a string of text, and choose either Response must contain or Response must not contain, depending on whether the text is supposed to appear or isn’t supposed to appear in the response.

Since many web apps return an HTTP success status code (such as 200) even if there is a human-readable error on the page, it’s often a good idea to validate the actual page content instead of relying solely on HTTP status codes.

JavaScript Validators

Speedway’s JavaScript validator is the most flexible and powerful, allowing you to declare your own JavaScript function to validate the response.

To add a JavaScript validator to an HTTP step, select Validate with JavaScript from the step context menu. Then, fill in the body of the function.

Your validation function must be called validate() and return a boolean value. If the function returns true, the step passed validation. If the function returns false or throws an error, the step failed validation and an error will be reported.

Speedway also provides some helpful utilities for use in your validator function. You can read about them in Code Blocks. Pretty much anything you can do in a code block, you can also do inside a validator function.

Here are a few examples.

Example: Validating a header with JavaScript

HTTP response headers are used to convey information about the response. The Content-Type header is an important header because it tells the client what type of content is contained in the response.

To create a custom validator that checks for a content type of application/json, create a function like this:

function validate(response) {
    return response.getHeader("Content-Type") == "application/json";
}

If you need to look at multiple headers, you can access the headers array directly. In this example we make sure at least three separate Set-Cookie headers came back in the response.

function validate(response) {
    let allHeaders = response.headers;
    let cookieHeaders = allHeaders.filter(header => header.name == 'Set-Cookie');

    console.log(`We got ${cookieHeaders.length} cookies`);

    return cookieHeaders.length >= 3; // only valid if we got 3+ cookies
}

Example: Validating body content with a regular expression

Speedway’s content validator makes it easy to search for a fixed string in the response, but what about an expression that might be a bit different each time, like a tracking number that was generated by your application?

With a JavaScript validator you can make sure that the response content matches a certain regular expression, throwing a validation error if it doesn’t.

function validate(response) {
    const regex = /Confirmation #AWB(\d{8})Z/gm;
    const matched = regex.test(response.string());

    return matched; // Validation is successful if the regex matched
}

Example: Validating a JSON or XML response

Validating a JSON response in Speedway can be done with JavaScript, as follows:

// {"books": [
//      {"id": 4, "name": "Slaughterhouse Five"},
//      {"id": 11, "name": "The Old Man and the Sea"},
//      {"id": 25, "name": "Don Quixote"}
//  ]}

function validate(response) {
    var json = JSON.parse(response.string());

    return json.books && json.books.length > 0; // Make sure at least one book was returned
}

Similarly, validating an XML document is just as easy:

// <cars>
//     <car id="13"><make>Volkswagen</make><model>Thing</model></car>
//     <car id="80"><make>Land Rover</make><model>Series IIA</model></car>
// </cars>

function validate(response) {
    var xml = XML.parse(response.string());
    var cars = xml.childNamed('cars');

    // Validation is successful if there was at least 1 car
    return cars && cars.childrenNamed('car').length > 0;
}

Pro tip: Speedway’s XML parser is based on xmldoc, a pure JavaScript XML parser. More documentation on how to use this parser is available on GitHub.

Testing Your Validation Rules

After adding or changing validation rules, play your script in the editor to make sure it still works. While it plays, watch the logs for validation messages.

If validation failed and you’re wondering why, you can use the Requests tab to look at the full HTTP response headers and body.

From the headers and body you should be able to tell why the response failed validation.

If validation fails consistently, you may need to adjust your script.

Troubleshooting JavaScript validation errors

Errors from JavaScript validators arise from your own validation function. If you’re having trouble, try using console.log() in your validate function to print debug messages.