Dynamic type validation in TypeScript

There is no doubt that TypeScript has enjoyed a huge adoption in the JavaScript community, and one of the great benefits it provides is the type checking of all the variables inside our code. It will check if performing any operation on a variable is possible given its type.

Most people think that by using TypeScript as their application language, they are “covered” from any emptiness error, like the classic “undefined is not a function” or, my favorite, “can’t read property X of undefined.” This assumption is wrong, and the best way to demonstrate it is with code!

I gave a talk on this topic at the TypeScript Berlin Meetup. This article and the talk cover the same content, so you can use either to learn about this topic!

Why TypeScript won’t always cover you 🕵

The following example does not present any TypeScript error.

// Typescript definition
type ExampleType = {
  name: string,
  age?: number,
  pets: {
    name: string,
    legs: number,
  }[],
};

// communicates with external API
const fetchData = (): Promise<ExampleType> => {};

const getBiped = async () => {
  const data = await fetchData();
  console.log(data);
  // { name: 'John' }
  return data.pets.find(pet => pet.legs === 2); // Boom!
};

The snippet contains:

• ExampleType – a type definition with two properties required, name and pets, and one optional property, age. The property pets is an array of objects with name and legs, both required
• fetchData – a function to retrieve data from an external endpoint
• getBiped – another function that will call fetchData, iterate over the pets, and return only the pets with two legs

So, why will my script fail when I execute it? The reason is because the external API is returning an object that doesn’t contain pets inside, and then when you try to execute data.pets.find(), you will receive the error Uncaught ReferenceError: Cannot read property 'find' of undefined.

In the official React documentation, you can find a very nice definition of what TypeScript is:

TypeScript is a programming language developed by Microsoft. It is a typed superset of JavaScript and includes its compiler. Being a typed language, TypeScript can catch errors and bugs at build time, long before your app goes live.

Given that definition, it’s possible to formulate a new assumption:

TypeScript performs static type validation. Developers should take advantage of dynamic validations.

Do you need to validate everything? 🤔

Simply put, no! 🎉

Checking all the variables in our application is time-consuming from both a development and performance perspective. A nice rule of thumb to follow is:

Validate all the external sources of your application.

External sources are everything that is external or doesn’t have access to your application. Some examples:

• API responses
• Content inside files
• Input from the user
• Untyped libraries

An application will always present at least one external source, otherwise, it would very likely be useless. Therefore, let’s take a look at how you can write validations for your objects in TypeScript.

To keep things simple, the original snippet will be considered the base, and on top, I will show how to implement each of the validation methods.

Manual validation

The most basic validation, it’s a set of conditions that check whether the structure is the expected one.

const validate = (data: ExampleType) => {
  if (!data.pets) return false;
  // perform more checks

  return true;
};

const getBiped = async () => {
  const data = await fetchData();
  console.log(data);
  // { name: 'John' }

  if (!validate(data))
    throw Error('Validation error: data is not complete ...');

  return data.pets.find(pet => pet.legs === 2);
};

As you can see, a new function has been defined, called validate, which receives as a parameter an ExampleType object, with which it checks whether the property pets is defined. If it is not, it will return false, which will end up throwing an error with a description message. Otherwise, it will continue the execution, and now, when evaluating data.pets.find, it won’t throw an error.

Be aware that the implementation of the validate function is quite simple, and there is room for many more checks, such as:

• name should exist
• name should be a string
• If age exists, it should be a number
• pets should be an array of objects
• Each pet object should have props name and legs

The more checks you add, the more robust your application will be — but the more time you need to invest, too.

The advantages of this method are:

• No external libraries required: Only pure TypeScript
• Business-centric: You can add any business logic inside these validators. For example, you can check that propertyA shouldn’t exist if propertyB is present

It also presents some disadvantages:

• Manual work: Every validation has to be manually coded, and this can be quite time-consuming
• Duplication of code: In the example, ExampleType already defines that there is a pets property, and that it is required. But again, inside the validation code, you should still check that it’s true
• Room for bugs: In the previous code, there were many “bugs” or places for improvement

Using a validation library

Why reinvent the wheel, right? This method consists of using any validation library to assert the structure of the objects. To name some of the most used libraries:

• ajv
• joi
• v8n
• validate.js

The validation library used for this article is ajv; nevertheless, all the conclusions also apply to the other libraries.

const Ajv = require('ajv');
const ajv = new Ajv();

const validate = ajv.compile({
  properties: {
    name: {
      type: 'string',
      minLength: 3,
    },
    age: { type: 'number' },
    pets: {
      type: 'array',
      items: {
        name: {
          type: 'string',
          minLength: 3,
        },
        legs: { type: 'number' },
      },
    },
  },
  required: ['name', 'pets'],
  type: 'object',
});

const getBiped = async () => {
  const data = await fetchData();
  console.log(data);
  // { name: 'John' }
  if (!validate(data)) {
    throw Error('Validation failed: ' + ajv.errorsText(validate.errors));
    // Error: Validation failed: data should have required property 'pets'
  }

  return data.pets.find(pet => pet.legs === 2);
};

Many validation libraries force you to define a schema wherein you can describe the structure to evaluate. Given that schema, you can create the validation function that will be used in your code.

The declaration of your schema will always depend on the library you are using; therefore, I always recommend checking the official docs. In the case of ajv, it forces you to declare in an object style, where each property has to provide its type. It’s also possible to set custom checkers for these values, like minLength for any array or string.

This method provides:

• A standardized way to create validators and checks: The idea behind the schema is to have only one way to check for specific conditions inside your application — especially in JavaScript, where there are many ways to accomplish the same task, such as checking the length of an array. This quality is great to improve communication and collaboration inside a team
• Improvement of error reporting: In case there is a mismatch on some property, the library will inform you of which one it is in a human-friendly way rather than just printing the stack trace

This new way of creating validations presents the following drawbacks:

• Introduction of new syntax: When a team decides to add a new library, it becomes more difficult to understand the whole codebase. Every contributor has to know about the validator schema to understand how to make a change on it
• Validators and types are not in sync: The definition of the schema and ExampleType is disconnected, which means that every time you make a change inside the ExampleType, you have to manually reflect it inside the schema. Depending on how many validators you have, this task can be quite tedious

One small comment regarding keeping validators and types in sync: some open-source projects address this issue, such as json-schema-to-typescript, which can generate a type definition from an existing schema. Then this won’t be considered a problem.

Dynamic type validator

This is the method I want to talk about, and it represents a change in paradigm regarding how to create validators and keep types in sync.

In the two other methods, the validator and the type can be seen as different entities: the validator will take the incoming object and check its properties, and the type statically belongs to the object. Combining both entities, the result is a validated type object.

Dynamic type validation allows a type to generate a validator from its definition. Now they are related — a validator depends entirely on a type, preventing any mismatch between structures.

Generation of validators

To generate these validators, I found an amazing open-source project called typescript-json-validator, made by @ForbesLindesay. The repo description states that is goal is to “automatically generate a validator using JSON Schema and AJV for any TypeScript type.”

For the test, let’s reuse the ExampleType definition, which now has been moved to a separate file inside the types folder.

// src/types/ExampleType.ts

type ExampleType = {
  name: string;
  age?: number;
  pets: {
    name: string;
    legs: number;
  }[];
};

This library exposes a handy CLI that can be called from anywhere, and given a file path and the name of the type, it will generate — in the same location as the file — a new file with the validator code.

> npx typescript-json-validator src/types/ExampleType.ts ExampleType
# ExampleType.validator.ts created!

The resulting validator can be a very long file, so let’s take a look piece by piece.

1. Creating the ajv instance

It also sets some default configuration for ajv.

/* tslint:disable */
// generated by typescript-json-validator
import { inspect } from 'util';
import Ajv = require('ajv');
import ExampleType from './ExampleType';

export const ajv = new Ajv({
  allErrors: true,
  coerceTypes: false,
  format: 'fast',
  nullable: true,
  unicode: true,
  uniqueItems: true,
  useDefaults: true,
});

ajv.addMetaSchema(require('ajv/lib/refs/json-schema-draft-06.json'));

export { ExampleType };

2. Definition of the schema from the type

This is the key of this approach.

// Definition of Schema
export const ExampleTypeSchema = {
  $schema: 'http://json-schema.org/draft-07/schema#',
  defaultProperties: [],
  properties: {
    age: {
      type: 'number',
    },
    name: {
      type: 'string',
    },
    pets: {
      items: {
        defaultProperties: [],
        properties: {
          legs: {
            type: 'number',
          },
          name: {
            type: 'string',
          },
        },
        required: ['legs', 'name'],
        type: 'object',
      },
      type: 'array',
    },
  },
  required: ['name', 'pets'],
  type: 'object',
};

3. Export validation function using the generated schema

It also takes care of throwing an exception in case there is an error.

export type ValidateFunction<T> = ((data: unknown) => data is T) &
  Pick<Ajv.ValidateFunction, 'errors'>;
export const isExampleType = ajv.compile(ExampleTypeSchema) as ValidateFunction<
  ExampleType
>;

export default function validate(value: unknown): ExampleType {
  if (isExampleType(value)) {
    return value;
  } else {
    throw new Error(
      ajv.errorsText(
        isExampleType.errors!.filter((e: any) => e.keyword !== 'if'),
        { dataVar: 'ExampleType' },
      ) +
        '\n\n' +
        inspect(value),
    );
  }
}

To use the validator, you just need to import from the respective path and call it. Be aware that this function is already checking whether there were any errors inside the object; therefore, it’s not necessary to add an if statement here, making the code much cleaner.

import validate from 'src/types/ExampleType.validator';

const getBiped = async () => {
  const data = validate(await fetchData());

  return data.pets.find(pet => pet.legs === 2);
};

TypeScript ❤️ ajv

This library uses ajv under the hood to create the validator function, which means you can make use of all the nice features it provides, such as custom validation for types.

Let’s create a new definition type for ExampleType.

interface ExampleType {
  /**
   * @format email
   */
  email?: string;
  /**
   * @minimum 0
   * @maximum 100
   */
  answer: number;
}

Above each property you’ll find some annotations made inside comment brackets. These will be translated into ajv rules when the library generates the final schema. This is the result:

export const ExampleTypeSchema = {
  $schema: 'http://json-schema.org/draft-07/schema#',
  defaultProperties: [],
  properties: {
    answer: {
      maximum: 100,
      minimum: 0,
      type: 'number',
    },
    email: {
      format: 'email',
      type: 'string',
    },
  },
  required: ['answer'],
  type: 'object',
};

The property answer presents now two more attributes that will check whether the number is between 0 and 100. In the case of email, it will check whether the string value belongs to a valid email address.

As these annotations are wrapped inside comments, they don’t present any conflict with the TypeScript compiler.

Making it part of your workflow

This method is based on the idea that the developer will run the CLI command and generate the validators; otherwise, there’s a possibility that the schema was generated with an older version of the type, which can then present mismatches.

Fixing this issue is quite easy: you simply have to add a script that will be executed before your code will run. You can call it prebuild or prestart, and this is how your package.json might look:

{
  "scripts": {
    "prebuild": "typescript-json-validator src/types/ExampleType.ts ExampleType",
    "start": "yarn prebuild && ts-node start.ts",
    "build": "yarn prebuild && tsc"
  }
}

One last recommendation: ignore any validator.ts file from your project. There is no point in committing these files to your repository since they are going to be generated every time you start your project.

My experience with this approach 🙋‍♂️

About two months ago, I open-sourced one of my side projects called gatsby-starter-linkedin-resume.

In summary, it’s a Gatsby starter that can retrieve your information from LinkedIn, using a LinkedIn crawler, and generate an HTML and PDF resume from it using JSON Resume.

The project presents two main flows:

1. Create the resume information: you will be asked to enter your Linked In credentials, then a crawler will open a new browser, read your profile values and finally save all this information inside a JSON file in your directory. After that, the project will transform the data extracted from the crawler into the structure for Json Resume.
2. Build the project: once the resume information has been processed, Gatsby can generate HTML and PDF with it.

At the beginning of this article, I mentioned that it’s advisable to validate your external sources. For this project they are:

1. Data coming from the LinkedIn crawler: When dealing with crawlers, you should always be very careful with their outcome because it’s highly attached to the website from which they get their data. In the event that there is a change on the website, the output from the crawler can be altered.
2. Local file with the resume information: This project allows you to change the content of your resume manually in case you want to skip the creation of the resume information and create it by yourself. If the structure of the resume data is wrong, JSON Resume won’t be able to generate the resume properly.

These are the type definitions for each case:

interface LinkedInSchema {
  contact: ContactItem[];
  profile: ProfileData;
  positions: LinkedInPosition[];
  educations: LinkedInEducation[];
  skills: Skill[];
  courses: Course[];
  languages: LinkedInLanguage[];
  projects: LinkedInProject[];
}

interface JsonResumeSchema {
  basics: JsonResumeBasics;
  work: JsonResumeWork[];
  volunteer?: JsonResumeVolunteer[];
  education: JsonResumeEducation[];
  awards?: JsonResumeAward[];
  publications?: JsonResumePublication[];
  skills?: JsonResumeSkill[];
  languages?: JsonResumeLanguage[];
  interests?: JsonResumeInterest[];
  references?: JsonResumeReference[];
  projects?: JsonResumeProject[];
}

Both types present similarities in terms of variable names, but their internal structure differs. This is why it’s necessary to transform from one structure to the other on the first flow.

After I set up my project to generate the validators from these types, checking the structure of the incoming object was a very easy task.

Validation of the crawler result

// src/index.ts
import { RESUME_PATH, LINKED_IN_PATH } from './utils/path';
import validateLinkedInSchema from './types/LinkedInSchema.validator';
import { saveJson, readJson } from './utils/file';
import { inquireLoginData, getLinkedInData } from './utils/linkedin';

// ❗️❗️ IMPORT OF THE VALIDATOR  ❗️❗️
import mapLinkedInToJSONResume from './utils/mapLinkedInToJSONResume';

export const main = async ({ renew }) => {
  if (renew || !readJson(LINKED_IN_PATH)) {
    const credentials = await inquireLoginData();
    const linkedInData = await getLinkedInData(credentials);

    saveJson(LINKED_IN_PATH, linkedInData);
  }

  // ❗️❗️ VALIDATION IN ACTION ❗️❗️
  const linkedInParsed = validateLinkedInSchema(readJson(LINKED_IN_PATH));

  const jsonResumeData = mapLinkedInToJSONResume(linkedInParsed);
  saveJson(RESUME_PATH, jsonResumeData);
};

Validation of the resume information

// gatsby-config.js
const { existsSync } = require('fs');

// ❗️❗️ IMPORT OF THE VALIDATOR  ❗️❗️
const {
  default: validateJsonResume,
} = require('./lib/types/JsonResumeSchema.validator');

if (!existsSync('./resume.json')) {
  throw new Error(
    'Please run "yarn generate-resume" to generate your resume information.',
  );
}

// ❗️❗️ VALIDATION IN ACTION ❗️❗️
const resumeJson = validateJsonResume(require('./resume.json'));

module.exports = {
  plugins: [
    {
      resolve: 'gatsby-theme-jsonresume',
      options: {
        resumeJson,
      },
    },
    'gatsby-plugin-meta-redirect',
  ],
};

Closing words 🗣

To sum it all up, I created this table comparing the three methods. The dynamic types approach grabs the best of the other methods, making it the recommended approach to validate your object.

Approach	No additional syntax	Validators and types sync	Standardization
Manual	✅	❌	❌
Library	❌	❓	✅
Dynamic types	✅	✅	✅

If you are working in a TypeScript codebase, I recommend you give this new method of validating your objects a try. It’s very easy to set up, and in the event you don’t find it useful, removing it from the codebase is as easy as removing an import from your files.