Automating Lambda Schema Validation with TypeScript
Enforce and document your API's query and body params at runtime using TypeScript
-
-API validation is essential to running a backend. Especially with tools like NoSQL databases, it is up to the API's business logic to enforce typing. Without these type checks, security vulnerabilities can be introduced.
Using TypeScript with Lambda is already awesome. Node.js is one of the fastest runtimes on AWS Lambda to cold start. Node.js also supports one of AWS's most mature SDK's.
Read this article if you're interested in setting up your Lambda API with TypeScript, but not sure how to set up your development environment. We'll assume you have a working Lambda/TypeScript environment for the rest of this tutorial.
Lambda Design Pattern
We're going to split our Lambda into a handler and service. This way we can write better unit tests, control error handling, and abstract away the aspects of creating an APIGatewayProxyResult for an HTTP response.
Additionally, our type checking system will be implemented in the handler. Invalid query params or body params will be filtered out before they can reach the service.
Our Lambda files will follow this pattern.
import { APIGatewayProxyEvent } from "aws-lambda/trigger/api-gateway-proxy";
import createHandler from "./lib/createHandler";
// If using CloudFormation or SAM, set the Lambda's handler as
// file/path/fileName.handler
export const handler = createHandle({
queryParamType: "QueryParamsType",
bodyParamType: "BodyParamsType",
service,
});
export async function service({
queryParams,
bodyParams,
}: ServiceParams<QueryParamsType, BodyParamsType>) {
// core logic here
}The createHandler() handler generator will start as this. The returned handler isn't doing much right now, but we'll add our type checks later.
interface ServiceParams<
queryParamType extends object,
bodyType extends object
> {
queryParams?: queryParamType;
body?: bodyType;
}
export function createHandle({
queryParamType,
bodyParamType,
service,
}: {
queryParamType?: string;
bodyParamType?: string;
service: (arg?: ServiceParams<any, any>) => Promise<any> | any;
}): (event: APIGatewayProxyEvent) => Promise<APIGatewayProxyResult> {
// instantiate stuff for the execution environment here
return async event => {
try {
// handler logic here
const res = await service();
return getSuccessRes(event, res);
} catch (error) {
return getErrorRes(event, error);
}
};
}Generating JSON Schema
Types and interfaces defined in TypeScript work well during development, but go away during runtime. In fact, without custom transformers in your tsconfig, it is impossible to access these types during runtime. Therefore, we need to create tooling that brings our TypeScript definitions to the runtime.
typescript-json-schema is a CLI that creates JSON Schema documents from TypeScript definitions. The JSONs follow the Draft 7 JSON Schema so schemas are understood by many applications. Add typescript-json-schema to your project as a dev dependency. We will be generating our JSON schemas ahead of runtime, so we can reduce our Lambda's overall package size by keeping this a dev dependency.
npm i -D typescript-json-schema
yarn add -D typescript-json-schemaLet's prepare our definitions. For example, let's design an endpoint to add teams. We'll require the team's id in a query string parameter called id, and the team object in the body param. Depending on your API design (REST, RPC, SOAP, etc.), you likely already have defined these types in your project, like Team.
export interface IdQueryParam {
id: string;
}
export interface Team {
name: string;
type: TeamType;
color?: string;
athletes: Athlete[];
}
interface Athlete {
name: string;
jerseyNo: number;
stats: {
[x: string]: string;
};
}
enum TeamType {
LOCAL_LEAGUE = 'LOCAL_LEAGUE',
SCHOOL_TEAM = 'SCHOOL_TEAM',
TRAVEL_TEAM = 'TRAVEL',
OTHER = 'OTHER',
}Save these in a types file. Now run the CLI with these arguments and replacing YOUR_TYPES_FILE_PATH. Most of the magic comes from --noExtraProps, which enforces only properties in our type definitions can be included. We also include --required to ensure only certain properties as required, so leaving out properties like color would still work. Refs are disabled with --refs false so inline definitions are added for each type (for example Athlete's schema is defined inside Team). Lastly, we save the schema output ./types.schema.json since the CLI defaults output to stdout.
More about the CLI options here.
typescript-json-schema --noExtraProps --required \
--refs false -o ./types.schema.json [YOUR_TYPES_FILE_PATH] *
Save this as a script in your package.json.
"scripts": {
"schema": "typescript-json-schema --noExtraProps --required --refs false -o ./types.schema.json \"./lib/Types.d.ts\" \"*\"",
}Once ran, your schema JSON file will now look like this.
{
"$schema": "http://json-schema.org/draft-07/schema#",
"definitions": {
"IdQueryParam": {
"additionalProperties": false,
"properties": {
"id": {
"type": "string"
}
},
"required": [
"id"
],
"type": "object"
},
"Team": {
"additionalProperties": false,
"properties": {
"athletes": {
"items": {
"additionalProperties": false,
"properties": {
"name": {
"type": "string"
},
"jerseyNo": {
"type": "number"
},
"stats": {
"additionalProperties": {
"type": "string"
},
"type": "object"
}
},
"required": [
"name",
"number",
"stats"
],
"type": "object"
},
"type": "array"
},
"color": {
"type": "string"
},
"name": {
"type": "string"
},
"type": {
"enum": [
"LOCAL_LEAGUE",
"OTHER",
"SCHOOL_TEAM",
"TRAVEL"
],
"type": "string"
}
},
"required": [
"athletes",
"name",
"type"
],
"type": "object"
}
}
}Validating JSON Schema at Runtime
Now that we've generated our JSON schema, we can use this file at runtime. We'll use AJV, a fast and popular JSON schema validator for Node.js and browsers.
npm i ajv
yarn add ajvNow we'll implement type checking in our handler generator. Generating handlers in side of createHandler() enables us to instantiate objects for our execution environment once. This way we can create objects only when the Lambda first boots up, not at every invocation.
We'll instantiate compiled validators for our query and body params. Compiling AJV validators is relatively slower than validation itself, so we only want to do this once. Once we've compiled our validators, we can use them for checks at runtime during each invocation.
You can write any validation logic you'd like, but here we'll throw an error if the types don't match, stopping execution before the service function.
import Ajv from 'ajv';
import schema from '../types.schema.json';
const ajv = new Ajv();
export function createHandle({
queryParamType,
bodyParamType,
service,
}: {
queryParamType?: string;
bodyParamType?: string;
service: (arg?: ServiceParams<any, any>) => Promise<any> | any;
}): (event: APIGatewayProxyEvent) => Promise<APIGatewayProxyResult> {
const isValidQueryParams = ajv.compile(
schema.definitions[queryParamType] || {},
);
const isValidBodyParams = ajv.compile(
schema.definitions[bodyParamType] || {},
);
return async event => {
try {
const serviceParams: ServiceParams<any, any> = {};
if (queryParamType) {
const params = event.queryStringParameters;
if (isValidQueryParams(params)) {
serviceParams.queryParams = params;
} else {
throw new Error('Invalid query string parameters');
}
}
if (bodyParamType) {
const params = JSON.parse(event.body);
if (isValidBodyParams(params)) {
serviceParams.body = params;
} else {
throw new Error('Invalid body parameters');
}
}
const res = await service(serviceParams);
return getSuccessRes(event, res);
} catch (error) {
return getErrorRes(event, error);
}
};
}If it isn't too much of a security concern, you could also have the API respond with the mismatched schema definition. That way, your API's interface is self documenting.
if (bodyParamType) {
const params = JSON.parse(event.body);
if (isValidBodyParams(params)) {
serviceParams.body = params;
} else {
throw new Error({
message: `Incorrect type. Must match ${bodyParamType} schema`,
schema: schema.definitions[bodyParamType],
});
}
}Conclusion
We can now write service() functions that input query and body params with guaranteed types. Here's a sample Lambda with our team creation example. We create the handle with queryParamType: "IdQueryParam" and bodyParamType: "Team" defined so our service can run guaranteed with an id and Team.
import { DocumentClient } from 'aws-sdk/clients/dynamodb';
import createHandler from "./lib/createHandler";
const docClient = new DocumentClient();
const { BLOG_TABLE } = process.env;
export const handler = createHandle({
queryParamType: "IdQueryParam",
bodyParamType: "Team",
service,
});
export async function service({
queryParams,
body,
}: ServiceParams<IdQueryParam, Team>) {
try {
const team: Team = body;
const { id } = queryParams;
const dynamoParams: DocumentClient.PutItemInput = {
TableName: BLOG_TABLE,
Item: {
PartitionKey: id,
...team,
},
};
await docClient.put(dynamoParams).promise();
return true;
} catch {
return false;
}
}P.S. - It's definitely cumbersome repeating TypeScript definitions as strings, but unfortunately I haven't found a simple workaround to this. It's as close as I've gotten to having TypeScript at runtime. If anyone has any suggestions for improvement (without tsconfig transformers), please let me know in the comments!