One year ago I was working at GoCardless, building a prototype of an Open Banking product.

As this was the first product work I’d done in a while (that wasn’t a Kubernetes operator or database tool) it felt sensible to revisit the API development scene, not wanting to do things as I always had if there were better options out there.

My hunch was right- the API scene had moved on. After looking at a variety of tools, we adopted a tool called Goa, helping us build an API at a level of polish that shocked me.

For almost no added effort, Goa helped:

Rapidly iterate on the API design
Generate type-safe server implementations, guaranteed to be consistent with the API spec
Produce transport specifications like OpenAPI specs
…from which we could generate typed API clients and documentation

One year later and I’m now working at incident.io, where one of my first contributions was to use Goa to power our API. It’s proved to be extremely valuable, especially in a start-up that wants to move fast while not breaking things.

This post explains how the toolchain works, and everything you get for free when you adopt it.

API design

So you have a product idea, and it needs an API.

Everything with Goa starts with the ‘design’, where you use the Goa DSL to specify services and methods, and the types they use.

Borrowing from my work at incident.io, let’s say we want an API to create an incident. Here’s what that might look like:

package design

import (
  . "goa.design/goa/v3/dsl"
)

var Incident = Type("Incident", func() {
  Attribute("id", String, "Unique identifier for the incident", func() {
    Example("01FDAG4SAP5TYPT98WGR2N7W91")
  })
  Attribute("name", String, "Name of the incident", func() {
    Example("Full service outage")
  })
  Required(
    "id",
    "name",
  )
})

var _ = Service("Incidents", func() {
  Description("Manage incidents")

  HTTP(func() {
    Path("/api/incidents")
  })

  Method("Create", func() {
    Description("Create a new incident")

    Payload(func() {
      Reference(Incident)
      Attribute("name")
    })

    Result(Incident)

    HTTP(func() {
      POST("/")
    })
  })
})

In this example, we define an Incident type and an Incidents service, with a Create method that accepts an incident name and returns an Incident. All of those definitions are abstracted from the transport, with HTTP blocks binding the application concepts to HTTP calls.

While this example includes HTTP bindings, you can easily include gRPC bindings, if that’s how you prefer to serve your API. It’s one of Goa’s strengths that the authors really understand APIs, and have created solid abstractions for each of the supported transports that rarely leak.

Returning to our example: right now all we have a design, but no implementation. We’ll use Goa to codegen the rest:

$ goa gen github.com/lawrencejones/goa-example/api/design -o api
api/gen/http/incidents/server/encode_decode.go
api/gen/http/incidents/server/paths.go
api/gen/http/incidents/server/server.go
api/gen/http/incidents/server/types.go
api/gen/http/openapi.json
api/gen/incidents/client.go
api/gen/incidents/endpoints.go
api/gen/incidents/service.go

Let’s break this down:

api/gen/incidents contains the application level interfaces for the incidents service, everything that is separate from the transport. We’ll implement these interfaces when we’re building our service.
api/gen/http/incidents binds HTTP level concepts (HTTP requests, HTTP response types) to the transport-agnostic service interfaces. encode_decode.go is about going from XML/JSON/etc to Go-native types, so developers can avoid transport concerns.
api/gen/http/openapi.json is an OpenAPI specification for the API.

The application interface Goa generates should be unsurprising:

package incidents

type Service interface {
  // Create a new incident
  Create(context.Context, *CreatePayload) (res *Incident, err error)
}

// Incident is the result type of the Incidents service Create method.
type Incident struct {
  // Unique identifier for the incident
  ID string
  // Name of the incident
  Name string
}

All that remains to be done, as a developer implementing this API design, is to write an implementation that satisfies this interface.

That looks like this:

package api

import (
  "context"
  "github.com/lawrencejones/goa-example/api/gen/incidents"
  "github.com/google/uuid"
)

func NewIncidents() incidents.Service {
  return &incidentsService{}
}

type incidentsService struct {
}

func (svc *incidentsService) Create(ctx context.Context, payload *incidents.CreatePayload) (*incidents.Incident, error) {
  inc := incidents.Incident{
    ID:   uuid.NewString(),
    Name: payload.Name,
  }

  return &inc, nil
}

Once you get used to this pattern–design, generate, implement–it becomes really easy to incrementally build your API. The flow emphasises the most important part of building any API, which is the design, and implementation is simplified as fixing the compiler errors (*incidentsService does not implement incidents.Service (missing Create method)).

That’s the hard part done- from here out, everything is easy.

Generate a client

Having used the API design to generate backend types, which the compiler will guarantee our implementation is consistent with, you can have total faith in the generated API specifications being correct and up-to-date.

Now we have faithful specifications, such as the OpenAPI spec, we can use them to generate API clients. Most projects I’ve worked with have a frontend component in TypeScript which can benefit from a strongly typed client guaranteed to be consistent with the server implementation.

OpenAPI Generator can help you here. It ships with a number of generators that use an OpenAPI specification to build clients in various flavours (language, frameworks, etc).

Picking the right generator is key, as quality can be inconsistent. We’ve had success with typescript-fetch, a vanilla TypeScript client that works best with the legacy openapi.yml that Goa produces:

$ make clients/typescript
java -jar openapi-generator-cli.jar \
    generate \
        --generator-name typescript-fetch \
        --input-spec api/gen/http/openapi.json \
        --skip-validate-spec \
        --additional-properties npmName=goa-example,typescriptThreePlus=true,modelPropertyNaming=original \
        --output clients/typescript

[main] INFO  o.o.codegen.DefaultGenerator - Generating with dryRun=false
[main] INFO  o.o.codegen.DefaultGenerator - OpenAPI Generator: typescript-fetch (client)
[main] INFO  o.o.codegen.DefaultGenerator - Generator 'typescript-fetch' is considered stable.
[main] INFO  o.o.codegen.TemplateManager - writing file /Users/lawrence/Projects/goa-example/clients/typescript/src/models/IncidentsCreateRequestBody.ts
...

Now you have an API client, giving integrators an ideal experience in their language of choice:

import { IncidentsApi, Configuration } from "clients/typescript";

const client = IncidentsApi(new Configuration());
const incident = await client
  .incidentsCreate({
    createRequestBody: {
      name: "Full service outage",
    },
  });

alert(`Created incident with ID=${incident.id}`);

TypeScript is just one of hundreds of generators- see the full list here.

Frontend benefits

At incident.io we have a monorepo with both the frontend web application (client) and backend implementation (server), with the backend using Goa to expose an API for the frontend to consume.

Introducing Goa has been great for many reasons, but Goa + TypeScript clients really shine when building frontend forms. We use react-hook-form for frontend form components, which has TypeScript bindings so you can strongly-type your form fields.

Now we have a TypeScript client, we can parameterise react-hook-forms with client types, allowing the TypeScript compiler to tell us when our form definitions are mismatched with our API definition:

import { useClient } from "contexts/ClientContext";
import { IncidentsCreateRequestBody } from "clients/typescript";

const IncidentCreateForm = ({
  closeCallback,
}: {
  closeCallback: () => void;
}): React.ReactElement | null => {
  const client = useClient();

  const { register, handleSubmit } = useForm<IncidentsCreateRequestBody>();

  const onSubmit = (body: IncidentsCreateRequestBody) => {
    client
      .incidentsCreate({createRequestBody: body})
      .then(() => {
        closeCallback();
      });
  };

  return (
    <form onSubmit={handleSubmit(onSubmit)}>
      <input {...register("name")} />
      <input type="submit" />
    </form>
  );
}

Before this, building forms that use an API would require care to produce requests consistent with the backend. Making changes to that API would require manual review of frontend code to find all uses of the API, hoping you catch them all before deploying.

In a world with Goa and generated clients, you:

Adjust the Goa design
Generate backend types and frontend client (make api/gen clients/typescript)
Fix backend compiler errors
Fix frontend compiler errors

If we changed the incident name field to be description, we’ll get a compiler error about our form immediately:

IncidentCreateForm.tsx:21:20 - error TS2322:
        Type '"name"' is not assignable to type '"description" | "severity_id"'.

21      register("name")
         ~~

I can’t overstate how useful this is, especially for maximising developer productivity and confidence in changes.

More than this

In projects where updating the specification (OpenAPI, etc) is optional in any sense, it will inevitably drift out-of-sync with reality.

At it’s heart, this toolchain is so valuable because you can trust the API specifications are accurate. That guarantee means a whole ecosystem of tools surrounding OpenAPI become even more compelling.

As just one example, go-swagger can generate documentation for OpenAPI specifications that are really high quality.

Most of my Goa projects have a make docs target:

$ make docs
docker run --platform=linux/amd64 -p 4000:4000 --rm -v "$(pwd)/api:/api" -it quay.io/goswagger/swagger:v0.28.0 \
                serve --no-open --port=4000 --host=0.0.0.0 /api/gen/http/openapi.json
2022/01/22 17:57:24 serving docs at http://localhost:4000/docs

Which is all you need to get docs worthy of a paid-for product:

Screenshot of go-swagger documentation for the incidents create API — go-swagger documentation for the incidents create API

It all feels too easy

When I first joined GoCardless in 2015, we had just built crank, an in-house tool for generating API client libraries from JSON hyperschema.

As a company who (at the time) thought of their product as the API, this was a no-brainer. Crank helped us maintain a suite of client libraries across several languages and automatically generated our docs, often the source of compliments and product referrals.

Maintaining crank was not fun, though. By the time I left in 2021, crank was still building our client libraries and docs, despite it being the tool everyone loved to hate.

With that in mind, finding a toolchain that takes a couple of days to setup and gives an amazing development experience feels like a cheat. Our internal APIs have documentation that rivals API products, something I will never fail to marvel at.

Adopting these tools at incident.io is one of the main reasons we can move so fast, and make product changes with such confidence. Whatever language you use, find a similar toolchain and try it out- I can recommend it with no caveats!

Discuss this post on Hackernews. If you liked this post and want to see more, follow me at @lawrjones.

Lawrence Jones

API development with type-safety across the entire stack

API design

Generate a client

Frontend benefits

More than this

It all feels too easy