Automating Kong API gateway setup with Terraform

by Saulius - posted in: Blog

Previous part: Kong API gateway - getting started.

This post is part of the series about using Kong API gateway in your technology stack.

In this tutorial we go through the steps of automating Kong API gateway setup. By using Terraform and setup scripts.

We will follow the infrastructure as a code pattern, which will allow us to recreate this setup every time we need it in the future workshops.

What will we create?

By the end of this tutorial you will have a set of Terraform scripts to automatically provision Kong API gateway instance on Digital Ocean platform.

It will become a useful setup for your projects. You will be able to re-create the fresh Kong API gateway instance with configs automatically, within minutes.

And we will use this setup for future Kong API gateway tutorials on this site. Source code for all files created in this tutorials are on GitHub.

Prerequisites

Access to Debian Linux server. Any virtual server works, you can use VirtualBox on your dev machine. Or any cloud platform which provides temporary VPS. I'm using Digital Ocean in this tutorial - they have generous free credits for all new signups.
API testing tool of your choice. It can be curl, httpie, any UI based tool. Here is a list of API testing tools. I'm using Flashpost in this post.

1. Automated VM creation

In our tutorials we aim for showing real world setup and configuration even while building proof of concept projects. This includes automation and infrastructure as code. It allows for reusability, time saving and reducing errors while building the cloud VMs.

In the previous tutorial, we documented the steps to use Terraform for a basic cloud VM setup.

Following steps are built on the foundations of this basic setup. You can get it from GitHub or visit the actual tutorial to build your own.

2. Install Kong and datastore

In this step we are going to use shell script from files directory.

Add the following shell commands to files/setup.sh:

# update & install 
# required dependencies curl, netstat, etc.
apt-get update
apt-get install -y apt-transport-https curl net-tools

# install & configure postgresql database
# create kong db user & kong database
apt-get install -y postgresql postgresql-contrib

# change to neutral directory
cd /tmp

# kong db user with credentials
su - postgres -c "createuser -s kong"
su - postgres -c "createdb kong"
sudo -u postgres psql -c "ALTER USER kong WITH PASSWORD 'kong';"

# install the Kong API gateway (3.8.0)
# guidelines from: https://docs.konghq.com/gateway/3.8.x/install/linux/debian/?install=oss
curl -Lo kong-3.8.0.deb "https://packages.konghq.com/public/gateway-38/deb/debian/pool/bullseye/main/k/ko/kong_3.8.0/kong_3.8.0_$(dpkg --print-architecture).deb"
sudo apt install -y ./kong-3.8.0.deb

# linux opend files limit setup
# recomended setting for kong
ulimit -n 4096

# bootstrap & start Kong
cd /etc/kong &&
cp /tmp/kong.conf /etc/kong/kong.conf
kong migrations bootstrap -c /etc/kong/kong.conf --v
kong start -c /etc/kong/kong.conf --v

# setup debian firewall
# only allow ports for
# ssh (22)
# kong apis (8000) - as demo setup
apt-get -y install ufw
ufw status verbose
ufw default deny incoming
ufw default allow outgoing
ufw allow ssh
ufw allow 22
ufw allow 8000
yes | ufw enable

The steps in this script will install PostgreSQL server, setup the db user, database and permissions, install Kong and configure the firewall.

3. Kong configuration file

In files directory create another file called files/kong.conf. It will hold configuration settings for Kong api gateway.

# run Kong in standalone mode
# with PostgreSQL db
database = postgres
pg_host = 127.0.0.1
pg_port = 5432
pg_timeout = 5000
pg_user = kong
pg_password = kong
pg_database = kong

# list of exposed interfaces
# admin apis will be only accessible
# from the local host (inside VM) 
proxy_listen = 0.0.0.0:8000
admin_listen = 127.0.0.1:8001

4. Instruct Terraform to copy files

Next, we have to instruct Terraform to copy the files to new VM.

We use file provisioner to accomplish this. Add the following to main.tf

# copy the files
provisioner "file" {
    source      = "files/kong.conf"
    destination = "/tmp/kong.conf"
}

provisioner "file" {
    source      = "files/setup.sh"
    destination = "/tmp/setup.sh"
}
...

5. Update Terraform exec steps

Now we can update the remote-exec provisioner steps to run setup.sh after the VM setup is done.

Update main.tf

# run all the necessary commands via ssh shell 
provisioner "remote-exec" {
    inline = [
        # update &amp; install dependencies
        "apt-get update",
        "chmod 755 /tmp/setup.sh",
        "/tmp/setup.sh"
    ]
}
...

6. Destroy and Recreate Kong api gateway

After you make all the changes, run terraform destroy and terraform plan followed by terraform apply again.

You will see Terraform deleting the previous VM and creating the new VM with updated setup within minutes.

Summary

We have now created a simple Terraform project which allows us to quickly spin up Kong api gateway for future tutorial parts.

All the source code for files we have created / update in the above steps you can find on GitHub.