API-First Design

Introduction

We have been applying API-First design principles for several years now and advise our customers to do the same. As the name suggests, API-First design means designing a system around an API rather than adding an API to an already designed system as an afterthought.

Note: when we say API, we mean a REST-API using JSON object representation – this is because virtually all APIs use REST these days, due to the fact that REST is based on http(s) which has emerged as the dominant communication protocol of the internet.

Why API-First design?

The main reasons for designing a system around an API are:

* Its canonical: the API is the canonical form of a system – i.e. stripping off details of presentation, storage etc, the API represents the system design in its most reduced, pure form. The API should remain valid even as you swap out client and server implementations.

* You can’t do without it: the fragmentation of client technologies (web, mobile etc.) means that we need to support multiple clients, making an API almost unavoidable.

* You’d better start with it: if you don’t make an API the starting point of your design, it’s going to be very hard to achieve a correct API without any bleeding of implementation details.

How to do API-First design?

The starting point of API-First design is that the API is authored and verified independently of any implementation. Crucially, this means creating the API with a set of specialised API design tools such as OpenAPI (which we use), NOT generating it from Java or Javascript classes. This lesson is often hard for developers to learn because they are used to thinking of an API as a way to invoke their code, rather than the other way around (namely that their code serves the API).

What this means in practice is that when your customers request an API, you must first design the correct API and then figure out what changes are required to the back-end code to serve that API. This is more difficult than it sounds – APIs follow standards with respect to statelessness, security, paging etc which may be very hard to serve with existing code. If you give into the temptation of asking your API users to live with the limitations of your back-end implementations, you will regret it – your customers will be unhappy from the start and you will likely have to live with those limitations for the lifetime of the system, even after you have swapped out the server implementation. Of course you can make v2 of your API, but you will still have to serve v1 because some customers will not be able or willing to upgrade their clients.

Connecting your API to your back-end systems

We have said in the previous section that you should not generate the API from the back-end system, but rather design it independently. So you may be wondering how the API gets connected to your back-end system. After you’ve designed your API (we use the OpenAPI Swagger editor to author the API), we use an “inflector” to wire-up the API to the back-end logic. OpenAPI provides inflectors for various languages, including our two preferred back-end languages, Java and Python. The inflector takes the API definition (OpenAPI uses a yaml or JSON definition) and takes care of invoking the back-end methods defined in the API definition. In practice this means that if you have existing back-end logic, you’ll usually need to modify it or create adapter classes to map between the API and the back-end. That’s the price of API-First, but its a price worth paying to achieve a usable and robust API.

Follow standards!

Try not to reinvent any wheels. There are emerging, de facto, standards for designing REST APIs. Follow them. People will generally not thank you for designing new ways to handle naming, authentication, state or paging, for which there are already standards. Its tempting to think that your business case is unique and therefore deserves a unique API with its own standards, however, remember that customers will often have experience writing clients for multiple APIs and they will be grateful whenever you follow patterns that they already understand.

For example, a REST GET method which is named /messages will be expected to return a list of JSON Message objects. That should be the same Message object which is returned by GET /message/. There should not be an arbitrary MessageList class (this type of construct was a typical side-effect of generating an API from an existing language-specific implementation). the required paging metrics, such as the total items returned by a query, should be handled by out-of-band variables, such as an X-Total-Count header, thus leaving the contents as a simple object list.

However, these are only emerging standards and there’s no single definitive guide. The best way to learn them is by consuming (i.e. writing clients for) several well known REST APIs and by reading best-practice articles (see below for some links).

More reading

Zalando restful API guidelines
Red Hat – thoughts on restful design

git workshop

Introduction

In this post, we’ll cover the basics of using the git source control system, which has emerged as the dominant tool in this space (especially since Microsoft paid $7.5B for github.com).

Audience

This workshop is intended for developers who are already aware what a source control system is and why you need it.

Platforms

We’ll do this workshop on Ubuntu, but git itself is completely cross-platform, so everything in this workshop is applicable for other platforms such as macOS and Windows. There are various GUIs available, but we’ll concentrate on the command line, since that’s what you should learn first. Many IDEs and editors (like VSCode) have git support built in, but it’s still important to be comfortable with the git command-line.

Installing git

Note: on macOS, git is preinstalled and on Windows you can install it from https://git-scm.com

Creating a repository

The first thing to note is that, unlike previous generations of source-control systems, git does not require a server – it creates its local repositories inside hidden folders (.git) in your project folder.

We’ll start by creating a “git-workshop” folder which we’ll use to hold our example project.

We’ll create a single index.html to represent our project source files.

Now we’ll create a new git repository:

Now we’ll check the status of the git repository with:

This tells us that index.html is untracked – i.e. hasn’t yet been added to the repository, so lets add it. We use “git add .” to tell git to add all files in the current directory to the git repository.

Adding files to the repository

git calls this “staging” files. When a file has been added to a git repository its considered to be in the staging area.

Now lets check the status again:

Now we see that the files are “staged”, which means that git knows about them. We still need to commit them to create a version of the file in the repository, so lets go ahead and do that with the “git commit” command:

Committing versions of your files

OK, git needs some additional configuration before the first commit, so lets do that:

Now we’re ready to commit our project files to the repository.

That’s better, now git has committed index.html to the repository.

If we now do a git status:

We can see that everything has been committed to the repository. So if we now make a change to index.html, we would expect to have one version of the file in our directory and another, older, version in the repository. Let’s go ahead and do that.

and check with git status:

We can see that git knows that we’ve modified index.html. git will also tell us what has changed if we use git diff:

OK, lets commit the changed version of index.html to the repository:

Note that we gain had to add index.html. This seems a little counter-intuitive, but it’s the way git works – each version of a file must be added and then committed.

We can now get a history of our index.html file with git log:

Rolling back changes

One of the reasons why you need git is in order to be able to revert to a previous version of your project when you realise that the changes you’re currently making are going nowhere.

Lets first make a change to index.html.

In a real project, we may have been working for hours and have made many changes, which we now regret and wish to roll back. There are several ways to do this with git, but the simplest one just gets us the latest committed version back again as follows:

Lets check that our changes have been rolled back:

Looks good!

Removing files

If you want to remove a file from the repository (“unstage” it), you can do it with git rm .

Deleting the git repository

You can also delete the whole repository simply by removing the .git folder (“$ rm -rf .git”). If you do that, you’ll need to start over again with git init.

Recap

So we’ve learnt that git creates a local repository in a “.git” subfolder when you type “git init”. We’ve also covered adding and committing files to the repository and reverting to a previously committed version.

Until now, you’re working alone with a local repository – this covers your own needs for maintaining versions as you work. However, the real power of version control is when you are working with other people on a project. In the next section we’ll therefore cover remote repositories.

Remote repositories

git supports multiple levels of repositories. Usually, a project will have local repositories on each of the developer’s machines and a central, main repository which holds the contributions of all the developers involved in the project.

This central repository may be in the cloud (like github.com) or it may be privately hosted by a company. For the purposes of this workshop, we’ll use github.com as remote repository.

I logged onto my account on github.com and created a repository called “rogerarmstrong/git-workshop”. Github helpfully gave me a list of onscreen instructions for setting this up as a remote repository for my project. I need two steps “git remote add” and “git push” as shown below:

So now my project is committed to both the local git repository and to the remote repository at github.com. New members of the project team can now do a “git clone https://github.com/rogerarmstrong/git-workshop.git” to start working on the project. This one line will initialise a local repository and get the latest project files from the central repository in a single step.

Working with the remote repository

We continue committing to the local repository as before and when we are ready to publish our changes to the rest of the project team, we push our changes to the remote repository.

We can double-check online that our changes have been pushed out to the remote repository by looking at the file online at github.com:

To get changes pushed to the remote repository by other developers, we need to do a “git pull”:

Branching

Branching allows you to commit to a separate path than the main (“master”) development. This means that you can continue to commit your changes, but they won’t affect team members who are working on the master or another branch. Below we create a branch called “experimental”:

If we now do a git status, we’ll see that we are on the branch “experimental”.

If we now make a change to index.html and commit it, it’ll be committed to the “experimental” branch.

We can now switch back and forward between “experimental” and master branches:

We can see all branches with:

If we now decide we like the experimental branch and want to merge it into the master:

Finally, we can delete the experimental branch:

Summary

Thats it for now. We hope you enjoyed our basic introduction to git. We have covered how to create a local git repository, how to add and commit files to it, how to create a remote repository and push changes out to it and pull changes from it and finally how to create branches. git has many more commands to deal with the complexities of real-world projects such as conflict resolution (during merging of changes from multiple developers) – we’ll leave you to explore these more advanced topics for yourself. Have fun!