Elixir, GraphQL, and End-to-End Tests

“How do I test that?”

Usually, this is one of the first questions that comes to my mind when I’m starting to learn a new technology. And considering the type of systems that we have to handle modern day, It’s not hard to understand why.

These days, It’s very unusual to find any project where a big part of the effort isn’t centered around creating and maintaining a suite of tests that not only serves as documentation of the expected behavior of the application, but also helps to mitigate some of the risks of its constant evolution.

This is not different in the APIs development, but while for REST endpoints tests are an already established subject, there is still some discussion around its implementation for GraphQL APIs.

In this post, we’ll bring up a couple of approaches and considerations regarding test implementations for those APIs. In this case, we’ll focus in Elixir, but the discussion is extensible to other stacks as well.

GraphQL In The Elixir World

Even though the extensive documentation of GraphQL and its many implementations, the fact that it is a relatively new technology means that some aspects are not totally covered yet: tests implementation are one example.

For Elixir, we have Absinthe project that provides an implementation of the GraphQL specification together with a series of packages that brings an extensive set of tools with the goal of making the developer’s work easier.

The project is well documented, even going above and beyond by providing different examples for more advanced subjects such as authentication and file upload, but the implementation of tests is an area that is completely left in the developer’s account.

Running Queries/Mutations with Absinthe

When you combine this missing point with the different forms of executing a GraphQL query/mutation, we find a very fertile ground of questions for the developer that is implementing his first API.

Essentially we have three main ways of performing a query/mutation using Absinthe, and with that, three ways of testing your GraphQL API:

1. Triggering a HTTP Post to your endpoint

This is maybe the approach closest to the ones usually used with traditional APIs. The idea here is to simulate a POST request to the endpoint and validate the results the same way that we would with a Rest API.

The difference here is only the details around the request and the fact that instead of a set of URIs representing the system resources, we have only one single endpoint that receives the POST request, and depending on the query provided, returns of the respective result.

defmodule MyAppWeb.AcceptancePostTest do
  use MyAppWeb.ConnCase, async: true
  
  describe "create user" do
    test "creates a new user with valid attributes" do
      mutation = """
      mutation createUser($name: String, $email: String, $password: String) {
        createUser(name: $name, email: $email, password: $password) {
          name
          email
        }
      }
      """

      variables = %{name: "John Doe", email: "[email protected]", password: "123456"}

      response =
        build_conn()
        |> post("/graphql", %{query: mutation, variables: variables})

      assert json_response(response, 200) == %{
               "data" => %{"createUser" => %{"email" => "[email protected]", "name" => "John Doe"}}
             }
    end
  end
end

Code wise, there is nothing surprising here in this test. As we need to build a Connection, we must include MyWebApp.ConnCase and use post and json_response functions to trigger the request and check the result.

2. Using Absinthe.run/3 to execute the query/mutation

Another way of running a query is through Absinthe.run/3 function. With it, we can specify a query and the parameters to be executed in a specific Schema and we receive a tuple with the result.

defmodule MyAppWeb.AcceptanceRunTest do
  use ExUnit.Case, async: false

  describe "create user" do
    test "creates a new user with valid attributes running the schema" do
      mutation = """
      mutation createUser($name: String, $email: String, $password: String) {
        createUser(name: $name, email: $email, password: $password) {
          name
          email
        }
      }
      """

      variables = %{"name" => "John Doe", "email" => "[email protected]", "password" => "123456"}

      result = Absinthe.run(mutation, MyAppWeb.Schema, variables: variables)

      assert result ==
               {:ok,
                %{data: %{"createUser" => %{"email" => "[email protected]", "name" => "John Doe"}}}}
    end
  end
end

In this version, the first change to be noted is the absence of the need to include ConnCase module functions. We don’t need to trigger an HTTP request considering that the test focus is to verify the return of the run function. All the code related to the POST request and the json response verification gives place to a call to Absinthe.run with the expected parameters to the test case.

3. Running the resolver

Another alternative would be go down an extra level and test only the resolver of que query/mutation under question. Considering that a resolver is nothing more than a function with a specific signature that transforms a GraphQL operation in data, a test like that don’t differentiate in any way from a test around a “normal” function.

defmodule MyAppWeb.AcceptanceResolverTest do
  use ExUnit.Case, async: false
  
  describe "create user" do
    test "creates a new user with valid attributes through resolver" do
      variables = %{"name" => "John Doe", "email" => "[email protected]", "password" => "123456"}

      result = CognitaWeb.Resolvers.Accounts.create_user(nil, variables, %{})

      assert {:ok, %MyApp.Accounts.User{email: "[email protected]", name: "John Doe"}} = result
    end
  end
end

And now? Which one to choose?

Among the different ways of implementing our tests, we have to make the call on which approach to follow. After researching a little bit, it is possible to find suggestions to avoid tests that run the Schema, and to move all the behavior to be tested into its own function, which would lead the third approach to be preferable.

Under a certain point of view, this suggestion makes sense. While testing only the resolver, we get rid of all the boilerplate related to the query execution that simplifies the test. What might go unnoticed with this is that the “simplification” has its own repercusion.

Different types of tests have different goals, which makes each of them work best for specific granularities. Unit and Integration tests strive to exercise isolated components or specific integrations, and that allows a higher granularity to be used for this type of tests..

In an end-to-end test, the goal is to exercise the interaction among all the components that works together in a specific scenario, and that means those tests have to be less granular.

In these cases, a test where we execute the HTTP request allows us to verify a series of interactions among the components of the API (Router, Middlewares, Resolvers, etc.) which is not possible in the other approaches.

In a nutshell, each approach has its own implications and must be selected based on its unique situation.

Summarizing…

Although existing recommendations to use less granular tests that only exercise on the resolver of your GraphQL API is not the more adequate form of implementing an end-to-end test, they bypass the integration among a series of different components that exist in the application.

For these cases, it is interesting to have a test that exercises all the components of the application as a way of verifying the system functionality when all the components are integrated. This justifies the cost of a test that handles HTTP requests and json responses.

The best way to handle the cost that comes from this type of test is to make sure that they represent just a small part of your test suite, focusing on checking only the happy path for all the flows, and maybe a few exceptional cases without, however, eliminating them from your test database.

Written by Ronualdo Maciel, originally on PipeBlog – May 24, 2019