What auditing 1000 endpoints told us about GraphQL Security Best Practices

What auditing 1000 endpoints  told us about GraphQL Security Best Practices
What Auditing 1000 endpoints has told us about GraphQL Security Best Practices

This post is a write-up of a talk I gave at the GraphQL San Francisco Conference. You can find the video on youtube. If you are in the bay area do not forget to also join the GraphQL SF group on meetup.

It's commonly said that GraphQL is less secure than REST by default.

As a company specialized in GraphQL Security, we have explored its security implications for more than a year on a wide range of production-grade endpoints.

In this talk, we use data from our free tool GraphQL.Security to explore what are the best practices for making GraphQL as secure, if not more, than REST.

About GraphQL.Security

GraphQL.Security is a free and simple online security checker for GraphQL. It quickly checks for 18 common security best practices on any GraphQL endpoint.

GraphQL.Security is simple & fast

It has been used by thousands of developers to quickly check the security of their endpoints, giving our team at Escape - GraphQL Security unique insights about the state of GraphQL Security.

The State of GraphQL Security

The following study has been done on a subset of 1000 endpoints scanned by GraphQL.Security over the last months. GraphQL.Security can only test endpoints that do not block all unauthenticated requests.

NB: Escape's full Live GraphQL Security Platform has authenticated endpoints scanning capabilities.

GraphQL.Security leverages our open source package GraphDNA to fingerprint the used engines, so we also have a unique insight into the market shares of each engine.

Not so surprisingly, Apollo is by far the most used engine here. Interestingly, GraphQLPHP is also quite used!

In total, we found 5869 security alerts among the tested endpoints, an average of 6 vulnerabilities per endpoint!

What were those endpoints vulnerable to?

Share of endpoints that have security misconfigurations
  • About 95% of endpoints still had HTTP level misconfigurations and potential CSRFs
  • 80% of endpoints were vulnerable to Denial of Service or Complexity based attacks
  • About 20% of endpoints were leaking sensitive information

Let's take a little break here: it looks like a lack of security is still endemic in the GraphQL ecosystem.

What are the GraphQL Security Best Practices and how to implement them?

I - Start with the basics: HTTP Layer and CSRF

It might look obvious and boring, but almost every production endpoint still lack proper security at the HTTP Level, and some are vulnerable to Cross-Site Request Forgery which could be easily available. Let's see how 👉

a) Set the right security headers

  • Activate the Strict-Transport-Security header to avoid request hijacking in non-secure environments. Even if redirection to HTTPS is activated.
  • Enforce Content-Type: application/json. multipart/form-data or
    application/x-www-form-urlencoded should not be accepted.
  • If your endpoint is not public, be sure that your Access-Control-Allow-Origin header is not set to “*”

b) Never allow mutations using GET requests

Contrarily to REST endpoints, GraphQL represents fetching and changing data using Queries and Mutations instead of GET's and POST's. Some endpoints forget to enforce HTTP status code to match Queries and mutations and allow Mutations through GET requests.

This is a bad practice, both from a development and a security point of view.

First, GET requests should be safe, idempotent, and cacheable per the HTTP specification. This is not compatible with mutations.

Secondly, when combined with Cookie authentication, it can lead to critical Cross-Site Request Forgery, when a malicious attacker can trick any of your users to execute a custom mutation just by clicking on a link.

II - Deeper into the Graph: Avoiding abusive queries

The anatomy of a GrapHQL Query

GraphQL is quite a full-featured language. Its queries have Operations, Fields, Directives, Parameters, Aliases, Batches, and Fragments. They have variable Width and Depth.

In fact, limits must be set for everything 🔒

Every single semantic token in a GraphQL Query can be abused to create a Denial of Service at the parser, or resolver level. Or even worse.

a) Limit Batching and Aliasing

GraphQL Batching is not part of the GraphQL specification itself. It's just a feature of some engines, including Apollo Server, that can process multiple GraphQL Operations sent in one single HTTP call.

GraphQL Aliasing on the other hand is quite equivalent but part of the GraphQL specification itself, thus supported by all engines. It allows to query the same field multiple times with different parameters using aliases.

The problem? Batching and Aliasing both can be used to bypass rate limiting at the HTTP level. In particular, if login is achieved through a mutation, this can open the door for bruteforce attacks.

Bruteforcing Login mutation using aliases

The solution (Apollo Server):

Deactivate Query Batching

const server = new ApolloServer({
  ...
  allowBatchedHttpRequests: false,
)}

Limit Query Aliasing

It's better not to deactivate aliasing completely but rather to limit overall request complexity. graphql-validation-complexity is a nice package that can help you do that quickly.

b) Limit Query Depth

The problem? Deep queries can be used to create arbitrary resource intensive operations.

query IAmEvil {
  author(id: "abc") {
    posts {
      author {
        posts {
          author {
            posts {
              author {
                # that could go on as deep as the client wants!
              }
            }
          }
        }
      }
    }
  }
}

Solution (Apollo Server)

You can easily limit the depth of queries with the very light graphql-depth-limit library.

Check how deep you expect legitimate queries to be and set a maximum depth accordingly.

import depthLimit from 'graphql-depth-limit'

const server = new ApolloServer({
  ...
  validationRules: [depthLimit(5)]
  });

c) Limit Query Complexity

The problem? Unlimited query complexity can be used to quickly gain access to a lot of data, and create denial of service.

Solution (Apollo Server)
Limit the complexity of queries to mitigate the abuse of GraphQL resources.

You can use a module to compute the complexity of each query and set a threshold on this complexity so that too broad requests are canceled.

To do so, you can either go with the simple to use graphql-validation-complexity module, which doesn't need you to modify your schema.

import { createComplexityLimitRule } from 'graphql-validation-complexity';

const ComplexityLimitRule = createComplexityLimitRule(1000);

const apolloServer = new ApolloServer({
    ...
    validationRules: [ComplexityLimitRule],
});

To get more information about complexity estimation, read : Securing Your GraphQL API from Malicious Queries

III - Avoiding Information Disclosure

Like a magician, a developer always keep his secrets 🔒

a) Deactivate Debug Mode in Production

The Problem? Stack Traces give too much information on what’s vulnerable in your app👀

Here, in this real life example, a stack trace in production give us information on the used version of Apollo-Gateway, which contains a publicly known vulnerability. Vulnerabilities databases like the CVE database make it very easy for malicious users to correlate stack traces with vulnerabilities.

How to avoid stack traces in production?

Always avoid database or code error stacktrace to be directly returned to the client.

When using Apollo, you can disable exception tracebacks in the response by either setting NODE_ENV to production or enforcing it:

const server = new ApolloServer({
  ...,
  debug: false
)}

For more details, see Apollo's documentation on omitting stacktraces (opens new window).

b) No Introspection? Disable Field Suggestion

The problem? Endpoints with disabled introspection generally still leak the full Schema through field suggestion.

Several engines, like Apollo Server, will send back an error message with suggestions when a field is entered with a typo:

Unfortunately, this can be leveraged to build back any introspection schema even when introspection is deactivated. An open source tool, Clairvoyance, even exists exactly for that purpose.

Example of a full introspection built back using Clairvoyance, with an unprotected update_admin mutation

At Escape, we have even found companies that were relying on disabling introspection only for protecting their admin endpoints. Yes, you heard me. Those were NOT AUTHENTIFIED, the developers thought that nobody could find them because introspection was deactivated.

But using Clairvoyance, we were able to quickly build back the full introspection, noticed an `update_admin` mutation that was unprotected, and were able to actually change the admin password.

How to avoid leaking introspection through field suggestions?

As of June 2022, Apollo has no option for deactivating field suggestions. A possible solution is to rewrite errors using a custom formatError middleware when in production.

const apolloServer = new ApolloServer({
  typeDefs: globalTypeDefs,
  resolvers: globalResolvers,
  ...
  // Mask errors from clients to mitigate security attacks
  formatError: (err) => {
    if (process.env.APP_ENV === 'production') {
      captureException(err);
      return new Error('Internal Server Error');
    }
    return err;
  },
});

Note: Escape's team is currently working on a plugin to solve that problem in a more convenient way.

Let's recap our best practices so far

  1. Start with the basics: HTTP layer and CSRF
    a. Set Strict-Transport-Security, Content-Type, and Access-Control-Allow-Origin headers
    properly
    b. Never allow mutations with GET requests
  2. Avoid abusive queries
    a. Limit query batching and aliasing
    b. Limit query depth
    c. Limit query complexity
  3. Avoid Information Disclosure
    a. Disable stack traces / debug mode in production
    b. If introspection is deactivated, also deactivate field suggestion

Going beyond best practices: Deep Graph Security Testing

At Escape, we understood that nice best practices weren't enough for making GraphQL as secure as REST.

Especially, good security testing capabilities are needed to be sure all resolvers, even deep in the graph, work properly from a security point of view.

But legacy API security testing is based on linear sequences of requests, which is not suitable for the non-linear nature of GraphQL.

We decided to create a whole new technical approach for testing the security of GraphQL Endpoints: Deep Graph Security Testing.

Instead of relying on a linear sequence of Requests, Deep Graph Security Testing dynamically explores the query graph and tests all the resolvers for:

  • Data leaks
  • Access control flaws
  • Injections in resolvers, even deeply nested ones
  • 40+ other kinds of vulnerabilities
List of All security tests performed by Escape Deep Graph Security Testing

Deep Graph Security Testing happens in two phases:
1) Our tool identifies the sensitive fields or objects inside the specification

2) Our Graph Exploration Algorithm explores all the paths leading to all fields, testing all resolvers, even deeply nested ones.

What have we found with that?

So far, hundreds of companies are already using Deep Graph Testing through Escape's platform.

Here is an example of what we have found:

Delivery, inc. (name has been changed), an uber eats competitor, was using a GraphQL Schema generator. It generated the GraphQL Schema using the database schema.

They were also logging users' login events in a specific table of their DB for analytics.

Sadly, this table also became accessible deeply in the GraphQL Schema under a userLoginHistory query, due to it being generated from the database. And even worse, this table actually contained the hashed password of the user that logged in.

Access control and data leak flaws are frequent in GraphQL even in production.

This vulnerability must not be overlooked as it is striking how many companies actually had critical security flaws in their GraphQL endpoints!

In fact, here is a non-exhaustive list of critical stuff found using Deep Graph Testing:

  • JWT Tokens
  • GCP Tokens
  • AWS keys
  • Personal Identifiable Information (Private phone numbers, Social Security Numbers)
  • SQL/NoSQL Injections
  • Database enumeration (IDORS)

Yes. That is scary. And that's why good security testing is a must.

Conclusion

By default, GraphQL is less secure than most REST implementations. Denial of Service, Cross-Site Request Forgery, and Sensitive Data Leaks are the most common flaws in GraphQL Endpoints.

Implementing Security Best Practices is a good start, but for real business-critical applications, security testing is a must.

To get security testing for GraphQL, you can try out Escape's Deep Graph Testing for free on our GraphQL Live Security Platform