Caching Tasks

Every JavaScript or TypeScript codebase will need to run package.json scripts, like build, test and lint. In Turborepo, we call these tasks.

Turborepo can cache the results and logs of your tasks - leading to enormous speedups for slow tasks.

Missing the cache

Each task in your codebase has inputs and outputs.

• A build task might have source files as inputs and outputs logs to stderr and stdout as well as bundled files.
• A lint or test task might have source files as inputs and outputs logs to stdout and stderr.

Let's say you run a build task with Turborepo using turbo run build:

1. Turborepo will evaluate the inputs to your task and turn them into a hash (e.g. 78awdk123).

2. Check the local filesystem cache for a matching cache artifact (e.g../node_modules/.cache/turbo/78awdk123.tar.zst).

3. If Turborepo doesn't find any matching artifacts for the calculated hash, Turborepo will then execute the task.

4. Once the task is successfully completed, Turborepo saves all specified outputs (including files and logs) into a new cache artifact, addressed by the hash.

Hitting the cache

Let's say that you run the task again without changing any of its inputs:

1. The hash will be the same because the inputs haven't changed (e.g. 78awdk123)

2. Turborepo will find the cache artifact with a matching hash (e.g. ./node_modules/.cache/turbo/78awdk123.tar.zst)

3. Instead of running the task, Turborepo will replay the output - printing the saved logs to stdout and restoring the saved output files to their respective position in the filesystem.

Restoring files and logs from the cache happens near-instantaneously. This can reduce your build times from minutes or hours down to seconds or milliseconds. Although specific results will vary depending on the shape and granularity of your codebase's dependency graph, most teams find that they can reduce their overall monthly build time by around 40-85% with Turborepo's caching.

Turn off caching

In some environments you don't want to write the cache output. To disable cache writes, append --no-cache to any command. For example, this will run dev (and all tasks that it dependsOn) in all workspaces, but it won't cache the output:

turbo run dev --no-cache

Note that --no-cache disables cache writes but does not disable cache reads. If you want to disable cache reads, use the --force flag.

You can also configure specific tasks to skip writing to the cache by setting the pipeline.<task>.cache configuration to false:

{
  "$schema": "https://turbo.build/schema.json",
  "pipeline": {
    "dev": {
      "cache": false,
      "persistent": true
    }
  }
}

Force overwrite cache

Conversely, if you want to disable reading the cache and force turbo to re-execute a previously cached task, add the --force flag:

# Run `build` npm script in all workspaces ignoring the cache.
turbo run build --force

Note that --force disables cache reads but does not disable cache writes. If you want to disable cache writes, use the --no-cache flag.

Handling Node.js versions

To account for Node.js versions, use the engines key in package.json (opens in a new tab). Turborepo will see the changes to your package.json and miss cache when the field is updated.

Handling platforms and other arbitrary hash contributors

For advanced use cases, you may want the operating system (OS), architecture, or other external factors to contribute to your hash. You can do this in four quick steps.

1. Write an arbitrary file to disk

First, create a script that accounts for the hash contributors that you are interested in. For example, here is a Node.js script that looks at platform and architecture and writes those details to a file (turbo-cache-key.json):

#!/usr/bin/env node
 
const { writeFileSync } = require('fs');
const { join } = require('path');
 
const { platform, arch } = process;
const file = "turbo-cache-key.json";
const str = JSON.stringify({ platform, arch });
console.log(`Generating cache key: ${str}`);
writeFileSync(file, str);

2. Add the file to your .gitignore

You won't want to commit this file to source control since it's dependent on environment. Add it to your .gitignore:

// .gitignore

+ turbo-cache-key.json

3. Add the file to the hash

Now, make sure that turbo is aware of the file by adding it to task inputs. You can do this two ways:

• For specific tasks: Include the file in the inputs array of the task(s):

{
  "$schema": "https://turbo.build/schema.json",
  "pipelines": {
    "build-for-platforms": {
      "dependsOn": ["^build"],
      "inputs": ["$TURBO_DEFAULT$", "turbo-cache-key.json"]
    }
  }
}

• For all tasks: Add the file to globalDependencies

{
  "$schema": "https://turbo.build/schema.json",
  "globalDependencies": ["turbo-cache-key.json"],
  "pipelines": {
    ...
  }
}

4. Generate the file before running turbo

Last, you'll want to ensure that you run the script before running turbo. For example:

{
  "scripts": {
    "build-for-platforms": "node ./scripts/create-turbo-cache-key.js && turbo run build"
  }
}

turbo run build will now take into account the contents of turbo-cache-key.json when calculating the hash for the build task.

Logs

Not only does turbo cache the output of your tasks, it also records the terminal output (i.e. combined stdout and stderr) to (<package>/.turbo/run-<command>.log). When turbo encounters a cached task, it will replay the output as if it happened again, but instantly, with the package name slightly dimmed.

Hashing

By now, you're probably wondering how turbo decides what constitutes a cache hit vs. miss for a given task. Good question!

First, turbo constructs a hash of the current global state of the codebase. This includes things like:

• Hash of the contents of any files that satisfy the glob patterns in globalDependencies
• The values of environment variables listed in globalEnv
• Select information from turbo.json, package.json, and any lockfile
• and more!

Then it adds more factors relevant to a given workspace's task:

• Hash of the contents of all version-controlled files in the workspace folder (or the files matching the inputs globs, if configured)
• The configured outputs specified in the pipeline
• The set of resolved versions of all installed dependencies, devDependencies, and optionalDependencies
• The workspace task's name
• The sorted list of environment variable key-value pairs specified in pipeline.<task>.env list.
• and more!

Once turbo encounters a given workspace's task in its execution, it checks the cache (both locally and remotely) for a matching hash. If it's a match, it skips executing that task, moves or downloads the cached output into place, and replays the previously recorded logs instantly. If there isn't anything in the cache (either locally or remotely) that matches the calculated hash, turbo will execute the task locally and then cache the specified outputs.

The hash of a given task is available to the task at execution time as an environment variable TURBO_HASH. This value can be useful in stamping outputs or tagging Dockerfile etc.