Caching
Unlike other tools you may have used, turbo
can cache emitted files and logs of previously run commands. It does this so it can skip work that's already been done, yielding incredible time savings.
By default, turbo run
considers any files emitted in the dist/**
or build/**
folders of a package task (defined in that package's package.json
scripts
object) to be cacheable. In addition, turbo run
treats logs (stderr
andstdout
), which are automatically written to .turbo/run-<command>.log
, as a cacheable artifact. By treating logs as artifacts, you can cache just about any command in your Turborepo.
Configuring Cache Outputs
Using pipeline
, you can configure cache conventions across your Turborepo.
To override the default cache output behavior, pass an array of globs to a pipeline.<task>.outputs
array. Any file that satisfies the glob patterns for a task will be treated as artifact.
{
"$schema": "https://turborepo.org/schema.json",
"pipeline": {
"build": {
"outputs": ["dist/**", ".next/**"],
"dependsOn": ["^build"]
},
"test": {
"outputs": [], // leave empty to only cache logs
"dependsOn": ["build"]
}
}
}
If your task does not emit any files (e.g. unit tests with Jest), set outputs
to an empty array (i.e. []
). Turborepo will automagically cache the logs for you.
Pro Tip for caching ESLint: You can get a cacheable pretty terminal output
(even for non-errors) by setting TIMING=1
variable before eslint
. Learn
more over at the ESLint
docs
With the above pipeline configuration and proper output folders, if you were to run:
# Run `build` npm script in all packages and apps and cache the output.
turbo run build test
And open up your node_modules/.cache/turbo
, you should see the cached artifacts.
Turn off caching
Sometimes you really don't need or want this cache behavior (such as when you're doing something with live reloading such as next dev
or react-scripts start
). To entirely disable caching, append --no-cache
to any command:
# Run `dev` npm script in all packages and apps in parallel,
# but don't cache the output
turbo run dev --parallel --no-cache
You can also always disable caching on a specific task by setting cache
to false
in your Turborepo pipeline
:
{
"$schema": "https://turborepo.org/schema.json",
"pipeline": {
"dev": {
"cache": false
}
}
}
Alter Caching Based on Environment Variables and Files
For some tasks, you may not want a cache miss if an irrelevant file has changed. For instance, updating README.md
might not need to trigger a cache miss for the test
task. You can use inputs
to restrict the set
of files turbo
considers for a particular task. In this case, only consider .ts
and .tsx
files relevant for
determining a cache hit on the test
task:
{
"$schema": "https://turborepo.org/schema.json",
"pipeline": {
// ...other tasks
"test": {
"outputs": [], // leave empty to only cache logs
"dependsOn": ["build"],
"inputs": ["src/**/*.tsx", "src/**/*.ts", "test/**/*.ts"]
}
}
}
When you use turbo
with tools which inline environment variables at build time (e.g. Next.js or Create React App), it is important you tell turbo
about it. Otherwise, you could ship a cached artifact with the wrong environment variables!
Luckily, you can control turbo
's cache fingerprinting (a.k.a. hashing) behavior based on the values of both environment variables and the contents of files:
- Including environment variables in a
dependsOn
in yourpipeline
definition prefixed by a$
will impact the cache fingerprint on a per-task or per-package-task basis. - Including environment variables in
globalDependencies
list prefixed by a$
will impact the cache fingerprint of all tasks. - Including files or globs of files in
globalDependencies
will impact the cache fingerprint of all tasks. - The value of any environment variable that includes
THASH
in its name will impact the cache fingerprint of all tasks.
{
"$schema": "https://turborepo.org/schema.json",
"pipeline": {
"build": {
"dependsOn": [
"^build",
// env vars will impact hashes of all "build" tasks
"$SOME_ENV_VAR",
],
"outputs": ["dist/**"]
},
"web#build": { // override settings for the "build" task for the "web" app
"dependsOn": [
"^build",
// env vars that will impact the hash of "build" task for only "web" app
"$STRIPE_SECRET_KEY",
"$NEXT_PUBLIC_STRIPE_PUBLIC_KEY",
"$NEXT_PUBLIC_ANALYTICS_ID",
],
"outputs": [".next/**"],
},
"docs#build": { // override settings for the "build" task for the "docs" app
"dependsOn": [
"^build",
// env vars that will impact the hash of "build" task for only "docs" app
"$STRIPE_SECRET_KEY",
"$NEXT_PUBLIC_STRIPE_PUBLIC_KEY",
"$NEXT_PUBLIC_ANALYTICS_ID",
],
"outputs": [".next/**"],
}
},
"baseBranch": "origin/main",
"globalDependencies": [
"$GITHUB_TOKEN",// env var that will impact the hashes of all tasks,
"tsconfig.json", // file contents will impact the hashes of all tasks,
".env.*", // glob file contents will impact the hashes of all tasks,
]
}
Pro Tip: In most monorepos, you don't often use environment variables in shared packages, but mostly only in applications. Thus, to get higher cache hit rates, you should likely only include environment variables in the app-specific tasks where they are used/inlined.
Force overwrite cache
Conversely, if you want to force turbo
to re-execute a previously cached task, add the --force
flag:
# Run `build` npm script in all packages and apps,
# ignoring cache hits.
turbo run build --force
Logs
Not only does turbo
cache the output of your tasks, it also records the terminal output (i.e. combined stdout
and stderr
) to (<package>/.turbo/run-<command>.log
). When turbo
encounters a cached task, it will replay the output as if it happened again, but instantly, with the package name slightly dimmed.
Hashing
By now, you're probably wondering how turbo
decides what constitutes a cache hit vs. miss for a given task. Good question!
First turbo
constructs a hash of the current global state of the monorepo:
- The contents of any files that satisfy the glob patterns and any the values of environment variables listed in
globalDependencies
- The sorted list environment variable key-value pairs that includes
THASH
anywhere in their names (e.g.STRIPE_PUBLIC_THASH_SECRET_KEY
but notSTRIPE_PUBLIC_KEY
)
Then it adds on more factors relative to a given package's task:
- Hash the contents of all not-gitignored files in the package folder or the files matching the
inputs
globs, if present - The hashes of all internal dependencies
- The
outputs
option specified in thepipeline
- The set of resolved versions of all installed
dependencies
,devDependencies
, andoptionalDependencies
specified in a package'spackage.json
from the root lockfile - The package task's name
- The sorted list of environment variable key-value pairs that correspond to the environment variable names listed in applicable
pipeline.<task-or-package-task>.dependsOn
list.
Once turbo
encounters a given package's task in its execution, it checks the cache (both locally and remotely) for a matching hash. If it's a match, it skips executing that task, moves or downloads the cached output into place and replays the previously recorded logs instantly. If there isn't anything in the cache (either locally or remotely) that matches the calculated hash, turbo
will execute the task locally and then cache the specified outputs
using the hash as an index.
The hash of a given task is injected at execution time as an environment variable TURBO_HASH
. This value can be useful in stamping outputs or tagging Dockerfile etc.
As of turbo
v0.6.10, turbo
's hashing algorithm when using npm
or pnpm
differs slightly from the above. When using either of these package managers,
turbo
will include the hashed contents of the lockfile in its hash
algorithm for each package's task. It will not parse/figure out the resolved
set of all dependencies like the current yarn
implementation.