Preventing DerivedData Locks in Parallel Xcode Builds

When two Xcode pipelines run concurrently on the same cloud Mac, intermittent database is locked messages, corrupted index databases, or linker errors that “cannot be reproduced locally” are often unrelated to code changes. Start by checking whether both jobs use the default ~/Library/Developer/Xcode/DerivedData. This directory is more than a cache: it contains build databases, indexes, module caches, and intermediate artifacts that jobs continuously rewrite. Treating it as a shared cache that supports concurrent writes will inevitably lead to failures.

Locate the Source of Contention First

Do not delete every cache as soon as a build fails. First preserve the failed job’s complete xcodebuild command, start time, working directory, and process list. Then search the logs for locked, database, unable to attach, malformed, and messages about duplicate outputs.

pgrep -alf 'xcodebuild|XCBuildService|swift-frontend'
find "$HOME/Library/Developer/Xcode/DerivedData" -name build.db -print
grep -Eini 'locked|database|malformed|multiple commands produce' build.log

If both jobs use the same -derivedDataPath, or neither explicitly sets the option, you have already identified the primary risk. Also check whether a pipeline timeout terminated only the outer script while leaving xcodebuild or compiler child processes running and continuing to write to the directory.

A successful retry does not prove that the problem is fixed. The second build may happen to pass because the contention window is shorter, while the shared-write relationship remains unchanged.

Distinguish Lock Contention from Ordinary Compilation Failures

Source compilation errors usually recur in the same file and at the same line. Lock contention is more sensitive to concurrency timing and may surface unpredictably while resolving dependencies, generating modules, or linking. If builds pass consistently with parallel jobs disabled and fail again when concurrency is restored, audit directory ownership before changing application code.

Assign an Isolated Workspace to Every Job

A reliable setup gives the repository checkout, DerivedData, result bundle, and temporary directory a unique job identifier. In a pipeline, that identifier can come from the job number; for local validation, the process ID is a suitable fallback.

set -euo pipefail

JOB_KEY="${CI_JOB_ID:-local-$$}"
ROOT="${RUNNER_TEMP:-$HOME/ci-work}/$JOB_KEY"
DERIVED_DATA="$ROOT/DerivedData"
RESULT_BUNDLE="$ROOT/TestResults.xcresult"

mkdir -p "$DERIVED_DATA"

xcodebuild \
  -workspace App.xcworkspace \
  -scheme App \
  -configuration Debug \
  -derivedDataPath "$DERIVED_DATA" \
  -resultBundlePath "$RESULT_BUNDLE" \
  build-for-testing

test -d "$RESULT_BUNDLE"

Each path must be created by one job and deleted only by that job. Do not use the project name as the sole directory identifier, because two branches of the same project will still collide. Cleanup scripts should also avoid broad matching, such as deleting every DerivedData directory whose name contains App.

Separate Dependency Resolution from Compilation

After isolating DerivedData, multiple jobs can still shift the contention to package resolution if they modify the same dependency checkout directory. Resolve dependencies first, then prevent the main build from changing dependency versions automatically. The repository should contain a reviewed lockfile, and the pipeline must not update it silently.

PACKAGES="$ROOT/SourcePackages"

xcodebuild \
  -resolvePackageDependencies \
  -workspace App.xcworkspace \
  -scheme App \
  -clonedSourcePackagesDirPath "$PACKAGES"

xcodebuild \
  -workspace App.xcworkspace \
  -scheme App \
  -derivedDataPath "$DERIVED_DATA" \
  -clonedSourcePackagesDirPath "$PACKAGES" \
  -disableAutomaticPackageResolution \
  build

If caching is required, cache downloaded content or a verified immutable snapshot rather than allowing multiple jobs to write to one active directory. At a minimum, the cache key should include the lockfile digest, Xcode version, and target architecture. Even after a cache hit, verify that the lockfile has not changed.

Define an Objective Acceptance Checklist

A single successful run is not enough to validate the fix. Start several rounds of jobs concurrently from the same commit, then verify the boundaries between directories, processes, and artifacts.

Run at least two identical builds concurrently, followed by a set of builds from different branches. The first test validates write isolation. The second exposes working directories, module caches, or artifact paths that are still shared by project name.

Preserve Evidence and Clean Up Safely

A failed job should not destroy its state immediately. First archive the build log, result bundle, Xcode version, lockfile digest, remaining disk space, and job paths. Source code, environment variables, and tokens in logs must be redacted beforehand. Successful jobs can be cleaned up after confirming that their artifacts have been copied to an independent release directory.

Cleanup should be scoped to the job root and protected by a path-prefix check. If a job is forcibly terminated, a later reclamation process should use the job identifier to remove its leftover directories instead of clearing the entire machine’s developer directory. When running multiple pipelines on NUMACS, confirm the currently available configurations in the console and set job limits according to the required concurrency. Higher concurrency is not a substitute for isolating writes.

The goal is not merely to make errors occur less often. It is to establish clear invariants: each parallel job has exactly one writer, dependency versions remain unchanged during the build, failed-job evidence remains traceable, and cleanup cannot affect other jobs. Once these four conditions hold, DerivedData stops being a source of random failures and becomes manageable build data again.

Frequently asked questions