How to Export Data from Sitecore: Extraction Methods, API Limits & Portability
Learn every method for exporting data from Sitecore: SCS, SPE, GraphQL, Content Transfer API, xConnect, and more — with real API rate limits and coverage gaps.
Planning a migration?
Get a free 30-min call with our engineers. We'll review your setup and map out a custom migration plan — no obligation.
Schedule a free call- 1,500+ migrations completed
- Zero downtime guaranteed
- Transparent, fixed pricing
- Project success responsibility
- Post-migration support included
How to Export Data from Sitecore: Extraction Methods, API Limits & Portability
Sitecore has no single "Export Everything" button. Your export method depends on which Sitecore product you're running (XP, XM, XM Cloud, Content Hub, CDP), what data you need (content items, media, analytics, customer profiles), and whether the target is another Sitecore environment or a different platform entirely.
Sitecore's architecture is not one system — it's a constellation of products, each with its own storage model, APIs, and extraction constraints. Content items in Sitecore XP live in SQL Server databases using an Entity-Attribute-Value (EAV) schema. XM Cloud content is managed through cloud-hosted authoring and delivered via Experience Edge's GraphQL API. Content Hub stores digital assets behind a throttled REST API. CDP behavioral data sits in a managed data lake exportable through Amazon S3.
A migration-ready extract has a specific meaning: relational links resolved to absolute URLs, media blobs extracted to standard files with correct MIME-to-extension mapping, rich text fields with internal dynamic links converted to paths, all language versions captured, and workflow states recorded — in a format readable by a system that has no knowledge of Sitecore's internal schema. Achieving this requires programmatic extraction and usually a combination of tools. Much like Kentico's native export packages, a standard Sitecore Package (.zip) is only useful if you are moving data to another Sitecore environment; it is practically useless for external migrations.
TL;DR: Use Sitecore Content Serialization (SCS) for YAML-based item export on XP 10.x+ and XM Cloud. Use SPE for custom bulk CSV/JSON extracts on XP. Use the Content Transfer API for XM Cloud environment-to-environment moves. Use the Experience Edge GraphQL API for published content. Use xConnect or the Data Export Tool for analytics. Every method has coverage gaps — media binaries, version history, and permissions each require separate handling.
Sitecore Export Methods Compared
| Method | Content Items | Media | Users/Permissions | Version History | Analytics | Best For |
|---|---|---|---|---|---|---|
| Sitecore Content Serialization (SCS) | ✅ YAML | ✅ (serialized) | Partial | ✅ (if configured) | ❌ | Dev sync, CI/CD, cross-env migration |
| Content Transfer API + Item Transfer API | ✅ .raif | ✅ | ✅ | ✅ | ❌ | XM Cloud env-to-env migration |
| PowerShell Extensions (SPE) | ✅ CSV/XML/JSON | ✅ (scripted) | ✅ (scripted) | ✅ (scripted) | ❌ | Custom bulk extraction for replatforming |
| Experience Edge GraphQL | ✅ JSON (published only) | URLs only | ❌ | ❌ | ❌ | Headless delivery, published content |
| Content Hub REST API | ✅ JSON | ✅ (Download API) | ❌ | ❌ | ❌ | DAM asset extraction |
| xConnect / Data Export Tool | ❌ | ❌ | ❌ | ❌ | ✅ | Contact and interaction exports |
| CDP Data Lake Export | ❌ | ❌ | ❌ | ❌ | ✅ (behavioral) | Full tenant data to S3 |
| Package Designer (.zip) | ✅ (Proprietary XML) | ✅ (Bundled) | ✅ | Partial | ❌ | Moving items between Sitecore instances |
| Unicorn / TDS | ✅ YAML/XML | ❌ | ❌ | Partial | ❌ | XP 7.x–9.x serialization (pre-SCS) |
| Direct SQL Queries | ✅ (raw EAV) | Partial | Partial | ✅ | ❌ | Emergency extraction, on-prem only |
Version Compatibility Matrix
Not all export methods are available on all Sitecore versions. This is a critical decision variable that narrows your options before anything else.
| Export Method | Sitecore 8.x | Sitecore 9.x | Sitecore 10.x | XM Cloud |
|---|---|---|---|---|
| SCS (dotnet CLI) | ❌ | ❌ | ✅ (10.1+) | ✅ |
| Unicorn | ✅ | ✅ | ✅ | ❌ |
| TDS (Team Development for Sitecore) | ✅ | ✅ | ✅ | ❌ |
| SPE (PowerShell Extensions) | ✅ (2.x+) | ✅ | ✅ | ❌ (no server access) |
| Package Designer | ✅ | ✅ | ✅ | ❌ (deprecated July 2026) |
| Content Transfer API | ❌ | ❌ | ❌ | ✅ |
| Experience Edge GraphQL | ❌ | ❌ | ❌ | ✅ |
| XM to XM Cloud Migration Tool v2 | ❌ | ✅ (9.1+) | ✅ | Target only |
| xConnect / Data Export Tool | ❌ | ✅ | ✅ | ❌ |
| CDP Data Lake (S3) | ❌ | ❌ | ❌ | ✅ (CDP-licensed) |
| Direct SQL | ✅ | ✅ | ✅ | ❌ |
Key takeaway: Teams on Sitecore 8.x or 9.x cannot use SCS. Their serialization options are Unicorn (open source, YAML-based) or TDS (commercial, MSBuild-integrated). Both tools are still actively maintained and widely deployed across the existing XP install base — any migration plan that ignores them is incomplete.
How Does Sitecore Store Data?
Sitecore XP uses SQL Server with a proprietary schema across three primary databases. The Master database contains all versions of all content supporting CMS users. The Web database contains published versions from Master supporting the website(s). The Core database contains data controlling the Sitecore CMS user interfaces.
The item data is split across four key tables: Items (all items), VersionedFields (fields with different values per version and language), UnversionedFields (language-specific but version-independent fields), and SharedFields (values identical across all languages and versions).
This is an Entity-Attribute-Value (EAV) structure. A single blog post might be scattered across hundreds of rows in these tables, linked only by GUIDs. You cannot SELECT * FROM pages — extracting a single content item with all its field values requires joining across multiple tables with GUID-based field IDs, filtering by language, and selecting the highest version number.
Sitecore's own documentation states: you should not use SQL commands to access Sitecore databases. The supported extraction paths use Sitecore's APIs, serialization tools, or PowerShell — not raw SQL.
XM Cloud replaces the local SQL databases with a cloud-hosted authoring environment. Content is delivered through Experience Edge's GraphQL API. You don't have direct database access.
Content Hub stores digital assets and metadata behind its own REST API layer. Sitecore CDP stores behavioral data in a managed data lake. xDB (Experience Database) in XP 9.x+ supports both MongoDB and SQL Server as the collection database backend — Microsoft SQL Server for xDB was introduced as a supported option in XP 9.0.
Sitecore Content Serialization (SCS): The Modern Export Path
Sitecore Content Serialization (SCS) is the recommended method for extracting content items as files from Sitecore XP 10.1+ and XM Cloud. SCS is not available for Sitecore 8.x or 9.x — for those versions, use Unicorn or TDS.
You serialize content items in and out of a Sitecore instance using SCS by configuring which content items to include and which ones to exclude, and which operations to perform. SCS serializes content items into your project folder as YAML files. The default path is \serialization relative to the module file, but you can configure the relative serialization path.
Authentication requirements
SCS CLI requires authentication against the Sitecore instance before pull/push operations can execute. Run dotnet sitecore login and authenticate against the CM endpoint. For XM Cloud, authentication uses Sitecore Cloud Portal credentials with environment-specific tokens. For XP 10.x, you configure the endpoint URL and credentials in user.json.
How to pull items with SCS
Define what to serialize in your Project.module.json:
{
"items": {
"includes": [
{
"name": "content",
"path": "/sitecore/content/home"
}
]
}
}Then run:
dotnet sitecore ser pullThe pull subcommand takes items from Sitecore, serializes them, and stores them as .yml files on a given file system path. The subcommand uses Module.json files to determine what items are included in the serialization process.
What an SCS YAML file looks like
A serialized content item in SCS YAML format:
---
ID: "11111111-1111-1111-1111-111111111111"
Parent: "22222222-2222-2222-2222-222222222222"
Template: "33333333-3333-3333-3333-333333333333"
Path: /sitecore/content/Home/Articles/My-Article
SharedFields:
- ID: "44444444-4444-4444-4444-444444444444"
Hint: __Workflow
Value: ""
Languages:
- Language: en
Versions:
- Version: 1
Fields:
- ID: "55555555-5555-5555-5555-555555555555"
Hint: Title
Value: My Article Title
- ID: "66666666-6666-6666-6666-666666666666"
Hint: Body
Value: "<p>Article content with <a href=\"~/link.aspx?_id=ABC&_z=z\">internal link</a></p>"Note the Body field: the internal link is stored as a Sitecore dynamic link (~/link.aspx?_id=...), not a URL. Any tool consuming this YAML for a non-Sitecore target must resolve these before import.
What SCS does not export
- Analytics and behavioral data — these live in xConnect/xDB, not the content databases
- Workflow state execution history — the current workflow state serializes, but the audit trail does not
- Computed or rendered output — SCS captures raw field values, not the HTML your renderings produce
- Content Hub assets — Content Hub is a separate product with its own API
- ASP.NET membership data — user accounts and roles require SPE or the XM Migration Tool
For cross-version migrations (e.g., Sitecore 9.x to XM Cloud), SCS alone isn't enough. Teams commonly use PowerShell scripts to export item field data into CSV files, which are later imported into XM Cloud using import scripts. This is especially true when the target uses entirely new templates and renderings.
Unicorn and TDS: Pre-SCS Serialization for XP 8.x–10.x
For Sitecore versions before 10.1, or teams that adopted these tools before SCS was introduced, Unicorn and TDS are the standard serialization options. Both remain in active use across the existing XP install base.
Unicorn (open source, GitHub: SitecoreUnicorn/Unicorn) serializes items to YAML using the Rainbow serialization format. Configuration uses .config files with predicate definitions that specify which Sitecore paths to include. Sync operations run via the /unicorn.aspx control panel or via the Unicorn CLI tool (Unicorn.MicroCLI.exe). Unicorn does not require a separate license.
TDS (Team Development for Sitecore) (commercial, Hedgehog Development) serializes items to a proprietary XML format (.item files) and integrates directly into Visual Studio's MSBuild pipeline. TDS supports automated deployment, validation rules, and delta deployments. It is more tightly integrated with CI/CD pipelines than Unicorn but requires a per-developer license.
Key differences from SCS:
- Neither Unicorn nor TDS serializes media binaries
- TDS
.itemformat is not human-readable YAML; it requires the TDS toolchain to consume - Unicorn YAML is structurally similar to SCS YAML but uses a different schema — they are not interchangeable without conversion
- Neither tool works on XM Cloud
If you are migrating from XP 8.x or 9.x to XM Cloud, your first step is typically converting Unicorn or TDS serialization to SCS-compatible YAML, or using the XM to XM Cloud Migration Tool (v2, which supports XM 9.1+) to bypass serialization entirely.
Content Transfer API: XM Cloud's Package Designer Replacement
The Content Transfer API and Item Transfer API are Sitecore's replacement for the Package Designer in XM Cloud. Starting with base image version 1.8.24 and effective July 7, 2026, the Package Designer is no longer available in XM Cloud. If you previously used the Package Designer to transfer content between environments, you can now use the Content Transfer API and the Item Transfer API to accomplish the same tasks.
The Content Transfer API securely exports content items from one environment into another. The API packages content data from your source environment into .raif files in the destination environment. The Item Transfer API loads the .raif files into the destination database.
The key advantage: the Content Transfer API uses chunked streaming, so there is no need to break large migrations into multiple smaller packages.
Authentication requirements: The Content Transfer API requires an XM Cloud OAuth token scoped to the specific environment. Tokens are obtained via the Sitecore Cloud Portal API or Management API. Operations require the ContentTransfer.Admin role on the source and destination environments.
The XM to XM Cloud Migration Tool (Version 2) delivers fast data transfers built on Google's Protocol Buffers (Protobuf) format and supports migrations from Sitecore XM on-premises and PaaS versions 9.1 and later.
The Content Transfer API moves data between Sitecore environments. If your target is a non-Sitecore platform, these APIs won't help — you'll need SCS, SPE, or the GraphQL API to extract data into a portable format.
How Do You Extract Data Using Sitecore PowerShell Extensions (SPE)?
Sitecore PowerShell Extensions (SPE) is the most flexible extraction tool for Sitecore XP and the de facto standard for custom bulk exports. It runs within the Sitecore context, meaning it has full access to the Sitecore API to resolve links, determine language fallbacks, and traverse the content tree.
SPE lets you script arbitrary queries against the content tree and output to CSV, JSON, XML, or Sitecore packages. This is the primary method teams use when migrating to a non-Sitecore target, because it gives you flat, mappable data with full control over field mapping.
Authentication requirements: SPE runs inside the Sitecore instance and uses standard Sitecore authentication. The executing user account requires read access to the content paths being exported, plus the PowerShell Extensions role. SPE is not available on XM Cloud without custom workarounds involving the Management API.
SPE extraction runbook
When writing an SPE export script, handle these Sitecore-specific quirks:
- Context Database: Run against the
masterdatabase for the latest working versions, or thewebdatabase if you strictly need only published content. - Field Types: Complex fields (Multilist, Treelist, Droplink) store data as pipe-separated GUIDs. Your script must split these and look up the target items to extract their actual values.
- Language Versions: Sitecore supports multiple languages and versioning. Explicitly request the latest version of each target language.
- Dynamic Links: Rich text fields contain internal Sitecore links stored as GUIDs. Use the
LinkManagerto expand these into absolute URLs during export — not after import.
Here is a conceptual example of using SPE to extract articles to JSON:
$exportPath = "master:/sitecore/content/Home/Articles"
$articles = Get-ChildItem -Path $exportPath -Recurse | Where-Object { $_.TemplateName -eq "Article" }
$exportData = @()
foreach ($article in $articles) {
$itemData = [PSCustomObject]@{
Id = $article.ID.ToString()
Title = $article.Fields["Title"].Value
# Resolve dynamic links to absolute URLs
Body = [Sitecore.Links.LinkManager]::ExpandDynamicLinks($article.Fields["Body"].Value)
Path = $article.Paths.Path
Language = $article.Language.Name
}
$exportData += $itemData
}
$exportData | ConvertTo-Json -Depth 5 | Out-File -FilePath "C:\exports\articles.json"For package creation:
$package = New-Package "Content-Export"
$source = Get-Item -Path "master:\content\home" |
New-ItemSource -Name 'Home Content' -InstallMode Overwrite
$package.Sources.Add($source)
Export-Package -Project $package -Path "$($package.Name).zip" -Zip
Download-File "$SitecorePackageFolder\$($package.Name).zip"SPE performance characteristics and limits
SPE runs synchronously in the Sitecore UI by default. Based on typical XP 10.x deployments with 8–16 GB IIS worker memory allocation:
- Under 5,000 items: Synchronous SPE exports typically complete without issue
- 5,000–15,000 items: Risk of IIS application pool recycling due to memory pressure; background sessions recommended
- 15,000–50,000 items: Synchronous export will time out or crash; chunked export by subtree or date range required
- 50,000+ items: SPE alone is impractical; use SCS pull for item metadata and script media downloads separately
There is no built-in pagination in SPE. To mitigate this: wrap large exports in Sitecore Scheduled Tasks or use SPE's background session capabilities (Start-ScriptSession). For datasets exceeding 10,000 records, chunk the export by date modified or by sub-folder path.
Common failure modes:
System.OutOfMemoryExceptionwhen accumulating large$exportDataarrays in memory before writing- IIS worker recycle mid-export with no partial output saved (mitigate by writing incrementally to file)
Sitecore.Data.Items.Itemaccess returning null for items with broken links or missing templates- Timeout on the SPE remote session when running via the ISE console (increase
remoteScriptExecutionTimeoutin SPE config)
What Are the Experience Edge GraphQL API Limits?
Experience Edge enforces a rate limit of 80 requests per second per environment. The Delivery GraphQL API rate limit is 80 requests per second, visible in the X-Rate-Limit-Limit response header. If you exceed the limit, Experience Edge returns a 429 response.
Other hard limits:
- A single query can return a maximum of 1,000 entities. To retrieve more entities, use pagination.
- Sitecore imposes a query complexity limit (default: 250) — fields, nesting, and depth all contribute.
- The
allSiteInfoquery's offset (pageNumber × pageSize) cannot exceed 10,000. - The sum of strings provided to
includedPathsandexcludedPathsarguments must not exceed 100. - Maximum media item size: 50 MB
Experience Edge only serves published content. If you need draft versions, version history, or workflow states, you must use SCS, SPE, or direct API access to the authoring environment — not the Edge delivery endpoint.
Querying Experience Edge for bulk extraction
query GetArticles($cursor: String) {
search(
where: {
name: "_templates",
value: "{ARTICLE-TEMPLATE-GUID}",
operator: CONTAINS
}
first: 100
after: $cursor
) {
pageInfo {
hasNextPage
endCursor
}
results {
... on Article {
id
title {
value
}
content {
value
}
url {
path
}
}
}
}
}Implement cursor-based pagination using hasNextPage and endCursor. At 80 req/sec and 100 items per page, extracting 10,000 published items takes approximately 2 minutes of sustained requests, not accounting for query complexity throttling. If your migration script hits a 429, implement exponential backoff — start at 1 second and double up to a maximum of 30 seconds.
Common failure modes:
- 429 errors from bursting above 80 req/sec during parallel extraction threads
- Query complexity errors when deeply nesting component datasource references
endCursorbecoming invalid if content is published or unpublished during extraction (treat as a restart condition)- Media URLs returning 403 if the Edge CDN token has expired mid-extraction
Experience Edge automatically resolves internal links and media URLs, bypassing the classic Sitecore GUID problem that plagues SPE and direct SQL exports. This is its primary advantage for extraction to non-Sitecore targets.
What Are the Sitecore Content Hub API Limits?
Content Hub's default rate limit is 15 API calls per second per integration user. Sitecore Content Hub backend services apply throttling to provide consistent performance across all services. When the backend is receiving too many requests, it returns 429 (Too Many Requests). Make sure a rate limit of 15 calls per second (per API integration user) is respected.
Customers can opt to increase this in 15 calls/second increments by contacting their Sitecore Account Manager.
Sitecore recommends using the WebSDK, which has embedded throttling, over the REST API (for which you have to code the retry mechanism yourself). The WebSDK's built-in retry policy retries any call returning HTTP 429 nine times by default.
For UI-based exports: exporting data from Content Hub through the UI limits the selection to 2,000 items — the maximum number of entities that can be exported. Since there is a limit to export 2,000 items through UI, REST APIs can be leveraged to export all the entities for the entity definition.
Content Hub ONE (the lightweight headless CMS, distinct from the full DAM product) has a tighter limit: 10 requests per second per tenant. These two products share branding but have different API limits, different authentication models (Content Hub uses OAuth 2.0 with client credentials; Content Hub ONE uses API keys), and different data models. Do not treat their export procedures as interchangeable.
Practical extraction rate at 15 calls/sec for Content Hub DAM:
- A library of 50,000 assets at 15 req/sec with 100 assets per page response requires ~334 list API calls ≈ 23 seconds of list calls
- Each asset then requires separate download API calls for the binary — download calls count against the same rate limit
- Total extraction time for 50,000 assets with full binary download: 3–8 hours depending on file sizes and network throughput
Authentication requirements: Content Hub REST API uses OAuth 2.0 with client credentials flow. Integration users must be created in the Content Hub admin panel with appropriate entity permissions. The WebSDK handles token refresh automatically; custom REST implementations must refresh tokens before expiry (tokens are valid for 1 hour by default).
The Internal Link Problem (Dynamic Links)
Sitecore does not store standard HTML links in its rich text fields. If a content editor links to an internal "About Us" page, Sitecore stores a dynamic link:
<link text="Click here" linktype="internal" class="" title="" target="" querystring="" id="{11111111-1111-1111-1111-111111111111}" />
This architecture prevents broken links when items are renamed or moved within the content tree. But it makes raw database exports and naive API extracts useless for external systems.
Resolution approaches by method:
| Method | Link Resolution | Notes |
|---|---|---|
| SPE | Manual — call LinkManager.ExpandDynamicLinks() |
Must be explicit in script |
| SCS YAML | None — raw dynamic link stored | Must resolve during post-processing |
| Experience Edge GraphQL | Automatic | Edge resolves to CDN URLs at query time |
| Direct SQL | None — raw GUID stored | Must build resolution table from Items schema |
| Content Transfer API | Preserved as-is (Sitecore-to-Sitecore) | Works because destination is also Sitecore |
If you use SPE or the Sitecore API targeting a non-Sitecore destination, you must explicitly call the LinkManager to expand these GUIDs into absolute or relative URLs before writing data to your export file. Skip this step and your target CMS will import thousands of broken, proprietary tags — a problem similar to extracting raw HubL code from HubSpot.
How to Export the Sitecore Media Library
Extracting images, PDFs, and other media from Sitecore presents its own challenges. Sitecore stores media in two ways, depending on how the system was originally configured:
- Database Storage (Default): Media files are stored as binary blobs directly in the SQL database, in the
Blobstable of the Master and Web databases. - File System Storage: Media files are stored on the server's disk under the
/App_Data/MediaCacheor a configured path, and the database holds only metadata.
For database-stored media, you cannot simply copy a folder. You must write an SPE script or use the Sitecore API to download each blob, reconstruct the file extension based on the MIME type field stored on the media item, and save it to disk.
SCS serializes media item metadata (name, alt text, dimensions, MIME type) but does not extract the binary blob for large media libraries. For those, use SPE's [Sitecore.Resources.Media.MediaManager]::GetMedia() approach or download via the Sitecore media URL (/-/media/) for each item.
Practical extraction rates for media:
- SPE-scripted media download via the application layer: approximately 200–500 MB/hour on a typical XP instance under production load
- Direct blob extraction from SQL (emergency only): faster but bypasses MIME resolution and security
- For media libraries exceeding 10 GB, plan media extraction as a separate overnight workstream
Media extraction scripts are notoriously slow. Downloading 50 GB of blobs through the Sitecore application layer will take hours and heavily tax the server. Schedule media extraction during low-traffic windows and write incrementally to disk — do not buffer the full library in memory.
How to Export Analytics and Behavioral Data
xConnect Data Export Tool (Sitecore XP)
If you are using Sitecore XP 9.x or 10.x, user data, contact profiles, and interaction history are not stored in the master SQL database. They live in the Experience Database (xDB). xDB supports two collection database backends: MongoDB (common in XP 9.0 and earlier deployments) and Microsoft SQL Server (supported from XP 9.0+, more common in 10.x). Both backends are accessed through the xConnect service layer — you do not query the underlying store directly.
The Data Export Tool uses the Processing Engine distributed worker together with the xConnect Data Extraction API to extract data from your collection database. You can export contacts or interactions with all the registered contact or interaction facets in the model. The export always exports all facets. You can filter the data using a set of conditions, for example, cut-off days and percentile ranges.
The export endpoints:
POST <instance>/sitecore/api/datatools/export/tasks/contacts
POST <instance>/sitecore/api/datatools/export/tasks/interactions
The BatchSize parameter controls the amount of data extracted from the collection database for each cursor read, defaulting to 100. Tune this upward (to 500–1000) for large exports, but monitor memory on the processing worker — higher batch sizes increase heap pressure on the xConnect search indexer.
Authentication requirements: xConnect uses certificate-based mutual TLS authentication. Client certificates must match the thumbprint configured in xconnect.collection.client.certificateThumbprint in the xConnect role configuration. Calls without a valid client certificate return 403 regardless of credentials.
For programmatic extraction:
using (var client = SitecoreXConnectClientConfiguration.GetClient())
{
var queryable = client.Contacts
.Where(c => c.Identifiers.Any(i => i.Source == "Alias"))
.WithExpandOptions(new ContactExpandOptions(
PersonalInformation.DefaultFacetKey,
EmailAddressList.DefaultFacetKey));
var results = await queryable.GetBatchEnumerator(100);
while (await results.MoveNext())
{
foreach (var contact in results.Current)
{
// Map contact facets to external format
}
}
}Because xDB can contain millions of interactions, use GetBatchEnumerator to paginate through the data. Facets must be explicitly requested via ContactExpandOptions — omitting a facet key means that data is not returned even if it exists. The xConnect OData search endpoint has a maximum page size of 1,000 contacts per request.
Common failure modes:
- Certificate thumbprint mismatch causing 403 on all xConnect calls
- Missing facet keys in
ContactExpandOptionsresulting in null fields that appear to be empty data BatchSizeset too high causing the Processing Engine worker to run out of memory- xDB SQL indexes not covering the query predicate, resulting in full table scans on large datasets
CDP Data Lake Export
For Sitecore CDP, the extraction path is fundamentally different. Sitecore CDP securely stores your organization's data in an Amazon S3 bucket. After you get access, the data lake export service will run daily — all your organization's data will be exported into an export folder.
The exported files use the Apache Parquet file format. Parquet is an open source format that stores nested data structures in a flat columnar format, and it's supported by most modern databases and analytics tools (Spark, Athena, DuckDB, Pandas).
CDP Parquet export schema — partition structure:
| Partition | Key Fields | Notes |
|---|---|---|
guests |
guestRef, email, createdAt |
One row per guest profile |
orders |
orderRef, guestRef, orderedAt |
Transactional data |
order_items |
orderRef, sku, quantity |
Line-item detail |
experience_definitions |
experienceId, type |
Experiment and experience config |
events |
guestRef, sessionRef, eventType, timestamp |
Behavioral events (page views, clicks) |
sessions |
sessionRef, guestRef, startedAt, channel |
Session-level grouping |
Your data is partitioned into these six entity types. Reading Parquet files requires a Parquet-capable tool — AWS Athena, Apache Spark, DuckDB, or the Python pyarrow/pandas library are common choices.
For audience-specific exports, you can choose whether to export the data as CSV or JSON output via the File Format tab in the CDP Audience Builder interface.
Decision Framework: Choosing the Right Export Method
Use this decision tree to select your extraction approach before committing to tooling:
START: What is your Sitecore product?
│
├── XM Cloud
│ ├── Target = another XM Cloud env? → Content Transfer API + Item Transfer API
│ ├── Target = non-Sitecore platform?
│ │ ├── Published content only? → Experience Edge GraphQL API
│ │ └── All content (drafts/versions)? → SCS + scripted post-processing
│ └── Media assets? → Experience Edge media URLs (published) or Management API
│
├── Sitecore XP / XM (on-prem or PaaS)
│ ├── Version 10.1+?
│ │ ├── Target = Sitecore env? → SCS pull + Content Transfer API
│ │ └── Target = external platform? → SPE (custom) + SCS (templates)
│ ├── Version 9.x?
│ │ ├── Target = XM Cloud? → XM to XM Cloud Migration Tool v2
│ │ └── Target = external platform? → Unicorn/TDS export + SPE custom scripts
│ └── Version 8.x?
│ └── Any target → SPE + TDS/Unicorn + manual SQL fallback
│
├── Content Hub (DAM)
│ ├── Under 2,000 items? → UI export
│ └── Over 2,000 items? → REST API with WebSDK (15 req/sec limit)
│
├── Content Hub ONE
│ └── Any extraction → REST API (10 req/sec limit, API key auth)
│
└── Analytics / behavioral data
├── Sitecore XP xDB? → xConnect Data Export Tool
└── Sitecore CDP? → S3 Data Lake (Parquet format)
Moving between Sitecore XM Cloud environments? Use the Content Transfer API with the Item Transfer API. It handles chunked streaming for large datasets and preserves item integrity.
Migrating from Sitecore XP to XM Cloud? The XM to XM Cloud Migration Tool helps move content, media, and user data from a source Sitecore XM instance to a target XM Cloud environment. It is middleware that simplifies the task of entering and selecting data, and orchestrating the migration. It supports XM 9.1+.
Migrating from Sitecore to a non-Sitecore platform? Use SPE to export content as structured CSV or JSON, with field-level control over what's included. Combine with SCS YAML exports for template and rendering definitions that inform your field mapping. For media, script a bulk download from the media library.
Extracting Content Hub assets? Use the REST API with export profiles to control which fields and relations are included. Budget for the 15 calls/second rate limit — a library of 50,000 assets at that rate, accounting for pagination overhead, can take several hours.
Need analytics or CDP data? These are entirely separate systems with their own export paths. xConnect exports go through the Processing Engine with certificate-based auth; CDP data goes through the S3 data lake in Parquet format. Neither can be extracted through content-side APIs.
What Gets Left Behind: Common Export Gaps
No single Sitecore export method captures everything. These data types consistently fall through the cracks:
| Data Type | Typical Gap | Workaround |
|---|---|---|
| Media binaries | SCS serializes metadata; binaries need separate handling for large libraries | SPE scripted download or blob storage copy |
| Personalization rules | Rules serialize, but A/B test results and analytics context don't | Export rules via SCS; analytics via xConnect separately |
| Rendering configurations | Layout details serialize; custom controller logic and view files don't | Package renderings + code via deployment pipeline |
| User accounts/roles | SCS doesn't cover ASP.NET membership tables | SPE scripted export or XM Migration Tool |
| Workflow history | Current state serializes; historical transitions don't | SQL query against WorkflowHistory if needed |
| URL redirects | Not included in standard serialization | Export redirect map separately |
| Scheduled publishing | Publishing restrictions serialize; scheduled publish jobs don't | Document and recreate manually |
| Unicorn/TDS config | Predicates and sync config don't transfer to SCS | Manually recreate module.json includes from Unicorn predicates |
| Delta/incremental sync state | No method captures "what changed since last export" natively | Track by __Updated field timestamp; query items modified after a cutoff date |
Direct SQL Extraction: When and Why to Avoid It
For on-premises Sitecore XP instances where the application layer is non-functional, direct SQL queries against the Master or Web database are technically possible. Unlike Xperience by Kentico, where direct SQL is often the recommended extraction path, Sitecore's own documentation advises against this — it bypasses security, caching, and pipeline processing — but it can salvage data from a database backup when no other option exists.
A basic extraction query:
SELECT Items.ID, Items.Name,
VersionedFields.Value AS VersionedValue,
SharedFields.Value AS SharedValue
FROM Items
INNER JOIN VersionedFields ON Items.ID = VersionedFields.ItemId
INNER JOIN SharedFields ON Items.ID = SharedFields.ItemId
WHERE Items.TemplateID = '{YOUR-TEMPLATE-GUID}'The problems with this approach:
- Field IDs are GUIDs, not human-readable names — you need a mapping table built from the
Itemstable (where template field definitions are themselves items) to make sense of the output - Rich text fields contain internal Sitecore dynamic links (e.g.,
~/link.aspx?_id=...) that must be resolved post-extraction - Media references are item IDs, not file paths or URLs
- Multi-language content requires joining with language-specific version rows and filtering by max version number
- No link resolution — the database has no concept of published URLs
Direct SQL should be a last resort — used only when the Sitecore application layer is non-functional and you need to salvage data from a database backup.
Delta Sync Strategies
Most migrations involve a bulk initial extract followed by delta syncs to keep the source and target aligned until go-live. Sitecore does not provide a native change data capture (CDC) mechanism, but these approaches work:
For content items (XP/XM): Filter by the __Updated shared field, which Sitecore automatically stamps with the UTC datetime of the last save. An SPE or SCS pull filtered by __Updated >= [last_sync_timestamp] captures items modified since the last run.
$cutoff = [datetime]"2024-01-01T00:00:00Z"
$changedItems = Get-ChildItem -Path "master:/sitecore/content" -Recurse |
Where-Object { [Sitecore.DateUtil]::IsoDateToDateTime($_["__Updated"]) -gt $cutoff }For xConnect contacts and interactions: The xConnect Data Export Tool supports cut-off date filtering via the interactionStartDateRange and interactionEndDateRange parameters on the interactions export endpoint.
For CDP: The S3 data lake export is partitioned by date. Daily incremental exports are the native mechanism — each day's folder contains only that day's data.
For Content Hub assets: Filter by ModifiedOn date using the REST API search endpoint. Assets modified after a given timestamp can be queried and re-exported without re-extracting the full library.
Migration-Ready Extract Checklist
Before treating an export as complete, verify each of the following:
- Dynamic links resolved — all
~/link.aspx?_id=...references converted to absolute or relative URLs - Media binaries extracted — not just metadata; actual files on disk with correct extensions
- MIME-to-extension mapping verified — each media item's file extension matches its stored MIME type
- All language versions captured — not just the default language
- Latest version selected — not all versions unless version history is required
- Multilist/Treelist fields resolved — pipe-separated GUIDs converted to referenced item values
- Field ID-to-name mapping complete — GUID field IDs resolved to human-readable names for target system mapping
- Workflow state recorded — current state captured even if history is not
- User accounts exported separately — ASP.NET membership not included in item serialization
- URL redirects exported — redirect rules captured outside the content tree
- Record count verified — source item count matches extracted item count per template
Planning Your Sitecore Export
Inventory before you extract. Know your item count by template, your media library size in GB, and the number of language versions. These three numbers determine which export method is practical and how long extraction will take. Run this SPE query first:
Get-ChildItem -Path "master:/sitecore/content" -Recurse |
Group-Object TemplateName |
Select-Object Name, Count |
Sort-Object Count -DescendingTest with a subtree first. Run your export against a single site or content branch before attempting the full tree. This catches field mapping issues, serialization errors, and rate limit surprises early.
Preserve internal link context. Any export to a non-Sitecore target must resolve dynamic links to paths or URLs during transformation — not after import.
Account for the two-clock problem. Exporting data from Sitecore is one workstream. Rebuilding templates, renderings, components, and personalizations in the target platform is a completely separate effort. (This structural complexity is exactly why migrating into Sitecore from simpler platforms requires custom ETL scripts). The data export might take days; the replatform project takes months. Don't conflate the two timelines.
Plan extraction windows around traffic. Media extraction and large SPE exports should run during off-peak hours. xConnect exports using the Processing Engine are resource-intensive and can degrade response times on shared infrastructure.
Frequently Asked Questions
- How do I export content from Sitecore XM Cloud?
- Use Sitecore Content Serialization (SCS) to pull items as YAML files via the CLI (`dotnet sitecore ser pull`). For environment-to-environment moves, use the Content Transfer API and Item Transfer API, which replace the Package Designer effective July 2026. For published content extraction, query Experience Edge's GraphQL API (rate-limited to 80 requests/second, max 1,000 entities per query).
- What are the Sitecore Experience Edge API rate limits?
- Experience Edge enforces a hard limit of 80 GraphQL requests per second per environment. Each query can return a maximum of 1,000 entities, and the default query complexity limit is 250. Exceeding these returns a 429 status. Use cursor-based pagination and exponential backoff for bulk extraction.
- Can I export Sitecore data using direct SQL queries?
- Technically yes for on-premises Sitecore XP, but Sitecore explicitly advises against it. Content is stored in an EAV schema across Items, VersionedFields, UnversionedFields, and SharedFields tables. Field IDs are GUIDs, internal links need resolution, and you bypass security and caching layers. Use SPE or SCS instead unless the application layer is non-functional.
- Why do my exported Sitecore text fields contain GUIDs instead of links?
- Sitecore stores internal links as dynamic XML tags containing GUIDs (e.g., ) to prevent broken links if items move. You must use the Sitecore LinkManager API during export to resolve these GUIDs into absolute URLs. Experience Edge's GraphQL API handles this automatically.
- How do I export analytics data from Sitecore?
- For Sitecore XP, use the xConnect Data Export Tool — it extracts contacts and interactions from the collection database via the Processing Engine. For Sitecore CDP, use the data lake export service, which runs daily and exports all tenant data (guests, orders, sessions, events) as Apache Parquet files to Amazon S3.