How to Export Data from ServiceNow: API Limits, Methods, and Pitfalls

How to Export Data from ServiceNow: API Limits, Methods, and Pitfalls

ServiceNow stores data well but doesn't give it up easily. The default UI export caps at 10,000 records, the Table API has the same default sysparm_limit, and attachments have no bulk-export feature at all. If you're planning a migration, feeding a data warehouse, or just trying to get incident history into a spreadsheet, you need to know which extraction method fits your volume, your timeline, and your tolerance for platform impact.

This guide covers every practical extraction path — from right-click CSV exports to paginated API scripts — with the real limits, the real edge cases, and the decisions that matter.

What are ServiceNow's default export limits?

ServiceNow enforces format-specific default export limits to prevent performance degradation on the instance. The platform checks for a format-specific property first, falls back to the general glide.ui.export.limit property, and then applies the default if neither is set.

Here are the defaults from ServiceNow's official documentation (Zurich release; behavior is consistent through Xanadu but property names should be verified against your specific release notes):

The default export limit is 10,000 records per export for most formats. PDF is the odd one out: it defaults to 1,000 rows with a hard ceiling of 5,000, and a maximum of 250 detail pages and 25 columns per page.

You can raise these limits via System Properties > Import Export, but ServiceNow's own documentation warns that exceeding default values can impact instance performance.

Version note: Rate limit properties, API behavior, and some system property names differ across ServiceNow releases (Utah, Vancouver, Xanadu). This guide reflects documented behavior through the Xanadu release. Always validate property names against your instance's release documentation before applying them in production.

Key terms defined

Before proceeding, two terms used throughout this guide:

• Keyset pagination: A technique for iterating through large datasets by using the last retrieved record's unique field value (e.g., sys_id) as the starting point for the next query. Unlike offset pagination, which must scan and discard all preceding rows, keyset pagination starts exactly where the previous page ended. This makes each page retrieval constant-time regardless of depth into the dataset.

• Offset pagination: The default approach where sysparm_offset=N tells the database to skip the first N records and return the next page. Simple to implement, but degrades linearly as offset grows.

UI list export vs. Export Sets: which one to use

UI list export is the right-click-and-download option most admins know. It's fine for ad hoc pulls under 10,000 records. A List Export is designed for quick, ad hoc exports directly from a ServiceNow list — useful for smaller datasets, zero configuration required.

Export Sets are a different tool entirely. A ServiceNow Export Set is a configurable container used to gather specific table data for export. Administrators define the source table, filters, target, and output format, then run the export on demand or on a schedule.

When to use which:

• UI list export: One-time pull, under 10K records, human reviewing data. Fast, zero setup.
• Export Sets: Recurring exports, larger datasets, delivery to FTP/SFTP or external targets, automated via scheduled jobs. Export Sets provide greater control over what data is exported, where it is sent, and when — making them better suited to larger or recurring batch exports.

Export Sets are also the standard mechanism for pushing files to a MID Server for transfer to external systems — a pattern commonly used for Workday, Snowflake, or data lake integrations.

How to export ServiceNow data via the REST Table API

The Table API (/api/now/table/{tableName}) is the primary programmatic extraction path. ServiceNow does not "export a report" — it returns records from a table based on a query, a sort, and a limit.

A basic GET request looks like this:

curl -u 'svc_integration:password' \
  'https://your-instance.service-now.com/api/now/table/incident?sysparm_query=active=true&sysparm_fields=number,short_description,state,sys_id&sysparm_limit=1000&sysparm_offset=0' \
  -H 'Accept: application/json'

Key parameters:

• sysparm_limit — Records per response. Default is 10,000; unusually large values can affect system performance.
• sysparm_offset — Skip N records (simple offset pagination; see limitations below).
• sysparm_query — Encoded query string using the same syntax as the ServiceNow UI URL bar.
• sysparm_fields — Comma-separated field list. Always specify this — it reduces payload size and database load significantly.
• sysparm_display_value — Set to true for display values, false for raw internal values, or all to return both simultaneously (recommended for migrations).
• sysparm_exclude_reference_link — Set to true to suppress the auto-generated reference link objects in API responses. Omitting this parameter causes the API to embed full URL objects for every reference field, which can double or triple response payload size on tables with many reference fields. This parameter is widely underdocumented but has a significant impact on throughput.

Why sysparm_offset pagination breaks at scale

Offset-based pagination works fine for the first few pages. It degrades badly on large tables. sysparm_offset forces the database to re-query and discard all preceding records, leading to progressively slower response times as the offset increases.

If you're iterating over a 500K-record table with offset pagination and a page size of 1,000, page 400 requires the database to scan and discard 400,000 rows before returning your 1,000. Response times grow linearly — or worse. In practice, on a production incident table with 500K records, expect page 1 response times of 200–500ms and page 400+ response times exceeding 5–10 seconds per page, depending on instance load and indexing.

Use keyset pagination instead

Keyset pagination — defined above — is the documented best practice for iterating through large ServiceNow tables via the REST Table API. Sort results by an indexed, unique field (sys_id or sys_updated_on) and use the last record's value from one page as the filter for the next.

In practice:

# First page — sort by sys_id
/api/now/table/incident?sysparm_query=ORDERBYsys_id&sysparm_limit=1000&sysparm_exclude_reference_link=true

# Subsequent pages — use the last sys_id from the previous response
/api/now/table/incident?sysparm_query=sys_id>LAST_SYS_ID^ORDERBYsys_id&sysparm_limit=1000&sysparm_exclude_reference_link=true

This lets the database start exactly where the last page ended — no scanning, no discarding. Each page is constant-time regardless of how deep you are into the table.

Throughput benchmarks (approximate, single-threaded):

Figures are estimates based on typical production instances; actual throughput varies by instance tier, table indexing, field count, and network latency. Measure against your own instance before designing an extraction job.

For page size, a value between 500 and 2,000 is typically a good balance between payload size and request overhead. A page size of 1,000 is a reliable default across most customer instances. At these rates, a 500K-record table with keyset pagination and page size 1,000 takes approximately 15–20 minutes single-threaded; parallelizing by date range or sys_id range partitions can reduce this to under 5 minutes.

How ServiceNow API rate limiting actually works

ServiceNow API rate limits are instance-configured, not globally fixed. Your instance administrator controls them — which means two customers on the same ServiceNow plan can have entirely different rate limits. There is no universal "ServiceNow API rate limit" you can look up in a vendor spec sheet.

What the documentation specifies:

• ServiceNow applies rate limiting typically in the range of 1,800–7,200 requests per hour per user, depending on instance configuration.
• ServiceNow uses semaphores to manage concurrent transactions. When all active semaphores on a node are busy, incoming requests are queued rather than immediately rejected.
• A 429 error is only returned once the queue itself is full — meaning the system can absorb bursts without immediately rejecting them, but sustained high throughput will eventually trigger queuing and then 429s.
• The Retry-After response header specifies the number of seconds to wait before retrying. Honor this header exactly — it reflects the actual queue state.

Rate limit rules can be scoped per user, per role, or globally, with user-level rules taking precedence over role-level, and role-level over all-users.

HTTP error taxonomy for ServiceNow API extractions

Understanding what each error means in the ServiceNow context prevents misdiagnosis:

Handling 429 errors in extraction scripts

Every extraction script must handle 429s. A naive script that ignores them will silently miss records or fail partway through.

import requests
import time
 
def fetch_page(url, auth, max_retries=5):
    for attempt in range(max_retries):
        resp = requests.get(url, auth=auth, headers={"Accept": "application/json"})
        if resp.status_code == 200:
            return resp.json()
        elif resp.status_code == 429:
            wait = int(resp.headers.get("Retry-After", 30))
            time.sleep(wait)
        else:
            resp.raise_for_status()
    raise Exception(f"Failed after {max_retries} retries")

The critical detail: honor the Retry-After header. Don't use a fixed sleep. ServiceNow tells you exactly how long to wait — use it.

How to export ServiceNow attachments

ServiceNow stores attachments across two tables:

• sys_attachment — metadata: file name, content type, size, parent record sys_id, and table name
• sys_attachment_doc — the actual binary content, chunked into segments

There is no out-of-box solution for bulk attachment export. The Attachment API endpoint for downloading file content is:

GET /api/now/attachment/{sys_id}/file

To bulk-export attachments programmatically:

1. Query sys_attachment for the records you need (filter by table_name, table_sys_id, date ranges, file size, content type, etc.).
2. Iterate through results and download each file using the Attachment API, using the sys_id from sys_attachment.
3. Store with metadata — preserve the parent record's sys_id, table name, and file name to reconstruct the relationship in the target system.

Attachment exports are slow because each file is a separate HTTP request — there is no /api/now/attachment/bulk endpoint for binary file content in standard ServiceNow. For instances with tens of thousands of attachments, expect single-threaded extraction to take several hours. Parallelizing across 3–5 concurrent threads is feasible within typical rate limits; more than that risks triggering 429s on instances with tighter semaphore configurations.

Attachment export sizing: If sys_attachment shows 50,000 records with an average file size of 200KB, plan for approximately 10GB of data transfer. At 1 request/second (a conservative rate-limit-safe pace), that's ~14 hours single-threaded. With 5 parallel workers, ~2.8 hours.

The sys_archive problem: records that silently disappear

ServiceNow's data archiving feature moves records older than a configured threshold from active tables (e.g., incident) to archive tables (e.g., incident_archive or sys_archive). Standard Table API queries against incident will not return archived records — they simply aren't there.

This is one of the most common silent failures in ServiceNow extractions: your query returns 85,000 records against a table you were told has 120,000, with no error or warning. The remaining 35,000 are in the archive.

To check whether archiving is active on a table: navigate to System Archiving > Archive Rules and filter by table name. If rules exist and are active, query the corresponding archive table separately and merge the results.

Note: archived records may have a different field structure or missing journal data, so validate the schema before merging.

Field-level encryption: when exports return masked values

ServiceNow supports field-level encryption for sensitive data (SSNs, passwords, PII). When a field is encrypted, the Table API returns the encrypted or masked value unless the requesting user has explicit access to the encryption key.

If your extraction shows unexpected asterisks (****) or base64-looking strings in fields that should contain plain text, the field is encrypted and your service account lacks key access. Remediation requires the instance administrator to grant the service account the appropriate encryption key role — typically admin or a custom role with glide_encryption_key access.

This also affects XML exports: encrypted fields in XML exports contain the ciphertext, not plaintext, unless the exporting user has key access.

CMDB and relational data: the hard part

Flat-table exports (incidents, changes, requests) are straightforward. CMDB exports are not.

The CMDB is a graph of Configuration Items (CIs) connected by relationships stored in cmdb_rel_ci. Exporting the cmdb_ci_server table gives you servers, but not their relationships to applications, services, or networks. A CMDB export that doesn't include relationship data is incomplete by definition.

To export CMDB data properly:

• Export the CI tables you need (cmdb_ci_server, cmdb_ci_app_server, cmdb_ci_database, etc.)
• Export cmdb_rel_ci with a query that includes CIs from your target tables
• Export cmdb_rel_type to understand what each relationship means ("Runs on," "Depends on," "Hosted on," etc.)
• Preserve sys_id values throughout — they're the foreign keys that tie everything together

The table inheritance problem: ServiceNow's CMDB uses table inheritance. A CI of class cmdb_ci_server has fields in three places:

1. cmdb_ci — base class fields (name, sys_class_name, sys_id)
2. cmdb_ci_hardware — hardware fields (serial number, manufacturer)
3. cmdb_ci_server — server-specific fields (OS, RAM, CPU count)

If you export only cmdb_ci_server, you get all three layers because ServiceNow's table API joins them automatically. But if you export via XML and import only cmdb_ci_server, you may miss fields stored in parent tables depending on how the target system handles class hierarchy. Validate your field coverage by comparing a sample record's field count via the API against what appears in the UI.

Scheduled extraction with GlideRecord scripts

For recurring server-side exports without using the REST API, Scheduled Script Executions with GlideRecord queries can generate and email CSV files or attach them to records. This approach stays within the instance, bypasses API rate limits entirely, and is suitable for moderate-volume recurring exports.

// Example: Export active incidents to a CSV attachment
var gr = new GlideRecord('incident');
gr.addQuery('active', true);
gr.query();
 
var csv = 'number,short_description,state\n';
while (gr.next()) {
    csv += gr.getValue('number') + ',' +
           gr.getValue('short_description').replace(/,/g, ' ') + ',' +
           gr.getValue('state') + '\n';
}
 
// Attach CSV to a record or send via email
var att = new GlideSysAttachment();
att.write(
    gr, // attach to a record
    'incident_export_' + new GlideDateTime().getDisplayValue() + '.csv',
    'text/csv',
    csv
);

Constraints:

• Default server-side script execution timeout is 300 seconds — a GlideRecord loop over 100K+ records will time out.
• Large GlideRecord loops consume significant instance heap memory; on shared instances this can trigger automatic termination.
• For volumes above ~50K records, prefer API-based extraction or Export Sets over server-side scripts.

MID Server extraction: when and why

A MID Server (Management, Instrumentation, and Discovery Server) is a Java application that runs in your network and proxies communication between a ServiceNow instance and systems that aren't directly internet-accessible.

For data extraction, MID Servers are relevant in two scenarios:

1. Export Sets targeting internal systems: When an Export Set needs to deliver a file to an internal FTP server, shared drive, or on-premises data warehouse, the MID Server acts as the relay. The instance generates the file, the MID Server receives it, and the MID Server delivers it to the internal target. This bypasses the need to open inbound ports from ServiceNow's cloud to your internal network.

2. Scripted integrations accessing internal databases: If your extraction workflow involves writing directly to an on-premises database (rather than pulling via API), the MID Server hosts the JDBC driver and executes the write. The REST Table API is still used to read from ServiceNow; the MID Server handles the outbound write.

MID Server extraction does not bypass API rate limits — the ServiceNow instance still enforces rate limits on outbound data queries. What it eliminates is the network architecture problem of getting data into systems that aren't publicly accessible.

ETL tools vs. API scripts vs. replication: decision framework

Three common architectures for getting data out of ServiceNow. Use this framework to select the right approach:

Is this a one-time extraction or recurring?
├── One-time
│   ├── Volume < 50K records AND flat tables → UI Export or quick API script
│   ├── Volume 50K–1M records → Python/PowerShell with keyset pagination + retry logic
│   └── Volume > 1M records OR includes CMDB/attachments → Dedicated extraction tool or ETL platform
└── Recurring
    ├── Frequency: daily/weekly → Export Sets with scheduled jobs, or ETL platform
    ├── Frequency: hourly → ETL platform (MuleSoft, Boomi, Informatica, Talend)
    └── Frequency: near-real-time (minutes) → Data replication (Perspectium, custom CDC)

Detailed comparison:

Pagination doesn't eliminate API demand. If retrieving a dataset requires 500 pages, you've made each individual request manageable — but you're still making 500 API calls. Design your rate limit budget accordingly.

Common failure modes (and how to avoid them)

1. Exporting display values instead of sys_ids. If your export contains "John Smith" instead of 6816f79cc0a8016401c5a33be04be441, you've lost the foreign key. Set sysparm_display_value=all in API calls to get both display and raw values simultaneously. For migrations, always preserve both.

2. Forgetting journal fields. Comments and work notes live in sys_journal_field, not on the incident record itself. A standard incident export won't include them. You need a separate extraction from sys_journal_field, filtered by element_id matching your incident sys_id values, and joined at load time. The relevant fields are element (either comments or work_notes), value (the text), and sys_created_on.

3. Ignoring timezone handling. ServiceNow stores all dates in UTC internally. sysparm_display_value=true returns localized times based on the requesting user's profile timezone — which may not match your target system's expected timezone. For migrations, always extract raw UTC values (sysparm_display_value=false for date fields) and handle timezone conversion in your ETL layer.

4. Not accounting for ACLs. The integration user's ACL determines what records and fields are visible. Missing records are often an ACL problem, not a data problem. In production, use a dedicated service account with role-scoped access to specific tables and fields — not a personal admin account. Verify coverage against a known record count before running a full extraction.

5. Encrypted fields returning masked values. See the field-level encryption section above. Check for **** patterns in exported data and verify service account encryption key access before running production extractions.

6. Archived records excluded from results. Query sys_archive rules for your target tables. If archiving is active, extract from both the active table and the archive table.

7. Running large exports during business hours. Large exports consume memory, processing capacity, and semaphores. Exporting tens or hundreds of thousands of records in a single transaction increases the risk of timeouts, incomplete exports, and performance degradation for other users. Schedule heavy extraction jobs for off-peak windows — typically 10pm–4am local time for the instance's primary user base.

8. Not setting sysparm_exclude_reference_link=true. On tables with many reference fields (CMDB tables are the worst offenders), omitting this parameter causes each reference field to return an embedded object with a full URL, inflating response payloads by 2–5x. On a table with 20 reference fields and 1,000 records per page, this can add 500KB–2MB per page unnecessarily.

What about ServiceNow's built-in backup?

ServiceNow's built-in backup is not a data export tool. It's an instance recovery mechanism. Many organizations require more advanced capabilities for data backups and archiving than ServiceNow provides out-of-the-box. The restoration process takes significant time, and while it runs, the instance is unavailable. You cannot selectively restore individual tables or records.

If your goal is data portability — moving data to another system, building a reporting layer, or maintaining a portable archive — you need one of the extraction methods described above. Built-in backups are appropriate for disaster recovery, not data migration or analytics.

Making the right call

The best extraction method depends on three variables: volume (how many records), frequency (one-time or ongoing), and complexity (flat tables or relational structures like CMDB).

Use the decision framework above to select your approach. The key technical decisions that most extractions get wrong:

1. Use keyset pagination, not offset pagination, for any table over 50K records
2. Set sysparm_exclude_reference_link=true on every API call
3. Extract sys_journal_field separately for any workflow involving incident or case history
4. Check for archiving rules before assuming your row count is complete
5. Verify encrypted field access before running production volume
6. Run extraction jobs in off-peak windows and budget your rate limit accordingly

For one-time pulls of a few thousand records, the UI export or a quick API script works fine. For recurring large-scale extractions, invest in proper keyset pagination, retry logic, and off-peak scheduling. For continuous data sync across platforms, you need infrastructure — whether that's a managed integration platform or a dedicated replication service.