Error Library / AEP

Batch ingestion: columns missing from the dataset after upload

What you seeBatch succeeded, but some source columns are missing from the dataset

Only fields that exist in the schema can land in the dataset. Depending on the ingestion path, a column that isn't mapped to a schema field is either dropped or rejected — either way the data isn't where you expect.

Why it happens

  • The schema was designed before the export changed, or the export has extra columns nobody modelled.
  • Column names in the file don't exactly match the mapping (case, spaces, underscores).
  • Custom fields belong under your organisation's tenant namespace in XDM, and a mapping that points at the wrong path won't match.

How to fix it

  1. Compare the file header with the schema and list every column without a destination.
  2. Add the missing fields to the schema (in a field group) if you need them, then update the mapping.
  3. Normalise header names before upload, or map by exact name.
  4. After ingesting, preview a few records and confirm each expected field is populated.

Before and after

Fails
search_term,consent_status → no matching fields in the schema
Works
search_term → a custom field such as _tenant.search.term; consent_status → consent field group

Common questions

Why did the batch say "success" if data is missing?

Success means the rows that were written passed validation. It doesn't mean every column had somewhere to go.

Practise the fix

Build the Customer Profile

Reproduce this error in your own AEP practice workspace, fix it, and earn a verifiable credential for the skills involved: XDM schema design, Identity descriptors, Real-Time Customer Profile enablement, Dataset configuration.

See the Lab plan

Related errors

XDM ingestion error: "Missing required field" (_id, timestamp)

XDM error: field "expected type object but got string" (ECID, identityMap)

AdsBot is an independent training provider, not affiliated with Adobe. Behaviour can differ between ingestion methods and product versions; always test with a small batch first.