Change & Migration
How to Control Full and Delta Migration During a GxP Cutover
A production migration often starts before the actual cutover. The final full source extract may be taken on one day and the corresponding full migration may take hours or several days to transform, load, verify and reconcile. During that period the source system may still be live and regulated records may continue to change.
That creates the real delta problem: everything that changes after the final full extract must be accounted for before the target becomes authoritative.
A rehearsal is still useful. It can show that the migration tooling, mapping, sequencing, timing and reconciliation method are capable of working. But the rehearsal source state is not the production delta boundary.
Fix the final full-extract cut-off
Define the exact source state represented by the final full extract. Record the source systems, extraction date and time, sequence or audit boundary, extraction rules, reference-data state, included populations and approved exclusions.
The cut-off must be queryable later. A label such as "final dump" is not enough if the team cannot determine whether a record belongs before or after it.
Where timestamps are used, confirm their semantics. Ask whether every relevant create, update, deletion, attachment, child-record change or state transition updates the marker used by the delta query. If not, use a more reliable change mechanism or additional detection logic.
The control objective is simple: no relevant source change should be able to occur after the full extract and remain invisible to the delta method.
Allow for the full migration to run while the source remains active
The full migration and the delta window can overlap in calendar time without overlapping in source population.
For example:
T0is the final full-extract cut-off;- the full population as of
T0is transformed and loaded over the next two or three days; - the source remains active after
T0; - changes after
T0accumulate into the first delta population; - later delta extracts may reduce the final cutover window;
- a final freeze or last controlled delta closes the remaining source activity before the target is accepted.
This means the migration plan needs both a load sequence and a source-change sequence. Treating them as the same timeline can hide gaps.
Define the delta population explicitly
The delta is not only newly created records. Assess separately:
- records created after the full-extract cut-off;
- existing records updated after the cut-off;
- records deleted, cancelled, merged or otherwise removed;
- attachments, document versions or child records added later;
- reference-data changes that alter transformation or target meaning;
- late corrections to records already included in the full extract;
- state changes that alter whether a record should migrate;
- changes to relationships, ownership or other dependent data needed to preserve the record's meaning.
For each class, state how it is detected and whether it is inserted, updated, reloaded, deleted, excluded or otherwise dispositioned in the target.
Use bounded delta windows
Large programmes may need more than one delta migration. If so, define explicit high-water marks so the windows neither overlap ambiguously nor leave gaps.
For example, the first delta might capture changes from T0 up to T1, and the second from T1 up to T2. The exact technical mechanism may use timestamps, sequence values, audit events or another source-specific marker.
The important control is not the notation. It is proving that each relevant change belongs to exactly one controlled population or has an explicit reason to be processed again.
Pay particular attention to:
- records changed several times across the window;
- deletes where the source does not retain a simple current-state row;
- child or attachment changes that do not update the parent marker;
- clocks, time zones and inclusive or exclusive timestamp boundaries;
- in-flight transactions around the cut-off;
- corrections made after a record has already been extracted for a delta.
Control dependency order
A full or delta population can be numerically complete and still fail if dependent data arrives in the wrong order. Parent records may need to exist before children, reference values before transactions and users before ownership or audit relationships.
Define the load order and the handling for dependencies that cannot yet be satisfied. Veeva's migration guidance discusses migration environments and technical controls around custom functionality during migration. Those product features help establish the load conditions, but they do not replace the customer's source-population and sequencing rationale.
Where a later delta changes a record that already exists in the target, make the update semantics explicit. The team should know whether the load replaces, merges, versions or otherwise transforms the previous target state.
Reconcile the full migration, each delta and the final target state
A strong acceptance record answers four questions:
- Did the final full population as of the cut-off reconcile?
- Did every delta population reconcile against its defined change window?
- Were migration exceptions and repeated changes dispositioned without creating gaps or double counting?
- Does the accepted target state represent the complete intended source state at the cutover boundary after approved transformations, exclusions and business rules?
The fourth question matters because full + delta = target is not always simple arithmetic. Migrations can merge records, transform values, version documents, exclude approved populations or change relationships. The reconciliation model must reflect the actual mapping rules.
Use counts only where counts genuinely prove completeness. For consequential records, include identifiers, transformation results, relationship checks, exception dispositions and targeted content verification where needed.
Close the source before cutover
Define how the final live-source activity ends. Depending on the process, this may be:
- a formal source-system freeze;
- a final short delta after the freeze;
- a controlled micro-delta for specific late populations;
- temporary manual capture under an approved procedure;
- a stop condition if the final population cannot be proven complete.
The cutover decision should not occur merely because the last load job completed. It should occur when the source-to-target population through the final boundary has been reconciled and the remaining exceptions are acceptable.
Link rollback to data created after the switch
Rollback also needs a data rule. If users or interfaces create valid target records after the target opens, returning to the source may require those records or decisions to be reconciled back or otherwise controlled.
The last recoverable point, authority to invoke rollback and treatment of post-switch target activity belong in the cutover plan. The delta method must connect to those decisions rather than stopping at the final load.
Important boundaries
This article does not replace the full migration validation strategy, cutover plan or exception process. It addresses the specific evidence gap between the final full production extract and the later cutover state.
MHRA data-integrity guidance and EU GMP Annex 11 support robustly controlled and validated data transfer, preservation of record meaning and lifecycle control. They do not mandate a document called a full-and-delta migration plan. The cut-off, bounded delta-window and combined reconciliation method described here is practitioner guidance.
Sources
- MHRA, GxP Data Integrity Guidance: guidance on robustly designed and validated data transfer and migration controls and preservation of record meaning.
- Veeva, Vault Migrations: current product guidance for migration environments, loading and control of custom functionality during migration.
- European Commission, EU GMP Annex 11: Computerised Systems: validation, data-transfer, change and lifecycle expectations for GMP computerised systems.