AI data engineering · Data migration · 2026
Procurement Data
Modernization Workbench
A traceable migration and evidence-retrieval system built from public World Bank procurement records.
From fragmented records to retrievable evidence.
300 notices + 300 awards
Verified local prototype using public World Bank procurement records. Independent portfolio project, not a World Bank product.
The engineering question
How do fragmented records become trustworthy enough for analytics and AI?
Procurement notices and contract awards arrive through different structures, naming conventions, and levels of completeness. Moving them into one database is not enough. The migration must preserve lineage, reconcile meaning, and make every transformation reviewable.
I built the workbench to demonstrate the full path from bounded API ingestion to validated relational data, project features, and citation-grounded retrieval. The interface is a proof surface for the engineering beneath it, not a decorative dashboard.
Verified scope
A complete bounded run, not a scripted mockup.
The same artifacts drive the pipeline ledger, quality evidence, retrieval corpus, and recruiter interface.
Migration architecture
Five layers, each independently inspectable.
The design preserves the source before applying interpretation, then separates validation, modeling, feature engineering, and retrieval.
- 01Raw evidence
Immutable API responses, retrieval timestamps, source URLs, and checksums.
- 02Standardized
Normalized identifiers and dates with original values retained for audit.
- 03Curated SQL
Nine relational tables with explicit grain, keys, joins, and run history.
- 04Project features
272 project-level rows covering activity, suppliers, linkage, and quality.
- 05Evidence index
759 source-linked chunks with metadata and deterministic vectors.
Record lineage
One award, traced from source value to curated field.
{
"projectid": " p166309 ",
"contr_sgn_date": "29-Jul-2026",
"total_contr_amnt": "1285000"
}DQ-002 + DQ-004
{
"project_id": "P166309",
"signed_date": "2026-07-29",
"amount": 1285000.00
}Data integrity
Uncertainty is recorded, not silently corrected.
Every control has a rule, severity, affected-record count, and recommended human handling. The source value remains available.
Preserve and quarantine
Retain original value
Retain and retrieve later
Compare official sources
Responsible retrieval
Retrieve evidence, cite the source, or explicitly stop.
The browser demo searches the exported SQLite evidence index. Results are ranked from stored vectors and never composed from a hard-coded answer list.
P166309 · Pakistan · official source retained
No indexed evidence means no generated answer.
Production translation
Local proof, enterprise path.
Reliable AI begins before the model. It begins with source lineage, explicit transformation rules, defensible joins, and the discipline to return no answer when evidence is insufficient.