Middle Data Engineer

Core Requirements: — Strong experience with Databricks and Lakehouse medallion architecture (bronze / silver / gold) — Advanced PySpark skills for building transformations in notebooks — Hands-on experience with Delta Lake — Experience with Data Vault modelling (hubs, links, satellites) — Experience with dimensional modelling (facts and dimensions) — Experience ingesting data with Fivetran and custom API integrations — Experience working with Oracle SQL databases as source systems — Spark performance tuning, including skew handling and Liquid Clustering — Experience deploying with Databricks Asset Bundles (DABs) — Experience with Unity Catalog, group-based RBAC, and PII masking Nice to Have: — Experience with Great Expectations or similar data quality frameworks — Experience handling unstructured files in Blob storage — Experience supporting LLM / agent and ML use cases with curated data What You'll Work On: — Building and maintaining a Databricks Lakehouse on a medallion architecture — Ingesting data from Oracle systems and APIs into the bronze layer — Modelling the silver layer with Data Vault and the gold layer with dimensional models — Optimising Spark performance and Delta table layouts — Implementing governance, access control, and PII masking in Unity Catalog — Replacing manual scheduling processes and spreadsheets with a unified data platform — Collaborating with the application team that builds on top of the gold layer for the web app, LLMs, and ML models