Middle Data Engineer — Databricks / Python / LLM / Neo4j

Core Requirements: — Strong hands-on experience with Databricks, especially Databricks Jobs — Advanced Python skills, writing notebooks daily from ingestion to the gold layer — Solid understanding of medallion architecture and how to define bronze / silver / gold boundaries — Experience working with Delta tables — Experience processing large volumes of PDF documents (100K+ files) — Experience calling LLM endpoints from notebooks and tracking token usage and cost — Experience with Azure cloud — Ability to design and consolidate job orchestration pipelines Nice to Have: — Experience with Neo4j or other graph databases — Experience with healthcare or clinical-research data — Experience building data layers consumed by AI agents What You'll Work On: — Extracting and structuring data from large collections of clinical-research PDFs — Building bronze, silver, and gold layers in Databricks with Python notebooks and Delta tables — Integrating LLM endpoints into data pipelines and monitoring token spend — Building a Neo4j graph alongside Delta tables for agents to query — Preparing gold-layer data for an agentic layer used as an API tool — Consolidating scattered and redundant Databricks job pipelines