Data Engineer — AI & Agentic Solutions

Build, test, and maintain robust production data pipelines across Clinical and Non-Clinical data domains using lakehouse medallion architectures (Bronze/Silver/Gold).

Implement core pipeline patterns including Change Data Capture (CDC), incremental batch loads, schema evolution, and idempotent reprocessing.

Build, deploy, and optimize agentic AI components, Retrieval-Augmented Generation (RAG) chains, tool-calling agents, and multi-agent workflow nodes using modern LLM frameworks.

Implement dynamic model-routing configurations to optimize LLM tasks based on cost, latency, accuracy, and enterprise governance constraints.

Integrate data and agentic workloads directly with Lilly's internal enterprise data platform, catalog services, data lineage trackers, and metadata repositories.

Construct and refine semantic retrieval pipelines, chunking strategies, vector embeddings, hybrid search indexes, and lineage-aware knowledge graphs.

Instrument agentic systems and pipelines with robust logging, tracing, evaluation harnesses, and observability metrics suitable for regulated GxP environments.

Drive AI-native development workflows, including AI-assisted test generation, schema drift detection, automated pipeline authoring, and data quality scoring.

Enforce strict access control, encryption, data lineage tracking, and prompt/data-leakage prevention on regulated clinical datasets.

Maintain rigorous software engineering best practices, including Git version control, automated regression testing, CI/CD deployment, and GxP / 21 CFR Part 11 compliance.

Collaborate with data architects, senior engineers, and clinical domain experts to translate complex functional requirements into production-grade data and AI assets.