Lead Data Engineer Job at Altimetrik, Cary, NC

  • Altimetrik
  • Cary, NC

Job Description

Programming & Data Engineering

Expert - level proficiency in Python, Scala, and PySpark, with a strong track record of designing and delivering production-ready, modular, and well-tested solutions; developing and troubleshooting Spark workloads; and optimizing large-scale batch and streaming data pipelines using Delta Lake and Spark technologies.

Strong SQL and data modelling — dimensional and normalised; schema design and data contract definition.

Databricks expertise — Delta Lake, Unity Catalog, Jobs & Workflows, cluster and pool management, performance tuning, Model Serving.

Azure data stack — ADLS Gen2 (zone design, ACLs, lifecycle), Azure Data Factory (parameterized / metadata-driven frameworks, error handling), Azure Event Hubs.

AI & Machine Learning

3+ years designing and shipping LLM-based systems in production: RAG pipelines, agentic / tool-calling workflows, structured output, chunking and embedding strategy, vector and hybrid retrieval, and prompt engineering.

Evaluation discipline — golden datasets, regression suites, accuracy and hallucination tracking, human-in-the-loop feedback loops; you measure AI quality, not assert it.

Hands-on experience with LangChain, LlamaIndex, or LangGraph, plus at least one provider stack (Azure OpenAI, OpenAI, or Databricks Model Serving).

Metadata-driven thinking — schema inference, data profiling, lineage, catalogs, and configuration-driven frameworks that onboard the next source without new code.

Job Tags

Full time, Contract work

Similar Jobs

Public Storage

Copywriter Job at Public Storage

 ...with AP Style Experience with content marketing, storytelling, or brand journalism Experience with writing press releases Bilingual or multicultural content experience a plus Additional Information Workplace One of our values pillars is to work as... 

GardaWorld

Mobile Patrol Security Officer Job at GardaWorld

 ...Mobile Patrol Security Officer $17.50 / hour | South Philadelphia, PA | Active Driver's License Required Ready to represent GardaWorld?...  ...a Mobile Patrol Security Officer, you will conduct mobile vehicle patrols across the site while serving as a visible, professional... 

Confidential

A level English Teacher Job at Confidential

Job vacancy in September, 2026Job Requirements 1. Native English speaker; 2. Masters Degree or higher; 3. Qualified for Working Visa and currently reside in China; 4. ESL, TEFL, TESOL, or CELTA certificates; 5. 3-5years of teaching experience in elementary ...

Wegmans Food Markets

Cashier Job at Wegmans Food Markets

 ...Location: Lake Grove, NY Address: 3270 Middle Country Rd Pay: $17.50 - $18 / hour Job Posting: 10/03/2026 Job Posting End...  ...on a recognized holiday # Health care benefits that provide a high level of coverage at a low cost to you # Retirement plan with a... 

Ogden City Corporation

FT - Arts Administrator - Cultural Services Division Job at Ogden City Corporation

 ...Job Description Job Description ARTS ADMINISTRATOR Department: Community and Economic Development Division: Cultural Services Pay Grade: GRADE 145: $64,176.38 - $93,055.74 Annual Salary ($30.85 - $44.74) Hourly Rate FLSA Status: Non-Exempt, Merit...