Insight Global logo

Sr. Data Engineer - Databricks

Insight Global

Lower Paxton, PAJobSeen todaySeen in employer's feed today

Evaluate my fit

Most applications go out cold — see where you stand first. No sign-up to start.

Watch jobs like this.

Don't just apply. Show up ready.

Olive works from this exact posting.

At a glance

Compensation
No compensation found
Location
Lower Paxton, PA
Work Authorization
Not specified

Olive lists jobs from US employers, including remote roles you can work from the United States.

Job overview

Insight Global’s client is seeking a Senior Data Engineer to support a data platforms team within a data science and insights department. The role focuses on building production-grade Databricks pipelines and reusable curated data products that combine current-state SAP data with legacy and non-SAP history.

Skills & qualifications

RequiredNice to have

Skills

Data EngineeringDatabricksSparkPySparkSpark SQLDelta LakeMedallion DesignLakehouse DesignSQLWorkflow OrchestrationUnity CatalogRBACMonitoringRecoveryData QualityPerformance TuningAzureADLSSAP Datasphere IntegrationDelta SharingNetezza ModernizationDataStage ModernizationDbtKafkaEvent-Driven Patterns

Qualifications

Bachelor's Degree or Equivalent8+ Years Related Experience or Training5+ Years Data Engineering2+ Years Recent Databricks/Spark

Full job description

Job Description

One of Insight Global's clients is seeking a Sr. Data Engineer supporting data platforms with a data science & insights department. The primary mission is to build production-grade Databricks pipelines and reusable curated data products that combine SAP current-state data with legacy and non-SAP history.

Responsibilities:

Ingestion and transformation pipelines; Bronze/Silver/Gold tables; incremental/CDC processing; ; operational monitoring and runbooks; performance; knowledge transfer to the client.

Skills and Requirements

  • Bachelor’s degree (B. A.) from four-year college or university; or 8+ years related experience and/training; or equivalent combination of education and experience.
  • 5+ years data engineering with at least 2-3 years recent production Databricks/Spark work
  • Hands-on Databricks; PySpark and Spark SQL; Delta Lake; medallion/lakehouse design; SQL; orchestration with Workflows/Lakeflow or comparable; Unity Catalog/RBAC; monitoring, recovery, data quality and performance tuning
  • Azure/ADLS; SAP Datasphere-to-Databricks integration; Delta Sharing; Netezza/DataStage modernization; dbt; Kafka/event-driven patterns.

We are a company committed to creating diverse and inclusive environments where people can bring their full, authentic selves to work every day. We are an equal employment opportunity/affirmative action employer that believes everyone matters. Qualified candidates will receive consideration for employment without regard to race, color, ethnicity, religion, sex (including pregnancy), sexual orientation, gender identity and expression, marital status, national origin, ancestry, genetic factors, age, disability, protected veteran status, military or uniformed service member status, or any other status or characteristic protected by applicable laws, regulations, and ordinances. If you need assistance and/or a reasonable accommodation due to a disability during the application or the recruiting process, please send a request to [email protected].

Similar jobs, posted recently

Open roles like this one, listed in the last 30 days.

You've read the whole posting — now see how you match it.

Evaluate my fit