Senior Spark Developer

Job Description

A large financial company is looking for a Senior Apache Spark Developer to help build and scale a large enterprise data processing platform. This person will be very hands-on, developing high-volume data applications using Apache Spark and Kafka, while also helping move the environment toward Databricks and AWS. The ideal candidate has strong Spark development experience and understands how to build distributed systems that process large amounts of both real-time and batch data. This resource will be: o Building and optimizing applications using Apache Spark. o Developing both real-time streaming and batch processing solutions. o Integrating Spark with Kafka. o Helping migrate existing on-prem data platforms into Databricks and AWS. o Working with architects and platform engineers on larger cloud modernization efforts. o Improving Spark jobs for performance, scalability, and reliability. o Building reusable data engineering frameworks and components. o Troubleshooting issues across Spark, Kafka, Databricks, AWS, and enterprise messaging systems. o Supporting testing, code reviews, CI/CD, monitoring, and production environments.

Skills and Requirements

5+ years of hands-on Apache Spark development experience within a financial environment. Strong experience with: Spark SQL, Structured Streaming, DataFrames/Datasets, Spark performance tuning, Apache Kafka Strong coding skills in: Java & Python Experience building and supporting large-scale distributed systems.

We are a company committed to creating diverse and inclusive environments where people can bring their full, authentic selves to work every day. We are an equal employment opportunity/affirmative action employer that believes everyone matters. Qualified candidates will receive consideration for employment without regard to race, color, ethnicity, religion, sex (including pregnancy), sexual orientation, gender identity and expression, marital status, national origin, ancestry, genetic factors, age, disability, protected veteran status, military or uniformed service member status, or any other status or characteristic protected by applicable laws, regulations, and ordinances. If you need assistance and/or a reasonable accommodation due to a disability during the application or the recruiting process, please send a request to [email protected].