Research Scientist (Pyspark/Databricks)

Job Description

Responsibilities Enhance and maintain existing Python and PySpark-based data processing solutions Extract, transform, and prepare large datasets for machine learning and analytics initiatives Design and optimize data pipelines supporting research and experimentation efforts Work with structured and unstructured data, including text-heavy datasets and NLP use cases Collaborate with data scientists, researchers, and engineering teams to develop scalable solutions Troubleshoot data quality, pipeline, and performance issues across large-scale environments Conduct exploratory analysis and support model development activities Evaluate new approaches, tools, and technologies that improve research outcomes Provide technical mentorship, code reviews, and guidance to junior team members Translate business and research requirements into practical data solutions

Skills and Requirements

Required Qualifications Experience in Data Science, Machine Learning, Applied AI, or a related field Strong Python programming experience Hands-on experience with PySpark and distributed data processing Experience working within Databricks environments Advanced SQL skills Experience building, maintaining, or optimizing large-scale data pipelines Understanding of machine learning concepts, model development, and experimentation workflows Experience working with datasets ranging from millions of records Ability to navigate and enhance existing codebases Strong communication and mentoring abilities Natural Language Processing (NLP) experience Experience working with text analytics, document processing, language models, or related NLP applications Exposure to computer vision projects Spark performance tuning and optimization experience Experience supporting migrations between data platforms Experience partnering with research, analytics, and engineering teams in enterprise environments

We are a company committed to creating diverse and inclusive environments where people can bring their full, authentic selves to work every day. We are an equal employment opportunity/affirmative action employer that believes everyone matters. Qualified candidates will receive consideration for employment without regard to race, color, ethnicity, religion, sex (including pregnancy), sexual orientation, gender identity and expression, marital status, national origin, ancestry, genetic factors, age, disability, protected veteran status, military or uniformed service member status, or any other status or characteristic protected by applicable laws, regulations, and ordinances. If you need assistance and/or a reasonable accommodation due to a disability during the application or the recruiting process, please send a request to [email protected].