Databricks Data Scientist/Engineer (100% Remote)

Job Description

Insight Global is looking to hire a Databricks Data Scientist/Engineer to support the implementation and long-term operation of Databricks within a complex Government environment. This person will work closely with stakeholders during the initial rollout of Databricks and support data engineering projects as the platform is expanded throughout the environment. This includes helping design and implement scalable Databricks solutions, integrating Databricks with other data platforms, and ensuring solutions are built with scalability and data governance in mind. The ideal candidate will bring strong hands-on Databricks and AWS experience and be comfortable supporting both the initial implementation and continued use of the platform.

Responsibilities Include:

• Lead data engineering efforts within Databricks to include pipeline development within Databricks on AWS in an enterprise environment
• Provide unification for organization and enterprise data capabilities to break down organizational data and operational silos, including mechanisms to standardize and migrate existing data to a unified data plane/warehouse
• Support the integration of Databricks with existing analytics platforms and metadata solutions
• Serve as a technical advisor to stakeholders and help upskill internal teams through guidance/documentation
• Assist and develop customer use case applications of Databricks

Compensation: $55.00/hr to $61.00/hr.

Exact compensation may vary based on several factors, including skills, experience, and education.

Employees in this role will enjoy a comprehensive benefits package starting on day one of employment, including options for medical, dental, and vision insurance. Eligibility to enroll in the 401(k) retirement plan begins after 90 days of employment. Additionally, employees in this role will have access to paid sick leave and other paid time off benefits as required under the applicable law of the worksite location.

Skills and Requirements

• Ability to obtain a DHS EOD security clearance
• Bachelor’s Degree (in IT or a related technical field)
• 7+ years of hands-on experience architecting, designing, data engineering with Databricks and AWS in a government setting.
• Hands-on Databricks experience including building pipelines and integrating Databricks with other analytics platforms.
• Strong understanding and experience in areas such as Apache Spark, Lakebase, Lakehouse architectures, Delta Lake, Delta Sharing, Medallion Architecture, Data & AI Governance, and use case acceleration.
• Demonstrated hands-on experience with AWS Cloud Native data and ETL capabilities (S3, Lambdas, Glue, etc.). Strong equivalent Azure experience is also acceptable.
• Demonstrated experience leveraging data to provide search, enrichment, correlation, AI/ML Model Training, in alignment with government mission and business use cases
• Strong Python skills.
• Prior consulting or customer-facing experience in data architecture or modernization initiatives.
• Experience in Agile environments and managing cross-functional technical delivery.
• Federal background/experience (experience supporting DHS is a plus)
• Experience supporting Databricks implementations at DHS agencies (e.g., CBP, USCIS, FEMA, ICE, USCG)
• Experience with the DHS System Engineering Lifecycle (SELC)
• Experience with Unity Catalog and MLFlow within Databricks
• Experience with agentic AI tools such as Databricks' Agent Bricks and Genie
• Familiarity with the concepts of a data mesh
• AWS Solutions Architect Associate or Solutions Architect Professional Certification
• Experience building reference architectures, playbooks, or platform documentation
• Experience with metadata platforms (e.g., DataHub, Collibra) and integrating Databricks with them
• Databricks certifications (Architect, Data Engineer)

We are a company committed to creating diverse and inclusive environments where people can bring their full, authentic selves to work every day. We are an equal employment opportunity/affirmative action employer that believes everyone matters. Qualified candidates will receive consideration for employment without regard to race, color, ethnicity, religion, sex (including pregnancy), sexual orientation, gender identity and expression, marital status, national origin, ancestry, genetic factors, age, disability, protected veteran status, military or uniformed service member status, or any other status or characteristic protected by applicable laws, regulations, and ordinances. If you need assistance and/or a reasonable accommodation due to a disability during the application or the recruiting process, please send a request to [email protected].