Senior Platform Architect - Databricks

CLA is seeking a Data Platform Architect to lead the architecture, design, implementation, and continued evolution of our enterprise data platform ecosystem, with Azure Databricks serving as a strategic platform for data, analytics, and artificial intelligence workloads.

In this role, you will define how enterprise data platforms enable governed data products, analytics, artificial intelligence, machine learning, application development, and data integration. You will establish scalable, secure, reliable, and supportable patterns across data architecture, platform services, networking, identity, governance, Infrastructure as Code (IaC), CI/CD, integration, observability, cost management, and operational support.

The ideal candidate brings proven experience designing and implementing modern cloud data platforms in enterprise environments, including deep expertise with Azure Databricks, and can demonstrate solutions taken from architectural design through production implementation, adoption, and ongoing support.

How You'll Make an Impact Data Platform Architecture and Strategy

• Define and maintain the enterprise data platform architecture strategy and partner with platform management on the product roadmap, with Azure Databricks as a core strategic capability.
• Establish reference architectures, design standards, architectural decision records, implementation patterns, reusable services, and platform guardrails.
• Define architectural boundaries and integration patterns across Databricks, cloud data services, operational systems, enterprise platforms, SaaS products, and external applications.
• Lead data platform architecture reviews and ensure solutions align with enterprise architecture, data strategy, security, networking, compliance, and operational standards.
• Assess architectural risks, identify platform capability gaps, and develop actionable roadmaps for product onboarding, modernization, migration, and technical debt remediation.

Cloud Platform, Network Architecture, and Connectivity

• Design secure cloud data platform network architectures using Azure virtual networks, subnets, network security groups, routing, firewalls, DNS, Private Link, private endpoints, and controlled egress patterns.
• Define connectivity patterns for Databricks workspaces, serverless and classic compute, storage, databases, messaging services, external clients, account-level resources, and integrated cloud services.
• Partner with Network Engineering and Cybersecurity teams to define private connectivity, traffic inspection, network isolation, and controlled internet access patterns.
• Evaluate emerging Azure and Databricks connectivity capabilities and translate them into approved enterprise architecture standards.

Data Governance, Identity, Security, and Access

• Establish an enterprise access model across data platforms, including Databricks accounts and workspaces, Unity Catalog, catalogs, schemas, tables, views, volumes, models, applications, compute, and related cloud resources.
• Partner with Identity and Access Management, Cybersecurity, Risk, and Compliance teams to establish access review, privileged access, separation-of-duties, and audit requirements.
• Define security and governance standards for structured and unstructured data, artificial intelligence, machine learning, agents, foundation models, data sharing, and external-facing data applications.

Platform Product and Lifecycle Management

• Define architecture, configuration, security, deployment, support, and governance standards for enterprise data platform products, with detailed standards for Databricks capabilities.
• Establish a product lifecycle framework covering General Availability, Beta, Public Preview, deprecation, migration, and retirement.
• Define evaluation and adoption criteria for new data platform capabilities, including business value, architectural fit, interoperability, security, compliance, data protection, operational readiness, cost, supportability, and production suitability.
• Determine which capabilities are appropriate for experimentation, development, limited production use, or enterprise-wide adoption.
• Define controlled proof-of-concept and pilot processes for emerging data platform capabilities.
• Translate validated capabilities into repeatable platform services, implementation patterns, documentation, enablement materials, and support procedures.

Infrastructure as Code, CI/CD, and Automation

• Design and implement Infrastructure as Code for Azure, Databricks, and supporting data platform resources using Terraform.
• Establish reusable Terraform modules for accounts, workspaces, networking, storage, governance, identities, permissions, compute policies, integrations, and shared platform services.
• Design CI/CD patterns for data products and platform services using Databricks Asset Bundles, Git repositories, GitHub Actions, Databricks CLI and SDKs, and automated deployment tooling.
• Implement automated validation, security scanning, policy enforcement, unit and integration testing, release controls, deployment verification, and rollback patterns.
• Build automation that reduces manual platform administration while improving consistency, reliability, auditability, and developer experience.

Data Integration and Enterprise Ecosystem

• Establish architecture standards for batch and streaming ingestion, APIs, event-driven integration, data federation, data sharing, marketplaces, external locations, storage credentials, service credentials, and interoperability with enterprise data tools.

Observability, FinOps, Reliability, and Operational Support

• Define enterprise observability and reliability architecture for platform health, data quality, lineage, security, performance, workload execution, service levels, usage, and cost.
• Standardize monitoring, alerting, capacity management, chargeback and showback, incident response, disaster recovery, and operational dashboards across Databricks and integrated data services.

Collaboration and Technical Leadership

• Partner with Cloud Architecture, Cybersecurity, Network Engineering, Identity and Access Management, DevOps, Cloud Operations, Data Engineering, and product teams.
• Serve as a primary architecture authority for enterprise data platform capabilities, with deep ownership of Databricks architecture and standards.
• Lead architecture reviews, technical design sessions, proof-of-concept evaluations, platform-readiness assessments, and modernization planning.
• Translate complex data platform decisions into clear business outcomes, risks, costs, tradeoffs, and recommendations.
• Develop architecture documentation for engineering teams, support organizations, leadership, auditors, data owners, and business stakeholders.
• Influence enterprise technology decisions and promote standardization across teams and workloads.
• Collaborate with product owners to prioritize platform capabilities, technical debt, operational improvements, and product adoption

What You Will Need Required Qualifications 4 years relevant experience required.

• Experience in platform engineering, cloud architecture, SRE, or infrastructure engineering preferred.

Preferred Qualifications

• Experience with Terraform, Bicep, Ansible, Azure DevOps, GitHub Actions, or similar technologies.
• Experience implementing Site Reliability Engineering (SRE) practices.
• Experience supporting enterprise-scale platform engineering initiatives.
• Azure certifications such as Azure Solutions Architect Expert or equivalent.
• Experience leading architecture reviews and technology modernization efforts.

Education Bachelor's degree is required. Combination of relevant experience, education, and training may be accepted in lieu of degree.

• Degree in Information Management, Computer Science, or a related field preferred.

Technical Competencies

• Six or more years of experience architecting, implementing, or supporting enterprise data platforms in cloud environments, including significant hands-on Azure Databricks experience.
• Proven experience delivering at least one enterprise data platform from architectural design through production implementation, adoption, and operational support.
• Proven experience designing data platform network, identity, security, governance, compute, storage, integration, data access, and deployment architectures.
• Expertise in modern cloud data platform architecture, including Azure Databricks account, workspace, compute, networking, governance, security, automation, integration, and operations.
• Advanced knowledge of Azure networking, including virtual networks, subnets, network security groups, route tables, private endpoints, Private Link, Azure Firewall, DNS, and secure egress.
• Advanced knowledge of data governance and Unity Catalog architecture, including authorization, credential models, object ownership, lineage, discovery, isolation, data quality, and data-sharing patterns.
• Advanced proficiency with Terraform, including modular Infrastructure as Code, state management, provider configuration, automated validation, testing, and deployment.
• Advanced proficiency designing GitHub Actions workflows for infrastructure and Databricks product deployments.
• Experience defining CI/CD patterns for data pipelines, notebooks, jobs, applications, models, configurations, infrastructure, and data products.
• Strong understanding of Git, GitOps, branching strategies, pull-request controls, release management, artifact management, and environment promotion.
• Knowledge of identity and access management, attribute-based access control, role-based access control, privileged access, secrets management, workload identity, and least-privilege design.
• Experience designing observability, logging, auditing, performance monitoring, cost management, alerting, and operational dashboards.
• Knowledge of ServiceNow incident, request, change, problem, configuration, and knowledge-management processes.
• Strong understanding of data modeling, data engineering, batch and streaming integration, lakehouse and warehouse architecture, business intelligence, artificial intelligence, machine learning, model governance, and data application architecture.
• Ability to evaluate General Availability, Beta, and Public Preview capabilities and make risk-based recommendations for enterprise adoption.
• Ability to identify architectural gaps, assess technical and operational risks, and develop scalable remediation strategies.
• Ability to balance security, governance, reliability, cost, performance, user experience, delivery speed, and operational supportability.
• Proficiency in communicating complex architectural decisions through diagrams, standards, roadmaps, presentations, and business-focused recommendations.

#LI-JH1

Equal Opportunity Employer/Protected Veterans/Individuals with Disabilities

Click here to learn about your hiring rights.

Wellness at CLA To support our CLA family members, we focus on their physical, financial, social, and emotional well-being and offer comprehensive benefit options that include health, dental, vision, 401k and much more.

To view a complete list of benefits, click here.