Senior Data Engineer

Data Engineering:Lead the design, development, and automation of scalable, high-performance data pipelines across institutional systems, AWS, Databricks, and external vendor APIs.Implement Databricks Lakehouse architectures to unify structured and unstructured data, enabling AI-ready data platforms that support advanced analytics and machine learning use cases.Build robust and reusable ETL/ELT workflows using Databricks, Spark, Delta Lake, and Python to support batch and streaming integrations.Ensureperformance, reliability, and data quality of data pipelines through proactive monitoring, optimization, and automated alerting.Partner with business and technical stakeholders to define and manage data pipeline parametersincluding load frequency, transformation logic, and delivery mechanismsensuring alignment with analytical and AI goals.Ensure all data engineering solutions adhere touniversitysecurity, compliance, and governance guidelines, whileleveragingbest practices in cloud-native data development.Develop andmaintaincomprehensive technical documentation of data pipeline designs, data flows, and operational procedures.Collaborate with enterprise data architects, data modelers, data stewards, and subject matter experts to ensure data consistency, lineage, and semantic alignment across the ecosystem.Continuously evaluate and introduce emerging technologiessuch as Databricks Unity Catalog,MLflow, Delta Live Tables, and AI-driven data observability toolsto enhance the data engineering landscape.Drive innovation by modernizing existing pipelines toward AI-readiness, enabling future integration with predictive analytics and machine learning models.Stay current with advances in Databricks, AI-driven data engineering, and cloud technologies, and advocate for their responsible adoption.Contribute to the vision of building a modern, AI-ready data ecosystem that powers advanced analytics, automation, and decision intelligence across the University.Collaboration, Support, and Communication:Work both independently and collaboratively within cross-functional teams to deliver data products and pipelines that meet the Universitys evolving data and analytics needs.Communicate clearly and effectively with technical and non-technical stakeholdersregardingproject progress, risks, dependencies, and technical challenges.Promote collaboration and knowledge sharing within the Data Engineering team through brainstorming sessions, design reviews, andDatabricksbest-practice discussions.Foster a culture of learning and innovation,supportingteam morale and professional growth.Provide mentorship and peer guidance to junior data engineers on data pipeline design, Databricks workflows, and coding best practices.Participate in change management processes to ensure transparency and coordination across teams during system enhancements or platform migrations.Perform other related functions as assigned.