Architect Big Data

Responsibilities:

• Develop a long-term database management plan, including architecture to support multi-tenancy and infrastructure scaling.
• Monitor database health, and storage growth, and predict potential performance issues.
• Conduct query cost analysis and optimize performance.
• Collaborate with DB Managed Services vendors to identify and mitigate performance issues.
• Work with AWS for regular database operational and performance reviews.
• Develop and maintain BigBasket-specific database standards and ensure compliance.
• Analyze and optimize database and team costs, ensuring timely setup and renewal of reservations to reduce costs.
• Collaborate with development teams to review and optimize DB queries.
• Plan and manage regular database maintenance activities.
• Oversee DB Managed Services vendor performance and evaluate monitoring setups, alerts, and thresholds.
• Perform regular database upgrades and ensure database hardening to eliminate vulnerabilities.
• Address data security requirements, including encryption and sanitization during non-prod DB refresh activities.
• Handle user security and audit activities.
• Evaluate new tools and capabilities.
• Develop and maintain best practices for development teams.

Requirements:

• Bachelor's degree in computer science or equivalent practical experience.
• At least 9-12 years of experience in data architecture, data engineering, or related fields, with a focus on designing and implementing large-scale data solutions.
• Strong experience in leading data architecture initiatives, including building data warehouses, data lakes, and cloud-based data platforms.
• Proven track record in designing and optimizing data systems for high availability, scalability, and performance.
• Expertise in relational and NoSQL databases (e. g., PostgreSQL, MySQL, Cassandra, Timescale).
• Extensive experience with cloud data platforms (e. g., AWS)) and cloud-native data services (e. g., Redshift, BigQuery, Snowflake, Databricks).
• Strong expertise in data modeling, ETL/ELT processes, and data integration tools (e. g., Apache Kafka, Talend, Informatica, dbt).
• Proficiency in data pipeline automation, and orchestration tools (e. g., Airflow, Apache NiFi).
• Hands-on experience with big data processing frameworks (e. g., Hadoop, Spark).
• Proficient in SQL, Scala, and/or Java for data management and automation.
• Strong leadership and mentorship abilities, with a proven track record of leading technical teams.
• Excellent communication skills, with the ability to engage with both technical and non-technical stakeholders.
• Ability to influence and drive change within a cross-functional team, advocating for best practices in data management.
• Analytical and problem-solving mindset with the ability to think strategically and execute tactically.
• Self-motivated and adaptable to changing business needs and evolving technologies.