Overview:
Randstad Digital are partnering with a firm in the search for a Technical Lead (Lead Data Engineer) to join their team on a 2 year fixed term basis. Our client is building a new modern data and analytics platform using the Azure Cloud stacks (Azure Data Factory, Azure Databrick and ADLS).
In this transformation, you will play a critical role as the Lead SME in the design, development and implementation of data solutions on Azure Cloud and Azure Databricks in a greenfield environment.
What we are seeking:
We are seeking an experienced candidate with a proven track record in delivering robust, scalable and high quality data solutions on Azure Cloud and Databricks in large, enterprise environments.
With a passion for driving best practices and efficient data systems, you will have technical leadership skills across mentoring a skilled team, driving continuous improvement and aligning a data roadmap into effective project delivery.
Responsbilities:
Design and Development:
- Lead the design and implementation of data pipelines, ETL processes and data models on Azure Databricks.
- Ensure the adoption of best practices in data engineering, including data quality, optimisation and security
- Translate high-level architecture into actionable implementation plans
Implementation & Best Practices:
- Ensuring solutions are scalable, reliable and secure along with mentoring team members
- Conduct code reviews, design reviews and ensure the team follows best practices in version control, CI/CD and DevOps for data engineering
Collaboration & Communication:
- Collaborate with cross-functional teams, including data architects, data modellers, analysts, and IT operations, to ensure seamless integration of data solutions.
- Communicate technical concepts to non-technical stakeholders, providing insights and recommendations to drive data-driven decision-making.
Governance Quality:
- Ensure data governance, security and quality standards are maintained throughout the data lifecycle
- Monitor and support data management initiatives, including within specific environments like Microsoft Azure/Databricks
Key Knowledge and Experience
- 6+ years of experience in data engineering, with a strong focus on Azure Cloud and Azure Databricks.
- Strong understanding of data architecture, data warehousing, data modelling, ETL processes and cloud based technologies.
- Demonstrated experience in mentoring technical teams
- Proficiency across Python, SQL and Scala
- Excellent problem-solving skills and the ability to work under pressure in a fast-paced environment
- Strong communication skills, with the ability to interact effectively with technical and non-technical stakeholders
Our client is looking to commence interviews ASAP with the successful candidate to be onboarded this year. For more information and a confidential chat, please reach out immediately to matt.andrews@randstaddigital.com.au with your updated cv.
At Randstad Digital, we are passionate about providing equal employment opportunities and embracing diversity to the benefit of all. We actively encourage applications from any background.
...
