Randstad Digital are seeking a highly skilled and Senior Data Engineer to play a crucial role in designing, developing, and deploying data pipelines for our client’s lake house platform!
... Key Responsibilities & Deliverables:
- Design and maintain data pipelines for ETL processes.
- Create data models to meet business needs.
- Integrate data from various systems for a unified view.
- Ensure data accuracy and compliance with governance policies.
- Optimise infrastructure for data processing and performance.
- Build scalable, reliable data systems for large volumes.
- Implement security measures for data privacy.
- Collaborate with BI developers, data modelers, scientists, stakeholders, and tech partners.
- Evaluate new technologies to enhance data capabilities.
- Troubleshoot data issues for smooth operations.
- Support data-driven decisions by ensuring data availability and accessibility.
Necessary Skills & Qualifications:
- 5+ years as a Data Engineer with Azure cloud experience.
- Proficiency in Azure Databricks, PySpark, Spark SQL, and cluster management.
- Experience in Data Lake or Lake House solutions with Azure Databricks.
- Advanced skills in Azure Data Factory or similar tools.
- Knowledge of Azure DevOps or equivalent.
- Expertise in Python, Spark SQL, SQL, and ETL techniques.
- Experience with REST API data extraction.
- Ability to work autonomously and collaboratively.
- Results-oriented with a growth mindset.
- Strong analytical and interpersonal skills.
- Solid background in data analysis and frameworks.
- Experience as a dedicated data modeler would be highly regarded.
Interested? Apply Now!
Send applications by clicking “Apply Now” and following the prompts, or alternatively send your updated CV to Shaun.rowe@randstaddigital.com.au
At Randstad Digital, we are passionate about providing equal employment opportunities and embracing diversity to the benefit of all. We actively encourage applications from any background.