Lead Data Engineer - Solution Design Consultant - London
Our client are seeking a Lead Data Engineer to join their UK consulting team. They are an established specialist consultancy renowned for their expertise in the Data and Analytics sector. The reputation they hold means they are continuing to grow and are looking for a data engineer to provide hands-on technical leadership to a delivery team, whilst also acting as a design authority "compass" for solution architecture.
In return, you will get to work on some of the most interesting projects in the UK, incorporating the latest technologies in data processing at scale. You will also have the opportunity to shape the future of a Data Engineering & Analytics practice.
Ideally we are looking for you to have experience in:
- Delivery experience working with the following technologies:
- Hadoop ecosystem
- Apache Spark ecosystem - deployed at scale on cloud infrastructure a big plus
- Git version control & versioning approaches
- Big Data Cloud Technologies - AWS, Google Cloud Platform, Azure
- ETL tooling e.g. Informatica, Ab Initio, Talend etc.
- Working knowledge of the following technologies are also preferable:
- Apache Beam (stream processing) and stream processing concepts
- Jenkins continuous integration & Continuous Delivery/Automated Testing practices
- Cron scheduling and/or Apache Airflow
- Data Visualisation tooling - Tableau, Looker, Qlik, Power BI
- SQL, Python, command line/Linux scripting
- Awareness of statistical and data mining techniques used to answer business questions through data
- Active interest in the latest technology trends, autonomously trialling & integrating you're your toolkit stack where deemed worthy
- Focus on collaboration and delivering iteratively - try > succeed/fail > adjust, rather than purely focusing on extensive up-front analysis
- Capable of defining and documenting solution architecture, and communicating to others the rationale behind design decisions
- Capable of considering and incorporating non-functional requirements for delivering Production-grade enterprise solutions
The role will be based from London/ Milton Keynes and you will be primarily be working with customers in the South East. Please note, that this is with a consultancy and whilst every effort is made to keep consultants local to home some travel across the UK is inevitable and must be something you can live with.
This vacancy is advertised by Agile Recruitment Ltd. The services advertised by Agile Recruitment Ltd for contract and interim recruitment are those of an Employment Business and for permanent and fixed-term recruitment are those of an Employment Agency.