Big Data Consultant - Edinburgh
We have an exciting opportunity to lead the Big Data projects of a growing Information Management company. You must have excellent communication skills as you will be in client-facing situations from the outset.
You should have the following skills:
Primary skill in data processing programming on Big Data technologies (including performance) with secondary skills on how to optimally structure data for processing on Big Data technologies.
- In depth understanding of big data processing technologies - Hadoop/Spark/Hive/MapReduce/Impala/Kafka etc.
- Programming experience - Java, Python, SQL etc.
- Provide project guidance/consultancy on limitations and capabilities, performance etc
- Undertake reverse engineering of physical data models
- Analyse data-related system integration issues and propose appropriate solutions
- Undertake query performance analysis, provide guidance and feedback into data model change as necessary
- Financial Services experience with an aligned standard logical model
- Develop best practices in physical modelling
- Analysis of logical data models and creation of appropriate physical data models
- Structure data on HDFS for optimal performance and efficiency
- Selection of optimal data structures for Hive/Impala processing including normalising/de-normalising approach
- Understanding of storage options on Hadoop including compression, file formats
- Develop partitioning, bucketing and indexing strategy
This vacancy is advertised by Agile Recruitment Ltd. The services advertised by Agile Recruitment Ltd for contract and interim recruitment are those of an Employment Business and for permanent and fixed-term recruitment are those of an Employment Agency.