Senior Technical Software Engineer - Big Data

Job Summary

As a key technologist in the Software Engineering group, help drive software engineering strategies, standards and guidelines at product and program level. Engineer business critical technology solutions through the application of software programming, quality management, and data management. Ensure robust engineering practices for solution planning, technology development, deployment, release, hyper-care and operation of technology solution. Leverage a wide ranging technical expertise to enable business to solve complex analytical problems in areas like planning, commerce, customer experience and operations.

Job Outline

1. Lead the discovery phase of small to medium projects to come up with high level design. Partner with the Product Managers, Architects and others in the team.
2. Lead technical design and build for medium to large sized solutions in a team. Translate complex functional and non-functional requirements into fit for purpose technical design. For Data and Analytics solutions, conduct required source system analysis, data profiling and complex source-to-target mapping activities to ensure solutions deliver on business needs. Design and review data models (both logical and physical) for solution fit.
3. Debugging and solving issues and follow design documents and implement/test codes and make design decisions while coding and using strong coding principles. Own any feature of the solution end to end. Update data inventories and registries as required to keep metadata and data lineage up-to-date, following agreed Data Governance standards, guidelines and principles.
4. Enable test automation while ensuring the CI/CD pipeline is always in good health and is responsible for the technical design and high-quality coding for technology products that passes the QA requirements and is production deployment ready. Automate testing and carry out functional and performance testing in collaboration with test engineers and quality assurance.
5. Build, maintain and operate technical environments for software development, continuous integration and continuous deployment for data solutions.
6. Contribute to software and data engineering playbooks for use by relevant people in information technology and data scientists in the business.
7. Contribute to sprint/work design and planning by providing subject matter expertise on business domain and relevant technologies and contribute to software engineering organisation in relevant technology and ensures participation in cross skilling using expertise.
8. Support acceptance testing of the technology solutions by the business ensuring the technology solution meets the business requirements Drive corrective, adaptive, preventative and perfective maintenance of solutions with support from environment management, service management, infrastructure management and cyber security.
9. Network with senior internal and external colleagues in area of expertise. Work with others to identify trade-offs or potential conflicts between functional goals and objectives.
10. Solve complex issues where analysis of situations or data requires an in-depth evaluation of variable factors. Exercises judgment in selecting methods, techniques and evaluation criteria for obtaining results.

Qualifications & Experience

Qualifications

Bachelor degree in a relevant field such as Computer Science, Computational Mathematics, Computer Engineering or Software Engineering. Master Degree will be a plus.

Experience

• Minimum of 5 years of experiance as Software Engineer.
• Experience of developing advanced technology solutions for large, complex organisations in a multi-functional environment.
• Experience in providing production support for business solutions.
• Deep experience in front end development, full stack development, data engineering or AI/ML development.
• Technology evaluation, vendor selection and technology vendor management.
• Implementation of contemporary engineering practices like lean software development, test driven development, continuous integration and continuous deployment.
• Establishing and managing technology platforms for DevOps, Data Pipelines, Code Management, Testing, Continuous Integration and Continuous Deployment.
• Good understanding of system integration architecture, and data architecture and models.

Knowledge/skills

• Understand the source systems data, gather & able to transfer the data into HDFS as RAW and Decomposed layer (including writing Oozie workflow, coordinators, Sqoop and Flume).
• Exposure in handling to massive quantities of data by taking advantage of both batch and speed methods.
• Strong knowledge of and experience with statistics; potentially other advanced math as well and good working knowledge of SQL ability.
• Deep knowledge in data mining, data modelling, data processing, machine learning, natural language processing or information retrieval.
• Experience in processing large amounts of structured/unstructured data.
• Capability to design and document solutions independently.
• Technology Domain.
• Key Technologies/Tools.
• Data Analysis, Modelling & Reporting.
• SQL, Excel, JSON, XSD, XML, Hive, AVRO, PARQUET, API, Data Vault 2.0, Microstrategy, Power BI, Business Intelligence, Business.
• Objects and Tabulo.
• Cloud technologies.
• ADLS, Data Factory, Logicapps, Azure functions & services, Databricks and Cloudera.
• Big Data.
• HDFS, Hadoop, Spark, Scala, Python, HBase, Oozie, Airflow and Apache NiFi.
• Data Integrations.
• Streaming (e.g. Kafka & Spark Streaming), Snaplogic, Azure Data Factory(ADF) & Sqoop.
• Database & NoSQL Technologies.
• Elasticsearch, Snowflake, Couchbase, Cassandra, Graph DB, Neo4j, Cosmos DB, Oracle, MySQL and SQL server.
• Machine Learning (ML) engineering.
• DataiKU, Python, R, Scala, Spark Mllib, Azure Databricks, Tensorflow, IBM Watson, NLP, Deep Learning, NumPy, Pandas and Scikit learn
  Continuous Integration and Deployment (CI/CD).
• GIT Bitbucket, SonarQube, Shiftleft, Jenkins, Jasmin, Karma, Quick & Nimble, Selenium, Cucumber, Protractor, Jmeter, Gatling, Appium, XCUITest, Swift, Docker, Openshift, Kubernetes, Jira-Agile, Confluence, VRA-CodeStream, Appdynamics and ShiftLeft.
• Automation.
• Java, Selenium, Oracle with VB Script, RESTAssured, SOAP, JS, Scala ( Gatling), APPD, HP Load Runner, Jmeter, HP UFT, Python, Nose, numpy, scikit-learn, pandas, Sikuli, Chai-Mocha, Junit, Nosetest, OneAutomation and Automation anywhere.

Salary & Benefits

Join us in Dubai and enjoy an attractive tax-free salary and travel benefits that are exclusive to our industry, including discounts on flights and hotels stays around the world.