19 feb
Intuitive Cloud
Monterrey
**ROLES & RESPONSIBILITIES**:
- Design, develop, implement, and tune large-scale distributed systems and pipelines that process large volume of data; focusing on scalability, low-latency, and fault-tolerance in every system built
- Developing scalable and re-usable frameworks for ingesting large data from multiple sources.
- Modern Data Orchestration engineering - query tuning, performance tuning, troubleshooting, and debugging big data solutions.
- Provides technical leadership, fosters a team environment, and provides mentorship and feedback to technical resources.
- Deep understanding of ETL/ELT design methodologies, patterns, personas, strategy, and tactics for complex data transformations.
- Data processing/transformation using various technologies such as spark and cloud Services.
- Understand current data engineering pipelines using legacy SAS tools and convert to modern pipelines.
**Data Infrastructure Engineer Strategy Objectives: End to End Strategy**
Define how data is acquired, stored, processed, distributed, and consumed. Collaboration and Shared responsibility across disciplines as partners in delivery for progressing our maturity model in the End-to-End Data practice.
- Understanding and experience with modern cloud data orchestration and engineering for one or more of the following cloud providers - AWS, Azure, GCP.
- Leading multiple engagements to design and develop data logistic patterns to support data solutions using data modeling techniques (such as file based, normalized or denormalized, star schemas, schema on read, Vault data model, graphs) for mixed workloads, such as OLTP, OLAP, streaming using any formats (structured, semi-structured, unstructured).
- Understanding of one or more is a big plus -CI/CD, cloud devops, containers (Kubernetes/Docker, etc.),
Python/PySpark/JavaScript.
- Implementing cloud data orchestration and data integration patterns (AWS Glue, Azure Data Factory, Event Hub, Databricks, etc.), storage and processing (Redshift, Azure Synapse, BigQuery, Snowflake)
- Possessing a certification(s) in one of the following is a big plus - AWS/Azure/GCP data engineering, and Migration.
**KEY REQUIREMENTS**:
- 10+ years’ experience as data engineer.
- Must have 5+ Years in implementing data engineering solutions with multiple cloud providers and toolsets.
- This is hands on role building data pipelines using Cloud Native and Partner Solutions. Hands-on technical experience with Data at Scale.
- Must have deep expertise in one of the programming languages for data processes (Python, Scala). Experience with Python, PySpark, Hadoop,
Hive and/or Spark to write data pipelines and data processing layers.
- Must have worked with multiple database technologies and patterns. Good SQL experience for writing complex SQL transformation.
- Performance Tuning of Spark SQL running on S3/Data Lake/Delta Lake/ storage and Strong Knowledge on Databricks and Cluster Configurations.
- Nice to have Databricks administration including security and infrastructure features of Databricks.
- Experience with Development Tools for CI/CD, Unit and Integration testing, Automation and Orchestration
Ability to commute/relocate:
- Monterrey, N. L.: Reliably commute or planning to relocate before starting work (required)
**Speak with the employer**
+91 (phone hidden)
Muestra tus habilidades a la empresa, rellenar el formulario y deja un toque personal en la carta, ayudará el reclutador en la elección del candidato.