Data Engineer
- תל אביב
- משרה קבועה
- משרה מלאה
- Be a part of a cross functional team of data, backend and DevOps engineers.
- Be responsible for ingesting large volumes of new data, followed by deep understanding and inspection of the data in close collaboration with data scientists.
- Lead the architecture, planning, design and development of mission-critical, diverse and large-scale data pipelines over both public and on-prem cloud solutions.
- At least 3 years of experience working as a Data Engineer
- At least 3 years of experience coding with Object-Oriented Programming (OOP) languages.
- Hands-on experience with Spark for large-scale batch processing.
- Hands-on experience working with AWS data processing tools and concepts.
- Proven understanding in designing, developing and optimizing complex solutions that move and/or manipulate large volumes of data.
- Sound understanding of partitioning and optimization techniques of different big data file formats (such as Parquet, DeltaLake, AVRO, HDF5)
- Experience with Docker, Linux, CI/CD tools and concepts, Kubernetes.
- Experience with data pipelining tools such as Airflow, Kubeflow or similar.
- BSc / MSc degree in Computer Science/Engineering / Mathematics or Statistics.
- Understanding of ML concepts and processes.
- Hands-on experience with DataBricks platform.
- Experience working on large scale and complex on-premises systems.
- Hands-on experience in ML frameworks and models training and implementation.
- Hands-on experience in lower-level programming languages such as C++ or RUST
- Develop an...
Mploy