Data Engineer
- לוד
- משרה קבועה
- משרה מלאה
- Ingestion and processing of large volumes of new data
- Deep understanding and analysis of data in close collaboration with Data Scientists
- Design and development of critical, diverse, and large-scale data processes
- Working in both Cloud and On-premise environments
- One work-from-home day per week
- Working in a stable and established company
- Advanced technological environment
- Professional and challenging team
- At least 3 years of experience as a Data Engineer
- 3 years of experience working with Object-Oriented Programming (OOP) languages
- 3 years of experience with Python
- Hands-on experience with Spark for Large Scale Data
- Bachelor's degree in Computer Science, Engineering, Mathematics, or Statistics
- At least 2 years of hands-on experience with AWS Services: Athena/Glue/StepFunction/EMR/Redshift/RDS
- Deep understanding of design, development, and optimization of complex solutions for processing large volumes of data
- Understanding of optimization techniques and working with partitions of various data types (Parquet, Avro, HDF5, Delta Lake)
- Experience working with Docker, Linux, CI/CD Tools, Kubernetes
- Experience with data pipeline tools like Airflow, Kubeflow
- Understanding of concepts and processes in Machine Learning
- Familiarity with GenAI/Prompt Engineering solutions
Mploy