Data Engineer
- תל אביב
- משרה קבועה
- משרה מלאה
- Implement scalable data processing pipelines using Apache Spark for both batch and streaming use cases.
- Manage and optimize storage and retrieval of high-dimensional data using Vector Databases.
- Develop flexible data models and maintain large-scale datasets using NoSQL databases.
- Build and maintain core data services and tools using Java and Python, with a focus on performance and maintainability.
- Deploy, scale, and manage containerized applications using Docker and Kubernetes (k8s).
- Collaborate with data scientists, ML engineers, and platform teams to deliver high-quality data solutions.
- Apply best practices in data governance, quality assurance, and operational monitoring to ensure data integrity and reliability.
- 4+ years of hands-on experience in big data engineering or a similar role.
- Proficiency in Apache Spark, including batch/streaming and performance tuning.
- Strong programming skills in Java and Python.
- Proven ability to design and manage workflows using Apache Airflow.
- Hands-on experience with containerization and orchestration tools: Docker and Kubernetes.
- Solid understanding of distributed systems and scalable data architectures.
- Familiarity with CI/CD processes and tools.
- Proficiency with Elastic Search, Vespa.AI or any other Vecors DB as a big advantage.
- Background in NLP, computer vision, or other relevant ML fields
- Familiarity with the Hadoop ecosystem
Mploy