Lead Data Engineer

תל אביב
משרה קבועה
משרה מלאה

לפני חודש

Modern vehicles create an enormous amount of data. Upstream turns that data into real-time cybersecurity detection & response (XDR), API security, and proactive quality detection for the world's largest automotive and smart mobility companies. Our Data Engineering & Data Science Group leads the development of our Iceberg-based data platform, including data lake, query engine, and ML-Ops tools, serving as a solid AI-ready foundation for all our products.At Upstream, you'll have real impact, your work powers every security alert and analytics view we deliver. You'll have room to innovate, pick new tools, refactor old ones, and set best practices. All within a people-first culture that values hybrid work, shared knowledge, and sustainable pace.Technological background: Iceberg, Trino, Prefect ,GitHub Actions, Kubernetes. JupyterHub, MLflow, dbtThis role is full time and is Herzliya, Israel based.Responsibilities:

Spend about 70% of your time coding, building and tuning the data lake (Iceberg + Trino + dbt) for fast, cost-effective queries.
Lead and mentor skilled data engineers: code reviews, pairing, growth plans, and future hiring.
Own the platform roadmap including schema design, query-engine upgrades, governance and new technology choices.
Design and maintain well-partitioned tables and views so DS, DA, users and product teams can work faster and effectively.
Automate ETL data-digests and ML model training with Prefect, and own the pipelines for deployments (GitHub Actions → Kubernetes), data-quality checks, monitoring and SLOs.
Enhance and scale our ML platform, and automate the machine-learning lifecycle (MLflow, feature stores, model builds and deployment).
Stay focused on the data lake.

Requirements:

8+ years of software/data engineering (strong Python preferred; JVM/Go welcome).
4+ years working with large-scale data lakes/lakehouses, including Iceberg (or similar formats). Experience with Apache Spark for heavy batch / ML pipelines is a plus.
2+ years leading engineers, while still being hands-on.
Solid SQL skills and experience with much of our stack: Trino/Presto, dbt, Iceberg, Prefect/Airflow, Kubernetes, GitHub Actions.
A builder's mindset: measure, improve, document, then automate.
Clear communication in English and a collaborative attitude.

Upstream is an equal opportunity employer. All candidates for employment will be considered without regard to race, color, religion, sex, national origin, physical or mental disability, veteran status, or any other basis protected by applicable federal, state or local law.⚠הגש מועמדות באתר החברהמשרות דומות שיכולות לעניין אותך25/07/2025תל אביבSilverfort**Silverfort is a cyber-security startup that develops a revolutionary identity protection platform. Using patented technology, our product enables st...20/07/2025תל אביבElsight - UAV & UGV Connection ConfidenceDescriptionWe are looking for a driven and independent Sales Manager to lead our growth efforts across *Israel and the Asia-Pacific region...29/07/2025תל אביבQualitest Israel****Qualitest, The World's Leading AI-Powered Quality Engineering Company****At Qualitest we're all about ensuring everything runs smoothly, whe...20/07/2025תל אביבCirclesAbout Circles:At Circles, we believe in providing emotional support through expert-led, real time support groups. Our transformative group suppo...20/07/2025תל אביבRiskifiedAbout UsRiskified empowers businesses to unleash ecommerce growth by taking risk off the table. Many of the world's biggest brands and publicly trad...30/07/2025תל אביבSilverfort**Silverfort is a cyber-security startup that develops a revolutionary identity protection platform. Using patented technology, our product enables st...29/07/2025כפר סבא, 15.72 ק"מ ממיקומךDialogA high-tech company specializing in streaming technology.The company develops and delivers advanced solutions for live video broadcasting on the int...16/07/2025תל אביבPlanck**Who are we?**Planck, now a part of Applied Systems Inc., is revolutionizing the insurance market with its advanced use of Generative AI (GenAI...16/07/2025תל אביבPlanck**Who are we?**Planck, now a part of Applied Systems Inc., is revolutionizing the insurance market with its advanced use of Generative AI (GenAI...קצת עלינוMploy הוא לוח דרושים מבוסס AI, שנועד לסייע למחפשי עבודה ולמעסיקים כאחד, תוך יצירת פלטפורמה חדשנית, איכותית המובילה את שוק העבודה בישראל.אנו מאגדים משרות עדכניות מאלפי מקורות בארץ, ומנגישים אותן ביעילות באמצעות סוכן AI חכם שמתאים משרות רלוונטיות למועמדים ומאפשר הגשת מועמדות בלחיצת כפתור.הפלטפורמה שלנו מציעה התאמות משרות מבוססות בינה מלאכותית עם אחוז התאמה אישי, קבוצות WhatsApp ייעודיות לפי תחום, ואפליקציה מתקדמת שמאפשרת חיפוש ושליחת קורות חיים מכל מקום ובכל זמן.Mploy אצלכם בוואטסאפ✨ רוצים להתעדכן בכל המשרות הכי שוות ישר לנייד?הצטרפו לקבוצות הוואטסאפ שלנו וקבלו את כל ההצעות המתאימות - בלי לחפש, ובלי לפספס. מחכים לכם! 📱😊

Mploy

הגש מועמדות