About this role
AI/ML Development: Design, develop, and deploy machine learning models, LLMs, and AI workflows within the Databricks environment.
Data Pipeline Engineering: Build scalable, efficient Delta Lake pipelines using PySpark and Python to ingest, clean, and preprocess large-scale datasets for AI applications.
MLOps Implementation: Set up and manage the ML lifecycle using MLflow for model tracking, versioning, deployment, and monitoring.
Optimization & Performance: Optimize PySpark jobs, Databricks clusters, and Python code for maximum performance and cost efficiency.
Collaboration & Integration: Partner with Data Scientists, Data Engineers, and DevOps teams to integrate AI models into core business applications and APIs.
Governance & Security: Implement Unity Catalog for data governance, access control, and lineage tracking across AI assets.
.