Research Scientist, ML H/W-SW Codesign

Reality Labs (RL) is bringing Meta's vision to life through virtual reality (VR), augmented reality (AR), and wearable technology. Meeting the demanding compute and power efficiency needs of these products requires purpose-built silicon. The Reality Labs Silicon team is pushing boundaries with pioneering advances in computer vision, machine learning, mixed reality, graphics, displays, sensors, and novel approaches to human body mapping. Our custom chips will power AR and VR experiences that seamlessly blend the physical and digital worlds throughout everyday life. We're convinced that realizing this vision demands a full-stack approach: spanning transistors, architecture, firmware, and algorithms.

Meta is seeking a HW Research Scientist to join our Research and Development teams. The candidate will have experience working on AI models, hardware acceleration, and software systems related topics. The position will involve taking these skills and applying them to solve for some of the most crucial & exciting problems that exist in Meta Reality Labs. The primary objective will be to develop novel solutions that enable compute and power-efficient training and on-device inference of vision and language models for use cases in AR, VR, and edge devices. We are hiring in multiple locations.

Responsibilities

Identify and solve multi-discipline ML acceleration problems involving algorithms, network design, hardware architecture, and AR/VR use cases. Many of these would be first-time solutions in the industry
Work across hardware and software, to solve deep co-design problems with other Research scientists working in this area
Codesign and invent novel ML accelerator and system architecture solutions, and facilitate the integration of algorithms and software to utilize these enhancements
Develop state-of-the-art model compression and scalability techniques using numerics, pruning, distillation, etc
Optimize models on hardware accelerators to achieve the best performance given various real-time latency and power constraints
Deliver impact directly or by influencing partners through thorough, data-driven analysis
Define use cases, and develop a methodology and benchmarks to evaluate different approaches
Apply in-depth knowledge of how the ML acceleration interacts with the other systems around it
Attend conferences, interpret papers, and stay up to date with the latest research advancements in the field of ML acceleration; patent and/or publish novel outcomes in peer-reviewed conferences and journals

Qualifications

Masters or PhD in Electrical Engineering, Computer Science or equivalent experience
2+ years of specialized experience in one or more of the following machine learning/deep learning domains: Model compression, hardware-aware model optimizations, hardware accelerators architecture, GPU architecture, machine learning compilers, or ML systems, AI infrastructure, high-performance computing, performance optimizations, or machine learning frameworks (e.g., PyTorch), numerics and SW/HW co-design
Experience developing AI system infrastructure, AI algorithms, or AI hardware acceleration in C/C++ or Python
Must obtain work authorization in the country of employment at the time of hire, and maintain ongoing work authorization during employment Experience or knowledge of training/inference of large-scale AI models - CV and/or LLMs
Experience or knowledge of architecting ML hardware accelerators and systems
Experience working and communicating cross-functionally in a team environment
Experience with PyTorch, TensorFlow, or similar machine learning toolsets
Experience or knowledge of on-device algorithm development including hardware-aware ML models and/or optimizing ML compilers for efficient deployment on AI accelerators
Proven track record of achieving significant results as demonstrated by grants, fellowships, patents, as well as publications at leading workshops, journals, or conferences such as ICLR, NeurIPS, CVPR, ACL, ICML, MLSys, ISCA, MICRO, DAC, ASPLOS, etc
Demonstrated research and engineering experience via an internship, work experience, coding competitions, or widely used contributions in open source repositories (e.g., GitHub)
Experience solving complex problems and comparing alternative solutions, trade-offs, and diverse points of view to determine a path forward