AI Research Engineer, VLM, Autonomy & Robotics

Company: Tesla, Inc.
Location: Palo Alto
Posted on: February 9, 2025

Job Description:

AI Research Engineer, VLM, Autonomy & RoboticsState-of-the-art Vision Language Models (VLMs) have advanced rapidly, yet they still struggle with physical reasoning and real-world understanding-often due to a "text first, vision second" training paradigm and insufficient large-scale, diverse, real-world datasets. By leveraging Tesla's extensive global vehicle fleet and our rapidly growing humanoid robot platforms, we aim to reshape how VLMs perceive and interpret the physical world.In this role, you'll have access to unparalleled compute resources, massive multimodal real-world datasets, and close collaboration with a small team of world-class AI research engineers. You'll be involved in every stage of the VLM pipeline-pre-training, alignment, post-training, reinforcement learning, evaluation, distillation, deployment, and efficient inference-pushing the boundaries of vision-language integration for real-world applications.What You'll Do

• Compute and verify scaling laws for real-world understanding using large GPU clusters and extensive datasets
• Develop and debug large distributed training jobs spanning tens of thousands of GPUs
• Align our pre-trained foundation vision models with large language models for unified perception and language comprehension
• Build new human-labeled and synthetic datasets addressing real-world tasks and physical reasoning
• Explore reward functions and SOTA RL techniques to enhance real-world understanding and problem-solving
• Leverage Tesla's data to create robust evaluation sets focused on real-world scenarios and physical accuracy
• Perform knowledge distillation from larger models to smaller, edge-optimized models deployable across Tesla cars and robots
• Apply quantization, inference-time optimizations, and device-specific tweaks to reduce power consumption and latencyWhat You'll Bring
  • Deep Learning Background: Experience with large-scale vision-language models, multimodal transformers, or related architectures
  • Distributed Systems Expertise: Proven ability to train and optimize models on high-performance clusters (thousands of GPUs)
  • Practical Dataset Management: Comfort curating or generating large, diverse datasets-human-labeled, synthetic, or both
  • Reinforcement Learning Knowledge: Familiarity with RL algorithms and reward function design, especially for complex real-world tasks
  • Hands-On Approach: Willingness to iterate quickly on experimental ideas-from pre-training to final deployment
  • Collaboration & Communication: Strong cross-functional skills, able to work with AI research engineers, robotics teams, and software groupsCompensation and BenefitsAlong with competitive pay, as a full-time Tesla employee, you are eligible for the following benefits at day 1 of hire:
    • Aetna PPO and HSA plans > 2 medical plan options with $0 payroll deduction
    • Family-building, fertility, adoption and surrogacy benefits
    • Dental (including orthodontic coverage) and vision plans, both have options with a $0 paycheck contribution
    • Company Paid (Health Savings Account) HSA Contribution when enrolled in the High Deductible Aetna medical plan with HSA
    • Healthcare and Dependent Care Flexible Spending Accounts (FSA)
    • 401(k) with employer match, Employee Stock Purchase Plans, and other financial benefits
    • Company paid Basic Life, AD&D, short-term and long-term disability insurance
    • Employee Assistance Program
    • Sick and Vacation time (Flex time for salary positions), and Paid Holidays
    • Back-up childcare and parenting support resources
    • Voluntary benefits to include: critical illness, hospital indemnity, accident insurance, theft & legal services, and pet insurance
    • Weight Loss and Tobacco Cessation Programs
    • Tesla Babies program
    • Commuter benefits
    • Employee discounts and perks programExpected Compensation$116,000 - $360,000/annual salary + cash and stock awards + benefitsPay offered may vary depending on multiple individualized factors, including market location, job-related knowledge, skills, and experience. The total compensation package for this position may also include other elements dependent on the position offered. Details of participation in these benefit plans will be provided if an employee receives an offer of employment.Tesla is an Equal Opportunity employer. All qualified applicants will receive consideration for employment without regard to any factor, including veteran status and disability status, protected by applicable federal, state or local laws.Tesla is also committed to working with and providing reasonable accommodations to individuals with disabilities. Please let your recruiter know if you need an accommodation at any point during the interview process.Privacy is a top priority for Tesla. We build it into our products and view it as an essential part of our business. To understand more about the data we collect and process as part of your application, please view our Tesla Talent Privacy Notice.
      #J-18808-Ljbffr

Keywords: Tesla, Inc., Palo Alto , AI Research Engineer, VLM, Autonomy & Robotics, Engineering , Palo Alto, California