How Tesla's Autopilot works

Recently, Andrej Karpathy, Head of Artificial Intelligence (AI) on Tesla’s Autopilot Team, offered several interesting insights on how Autopilot works. Read ARK Invest's key takeaways.

How Tesla's Autopilot works

April 27, 2020
Recently, Andrej Karpathy, Head of Artificial Intelligence (AI) on Tesla’s Autopilot Team, offered several interesting insights on how Autopilot works. Read ARK Invest's key takeaways.
Read Transcript

Recently, Andrej Karpathy, Head of Artificial Intelligence (AI) on Tesla’s Autopilot Team, offered several interesting insights on how Autopilot works in a video. First, he detailed how Tesla trains Autopilot with its customer-collected data feed using long tail examples like stop signs painted on buildings or those occluded by tree branches. To improve Autopilot’s reaction to these “corner cases”, Tesla sends a software detector to its 800,000+ vehicle fleet to identify images of, say, occluded stop signs. In contrast, with fleets of hundreds instead of hundreds of thousands driving in several cities instead of nationwide, GM’s Cruise Automation and Waymo have limited access to data on corner cases in training their vehicles.

Karpathy also explained how Tesla is bridging the gap between cameras and LiDAR. We previously heard about an Autopilot update involving 3D video labeling as opposed to 2D image labeling, enabling faster, more accurate image detection and path planning - Tesla’s solution to the LiDAR gap. LiDAR recognizes images and assesses direct depth more accurately than do cameras. In a two-step indirect process, cameras take shots of images and software gauges depth by analyzing the pixels. Mistakes involving a few pixels can translate into meters or yards of inaccuracy. By labeling 3D videos of driving scenes, Tesla is compensating for the camera’s weakness as the primary image sensor in its vehicles.

Karpathy also discussed Tesla’s local mapping which includes much less detail than the high definition maps its competitors use. While a Waymo vehicle drives with preloaded information about the exact location of a stop sign, within centimeters of accuracy, a Tesla would detect only the presence of a stop sign somewhere in the vicinity. In addition to vague local maps and its camera-based approach, 3D video labeling separates Tesla from its competitors, enabling the recognition of corner cases in solving for full autonomy.

We believe Tesla’s approach is highly differentiated and will be almost impossible for a competitor to replicate. While autonomous driving is an extremely complex problem to solve, Tesla could enjoy a near-monopoly in autonomous ride hailing if it is successful.

 

Catherine Wood, CEO and CIO of ARK Invest pitched Tesla at the 2019 Sohn Hearts & Minds Conference.

----

ARK's statements are not an endorsement of any company or a recommendation to buy, sell or hold any security. For a list of all purchases and sales made by ARK for client accounts during the past year that could be considered by the SEC as recommendations, click here. It should not be assumed that recommendations made in the future will be profitable or will equal the performance of the securities in this list. For full disclosures, click here.

Recently, Andrej Karpathy, Head of Artificial Intelligence (AI) on Tesla’s Autopilot Team, offered several interesting insights on how Autopilot works in a video. First, he detailed how Tesla trains Autopilot with its customer-collected data feed using long tail examples like stop signs painted on buildings or those occluded by tree branches. To improve Autopilot’s reaction to these “corner cases”, Tesla sends a software detector to its 800,000+ vehicle fleet to identify images of, say, occluded stop signs. In contrast, with fleets of hundreds instead of hundreds of thousands driving in several cities instead of nationwide, GM’s Cruise Automation and Waymo have limited access to data on corner cases in training their vehicles.

Karpathy also explained how Tesla is bridging the gap between cameras and LiDAR. We previously heard about an Autopilot update involving 3D video labeling as opposed to 2D image labeling, enabling faster, more accurate image detection and path planning - Tesla’s solution to the LiDAR gap. LiDAR recognizes images and assesses direct depth more accurately than do cameras. In a two-step indirect process, cameras take shots of images and software gauges depth by analyzing the pixels. Mistakes involving a few pixels can translate into meters or yards of inaccuracy. By labeling 3D videos of driving scenes, Tesla is compensating for the camera’s weakness as the primary image sensor in its vehicles.

Karpathy also discussed Tesla’s local mapping which includes much less detail than the high definition maps its competitors use. While a Waymo vehicle drives with preloaded information about the exact location of a stop sign, within centimeters of accuracy, a Tesla would detect only the presence of a stop sign somewhere in the vicinity. In addition to vague local maps and its camera-based approach, 3D video labeling separates Tesla from its competitors, enabling the recognition of corner cases in solving for full autonomy.

We believe Tesla’s approach is highly differentiated and will be almost impossible for a competitor to replicate. While autonomous driving is an extremely complex problem to solve, Tesla could enjoy a near-monopoly in autonomous ride hailing if it is successful.

 

Catherine Wood, CEO and CIO of ARK Invest pitched Tesla at the 2019 Sohn Hearts & Minds Conference.

----

ARK's statements are not an endorsement of any company or a recommendation to buy, sell or hold any security. For a list of all purchases and sales made by ARK for client accounts during the past year that could be considered by the SEC as recommendations, click here. It should not be assumed that recommendations made in the future will be profitable or will equal the performance of the securities in this list. For full disclosures, click here.

Disclaimer: This material has been prepared by ARK Invest, published on Apr 27, 2020. HM1 is not responsible for the content of linked websites or content prepared by third party. The inclusion of these links and third-party content does not in any way imply any form of endorsement by HM1 of the products or services provided by persons or organisations who are responsible for the linked websites and third-party content. This information is for general information only and does not consider the objectives, financial situation or needs of any person. Before making an investment decision, you should read the relevant disclosure document (if appropriate) and seek professional advice to determine whether the investment and information is suitable for you.

facebook
linkedin
All
Thank you! Your submission has been received!
Oops! Something went wrong while submitting the form.
No items found.
December 10, 2024

Professor Jane Butler: Sparking Hope for Spinal Cord Injuries

In this episode of the Hearts & Minds Podcast, we sit down with Professor Jane Butler to discuss her groundbreaking research into spinal cord injuries.

Read More
impact-podcasts
September 24, 2024

Asian Market Potential with Tom Naughton of Prusik

CIO Charlie Lancaster sits down with Tom Naughton, CIO of Prusik Investment Mgmt. Tom shares his investment philosophy, the opportunities and challenges in Asian markets, and how his 2023 conference stock pick, Swire Pacific (0019.HK), delivered an impressive 30% return.

Read More
investing
September 4, 2024

Building Hearts and Minds with Co-Founders Matthew Grounds and Guy Fowler

In this episode, co-founders Matthew Grounds AM and Guy Fowler OAM discuss their journey in building Hearts & Minds and its philanthropic model that has donated over $70 million to medical research.

Read More
investing
June 25, 2024

Navigating the Resource Sector with Jeremy Bond of Terra Capital

In this episode, we chat with Jeremy Bond, Founder of Terra Capital and HM1 Conference Fund Manager. Tune in for insights into the world of resource investments and the exciting opportunities that lie ahead.

Read More
investing
June 11, 2024

Prof. Nadia Badawi on Cerebral Palsy Breakthroughs and Neonatal Care

Dive deep into the groundbreaking work of Professor Nadia Badawi, an internationally recognised neonatologist and expert in Cerebral Palsy.

Read More
impact-podcasts
May 28, 2024

Investment Insights: Rikki Bannan on Top Picks and Trends

Join us for an engaging episode featuring Rikki Bannan, Portfolio Manager of IFM Investors and HM1 Conference Fund Manager. This episode explores Rikki's career journey, investment strategies, and her 2023 conference stock pick, Telix Pharmaceuticals (ASX.TLX).

Read More
investing
December 6, 2023

Peter Cooper talks building and instilling a culture of humility and excellence

In this episode, our guest is the renowned investor, Peter Cooper, founder and Chief Investment Officer of Cooper Investors (Core Fund Manager). A founding supporter of Hearts and Minds, Peter is a staunch advocate of our model and its philanthropic purpose, actively engaging in every facet of Hearts and Minds.

Read More
investing
November 28, 2023

Jun Bei Liu on her high conviction investment strategy

In this episode, HM1 Chief Investment Officer Charlie Lanchester is joined by Jun Bei Liu. Jun Bei is the Portfolio Manager of Tribeca’s Alpha Plus Fund and since taking over managing the Fund, she has quadrupled AUM.

Read More
investing
November 21, 2023

The world of rare genetic disease research

In this episode, we speak to Associate Professor Gina Ravenscroft. Gina is an Associate Professor in Neurogenetics at the Harry Perkins Institute of Medical Research in Perth. Her research interests are in rare genetic diseases, with a particular focus on neurogenetic diseases in babies and children.

Read More
impact-podcasts
November 14, 2023

Learn what makes a high conviction investment and how to avoid short-term noise

In this episode, our Core Fund Manager Magellan shares how they select top stocks for the HM1 portfolio.

Read More
investing
November 7, 2023

Delve into the world of kids critical care and trauma research

In thie episode, we are joined by Dr. Marino Festa, or Rino for short. He is the Medical Director of NSW Kids ECMO Referral Service and a senior specialist in Paediatric Intensive Care at Children’s Hospital at Westmead.

Read More
impact-podcasts
October 31, 2023

Where Regal's Phil King is searching for opportunities

HM1's CIO, Charlie Lanchester, talks to Phil King of Regal Funds about his passion for stocks, his ongoing search for opportunities, and some of the sectors he’s excited by right now. Phil King of Regal Funds, has been a tremendous supporter of Hearts & Minds since the beginning.

Read More
investing
October 24, 2023

Preventing recurrent miscarriages and birth defects

In this episode, CEO Paul Rayson is joined by renowned biomedical researcher Professor Sally Dunwoodie. Prof. Dunwoodie's groundbreaking work has revolutionised clinical practices and enabled genetic diagnostic tests worldwide. In 2017, her team achieved a double breakthrough with the potential to prevent recurrent miscarriages and various birth defects.

Read More
impact-podcasts
October 17, 2023

Nick Griffin on how he finds global winners

In this episode, CIO Charlie Lanchester chats with Nick Griffin, the founding partner and CIO of Munro Partners, one of HM1's Core Fund Managers. They go over his career to date, reflect on the lessons he’s learned, and trace the decisions that led to him starting Munro.

Read More
investing
October 10, 2023

How A/Prof Matt Call is teaching our body to kill cancer

In this episode, CEO Paul Rayson is joined by WEHI’s Associate Professor Matt Call to talk about his incredible research. Matt’s team teaches and trains the body's own immune cells to target and kill cancer cells.

Read More
impact-podcasts

No results found.

Please try a different search keyword or filter.