Perceptive Automata: Giving Robots the Ability to Understand People

Autonomous vehicles operate autonomously in very controlled environments today, in places where pedestrian operations are very predictable or where people are kept at a safe distance. But in more challenging environments, safety drivers or human surveillance will take over if the situation becomes too complex or risky. With Perceptive Automata’s SOMAI, the vehicle is able to understand pedestrians’ intentions so they can safely continue the driving task without human intervention.

Suppose the goal is full autonomy, where a robot can operate independently as a human does in challenging scenarios among humans. In that case, the value of a product like SOMAI becomes extremely clear. And it’s not merely “nice-to-have” software. It’s a critical, missing piece for true autonomous driving that can make systems safer and AVs operate more “human-like” on the road.

We recently interviewed Patrick Curley, Chief Product Officer of Perceptive Automata, to know more about SOMAI and Perceptive Automata’s vision for the future. Read on for the excerpts from the interview.

Q. Could you tell us a bit about your product SOMAI?

State of Mind AI (SOMAI) – is targeted at customers attempting to solve the problem of safe operation of autonomous vehicles around pedestrians. Current solutions operate effectively around a narrow class of human behavior – usually based on a linear prediction model. Specifically, if a person continues to do what they are currently doing, AVs understand and navigate those scenarios fairly effectively. But they utilize zero insight into when their behavior changes, or might change.

The cases we focus on at Perceptive Automata are the difficult ones. We solve for the pedestrians who are considering changing their behavior in a way that impacts the driving task. We’ve taught SOMAI to “read” people, the same way humans do, using their intuition. People often change their minds and their course of action, for a variety of reasons. SOMAI solves that class of driving challenges. It’s what makes SOMAI game changing.

We’ve hired the best minds in psychophysics, computer vision, and machine learning to build a software model that understands pedestrians the same way a human driver would. We study thousands of hours of real, live driving, and learn what human drivers would do, and how they know to do it. And we’ve implemented that same know-how into SOMAI. It knows what pedestrians are likely to do next, so the AV can plan a solution space to accommodate their changing behavior. That’s what makes this so exciting. This is the key to unlocking the promise of L5 autonomous operation, and we believe it is a necessary step all AVs will need to take to get there.

Q. What challenges are you addressing with SOMAI’s product suite?

Well, we are solving the hardest problem in robotics in general today: endowing AI with human-level understanding – human intuition. It’s not a pipedream. As the world’s leading experts in AI-powered autonomous systems that recognize and understand human behavior, we provide human-centric AI solutions that will one day support many applications. We believe our technology is relevant to any industry that could use AI-powered autonomous systems, such as sidewalk delivery robots, worksite robots, drones for industry, and defense robotics with diverse challenges in human-machine interaction, etcetera. Even Industry 5.0, where people are working alongside robots and smart machines, is interesting to us. I guess you can say that we’re developing a “product suite” in the sense that we’re listening to the needs of our partners – from some of the world’s largest companies – taking on their critical pain points, and trying to apply our approach to solve their problems and edge cases.

Right now, we’re focused on autonomous vehicles and helping solve challenges they may encounter in roadways among people – which could hamper their pursuit of full-deployment and public acceptance if NOT solved. Consider robotaxis for example. Robotaxis have dominated the news recently and rightly so. These types of autonomous vehicles offer tremendous potential to transform ridesharing, and, really, urban mobility, in the near future. That’s why so many are currently being tested in California, China, Germany and other places throughout the world. But, as recent articles have shown, navigating pick-up and drop-off areas coherently has become one of the bigger challenges for robotaxis. That’s usually because traditional artificial intelligence has a hard time understanding if the person in front of a hotel entrance looking at their phone is going to stay still, or stride out towards the street to pick up a rideshare car. Also, how does a robotaxi tell the difference between its would-be passenger if she/he is among a group of people? Issues like those are very important to the industry. And SOMAI is going to solve these difficult cases!

Q. As the CPO of Perceptive Automata, what are your main focus areas?

My main focus since I joined Perceptive Automata is to deepen our understanding of customer problems. We’ve proven the technology. We absolutely can do what we said we could do. We can replicate human intuition about pedestrians’ intentions in real time. Now, it’s a matter of understanding and solving for customer pain points. Where their technology is effective and where it’s not. We are convinced our approach is not only sufficient, but necessary to solve these problems. And I’m excited to see what the next year brings in the realization of fully autonomous vehicle operation. I can’t wait to hail a robotaxi the same way I would an Uber today. They will be ubiquitous, the same way Uber & Lyft are.

Q. How hard does Perceptive Automata work on enhancing the product experience?

We have been busy working on the tooling for customers to use SOMAI. When you get past the demonstration and sales process, our customers need to work with SOMAI directly, get their hands dirty using their own data sets. We also have an integrations team that can work with our customers’ development teams to make SOMAI work seamlessly in their own environment.

During integration, there are always edge cases or problems that come up. Our development environment and technical staff is positioned to isolate and solve any problems that might arise. There are no “Black Boxes” with SOMAI. It’s our technology top to bottom. And we’re prepared to support our customers with the resources necessary to make them successful.

Q. What is Perceptive Automata’s vision for the future?

Currently, we are talking with partners in the Level 4 or 5 autonomous vehicle space. Level 4 and 5 being the highest levels of operation of an autonomous vehicle. But maybe more importantly than space is the environments we are enabling. Perceptive Automata’s vision is to enable autonomous vehicles to operate in some of the most challenging environments possible. Think high-traffic areas for both AVs and Pedestrians. Think taxi service at hotel lobbies, airports, or sport events. Why are these so attractive? Because they represent the most profitable applications for AVs. Those applications are exactly where we want to go.

About the Leader

Patrick Curley, Chief Product Officer

Patrick is a successful Product Executive having worked with many growth stage companies. His specialty is Product Market Fit, optimizing the core competency and assets of a company with emerging customer needs. With his understanding of both the technical and customer sides of the house, he is uniquely qualified to find the sweet spot where customer value is maximized.

"We’ve taught SOMAI to ‘read’ people, the same way humans do, using their intuition.”