“Neolix, a start-up so Chinese that it has only barely has an English-language website, has announced mass-production of its autonomous delivery vehicles and declared itself the first company in the world to do this, according to Bloomberg.”
“Perhaps radar is even underappreciated. Venture capital has flowed into lidar and camera-based solutions for automated vehicles; radar has been viewed as a commodity.”
That seems right to me.
The article highlights three companies working on different approaches to more advanced radar for self-driving cars. The work from Bosch to create radar-based high-definition maps seems particularly interesting.
“By coupling these two inputs [radar and GPS], Bosch’s system can take that real-time data and compare it to its base map, match patterns between the two, and determine its location with centimeter-level accuracy.”
Bosch calls this approach, “radar road signature” and posits that it can provide centimeter-level accuracy while using half as much data as a camera-based map.
Bosch is highlighting their work with TomTom to build radar-based HD maps. They divide these maps into three layers:
Localization: calculating where the car is in the world
Planning: deciding which actions are available to the car
Dynamic: predicting what other actors in the environment will do
This is exciting work because high-definition (HD) maps are usually the domain of lidar. Lidar point clouds are used to generate maps against which a vehicle can compare later sensor readings.
Some work has gone into attempts to build such maps with camera data. Visual SLAM is one example of this. By comparison, relatively little work has gone into building HD maps from radar data. That makes Bosch’s endeavor novel and exciting.
Bosch is positioning this as a fleet-based mapping system, with map data generated by ordinary consumer cars, not necessarily specialized mapping vehicles. It’s hard to know how realistic that really is, but it would play to Bosch’s strength of scale.
“One million vehicles will keep the high-resolution map up to date.”
As the world’s largest automotive supplier, Bosch has a unique ability to pump a success into the automotive market.
Ford CTO Ken Washington, who used to be like my boss’s boss’s boss’s boss’s boss when I was at Ford, and seems like a great guy, has a post up about Digit, a humanoid robot that Ford is working on for last-mile deliveries.
Reading the post and watching the video, I have a few reactions:
This is awesome.
This will be insanely hard.
Giving a robot a lidar for a head is a stroke of genius, at least from an aesthetic perspective.
Ford is completely right that the last-mile (really, last-ten-yards) delivery problem is going to be a huge issue. Right now logistics companies rely on drivers to both operate a vehicle and to walk deliveries to customers’ front doors. Self-driving cars solve the first problem, but in a lot of cases that won’t ultimately have much of an impact if we can’t solve the second task.
So the motivation for Digit is spot-on.
But walking robots are bananas-level difficult.
Look no further than this video with the awesome title, “A Compilation of Robots Falling Down at the DARPA Robotics Challenge”:
Granted, this video is from 4 years ago and progress has been made, but my impression is that walking robots make self-driving cars look like an easy problem.
I remember taking an edX course from Russ Tedrake at MIT called “Underactuated Robotics” that was concerned with, among other things, walking robots. This course was so, so hard. The control problems inherent in a multi-joint, walking robot are of a staggering level of mathematical complexity.
Digit’s demo video is awesome, but we’ve all learned to be skeptical of demo videos. If Ford, together with Agility Robotics, can really crack the nut on a walking robot that can deliver packages, then they won’t even need to solve the autonomous vehicle problem. They’ll have the whole world beating down their door.
I’ve been thumbing through Sebastian’s magnum opus, Probabilistic Robotics. The book is now 13 years old, but it remains a great resource for roboticists. Kind of funny to think that, when Sebastian wrote this, he hadn’t even started to work on self-driving cars yet!
The chapter on Markov decision processes (MDPs) covers how to make robotic planning decisions under uncertainty. One of the key assumptions of MDPs is that the agent (robot) can observe its environment perfectly. This turns out to be an unrealistic assumption, which leads to further types of planning algorithms, principally partially observable Markov decision processes (POMDPs).
Nonetheless, ordinary Markov decision processes are a helpful place to start when thinking about motion planning.
“In this exercise, you are asked to extend dynamic programming to an environment with a single hidden state variable. The environment is a maze with a designated start marked “S”, and two possible goal states, both marked “H”.
“What the agent does not know is which of the two goal states provides a positive reward. One will give +100, whereas the other will give -100. There is a .5 probability that either of those situations is true. The cost of moving is -1; the agent can only move into the four directions north, south, east, and west. Once a state labeled “H” has been reached, the play is over.”
So far, so good.
“(a) Implement a value iteration algorithm for this scenario. Have your implementation compute the value of the starting state. What is the optimal policy?”
The value of the starting state is 0, because the optimal policy is to stay put.
This is basically just counting. Each move costs -1. The expected reward from reaching state “H” is 0, because there’s a 50% chance of a +100 reward, but also a 50% chance of a -100 reward.
0.5 * (+100) + 0.5 * (-100) = 50 + (-50) = 0
“(b) Modify your value algorithm to accommodate a probabilistic motion model: with 0.9 chance the agent moves as desired; with 0.1 chance it will select any of the other three directions at random. Run your value iteration algorithm again, and compute both the value of the starting state, and the optimal policy.”
Now we really need a computer to calculate the value function, because we have to iterate over all the cells on the map until values converge. For each cell, we have to look at each action and sum the 90% probability that the action will execute properly, and the 10% probability that the action will misfire randomly. Then we pick the highest-value action. Once we do this for every cell, we repeat the cycle over all the cells again, and we keep doing this until the values stabilize.
The first pass in the iteration sets all cells to 0. Depending on which direction we iterate from, the next step might look like this:
Nonetheless, even without a computer, it seems pretty clear that the optimal policy is still for our agent to stay put in the start cell. Without any information about which “H” is heaven and which is hell, there’s no ultimate reward for going anywhere.
“(c) Now suppose the location labeled X contains a sign that informs the agent of the correct assignment of rewards to the two states labeled “H”. How does this affect optimal policy?”
Without computing the policy, it seems likely that the optimal policy will involve going to the sign, identifying heaven and hell on the map, and then proceeding to heaven.
This policy seems qualitatively clear because of the relatively high payoff for reaching heaven (+100), the relatively low cost of motion (-1), the relatively high probability of the motion executing accurately (0.9), and the relatively small size of the map (distance from S to X to H = 19).
It’s easy to imagine tweaking these parameters such that it’s no longer so obvious that it makes sense to go find the sign. With different parameters, it might still make sense to stay put at S.
“(d) How can you modify your value iteration algorithm to find the optimal policy? Be concise. State any modifications to the space over which the value function is defined.”
Basically, we need to figure out the value of reaching the sign. There are essentially two value functions: the value function when we cannot observe the state, and the value function when we can.
Another way to put this is that going to the sign is like taking a measurement with a sensor. We have prior beliefs about the state of the world before we reach the sign, and then posterior beliefs once we get the information from the sign. Once we transition from prior to posterior beliefs, we will need to recalculate our value function.
An important point here is that this game assumes the sign is 100% certain, which makes the model fully observable. That’s not the case with normal sensors, which is why real robots have to deal with partially observable Markov decision processes (POMDPs).
“(e) Implement your modification, and compute both the value of the starting state and the optimal policy.”
Again, we’d need to write code to actually implement this, but the general idea is to have two value functions. The value of X will be dependent on the posterior value function (the value function that we can calculate once we know which is heaven and which is hell). Then we use that value of X to calculate our prior distribution.
For example, here are the value functions, assuming perfect motion:
The posterior value function, after reading the sign at “X”.The posterior value function, before reading the sign at “X”.
Probably it’s worth programming this little game to make sure I got all this correct.
My Udacity colleague, Cezanne Camacho, is preparing a presentation on capsule networks and gave a draft version in the office today. Cezanne is a terrific engineer and teacher, and she’s already written a great blog post on capsule networks, and she graciously allowed me to share some of that here.
Capsule Networks provide a way to detect parts of objects in an image and represent spatial relationships between those parts. This means that capsule networks are able to recognize the same object in a variety of different poses even if they have not seen that pose in training data.
Love the Pacman GIF. Did I mention Cezanne is also an artist?
Cezanne explains that a “capsule” encompasses features that make up a piece of an image. Think of an image of a face, for example, and imagine capsules that capture each eye, and the nose, and the mouth.
These capsules organize into a tree structure. Larger structures, like a face, would be parent nodes in the tree, and smaller structures would be child nodes.
“In the example below, you can see how the parts of a face (eyes, nose, mouth, etc.) might be recognized in leaf nodes and then combined to form a more complete face part in parent nodes.”
“Dynamic routing” plays a role in capsule networks:
“Dynamic routing is a process for finding the best connections between the output of one capsule and the inputs of the next layer of capsules. It allows capsules to communicate with each other and determine how data moves through them, according to real-time changes in the network inputs and outputs!”
Dynamic routing is ultimately implemented via an iterative routing process that Cezanne does a really nice job describing, along with the accompanying math, in her blog post.
Capsule networks seem to do well with image classification on a few datasets, but they haven’t been widely deployed yet because they are slow to train.
“The transcript highlights, in particular, the distinction between NXP’s traditional automotive semiconductor business, which declined, and its advanced driver assistance systems (ADAS) and battery management systems (BMS), both of which grew dramatically, albeit from small bases.”
NXP is kind of like the automotive industry in miniature: vehicle sales are declining today, causing decreases in revenue associated with traditional automotive manufacturing. But in the not-so-distant future, mobility will change and new products, like advanced driver assistance systems and battery management systems, will grow quickly.
My Udacity colleague Vienna Harvey sat down with Australian podcaster Zoe Eather to discuss the role of both ethics and education as they relate to self-driving cars. It’s a fun episode :-)
This interview is part of Zoe’s Smart Community podcast, which covers everything from infrastructure, to data, to climate change, to mobility.
Prior to Vienna’s interview, I got to take Zoe for a spin in Carla, Udacity’s self-driving car. Zoe was delightful and I think you’ll enjoy listening to her and Vienna geek out about self-driving cars.