【课程】MOB LEC2 传感器和自动驾驶底层架构

Sensors for robot perception

Sensors: Sensor is a device that measures or detects a property of the environment, or changes to a property.

Categorization of sensors: Exteroceptive (extero or surroundings), Proprioceptive (proprio or internal).

Type of Sensor	Feature	Weakness	Future Trend	More words
	Essential for robot to perceive environment with its rich semantics.	?	HD, wide dynamic ranges	Comparison Metrics: Resolution, Field of view, Dynamics range
	This simulates human binocular vision and able to give scales/metrics for images.	?	?	Definition: A type of camera with two or more lenses with a separate image sensor or film frame for each lens.
	It provides robots with detailed 3D scene geometry in the form of 3D point cloud.	?	HD, low-cost, solid state LiDAR	Definition: A sensing device that detects objects and maps their distances by illuminating a target with an optical pulse.
	Robust sensing tech. against poor visibility caused by fog, rain, dust and sand storms.	?	single-chip, 4D imaging radar	Definition: A sensing technology for detection of objects and providing the range, velocity and angle of these objects. It uses radio frequency technology operating in the spectrum between 30GHz and 300GH. More information: Doppler effect.
	use robust mechanical wave, all-weather and all-lighting distance measurement	Short-range (<5m)	evolve to 3D ultrasonic, produce point clouds for close objects.	Definition: An instrument that measures the distance to objects using ultrasonic sound waves.
[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-Hn6AvhA3-1666255627971)(https://image.discover304.top/blog-img/s19595309262022.png)]	?	?	?	Full name: Global Navigation Satellite Systems. Measuring Position and Velocity and Heading.
	?	?	?	Full name: Inertial measurement units. Measuring Angular rotation rate and Acceleration and Heading.
	?	Speed accuracy and Position drift	?	Tracks wheel velocity and orientation with a rotary encoder.

Synchronized Hardware

To synchronize different modules / sensors on a robot to provide a common clock. For autonomous driving, GPS and NTP (Network Time Protocol) time is often used as sources for a reference clock.

Time sync is crucial to multi-robot systems (e.g., a fleet of cars /drones) in which data will be shared and integrated for joint decision making.

Time sync is never a challenge, but the sync itself is a real challenge.

Computing hardware

Making decision

Takes in all sensory data and computes actions for the robot. Service robots have low requirements for the computing hardware (RPi 3/4 can be fine) but autonomous vehicles require powerful and customized computing units.

Parallel computing due to heavy loads

Many tasks in autonomous robots can easily drain the compute resource, such as using images or LiDAR point cloud for image processing, object detection and mapping.

Graphic Processing Unit (GPU)
Pending update

Field Programmable Gate Array (FPGA)
Pending update

Application Specific Integrated Chip (ASIC)
Pending update

Software decomposition for mobile robots

Environment Perception
Pending update

Environment Mapping
Pending update

Motion Planning
Pending update

Controller
Pending update

System Supervisor
Pending update

Question looking for answer

What are the basic components of a mobile robot?

How a mobile robot observes the physical word when moving around in the real world?

What is the necessary hardware and desirable software features of a mobile robot?

Design an omni-directional multi-sensor system for a mobile robot.

Describe the architecture of a typical (mobile) robotic system.

Understand the hardware and software components needed for a basic robot.

Why we cannot use lidar in harsh sunlight?

Smaller topics

Can we use ultrasonic sensor to measure the speed?
Maybe, Amazon has some research on this topic.

Is there any good purpose to do so?
In water.

Can we use other mechanic wave in autonomous vehicles? What about infrasonic?
Research project is possible.

What’s the purpose to have more lenses?
Generally on a robot, if a stereo camera have more than 2 cameras, only 2 of them are used for stereo vision.

Take the Intel D435 for example. The 2 cameras for stereo are black&white cameras. A third camera is added for RGB vision.

There are analogue cameras with more than 2 cameras. They are used for photography and artistic creation, not for robotics.

What is separate film frame?

Film frame is where the film is located on a film camera.

A film frame on a film camera equals to a sensor (CCD or CMOS) on a digital camera.

This definition of stereo cameras also include stereo film cameras. But they are not used in robotics.

Supplement material

• Computing Systems for Autonomous Driving: State-of-the-Art and Challenges
• How Self-driving Cars Work: Sensor Systems
• Tomorrow’s car silicon brain, how is it made?

---

Origin: Dr. Chris Lu (Homepage)
Translate + Edit: YangSier (Homepage)

🍀碎碎念🍀
Hello米娜桑，这里是英国留学中的杨丝儿。我的博客的关键词集中在编程、算法、机器人、人工智能、数学等等，点个关注吧，持续高质量输出中。