Sensor Fusion Methods for UAVs in GPS-denied Environments

A redundant flight control and navigation system for drones that ensures high safety and reliability through dual-redundancy architecture. The system comprises multiple sensors, a data fusion and filtering unit, a sensor information determination system, a flight control computer, and a control information determination system, all of which operate in parallel to provide fault-tolerant operation. The system can also interface with multiple actuators to execute control decisions.

Source

Autonomous drone system for exploration and reconnaissance in unknown environments. The system allows the drone to autonomously fly, map the environment, locate targets, avoid obstacles, and return home. The drone acquires data from cameras, lidar, and IMU to estimate pose, recognize targets, and generate a 3D map. It plans safe paths using ray-casting and sensor fusion. The drone applies the path to fly autonomously. This allows it to explore unknown areas, accurately locate targets, avoid obstacles, and return home.

Source

Controlling a UAV's flight altitude and posture in environments where satellite signals are poor or blocked, like flying under bridges, by using a secondary barometer to compensate for wind-induced air pressure changes. The UAV's onboard barometer provides the initial air pressure reading. This is synchronized with an external reference barometer's reading and recalculated to account for any wind effects. The compensated air pressure is then fused with other sensor data to determine the target altitude and posture. This allows accurate flight control even when satellite signals are unavailable.

Source

Facilitating managing of paths for unmanned vehicles using a hierarchical multi-drone/sensor platform with information lateralization and federated path planning. The platform involves multiple drones, ground robots, and sensors with complementary functions acquiring heterogeneous information at different resolutions. This information is integrated and fed back to adjust planned paths for missions. The platform is modeled after brain lateralization, where drones/sensors have specialized roles like human hemispheres.

Source

A method for calibrating a camera and lidar sensor using a calibration board with known patterns and reflection areas. The method determines the intrinsic camera parameters and then uses the lidar sensor's projected laser lines to detect the calibration board's pose. The calibration board's known patterns and reflection areas enable simultaneous calibration of both sensors without requiring prior camera calibration.

Source

An autonomous control system for unmanned aerial vehicles (UAVs) that uses machine learning to detect landing spaces from camera images. The system generates pixel-level labels for the images, identifying landing spaces, occupied spaces, and non-landing spaces, and uses these labels to determine the UAV's relative position and control its movements.

Source

An autonomous mobile object's self-position estimation method is dynamically switched between two modes based on the object's state. When the object is moving, it uses a successive estimation method that integrates internal sensor data. When the object is stopped, it switches to a discrete estimation method that uses external sensor data, enabling more accurate positioning in environments where GPS is unreliable.

Source

A sensor fusion system for vehicle control applications that integrates visible light cameras with shortwave infrared (SWIR) and longwave infrared (LWIR) cameras to enhance object detection and recognition capabilities in adverse weather conditions. The system determines environmental conditions, such as low light or glare, and automatically switches to alternative sensor modalities to maintain reliable object detection and tracking. The fused images from multiple sensors are processed to generate a comprehensive understanding of the vehicle's surroundings, enabling proactive hazard detection and prediction.

Source

Autonomous drone navigation control for stable flight in extreme wind conditions using onboard sensors and physics-based modeling. The drone estimates thrust and drag forces from rotor geometry, measures wind, and uses feedforward flight control to compensate for winds. It also has a relationship between RPM and throttle input for rotors. This allows accurate flight path following in severe winds without relying on complex aerodynamic models.

Source

A navigation system for autonomous vehicles that integrates optical sensor data with motion sensor data to provide accurate positioning in environments where traditional GNSS systems are unreliable. The system uses a nonlinear state estimation technique that incorporates a measurement model for optical samples, enabling direct integration of optical data with motion sensor data to generate a precise navigation solution. The system can operate in environments with limited GNSS visibility, such as dense urban areas, and can also be used in pedestrian navigation applications.

Source

An information processing device, method, and system for accurate calibration of multiple sensors, including cameras and lidars, regardless of sensor type or range. The system uses a third sensor to measure 3D information of calibration objects placed within the measurement ranges of the first and second sensors, and calculates the relative positions and orientations of the sensors based on the third sensor data and sensor data from the first and second sensors.

Source

A lightweight pose estimation system for autonomous vehicles that determines vehicle position and orientation using a combination of inertial, kinematic, and odometry sensors. The system generates a pose value by integrating sensor readings and can adjust measurements based on observed noise. It enables efficient and real-time pose estimation for autonomous maneuvers, particularly in emergency braking applications.

Source

Calibration of an environment sensor system of an infrastructure device, such as a camera system or lidar system, for autonomous vehicle navigation. The system identifies two independent marking elements in the sensor's field of view, determines their relative pose, and compares it to a predetermined target value. Based on the comparison result, the system corrects the sensor's pose information in a reference coordinate system.

Source

A high-reliability and high-precision navigation and positioning method for drones in GPS-denied environments. The method combines inertial sensor data with binocular camera images and ranging radar measurements to achieve accurate pose estimation. A nonlinear graph optimization algorithm based on a sliding window is used to fuse the sensor data and obtain high-precision pose estimates. The system also includes a loop closure detection module that enables four-degree-of-freedom pose graph optimization when the drone revisits a previously mapped location. The optimized pose data is then packaged into a pseudo-GPS signal that is input to the drone's flight controller for positioning and route planning.

Source

A method for determining the position of an unmanned aerial system (UAS) using a network of radar nodes. Each node measures the slant distance and elevation angle to the UAS, and transmits its position and azimuthal bounds to neighboring nodes. By combining these measurements, the nodes can solve for the UAS position without explicit measurement of its azimuthal position, providing a unique solution without ambiguity.

Source

A method for determining a position estimate of a mobile device using visual positioning. The method includes obtaining sensor information, detecting identifiable features in the sensor information, determining a range to the features, obtaining coarse map information, and determining the position estimate based on the range and map information. The method can utilize various sensors, including cameras, lidar, and radar, and can leverage coarse map data from remote servers.

Source

Autonomous navigation system for unmanned aerial vehicles (UAVs) that enables reliable flight in environments where traditional navigation sensors are unreliable. The system uses a combination of GNSS location signals, inertial measurement unit (IMU) data from accelerometers and gyroscopes, and quadcopter aerodynamics to determine the UAV's position, velocity, and orientation. By leveraging the unique characteristics of quadcopters, the system can maintain stable flight and navigate through challenging environments without relying on cameras, compasses, or magnetometers.

Source

A method for radar-to-lidar calibration in autonomous vehicles that eliminates the need for specialized calibration targets. The method uses point cloud registration and entropy minimization to align radar and lidar point clouds gathered from different vehicle poses, enabling calibration in unstructured environments. The process aggregates radar and lidar point clouds using vehicle odometry and SLAM data, and then minimizes entropy over multiple drive segments to achieve accurate calibration.

Source

Automated extrinsic calibration system for lidars, cameras, radars, and ultrasonic sensors on vehicles and robots, comprising a turntable system, calibration target system, and distributed imaging systems. The system enables precise calibration of sensor systems through automated scanning, target configuration, and image capture, eliminating the need for manual measurement and technician expertise.

Source

An object ranging apparatus for improving accuracy and stability of object ranging in autonomous vehicles, comprising: an object recognition unit; a first distance estimation unit using depth estimation; a second distance estimation unit using motion parallax; and a combining unit that combines the results of the two estimation methods based on factors such as steering wheel angle and acceleration.

Source

Determining the location and/or navigation path of a moving platform. The method includes using a vision system on a moving platform to identify a region of interest, classifying objects within the region of interest, directing random-access LIDAR to ping one or more of the classified objects, and locating the platform in three dimensions using data from the vision system and LIDAR.

Source

System for validating navigation solution outputs by monitoring sensor integrity and automatically eliminating faulty data. The system continuously assesses the accuracy and reliability of multiple navigation sensors, including GNSS, accelerometers, gyroscopes, and others, and dynamically adjusts the navigation solution based on sensor performance. When sensor integrity is compromised, the system automatically rejects the affected data to ensure reliable position and timing outputs.

Source

A multi-modal sensing approach for environmental mapping in autonomous systems that adapts data capture to object identification. The system uses a hierarchical structure to mimic human visual processing, initially capturing low-resolution data over a wide field of view and then selectively applying high-resolution sensing to areas of interest. Object recognition algorithms inform the distribution of sensing resources, enabling efficient data collection and object classification.

Source

A drone system that enables advanced autonomous flight capabilities through a novel interface device that connects an onboard computer to the flight controller and inertial measurement unit. The interface device performs real-time data processing and frequency conversion to enable efficient communication between the onboard computer and flight controller, while also supporting the integration of additional sensors. This enables the development of advanced autonomous flight modes, including AI-powered control systems that can learn from human pilots.

Source

Optimizing sensor configuration for autonomous vehicles by comparing target and capability specification maps to identify regions where sensors are insufficient for a task. The method involves determining a target specification map indicating sensor parameters needed for a task, and a capability map showing what sensors can provide. Comparing the maps identifies regions where sensors fall short, allowing modifications to improve coverage. This allows evaluating sensor sufficiency before deployment.

Source

A method of operating a vehicle that improves navigation accuracy by combining data from multiple sources with statistical weights based on their reliability. The method collects navigation parameters from sensors, GPS, and inertial systems, determines their uncertainties, and assigns weights to each parameter based on its reliability. A navigational solution is then formed by blending the weighted parameters using a Kalman filter, providing an optimized navigation solution with overall uncertainty estimates.

Source

Method for estimating camera pose and scale that achieves both high speed and high accuracy by incorporating prior knowledge of rotation and scale into the estimation process. The method uses prior parameters derived from inertial sensor measurements to bias the cost function, accelerating the estimation process and improving accuracy compared to conventional approaches. The prior parameters selectively influence the cost function through rotation and scale weights, which are adjusted based on sensor noise. The method determines the camera pose and scale by optimizing the similarity transformation that minimizes the cost function below a threshold.

Source

Map generation apparatus for vehicle positioning that extracts feature points from sensor data, generates a map incorporating these points, recognizes landmarks, determines their importance, and adjusts the map's feature point density based on landmark significance.

Source

Indoor autonomous aerial system that enables efficient and precise navigation of micro aerial vehicles (MAVs) using machine learning algorithms. The system combines MAVs with image capture capabilities to autonomously navigate to desired locations within a deployable space, leveraging input from visual data to guide the MAV. By analyzing visual patterns and features, the system can extract valuable location data and present it to the MAV controller, enabling precise navigation and sign presentation applications. The system also incorporates battery level monitoring through onboard sensors, eliminating the need for traditional GPS.

Source

Automatic calibration of vehicular sensor systems using visual data from cameras and other sensors. The system detects stationary objects, such as traffic signs, and tracks their positions across multiple frames to estimate camera pose and sensor offsets. It filters object detection tracks based on quality metrics and jointly computes camera calibration parameters and object locations using the filtered tracks. The system can also incorporate data from other sensors, such as GPS, IMU, and wheel odometry, to improve accuracy and robustness.

Source

A 3D position and orientation calculation and robotic application structure that enables precise and repeatable robotic operations by combining inertial measurement unit (IMU) and string-encoder position sensors. The system records the movements of a portable recording apparatus using the IMU and string-encoder sensors, and then applies these movements to a robot for unmanned operation. The system provides high accuracy and precision, eliminates human error, and enables the creation of precise robotic programs for repetitive tasks.

Source

Computing flight controls for safe landing of unmanned aerial vehicles (UAVs) using sensor data fusion. The system involves a UAV with a multi-sensor suite including radar, cameras, AHRS, and GPS. Before descent, the UAV receives landing confirmation from a service. During descent, the sensors cross-check to confirm clearance, alignment, altitude, descent rate, obstacles, and beacon visibility. If conflicts arise, the UAV stops. If all checks pass, the UAV lands. This comprehensive sensor fusion enables safe and reliable landing by coordinating multiple sensors and data sources.

Source

A device for more appropriate action control in autonomous mobile objects like drones. It selects the best self-position estimation method from multiple options based on indexes like GPS accuracy. This allows optimized positioning for different situations. The selected positioning is used to obtain the mobile object's state, and actions are then controlled based on that state using predefined plans.

Source

A collaborative localization system for multiple platforms that enables accurate positioning in GPS-denied environments through real-time, distributed information sharing. The system uses visual feature matching and inertial measurement data to determine platform poses, with each device contributing to a shared map of geo-referenced visual features. When a device encounters a new feature, it uses IMU data to estimate its position, while matching features to the shared map enables precise localization. The system's distributed architecture enables organic collaboration without requiring special behaviors, making it suitable for applications where multiple platforms operate independently.

Source

System for accurate localization and tracking of unmanned aerial vehicles (UAVs) using a mobile ground vehicle. The system has a wheeled chassis with pose sensors that track UAV positions. A processor calculates UAV poses relative to the chassis using the sensor data. This allows UAV navigation even in nested environments where fixed sensors can't see them. The chassis can also provide extended power and computation resources for the UAVs.

Source

Autonomous aerial drone that enables self-sustaining flight capabilities through advanced AI-driven navigation and obstacle avoidance. The drone integrates multiple sensors, including cameras and sensors, to detect and respond to environmental changes, while its onboard AI system continuously optimizes flight paths and collision avoidance strategies. The system enables autonomous flight, route planning, and real-time obstacle avoidance, eliminating the need for human operator intervention.

Source

A method for determining a vehicle's position in urban environments using a combination of a last known position, stable landmarks, and a reference map. The method identifies stable landmarks by analyzing multiple images of the area over time, and uses these landmarks in conjunction with GNSS and IMU data to accurately determine the vehicle's current position when GNSS accuracy falls below a threshold.

Source

A radar-based positioning system for aircraft that provides accurate, high-integrity, and continuous positioning in all weather conditions. The system uses multiple antennas with multiple receivers to measure range and velocity from reflected radar signals, enabling determination of position, velocity, and orientation in three dimensions. The system's radar componentry operates independently of GPS, providing a redundant and reliable positioning solution.

Source

A vision-based navigation system for aircraft that leverages onboard sensors and databases to output accurate latitude-longitude coordinates without GPS. The system uses computer vision techniques to analyze images from visible light, night vision, and infrared cameras, as well as synthetic aperture radar, to determine aircraft position and orientation. It references georeferenced databases of high-resolution satellite images, DTED, and NIR data to achieve GPS-like precision.

Source

A method and system for providing odometry information for autonomous vehicles using radar sensors. The system employs a deep neural network to process radar measurements and detect static objects, which are then used to estimate the vehicle's pose and motion. The radar sensor can be a frequency-modulated continuous wave (FMCW) radar or a pulse-based radar, and the system can operate in various frequency bands, including 24 GHz and 77 GHz. The neural network-based approach enables accurate odometry estimation even in the absence of GPS signals, making it suitable for applications where GNSS is unreliable or unavailable.

Source

Synchronizing image capture and lidar scans in sensor systems like autonomous vehicles to provide temporally coordinated sensor information. The technique involves capturing high resolution images, low resolution images, and lidar scans simultaneously during a single scan interval. It uses a technique called correlated double sampling to remove defective pixel noise from the images. By synchronizing the sensor functions, it allows precise spatial correlation between the lidar point cloud and the images for better perception and sensor fusion.

Source

Decrating a physical state of a movable object to improve the operational safety margin/factor of unmanned vehicles. The determination includes obtaining a plurality of sets of sensor data from a plurality of sensing systems coupled to the movable object, the plurality of sets of sensor data comprising a distinct set of sensor data for each sensing system of the plurality of sensing systems; determining, for each sensing system of the plurality of sensing systems, a deviation between the corresponding set of sensor data and another set of sensor data for a different sensing system of the plurality of sensing systems; selecting one or more sets of sensor data from the plurality of sets of sensor data based at least in part on the deviations of the one or more sets of sensor data; and determining a physical state of the movable object based at least in part on the selected one or more sets of sensor data.

Source

Self-localization device which can be used for mapping a map and performing steering control of the mobile object using a result of detection of the position and the posture. The device includes an acquisition unit, an estimation unit, a key frame registering unit, a map reading unit, and a re-localization processing unit.

Source

Autonomous landing system for unmanned aerial vehicles (UAVs) that enables precise landing on moving ground vehicles (GVs) in GPS-denied environments. The system uses multiple monocular cameras to detect and track markers on the GVs, and employs a calibration phase to establish camera-marker relationships. During flight, the cameras generate pose estimates that are fused to provide accurate position and orientation data for the UAV's flight controller. The system enables autonomous takeoff, tracking, and landing of UAVs on moving GVs, with applications in surveillance, inspection, and logistics.

Source

An automatic driving object detection and positioning system that integrates stereo vision, inertial navigation, and satellite navigation to achieve high-accuracy positioning and obstacle detection. The system uses a direct binocular method to calculate 3D coordinates without feature extraction, and tightly couples the measurement data from the three sensors to correct inertial navigation errors and improve overall positioning accuracy.

Source

Precision localization for autonomous vehicles using signal analysis instead of GPS. The method involves detecting, tracking, and classifying distinctive landmarks from signals like images or LiDAR scans. By associating landmarks with locations, vehicles can determine their position without relying on GPS. This enables real-time, high-precision mapping and localization without GPS, which is useful for autonomous driving and other applications where GPS is unavailable. The landmark detection, tracking, classification, and parameter extraction are done onboard the vehicle. The landmark associations are stored remotely and retrieved to provide global position.

Source

Guidance system for unmanned aircraft that enables precise navigation along predetermined air routes, particularly in environments where GPS signals are unreliable or unavailable. The system employs a network of optical guidance lines and beams that are detected by onboard sensors, allowing the aircraft to maintain precise alignment and trajectory control. The system also includes a collision avoidance mechanism for managing multiple aircraft operating in the same airspace.

Source

A calibration system for vehicle sensors that uses a polyhedral target to simultaneously calibrate multiple sensor types, including cameras and LIDAR systems, by capturing data from multiple angles and mapping sensor representations to a common coordinate system. The system enables accurate sensor calibration despite manufacturing variations and environmental degradation, ensuring reliable sensor data for autonomous vehicles and sensor-equipped vehicles.

Source

A method for determining the position of a moveable element in a vehicle, comprising acquiring position-related data from multiple sensors, selecting the most accurate data based on predetermined criteria, and fusing the selected data to determine the element's position. The method iteratively refines the position estimate as more accurate data becomes available, enabling precise tracking of elements like cameras and steering columns despite movement and thermal expansion.

Source

A method and apparatus for generating an optimal path for an unmanned aerial vehicle (UAV) that avoids obstacles detected by a sensor during autonomous flight. The method uses a particle swarm optimization algorithm to determine a smooth trajectory connecting waypoints, with the algorithm iteratively updating particle positions and velocity vectors based on fitness function values. The optimal path is generated by establishing an integrated path plan that takes into account both waypoints and the connecting trajectory, with the algorithm converging to a solution that balances exploration and exploitation.

Source