The combination of visual and inertial sensors for state estimation has recently found wide echo in the robotics community, especially in the aerial robotics field, due to the lightweight and complementary characteristics of the sensors data. However, most state estimation systems based on visual-inertial sensing suffer from severe processor requirements, which in many cases make them impractical. In this paper, we propose a simple, low-cost and high rate method for state estimation enabling autonomous flight of micro aerial vehicles, which presents a low computational burden. The proposed state estimator fuses observations from an inertial measurement unit, an optical flow smart camera and a time-of-flight range sensor. The smart camera provides optical flow measurements up to a rate of 200 Hz, avoiding the computational bottleneck to the main processor produced by all image processing requirements. To the best of our knowledge, this is the first example of extending the use of these smart cameras from hovering-like motions to odometry estimation, producing estimates that are usable during flight times of several minutes. In order to validate and defend the simplest algorithmic solution, we investigate the performances of two Kalman filters, in the extended and error-state flavors, alongside with a large number of algorithm modifications defended in earlier literature on visual-inertial odometry, showing that their impact on filter performance is minimal. To close the control loop, a non-linear controller operating in the special Euclidean group SE(3) is able to drive, based on the estimated vehicle’s state, a quadrotor platform in 3D space guaranteeing the asymptotic stability of 3D position and heading. All the estimation and control tasks are solved on board and in real time on a limited computational unit. The proposed approach is validated through simulations and experimental results, which include comparisons with ground-truth data provided by a motion capture system. For the benefit of the community, we make the source code public.
The final publication is available at link.springer.com
Santamaria, A.; Grosch, P.; Lippiello, V.; Solá, J.; Andrade-Cetto, J. IEEE-ASME transactions on mechatronics Vol. 22, num. 4, p. 1610-1621 DOI: 10.1109/TMECH.2017.2682283 Data de publicació: 2017-08-01 Article en revista
This paper addresses the problem of autonomous servoing an unmanned redundant aerial manipulator using computer vision. The overactuation of the system is exploited by means of a hierarchical control law, which allows to prioritize several tasks during flight. We propose a safety-related primary task to avoid possible collisions. As a secondary task, we present an uncalibrated image-based visual servo strategy to drive the arm end-effector to a desired position and orientation by using a camera attached to it. In contrast to the previous visual servo approaches, a known value of camera focal length is not strictly required. To further improve flight behavior, we hierarchically add one task to reduce dynamic effects by vertically aligning the arm center of gravity to the multirotor gravitational vector, and another one that keeps the arm close to a desired configuration of high manipulability and avoiding arm joint limits. The performance of the hierarchical control law, with and without activation of each of the tasks, is shown in simulations and in real experiments confirming the viability of such prioritized control scheme for aerial manipulation.
We address in this paper the problem of loop closure detection for laser-based simultaneous localization and mapping (SLAM) of very large areas. Consistent with the state of the art, the map is encoded as a graph of poses, and to cope with very large mapping capabilities, loop closures are asserted by comparing the features extracted from a query laser scan against a previously acquired corpus of scan features using a bag-ofwords (BoW) scheme. Two contributions are here presented. First, to benefit from the graph topology, feature frequency scores in the BoW are computed not only for each individual scan but also from neighboring scans in the SLAM graph. This has the effect of enforcing neighbor relational information during document matching. Secondly, a weak geometric check that takes into account feature ordering and occlusions is introduced that substantially improves loop closure detection performance. The two contributions are evaluated both separately and jointly on four common SLAM datasets, and are shown to improve the state-of-the-art performance both in terms of precision and recall in most of the cases. Moreover, our current implementation is designed to work at nearly frame rate, allowing loop closure query resolution at nearly 22 Hz for the best case scenario and 2 Hz for the worst case scenario.
Vaquero, V.; del Pino, I.; Moreno-Noguer, F.; Solá, J.; Sanfeliu, A.; Andrade-Cetto, J. European Conference on Mobile Robots p. 1-7 DOI: 10.1109/ECMR.2017.8098657 Data de presentació: 2017 Presentació treball a congrés
Vehicle detection and tracking is a core ingredient for developing autonomous driving applications in urban scenarios. Recent image-based Deep Learning (DL) techniques are obtaining breakthrough results in these perceptive tasks. However, DL research has not yet advanced much towards processing 3D point clouds from lidar range-finders. These sensors are very common in autonomous vehicles since, despite not providing as semantically rich information as images, their performance is more robust under harsh weather conditions than vision sensors. In this paper we present a full vehicle detection and tracking system that works with 3D lidar information only. Our detection step uses a Convolutional Neural Network (CNN) that receives as input a featured representation of the 3D information provided by a Velodyne HDL-64 sensor and returns a per-point classification of whether it belongs to a vehicle or not. The classified point cloud is then geometrically processed to generate observations for a multi-object tracking system implemented via a number of Multi-Hypothesis Extended Kalman Filters (MH-EKF) that estimate the position and velocity of the surrounding vehicles. The system is thoroughly evaluated on the KITTI tracking dataset, and we show the performance boost provided by our CNN-based vehicle detector over a standard geometric approach. Our lidar-based approach uses about a 4% of the data needed for an image-based detector with similarly competitive results.
del Pino, I.; Vaquero, V.; Massini, B.; Solá, J.; Moreno-Noguer, F.; Sanfeliu, A.; Andrade-Cetto, J. Iberian Robotics Conference p. 287-298 DOI: 10.1007/978-3-319-70833-1_24 Data de presentació: 2017 Presentació treball a congrés
Vehicle detection and tracking in real scenarios are key com- ponents to develop assisted and autonomous driving systems. Lidar sen- sors are specially suitable for this task, as they bring robustness to harsh weather conditions while providing accurate spatial information. How- ever, the resolution provided by point cloud data is very scarce in com- parison to camera images. In this work we explore the possibilities of Deep Learning (DL) methodologies applied to low resolution 3D lidar sensors such as the Velodyne VLP-16 (PUCK), in the context of vehicle detection and tracking. For this purpose we developed a lidar-based sys- tem that uses a Convolutional Neural Network (CNN), to perform point- wise vehicle detection using PUCK data, and Multi-Hypothesis Extended Kalman Filters (MH-EKF), to estimate the actual position and veloci- ties of the detected vehicles. Comparative studies between the proposed lower resolution (VLP-16) tracking system and a high-end system, using Velodyne HDL-64, were carried out on the Kitti Tracking Benchmark dataset. Moreover, to analyze the influence of the CNN-based vehicle detection approach, comparisons were also performed with respect to the geometric-only detector. The results demonstrate that the proposed low resolution Deep Learning architecture is able to successfully accom- plish the vehicle detection task, outperforming the geometric baseline approach. Moreover, it has been observed that our system achieves a similar tracking performance to the high-end HDL-64 sensor at close range. On the other hand, at long range, detection is limited to half the distance of the higher-end sensor.
The final publication is available at link.springer.com
Corominas, A.; Vallve, J.; Solá, J.; Flores, I.; Andrade-Cetto, J. IEEE International Conference on Robotics and Automation p. 3161-3166 DOI: 10.1109/ICRA.2016.7487484 Data de presentació: 2016-06 Presentació treball a congrés
Localization is the key perceptual process closing the loop of autonomous navigation, allowing self-driving vehicles to operate in a deliberate way. To ensure robust localization, autonomous vehicles have to implement redundant estimation processes, ideally independent in terms of the underlying physics behind sensing principles. This paper presents a stereo radar odometry system, which can be used as such a redundant system, complementary to other odometry estimation processes, providing robustness for long-term operability. The presented work is novel with respect to previously published methods in that it contains: (i) a detailed formulation of the Doppler error and its associated uncertainty; (ii) an observability analysis that gives the minimal conditions to infer a 2D twist from radar readings; and (iii) a numerical analysis for optimal vehicle sensor placement. Experimental results are also detailed that validate the theoretical insights.
Event-based cameras have an incredible potential in real-time and real-world robotics. They would enable more efficient algorithms in applications where high demanding requirements, such as rapid dynamic motion and high dynamic range, make standard cameras run into problems rapid dynamic motion and high dynamic range. While traditional cameras are based in the frame-base paradigm - a shutter captures a certain amount of pictures per second -, the bio-inspired event cameras have pixels that respond independently to the change of log-intensity generating asynchronous events. An special appeal for this type of cameras is their low band-width, since the stream of events contain all the information getting rid of the redundancy. This sensors that mimic some properties of the human retina has microseconds latency and 120 dB dynamic range (in contrast to the 60 dB of the standard cameras).
However, the current impact of the event cameras has been tiny due to the necessity of completely new algorithm, there is no global measurement of the intensity which would allow the use of current methods. The fact that an event corresponds to an asynchronous local intensity difference turns out to be a challenging problem if one wants to recover the motion as well as the scene. This article tries to illustrate the several problems that are needed to face when dealing with this problem and some of the different approaches taken.
First of all, we will explain the generative model of the event camera and the preliminaries, followed by the different approaches. Finally will the conclusions and a glossary of the code.
The lack of depth information in camera images has triggered much work on their use for localization and mapping in robotics. In particular, specific landmark parametrizations that isolate the unknown depth in one variable, and that allows to handle the associated large uncertainties have been proposed. Recently, an innovative parametrization (Parallax Angle) has shown to outperform the others in the context of a Bundle Adjustment approach. This paper investigates the way to exploit this parametrization in an incremental graph-based SLAM approach, in a robotics context in which motions measures can be incorporated in the overall estimation. It presents the factors required to initialize landmarks and manage their observations. Simulation results show that the proposed algorithms are able to incrementally incorporate observations, and a discussion analyzes how the incremental updates on ISAM2 are affected by these new factors.
Santamaria, A.; Solá, J.; Andrade-Cetto, J. IEEE/RSJ International Conference on Intelligent Robots and Systems p. 1864-1871 DOI: 10.1109/IROS.2015.7353621 Data de presentació: 2015 Presentació treball a congrés
This paper develops a new method for 3D, high rate vehicle state estimation, specially designed for free-flying Micro Aerial Vehicles (MAVs). We fuse observations from inertial and optical flow low-cost measurement units, and extend the current use of this optical sensors from hovering purposes to odometry estimation. Two Kalman filters, with its extended and error-state versions, are developed, and benchmarked alongside a large number of algorithm variations, using both simulations and real experiments with precise ground-truth. In contrast to state-of-the-art visual-inertial odometry methods, the proposed solution does not require image processing in the main CPU. Instead, the data correction is done taking advantage of the recently appeared optical flow sensors, which directly provide metric information about the MAV motion. We hence reduce the computational load of the main processor unit, and obtain an accurate estimation of the vehicle state at a high update rate.
The paper has supplementary downloadable material available at http://ieeexplore.ieee.org, provided by the authors. The material includes a video of the state estimation presented in the paper.