Robots, traditionally confined into factories, are nowadays moving to domestic and assistive environments, where they need to deal with complex object shapes, deformable materials, and pose uncertainties at human pace. To attain quick 3D perception, new cameras delivering registered depth and intensity images at a high frame rate hold a lot of promise, and therefore many robotics researchers are now experimenting with structured-light RGBD and Time-of-Flight (ToF) cameras. In this paper both technologies are critically compared to help researchers to evaluate their use in real robots. The focus is on 3D perception at close distances for different types of objects that may be handled by a robot in a human environment. We review three robotics applications. The analysis of several performance aspects indicates the complementarity of the two camera types, since the user-friendliness and higher resolution of RGBD cameras is counterbalanced by the capability of ToF cameras to operate outdoors and perceive details.
Kazmi, Wajahat; Foix Salmeron, Sergi; Alenyà Ribas, Guillem; Andersen, Hans Jørgen ISPRS journal of photogrammetry and remote sensing Vol. 88, p. 128-146 DOI: 10.1016/j.isprsjprs.2013.11.012 Date of publication: 2014-02-01 Journal article
In this article we analyze the response of Time-of-Flight (ToF) cameras (active sensors) for close range imaging under three different illumination conditions and compare the results with stereo vision (passive) sensors. ToF cameras are sensitive to ambient light and have low resolution but deliver high frame rate accurate depth data under suitable conditions. We introduce metrics for performance evaluation over a small region of interest. Based on these metrics, we analyze and compare depth imaging of leaf under indoor (room) and outdoor (shadow and sunlight) conditions by varying exposure times of the sensors. Performance of three different ToF cameras (PMD CamBoard, PMD CamCube and SwissRanger SR4000) is compared against selected stereo-correspondence algorithms (local correlation and graph cuts). PMD CamCube has better cancelation of sunlight, followed by CamBoard, while SwissRanger SR4000 performs poorly under sunlight. Stereo vision is comparatively more robust to ambient illumination and provides high resolution depth data but is constrained by texture of the object along with computational efficiency. Graph cut based stereo correspondence algorithm can better retrieve the shape of the leaves but is computationally much more expensive as compared to local correlation. Finally, we propose a method to increase the dynamic range of ToF cameras for a scene involving both shadow and sunlight exposures at the same time by taking advantage of camera flags (PMD) or confidence matrix (SwissRanger). (C) 2013 International Society for Photogrammetly and Remote Sensing, Inc. (ISPRS) Published by Elsevier B.V. All rights reserved.
ToF cameras are now a mature technology that is widely being adopted to provide sensory input to robotic applications. Depending on the nature of the objects to be perceived and the viewing distance, we distinguish two groups of applications: those requiring to capture the whole scene and those centered on an object. It will be demonstrated that it is in this last group of applications, in which the robot has to locate and possibly manipulate an object, where the distinctive characteristics of ToF cameras can be better exploited.
After presenting the physical sensor features and the calibration requirements of such cameras, we review some representative works highlighting for each one which of the distinctive ToF characteristics have been more essential. Even if at low resolution, the acquisition of 3D images at frame-rate is one of the most important features, as it enables quick background/foreground segmentation. A common use is in combination with classical color cameras. We present three developed applications, using a mobile robot and a robotic arm, to exemplify with real images some of the stated advantages.
A common strategy to teach a robot certain skills involves demonstration. While the demonstrations are best made by directly manipulating the robot, in hazardous conditions the only choice is teleoperation. Even though haptic devices offer fairly good results, using natural movements would give a better feeling of control.
Leap Motion sensor (Leap Motion Inc., USA) is a device that detects hands and it can be used to control the robot arm in a more natural way. In this work, a system that will control the WAM arm (Barret Technology Inc., USA) using the Leap Motion sensor will be explained. Later this system will be tested in order to grasp a polo shirt which will be used to train grasping points on the shirt.
Robotic handling of textile objects in household environments is an emerging application that has recently received considerable attention thanks to the development of domestic robots. Most current approaches follow a multiple re-grasp strategy for this purpose, in which clothes are sequentially grasped from different points until one of them yields a desired configuration.
In this work we propose a vision-based method, built on the Bag of Visual Words approach, that combines appearance and 3D information to detect parts suitable for grasping in clothes, even when they are highly wrinkled.
We also contribute a new, annotated, garment part dataset that can be used for benchmarking classification, part detection, and segmentation algorithms. The dataset is used to evaluate our approach and several state-of-the-art 3D descriptors for the task of garment part detection. Results indicate that appearance is a reliable source of information, but that augmenting it with 3D information can help the method perform better with new clothing items.
Martínez Martínez, David; Alenyà Ribas, Guillem; Torras, Carme IROS Machine Learning in Planning and Control of Robot Motion Workshop p. 1-6 Presentation's date: 2014 Presentation of work at congresses
Task learning in robotics is a time-consuming process, and model-based reinforcement learning algorithms have been proposed to learn with just a small amount of experiences. However, reducing the number of experiences used to learn implies that the algorithm may overlook crucial actions required to get an optimal behavior. For example, a robot may learn simple policies that have a high risk of not reaching the goal because they often fall into dead-ends.
We propose a new method that allows the robot to reason about dead-ends and their causes. Analyzing its current model and experiences, the robot will hypothesize the possible causes for the dead-end, and identify the actions that may cause it, marking them as dangerous. Afterwards, whenever a dangerous action is included into a plan which has a high risk of leading to a dead-end, the special action request teacher confirmation will be triggered by the robot to actively confirm with a teacher that the planned risky action should be executed.
This method permits learning safer policies with the addition of just a few teacher demonstration requests. Experimental validation of the approach is provided in two different scenarios: a robotic assembly task and a domain from the international planning competition. Our approach gets success ratios very close to 1 in problems where previous approaches had high probabilities of reaching dead-ends
Alenyà Ribas, Guillem; Rivero Partida, José Luís; Rull Sanahuja, Aleix; Grosch Obregon, Patrick John; Hernandez Juan, Sergi International Conference on Computer Supported Education p. 213-220 Presentation's date: 2014 Presentation of work at congresses
The HumanoidLab is a more than 5 year old activity aimed to use educational robots to approach students to our Research Centre. Different commercial educative humanoid platforms have been used to introduce students to different aspects of robotics using projects and offering guidance and assistance. About 40 students have performed small mechanics, electronics or programming projects that are used to improve the robots by adding features. Robotics competitions are used as a motivation tool. A two weeks course was started that has received 80 undergraduate students, and more than 100 secondary school students in a short version. The experience has been very positive for students and for the institution: some of these students have performed their scholar projects and research in robotics and continue enrolled in the robotics field, and some of them are currently in research groups at IRI.
Husain, Syed Farzad; Colomé Figueras, Adrià; Dellen, Babette Karla Margarete; Alenyà Ribas, Guillem; Torras, Carme IEEE International Conference on Robotics and Automation p. 2617-2622 DOI: 10.1109/ICRA.2014.6907234 Presentation's date: 2014 Presentation of work at congresses
In this paper we present an automated system that is able to track and grasp a moving object within the workspace of a manipulator using range images acquired with a Microsoft Kinect sensor. Realtime tracking is achieved by a geometric particle filter on the affine group. Based on the tracked output, the pose of a 7-DoF WAM robotic arm is continuously updated using dynamic motor primitives until a distance measure between the tracked object and the gripper mounted on the arm is below a threshold. Then, it closes its three fingers and grasps the object. The tracker works in real-time and is robust to noise and partial occlusions. Using only the depth data makes our tracker independent of texture which is one of the key design goals in our approach. An experimental evaluation is provided along with a comparison of the proposed tracker with state-of-the-art approaches, including the OpenNI-tracker. The developed system is integrated with ROS and made available as part of IRI's ROS stack.
Supervision of long-lasting extensive botanic
experiments is a promising robotic application that some recent technological advances have made feasible. Plant modeling for this application has strong demands, particularly in what concerns three-dimensional (3-D) information gathering and speed.
Perform the setup and prepare WAM robots to use the the motion planning and obstacle avoid-
ance facilities present at the MoveIt! framework. In particular, determine the configurations
and API function calls to set the target of the robot arm with joint state and add collision
objects programatically, without using the graphical interface. Supporting page with videos:
A method to perform cleaning tasks is presented where
a robot manipulator autonomously grasps a textile and
uses different dragging actions to clean a surface. Ac-
tions are imprecise, and probabilistic planning is used
to select the best sequence of actions. The character-
ization of such actions is complex because the initial
autonomous grasp of the textile introduces differences
in the initial conditions that change the efficacy of the
robot cleaning actions. We demonstrate that the action
outcome probabilities can be learned very fast while the
task is being executed, so as to progressively improve
robot performance. The learner adds only a little over-
head to the system compared to the improvements ob-
tained. Experiments with a real robot show that the most
effective plan varies depending on the initial grasp, and
that plans become better after only a few learning itera-
Colomé Figueras, Adrià; Pardo Ayala, Diego Esteban; Alenyà Ribas, Guillem; Torras, Carme IEEE International Conference on Robotics and Automation p. 3535-3540 DOI: 10.1109/ICRA.2013.6631072 Presentation's date: 2013 Presentation of work at congresses
This paper presents a method to estimate external forces exerted on a manipulator, avoiding the use of a sensor. The method is based on task-oriented dynamics model learning and a robust disturbance state observer. The combination of both leads to an efficient torque observer that can be incorporated to any control scheme. The use of a learned based approach avoids the need of analytical models of friction or Coriolis dynamics effects.
Colomé Figueras, Adrià; Alenyà Ribas, Guillem; Torras, Carme ICRA Workshop on Novel Methods for Learning and Optimization of Control Policies and Trajectories for Robotics Presentation's date: 2013 Presentation of work at congresses
Dynamic Motor Primitives (DMP) are nowadays widely used as movement parametrization for learning trajectories, because of their linearity in the parameters, rescalation robustness and continuity. However, when learning a movement with DMP, where a set of gaussians distributed along the trajectory is used to approximate an acceleration excitation function, a very large number of gaussian approximations need to be performed. Adding them up for all joints yields too many parameters to be explored, thus requiring a prohibitive number of experiments/simulations to converge to a solution with an optimal (locally or globally) reward. We propose here two strategies to reduce this dimensionality: the first is to explore only the most significant directions in the parameter space, and the second is to add a reduced second set of gaussians that should only optimize the trajectory after fixing the gaussians that approximate the demonstrated movement.
Ramisa Ayats, Arnau; Alenyà Ribas, Guillem; Moreno Noguer, Francesc d'Assis; Torras, Carme IEEE/RSJ International Conference on Intelligent Robots and Systems p. 824-830 DOI: 10.1109/IROS.2013.6696446 Presentation's date: 2013 Presentation of work at congresses
Most current depth sensors provide 2.5D range images in which depth values are assigned to a rectangular 2D array. In this paper we take advantage of this structured information to build an efficient shape descriptor which is about two orders of magnitude faster than competing approaches, while showing similar performance in several tasks involving deformable object recognition. Given a 2D patch surrounding a point and its associated depth values, we build the descriptor for that point, based on the cumulative distances between their normals and a discrete set of normal directions. This processing is made very efficient using integral images, even allowing to compute descriptors for every range image pixel in a few seconds. The discriminative power of our descriptor, dubbed FINDDD, is evaluated in three different scenarios: recognition of specific cloth wrinkles, instance recognition from geometry alone, and detection of reliable and informed grasping points.
Foix Salmeron, Sergi; Kriegel, Simon; Fuchs, Stefan; Alenyà Ribas, Guillem; Torras, Carme International Conference on Advanced Concepts for Intelligent Vision Systems p. 36-47 DOI: 10.1007/978-3-642-33140-4_4 Presentation's date: 2012 Presentation of work at congresses
Active view planning for gathering data from an unexplored 3D complex scenario is a hard and still open problem in the computer vision community. In this paper, we present a general task-oriented approach based on an information-gain maximization that easily deals with such a problem. Our approach consists of ranking a given set of possible actions, based on their task-related gains, and then executing the best-ranked action to move the required sensor.
An example of how our approach behaves is demonstrated by applying it over 3D raw data for real-time volume modelling of complex-shaped objects. Our setting includes a calibrated 3D time-of-flight (ToF) camera mounted on a 7 degrees of freedom (DoF) robotic arm. Noise in the sensor data acquisition, which is too often ignored, is here explicitly taken into account by computing an uncertainty matrix for each point, and refining this matrix each time the point is seen again. Results show that, by always choosing the most informative view, a complete model of a 3D free-form object is acquired and also that our method achieves a good compromise between speed and precision.
Ramisa Ayats, Arnau; Alenyà Ribas, Guillem; Moreno Noguer, Francesc d'Assis; Torras, Carme IEEE International Conference on Robotics and Automation p. 1703-1708 DOI: 10.1109/ICRA.2012.6225045 Presentation's date: 2012 Presentation of work at congresses
Detecting grasping points is a key problem in cloth manipulation. Most current approaches follow a multiple regrasp
strategy for this purpose, in which clothes are sequentially grasped from different points until one of them yields to a
desired configuration. In this paper, by contrast, we circumvent the need for multiple re-graspings by building a robust detector that identifies the grasping points, generally in one single step,
even when clothes are highly wrinkled.
In order to handle the large variability a deformed cloth may have, we build a Bag of Features based detector that combines
appearance and 3D geometry features. An image is scanned using a sliding window with a linear classifier, and the candidate
windows are refined using a non-linear SVM and a “grasp goodness” criterion to select the best grasping point.
We demonstrate our approach detecting collars in deformed polo shirts, using a Kinect camera. Experimental results show
a good performance of the proposed method not only in identifying the same trained textile object part under severe deformations and occlusions, but also the corresponding part in other clothes, exhibiting a degree of generalization.
Monsó Purtí, Pol; Alenyà Ribas, Guillem; Torras, Carme IEEE/RSJ International Conference on Intelligent Robots and Systems p. 1324-1329 DOI: 10.1109/IROS.2012.6386011 Presentation's date: 2012 Presentation of work at congresses
Rigid object manipulation with robots has mainly
relied on precise, expensive models and deterministic sequences.
Given the great complexity of accurately modeling deformable
objects, their manipulation seems to call for a rather different
approach. This paper proposes a probabilistic planner, based
on a Partially Observable Markov Decision Process (POMDP),
targeted at reducing the inherent uncertainty of deformable
object sorting. It is shown that a small set of unreliable
actions and inaccurate perceptions suffices to accomplish the
task, provided faithful statistics on both of them are collected
beforehand. The planner has been applied to a clothes sorting
task in a real case context with a depth and color sensor and
a robotic arm. Experimental results show the promise of the
approach since more than 95% certainty of having isolated a
piece of clothing is reached in an average of four steps for quite
entangled initial clothing configurations.
Alenyà Ribas, Guillem; Ramisa Ayats, Arnau; Moreno Noguer, Francesc d'Assis; Torras, Carme IEEE International Conference on Robotics and Automation p. 1-6 Presentation's date: 2012 Presentation of work at congresses
Grasping highly deformable objects, like textiles,
is an emerging area of research that involves both percep-
tion and manipulation abilities. As new techniques appear,
it becomes essential to design strategies to compare them.
However, this is not an easy task, since the large state-space
of textile objects explodes when coupled with the variability
of grippers, robotic hands and robot arms performing the
manipulation task. This high variability makes it very difficult
to design experiments to evaluate the performance of a system
in a repeatable way and compare it to others. We propose
a framework to allow the comparison of different grasping
methods for textile objects.
Instead of measuring each component separately, we there-
fore propose a methodology to explicitly measure the vision-
manipulation correlation by taking into account the throughput
of the actions. Perceptions of deformable objects should be
grouped into different clusters, and the different grasping
actions available should be tested for each perception type to
obtain the action-perception success ratio. This characterization
potentially allows to compare very different systems in terms
widely useful actions
with the cost of performing each action. We will also show
that this categorization is useful in manipulation planning of
Rigual Aparici, Ferran; Ramisa Ayats, Arnau; Alenyà Ribas, Guillem; Torras, Carme International Conference of the Catalan Association for Artificial Intelligence p. 123-132 DOI: 10.3233/978-1-61499-139-7-123 Presentation's date: 2012 Presentation of work at congresses
In this work we address the problem of object detection for the purpose of object manipulation in a service robotics scenario. Several implementations of state-of-the-art object detection methods were tested, and the one with the best performance was selected. During the evaluation, three main practical limitations of current methods were identified in relation with long-range object detection, grasping point detection and automatic learning of new objects; and practical solutions are proposed for the last two. Finally, the complete pipeline is evaluated in a real grasping experiment.
Colomé Figueras, Adrià; Pardo Ayala, Diego Esteban; Alenyà Ribas, Guillem; Torras, Carme IROS Workshop Beyond Robot Grasping: Modern Approaches for Learning Dynamic Manipulation p. 1 Presentation's date: 2012 Presentation of work at congresses
Our current work on external force estimation without end-effector force sensor is resented.To verify if a grasp of a textile has been successful, the external wrench applied on the robot is computed online, with a state observer based on a LWPR  model of a task.
Alenyà Ribas, Guillem; Dellen, Babette Karla Margarete; Foix Salmeron, Sergi; Torras, Carme IROS Workshop on Agricultural Robotics: Enabling Safe, Efficient, Affordable Robots for Food Production (IROS AGROBOTICS) p. 1-6 Presentation's date: 2012 Presentation of work at congresses
We present a novel method for the robotized
probing of plant leaves using Time-of-Flight (ToF) sensors.
Plant images are segmented into surface patches by combining
a segmentation of the infrared intensity image, provided by
the ToF camera, with quadratic surface fitting using ToF
depth data. Leaf models are fitted to the boundaries of the
segments and used to determine probing points and to evaluate
the suitability of leaves for being sampled. The robustness of
the approach is evaluated by repeatedly placing an especially
adapted, robot-mounted spad meter on the probing points
which are extracted in an automatic manner. The number of
successful chlorophyll measurements is counted, and the total
time for processing the visual data and probing the plant with
the robot is measured for each trial. In case of failure, the
underlying causes are determined and reported, allowing a
better assessment of the applicability of the method in real
Pardo Ayala, Diego Esteban; Rozo Castañeda, Leonel; Alenyà Ribas, Guillem; Torras, Carme Workshop on Learning and Interaction in Haptic Robots p. 1-2 Presentation's date: 2012 Presentation of work at congresses
This work presents a probabilistic model for learning robot tasks from human demonstrations using kinesthetic teaching. The difference with respect to previous works is that a complete state of the robot is used to obtain a consistent representation of the dynamics of the task. The learning framework is based on hidden Markov models and Gaussian mixture regression, used for coding and reproducing the skills. Benefits of the proposed approach are shown in the execution of a simple self-crossing trajectory by a 7-DoF manipulator.
Kazmi, Wajahat; Alenyà Ribas, Guillem; Foix Salmeron, Sergi IEEE International Symposium on Robotic and Sensors Environments p. 192-197 DOI: 10.1109/ROSE.2012.6402615 Presentation's date: 2012 Presentation of work at congresses
In this article, we analyze the effects of ambient light on Time of Flight (ToF) depth imaging for a plant’s leaf in sunlight, shadow and room conditions. ToF imaging is sensitive to ambient light and we try to find the best possible integration
times (IT) for each condition. This is important in order to optimize camera calibration. Our analysis is based on several
statistical metrics estimated from the ToF data. We explain the estimation of the metrics and propose a method of predicting the deteriorating behavior of the data in each condition using camera flags. Finally, we also propose a method to improve the quality of a ToF image taken in a mixed condition having different ambient light exposures.
Simo Serra, Edgar; Ramisa Ayats, Arnau; Torras, Carme; Alenyà Ribas, Guillem; Moreno Noguer, Francesc d'Assis IEEE Conference on Computer Vision and Pattern Recognition p. 2673-2680 DOI: /10.1109/CVPR.2012.6247988 Presentation's date: 2012 Presentation of work at congresses
Markerless 3D human pose detection from a single image is a severely underconstrained problem because different 3D poses can have similar image projections. In order to handle this ambiguity, current approaches rely on prior shape models that can only be correctly adjusted if 2D image features are accurately detected. Unfortunately, although current 2D part detector algorithms have shown promising results, they are not yet accurate enough to guarantee a complete disambiguation of the 3D inferred shape.
In this paper, we introduce a novel approach for estimating 3D human pose even when observations are noisy. We propose a stochastic sampling strategy to propagate
the noise from the image plane to the shape space. This provides a set of ambiguous 3D shapes, which are virtually undistinguishable from their image projections. Disambiguation is then achieved by imposing kinematic constraints that guarantee the resulting pose resembles a 3D
human shape. We validate the method on a variety of situations in which state-of-the-art 2D detectors yield either inaccurate estimations or partly miss some of the body parts.
This paper reviews the state-of-the art in the field of lock-in ToF cameras, their advantages, their limitations, the
existing calibration methods, and the way they are being used, sometimes in combination with other sensors. Even though lockin ToF cameras provide neither higher resolution nor larger ambiguity-free range compared to other range map estimation
systems, advantages such as registered depth and intensity data at a high frame rate, compact design, low weight and reduced power
consumption have motivated their increasing usage in several research areas, such as computer graphics, machine vision and
Dellen, Babette Karla Margarete; Alenyà Ribas, Guillem; Foix Salmeron, Sergi; Torras, Carme Winter Vision Meeting: Workshop on Applications of Computer Vision p. 591-598 DOI: 10.1109/WACV.2011.5711558 Presentation's date: 2011 Presentation of work at congresses
We present a new method for segmenting color images into their composite surfaces by combining color segmentation with model-based fitting utilizing sparse depth data, acquired using time-of-flight (Swissranger, PMD CamCube) and stereo techniques. The main target of our work is the segmentation of plant structures, i.e., leaves, from color-depth images, and the extraction of color and 3D shape information for automating manipulation tasks. Since segmentation is performed in the dense color space, even sparse, incomplete, or noisy depth information can be used. This kind of data often represents a major challenge for methods operating in the 3D data space directly. To achieve our goal, we construct a three-stage segmentation hierarchy by segmenting the color image with different resolutions-assuming that “true” surface boundaries must appear at some point along the segmentation hierarchy. 3D surfaces are then fitted to the color-segment areas using depth data. Those segments which minimize the fitting error are selected and used to construct a new segmentation. Then, an additional region merging and a growing stage are applied to avoid over-segmentation and label previously unclustered points. Experimental results demonstrate that the method is successful in segmenting a variety of domestic objects and plants into quadratic surfaces. At the end of the procedure, the sparse depth data is completed using the extracted surface models, resulting in dense depth maps. For stereo, the resulting disparity maps are compared with ground truth and the average error is computed.
Foix Salmeron, Sergi; Alenyà Ribas, Guillem; Torras, Carme International Conference of the Catalan Association for Artificial Intelligence p. 101-109 DOI: 10.3233/978-1-60750-842-7-101 Presentation's date: 2011 Presentation of work at congresses
Monitoring plants using leaf feature detection is a challenging perception
task because different leaves, even from the same plant, may have very different
shapes, sizes and deformations. In addition, leaves may be occluded by other leaves
making it hard to determine some of their characteristics. In this paper we use a
Time-of-Flight (ToF) camera mounted on a robot arm to acquire the depth information
needed for plant leaf detection. Under a Next Best View (NBV) paradigm,
we propose a criterion to compute a new camera position that offers a better view
of a target leaf. The proposed criterion exploits some typical errors of the ToF camera,
which are common to other 3D sensing devices as well. This approach is also
useful when more than one leaf is segmented as the same region, since moving the
camera following the same NBV criterion helps to disambiguate this situation.
Alenyà Ribas, Guillem; Dellen, Babette Karla Margarete; Torras, Carme IEEE International Conference on Robotics and Automation p. 3408-3414 DOI: 10.1109/ICRA.2011.5980092 Presentation's date: 2011 Presentation of work at congresses
Supervision of long-lasting extensive botanic experiments is a promising robotic application that some recent technological advances have made feasible. Plant modelling for this application has strong demands, particularly in what concerns 3D information gathering and speed. This paper shows that Time-of-Flight (ToF) cameras achieve a good compromise between both demands, providing a suitable complement to color vision. A new method is proposed to segment plant images into their composite surface patches by combining hierarchical color segmentation with quadratic surface fitting using ToF depth data. Experimentation shows that the interpolated depth maps derived from the obtained surfaces fit well the original scenes. Moreover, candidate leaves to be approached by a measuring instrument are ranked, and then robot-mounted cameras move closer to them to validate their suitability to being sampled. Some ambiguities arising from leaves overlap or occlusions are cleared up in this way. The work is a proof-of-concept that dense color data combined with sparse depth as provided by a ToF camera yields a good enough 3D approximation for automated plant measuring at the high throughput imposed by the application.
Perception and manipulation of rigid objects has
received a lot of attention, and several solutions have been
proposed. In contrast, dealing with deformable objects is a
relatively new and challenging task because they are more
complex to model, their state is difficult to determine, and
self-occlusions are common and hard to estimate. In this
paper we present our progress/results in the perception of
deformable objects both using conventional RGB cameras and
active sensing strategies by means of depth cameras.We provide
insights in two different areas of application: grasping of textiles
and plant leaf modelling.
Ramisa Ayats, Arnau; Alenyà Ribas, Guillem; Moreno Noguer, Francesc d'Assis; Torras, Carme International Conference of the Catalan Association for Artificial Intelligence p. 199-207 DOI: 10.3233/978-1-60750-842-7-199 Presentation's date: 2011 Presentation of work at congresses
In this paper we address the problem of finding an initial good grasping point for the robotic manipulation of textile objects lying on a flat surface. Given as input a point cloud of the cloth acquired with a 3D camera, we propose choosing as grasping points those that maximize a new measure of wrinkledness, computed from the distribution of normal directions over local neighborhoods. Real grasping experiments using a robotic arm are performed, showing that the proposed measure leads to promising results.
An algorithm to estimate camera motion from the progressive deformation of a tracked contour in the acquired video stream has been previously proposed. It relies on the fact that two views of a plane are related by an affinity, whose six parameters can be used to derive the six degrees-of-freedom of camera motion between the two views. In this paper we evaluate the accuracy of the algorithm. Monte Carlo simulations show that translations parallel to the image plane and rotations about the optical axis are better recovered than translations along this axis, which in turn are more accurate than rotations out of the plane. Concerning covariances, only the three less precise degrees-of-freedom appear to be correlated. In order to obtain means and covariances of 3D motions quickly on a working robot system, we resort to the Unscented Transformation (UT) requiring only 13 samples per view, after validating its usage through the previous Monte Carlo simulations. Two sets of experiments have been performed: short-range motion recovery has been tested using a Staübli robot arm in a controlled lab setting, while the precision of the algorithm when facing long translations has been assessed by means of a vehicle-mounted camera in a factory floor. In the latter more unfavourable case, the obtained errors are around 3%, which seems accurate enough for transferring operations
La propuesta se enmarca dentro del proyecto Humanoid Lab del Institut de Robòtica i Informàtica Industrial (IRI). El grupo dispone de múltiples plataformas humanoides educativas (Robonova y Bioloid). Existe una primera versión de un sistema de realimentación de fuerza para los pies de estos robots, desarrollada en el propio grupo, que funciona correctamente pero presenta algunas deficiencias y limitaciones que se pretenden subsanar. El objetivo de este trabajo es diseñar e implementar un nuevo sistema de sensores del pie.
This document outlines the most important concepts presented during a workshop about the iCub robot done at the Instituto Italiano di Technologia in Genova. Mechanical, electronic as well as rmware and software issues are presented, and the basic procedures to detect and solve the most common problems are described. The most important goal of this workshop was to get the necessary skills to perform the most
basic maintenance of the robot without having to depend on the support from IIT. Also a brief introduction to the main issues of the control of the robot were provided.
This technical report reviews the state-of-the art in the field of ToF cameras, their advantages, their limitations, and their present-day applications sometimes in combination with other sensors. Even though ToF cameras provide neither higher resolution nor larger ambiguity-free range compared to other range map estimation systems, advantages such as registered depth and intensity data at a high frame rate, compact design, low weight and reduced power consumption have motivated their use in numerous areas of research. In robotics, these areas range from mobile robot navigation and map building to vision-based human motion capture and gesture recognition, showing particularly a great potential in object modeling and recognition.
Foix Salmeron, Sergi; Alenyà Ribas, Guillem; Andrade Cetto, Juan; Torras, Carme IEEE International Conference on Robotics and Automation p. 1306-1312 DOI: 10.1109/ROBOT.2010.5509197 Presentation's date: 2010 Presentation of work at congresses
Time-of-Flight (ToF) cameras deliver 3D images at 25 fps, offering great potential for developing fast object modeling algorithms. Surprisingly, this potential has not been extensively exploited up to now. A reason for this is that, since the acquired depth images are noisy, most of the available registration algorithms are hardly applicable. A further difficulty is that the transformations between views are in general not accurately known, a circumstance that multi-view object modeling algorithms do not handle properly under noisy conditions. In this work, we take into account both uncertainty sources (in images and camera poses) to generate spatially consistent 3D object models fusing multiple views with a probabilistic approach. We propose a method to compute the covariance of the registration process, and apply an iterative state estimation method to build object models under noisy conditions.
A planning framework is proposed for the task of cleaning a table and stack an unknown number of objects of different size on a tray. We propose to divide this problem in two, and combine two different planning algorithms. One, plan hand motions in the euclidean space to be able to move the hand in a noisy scenario using a novel Time-of-Flight camera (ToF) to perform the perception of the environment.
The other one, chooses the strategy to effectively clean the table, considering the symbolic position of the objects, and also its size for stacking considerations. Our formulation does not use information about the number of objects available, and thus is general in this sense. Also, it can deal with different object sizes, planning adequately to stack them. The special definition of the possible actions allows a simple and elegant way of characterizing the problem, and is one of the key ingredients of the proposed solution. Some experiments are provided in simulated and real scenarios that validate our approach.
Alenyà Ribas, Guillem; Nègre, Amaury; Crowley, James L IEEE/RSJ International Conference on Intelligent Robots and Systems p. 4565-4570 Presentation's date: 2009-10-15 Presentation of work at congresses
Time to Contact (TTC) is a biologically inspired method for obstacle detection and reactive control of motion that does not require scene reconstruction or 3D depth estimation. Estimating TTC is difficult because it requires a stable and reliable estimate of the rate of change of distance
between image features. In this paper we ropose a new method to measure time to contact, Active Contour Affine Scale (ACAS). We experimentally and analytically compare ACAS with two other recently proposed methods: Scale Invariant Ridge Segments (SIRS), and Image Brightness Derivatives
(IBD). Our results show that ACAS provides a more accurate estimation of TTC when the image flow may be approximated by an affine transformation, while SIRS provides an estimate that is generally valid, but may not always be as accurate as ACAS, and IBD systematically over-estimate time to contact.
We propose a novel algorithm for stereo matching using a dynamical systems approach. The stereo correspondence problem is first formulated as an energy minimization problem. From the energy function, we derive a system of differential equations describing the corresponding dynamical system of interacting elements, which
we solve using numerical integration. Optimization is introduced by means of a damping term and a noise term, an idea similar to simulated annealing. The algorithm is tested on the Middlebury stereo benchmark.
The interest of the robotics community on humanoid
robots is growing, specially in perception,
scene understanding and manipulation in humancentered
environments, as well as in human-robot
interaction. Moreover, humanoid robotics is one
of the main research areas promoted by the European
research program. Here we present some
projects and educational initiatives in this direction
carried out at the Institut de Robòtica i Informàtica Industrial, CSIC-UPC.