R & D Activities
Research and development activities at CVRL address the following fundamental questions in an effort to arrive at robust solutions to corresponding theoretical, computational, and design problems.
What is the maximum information that a robot can extract by sensing its environment? More importantly, what is the actual information that a robot needs to extract from its environment in order to exhibit certain desired and possibly intelligent behaviors?
Robot behaviors and sensory-motor coordination:
How should perceptual capabilities and control strategies be combined to obtain the desired robot behaviors? How should the characteristics of the environment and the mechanical properties of a robot be taken into account in this process?
How can a robot acquire skills in a developmental manner, through interaction with its environment? How can a robot accumulate experience, thus improving its acquired skills?
DARWIN - Dextrous Assembler Robot Working with Embodied Intelligence
The project aims at developing robotic technology that will be able to assemble objects from their constituent parts. The sequence of operations needed, during the assembly process, will be provided by an executive process in the robotic agent. The executive process will accommodate a wide spectrum of requirements and constraints of the general assembly problem. On the one end of the spectrum, it will allow for CAD-CAM systems, used in industrial manufacturing, to provide the detailed sequence of instructions while at the other end it will support stand-alone operation in cases where external knowledge cannot be provided. In this mode the robot will be able to use previous knowledge of tools and assembly strategies, acquired by 'playing' with the tools and the objects in question and using mental simulation capabilities in order to 'imagine' the outcome of decisions during the assembly process. In addition to trial and error learning mode the robot will also support learning by observation of another agent assembling the same object. Finally a third learning mode will be supported where a series of snapshots of the object-to-be-assembled will be provided and from these the robot will discover by itself the necessary sequence through reasoning.
JAMES - Joint Action for Multimodal Embodied Social Systems
The JAMES project aims to develop a socially intelligent humanoid robot combining efficient task-based behaviour with the ability to understand and respond in a socially appropriate manner to a wide range of multimodal communicative signals in the context of realistic, open-ended, multi-party interactions. The work in JAMES will build on state of-the-art results and techniques in seven areas: social robotics, social signal processing, machine learning, multimodal data collection, planning and reasoning, visual processing, and natural language interaction.
FIRST-MM - Flexible Skill Acquisition and Intuitive Robot Tasking for Mobile Manipulation in the Real World
The project aims to build the basis for a new generation of autonomous mobile manipulation robots that can flexibly be instructed to perform complex manipulation and transportation tasks. The project will develop a novel robot programming environment that allows even non-expert users to specify complex manipulation tasks in real-world environments. In addition to a task specification language, the environment includes concepts for probabilistic inference and for learning manipulation skills from demonstration and from experience. The project will build upon and extend recent results in robot programming, navigation, manipulation, perception, learning by instruction, and statistical relational learning to develop advanced technology for mobile manipulation robots that can flexibly be instructed even by non-expert users to perform challenging manipulation tasks in real-world environments, designed to autonomously navigate in urban environments outdoors as well as in shopping malls and shops to provide various services to users including guidance, delivery, and transportation.
European Commission FP7
Integrated Project Cognitive Systems
Objectives:The aim of GRASP is the design of a cognitive system capable of performing grasping and manipulation tasks in open-ended environments, dealing with novelty, uncertainty and unforeseen situations. GRASP will develop means for robotic systems to reason about graspable targets, to explore and investigate their physical properties and finally to make artificial hands grasp any object. To meet these objectives, we will use theoretical, computational and experimental studies to model skilled sensorimotor behaviour based on known principles governing grasping and manipulation tasks performed by humans. As widely recognised, to design and evaluate such a complex system, we need to integrate computational techniques from machine learning, computer vision, control theory and signal processing together with experimental frameworks that include real robotic and simulation tools that allow for a long-term, experimental control over sensory inputs and tasks. Hence, the objective of GRASP is to integrate findings from disciplines such as neuroscience, cognitive science, robotics, multimodal perception and machine learning to achieve a core cognitive capability: Grasping any object by building up relations between task setting, embodied hand actions, object attributes, and contextual knowledge such that learnt grasps are extendable toward new, never seen objects in new situations.
INDIGO -Interaction with Personality and Dialogue Enabled Robots
Objectives:INDIGO aims to develop a new generation of intelligent mobile robotic systems that will operate and serve tasks in every-day environments addressing their interaction with humans. Unlike any related previous research project, the emphasis in the INDIGO sys-tem will be on advancing human-robot communication. This will be achieved both by perceiving and understanding human behavior, and by interpreting and expressing robot actions in a human-oriented way.
OCTOPUS - Novel Design Principles and Technologies for a New Generation of High Dexterity Soft-bodied Robots Inspired by the Morphology and Behaviour of the Octopus
The grand challenge of the OCTOPUS IP is investigating and understanding the principles that give rise to the octopus sensory-motor capabilities and incorporating them in new design approaches and ICT and robotics technologies to build an embodied artefact, based broadly on the anatomy of the 8-arm body of an octopus, and with similar performance in water, in terms of dexterity, speed, control, flexibility, and applicability.
VECTOR - Versatile Endoscopic Capsule for Gastrointestinal Tumor Recognition and Therapy
The objective of the VECTOR project is to develop intelligent endoscopic capsules using innovations in micro- and nanotechnology. The VECTOR project aims at investigating and developing a miniaturised robotic pill for advanced diagnostics and therapy in the human digestive tract. The project mission is to make a significant contribution to the diagnosis and treatment of digestive cancers and their precursors and to strengthen the competitiveness of the European biomedical industry through innovative technologies.
Project XENIOS aims at the development of a user-robot interaction system, based on understanding of visual input, speech processing and natural language generation. The developed system will be used to facilitate interaction of tour-guide robots with on-site as well as web-visitors of museums and exhibition areas.
MATHESIS - Observational Learning in Cognitive Agents
The MATHESIS project aims to explore fundamental aspects of social communication and adaptive behavior, especially the process of assigning meaning to the actions of other subjects. This will be demonstrated by developing and validating artificial cognitive agents, primarily robotic ones, able to acquire a repertory of motor actions by observational learning. Observational learning is understood here as the capacity to acquire an action strategy only through observation of other agents, without the experimentation needed in other learning procedures. Neurophysiological investigations will attempt to establish that the neural representations of action-execution, action-observation and action-recall overlap extensively within the cortex, as suggested by our preliminary results. A major implication of this, both for biological and for artificial agents, is that it should be possible to train their motor system by simple action-observation and action-recall, as in the traditional use of observational learning and mental training (e.g., the use of videotapes to train athletes). The MATHESIS project will assess the generality, scalability, accuracy and robustness of such cognitive architectures. Furthermore, it will establish the developmental stage at which observational learning can be used efficiently in infants and children.
MUSCLE Network of Excellence (Multimedia Understanding through Semantics, Computation and Learning)
European Commission FP6
Unit E2 : Knowledge Management and Content Creation
MUSCLE aims at creating and supporting a pan-European Network of
Excellence to foster close collaboration between research groups in
multimedia datamining on the one hand and machine learning on the
other in order to make breakthrough progress towards the following
objectives: (b) Harnessing the full potential of machine learning
and cross-modal interaction for the (semi-) automatic generation of
metadata with high semantic content for multimedia documents; (b)
Applying machine learning for the creation of expressive, context-aware,
self-learning, and human-centered interfaces that will be able to
effectively assist users in the exploration of complex and rich multimedia
content; (c) Improving interoperability and exchangeability of heterogeneous
and distributed (meta)data by enabling data descriptions of high semantic
content (e.g. ontologies, MPEG7 and XML schemata) and inference schemes
that can reason about these at the appropriate levels. (d) Through
dissemination, training and industrial liaison, contribute to the
distribution and uptake of the technology by relevant end-users such
as industry, education, and the service sector. In particular, close
interactions with other IP's and NOE's in this and related activity
fields are planned. (e) Through accomplishing the above, MUSCLE will
facilitate the broad and democratic access to information and knowledge
for all European citizens (e.g. e-Education, enriched cultural heritage).
The objective of the project is to understand motion and perception systems of lower animal forms and to design and fabricate mini- and micro-machines inspired by such biological systems. The long term goal of the project is to develop an entirely new generation of autonomous smart machines able to interact with the environment in a life-like way and with performance comparable to those of the animals by which they are inspired.
These bio-inspired machines may lead to many useful applications in various fields related to the "inspection" problem: for example in medicine and, in particular, in microendoscopy. The bio-like machines will be not only designed, but also fabricated, according to a biomimetic approach which merges sensing and actuation capabilities, by exploiting hybrid manufacturing techniques, high integration of active materials and sensors, and embedded low level control.
RECOVER - Photorealistic 3D Reconstruction of Perspective Paintings and Pictures
EC 6th Framework research program: Co-operative Research Project
RECOVER aims to develop a system for the semi-automatic extraction of three-dimensional (3D) models of scenes depicted in perspective paintings. 3D models of paintings constitute a new and exciting way for the general public to experience and appreciate fine art. The viewer can experience a feeling of immersion; paintings are no longer perceived as static artefacts from a long-gone past but as living, vibrant entities. With the aid of appropriate software, the viewer can literally dive into the painting, interacting with it and observing it from various viewpoints in impressive walk-throughs and inspiring fly-bys. This enables non-specialists to step into history and experience the scene in the space and time frame perceived by the artist. Ultimately, the viewing of paintings becomes a more appealing, exploratory endeavour, arousing the publicís interest in fine art and cultural heritage in general.
GNOSYS- An Abstraction Architecture for Cognitive Agents
European Commission FP6
Unit E5: Cognitive Systems
Specific Targeted Research or Innovation Project (STREP)
ObjectivesThe project's vision is to develop an architecture for cognitive agents and to validate it in a robotic embodiment in an unknown outdoors environment. The architecture will include major building blocks found in human cognitive processes and it will integrate the cycle of perception-knowledge acquisition-abstraction-reasoning-action generation. Knowledge acquisition will be supported by a suitable concept system with a corresponding abstraction mechanism. The concept system is the representation of the knowledge that the agent possesses of its environment and itself. Objects, relations, goals, context information, and solution strategies are considered as knowledge about a situation. The abstraction mechanism is responsible for creating and organising a hierarchy of concepts while the reasoning process operates on the concept system in order to make inferences for virtual actions and select the one that will realise the greatest reward. The architecture will include attention control as a means of handling complexity, prioritising responses, detecting novelty and creating new goals. Both sensory and motor attention will be used. A goals-oriented computational model will allow the fusion together of user tasks with tasks originating from the agent. A goals generation system will enable the agent to produce its own goals. Reinforcement learning will provide the means by which the agent learns solution strategies for the satisfaction of a goal. The loop closes by having new actions modifying the current knowledge through perception. The architecture will be implemented in a robotics application, namely of robot navigation in unknown outdoors environment but it will be in no way specific to this domain. Different outdoors environments will be used for testing.
MultiSens - Cameras as Multifunctional Sensors for Automated Processes
EC 6th Framework research program: Co-operative Research Project
Goal of the project is the development of a ''smart'' camera for process control and fault detection in automation. The camera will take the role of several sensors and thus reduce the setup time for machines substantially. Two prototypes will be developed that are targeted at low-speed (40ms) as well as high-speed (1ms) applications.
In most automatic assembly machines a large number of (sometimes expensive) sensors are used to check whether the machine is still operating correctly or to avoid collisions. A typical situation is that a sensor has to make sure that a gripper is retracted, before the handled object moves on to the next station. The setup of these sensors and programming of the right timing in a PLC (programmable logic controller) program is a very time consuming process. Within the project a camera system will be developed that will help the 135000 SMEs in the European mechanical engineering industry in building their machines.
The proposed device is essentially a camera which is combined with a processing unit and will be able to visually interpret image sequences of automated processes, report malfunctions and deviations in the process and assist the task of programming the right sequence of events. Within this project there are two SMEs working in the field of integrated camera-processor systems (ProDesign, GER and Analogic Computers, HU). They will add this device to their range of products and benefit directly from the results of this project. Three SMEs from various fields of mechanical engineering (Hage Sondermaschinenbau, automation for automotive industry, Austria; Hajek-Engineering, food processing and packaging machines, Austria and Hochrainer, assembly automation, Germany) will provide a wide range of different automated processes upon which the developments will be based. The researchers (Profactor, Austria; FORTH, Greece; MTA-SZTAKI, Hungary) will develop the algorithms and basic methods for the interpretation of image-sequences and online fault detection.
ActiPret - Interpreting and Understanding Activities of Expert Operators for Teaching and Education
The objective of ActIPret is to develope a cognitive vision methodology that interprets and records the activities of people handling tools. Focus is on active observation and interpretation of activities, on parsing the sequences into constituent behaviour elements, and on extracting the essential activities and their functional dependence. By providing this functionality ActIPret will enable observation of experts executing intricate tasks such as repairing machines and maintaining plants. The expert activities are interpreted and stored using natural language expressions in an activity plan. The activity plan is an indexed manual in the form of 3D reconstructed scenes, which can be replayed at any time and location to many users using Augmented Reality equipment.
WebFair - Web Access to Commercial Fairs Through Mobile Agents
WebFAIR addresses the marketing and promotion requirements of large commercial exhibitions by providing broad access to information, services and commodities exhibited at the event. Essentially, WebFAIR aims at providing the means to remote corporate and private users for active and personalised workplace exploration and information visualisation for commercial purposes.
The goal of the proposed project is the development and validation of an interactive tele-presence system (WebFAIR) based on mobile robotic systems, able to provide individual access to large exhibitions and commercial trade-fairs over the World-Wide Web. The user observes through the "eyes" (cameras) of the robot and hears though its "ears" (microphones). Through a Web-interface, users all over the world will be able to tele-control the robotic avatar and specify exhibits, or interesting places that they may wish to visit.
LIFEPLUS proposes an innovative 3D reconstruction of ancient frescos-paintings through the real-time revival of their fauna and flora, featuring groups of virtual animated characters with artificial life dramaturgical behaviours, in an immersive AR environment. In greater detail LIFEPLUS objectives are:
-Real-time realistic virtual life in AR environments.
-Automatic Real-time Camera Tracking in unknown environments.
-Design of successful character based installations
-Expressive autonomous cinematography for interactive Virtual Environments.
Although initially targeted at Cultural Heritage Centres, the paradigm is by no means limited to such subjects, but encompasses all types of future Location-Based Entertainments, E-visitor Attractions as well as on-set visualisations for the TV/movie industry.
EU-IST Programme DG-XIII
The goal of TOURBOT is the development of an interactive tour-guide robot able to provide individual access to museums' exhibits and cultural heritage over the Internet. TOURBOT operates as the user's avatar in the museum by accepting command over the web that direct it to move in its workspace and visit specific exhibits; besides, TOURBOT can also act as a flexible, on-site museum-guide.
PRIME- Prediction of Congestion and Incidents in Real Time For Intelligent Incident Management and Emergency Traffic Management
EU-IST Programme DG-XIII
PRIME aims to improve road safety on motorways and adjacent urban networks by increasing the effectiveness of incident detection and incident management through the development of innovative methods. The project will build on recent achievements in the management of incidents and road emergencies in related EU projects, particularly IN-RESPONSE, and will enhance the complete incident and road emergency management chain. The innovative methods include improvements in the dynamic prediction, detection, and verification of incidents. These provide benefits to the traffic management controller and facilitate the integration of high-level technologies now reaching the marketplace. The new methods also address the integration of motorway and urban incident management technologies which leads to improved overall incident management and safety. A prototype will be developed including databases, models and algorithms to be tested off-line (using data collected from three EU sites) and on-line (using data from two EU sites). Evaluation will include separate testing of each prototype module, (detection, prediction and verification) and a simulation tool will evaluate the system components.
DRIVER - Integrated System for Assistive Navigation of Robotic Wheelchair Platforms
Period: June 1999 - Nov. 2000
General Secretariat for Research and Technology, Greece, Programme for People with Special Needs (EPET-AMEA).
Objectives:DRIVER aims to help people with special needs to navigate an electric wheel chair. For this purpose, the navigational capabilities of robotic wheel chairs will be properly enhanced to allow semi-autonomous navigation. The user controls the navigation proccess and determines the destination. The resulted system will be secure, user-friendly and easily adaptive to the current robotic wheel chairs. Thus, the DRIVER's main goal is the development, evaluation and assessment of a navigational support system that could be used as an add-on module on robotic wheel chairs.
TACIT - Theory and Applications of Continuous Interaction Techniques
Period: 3 years
Training and Mobility of Researchers (Research Networks), DG XII
Objectives:The aim of the TACIT network is the development of a Theory of Continuous Interaction Techniques. These techniques comprise interaction between human and computer by means of gestures, speech and body expression recognition. The theory is developed within a multi-disciplinary setting encompassing different perspectives and disciplines such as computer science, linguistics, cognitive psychology and semiotics. It aims at the development of a unified and integrated body of knowledge enabling and facilitating the application of a combination of existing and novel interaction techniques that are expected to be required in a new generation of interfaces around the year 2010. The theory will be applied to a number of selected scenarios and assessed by means of both experimental techniques and formal validation. The scenarios are selected with particular attention to the industry's view of potential markets and the likely timeliness required for the development of the technologies. The industrial view is taken care of by means of the close links between the participating organisations and industry within joint research projects.
NEMESIS - New Multimedia Services using Analysis Synthesis
The NEMESIS project targets the applications of professional digital video post-production and new multimedia editing. This project relies on a new powerful content-based image sequence representation scheme to make possible the use of cutting edge image analysis tools in the domain of multimedia contents production. NEMESIS proposes to integrate and to develop automatic image analysis/synthesis techniques providing a set of objects (e.g. regions, layers and 3D objects over time), from sequences of monocular natural images. The NEMESIS project will develop a coherent software platform allowing new functionnalities for image sequence manipulation outperforming existing post-production tools.
Period: 3 years
Training and Mobility of Researchers (Research Networks), DG XII
Objectives:The goal of the VIRGO network is to coordinate European research and postgraduate training activities that address the development of intelligent robotic systems able to navigate in (partially) unknown and possibly changing environments. This goal will be achieved through a framework which enhances RTD activities in European laboratories, already established in the aforementioned scientific area. Specifically, in pursuing this goal, alternative environment representations based on visual information will be studied and methods to process these representations and use them to control the motion of the mechanical parts of a robot will be developed. The VIRGO network consortium consists of ten organisations from eight European countries: FORTH (network coordinator), AUC, DIST, TUG, KTH, U-BONN, INRIA, GMD, DIKU, U-ZURICH.
NAVIGATOR - Robot Navigation in Unknown Environments Using Visual Information
Period: Jan. 1996 - Dec. 1997
General Secretariat for Research and Technology, Greece
Objectives:NAVIGATOR addresses issues of robot navigation in partially unknown environments. It employs landmarks as well as spatial relationships between them for space perception and localization purposes. The system's knowledge of the environment is specified from the contents of a visual memory, which contains a large number of patterns corresponding to landmark descriptions known to the system. The visual recognition of landmarks, using a set of qualitative descriptors, in conjunction with the visual memory, facilitates navigation without the need of environment maps and beacons, used in traditional approaches.
ARHON - Multimedia System for Archival, Annotation and Retrieval of Historical Documents
ARHON pursues the development of a multimedia system for the archival and retrieval of historical documents. Such documents are acquired and stored as image patterns; they are annotated using a semantic indexing system which facilitates their retrieval using semantic information. Since these documents are dated back a few centuries, their visual appearance is degraded; image processing techniques are therefore employed to restore them to an acceptable quality.
DIVAN - Distributed audio-Visual Archives Network