Search results

As the capabilities of the mobile robots as well as their abilities to perform more tasks in an autonomous manner are increased, we need to think about the interactions that humans will have with these robots. Human-robot interaction (HRI) has recently received considerable attention in the academic community, government labs, technology companies, and through the media. The interdisciplinary nature of HRI requires researchers in the field to understand their research within a broader context. Since natural language is the easiest and most natural mode of communication for humans, it is desirable to use it to instruct the robot and to generate easy-to-understand messages for the user. Using natural language to teach a navigation task to a robot is an application of a more general instruction-based learning methodology. It can be used to instruct the robot with higher-level goals or to handle certain behaviors and modify their execution. One effective way is to describe the route to the robot in a multimodal way. On the other hand, significant progress has been made towards stable robotic bipedal walking in the last few years. This is creating an increased research interest in developing autonomous navigation strategies which are tailored specifically to humanoid robots. Efficient approaches to perception and motion planning, which are suited to the unique characteristics of bipedal humanoid robots and their typical operating environments, are receiving special interest. One important area of research involves the design of algorithms to compute robust navigation strategies for humanoid robots in human environments. Therefore, autonomous robot navigation based on route instruction is becoming an increasingly important research topic with regard to both humanoid and other mobile robots. In this dissertation, the problem of humanoid robot navigation in indoor environments is addressed. A complete framework is presented for humanoid robot navigation based on a multimodal cognitive interface. First, a spatial language to describe route-based navigation tasks for a mobile robot is proposed. This language is implemented to present an intuitive interface that enables novice users to easily and naturally describe a route to a mobile robot in indoor environments. An instruction interpreter is implemented to analyze the user's route to generate its equivalent symbolic and topological map representations which are used as an initial path estimation for the humanoid robot. Second, a robust lightweight object processing system with a high detection rate is developed. It can actually be used by mobile robots and meet their hard constraints to recognize landmarks during navigation. A landmark processing system is developed to detect, identify, and localize different types of landmarks during robot navigation in indoor or miniature city environments. The system is based on a two-step classification stage which is robust and invariant towards scaling and translations. By combining the strengths of appearance-based and model-based object classification techniques, it provides a good balance between fast processing time and high detection accuracy. Finally, a time-efficient hybrid motion planning system for a humanoid robot in indoor environments is implemented. The proposed technique is a combination of sampling-based planner and D* Lite search to generate dynamic footstep placements in unknown environments. A modified cylinder model is used to approximate the trajectory for the robot's body-center during navigation. It calculates the actual distances required to execute different actions of the robot and compares them to the distances from the nearest obstacles. D* Lite search is then used to find dynamic and low-cost footstep placements within the resulting configuration space. ; Da die Fähigkeiten von mobilen Robotern einschließlich ihrer Möglichkeiten, Aufgaben autonom durchzuführen, erweitert wurden, muss die Interaktion zwischen Mensch und Roboter neu betrachtet werden. Human-Robot-Interaction (HRI) ist ein aktuelles Thema in der Forschung, in Technologie-Unternehmen und in den Medien. Der interdisziplinäre Charakter des HRI-Bereiches erfordert Forschung innerhalb eines breiten Themenkomplexes. Da natürliche Sprache das einfachste und natürlichste Mittel der Kommunikation für Menschen ist, ist es wünschenswert, diese Form der Kommunikation auch bei der HRI zu nutzen, um einem Roboter Anweisungen zu geben und leicht verständliche Botschaften für den Benutzer zu generieren. Die Verwendung natürlicher Sprache zur Instruierung bei Navigations-Aufgaben ist eine Anwendung einer allgemeineren instruktions-basierten Lernmethodologie. Dem Roboter können so übergeordnete Ziele mitgeteilt werden, bestimmte Verhaltensweisen geändert oder auch die Ausführung einzelner Aktionen modifiziert werden. Eine effiziente Methode zur Beschreibung der Route ist die Verwendung multimodaler Anweisungen. Weil die vergangenen Jahre einen bedeutenden Fortschritt auf dem Gebiet der humanoiden Roboter und des stabilen zweibeinigen Gehens gebracht haben, besteht ein verstärktes Forschungsinteresse an der Entwicklung autonomer Navigationsstrategien, die speziell auf humanoide Roboter zugeschnitten sind. Von besonderem Interesse sind effiziente Ansätze zur kombinierten Perzeptions- und Aktionsplanung, die an die speziellen Eigenschaften von zweibeinigen humanoiden Robotern und ihre typischen Betriebsumgebungen angepasst sind. Ein wichtiges Gebiet der Forschung ist der Entwurf von Algorithmen zur Berechnung von robusten Navigations-Strategien für humanoide Roboter in menschlicher Umgebung. Aus diesem Grunde ist die auf Routen-Instruktion beruhende autonome Roboter-Navigation ein zunehmend interessantes Thema im Hinblick auf humanoide und andere mobile Roboter. Diese Dissertation befasst sich mit dem Problem der humanoiden Roboter-Navigation in Innenräumen. Es wird ein komplettes Framework für humanoide Roboter-Navigation basierend auf einer multimodalen Schnittstelle vorgestellt. Zunächst wird eine formale Sprache eingeführt, mit der die routen-basierten Navigationsaufgaben beschrieben werden können. Diese Sprache stellt eine intuitive Schnittstelle bereit, mit der auch unerfahrene Anwender leicht einen mobilen Roboter in einer Route in Innenräumen instruieren können. Ein Befehls-Interpreter analysiert die Benutzer-Eingabe und generiert entsprechende symbolische und topologische Darstellungen, die als erste Pfad-Schätzung für den humanoiden Roboter verwendet werden. Des Weiteren wird in dieser Arbeit ein robustes und effizientes Objekterkennungssystem mit einer hohen Erkennungsrate entwickelt. Es kann bei mobilen Robotern eingesetzt werden und erfüllt die Anforderung, Landmarken während der Navigation zu erkennen. Das Landmarken-Detektions-System ist in der Lage, während der Roboter-Navigation in einer Miniatur-Stadt verschiedene Typen von Landmarken zu detektieren, identifizieren und zu lokalisieren. Das System basiert auf einem zweistufigen Klassifikations-Prozess, der robust und invariant gegenüber Skalierung und Translation ist. Durch die Kombination der Stärken der erscheinungs-basierten und modell-basierten Objekt-Klassifikation bietet es einen guten Kompromiss zwischen schnellen Bearbeitungszeiten und hoher Erkennungsgenauigkeit. Ebenfalls Bestandteil dieser Arbeit ist die Implementierung eines zeiteffizienten hybriden Bewegungs-Planungs-Systems für humanoide Roboter in einer Innenraum-Umgebung. Die vorgeschlagene Technik ist eine Kombination aus Sampling-basierter Planung und "D * Lite"-Suche, die ermöglicht, dynamisch Tritt-Platzierungen in unbekannten Umgebungen zu erzeugen. Ein modifiziertes Zylinder-Modell wird verwendet, um die Trajektorie des Roboters während der Navigation näherungsweise zu bestimmen. Die Planungskomponente berechnet die benötigten Freiräume, um verschiedene Aktionen des Roboters auszuführen und vergleicht sie mit der aktuellen Entfernung zu den nächstgelegenen Hindernissen. "D* Lite"-Suche wird dann verwendet, um eine dynamische und effiziente Platzierung der Schritte innerhalb des resultierenden Konfigurations-Raumes zu finden.

In this brief, multimodal wavelet transforms for face recognition is proposed for feature extraction for improve the efficiency for the face recognition. Face recognition system is widely used for detect the given input image from the different database. Face recognition scheme applications are military, medical, entertainment and security purpose. So far number of feature extraction techniques has been researched on different wavelets. In this paper to enhance image retrieval recognition efficiency detect the edge contours of the given input data. Face recognition algorithms concentrate on the different distance measure techniques. In this multimodal analysis introduced the different combinations of discrete wavelets such as discrete wavelet transform (DWT) and contourlet transform. The experimental results of the proposed paper uses YALE and ORL database. As compared to existing techniques this proposed method gives better results in terms of recognition efficiency and computaltional rates.

Content-based information retrieval is based on the information contained in documents rather than using metadata such as keywords. Most information retrieval methods are either based on text or image. In this paper, we investigate the usefulness of multimodal features for cross-lingual news search in various domains: politics, health, environment, sport, and finance. To this end, we consider five feature types for image and text and compare the performance of the retrieval system using different combinations. Experimental results show that retrieval results can be improved when considering both visual and textual information. In addition, it is observed that among textual features entity overlap outperforms word embeddings, while geolocation embeddings achieve better performance among visual features in the retrieval task.

Zusammenfassung
Zielsetzung Dieser Scoping Review zielt darauf ab, einen Überblick über
bisher publizierte Behandlungsstrategien zu geben, die multimodaler und nicht
rein medikamentöser Art sind und für haus- oder bettgebundene ME/CFS-Patienten
in Frage kommen. Der Schwerpunkt liegt somit auf der Analyse von
telemedizinischen und aufsuchenden Behandlungselementen. Darüber hinaus werden
die in den betrachteten Studien verwendeten Evaluations- und Auswertungsmethoden
herausgearbeitet.
Methode Nach einer Literaturrecherche gemäß der Scoping Review-Methode und
unter Verwendung von vordefinierten Kriterien wurden 14 Publikationen
eingeschlossen und ausgewertet. Voraussetzung für den Einschluss waren Modelle,
die auf hausgebundene Personen mit ME/CFS anwendbar sind und sich auf
sozialmedizinische, psychologische Unterstützungsangebote und nicht auf einzelne
medikamentöse Strategien beziehen.
Ergebnis Die als geeignet bewerteten Behandlungen fanden überwiegend in
einem aufsuchenden (Hausbesuche) (n=5) oder telemedizinischen (n=7) Format
statt. Studien mit einem anderen Setting wurden aus dem Grund einbezogen, dass
das beschriebene Modell in ein telemedizinisches Konzept umwandelbar wäre.
Darüber hinaus wurden in mehreren Studien (n=8) Individualisierung und
Flexibilität der Behandlungsmethode - und somit die Möglichkeit, auf das
tagesaktuelle Ausmaß der Beeinträchtigung eingehen zu können - ebenfalls als
wichtige Faktoren im Umgang mit dieser Patientengruppe herausgearbeitet. In
sechs Studien wurde eine zusätzliche, ausdrückliche Einbindung der Familien in
das Behandlungskonzept beschrieben. In zehn Artikeln wurde die Evaluation eines
Behandlungskonzepts beschrieben, meist unter Verwendung von Fragebögen (n=7). Es
wurde keine einheitliche Verwendung spezifischer Fragebögen festgestellt.
Qualitative Auswertungen wurden durchweg mittels thematischer Analyse nach Braun
und Clarke durchgeführt (n=3).
Schlussfolgerung Die Gesamtzahl der Publikationen, die sich mit
multimodalen Behandlungsstrategien für hausgebundene ME/CFS-Patienten
beschäftigen, ist gering. Es finden sich jedoch aufsuchende und telemedizinische
Ansätze. Die Mehrzahl der identifizierten Publikationen adressieren die
Notwendigkeit einer individualisierten, flexiblen Versorgung. Einige widmen sich
dem möglichen Mehrgewinn durch explizite Einbindung der Angehörigen. Die
recherchierten Daten belegen die besonderen Herausforderungen bei der Versorgung
von schwer betroffenen ME/CFS-Patienten und implizieren, dass diese
Besonderheiten auch im Forschungskontext berücksichtigt werden müssen.

Multimodal freight transport can facilitate the economic development of Operators. The Study shows particularly in fright transportation, there is developing the single mode of transport in to multimodal operation system, the characteristics of multimodal travel today in Ethiopia and assesses its future potential in Railway Corporation, the factors required to implement the multimodal operating system and make Railway will be competitive rather than shipping line. Multimodal transport requires new organizational and financial arrangements between all actors involved. Transporting freight plays a major role in the country and Ethiopian railway will start to work in the route of Addis Ababa-Djibouti in a single mode of transport. this type of transport is not sufficient to return the loan of railway in short period of time and in the sake of increasing socio- economic benefit. In this study by interviewing the groups of professional from Governmental agencies and private sector the information which is necessary for this paper is collected and the required data for this particular investigation purpose is taking from concerned sectors especially ESLSE . The previous research which is related to this title is used as a base for the theoretical and empirical framework of the study. Additionally, computerized search strategy was selected. Finally implementing multimodal operation requires solving the transportation cost, transportation time, transportation capacity constraints, and transportation allocation problem and also transporting in multimodal makes to saving 62.2USD/ton or transporting in unimodal is 1.24times that of multimodal cost and Operating Multimodal transport is 7.07 times greater profit than operating single mode of transport. Therefore By Considering enough performance of railway and its profitability implement Operating Multimodal transport is essential. Hence Ethiopian Railway use multimodal operator for their better economic efficiency and social effectiveness

Comunicació presentada a: IV Jornadas en Tecnología del Habla, celebrat del 8 al 10 de novembre de 2006 a Saragossa. ; In this paper we present a person identification system based on a combination of acoustic features and 2D face images. We address the modality integration issue on the example of a smart room environment. In order to improve the results of the individual modalities, the audio and video classifiers are integrated after a set of normalization and fusion techniques. First we introduce the monomodal acoustic and video identification approaches and then we present the use of combined input speech and face images for person identification. The various sensory modalities, speech and faces, are processed both individually and jointly. The result obtained in the CLEAR'06 Evaluation Campaign shows that the performance of the multimodal approach results in improved performance in the identification of the participants. ; This work has been partially sponsored by the EC-funded project CHIL (IST-2002 − 506909) and by the Spanish Government-funded project ACESCA (TIN2005 −08852).

Recent advances in thermal infrared imaging (LWIR) has allowed its use in applications beyond of the military domain. Nowadays, this new family of sensors is included in different technical and scientific applications. They offer features that facilitate tasks, such as detection of pedestrians, hot spots, differences in temperature, among others, which can significantly improve the performance of a system where the persons are expected to play the principal role. For instance, video surveillance applications, monitoring, and pedestrian detection.During the dissertation the next question is stated: \textit{Could a couple of sensors measuring different bands of the electromagnetic spectrum, as the visible and thermal infrared, be used to extract depth information?} Although it is a complex question, we shows that a system of these characteristics is possible as well as their advantages, drawbacks, and potential opportunities.In this research an experimental study that compares different cost functions and matching approaches is performed, in order to build a multimodalstereovision system. Furthermore, the common problems in infrared/visible stereo, specially in the outdoor scenes are identified. Our framework summarizes the architecture of a generic stereo algorithm, at different levels: computational, functional, and structural, which can be extended toward high-level fusion (semantic) and high-order (prior).The proposed framework is intended to explore novel multimodal stereo matching approaches, going from sparse to dense representations (both disparity and depth maps). Moreover, context information is added in form of priors and assumptions. Finally, the dissertation shows a promissory way toward the integration of multiple sensors for recovering three-dimensional information.

International audience ; The paper introduces an annotation scheme for a political debate dataset which is mainly in the form of video, and audio annotations. The annotation contains various infor- mation ranging from general linguistic to domain specific information. Some are annotated with automatic tools, and some are manually annotated. One of the goals is to use the information to predict the categories of the answers by the speaker to the disruptions. A typology of such answers is proposed and an automatic categorization system based on a multimodal parametrization is successfully performed.

International audience ; The paper introduces an annotation scheme for a political debate dataset which is mainly in the form of video, and audio annotations. The annotation contains various infor- mation ranging from general linguistic to domain specific information. Some are annotated with automatic tools, and some are manually annotated. One of the goals is to use the information to predict the categories of the answers by the speaker to the disruptions. A typology of such answers is proposed and an automatic categorization system based on a multimodal parametrization is successfully performed.

International audience ; The paper introduces an annotation scheme for a political debate dataset which is mainly in the form of video, and audio annotations. The annotation contains various infor- mation ranging from general linguistic to domain specific information. Some are annotated with automatic tools, and some are manually annotated. One of the goals is to use the information to predict the categories of the answers by the speaker to the disruptions. A typology of such answers is proposed and an automatic categorization system based on a multimodal parametrization is successfully performed.

AbstractThis paper analyzes humorous comments created through a popular viewing-and-commenting system used in China and Japan, known asdanmu(ordanmaku). This system enables its users to superimpose anonymous comments on the video frame, which are displayed in subsequent viewing. We collected 327 user-selected 'funniest' screenshots of comments fromdanmuvideo sharing sites. Using content and discourse analysis, we re-contextualized the comments and identified main mechanisms of humor. Results show that speakers make fun of the plot, characters and of each other, relating to the video frame, Chinese culture and Japanese fandom. They rely on non-aggressive but rather playful teasing, allusions and retorts, and apply multimodal resources such as color, layout, and symbols to enhance the humorous effect. Our study contributes to the emerging research focus on multimodal humor (Yus 2016), social semiotics and a discursive approach todanmu-mediated communication.

The term localisation is derived from the word locale, which traditionally means a small area or vicinity. In ancient days, localisation meant navigation -- an art of finding the way from one place to another. Tremendous advancement in the science of navigation dates back to the sixteenth century, when instruments like compasses, sextants and the first ever clock to keep the time exactly were devised. Advancement in navigation brought ways and means to explore the world, be it for expansion of the territories or for promoting trade and business. Since then localisation has been explored for several decades as a classical problem in many disciplines -- including robotics, virtual reality, navigation. Now we are in the era of ubiquitous computing -- a term coined by the visionary Mark Weiser in the early 1990s. Weiser sees technology only as a means to an end, which should take a back seat in order to allow user to fully concentrate on the task at hand. Looking from a technological standpoint, today we are surrounded by a wealth of devices enriched with sensing, computing and communication capabilities which are seamlessly integrated in our daily lives. Knowing the location of an object is an important cornerstone and fertile research area in ubiquitous computing. The growing need of location systems underscores the importance of addressing this problem -- government initiatives to locate emergency call by cellular network providers and the increasing usage of global positioning systems (GPS) in many commercial applications as in navigation are just a few examples. Since the field is active and vibrant, new services and market players are constantly emerging. Google have just launched a new service called Latitude, which lets smart phone and laptop users share their location with friends and allows those friends to share their locations in return. Latitude uses satellites and cell towers to estimate location. The market for GPS products and services alone is expected to grow to USD200 billion by 2015. Real-time location systems (RTLS) in the transport and logistics sector drive the penetration of several location-based solutions. The number of RTLS suppliers is expected to increase from 50 to 200 by 2013, reflecting a market growth from USD 145 million in 2008 to USD 2.7 billion in 2013. Despite the extraordinary advances in outdoor localisation and navigation, indoor localisation still remains an open challenge. Fundamental to any location system are the algorithms used to estimate location. This thesis focuses on formulation of localisation algorithms with the capability of fusing measurements from multiple modalities. We begin by systematically analysing the basic principles of localisation through a review and classification of the state of the art. From our detailed survey, it is evident that no location system is error-free and suited for all situations. For example, pure inertial sensors suffer from drift, ultrasound sensors require clear line of sight and magnetic sensors are affected by ferromagnetic and conductive materials in the environment. Thus, we rationalise "multimodal localisation" as one of the promising ways for improving location accuracy. Apart from improving performance of the location system in limited measurement volumes, fusion of heterogeneous sensing systems will ultimately allow people to move between places covered by different sensing systems without loss of location knowledge. We explore localisation algorithms that use multiple sensing modalities to improve accuracy and robustness. To ground our work, we have chosen three specific applications covering both infrastructure-based positioning and ad hoc-based positioning systems. From our taxonomy, we create a blueprint of location technologies that would meet those three application needs. -Localisation in office environments to facilitate social networking, as a way to help coordination of people and understand social patterns. We leverage the existing wireless local-area networks (WLAN) infrastructure to sense motion and location with the main motivation of building wide-area location services. Our contributions include-- (i) in-depth characterisation of received signal strength (RSSI), (ii) novel algorithms to deduce motion by observing fluctuations in RSSI across all the access points in range, and (iii) performance comparison using real data against common deterministic location algorithms with and without adding motion information. -Transport and logistics operation (e.g. in warehouses), motivating the need of fine-grained location information. We use ultra-wideband (UWB) as it copes with harsh indoor environments better than conventional radio technologies. Our contributions include-- (i) characterisation of heterogeneous observations (pseudoranges and angles) obtained from two deployments, mimicking real-world (low-overhead) vs. ideal deployment (carefully planned and calibrated), (ii) formulation of algorithms to fuse heterogeneous observations and (iii) a thorough evaluation for both static and dynamic tracking. -Emergency response scenarios, motivating the need for ad hoc positioning capabilities. In particular, we use a combination of inertial sensors and ultrasound sensors. The position error in a purely inertial system increases with time and requires correction from external sources. We address this problem by deploying ultrasound sensors as landmarks correcting for the inertial drift. Our contributions include-- (i) characterisation of inertial and ultrasound data, (ii) algorithms to support guidance and tracking and (iii) a thorough evaluation from data gathered from real deployments. While the chosen technologies and applications are not exhaustive, they are representative as they cover a broad spectrum across several dimensions: accuracy -- fine grained to coarse grained, coverage -- room-level to wide-area, dependence -- dense infrastructure to ad hoc, cost -- expensive to minimal cost. In every instance, we have illustrated the benefits of combining multiple modalities. In short, our contributions include algorithms for motion detection and technology independent localisation algorithms that have the ability to fuse readings across different sensing technologies and incorporate motion models to improve accuracy significantly. Another important aspect of the work presented in this thesis is the characterisation of the raw measurement errors of the individual modalities. In all cases, we perform a rigorous evaluation of the presented algorithms by using measurements collected from real deployments.