Suchergebnisse

Vector-space word representations obtained from neural network models have been shown to enable semantic operations based on vector arithmetic. In this paper, we explore the existence of similar information on vector representations of images. For that purpose we define a methodology to obtain large, sparse vector representations of image classes, and generate vectors through the state-of-the-art deep learning architecture GoogLeNet for 20 K images obtained from ImageNet. We first evaluate the resultant vector-space semantics through its correlation with WordNet distances, and find vector distances to be strongly correlated with linguistic semantics. We then explore the location of images within the vector space, finding elements close in WordNet to be clustered together, regardless of significant visual variances (e.g., 118 dog types). More surprisingly, we find that the space unsupervisedly separates complex classes without prior knowledge (e.g., living things). Afterwards, we consider vector arithmetics. Although we are unable to obtain meaningful results on this regard, we discuss the various problem we encountered, and how we consider to solve them. Finally, we discuss the impact of our research for cognitive systems, focusing on the role of the architecture being used. ; This work is partially supported by the Joint Study Agreement no. W156463 under the IBM/BSC Deep Learning Center agreement, by the Spanish Government through Programa Severo Ochoa (SEV-2015-0493), by the Spanish Ministry of Science and Technology through TIN2015-65316-P project and by the Generalitat de Catalunya (contracts 2014-SGR-1051), and by the Core Research for Evolutional Science and Technology (CREST) program of Japan Science and Technology Agency (JST). ; Peer Reviewed ; Postprint (published version)

Vector-space word representations obtained from neural network models have been shown to enable semantic operations based on vector arithmetic. In this paper, we explore the existence of similar information on vector representations of images. For that purpose we define a methodology to obtain large, sparse vector representations of image classes, and generate vectors through the state-of-the-art deep learning architecture GoogLeNet for 20 K images obtained from ImageNet. We first evaluate the resultant vector-space semantics through its correlation with WordNet distances, and find vector distances to be strongly correlated with linguistic semantics. We then explore the location of images within the vector space, finding elements close in WordNet to be clustered together, regardless of significant visual variances (e.g., 118 dog types). More surprisingly, we find that the space unsupervisedly separates complex classes without prior knowledge (e.g., living things). Afterwards, we consider vector arithmetics. Although we are unable to obtain meaningful results on this regard, we discuss the various problem we encountered, and how we consider to solve them. Finally, we discuss the impact of our research for cognitive systems, focusing on the role of the architecture being used. ; This work is partially supported by the Joint Study Agreement no. W156463 under the IBM/BSC Deep Learning Center agreement, by the Spanish Government through Programa Severo Ochoa (SEV-2015-0493), by the Spanish Ministry of Science and Technology through TIN2015-65316-P project and by the Generalitat de Catalunya (contracts 2014-SGR-1051), and by the Core Research for Evolutional Science and Technology (CREST) program of Japan Science and Technology Agency (JST). ; Peer Reviewed ; Postprint (published version)

Deep learning diagnostic algorithms are proving comparable results with human experts in a wide variety of tasks, and they still require a huge amount of well-annotated data for training, which is often non affordable. Metric learning techniques have allowed a reduction in the required annotated data allowing few-shot learning over deep learning architectures. Aims and Objectives: In this work, we analyze the state-of-the-art loss functions such as triplet loss, contrastive loss, and multi-class N-pair loss for the visual embedding extraction of hematoxylin and eosin (H&E) microscopy images and we propose a novel constellation loss function that takes advantage of the visual distances of the embeddings of the negative samples and thus, performing a regularization that increases the quality of the extracted embeddings. Materials and Methods: To this end, we employed the public H&E imaging dataset from the University Medical Center Mannheim (Germany) that contains tissue samples from low-grade and high-grade primary tumors of digitalized colorectal cancer tissue slides. These samples are divided into eight different textures (1. tumour epithelium, 2. simple stroma, 3. complex stroma, 4. immune cells, 5. debris and mucus, 6. mucosal glands, 7. adipose tissue and 8. background,). The dataset was divided randomly into train and test splits and the training split was used to train a classifier to distinguish among the different textures with just 20 training images. The process was repeated 10 times for each loss function. Performance was compared both for cluster compactness and for classification accuracy on separating the aforementioned textures. Results: Our results show that the proposed loss function outperforms the other methods by obtaining more compact clusters (Davis-Boulding: 1.41 ± 0.08, Silhouette: 0.37 ± 0.02) and better classification capabilities (accuracy: 85.0 ± 0.6) over H and E microscopy images. We demonstrate that the proposed constellation loss can be successfully used in the medical domain in situations of data scarcity. ; This work was partially supported by PICCOLO project. This project has received funding from the European Union's Horizon2020 Research and Innovation Programme under grant agreement No. 732111.

In this thesis, we present a simple and effective architecture called Bilinear Convolutional Neural Networks (B-CNNs). These networks represent an image as a pooled outer product of features derived from two CNNs and capture localized feature interactions in a translationally invariant manner. B-CNNs generalize classical orderless texture-based image models such as bag-of-visual-words and Fisher vector representations. However, unlike prior work, they can be trained in an end-to-end manner. In the experiments, we demonstrate that these representations generalize well to novel domains by fine-tuning and achieve excellent results on fine-grained, texture and scene recognition tasks. The visualization of fine-tuned convolutional filters shows that the models are able to capture highly localized attributes. We present a texture synthesis framework that allows us to visualize the pre-images of fine-grained categories and the invariances that are captured by these models. In order to enhance the discriminative power of the B-CNN representations, we investigate normalization techniques for rescaling the importance of individual features during aggregation. Spectral normalization scales the spectrum of the covariance matrix obtained after bilinear pooling and offers a significant improvement. However, the computation involves singular value decomposition, which is not computationally efficient on modern GPUs. We present an iteration-based approximation of matrix square-root along with its gradients to speed up the computation and study its effect on fine-tuning deep neural networks. Another approach is democratic aggregation, which aims to equalize the contributions of individual feature vector into the final pooled image descriptor. This achieves a comparable improvement, and can be approximated in a low-dimensional embedding unlike the spectral normalization. Therefore, this approach is friendly to aggregating higher-dimensional features. We demonstrate that the two approaches are closely related, and we discuss their trade-off between performance and efficiency.

International audience ; The amount of digital visual data (image, video and 3D object) has increased rapidly on the Internet. Image, video and 3D object security becomes increasingly important for many applications, e.g., confidential transmission, video surveillance, military and medical applications. For example, the necessity of fast and secure diagnosis is vital in the medical world. Nowadays, the transmission of visual data is a daily routine and it is necessary to find an efficient way to transmit them over networks. Two main groups of technologies have been developed for this purpose. The first one is based on content protection through encryption. In this group, proper decryption of data requires a key. The second group bases the protection on digital watermarking or data hiding, aimed at secretly embedding a message into the data. In order to not increase the processing time, these two approaches must be combined with the compression stage. Nowadays, the challenge is to perform simultaneously for example image encryption and compression.

International audience ; The amount of digital visual data (image, video and 3D object) has increased rapidly on the Internet. Image, video and 3D object security becomes increasingly important for many applications, e.g., confidential transmission, video surveillance, military and medical applications. For example, the necessity of fast and secure diagnosis is vital in the medical world. Nowadays, the transmission of visual data is a daily routine and it is necessary to find an efficient way to transmit them over networks. Two main groups of technologies have been developed for this purpose. The first one is based on content protection through encryption. In this group, proper decryption of data requires a key. The second group bases the protection on digital watermarking or data hiding, aimed at secretly embedding a message into the data. In order to not increase the processing time, these two approaches must be combined with the compression stage. Nowadays, the challenge is to perform simultaneously for example image encryption and compression.

International audience ; The amount of digital visual data (image, video and 3D object) has increased rapidly on the Internet. Image, video and 3D object security becomes increasingly important for many applications, e.g., confidential transmission, video surveillance, military and medical applications. For example, the necessity of fast and secure diagnosis is vital in the medical world. Nowadays, the transmission of visual data is a daily routine and it is necessary to find an efficient way to transmit them over networks. Two main groups of technologies have been developed for this purpose. The first one is based on content protection through encryption. In this group, proper decryption of data requires a key. The second group bases the protection on digital watermarking or data hiding, aimed at secretly embedding a message into the data. In order to not increase the processing time, these two approaches must be combined with the compression stage. Nowadays, the challenge is to perform simultaneously for example image encryption and compression.

Convolutional Neural Networks (CNN) are the most popular of deep network models due to their applicability and success in image processing. Although plenty of effort has been made in designing and training better discriminative CNNs, little is yet known about the internal features these models learn. Questions like, what specific knowledge is coded within CNN layers, and how can it be used for other purposes besides discrimination, remain to be answered. To advance in the resolution of these questions, in this work we extract features from CNN layers, building vector representations from CNN activations. The resultant vector embedding is used to represent first images and then known image classes. On those representations we perform an unsupervised clustering process, with the goal of studying the hidden semantics captured in the embedding space. Several abstract entities untaught to the network emerge in this process, effectively defining a taxonomy of knowledge as perceived by the CNN. We evaluate and interpret these sets using WordNet, while studying the different behaviours exhibited by the layers of a CNN model according to their depth. Our results indicate that, while top (i.e., deeper) layers provide the most representative space, low layers also define descriptive dimensions. ; This work was partially supported by the IBM/BSC Technology Center for Supercomputing (Joint Study Agreement, No. W156463), by the Spanish Government through Programa Severo Ochoa (SEV-2015-0493), by the Spanish Ministry of Science and Technology through TIN2015-65316-P project and by the Generalitat de Catalunya (contracts 2014-SGR-1051). ; Peer Reviewed ; Postprint (author's final draft)

Convolutional Neural Networks (CNN) are the most popular of deep network models due to their applicability and success in image processing. Although plenty of effort has been made in designing and training better discriminative CNNs, little is yet known about the internal features these models learn. Questions like, what specific knowledge is coded within CNN layers, and how can it be used for other purposes besides discrimination, remain to be answered. To advance in the resolution of these questions, in this work we extract features from CNN layers, building vector representations from CNN activations. The resultant vector embedding is used to represent first images and then known image classes. On those representations we perform an unsupervised clustering process, with the goal of studying the hidden semantics captured in the embedding space. Several abstract entities untaught to the network emerge in this process, effectively defining a taxonomy of knowledge as perceived by the CNN. We evaluate and interpret these sets using WordNet, while studying the different behaviours exhibited by the layers of a CNN model according to their depth. Our results indicate that, while top (i.e., deeper) layers provide the most representative space, low layers also define descriptive dimensions. ; This work was partially supported by the IBM/BSC Technology Center for Supercomputing (Joint Study Agreement, No. W156463), by the Spanish Government through Programa Severo Ochoa (SEV-2015-0493), by the Spanish Ministry of Science and Technology through TIN2015-65316-P project and by the Generalitat de Catalunya (contracts 2014-SGR-1051). ; Peer Reviewed ; Postprint (author's final draft)

Over the past few decades, the development of new methods of storing, transferring, analyzing, and reproducing data has become increasingly important, considering the widespread development of information technologies and multimedia means. These methods also include means of ensuring the reliability, security, safety, and confidentiality of information when it is transmitted through various communication channels. Today, the fulfillment of the necessary data confidentiality requirements is developing in the direction of cryptography and steganography.
The article discusses a method of ensuring data confidentiality by means of Digital Steganography, using GIF images as container files. The features of this method are considered, the main steps of the algorithm are described, and the resulting file is analyzed. The second part of the article studies a method that estimates the probability of hiding information in the blocks of a container file. This method utilizes Pearson's goodness-of-fit test. A number of test tests of the algorithm have been carried out to check the efficiency of this method. As a result, a computer program has been created using the Microsoft Visual Studio 2019 Community programming environment and the C# programming language (.NET Framework).

Cognitive architectures allow robots to perform their operations by drawing on a process that aims to simulate human reasoning. This paper presents an integrated semantic artificial memory system in cognitive architecture based on symbolic reasoning and a connective representation of the knowledge. This memory system attempts to simulate how humans learn to distinguish instances of particular objects within their class using a convolutional network to detect the relevant elements of an image. We use a vector with the extracted features to learn to discriminate an instance of another element from the same class. A novel feature of our approach is its autonomous learning process during the operation of the robot, integrating a deep learning embedding with a statistical classifier. The usefulness and robustness of this method are demonstrated by applying it to a social robot that learns to differentiate people. Finally, experiments are carried out to validate our approach, comparing the detection results with several alternative methods. ; This work has been funded by the Spanish Government TIN2016-76515-R grant for the COMBAHO project, supported with Feder funds. This work has also been supported by a Spanish grant for PhD studies ACIF/2017/243 and FPU16/00887.

Recognition of the value of landscapes, environmentally, economically, and to quality of life—and, importantly, the embedding of these concepts in legislation such as the European Landscape Convention—has led to the need for an 'objective' assessment of these values and the potential impact of changes to them. But studies relating preference information to metric analysis of planimetric viewsheds have so far provided only limited explanation of preference. It has been suggested that this is due to the effect of perspective on the visual topology of the view generating different metrics in perspective from those on the flat map. The completion of the Pan European Study under the EU Framework 5 Visulands project provided a large sample of preference responses to a limited number of computer-generated simple landscape scenarios. As such this is an ideal opportunity to test the significance of perspective on metric correlation with preference. This paper considers the degree to which metrics are altered by a panoramic or viewshed analysis, and the significance of this for any correlation with the preference scores. The implications for the role of the respective media in planning are considered, including that of 3D visualisation as a means for eliciting opinion on landscape preference.

FC -- Half Title -- ADVANCES IN SEMIOTICS -- Title -- Copyright -- Dedication -- Contents -- List of Figures -- Introduction -- 1 Introducing Visual Language -- What is "visual language"? -- The structure of visual language -- Outline of the book -- SECTION 1 Structur e of visual language -- Concern #1: Panels are not arbitrary signs -- Concern #2: There is no systematic lexicon of panels -- 2 The Visual Lexicon, Part 1: Visual Morphology -- A visual lexicon -- Open-class lexical items -- Combining schemas -- Conclusion -- Closed-class lexical items -- Bound morphemes/affixation -- Suppletion/umlaut -- Reduplication -- Conclusion -- 3 The Visual Lexicon, Part 2: Panels and Constructions -- Regularity in panels -- Suppletive panels -- Panel-level templates -- Panels as attention units -- Visual language constructions -- Multimodal constructions -- Conclusion -- 4 Visual Language Grammar: Narrative Structure -- Three ideas for sequential image comprehension -- 1. Panel transitions/linear coherence relationships -- 2. Promiscuous transitions -- 3. General cognitive scripts and schemas -- Conclusion -- Basic narrative categories -- Peaks -- Initials -- Releases -- Establishers -- Prolongations -- Orienters -- Summary -- Constituent structure in visual narrative -- Modification -- Summary -- Discourse and Film -- Conclusion -- 5 Navigation of External Compositional Structure -- The challenge of page layouts -- Variations in ECS -- Comprehension of ECS -- Constraints on external compositional structure -- Embedding structures -- Descriptive tree structures -- The infinite canvas -- Conclusion -- 6 Cognition of Visual Language -- Graphic morphology -- Motion lines -- Carriers -- Narrative grammar -- Narrative categories -- Separation of structure and meaning -- Constituent structure -- Visual narratives and brains -- Fluency.

Cover -- Half Title -- Series Page -- Title Page -- Copyright Page -- Table of Contents -- List of figures -- The logo as a scientific tool -- List of contributors -- Introduction -- PART I: Public agency as a new form of protest -- 1. Public space democracy, assembly and creativity -- 2. What theory for the new protest movements? -- 3. Embedding the prefigurations of the Gezi protests: The rhizomatic spread of new subjectivities and politicized identities -- 4. Çars¸ı in the Gezi Park protests in Istanbul: New forms of public agency in a square movement -- 5. Protest repertoires during Ukraine's Euromaidan: Historical traditions, memory politics and new public agency -- 6. Transmuting civic horizontality of 15M into civil verticality in Spain: Collective presences and representative governance -- PART II: Public culture and norm conflicts -- 7. Manaf Halbouni's "Monument" installation in Dresden (2017): Contesting memories and the politics of art -- 8. The case of M. F. Husain in democratic India: Art, politics and offence -- 9. Da'wa through conviviality and arts in Molenbeek: Beer, coffee and the frictions of the public -- 10. The case of AKM building in Istanbul: Public sphere under (re)construction -- PART III: Public memory, monuments and art forms -- 11. Sites, selfies and contemporary transnational commemoration -- 12. Armenian cultural heritage and architecture in Turkey: The emergence of plural memories in the public space -- 13. Martyr iconography in postwar Iran: When public memorialization leads to grieving obstruction -- 14. Contemporary artists on the traces of the Mausoleum of Georgi Dimitrov in Sofia: Memory debates and public space -- PART IV: Public transgressions and artistic interventions -- 15. "Art for demos" -- 16. Aesthetic struggles in Algiers (1988-2018) -- 17. Mutterzunge, the silence in the park.

Analysis of biomedical images requires computational expertize that are uncommon among biomedical scientists. Deep learning approaches for image analysis provide an opportunity to develop user-friendly tools for exploratory data analysis. Here, we use the visual programming toolbox Orange (http://orange.biolab.si) to simplify image analysis by integrating deep-learning embedding, machine learning procedures, and data visualization. Orange supports the construction of data analysis workflows by assembling components for data preprocessing, visualization, and modeling. We equipped Orange with components that use pre-trained deep convolutional networks to profile images with vectors of features. These vectors are used in image clustering and classification in a framework that enables mining of image sets for both novel and experienced users. We demonstrate the utility of the tool in image analysis of progenitor cells in mouse bone healing, identification of developmental competence in mouse oocytes, subcellular protein localization in yeast, and developmental morphology of social amoebae.