Theories like the picture superiority effect prove that visual information is vital in the acquisition of knowledge, suchas in language learning. Words can be graphically represented to illustrate the meaning of a message and facilitate itsunderstanding, but this rarely applies to abstract words. The current research turns to sign languages to explore thecommon semantic elements that link abstract words to each other, pointing towards the possibility of creating clusters oficonic meanings. By using sign language insight and VerbNets organisation of verb predicates, this study presents a novelorganisation of 500 English abstract verbs classified by visual shape. Graphic animation was used to visually represent 20classes of abstract verbs (see on www.vroav.online). An online survey was created to achieve judgements on the graphicvisuals representativeness. Significant agreement between participants suggests a positive way forward for further researchand applications within multimodal communication and computer assisted learning.

Ostracism is a social phenomenon, shared by most social animals, including humans. Its detection plays a crucial role forthe individual, with possible evolutionary consequences for the species.Considering (1) its relation with communication and therefore language and (2) its social nature, we hypothesised that thecombination of linguistic and community-level social features would have a positive impact on the automatic recognitionof ostracism in human online communities.We modelled a linguistic community through Reddit data and we analysed the performance of simple classification al-gorithms (Nave Bayes and SVM), particularly focusing on the feature selection. Comparing the accuracy scores of thealgorithms fed with a) linguistic features, b) extralinguistic features, and c) linguistic + extralinguistic features, we testedour hypothesis, showing how models based on c) generally outperform.To our knowledge, this is the first attempt to automatise the identification of such a complex phenomenon through NLPtechniques.

Drawing from the Conceptual Metaphor Theory and the Structure-Mapping Theory, this paper introduces two exploratory works in the field of metaphorical and visual reasoning using vision models and multimodal large language models. (i) The Multimodal Chain-of-Thought Prompting for Metaphor Generation task aimed to generate metaphorical linguistic expressions from non-metaphorical images by using the multimodal LLaVA 1.5 model and the two-step approach of multimodal chain-of-thought prompting. The results showed the model's ability to generate metaphorical expressions, as 92% of them were classified as metaphors by human evaluators. Additionally, the evaluation revealed interesting patterns in terms of metaphoricity, familiarity and appeal scores across the generated metaphors. (ii) The Metaphorical Visual Analogy (MeVA) task consisted in solving visual analogies of the kind "source_domain : target_domain :: source_element : ?" by choosing the correct target element among three difficult distractors, varying in semantic domains and roles. The results showed that all six models and humans performed higher than chance level, with only GPT-4o and ConvNeXt achieving higher than humans. Moreover, the error analysis showed that, in solving the analogies, the most frequent error was the selection of distractor 1. These works showed encouraging results for future research in the field of metaphorical and visual reasoning, contributing to the broader question of whether AI models serve as empirical tests of existing cognitive theories.

We describe a system that can associate images with Englishproverbs. We start from a corpus of proverbs, harvest relatedimages from the web and use this data to train two variants ofa convolutional neural network. We then collect a small set ofannotations, and use these to combine the outputs of the twonetworks into a single prediction for each input image. Wecarry out feature selection experiments on a set of features de-rived from the images and from the predicted proverbs, anddemonstrate that the metaphoricity of the proverbs plays a sig-nificant role in classification accuracy. An empirical evalua-tion with human raters confirms the system’s ability to abstractfrom the raw bits in the images and to learn meaningful, non-trivial associations.

The present paper addresses the study of cross-linguistic phonosemantic correspondences within a deep learning framework. An LSTM-based Recurrent Neural Network is trained to associate the phonetic representation of a word, encoded as a sequence of feature vectors, to its corresponding semantic representation in a multilingual and cross-family vector space. The processing network is then tested, without further training, in a language that does not appear in the training set and belongs to a different language family. The performance of the model is evaluated through a comparison with a monolingual and mono-family upper bound and a randomized baseline. After the assessment of the network's performance, the distribution of phonosemantic properties in the lexicon is inspected in relation to different (psycho)linguistic variables, showing a link between lexical non-arbitrariness and semantic, syntactic, pragmatic, and developmental factors.

Creativity is an essential factor in successful advertising wherecatchy and memorable media is produced to persuade the au-dience. The creative elements in the visual design and in theslogan of an advertisement elevate the overall appeal providinga perceptually grounded attractive message. In this study, wepropose the exploitation of creativity cues in textual and visualinformation for the appreciation prediction of multimodal ad-vertising prints. Moreover, as a novel dimension space of mul-timodality, we propose using the human sense (i.e., sight, hear-ing, taste, and smell) information embedded in the language.Our findings show that sensorial information is an invaluableindication of whether the advertisement is appreciated or not.Furthermore, combining linguistic and visual models signif-icantly improves the unimodal appreciation detection perfor-mances.

The assessment of how a deceptive message is produced in dif-ferent languages has received little attention, with the majorityof studies focused on the English language. Moreover, thereis no agreement about the stability of linguistic clues of deceitacross different languages. In this paper, we address this issueby analysing both theory-driven linguistic markers of decep-tion (cognitive load hypothesis) and standard text categorisa-tion features. After compiling a multilingual corpus of bothhonest and deceitful first-person opinions regarding five differ-ent topics, we assessed the cross-language applicability of fourdifferent features sets in within-topic, cross-topic and cross-language binary classification experiments. Results showedpromising classification performances in all the three experi-ments with few exceptions. Interestingly, linguistic markersof deceit linked to the cognitive load hypothesis exhibited thesame trend in the two languages under investigation and thecross-language evaluation highlighted their usefulness in spot-ting deceit between different languages.