Speech, hand, and facial gestures: a proposal of a multimodal approach to describe negative structures with não in Brazilian Portuguese / Fala, gestos das mãos e face: uma proposta de abordagem multimodal para descrever estruturas negativas com não no Português Brasileiro

Paloma Batista Cardoso

doi:10.17851/2237-2083.31.2.719-763

Speech, hand, and facial gestures: a proposal of a multimodal approach to describe negative structures with não in Brazilian Portuguese / Fala, gestos das mãos e face: uma proposta de abordagem multimodal para descrever estruturas negativas com não no Português Brasileiro

Paloma Batista Cardoso

Abstract

Abstract: In an interactional encounter, verbal structure, hand, and face gestures form usage events in which lexical items, together with these body movements, assume specific functions. In Brazilian Portuguese, negation can be expressed by não in pre-verbal, double, and post-verbal positions. Analysis based only on verbal structures assumes that these negative structures are three ways to express opposition. However, pragmatic studies assume that they express different conversational functions: denegation, thematic pause, return to a quantitative topic, face-saving, and negative evaluation. Multimodal approaches to gestures in German, English, Spanish, and Brazilian Portuguese suggest that negation has a gestural component. Based on that, we propose a multimodal analysis of negative structures with não in Brazilian Portuguese, assuming that negative structures with this particle suggest different functions, also characterized by hand and facial gestures.
Keywords: Multimodality; hand gestures; facial gestures; negative structures; usage events.

Resumo: Na interação, estrutura verbal, gestos das mãos e face formam eventos de uso nos quais os itens lexicais, juntamente com esses movimentos corporais, assumem funções específicas. No português brasileiro, a negação pode ser expressa por não em posições pré-verbal, dupla e pós-verbal. A análise baseada apenas nas estruturas verbais assume que essas estruturas negativas são três formas de expressar oposição. No entanto, os estudos pragmáticos assumem que elas expressam diferentes funções conversacionais: denegação, pausa temática, regresso a um tópico quantitativo, salvamento da face e avaliação negativa. Abordagens multimodais de gestos em alemão, inglês, espanhol e português do Brasil sugerem que a negação tem um componente gestual. Com base nisso, neste artigo, propomos uma análise multimodal das estruturas negativas com não no português brasileiro, assumindo que as estruturas negativas com tal partícula indicam diferentes funções, caracterizadas também por gestos das mãos e face.
Palavras-chave: Multimodalidade; gestos das mãos; gestos da face; estruturas negativas; eventos de uso.

Keywords

Multimodality; Hand gestures; Facial expressions; Negation; Multimodalidade; gestos das mãos; gestos da face; estruturas negativas; eventos de uso.

Full Text:

PDF

References

AMARAL, A. O dialeto caipira. Hucitec: São Paulo, 1976.

ANTUNES, L. B.; AUBERGÉ, V.; SASA, Y. Certainty and uncertainty in Brazilian Portuguese: methodology of spontaneous corpus collection and data analysis. In: 7th International Conference on Speech Prosody, 2014, Dublin. Proceedings of the 7th th International Conference on Speech Prosody. Dublin, Trinity College, p. 2014, p. 110-114. DOI: 10.21437/SpeechProsody.2014-10

AUSTIN, J. L. How to do things with words. Oxford: Oxford University Press, 1975.

BATEMAN, J. A. Peircean semiotics and multimodality: Towards a new synthesis. Multimodal Communication, Berlin/Boston, v. 7, n. 1, p.1-24, 2018. DOI: https://doi.org/10.1515/mc-2017-0021

BOUTET, D. Structuration physiologique de la gestuelle: modèle et tests. Lidil. Revue de linguistique et de didactique des langues, n. 42, p. 77-96, 2010. DOI: https://doi.org/10.4000/lidil.3070

BRESSEM, J. Characterizing gestural form features - Suggestions for a form-based notational system of coverbal gestures. Unpublished Manuscript, 2008. Available at . Accessed on: July 17, 2023.

BRESSEM, J. Repetitions in gesture: A cognitive-linguistic and usage-based perspective.Berlin/Boston: Walter de Gruyter GmbH & Co KG, 2021.

BRESSEM, J.; LADEWIG, S. H. Rethinking gesture phases: Articulatory features of gestural movement?. Semiotica, Berlin/New York, v. 2011, n. 184, p 53-91, 2011. . DOI: https://doi.org/10.1515/semi.2011.022

BRESSEM, J.; LADEWIG, S.; MÜLLER, C. Linguistic Annotation System for Gestures. In: MÜLLER, C; CIENKI, A.; FRICKE, E.; LADEWIG, S.; McNEILL, D.; TEßENDORF, S . Handbücher Zur Sprach - Und Kommunikationswissenschaft / Handbooks of Linguistics and Communication Science, Vol. 1. Berlin/Boston: De Gruyter Mouton, 2013. p. 1098-1124.

BRESSEM, J.; MÜLLER, C. The “Negative-Assessment-Construction”: A multimodal pattern based on a recurrent gesture?. Linguistics Vanguard, Berlin/Boston, v. 3, n. s1, p. 1-9. 2017. DOI: https://doi.org/10.1515/lingvan-2016-0053

BRESSEM, J; MÜLLER, C. The family of Away gestures: Negation, refusal, and negative assessment. In: MÜLLER C.; CIENKI, A.; FRICKE, E.; LADEWIG, S.; MCNEILL, D.;BRESSEM, J. (eds.). An international handbook on multimodality in human interaction. Berlin/ Boston: De Gruyter Mouton, 2014, p. 1592-1604.

BÜHLER, K. Theory of language: The representational function of language. Amsterdam/Philadelphia: John Benjamins Publishing Company, 1990.

CALBRIS, G. Elements of Meaning in Gesture. Amsterdam: John Benjamins Publishing Company, 2011.

CARDOSO. P. B. Efeitos linguísticos e paralinguísticos na inferência dos sentidos indicados por (eu) acho que em entrevistas sociolinguísticas. 2021. 100 f. Dissertação (Mestrado em Letras) - Programa de pós-graduação em Letras, Universidade Federal de Sergipe, 2021.

CHAFE, W. L. Discourse, Consciousness, and Time: The Flow and Displacement of Conscious Experience in Speaking and Writing. Chicago: University of Chicago Press, 1994.

CIENKI, A. From the finger lift to the palm-up open hand when presenting a point: A methodological exploration of forms and functions. Languages and Modalities, v. 1, n. 1, p.17-30, 2021. DOI: https://doi.org/10.3897/lamo.1.68914

CIENKI, A. Spoken language usage events. Language and Cognition, Cambridge, v. 7, n. 4, p. 499-514, 2015.DOI: https://doi.org/10.1017/langcog.2015.20

CROFT, W.; CRUSE, A. Cognitive linguistics. Cambridge: Cambridge University Press, 2004.

DAHL, Ö. The expression of negation. Berlin: De Gruyter Mouton, 2010.

DE JORIO, A. La mimica degli antichi investigata nel gestire napoletano [Gesture in Naples and Gesture in Classical Antiquity.], translated and with an introduction and notes by Adam Kendon. Bloomington: Fibreno, 1832[2000].

DE RUITER, J. P. The production of gesture and speech. In: McNEILL, D. (eds), Language and gesture. v. 2, Cambridge: Cambridge University Press, 2000. p.284-311. DOI: https://doi.org/10.1017/CBO9780511620850.018

EKMAN, P; FRIESEN, W. V. Unmasking the face: A guide to recognizing emotions from facial clues. Los Altos:Ishk, 2003.

FREITAG, R. M. K.; CRUZ, R. C. F.; NASCIMENTO, T. DA C. The grammar in the body: on embodied resources in meaning construction and negotiation. Cadernos de Linguística, v. 2, n. 1, p. 1-24, 2021. DOI: https://doi.org/10.25189/2675-4916.2021.V2.N1.ID354

FREITAG, R.; TEJADA, J.; BRITO, Í.; PINHEIRO, B.; SILVA, L. S.; CARDOSO, P. B; ANDRADE SOUZA, V. R.. Rating linguistics features and facial expressions: an approach of variation processing. Cadernos de Linguística, v. 1, n. 2, p. 01-19, 2020. DOI: https://doi.org/10.25189/2675-4916.2020.V1.N2.ID15

FURTADO DA CUNHA, M. A. O modelo das motivações competidoras no domínio funcional da negação. DELTA: Documentação de Estudos em Lingüística Teórica e Aplicada, São Paulo, v. 17, p. 1-30, 2001. DOI: https://doi.org/10.1590/S0102-44502001000100001.

GIBBS, R. Embodiment and cognitive science. New York: Cambridge University Press, 2005.

HARRISON, S. Evidence for node and scope of negation in coverbal gesture. Gesture, Amsterdam, v. 10, n. 1, p. 29-51, 2010. DOI: https://doi.org/10.1075/gest.10.1.03har

IRISKHANOVA, O. K.; CIENKI, A. The semiotics of gestures in cognitive linguistics: contribution and challenges. Voprosy Kognitivnoy Lingvistiki, v. 4, p. 25-35, 2018. DOI: 10.20916/1812-3228-2018-4-25-36

JAKOBSON, R. Lingüística e comunicação. São Paulo, Editora Cultrix, 2008.

JESPERSEN, O. Selected Writings of Otto Jespersen. New York: Routledge, 2010.

KANADE, T.; COHN, J. F.; TIAN, Y. Comprehensive database for facial expression analysis. In: IEEE International Conference on Automatic Face and Gesture Recognition, 4, 2000, Grenoble. Proceedings fourth IEEE international conference on automatic face and gesture recognition, 2000. p. 46-53. DOI: https://doi.org/10.1109/AFGR.2000.840611

KENDON, A. et al. Gesticulation and speech: Two aspects of the process of utterance. The relationship of verbal and nonverbal communication, v. 25, n. 1980, p. 207-227, 1980.

KENDON, A. Gesture: Visible action as utterance. Cambridge: Cambridge University Press, 2004.

KENDON, A. Language and gesture: Unity or duality. In: McNEILL, D. (ed). Language and gesture. Cambridge: Cambridge University Press, p. 47-62, 2000.

KENDON, A. Pragmatic functions of gestures: Some observations on the history of their study and their nature. Gesture, Amsterdam, v. 16, n. 2, p. 157-175, 2017. DOI: https://doi.org/10.1075/gest.16.2.01ken

KENDON, A. The study of gesture: Some remarks on its history. Boston: Springer, 1983. DOI: https://doi.org/10.1007/978-1-4615-9328-7_15

KOK, K. I.; CIENKI, A. Cognitive Grammar and gesture: Points of convergence, advances and challenges. Cognitive Linguistics, Berlin/Boston v. 27, n. 1, p. 67-100, 2016. DOI: https://doi.org/10.1515/cog-2015-0087

LADEWIG, S. H. Describing, searching and prompting - variants of a recurrent gesture. Language and Literature, v. 41, n. 1, p. 89-111, 2010. DOI: https://doi.org/10.1163/25890859-041-01-90000006

LADEWIG, S. H.; BRESSEM, J. New insights into the medium hand: Discovering recurrent structures in gestures. Semiotica, Berlin/Boston, v. 2013, n. 197, p. 203-231, 2013. DOI: https://doi.org/10.1515/sem-2013-0088

LADEWIG, S. Integrating gestures: The dimension of multimodality in cognitive grammar.Berlin/Boston: Walter de Gruyter GmbH & Co KG, 2020.

LAKOFF, G.; JOHNSON, M. Metaphors we live by. Chicago: University of Chicago Press, 2003.

LANGACKER, R. W. Interactive cognition: Toward a unified account of structure, processing, and discourse. International Journal of Cognitive Linguistics, v. 3, n. 2, p. 95-125, 2012.

McNEILL, D. Hand and Mind: What Gestures Reveal About Thought. Chicago: University of Chicago press. 1992.

McNEILL, D. So you think gestures are nonverbal? Psychological review, Washington, v. 92, n. 3, p. 350-371, 1985. DOI: https://doi.org/10.1037/0033-295X.92.3.350

McNEILL, D., CASSELL, J., & MCCULLOUGH, K.-E. Communicative Effects of Speech-Mismatched Gestures. Research on Language & Social Interaction, v. 27, n.3, p. 223-237, 1994.DOI: https://doi.org/10.1207/s15327973rlsi2703_4

MIGUEL, F. K. Psicologia das emoções: uma proposta integrativa para compreender a expressão emocional. Psico-usf, Campinas, v. 20, n. 1, p. 153-162, 2015. DOI: https://doi.org/10.1590/1413-82712015200114

MITTELBERG, I.; HINNELL, J. Gesture Studies and Semiotics. In: PELKEY. J.; COBLEY, P. (eds). New York: Bloomsbury Semiotics Volume 4: Semiotic Movements, 2023, p. 183-214.

MÜLLER, C. Iconicity and gesture. In: SANTI, S.; GUAÏTELLA, I.; CAVÉ, C. ; KONOPCZYNSKI, G. (eds). Oralité et gestualité: Communication multimodale, interaction. Paris: L’Harmattan, 1998, p. 321–328.

MÜLLER, C.; LADEWIG, S; BRESSEM, J. Towards a grammar of gestures: A form-based view. In: MÜLLER C.; CIENKI, A.; FRICKE, E.; LADEWIG, S.; MCNEILL, D.;BRESSEM, J. (eds.). Handbücher Zur Sprach- Und Kommunikationswissenschaft / Handbooks of Linguistics and Communication Science,vol. 1. Berlin/ Boston: De Gruyter Mouton, 2013, p. 707-733.

MÜLLER, C.; SPECKMANN, G. Gestos con una valoración negativa en la conversación cubana. DeSignis, Buenos Aires:,, n. 3, p. 91-103, 2002

PETRY, P.; GOLDNADEL, M.; LAMBERTI, L. Funções pragmáticas de enunciados com dupla negação em Porto Alegre no final do século XX: uma análise qualitativa de dados extraídos de entrevistas sociolinguísticas. Revista da ABRALIN, v. 20, n. 2, p. 1-38, 2021. DOI: https://doi.org/10.25189/rabralin.v20i2.1899

PINHEIRO, B. F. M. Pistas linguísticas e paralinguísticas para os sentidos diminutivos. 2021. 98 f. Dissertação (Mestrado em Letras) - Programa de Pós-Graduação em Letras, Universidade Federal de Sergipe, 2021.

SACKS, H.; SCHEGLOFF, H. E. A.; JEFFERSON, G. A simplest systematics for the organization of turn-taking for conversation. Language, v. 50, n.4, p. 696–735, 1974. DOI: https://doi.org/10.1016/B978-0-12-623550-0.50008-2

SANTOS, B. F. G. A construção de um repertório de gestos de negação para o português brasileiro: uma proposta cognitivo-gestual. 2021. 152 f. Dissertação (Mestrado em Linguística) - Programa de pós-graduação em Linguística, Universidade Estadual do Sudoeste da Bahia, 2021.

SAUSSURE, F. Curso de linguística geral. São Paulo: Editora Cultrix, 2012.

SCHWENTER, S. A. The pragmatics of negation in Brazilian Portuguese. Lingua, v. 115, n. 10, p. 1427-1456, 2004. DOI: https://doi.org/10.1016/j.lingua.2004.06.006

SEARLE, J. R. Speech acts. Brill, 1975

SEIXAS, V. C.; DE ALKMIM, M. G. R.; CHAVES, E. Construções negativas na fala de moradores da zona rural do município de Piranga, Estado de Minas Gerais: uma análise variacionista. Acta Scientiarum. Language and Culture, Maringá, v. 34, n. 2, p. 269-276, 2012. DOI: https://doi.org/10.4025/actascilangcult.v34i2.15149

SOUSA, L. T. de. A Gramaticalização do não no Português Brasileiro e a Etapa do Processo. Domínios de Lingu@gem, Uberlândia, v. 1, n. 2, 2011. DOI: https://doi.org/10.14393/DL2-v1n2a2007-5

STOKOE, W. Sign language structure. Buffalo: Buffalo University Press, 1960.

STREEK, J. Gesturecraft: The manu-facture of meaning. Amsterdam: John Benjamins, 2009.

TEBENDORF, S. Pragmatic and Metaphoric Gestures – Combining Functional With Cognitive Approaches in the Analysis of the “Brushing Aside Gesture”.” In: MÜLLER C.; CIENKI, A.; FRICKE, E.; LADEWIG,S.; MCNEILL, D.; BRESSEM, J. (eds.). Body – Language – Communication: An International Handbook on Multimodality in Human Interaction (Handbook of Linguistics and Communication Science 38.2.), Berlin/Boston: De Gruyter Mouton, 2014, p. 1540-1558.

DOI: http://dx.doi.org/10.17851/2237-2083.31.2.719-763