How to Speak Silently — Rethinking Materiality, Agency, and Communicative Competence in Virtual Reality

While thinkers of the material turn offer new conceptual resources for talking about non-human ontologies, interaction researchers are trying to reassemble the social situation fragmented by telecommunication. Conversation analysts tend to see technical objects in their situation-constitutive role, but they can also disrupt the current projects of the participants whilst remaining “unseen and unnoticed” (e.g. Zoom delays). We propose a conceptualization of the relationship between the participant and the interaction environment as a source of agency, which makes it possible to preserve an emic perspective. We illustrate our thesis by analyzing a case study of interaction between a Deaf and a hearing participant in VRChat.

In this case, virtual pencils that leave durable inscriptions in the air are usedand-answer by participants sequence for and communication demonstrate . thatWe:  analyzethe participants a simple  questiontreat the- inscriptions as material; the hearing participant is less capable of communicating in this space than the Deaf person; the answer to the question is produced jointly due to the instructional work of the Deaf participant. The results allow us to draw the following conclusions about the nature of materiality, agency and communicative competence: 1) the materiality of the environment is not a purely analytical category, but is constructed by the participants in the interaction;  2) the agency of the participants depends on the environment and at the same time has a distributed character; 3) communicative competence is not directly related to the “internal” characteristics of the agent, such as atypicality.

1. Erofeeva M. (2021) Achieving Intersubjectivity in Asymmetric Interaction. The Journal of Social Policy Studies, 19(4): 635–650.

2. Zizina O. (2019) The phenomenon of polite inattention in VR Chat. Blog of the MSSES Virtual Reality Laboratory. https://www.vrstudies.ru/ru/civil-inattention-vrchat/

3. Korbut A. M. (2021) Domestication of Artificial Intelligence: Smart Speakers and Transformation of Everyday Life. Monitoring of Public Opinion: Economic and Social Changes, 1: 193–216. — in Russ.

4. Avgustis I., Oloff F. (2022) Bringing smartphone users (back) into the conversation. 17th International Pragmatics Conference, Panel “Technology use in social interaction: enabling vs. constraining participation”, 27 June — 2 July, Winterthur, Switzerland (online).

5. Brassac C., Fixmer P., Mondada L., Vinck D. (2008) Interweaving Objects, Gestures, and Talk in Context. Mind, Culture, and Activity, 15(3): 208–233.

6. Caronia L., Cooren F. (2014) Decentering our analytical position: The dialogicity of things. Discourse & Communication, 8(1): 41–61.

7. Caronia L., Mortari L. (2015) The agency of things: how spaces and artefacts organize the moral order of an intensive care unit. Social Semiotics, 25(4): 401–422.

8. Cetina K. K. (1997) Sociality with Objects. Theory, Culture & Society, 14(4): 1–30.

9. Day D., Wagner J. (2014) Objects as tools for talk. M. Nevile, P. Haddington, T. Heinemann, M. Rauniomaa (eds.) Interacting with objects: Language, materiality, and social activity, Amsterdam, Philadelphia: John Benjamins Publishing Company.

10. Erofeeva M. (2021) Achieving Intersubjectivity in Asymmetric Interaction. The Journal of Social Policy Studies, 19(4): 635–650.

11. Fox B. A., Heinemann T. (2015) The Alignment of Manual and Verbal Displays in Requests for the Repair of an Object. Research on Language and Social Interaction, 48(3): 342–362.

12. Gibson J. J. (1986) The ecological approach to visual perception. Hillsdale, NJ: Erlbaum.

13. Goffman E. (1981) Forms of talk. University of Pennsylvania publications in conduct and communication, Philadelphia: University of Pennsylvania Press.

14. Goode D. (1994) A World without Words: The Social Construction of Children Born Deaf and Blind, Philadelphia: Temple University Press.

15. Goodwin C. (1981) Conversational organization: Interaction between speakers and hearers. Language, thought, and culture, New York, London: Academic Press.

16. Goodwin C. (1995) Co-Constructing Meaning in Conversations With an Aphasie Man. Research on Language and Social Interaction, 28(3): 233–260.

17. Goodwin C. (2000) Action and embodiment within situated human interaction. Journal of Pragmatics, 32(10): 1489–1522.

18. Goodwin C. (2018) Co-operative action. Learning in doing, Cambridge: Cambridge University Press.

19. Helping Hands mission statement (n.d). Retrieved from. https://docs.google.com/document/d/1KFTiPRlER_Lf0aIbqfHDOWGqsl9aoyY2DGzeoqz5zD0/edit

20. Hutchby I. (2001) Conversation and technology: From the telephone to the Internet, Cambridge: Polity.

21. Kendon A. (2009) Conducting interaction: Patterns of behavior in focused encounters (Transf. to digital printing). Studies in interactional sociolinguistics. Vol. 7. Cambridge, New York: Cambridge Univ. Press.

22. Klowait N. O. (2019) Interactionism in the age of ubiquitous telecommunication. Information, Communication & Society, 22(5): 605–621.

23. Cetina K. K. (2009) The Synthetic Situation: Interactionism for a Global World. Symbolic Interaction, 32: 61–87.

24. Latour B. (1996) On Interobjectivity. Mind, Culture, and Activity, 3(4): 228–245.

25. Latour B. (1994) On technical mediation. Common Knowledge, 3(2): 29–64.

26. Latour B. (2016) How Better to Register the Agency of Things. M. Matheson (eds.) The Tanner lectures on human values, Salt Lake City, Utah: University of Utah Press: 79–117. Latour B. (1993) We have never been modern, Cambridge, Mass.: Harvard University Press.

27. Licoppe C., Luff P. K., Heath C., Kuzuoka H., Yamashita N., Tuncer S. (2017) Showing Objects. G. Mark, S. Fussell, C. Lampe, m. Schraefel, J P. Hourcade, C. Appert, D. Wigdor (eds.) Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, New York, USA: ACM: 5295–5306.

28. Lindström J. K., Norrby C., Wide C., Nilsson J. (2017) Intersubjectivity at the counter: Artefacts and multimodal interaction in theatre box office encounters. Journal of Pragmatics, 108: 81–97.

29. Luff P., Heath C., Kuzuoka H., Hindmarsh J., Yamazak, K., Oyama S. (2003) Fractured Ecologies: Creating Environments for Collaboration. Human–Computer Interaction, 18(1-2): 51–84.

30. Mondada L. (2016) Challenges of multimodality: Language and the body in social interaction. Journal of Sociolinguistics, 20(3): 336–366.

31. Mondada L. (2019) Contemporary issues in conversation analysis: Embodiment and materiality, multimodality and multisensoriality in social interaction. Journal of Pragmatics, 145: 47–62.

32. Mondada L. (2001) Conventions for multimodal transcription. https://www.lorenzamondada.net/multimodal-transcription

33. Nevile M. (2015) The Embodied Turn in Research on Language and Social Interaction. Research on Language and Social Interaction, 48(2): 121–151.

34. Nevile M., Haddington P., Heinemann T., Rauniomaa M. (2014) Interacting with objects: Language, materiality, and social activity, Amsterdam, Philadelphia: John Benjamins Publishing Company.

35. Pels D., Hetherington K., Vandenberghe F. (2002) The Status of the Object. Theory, Culture & Society, 19(5-6): 1–21.

36. Pinch T. J., Bijker W. E. (1984) The Social Construction of Facts and Artefacts: Or How the Sociology of Science and the Sociology of Technology Might Benefit Each Other. Social Studies of Science, 14: 399–441.

37. Schegloff E. A. (2007) Sequence Organization in Interaction, Cambridge: Cambridge University Press.

38. Seuren L. M., Wherton J., Greenhalgh T., Shaw S. E. (2021) Whose turn is it anyway? Latencymatics, 172: 63–78.