Down to a r/science: Integrating Computational Approaches to the Study of Credibility on Reddit | Amsterdam University Press Journals Online
2004
Volume 3, Issue 1
  • E-ISSN: 2665-9085

Abstract

Abstract

Digital trace data enable researchers to study communication processes at a scale previously impossible. We combine social network analysis and automated content analysis to examine source and message factors’ impact on ratings of user-shared content. We found that the expertise of the author, the network position that the author occupies, and characteristics of the content the author creates have a significant impact on how others respond to that content. By observationally examining a large-scale online community, we provide a real-world test of how message consumers react to source and message characteristics. Our results show that it is important to think of online communication as occurring interactively between networks of individuals, and that the network positions people inhabit may inform their behavior.

Loading

Article metrics loading...

/content/journals/10.5117/CCR2021.1.004.HUBN
2021-03-01
2024-03-28
Loading full text...

Full text loading...

/deliver/fulltext/26659085/3/1/04_CCR2021.1_HUBN.html?itemId=/content/journals/10.5117/CCR2021.1.004.HUBN&mimeType=html&fmt=ahah

References

  1. Alexa(2019). The top 500 sites on the web. Retrieved from www.alexa.com/topsites
    [Google Scholar]
  2. Anderson, A. A., Brossard, D., Scheufele, D. A., Xenos, M. A., & Ladwig, P.(2014). The “nasty effect”: Online incivility and risk perceptions of emerging technologies. Journal of Computer-Mediated Communication, 19(3), 373-387. doi:10.1111/jcc4.12009
    [Google Scholar]
  3. Arya, D. J., Hiebert, E. H., & Pearson, P. D.(2011). The effects of syntactic and lexical complexity on the comprehension of elementary science texts. International Electronic Journal of Elementary Education, 4(1), 107-125. Retrieved from https://www.iejee.com/index.php/IEJEE/article/view/216
    [Google Scholar]
  4. Barthel, M., Mitchell, A., & Holcomb, J. (2016a, December 15). Many Americans believe fake news is sowing confusion.Pew Research Center. Retrieved from https://www.journalism.org/wp-content/uploads/sites/8/2016/12/PJ_2016.12.15_fake-news_FINAL.pdf
    [Google Scholar]
  5. Barthel, M., Stocking, G., Holcomb, J., & Mitchell, A. (2016b, February 25). Seven-in-ten Reddit users get their news on the site. Pew Research Center. Retrieved from https://www.journalism.org/wp-content/uploads/sites/8/2016/05/PJ_2016.05.26_Reddit_Final_Update.pdf
    [Google Scholar]
  6. Brin, S., & Page, L.(1998). The anatomy of a large-scale hypertextual web search engine. Computer Networks and ISDN Systems, 30(1-7), 107-117. doi:10.1016/S0169‑7552(98)00110‑X
    [Google Scholar]
  7. Bromme, R., Kienhues, D., & Porsch, T.(2010). Who knows what and who can we believe? Epistemological beliefs are beliefs about knowledge (mostly) to be attained from others. In L. D.Bendixen & F. C.Feucht (Eds.), Personal epistemology in the classroom: Theory, research, and implications for practice. Cambridge, UK: Cambridge University Press.
    [Google Scholar]
  8. Brossard, D.(2013). New media landscapes and the science information consumer. Proceedings of the National Academy of Sciences, 110(Supplement 3), 14096-14101. doi:10.1073/pnas.1212744110
    [Google Scholar]
  9. Bullock, O. M., Amill, D. C., Shulman, H. C., & Dixon, G. N.(2019). Jargon as a barrier to effective science communication: Evidence from metacognition. Public Understanding of Science, 28(7), 845-853. doi:10.1177/0963662519865687
    [Google Scholar]
  10. Cenci, W. (2016, October 10). Why scientists should learn to love Reddit. The Guardian. Retrieved from https://www.theguardian.com/higher-education-network/2016/oct/10/why-scientists-should-learn-to-love-reddit
    [Google Scholar]
  11. Chaiken, S.(1980). Heuristic versus systematic information processing and the use of source versus message cues in persuasion. Journal of Personality and Social Psychology, 39(5), 752-766. doi:10.1037/0022‑3514.39.5.752
    [Google Scholar]
  12. Cody, E. M., Reagan, A. J., Mitchell, L., Dodds, P. S., & Danforth, C. M.(2015). Climate change sentiment on Twitter: An unsolicited public opinion poll. PLoS ONE, 10(8). doi:10.1371/journal.pone.0136092
    [Google Scholar]
  13. Cooper, J., Bennett, E. A., & Sukel, H. L.(1996). Complex scientific testimony: How do jurors make decisions. Law and Human Behavior, 20(4), 379-394. doi:10.1007/BF01498976
    [Google Scholar]
  14. Csárdi, G.(2019). The igraph package for complex network research (Version 1.2.4.1) [Computer software]. Retrieved from https://cran.r-project.org/web/packages/igraph/
    [Google Scholar]
  15. Eastin, M. S.(2006). Credibility assessments of online health information: The effects of source expertise and knowledge of content. Journal of Computer-Mediated Communication, 6(4). doi:10.1111/j.1083‑6101.2001.tb00126.x
    [Google Scholar]
  16. Edwards, C., Edwards, A., Qing, Q., & Wahl, S. T.(2007). The influence of computer-mediated word-of-mouth communication on student perceptions of instructors and attitudes toward learning course content. Communication Education, 56(3), 255-277. doi:10.1080/03634520701236866
    [Google Scholar]
  17. Epstein, S.(1995). The construction of lay expertise: AIDS activism and the forging of credibility in the reform of clinical trials. Science, Technology, & Human Values, 20(4), 408-437. doi:10.1177/016224399502000402
    [Google Scholar]
  18. Eysenbach, G., & Köhler, C.(2003). What is the prevalence of health-related searches on the World Wide Web? Qualitative and quantitative analysis of search engine queries on the internet. AMIA Annual Symposium Proceedings Archive, 2003, 225-229.
    [Google Scholar]
  19. Faraj, S., & Johnson, S. L.(2011). Network exchange patterns in online communities. Organization Science, 22(6), 1464-1480. doi:10.1287/orsc.1100.0600
    [Google Scholar]
  20. Flesch, R.(1948). A new readability yardstick. Journal of Applied Psychology, 32(3), 221-233. doi:10.1037/h0057532
    [Google Scholar]
  21. Fogg, B. J.(1999). Persuasive technologies. Communications of the ACM, 42(5), 27-29. doi:10.1145/301353.301396
    [Google Scholar]
  22. Freeman, L. C.(1978). Centrality in social networks conceptual clarification. Social Networks, 1(3), 215-239. doi:10.1016/0378‑8733(78)90021‑7
    [Google Scholar]
  23. Galegher, J., Sproull, L., & Kiesler, S.(1998). Legitimacy, authority, and community in electronic support groups. Written Communication, 15(4), 493-530. doi:10.1177/0741088398015004003
    [Google Scholar]
  24. Haard, J., Slater, M. D., & Long, M.(2004). Scientese and ambiguous citations in the selling of unproven medical treatments. Health Communication, 16(4), 411-426. doi:10.1207/s15327027hc1604_2
    [Google Scholar]
  25. Hartley, J., & Cabanac, G.(2016). Are two authors better than one? Can writing in pairs affect the readability of academic blogs?Scientometrics, 109(3), 2119-2112. doi:10.1007/s11192‑016‑2116‑x
    [Google Scholar]
  26. Hayes, R. A., Carr, C. T., & Wohn, D. Y.(2016). One click, many meanings: Interpreting paralinguistic digital affordances in social media. Journal of Broadcasting & Electronic Media, 60(1), 171-187.
    [Google Scholar]
  27. Hilligoss, B., & Rieh, S. Y.(2008). Developing a unifying framework of credibility assessment: Construct, heuristics, and interaction in context. Information Processing Management, 44(4), 1467-1484. doi:10.1016/j.ipm.2007.10.001
    [Google Scholar]
  28. Hovland, C. I., Janis, I. L., & Kelley, H. H.(1953). Communication and technology: Psychological studies on opinion change. New Haven, CT: Yale University Press.
    [Google Scholar]
  29. Kincaid, J. P., Fishburne, R. P., Rogers, R. L., & Chissom, B. S.(1975). Derivation of new readability formulas (automated readability index, fog count and flesch reading ease formula) for navy enlisted personnel. No. RBR-8-75. Millington, TN: Naval Technical Training Command, Research Branch.
    [Google Scholar]
  30. Lazer, D., Pentland, A., Adamic, L., Aral, S., Barabasi, A.-L., Brewer, D., … & van Alstyne, M.(2009). Computational social science. Science, 323(5915), 721-723. doi:10.1126/science.1167742
    [Google Scholar]
  31. Leavitt, A., & Robinson, J. J.(2017). Upvote my news: The practices of peer information aggregation for breaking news on reddit. com. Proceedings of the ACM on Human-Computer Interaction, 1(CSCW), 1-18.
    [Google Scholar]
  32. Liang, K.-Y., & Zeger, S. L.(1986). Longitudinal data analysis using generalized linear models. Biometrika, 73(1), 13-22. doi:10.2307/2336267
    [Google Scholar]
  33. Liang, X., Su, L. Y.-F., Yeo, S. K., Scheufele, D. A., Brossard, D., Xenos, M., … & Corley, E. A.(2014). Building buzz: (Scientists) communicating science in new media environments. Journalism & Mass Communication Quarterly, 91(4), 772-791. doi:10.1177/1077699014550092
    [Google Scholar]
  34. Jenson, J. D.(2008). Scientific uncertainty in news coverage of cancer research: Effects of hedging on scientists’ and journalists’ credibility. Human Communication Research, 34(3), 347-369. doi:10.1111/j.1468‑2958.2008.00324.x
    [Google Scholar]
  35. Mahmoodi, J., Leckelt, M., van Zalk, M. W. H., Geukes, K., & Back, M. D.(2017). Big data approaches in social and behavioral science: Four key trade-offs and a call for integration. Current Opinion in Behavioral Sciences, 18, 57-62. doi:10.1016/j.cobeha.2017.07.001
    [Google Scholar]
  36. Mason, L., Boldrin, A., & Ariasi, N.(2010). Epistemic metacognition in context: Evaluating and learning online information. Metacognition and Learning, 5(1), 67-90. doi:10.1007/s11409‑009‑9048‑2
    [Google Scholar]
  37. McGinnies, E., & Ward, C. D.(1980). Better liked than right: Trustworthiness and expertise as factors in credibility. Personality and Social Psychology Bulletin, 6(3), 467-472. doi:10.1177/014616728063023
    [Google Scholar]
  38. Metzger, M. J., & Flanagin, A. J.(2013). Credibility and trust of information in online environments: The use of cognitive heuristics. Journal of Pragmatics, 59(B), 210-220. doi:10.1016/j.pragma.2013.07.012
    [Google Scholar]
  39. Metzger, M. J., Flanagin, A. J., & Medders, R. B.(2010). Social and heuristic approaches to credibility evaluation online. Journal of Communication, 60(3), 413-439. doi:10.1111/j.1460‑2466.2010.01488.x
    [Google Scholar]
  40. Michalke, M.(2018). koRpus: An R package for text analysis (Version 0.11-5) [Computer software]. Retrieved from https://cran.r-project.org/web/packages/koRpus/
    [Google Scholar]
  41. Mogull, S. A.(2017). Scientific and medical communication: A guide for effective practice. New York, NY: Routledge.
    [Google Scholar]
  42. Monge, P. R., & Contractor, N. S.(2001). Emergence of communication networks. In F. M.Jablin & L. L.Putnam (Eds.), The New Handbook of Organizational Communication: Advances in Theory. Newbury Park, CA: Sage.
    [Google Scholar]
  43. Monge, P. R., & Contractor, N. S.(2003). Theories of Communication Networks. Oxford, UK: Oxford University Press.
    [Google Scholar]
  44. Morrison, D., & Hayes, C.(2013). Here, have an upvote: Communication behavior and karma on Reddit. INFORMATIK 2013–Informatik angepasst an Mensch, Organisation und Umwelt.
    [Google Scholar]
  45. Mou, Y., Miller, M., & Fu, H.(2015). Evaluating a target on social media: From the self-categorization perspective. Computers in Human Behavior, 49, 451-459. doi:10.1016/j.chb.2015.03.031
    [Google Scholar]
  46. Oppenheimer, D. M.(2006). Consequences of erudite vernacular utilized irrespective of necessity: Problems with using long words needlessly. Applied Cognitive Psychology, 20(2), 139-156. doi:10.1002/acp.1178
    [Google Scholar]
  47. Petty, R. E., & Cacioppo, J. T.(1986). The elaboration likelihood model of persuasion. Advances in Experimental Social Psychology, 19, 123-205. doi:10.1016/S0065‑2601(08)60214‑2
    [Google Scholar]
  48. Pew Research Center (2019, June 12). Social media fact sheet. Retrieved from www.pewinternet.org/fact-sheet/social-media/
    [Google Scholar]
  49. Pornpitakpan, C.(2006). The persuasiveness of source credibility: A critical review of five decades of evidence. Journal of Applied Social Psychology, 34(2), 243-281. doi:10.1111/j.1559‑1816.2004.tb02547.x
    [Google Scholar]
  50. Reddit (n.d.). Science verified user program. Retrieved from https://www.reddit.com/r/science/wiki/flair
    [Google Scholar]
  51. Ruths, D., & Pfeffer, J.(2014). Social media for large studies of behavior. Science, 346(6213), 1063-1064. doi:10.1126/science.346.6213.1063
    [Google Scholar]
  52. Sattelberg, W. (2019, July 22). The demographics of Reddit: Who uses the site?TechJunkie. Retrieved from https://www.techjunkie.com/demographics-reddit/
    [Google Scholar]
  53. See, Y. H. M., Petty, R. E., & Evans, L. M.(2009). The impact of perceived message complexity and need for cognition on information processing and attitudes. Journal of Research in Personality, 43(5), 880-889. doi:10.1016/j.jrp.2009.04.006
    [Google Scholar]
  54. Sharma, R., Wigginton, B., Meurk, C., Ford, P., & Gartner, C. E.(2017). Motivations and limitations associated with vaping among people with mental illness: A qualitative analysis of Reddit discussions. International Journal of Environmental Research and Public Health, 14(1). doi:10.3390/ijerph14010007
    [Google Scholar]
  55. Shearer, E., & Gottfried, J. (2017, September 7). News use across social media platforms 2017. Pew Research Center. Retrieved from www.journalism.org/2017/09/07/news-use-across-social-media-platforms-2017/
    [Google Scholar]
  56. Spartz, J. T., Su, L. Y.-F., Griffin, R., Brossard, D., & Dunwoody, S.(2017). YouTube, social norms and perceived salience of climate change in the American mind. Environmental Communication, 11(1), 1-16. doi:10.1080/17524032.2015.1047887
    [Google Scholar]
  57. Su, L. Y.-F., Akin, H., Brossard, D.Scheufele, D. A., & Xenos, M.(2015). Science news consumption patterns and their implications for the understanding of science. Journalism & Mass Communication Quarterly, 92(3), 597-616. doi:10.1177/1077699015586415
    [Google Scholar]
  58. Suran, M., & Kilgo, D. K.(2017). Freedom from the press? How anonymous gatekeepers on Reddit covered the Boston Marathon bombing. Journalism Studies, 18(8), 1035-1051. doi:10.1080/1461670X.2015.1111160
    [Google Scholar]
  59. Sweitzer, M. D., & Shulman, H. C.(2018). The effects of metacognition in survey research: Experimental, cross-sectional, and content-analytic evidence. Public Opinion Quarterly, 82(4), 745-768. doi:10.1093/poq/nfy034
    [Google Scholar]
  60. Thomm, E., & Bromme, R.(2011). “It should at least seem scientific!” Textual features of “scientificness” and their impact on lay assessments of online information. Science Education, 96(2), 187-211. doi:10.1002/sce.20480
    [Google Scholar]
  61. Tolochko, P., & Boomgaarden, H. G.(2018). Analysis of linguistic complexity in professional and citizen media. Journalism Studies, 19(12), 1786-1803. doi:10.1080/1461670X.2017.1305285
    [Google Scholar]
  62. Wathen, C. N., & Burkell, J.(2002). Believe it or not: Factors influencing source credibility on the web. Journal of the American Society for Information Science and Technology, 53(2), 134-144. doi:10.1002/asi.10016
    [Google Scholar]
  63. Weiner, J. L., & Mowen, J. C.(1985). Source credibility: On the independent effect of trust and expertise. In R. J.Lutz (Ed.), Association for Consumer Research (Vol. 13). Las Vegas, NV: Association for Consumer Research.
    [Google Scholar]
  64. Wickham, H.(2019). stringr: Simple, consistent wrappers for common string operations (Version 1.4.0) [Computer software]. Retrieved from https://cran.r-project.org/web/packages/stringr/
    [Google Scholar]
  65. Winter, S., & Krämer, N. C.(2012). Selecting science information in Web 2.0: How source cues, message sidedness, and need for cognition influence users’ exposure to blog posts. Journal of Computer-Mediated Communication, 18(1), 80-96. doi:10.1111/j.1083‑6101.2012.01596.x
    [Google Scholar]
  66. Zemla, J. C., Sloman, S., Bechlivanidis, C., & Lagnado, D. A.(2017). Evaluating everyday explanations. Psychonometric Bulletin & Review, 24(5), 1488-1500. doi:10.3758/s13423‑017‑1258‑z
    [Google Scholar]
http://instance.metastore.ingenta.com/content/journals/10.5117/CCR2021.1.004.HUBN
Loading
/content/journals/10.5117/CCR2021.1.004.HUBN
Loading

Data & Media loading...

This is a required field
Please enter a valid email address
Approval was a Success
Invalid data
An Error Occurred
Approval was partially successful, following selected items could not be processed due to error