publications

2021

  • T. Draws, Z. Szlávik, B. Timmermans, N. Tintarev, K. R. Varshney, and M. Hind, “Disparate Impact Diminishes Consumer Trust Even for Advantaged Users,” in Persuasive Technology, Cham, 2021, p. 135–149. doi:10.1007/978-3-030-79460-6_11
    [BibTeX] [Abstract] [Download PDF]

    Systems aiming to aid consumers in their decision-making (e.g., by implementing persuasive techniques) are more likely to be effective when consumers trust them. However, recent research has demonstrated that the machine learning algorithms that often underlie such technology can act unfairly towards specific groups (e.g., by making more favorable predictions for men than for women). An undesired disparate impact resulting from this kind of algorithmic unfairness could diminish consumer trust and thereby undermine the purpose of the system. We studied this effect by conducting a between-subjects user study investigating how (gender-related) disparate impact affected consumer trust in an app designed to improve consumers’ financial decision-making. Our results show that disparate impact decreased consumers’ trust in the system and made them less likely to use it. Moreover, we find that trust was affected to the same degree across consumer groups (i.e., advantaged and disadvantaged users) despite both of these consumer groups recognizing their respective levels of personal benefit. Our findings highlight the importance of fairness in consumer-oriented artificial intelligence systems.

    @inproceedings{Draws2021,
    abstract = {Systems aiming to aid consumers in their decision-making (e.g., by implementing persuasive techniques) are more likely to be effective when consumers trust them. However, recent research has demonstrated that the machine learning algorithms that often underlie such technology can act unfairly towards specific groups (e.g., by making more favorable predictions for men than for women). An undesired disparate impact resulting from this kind of algorithmic unfairness could diminish consumer trust and thereby undermine the purpose of the system. We studied this effect by conducting a between-subjects user study investigating how (gender-related) disparate impact affected consumer trust in an app designed to improve consumers' financial decision-making. Our results show that disparate impact decreased consumers' trust in the system and made them less likely to use it. Moreover, we find that trust was affected to the same degree across consumer groups (i.e., advantaged and disadvantaged users) despite both of these consumer groups recognizing their respective levels of personal benefit. Our findings highlight the importance of fairness in consumer-oriented artificial intelligence systems.},
    address = {Cham},
    author = {Draws, Tim and Szl{\'{a}}vik, Zolt{\'{a}}n and Timmermans, Benjamin and Tintarev, Nava and Varshney, Kush R. and Hind, Michael},
    booktitle = {Persuasive Technology},
    doi = {10.1007/978-3-030-79460-6_11},
    editor = {Ali, Raian and Lugrin, Birgit and Charles, Fred},
    file = {:Users/tim/Surfdrive/Research/Library/Literature/Draws et al. - 2021 - Disparate Impact Diminishes Consumer Trust Even for Advantaged Users.pdf:pdf},
    keywords = {algorithmic fairness,consumer trust,disparate impact},
    pages = {135--149},
    publisher = {Springer International Publishing},
    title = {{Disparate Impact Diminishes Consumer Trust Even for Advantaged Users}},
    url = {http://timdraws.net/files/papers/Disparate_impact_diminishes_consumer_trust_even_for_advantaged_users.pdf},
    year = {2021}
    }

  • T. Draws, N. Tintarev, U. Gadiraju, A. Bozzon, and B. Timmermans, “Assessing Viewpoint Diversity in Search Results Using Ranking Fairness Metrics,” ACM SIGKDD Explorations Newsletter, vol. 23, iss. 1, p. 50–58, 2021. doi:10.1145/3468507.3468515
    [BibTeX] [Abstract] [Download PDF]

    The way pages are ranked in search results influences whether the users of search engines are exposed to more homogeneous, or rather to more diverse viewpoints. However, this viewpoint diversity is not trivial to assess. In this paper we use existing and novel ranking fairness metrics to evaluate viewpoint diversity in search result rankings. We conduct a controlled simulation study that shows how ranking fairness metrics can be used for viewpoint diversity, how their outcome should be interpreted, and which metric is most suitable depending on the situation. This paper lays out important ground work for future research to measure and assess viewpoint diversity in real search result rankings.

    @article{Draws2020assessing,
    abstract = {The way pages are ranked in search results influences whether the users of search engines are exposed to more homogeneous, or rather to more diverse viewpoints. However, this viewpoint diversity is not trivial to assess. In this paper we use existing and novel ranking fairness metrics to evaluate viewpoint diversity in search result rankings. We conduct a controlled simulation study that shows how ranking fairness metrics can be used for viewpoint diversity, how their outcome should be interpreted, and which metric is most suitable depending on the situation. This paper lays out important ground work for future research to measure and assess viewpoint diversity in real search result rankings.},
    author = {Draws, Tim and Tintarev, Nava and Gadiraju, Ujwal and Bozzon, Alessandro and Timmermans, Benjamin},
    doi = {10.1145/3468507.3468515},
    file = {:Users/tim/Surfdrive/Research/Library/Literature/Draws et al. - 2021 - Assessing Viewpoint Diversity in Search Results Using Ranking Fairness Metrics.pdf:pdf},
    journal = {ACM SIGKDD Explorations Newsletter},
    keywords = {ranking fairness,viewpoint diversity,web search},
    number = {1},
    pages = {50--58},
    title = {{Assessing Viewpoint Diversity in Search Results Using Ranking Fairness Metrics}},
    url = {http://timdraws.net/files/papers/Assessing_Viewpoint_Diversity_in_Search_Results_Using_Ranking_Fairness_Metrics.pdf},
    volume = {23},
    year = {2021}
    }

  • T. Draws, “Understanding How Algorithmic and Cognitive Biases in Web Search Affect User Attitudes on Debated Topics,” in Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval, New York, NY, USA, 2021, p. 2709. doi:10.1145/3404835.3463273
    [BibTeX] [Abstract] [Download PDF]

    Web search increasingly provides a platform for users to seek advice on important personal decisions but may be biased in several different ways. One result of such biases is the search engine manipulation effect (SEME): when a list of search results relates to a debated topic (e.g., veganism) and promotes documents pertaining to a particular viewpoint (e.g., by ranking them higher), users tend to adopt this advantaged viewpoint. However, the detection and mitigation of SEME are complicated by the current lack of empirical understanding of its underlying mechanisms. This dissertation aims to investigate which (and to what degree) algorithmic and cognitive biases play a role in SEME concerning debated topics. RQ1. What set of labels can accurately represent viewpoints of textual documents on debated topics? Studying algorithmic and cognitive biases in the context of web search on debated topics requires accurate labeling of documents. RQ1 investigates how to best represent viewpoints of textual documents on debated topics. The first step in this work was introducing perspectives as an additional dimension of viewpoint labels for textual documents (i.e., adding people’s underlying motivations for taking a given stance) and showing how they can be automatically discovered using Joint Topic Models. My future research will evaluate whether viewpoint labels consisting of stances and perspectives are accurate representations (or whether more nuanced notions are necessary) and describe how to obtain these labels. The work on RQ1 will result in a framework to accurately represent viewpoints on debated topics expressed by textual documents. This will allow for algorithmic assessment of viewpoint-related ranking bias in search results and alignment of document viewpoints with users’ viewpoints. RQ2. What methods can automatically measure viewpoint-related ranking bias in search results? Several methods have been proposed to measure ranking bias, fairness, and diversity in search results. RQ2 investigates which of these (or novel) methods can be used to assess viewpoint-related ranking bias. The first contribution to RQ2 was demonstrating how to assess viewpoint-related ranking bias in search results using ranking fairness metrics for categorical viewpoint labels and evaluated which specific methods work best in which situation. Going forward, I plan to develop methods that assess viewpoint-related ranking bias in more complex settings. Furthermore, I aim to assess viewpoint-related ranking bias in real search results on debated topics. This work will contribute novel evaluation metrics that measure viewpoint-related ranking bias in search results, a set of guidelines for when and how to use them using a web-based demo, as well as directions for practitioners regarding viewpoint-related ranking bias in real search results. RQ3. What cognitive biases may contribute to the process of attitude change on debated topics in users of web search engines? Being able to measure algorithmic ranking bias is not yet enough to understand its effect on human behavior. RQ3 aims at understanding which specific cognitive biases are responsible for SEME; i.e., what reasoning mistakes users make when they change their attitudes after viewing search results. The first contribution to RQ3 was evaluating in a user study whether order effects alone can cause SEME. We found that this may not be the case and describe exploratory results that show that exposure effects may play a more important role in causing SEME than previously anticipated. My future work in this area will consider findings from RQ1 and RQ2 to draw more realistic scenarios of SEME and study interactions between algorithmic and different cognitive biases. The result of this work will be a set of guidelines for how SEME could be avoided by mitigating cognitive user biases in web search.

    @inproceedings{Draws2020SIGIRDC,
    abstract = {Web search increasingly provides a platform for users to seek advice on important personal decisions but may be biased in several different ways. One result of such biases is the search engine manipulation effect (SEME): when a list of search results relates to a debated topic (e.g., veganism) and promotes documents pertaining to a particular viewpoint (e.g., by ranking them higher), users tend to adopt this advantaged viewpoint. However, the detection and mitigation of SEME are complicated by the current lack of empirical understanding of its underlying mechanisms. This dissertation aims to investigate which (and to what degree) algorithmic and cognitive biases play a role in SEME concerning debated topics. RQ1. What set of labels can accurately represent viewpoints of textual documents on debated topics? Studying algorithmic and cognitive biases in the context of web search on debated topics requires accurate labeling of documents. RQ1 investigates how to best represent viewpoints of textual documents on debated topics. The first step in this work was introducing perspectives as an additional dimension of viewpoint labels for textual documents (i.e., adding people's underlying motivations for taking a given stance) and showing how they can be automatically discovered using Joint Topic Models. My future research will evaluate whether viewpoint labels consisting of stances and perspectives are accurate representations (or whether more nuanced notions are necessary) and describe how to obtain these labels. The work on RQ1 will result in a framework to accurately represent viewpoints on debated topics expressed by textual documents. This will allow for algorithmic assessment of viewpoint-related ranking bias in search results and alignment of document viewpoints with users' viewpoints. RQ2. What methods can automatically measure viewpoint-related ranking bias in search results? Several methods have been proposed to measure ranking bias, fairness, and diversity in search results. RQ2 investigates which of these (or novel) methods can be used to assess viewpoint-related ranking bias. The first contribution to RQ2 was demonstrating how to assess viewpoint-related ranking bias in search results using ranking fairness metrics for categorical viewpoint labels and evaluated which specific methods work best in which situation. Going forward, I plan to develop methods that assess viewpoint-related ranking bias in more complex settings. Furthermore, I aim to assess viewpoint-related ranking bias in real search results on debated topics. This work will contribute novel evaluation metrics that measure viewpoint-related ranking bias in search results, a set of guidelines for when and how to use them using a web-based demo, as well as directions for practitioners regarding viewpoint-related ranking bias in real search results. RQ3. What cognitive biases may contribute to the process of attitude change on debated topics in users of web search engines? Being able to measure algorithmic ranking bias is not yet enough to understand its effect on human behavior. RQ3 aims at understanding which specific cognitive biases are responsible for SEME; i.e., what reasoning mistakes users make when they change their attitudes after viewing search results. The first contribution to RQ3 was evaluating in a user study whether order effects alone can cause SEME. We found that this may not be the case and describe exploratory results that show that exposure effects may play a more important role in causing SEME than previously anticipated. My future work in this area will consider findings from RQ1 and RQ2 to draw more realistic scenarios of SEME and study interactions between algorithmic and different cognitive biases. The result of this work will be a set of guidelines for how SEME could be avoided by mitigating cognitive user biases in web search.},
    address = {New York, NY, USA},
    author = {Draws, Tim},
    booktitle = {Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval},
    doi = {10.1145/3404835.3463273},
    file = {:Users/tim/Surfdrive/Research/Library/Literature/Draws - 2021 - Understanding How Algorithmic and Cognitive Biases in Web Search Affect User Attitudes on Debated Topics.pdf:pdf},
    keywords = {debated topics,joint topic models,perspective discovery,sentiment analysis,topic modeling},
    pages = {2709},
    publisher = {Association for Computing Machinery},
    series = {SIGIR '21},
    title = {{Understanding How Algorithmic and Cognitive Biases in Web Search Affect User Attitudes on Debated Topics}},
    url = {http://timdraws.net/files/papers/Understanding_How_Algorithmic_and_Cognitive_Biases_in_Web_Search_Affect_User_Attitudes_on_Debated_Topics.pdf},
    year = {2021}
    }

  • T. Draws, N. Tintarev, U. Gadiraju, A. Bozzon, and B. Timmermans, “This Is Not What We Ordered: Exploring Why Biased Search Result Rankings Affect User Attitudes on Debated Topics,” in Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval, New York, NY, USA, 2021, p. 295–305. doi:10.1145/3404835.3462851
    [BibTeX] [Abstract] [Download PDF]

    In web search on debated topics, algorithmic and cognitive biases strongly influence how users consume and process information. Recent research has shown that this can lead to a search engine manipulation effect (SEME): when search result rankings are biased towards a particular viewpoint, users tend to adopt this favored viewpoint. To better understand the mechanisms underlying SEME, we present a pre-registered, 5 × 3 factorial user study investigating whether order effects (i.e., users adopting the viewpoint pertaining to higher-ranked documents) can cause SEME. For five different debated topics, we evaluated attitude change after exposing participants with mild pre-existing attitudes to search results that were overall viewpoint-balanced but reflected one of three levels of algorithmic ranking bias. We found that attitude change did not differ across levels of ranking bias and did not vary based on individual user differences. Our results thus suggest that order effects may not be an underlying mechanism of SEME. Exploratory analyses lend support to the presence of exposure effects (i.e., users adopting the majority viewpoint among the results they examine) as a contributing factor to users’ attitude change. We discuss how our findings can inform the design of user bias mitigation strategies. CCS

    @inproceedings{Draws2021This,
    abstract = {In web search on debated topics, algorithmic and cognitive biases strongly influence how users consume and process information. Recent research has shown that this can lead to a search engine manipulation effect (SEME): when search result rankings are biased towards a particular viewpoint, users tend to adopt this favored viewpoint. To better understand the mechanisms underlying SEME, we present a pre-registered, 5 × 3 factorial user study investigating whether order effects (i.e., users adopting the viewpoint pertaining to higher-ranked documents) can cause SEME. For five different debated topics, we evaluated attitude change after exposing participants with mild pre-existing attitudes to search results that were overall viewpoint-balanced but reflected one of three levels of algorithmic ranking bias. We found that attitude change did not differ across levels of ranking bias and did not vary based on individual user differences. Our results thus suggest that order effects may not be an underlying mechanism of SEME. Exploratory analyses lend support to the presence of exposure effects (i.e., users adopting the majority viewpoint among the results they examine) as a contributing factor to users' attitude change. We discuss how our findings can inform the design of user bias mitigation strategies. CCS},
    address = {New York, NY, USA},
    author = {Draws, Tim and Tintarev, Nava and Gadiraju, Ujwal and Bozzon, Alessandro and Timmermans, Benjamin},
    booktitle = {Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval},
    doi = {10.1145/3404835.3462851},
    file = {:Users/tim/Surfdrive/Research/Library/Literature//Draws et al. - 2021 - This Is Not What We Ordered Exploring Why Biased Search Result Rankings Affect User Attitudes on Debated Topics.pdf:pdf},
    isbn = {9781450380379},
    keywords = {user attitudes,user-centered evaluation,web search},
    pages = {295--305},
    publisher = {Association for Computing Machinery},
    series = {SIGIR '21},
    title = {{This Is Not What We Ordered: Exploring Why Biased Search Result Rankings Affect User Attitudes on Debated Topics}},
    url = {https://timdraws.net/files/papers/This_Is_Not_What_We_Ordered__Exploring_Why_Biased_Search_Result_Rankings_Affect_User_Attitudes_on_Debated_Topics.pdf},
    year = {2021}
    }

  • F. Giunchiglia, S. Kleanthous, J. Otterbacher, and T. Draws, “Transparency Paths – Documenting the Diversity of User Perceptions,” in Adjunct Proceedings of the 29th ACM Conference on User Modeling, Adaptation and Personalization, New York, NY, USA, 2021, p. 415–420. doi:10.1145/3450614.3463292
    [BibTeX] [Abstract] [Download PDF]

    We are living in an era of global digital platforms, eco-systems of algorithmic processes that serve users worldwide. However, the increasing exposure to diversity online – of information and users – has led to important considerations of bias. A given platform, such as the Google search engine, may demonstrate behaviors that deviate from what users expect, or what they consider fair, relative to their own context and experiences. In this exploratory work, we put forward the notion of transparency paths, a process by which we document our position, choices, and perceptions when developing and/or using algorithmic platforms. We conducted a self-reflection exercise with seven researchers, who collected and analyzed two sets of images; one depicting an everyday activity, “washing hands,” and a second depicting the concept of “home.” Participants had to document their process and choices, and in the end, compare their work to others. Finally, participants were asked to reflect on the definitions of bias and diversity. The exercise revealed the range of perspectives and approaches taken, underscoring the need for future work that will refine the transparency paths methodology.

    @inproceedings{Giunchiglia2021,
    abstract = {We are living in an era of global digital platforms, eco-systems of algorithmic processes that serve users worldwide. However, the increasing exposure to diversity online – of information and users – has led to important considerations of bias. A given platform, such as the Google search engine, may demonstrate behaviors that deviate from what users expect, or what they consider fair, relative to their own context and experiences. In this exploratory work, we put forward the notion of transparency paths, a process by which we document our position, choices, and perceptions when developing and/or using algorithmic platforms. We conducted a self-reflection exercise with seven researchers, who collected and analyzed two sets of images; one depicting an everyday activity, “washing hands,” and a second depicting the concept of “home.” Participants had to document their process and choices, and in the end, compare their work to others. Finally, participants were asked to reflect on the definitions of bias and diversity. The exercise revealed the range of perspectives and approaches taken, underscoring the need for future work that will refine the transparency paths methodology.},
    address = {New York, NY, USA},
    author = {Giunchiglia, Fausto and Kleanthous, Styliani and Otterbacher, Jahna and Draws, Tim},
    booktitle = {Adjunct Proceedings of the 29th ACM Conference on User Modeling, Adaptation and Personalization},
    doi = {10.1145/3450614.3463292},
    file = {:Users/tim/Surfdrive/Research/Library/Literature/Giunchiglia et al. - 2021 - Transparency Paths - Documenting the Diversity of User Perceptions.pdf:pdf},
    isbn = {9781450383677},
    pages = {415--420},
    publisher = {Association for Computing Machinery},
    series = {UMAP '21},
    title = {{Transparency Paths - Documenting the Diversity of User Perceptions}},
    url = {http://timdraws.net/files/papers/Transparency Paths - Documenting the Diversity of User Perceptions.pdf},
    year = {2021}
    }

2020

  • C. J. van Lissa, W. Stroebe, P. Leander, M. Agostini, B. Gutzkow, J. Kreienkamp, J. Belanger, T. Draws, A. Grygoryshyn, C. S. Vetter, and P. Collaboration, “Early Indicators of COVID-19 Infection Prevention Behaviors: Machine Learning Identifies Personal and Country-Level Factors,” PsyArxiv Preprint, 2020.
    [BibTeX] [Abstract] [Download PDF]

    The Coronavirus is highly infectious and potentially deadly. In the absence of a cure or a vaccine, the infection prevention behaviors recommended by the World Health Organization constitute the only measure that is presently available to combat the pandemic. The unprecedented impact of this pandemic calls for swift identification of factors most important for predicting infection prevention behavior. In this paper, we used a machine learning approach to assess the relative importance of potential indicators of personal infection prevention behavior in a global psychological survey we conducted between March-May 2020 (N = 56,072 across 28 countries). The survey data were enriched with society-level variables relevant to the pandemic. Results indicated that the two most important indicators of self-reported infection prevention behavior were individual-level injunctive norms—beliefs that people in the community should engage in social distancing and self-isolation, followed by endorsement of restrictive containment measures (e.g., mandatory vaccination). Society-level factors (e.g., national healthcare infrastructure, confirmed infections) also emerged as important indicators. Social attitudes and norms were more important than personal factors considered most important by theories of health behavior. The model accounted for 52% of the variance in infection prevention behavior in a separate test sample—above the performance of psychological models of health behavior. These results suggest that individuals are intuitively aware that this pandemic constitutes a social dilemma situation, where their own infection risk is partly dependent on the behaviors of others. If everybody engaged in infection prevention behavior, the virus could be defeated even without a vaccine.

    @article{vanLissa2020early,
    abstract = {The Coronavirus is highly infectious and potentially deadly. In the absence of a cure or a vaccine, the infection prevention behaviors recommended by the World Health Organization constitute the only measure that is presently available to combat the pandemic. The unprecedented impact of this pandemic calls for swift identification of factors most important for predicting infection prevention behavior. In this paper, we used a machine learning approach to assess the relative importance of potential indicators of personal infection prevention behavior in a global psychological survey we conducted between March-May 2020 (N = 56,072 across 28 countries). The survey data were enriched with society-level variables relevant to the pandemic. Results indicated that the two most important indicators of self-reported infection prevention behavior were individual-level injunctive norms—beliefs that people in the community should engage in social distancing and self-isolation, followed by endorsement of restrictive containment measures (e.g., mandatory vaccination). Society-level factors (e.g., national healthcare infrastructure, confirmed infections) also emerged as important indicators. Social attitudes and norms were more important than personal factors considered most important by theories of health behavior. The model accounted for 52% of the variance in infection prevention behavior in a separate test sample—above the performance of psychological models of health behavior. These results suggest that individuals are intuitively aware that this pandemic constitutes a social dilemma situation, where their own infection risk is partly dependent on the behaviors of others. If everybody engaged in infection prevention behavior, the virus could be defeated even without a vaccine.},
    author = {van Lissa, Caspar J and Stroebe, Wolfgang and Leander, Pontus and Agostini, Maximilian and Gutzkow, Ben and Kreienkamp, Jannis and Belanger, Jocelyn and Draws, Tim and Grygoryshyn, Andrii and Vetter, Clara S and Collaboration, Psycorona},
    journal = {PsyArxiv Preprint},
    title = {{Early Indicators of COVID-19 Infection Prevention Behaviors: Machine Learning Identifies Personal and Country-Level Factors}},
    url = {https://psyarxiv.com/whjsb/},
    year = {2020}
    }

  • J. van Doorn, D. van den Bergh, U. Böhm, F. Dablander, K. Derks, T. Draws, A. Etz, N. J. Evans, Q. F. Gronau, J. M. Haaf, M. Hinne, Š. Kucharský, A. Ly, M. Marsman, D. Matzke, A. R. N. Gupta, A. Sarafoglou, A. Stefan, J. G. Voelkel, and E. J. Wagenmakers, “The JASP guidelines for conducting and reporting a Bayesian analysis,” Psychonomic Bulletin and Review, p. 1–14, 2020. doi:10.3758/s13423-020-01798-5
    [BibTeX] [Abstract] [Download PDF]

    Despite the increasing popularity of Bayesian inference in empirical research, few practical guidelines provide detailed recommendations for how to apply Bayesian procedures and interpret the results. Here we offer specific guidelines for four different stages of Bayesian statistical reasoning in a research setting: planning the analysis, executing the analysis, interpreting the results, and reporting the results. The guidelines for each stage are illustrated with a running example. Although the guidelines are geared towards analyses performed with the open-source statistical software JASP, most guidelines extend to Bayesian inference in general.

    @article{VanDoorn2020,
    abstract = {Despite the increasing popularity of Bayesian inference in empirical research, few practical guidelines provide detailed recommendations for how to apply Bayesian procedures and interpret the results. Here we offer specific guidelines for four different stages of Bayesian statistical reasoning in a research setting: planning the analysis, executing the analysis, interpreting the results, and reporting the results. The guidelines for each stage are illustrated with a running example. Although the guidelines are geared towards analyses performed with the open-source statistical software JASP, most guidelines extend to Bayesian inference in general.},
    author = {van Doorn, Johnny and van den Bergh, Don and B{\"{o}}hm, Udo and Dablander, Fabian and Derks, Koen and Draws, Tim and Etz, Alexander and Evans, Nathan J. and Gronau, Quentin F. and Haaf, Julia M. and Hinne, Max and Kucharsk{\'{y}}, {\v{S}}imon and Ly, Alexander and Marsman, Maarten and Matzke, Dora and Gupta, Akash R.Komarlu Narendra and Sarafoglou, Alexandra and Stefan, Angelika and Voelkel, Jan G. and Wagenmakers, Eric Jan},
    doi = {10.3758/s13423-020-01798-5},
    file = {:Users/tim/Surfdrive/Research/Library/Literature/van Doorn et al. - 2020 - The JASP guidelines for conducting and reporting a Bayesian analysis.pdf:pdf},
    isbn = {1342302001798},
    issn = {15315320},
    journal = {Psychonomic Bulletin and Review},
    keywords = {Bayesian inference,Scientific reporting,Statistical software},
    pages = {1--14},
    title = {{The JASP guidelines for conducting and reporting a Bayesian analysis}},
    url = {https://ir.cwi.nl/pub/30141/30141.pdf},
    year = {2020}
    }

  • D. van den Bergh, J. van Doorn, M. Marsman, T. Draws, E. J. van Kesteren, K. Derks, F. Dablander, Q. F. Gronau, Š. Kucharský, A. R. N. Gupta, A. Sarafoglou, J. G. Voelkel, A. Stefan, A. Ly, M. Hinne, D. Matzke, and E. J. Wagenmakers, “A Tutorial on Conducting and Interpreting a Bayesian ANOVA in JASP,” Annee Psychologique, vol. 120, iss. 1, p. 73–96, 2020. doi:10.3917/anpsy1.201.0073
    [BibTeX] [Abstract] [Download PDF]

    Analysis of variance (ANOVA) is the standard procedure for statistical inference in factorial designs. Typically, ANOVAs are executed using frequentist statistics, where p-values determine statistical significance in an all-or-none fashion. In recent years, the Bayesian approach to statistics is increasingly viewed as a legitimate alternative to the p-value. However, the broad adoption of Bayesian statistics-and Bayesian ANOVA in particular-is frustrated by the fact that Bayesian concepts are rarely taught in applied statistics courses. Consequently, practitioners may be unsure how to conduct a Bayesian ANOVA and interpret the results. Here we provide a guide for executing and interpreting a Bayesian ANOVA with JASP, an open-source statistical software program with a graphical user interface. We explain the key concepts of the Bayesian ANOVA using two empirical examples.

    @article{VanDenBergh2020,
    abstract = {Analysis of variance (ANOVA) is the standard procedure for statistical inference in factorial designs. Typically, ANOVAs are executed using frequentist statistics, where p-values determine statistical significance in an all-or-none fashion. In recent years, the Bayesian approach to statistics is increasingly viewed as a legitimate alternative to the p-value. However, the broad adoption of Bayesian statistics-and Bayesian ANOVA in particular-is frustrated by the fact that Bayesian concepts are rarely taught in applied statistics courses. Consequently, practitioners may be unsure how to conduct a Bayesian ANOVA and interpret the results. Here we provide a guide for executing and interpreting a Bayesian ANOVA with JASP, an open-source statistical software program with a graphical user interface. We explain the key concepts of the Bayesian ANOVA using two empirical examples.},
    author = {van den Bergh, Don and van Doorn, Johnny and Marsman, Maarten and Draws, Tim and van Kesteren, Erik Jan and Derks, Koen and Dablander, Fabian and Gronau, Quentin F. and Kucharsk{\'{y}}, {\v{S}}imon and Gupta, Akash R.Komarlu Narendra and Sarafoglou, Alexandra and Voelkel, Jan G. and Stefan, Angelika and Ly, Alexander and Hinne, Max and Matzke, Dora and Wagenmakers, Eric Jan},
    doi = {10.3917/anpsy1.201.0073},
    file = {:Users/tim/Surfdrive/Research/Library/Literature/van den Bergh et al. - 2020 - A Tutorial on Conducting and Interpreting a Bayesian ANOVA in JASP.pdf:pdf},
    isbn = {9782130822936},
    issn = {00035033},
    journal = {Annee Psychologique},
    keywords = {Analysis of Variance,Bayes Factor,Hypothesis Test,JASP,Posterior distribution,Tutorial},
    number = {1},
    pages = {73--96},
    title = {{A Tutorial on Conducting and Interpreting a Bayesian ANOVA in JASP}},
    url = {https://pdfs.semanticscholar.org/a79c/bca2076779f20037f87ea23c68bb3a7a2f29.pdf},
    volume = {120},
    year = {2020}
    }

  • T. Draws, J. Liu, and N. Tintarev, “Helping users discover perspectives: Enhancing opinion mining with joint topic models,” in International Conference on Data Mining Workshops (ICDMW), 2020, p. 23–30. doi:10.1109/ICDMW51313.2020.00013
    [BibTeX] [Abstract] [Download PDF]

    Support or opposition with respect to a debated claim such as abortion should be legal can have different underlying reasons, which we call perspectives. This paper explores how opinion mining can be enhanced with joint topic modeling, to identify distinct perspectives within the topic, providing an informative overview from unstructured text. We evaluate four joint topic models (TAM, JST, VODUM, and LAM) in a user study assessing human understandability of the extracted perspectives. Based on the results, we conclude that joint topic models such as TAM can discover perspectives that align with human judgments. Moreover, our results suggest that users are not influenced by their pre-existing stance on the topic of abortion when interpreting the output of topic models.

    @inproceedings{Draws2020helping,
    abstract = {Support or opposition with respect to a debated claim such as abortion should be legal can have different underlying reasons, which we call perspectives. This paper explores how opinion mining can be enhanced with joint topic modeling, to identify distinct perspectives within the topic, providing an informative overview from unstructured text. We evaluate four joint topic models (TAM, JST, VODUM, and LAM) in a user study assessing human understandability of the extracted perspectives. Based on the results, we conclude that joint topic models such as TAM can discover perspectives that align with human judgments. Moreover, our results suggest that users are not influenced by their pre-existing stance on the topic of abortion when interpreting the output of topic models.},
    author = {Draws, Tim and Liu, Jody and Tintarev, Nava},
    booktitle = {International Conference on Data Mining Workshops (ICDMW)},
    doi = {10.1109/ICDMW51313.2020.00013},
    file = {:Users/tim/Surfdrive/Research/Library/Literature/Draws, Liu, Tintarev - 2020 - Helping users discover perspectives Enhancing opinion mining with joint topic models.pdf:pdf},
    keywords = {debated topics,els,joint topic mod-,perspective discovery,sentiment analysis,topic modeling},
    pages = {23--30},
    title = {{Helping users discover perspectives: Enhancing opinion mining with joint topic models}},
    url = {http://timdraws.net/files/papers/Helping_users_discover_perspectives_in_controversial_debates_using_joint_topic_models.pdf},
    year = {2020}
    }

2019

  • A. Sarafoglou, A. van der Heijden, T. Draws, J. Cornelisse, E. Wagenmakers, and M. Marsman, “Combine Statistical Thinking With Scientific Practice: A Protocol of a Bayesian Thesis Project For Undergraduate Students,” arXiv preprint arXiv:1810.07496, 2019.
    [BibTeX] [Abstract] [Download PDF]

    Current developments in the statistics community suggest that modern statistics education should be structured holistically, i.e., by allowing students to work with real data and answer concrete statistical questions, but also by educating them about alternative statistical frameworks, such as Bayesian statistics. In this article, we describe how we incorporated such a holistic structure in a Bayesian thesis project on ordered binomial probabilities. The project was targeted at undergraduate students in psychology with basic knowledge in Bayesian statistics and programming, but no formal mathematical training. The thesis project aimed to (1) convey the basic mathematical concepts of Bayesian inference, (2) let students experience the entire empirical cycle including the collection, analysis, and interpretation of data, and (3) teach students open science practices.

    @article{Sarafoglou2019,
    abstract = {Current developments in the statistics community suggest that modern statistics education should be structured holistically, i.e., by allowing students to work with real data and answer concrete statistical questions, but also by educating them about alternative statistical frameworks, such as Bayesian statistics. In this article, we describe how we incorporated such a holistic structure in a Bayesian thesis project on ordered binomial probabilities. The project was targeted at undergraduate students in psychology with basic knowledge in Bayesian statistics and programming, but no formal mathematical training. The thesis project aimed to (1) convey the basic mathematical concepts of Bayesian inference, (2) let students experience the entire empirical cycle including the collection, analysis, and interpretation of data, and (3) teach students open science practices.},
    archivePrefix = {arXiv},
    arxivId = {1810.07496},
    author = {Sarafoglou, Alexandra and van der Heijden, Anna and Draws, Tim and Cornelisse, Joran and Wagenmakers, Eric-Jan and Marsman, Maarten},
    eprint = {1810.07496},
    file = {:Users/tim/Surfdrive/Research/Library/Literature/Sarafoglou et al. - 2019 - Combine Statistical Thinking With Scientific Practice A Protocol of a Bayesian Thesis Project For Undergradua.pdf:pdf},
    journal = {arXiv preprint arXiv:1810.07496},
    title = {{Combine Statistical Thinking With Scientific Practice: A Protocol of a Bayesian Thesis Project For Undergraduate Students}},
    url = {http://arxiv.org/abs/1810.07496},
    year = {2019}
    }

privacy