publications

Publications

2021

  • T. Draws, Z. Szlávik, B. Timmermans, N. Tintarev, K. R. Varshney, and M. Hind, “Disparate Impact Diminishes Consumer Trust Even for Advantaged Users,” in Persuasive, 2021.
    [BibTeX] [Abstract] [Download PDF]

    Systems aiming to aid consumers in their decision-making (e.g., by implementing persuasive techniques) are more likely to be effec- tive when consumers trust them. However, recent research has demon- strated that the machine learning algorithms that often underlie such technology can act unfairly towards specific groups (e.g., by making more favorable predictions for men than for women). An undesired disparate impact resulting from this kind of algorithmic unfairness could diminish consumer trust and thereby undermine the purpose of the system. We studied this effect by conducting a between-subjects user study investi- gating how (gender-related) disparate impact affected consumer trust in an app designed to improve consumers’ financial decision-making. Our results show that disparate impact decreased consumers’ trust in the sys- tem and made them less likely to use it. Moreover, we find that trust was affected to the same degree across consumer groups (i.e., advantaged and disadvantaged users) despite both of these consumer groups recognizing their respective levels of personal benefit. Our findings highlight the im- portance of fairness in consumer-oriented artificial intelligence systems.

    @inproceedings{Draws2021,
    abstract = {Systems aiming to aid consumers in their decision-making (e.g., by implementing persuasive techniques) are more likely to be effec- tive when consumers trust them. However, recent research has demon- strated that the machine learning algorithms that often underlie such technology can act unfairly towards specific groups (e.g., by making more favorable predictions for men than for women). An undesired disparate impact resulting from this kind of algorithmic unfairness could diminish consumer trust and thereby undermine the purpose of the system. We studied this effect by conducting a between-subjects user study investi- gating how (gender-related) disparate impact affected consumer trust in an app designed to improve consumers' financial decision-making. Our results show that disparate impact decreased consumers' trust in the sys- tem and made them less likely to use it. Moreover, we find that trust was affected to the same degree across consumer groups (i.e., advantaged and disadvantaged users) despite both of these consumer groups recognizing their respective levels of personal benefit. Our findings highlight the im- portance of fairness in consumer-oriented artificial intelligence systems.},
    author = {Draws, Tim and Szl{\'{a}}vik, Zolt{\'{a}}n and Timmermans, Benjamin and Tintarev, Nava and Varshney, Kush R. and Hind, Michael},
    booktitle = {Persuasive},
    file = {:Users/tim/Surfdrive/Research/Library/Literature/Draws et al. - 2021 - Disparate Impact Diminishes Consumer Trust Even for Advantaged Users.pdf:pdf},
    keywords = {algorithmic fairness,consumer trust,disparate impact},
    title = {{Disparate Impact Diminishes Consumer Trust Even for Advantaged Users}},
    url = {https://timdraws.net/files/papers/Disparate_impact_diminishes_consumer_trust_even_for_advantaged_users.pdf},
    year = {2021}
    }

2020

  • D. van den Bergh, J. van Doorn, M. Marsman, T. Draws, E. J. van Kesteren, K. Derks, F. Dablander, Q. F. Gronau, Š. Kucharský, A. R. N. Gupta, A. Sarafoglou, J. G. Voelkel, A. Stefan, A. Ly, M. Hinne, D. Matzke, and E. J. Wagenmakers, “A Tutorial on Conducting and Interpreting a Bayesian ANOVA in JASP,” Annee Psychologique, vol. 120, iss. 1, p. 73–96, 2020. doi:10.3917/anpsy1.201.0073
    [BibTeX] [Abstract] [Download PDF]

    Analysis of variance (ANOVA) is the standard procedure for statistical inference in factorial designs. Typically, ANOVAs are executed using frequentist statistics, where p-values determine statistical significance in an all-or-none fashion. In recent years, the Bayesian approach to statistics is increasingly viewed as a legitimate alternative to the p-value. However, the broad adoption of Bayesian statistics-and Bayesian ANOVA in particular-is frustrated by the fact that Bayesian concepts are rarely taught in applied statistics courses. Consequently, practitioners may be unsure how to conduct a Bayesian ANOVA and interpret the results. Here we provide a guide for executing and interpreting a Bayesian ANOVA with JASP, an open-source statistical software program with a graphical user interface. We explain the key concepts of the Bayesian ANOVA using two empirical examples.

    @article{VanDenBergh2020,
    abstract = {Analysis of variance (ANOVA) is the standard procedure for statistical inference in factorial designs. Typically, ANOVAs are executed using frequentist statistics, where p-values determine statistical significance in an all-or-none fashion. In recent years, the Bayesian approach to statistics is increasingly viewed as a legitimate alternative to the p-value. However, the broad adoption of Bayesian statistics-and Bayesian ANOVA in particular-is frustrated by the fact that Bayesian concepts are rarely taught in applied statistics courses. Consequently, practitioners may be unsure how to conduct a Bayesian ANOVA and interpret the results. Here we provide a guide for executing and interpreting a Bayesian ANOVA with JASP, an open-source statistical software program with a graphical user interface. We explain the key concepts of the Bayesian ANOVA using two empirical examples.},
    author = {van den Bergh, Don and van Doorn, Johnny and Marsman, Maarten and Draws, Tim and van Kesteren, Erik Jan and Derks, Koen and Dablander, Fabian and Gronau, Quentin F. and Kucharsk{\'{y}}, {\v{S}}imon and Gupta, Akash R.Komarlu Narendra and Sarafoglou, Alexandra and Voelkel, Jan G. and Stefan, Angelika and Ly, Alexander and Hinne, Max and Matzke, Dora and Wagenmakers, Eric Jan},
    doi = {10.3917/anpsy1.201.0073},
    file = {:Users/tim/Surfdrive/Research/Library/Literature/van den Bergh et al. - 2020 - A Tutorial on Conducting and Interpreting a Bayesian ANOVA in JASP.pdf:pdf},
    isbn = {9782130822936},
    issn = {00035033},
    journal = {Annee Psychologique},
    keywords = {Analysis of Variance,Bayes Factor,Hypothesis Test,JASP,Posterior distribution,Tutorial},
    number = {1},
    pages = {73--96},
    title = {{A Tutorial on Conducting and Interpreting a Bayesian ANOVA in JASP}},
    url = {https://pdfs.semanticscholar.org/a79c/bca2076779f20037f87ea23c68bb3a7a2f29.pdf},
    volume = {120},
    year = {2020}
    }

  • T. Draws, N. Tintarev, U. Gadiraju, A. Bozzon, and B. Timmermans, “Assessing Viewpoint Diversity in Search Results Using Ranking Fairness Metrics,” in Informal Proceedings of the Bias and Fairness in AI Workshop at ECML-PKDD (BIAS 2020), 2020.
    [BibTeX] [Abstract] [Download PDF]

    The way pages are ranked in search results influences whether the users of search engines are exposed to more homogeneous, or rather to more diverse viewpoints. However, this viewpoint diversity is not trivial to assess. In this paper we use existing and novel ranking fairness metrics to evaluate viewpoint diversity in search result rankings. We conduct a controlled simulation study that shows how ranking fairness metrics can be used for viewpoint diversity, how their outcome should be interpreted, and which metric is most suitable depending on the situation. This paper lays out important ground work for future research to measure and assess viewpoint diversity in real search result rankings.

    @inproceedings{Draws2020assessing,
    abstract = {The way pages are ranked in search results influences whether the users of search engines are exposed to more homogeneous, or rather to more diverse viewpoints. However, this viewpoint diversity is not trivial to assess. In this paper we use existing and novel ranking fairness metrics to evaluate viewpoint diversity in search result rankings. We conduct a controlled simulation study that shows how ranking fairness metrics can be used for viewpoint diversity, how their outcome should be interpreted, and which metric is most suitable depending on the situation. This paper lays out important ground work for future research to measure and assess viewpoint diversity in real search result rankings.},
    author = {Draws, Tim and Tintarev, Nava and Gadiraju, Ujwal and Bozzon, Alessandro and Timmermans, Benjamin},
    booktitle = {Informal Proceedings of the Bias and Fairness in AI Workshop at ECML-PKDD (BIAS 2020)},
    file = {:Users/tim/Surfdrive/Research/Library/Literature/Draws et al. - 2020 - Assessing Viewpoint Diversity in Search Results Using Ranking Fairness Metrics.pdf:pdf},
    keywords = {ranking fairness,viewpoint diversity,web search},
    title = {{Assessing Viewpoint Diversity in Search Results Using Ranking Fairness Metrics}},
    url = {https://timdraws.net/files/papers/Assessing_Viewpoint_Diversity_in_Search_Results_Using_Ranking_Fairness_Metrics.pdf},
    year = {2020}
    }

  • T. Draws, J. Liu, and N. Tintarev, “Helping users discover perspectives : Enhancing opinion mining with joint topic models,” International Conference on Data Mining Workshops (ICDMW), 2020.
    [BibTeX] [Abstract] [Download PDF]

    Support or opposition with respect to a debated claim such as abortion should be legal can have different underlying reasons, which we call perspectives. This paper explores how opinion mining can be enhanced with joint topic modeling, to identify distinct perspectives within the topic, providing an informative overview from unstructured text. We evaluate four joint topic models (TAM, JST, VODUM, and LAM) in a user study assessing human understandability of the extracted perspectives. Based on the results, we conclude that joint topic models such as TAM can discover perspectives that align with human judgments. Moreover, our results suggest that users are not influenced by their pre-existing stance on the topic of abortion when interpreting the output of topic models.

    @article{Draws2020helping,
    abstract = {Support or opposition with respect to a debated claim such as abortion should be legal can have different underlying reasons, which we call perspectives. This paper explores how opinion mining can be enhanced with joint topic modeling, to identify distinct perspectives within the topic, providing an informative overview from unstructured text. We evaluate four joint topic models (TAM, JST, VODUM, and LAM) in a user study assessing human understandability of the extracted perspectives. Based on the results, we conclude that joint topic models such as TAM can discover perspectives that align with human judgments. Moreover, our results suggest that users are not influenced by their pre-existing stance on the topic of abortion when interpreting the output of topic models.},
    archivePrefix = {arXiv},
    arxivId = {2010.12505},
    author = {Draws, Tim and Liu, Jody and Tintarev, Nava},
    eprint = {2010.12505},
    file = {:Users/tim/Surfdrive/Research/Library/Literature/Draws, Liu, Tintarev - 2020 - Helping users discover perspectives Enhancing opinion mining with joint topic models.pdf:pdf},
    journal = {International Conference on Data Mining Workshops (ICDMW)},
    keywords = {debated topics,els,joint topic mod-,perspective discovery,sentiment analysis,topic modeling},
    title = {{Helping users discover perspectives : Enhancing opinion mining with joint topic models}},
    url = {https://timdraws.net/files/papers/Helping_users_discover_perspectives_in_controversial_debates_using_joint_topic_models.pdf},
    year = {2020}
    }

  • J. van Doorn, D. van den Bergh, U. Böhm, F. Dablander, K. Derks, T. Draws, A. Etz, N. J. Evans, Q. F. Gronau, J. M. Haaf, M. Hinne, Š. Kucharský, A. Ly, M. Marsman, D. Matzke, A. R. N. Gupta, A. Sarafoglou, A. Stefan, J. G. Voelkel, and E. J. Wagenmakers, “The JASP guidelines for conducting and reporting a Bayesian analysis,” Psychonomic Bulletin and Review, p. 1–14, 2020. doi:10.3758/s13423-020-01798-5
    [BibTeX] [Abstract] [Download PDF]

    Despite the increasing popularity of Bayesian inference in empirical research, few practical guidelines provide detailed recommendations for how to apply Bayesian procedures and interpret the results. Here we offer specific guidelines for four different stages of Bayesian statistical reasoning in a research setting: planning the analysis, executing the analysis, interpreting the results, and reporting the results. The guidelines for each stage are illustrated with a running example. Although the guidelines are geared towards analyses performed with the open-source statistical software JASP, most guidelines extend to Bayesian inference in general.

    @article{VanDoorn2020,
    abstract = {Despite the increasing popularity of Bayesian inference in empirical research, few practical guidelines provide detailed recommendations for how to apply Bayesian procedures and interpret the results. Here we offer specific guidelines for four different stages of Bayesian statistical reasoning in a research setting: planning the analysis, executing the analysis, interpreting the results, and reporting the results. The guidelines for each stage are illustrated with a running example. Although the guidelines are geared towards analyses performed with the open-source statistical software JASP, most guidelines extend to Bayesian inference in general.},
    author = {van Doorn, Johnny and van den Bergh, Don and B{\"{o}}hm, Udo and Dablander, Fabian and Derks, Koen and Draws, Tim and Etz, Alexander and Evans, Nathan J. and Gronau, Quentin F. and Haaf, Julia M. and Hinne, Max and Kucharsk{\'{y}}, {\v{S}}imon and Ly, Alexander and Marsman, Maarten and Matzke, Dora and Gupta, Akash R.Komarlu Narendra and Sarafoglou, Alexandra and Stefan, Angelika and Voelkel, Jan G. and Wagenmakers, Eric Jan},
    doi = {10.3758/s13423-020-01798-5},
    file = {:Users/tim/Surfdrive/Research/Library/Literature/van Doorn et al. - 2020 - The JASP guidelines for conducting and reporting a Bayesian analysis.pdf:pdf},
    isbn = {1342302001798},
    issn = {15315320},
    journal = {Psychonomic Bulletin and Review},
    keywords = {Bayesian inference,Scientific reporting,Statistical software},
    pages = {1--14},
    title = {{The JASP guidelines for conducting and reporting a Bayesian analysis}},
    url = {https://ir.cwi.nl/pub/30141/30141.pdf},
    year = {2020}
    }

2019

  • A. Sarafoglou, A. van der Heijden, T. Draws, J. Cornelisse, E. Wagenmakers, and M. Marsman, “Combine Statistical Thinking With Scientific Practice: A Protocol of a Bayesian Thesis Project For Undergraduate Students,” arXiv preprint arXiv:1810.07496, 2019.
    [BibTeX] [Abstract] [Download PDF]

    Current developments in the statistics community suggest that modern statistics education should be structured holistically, i.e., by allowing students to work with real data and answer concrete statistical questions, but also by educating them about alternative statistical frameworks, such as Bayesian statistics. In this article, we describe how we incorporated such a holistic structure in a Bayesian thesis project on ordered binomial probabilities. The project was targeted at undergraduate students in psychology with basic knowledge in Bayesian statistics and programming, but no formal mathematical training. The thesis project aimed to (1) convey the basic mathematical concepts of Bayesian inference, (2) let students experience the entire empirical cycle including the collection, analysis, and interpretation of data, and (3) teach students open science practices.

    @article{Sarafoglou2019,
    abstract = {Current developments in the statistics community suggest that modern statistics education should be structured holistically, i.e., by allowing students to work with real data and answer concrete statistical questions, but also by educating them about alternative statistical frameworks, such as Bayesian statistics. In this article, we describe how we incorporated such a holistic structure in a Bayesian thesis project on ordered binomial probabilities. The project was targeted at undergraduate students in psychology with basic knowledge in Bayesian statistics and programming, but no formal mathematical training. The thesis project aimed to (1) convey the basic mathematical concepts of Bayesian inference, (2) let students experience the entire empirical cycle including the collection, analysis, and interpretation of data, and (3) teach students open science practices.},
    archivePrefix = {arXiv},
    arxivId = {1810.07496},
    author = {Sarafoglou, Alexandra and van der Heijden, Anna and Draws, Tim and Cornelisse, Joran and Wagenmakers, Eric-Jan and Marsman, Maarten},
    eprint = {1810.07496},
    file = {:Users/tim/Surfdrive/Research/Library/Literature/Sarafoglou et al. - 2019 - Combine Statistical Thinking With Scientific Practice A Protocol of a Bayesian Thesis Project For Undergradua.pdf:pdf},
    journal = {arXiv preprint arXiv:1810.07496},
    title = {{Combine Statistical Thinking With Scientific Practice: A Protocol of a Bayesian Thesis Project For Undergraduate Students}},
    url = {http://arxiv.org/abs/1810.07496},
    year = {2019}
    }