SweWinogender 2.0
type: Dataset
distribution: Landing page for the dataset
description: I. IDENTIFYING INFORMATION Title* SweWinogender v2.0 Subtitle A Swedish diagnostic set for gender bias in natural language inference models Created by* Yvonne Adesam (yvonne.adesam@gu.se), Gerlof Bouma (gerlof.bouma) Publisher(s)* Språkbanken Text Link(s) / permanent identifier(s)* https://spraakbanken.gu.se/en/resources/swewinogender License(s)* CC BY 4.0 Abstract* The SweWinogender test set is diagnostic dataset to measure gender bias in coreference resolution/textual entailment. It is modelled after the English Winogender benchmark, and is released with reference statistics on the distribution of men and women between occupations and the association between gender and occupation in modern corpus material. Funded by* Vinnova (dnr 2020-02523, dnr 2021-04165); Språkbanken Text Cite as [1] Related datasets Part of the SuperLim 2.0 collection (https://spraakbanken.gu.se/en/resources/superlim) Based upon/partially translated from Winogender Schemas [2] See SweWinogender v1.0 for a formulation of this task as a pronoun resolution problem. II. USAGE Key applications Diagnose gender bias in natural language inference systems Intended task(s)/usage(s) (1) Indirectly the pronoun interpretation task cast as a natural language inference problem: decide whether a discourse fragment containing a pronoun entails a sentence with the pronoun replaced by a candidate antecedent. (2) Compare system predictions between pronoun types (masc/fem/gender-neutral) (3) Compare system predictions with auxiliary statistics on gender and occupation Recommended evaluation measures (1) Krippendorff's alpha on binary label (2) “Gender parity”: The proportion of triples of items differing only by the type of pronoun, which receive identical labels See [3]. (3) Correlation (Spearman’s rho); plotting/visual inspection. See [2]. Dataset function(s) Diagnostics Recommended split(s) Test data only III. DATA Primary data* Text Language* Swedish Dataset in numbers* 624 test items from 104 templates. 312 p…
modified: 2023-03-30T00:00:00Z
issued: 2024-01-01T00:00:00Z
publisher: University of Gothenburg
qualified attribution: Attribution
  had role: principalInvestigator
  agent: Adesam, Yvonne
    type: Agent
    type: PrivateIndividual(s)
qualified attribution: Attribution
  had role: principalInvestigator
  agent: Bouma, Gerlof
    type: Agent
    type: PrivateIndividual(s)
keyword: Language Technology (Computational Linguistics)
keyword: corpus
language: SWE

Landing page for the dataset
type: Distribution
title: Landningssida för datasetet @sv
access URL: SweWinogender 2.0
availability: stable

University of Gothenburg
type: Agent
name: Göteborgs universitet @sv
type: Academia-ScientificOrganisation
source: 01tm6cn81