Scores of responses by doctors and ChatGPT on the Swedish family medicine specialist exam
https://doi.org/10.5878/j8jh-5128
These scores were compiled as part of a study which compared ChatGPT’s performance with real doctors on the Swedish family medicine licensing exam.
The scores from zero to ten for the cases of exam years 2017-2022. For more details, see README.txt.
Citation and access
Citation and access
Data access level:
Creator/Principal investigator(s):
- Rasmus Arvidsson - University of Gothenburg - Institute of Medicine, School of Public Health and Community Medicine
- Artin Entezarjou - University of Gothenburg - Institute of Medicine, School of Public Health and Community Medicine
Research principal:
Data contains personal data:
No
Citation:
Language:
Copyright:
Copyright is retained for the example case in the README file. See LICENSE.txt.
Method and outcome
Method and outcome
Population:
Anonymous responses from SFAM's specialist exam in general medicine 2017-2022 and responses from ChatGPT to the same cases.
Time method:
Study design:
- Observational study
Description of study design:
ChatGPT’s scores were compared with that of real doctors using cases from the Swedish family medicine specialist exam.
Sampling procedure:
Description of sampling:
1. Randomly selected doctor responses - a single response was selected randomly for each case. 2. Top tier doctor responses - a response for each case chosen by the exam reviewers as an example of a very good response. 3. ChatGPT responses - responses provided by ChatGPT.-4, August 3 Version 2023.
Time period(s) investigated:
Data format/data structure:
Data collection - Simulation
Data collection - Simulation
Mode of collection:
Simulation
Description of the mode of collection:
Questions prompted to ChatGPT-4
Time period(s) for data collection:
2023-08-23 - 2023-08-23
Data collector:
- University of Gothenburg
Opens a new window at ror.org.
ROROpens in a new tab
Instrument
Instrument
Name:
ChatGPT-4
Type:
Other
Data collection - Educational measurements and tests
Data collection - Educational measurements and tests
Mode of collection:
Educational measurements and tests
Description of the mode of collection:
SFAM's specialist exam in general medicine
Data collector:
- The Swedish Association of General Practice (SFAM)
Geographic coverage
Geographic coverage
Geographic location:
Administrative information
Administrative information
Responsible department/unit:
Institute of Medicine
Topic and keywords
Topic and keywords
Standard för svensk indelning av forskningsämnen 2025:
Metadata
Metadata
Version 1
