“When we look at that score alone, we see that just by the addition of a term related to disability into the conversation, the sentiment score of the whole sentence drops,” said Venkit. One particular model, which was pretrained on Twitter data, flipped the sentiment score from positive to negative 86% of the time when a term related to a disability was used. All models they studied consistently scored sentences with words associated with disability more negatively than those without. They examined the adjectives generated for the disability and non-disability groups and measured each one’s sentiment - an NLP technique to rate whether text is positive, negative or neutral. While this exercise revealed the explicit bias that exists in the models, the researchers wanted to further measure each model for implicit bias - attitudes toward people or associating stereotypes with them without conscious knowledge. “With the addition of a non-disability term, the effect of ‘good’ becomes ‘great.’ But when ‘good’ is associated with a disability-related term, we get the result of ‘bad.’ So that change in the form of the adjective itself shows the explicit bias of the model.” “For example, we selected the word ‘good,’ and wanted to see how it associated with terms related to both non-disability and disability,” explained Venkit. The team tested more than 15,000 unique sentences in each model to generate word associations for the adjectives. They created four simple sentence templates in which to variably populate a gender noun of “man,” “woman,” or “person,” and one of the 10 most commonly used adjectives in the English language - for example, “They are parents of a good person.” Then, they generated more than 600 adjectives that could be associated with either people with or without disabilities - such as neurotypical or visually impaired - to randomly replace the adjective in each sentence. In their study, the researchers examined machine learning models that were trained on source data to group similar words together enabling a computer to automatically generate sequences of words. “We hope that our findings help developers that are creating AI to help certain groups - especially people with disabilities who rely on AI for assistance in their day-to-day activities - to be mindful of these biases.” 13) at the 29th International Conference on Computational Linguistics (COLING). “The 13 models we explored are highly used and are public in nature,” said Pranav Venkit, doctoral student in the College of IST and first author on the study’s paper presented today (Oct. Previous research on pretrained language models - which are trained on large amounts of data that may contain implicit biases - has found sociodemographic biases against genders and races, but until now similar biases against people with disabilities have not been widely explored. The researchers found that all the algorithms and models they tested contained significant implicit bias against people with disabilities. However, the algorithms that drive this technology often have tendencies that could be offensive or prejudiced toward individuals with disabilities, according to researchers at the Penn State College of Information Sciences and Technology (IST). Natural language processing (NLP) is a type of artificial intelligence that allows machines to use text and spoken words in many different applications - such as smart assistants or email autocorrect and spam filters - helping automate and streamline operations for individual users and enterprises.
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |