#timnit_gebru

Quand le #comité_d’éthique du #CNRS se penche sur l’#engagement_public des chercheurs et chercheuses

#Neutralité ? #Intégrité ? #Transparence ?

Le Comité d’éthique du CNRS rappelle qu’il n’y a pas d’#incompatibilité de principe, plaide pour un « guide pratique de l’engagement » et place la direction de l’institution scientifique devant les mêmes obligations que les chercheurs.

Avec la crise climatique, la pandémie de covid-19, l’accroissement des inégalités, le développement de l’intelligence artificielle ou les technologies de surveillance, la question de l’#engagement public des chercheurs est d’autant plus visible que les réseaux sociaux leur permettent une communication directe.

Cette question dans les débats de société n’est pas nouvelle. De l’appel d’#Albert_Einstein, en novembre 1945, à la création d’un « #gouvernement_du_monde » pour réagir aux dangers de la #bombe_atomique à l’alerte lancée par #Irène_Frachon concernant le #Médiator, en passant par celle lancée sur les dangers des grands modèles de langage par #Timnit_Gebru et ses collègues, les chercheurs et chercheuses s’engagent régulièrement et créent même des sujets de #débats_publics.

Une question renouvelée dans un monde incertain

Le #comité_d'éthique_du_CNRS (#COMETS) ne fait pas semblant de le découvrir. Mais, selon lui, « face aux nombreux défis auxquels notre société est confrontée, la question de l’engagement public des chercheurs s’est renouvelée ». Il s’est donc auto-saisi pour « fournir aux chercheurs des clés de compréhension et des repères éthiques concernant l’engagement public » et vient de publier son #rapport sur le sujet [PDF].

Il faut dire que les deux premières années du Covid-19 ont laissé des traces dans la communauté scientifique sur ces questions de prises de paroles des chercheurs. Le COMETS avait d’ailleurs publié en mai 2021 un avis accusant Didier Raoult alors que la direction du Centre avait rappelé tardivement à l’ordre, en août de la même année, et sans le nommer, le sociologue et directeur de recherche au CNRS Laurent Mucchielli, qui appelait notamment à suspendre la campagne de vaccination.

Le COMETS relève que les chercheurs s’engagent selon des modalités variées, « de la signature de tribunes à la contribution aux travaux d’ONG ou de think tanks en passant par le soutien à des actions en justice ou l’écriture de billets de blog ». Il souligne aussi que les #réseaux_sociaux ont « sensiblement renforcé l’exposition publique des chercheurs engagés ».

La présidente du comité d’éthique, Christine Noiville, égrène sur le site du CNRS, les « interrogations profondes » que ces engagements soulèvent :

« S’engager publiquement, n’est-ce pas contraire à l’exigence d’#objectivité de la recherche ? N’est-ce pas risquer de la « politiser » ou de l’« idéologiser » ? S’engager ne risque-t-il pas de fragiliser la #crédibilité du chercheur, de mettre à mal sa réputation, sa carrière ? Est-on en droit de s’engager ? Pourrait-il même s’agir d’un devoir, comme certains collègues ou journalistes pourraient le laisser entendre ? »

Pas d’incompatibilité de principe

Le comité d’éthique aborde les inquiétudes que suscite cet engagement public des chercheurs et pose franchement la question de savoir s’il serait « une atteinte à la #neutralité_scientifique ? ». Faudrait-il laisser de côté ses opinions et valeurs pour « faire de la « bonne » science et produire des connaissances objectives » ?

Le COMETS explique, en s’appuyant sur les travaux de l’anthropologue #Sarah_Carvallo, que ce concept de neutralité est « devenu central au XXe siècle, pour les sciences de la nature mais également pour les sciences sociales », notamment avec les philosophes des sciences #Hans_Reichenbach et #Karl_Popper, ainsi que le sociologue #Max_Weber dont le concept de « #neutralité_axiologique » – c’est-à-dire une neutralité comme valeur fondamentale – voudrait que le « savant » « tienne ses #convictions_politiques à distance de son enseignement et ne les impose pas subrepticement ».

Mais le comité explique aussi, que depuis Reichenbach, Popper et Weber, la recherche a avancé. Citant le livre d’#Hilary_Putnam, « The Collapse of the Fact/Value Dichotomy and Other Essays », le COMETS explique que les chercheurs ont montré que « toute #science s’inscrit dans un #contexte_social et se nourrit donc de #valeurs multiples ».

Le comité explique que le monde de la recherche est actuellement traversé de valeurs (citant le respect de la dignité humaine, le devoir envers les animaux, la préservation de l’environnement, la science ouverte) et que le chercheur « porte lui aussi nécessairement des valeurs sociales et culturelles dont il lui est impossible de se débarrasser totalement dans son travail de recherche ».

Le COMETS préfère donc insister sur les « notions de #fiabilité, de #quête_d’objectivité, d’#intégrité et de #rigueur de la #démarche_scientifique, et de transparence sur les valeurs » que sur celle de la neutralité. « Dans le respect de ces conditions, il n’y a aucune incompatibilité avec l’engagement public du chercheur », assure-t-il.

Liberté de s’engager... ou non

Il rappelle aussi que les chercheurs ont une large #liberté_d'expression assurée par le code de l’éducation tout en n’étant pas exemptés des limites de droit commun (diffamation, racisme, sexisme, injure ...). Mais cette liberté doit s’appliquer à double sens : le chercheur est libre de s’engager ou non. Elle est aussi à prendre à titre individuel, insiste le COMETS : la démarche collective via les laboratoires, sociétés savantes et autres n’est pas la seule possible, même si donner une assise collective « présente de nombreux avantages (réflexion partagée, portée du message délivré, moindre exposition du chercheur, etc.) ».

Le comité insiste par contre sur le fait que, lorsque le chercheur s’engage, il doit « prendre conscience qu’il met en jeu sa #responsabilité, non seulement juridique mais aussi morale, en raison du crédit que lui confère son statut et le savoir approfondi qu’il implique ».

Il appuie aussi sur le fait que sa position privilégiée « crédite sa parole d’un poids particulier. Il doit mettre ce crédit au service de la collectivité et ne pas en abuser ».

Des #devoirs lors de la #prise_de_parole

Outre le respect de la loi, le COMETS considère, dans ce cadre, que les chercheurs et chercheuses ont des devoirs vis-à-vis du public. Notamment, ils doivent s’efforcer de mettre en contexte le cadre dans lequel ils parlent. S’agit-il d’une prise de parole en nom propre ? Le thème est-il dans le domaine de compétence du chercheur ? Est-il spécialiste ? A-t-il des liens d’intérêts ? Quelles valeurs sous-tendent son propos ? Le #degré_de_certitude doit aussi être abordé. Le Comité exprime néanmoins sa compréhension de la difficulté pratique que cela implique, vu les limites de temps de paroles dans les médias.

Une autre obligation qui devrait s’appliquer à tout engagement de chercheurs selon le COMETS, et pas des moindres, est de l’asseoir sur des savoirs « robustes » et le faire « reposer sur une démarche scientifique rigoureuse ».

Proposition de co-construction d’un guide

Le COMETS recommande, dans ce cadre, au CNRS d’ « élaborer avec les personnels de la recherche un guide de l’engagement public » ainsi que des formations. Il propose aussi d’envisager que ce guide soit élaboré avec d’autres organismes de recherche.

La direction du CNRS à sa place

Le Comité d’éthique considère en revanche que « le CNRS ne devrait ni inciter, ni condamner a priori l’engagement des chercheurs, ni opérer une quelconque police des engagements », que ce soit dans l’évaluation des travaux de recherche ou dans d’éventuelles controverses provoquées par un engagement public.

« La direction du CNRS n’a pas vocation à s’immiscer dans ces questions qui relèvent au premier chef du débat scientifique entre pairs », affirme-t-il. La place du CNRS est d’intervenir en cas de problème d’#intégrité_scientifique ou de #déontologie, mais aussi de #soutien aux chercheurs engagés « qui font l’objet d’#attaques personnelles ou de #procès_bâillons », selon lui.

Le comité aborde aussi le cas dans lequel un chercheur mènerait des actions de #désobéissance_civile, sujet pour le moins d’actualité. Il considère que le CNRS ne doit ni « se substituer aux institutions de police et de justice », ni condamner par avance ce mode d’engagement, « ni le sanctionner en lieu et place de l’institution judiciaire ». Une #sanction_disciplinaire peut, par contre, être envisagée « éventuellement », « en cas de décision pénale définitive à l’encontre d’un chercheur ».

Enfin, le Comité place la direction du CNRS devant les mêmes droits et obligations que les chercheurs dans son engagement vis-à-vis du public. Si le CNRS « prenait publiquement des positions normatives sur des sujets de société, le COMETS considère qu’il devrait respecter les règles qui s’appliquent aux chercheurs – faire connaître clairement sa position, expliciter les objectifs et valeurs qui la sous-tendent, etc. Cette prise de position de l’institution devrait pouvoir être discutée sur la base d’un débat contradictoire au sein de l’institution ».

▻https://next.ink/985/quand-comite-dethique-cnrs-se-penche-sur-engagement-public-chercheurs-et-cherc

Ex-Googler Timnit Gebru Starts Her Own AI Research Center | WIRED
▻https://www.wired.com/story/ex-googler-timnit-gebru-starts-ai-research-center/?bxid=5cec29ba24c17c4c6465ed0b&bxid=5cec29ba24c17c4c6465ed0b&cndid=57360427&c

https://media.wired.com/photos/61a80d55285453ba57827d11/191:100/w_1280,c_limit/timnet-gebru-1028811898.jpg

One year ago Google artificial intelligence researcher Timnit Gebru tweeted, “I was fired” and ignited a controversy over the freedom of employees to question the impact of their company’s technology. Thursday, she launched a new research institute to ask questions about responsible use of artificial intelligence that Gebru says Google and other tech companies won’t.

“Instead of fighting from the inside, I want to show a model for an independent institution with a different set of incentive structures,” says Gebru, who is founder and executive director of Distributed Artificial Intelligence Research (DAIR). The first part of the name is a reference to her aim to be more inclusive than most AI labs—which skew white, Western, and male—and to recruit people from parts of the world rarely represented in the tech industry.

Gebru was ejected from Google after clashing with bosses over a research paper urging caution with new text-processing technology enthusiastically adopted by Google and other tech companies. Google has said she resigned and was not fired, but acknowledged that it later fired Margaret Mitchell, another researcher who with Gebru co-led a team researching ethical AI. The company placed new checks on the topics its researchers can explore. Google spokesperson Jason Freidenfelds declined to comment but directed WIRED to a recent report on the company’s work on AI governance, which said Google has published more than 500 papers on “responsible innovation” since 2018.

The fallout at Google highlighted the inherent conflicts in tech companies sponsoring or employing researchers to study the implications of technology they seek to profit from. Earlier this year, organizers of a leading conference on technology and society canceled Google’s sponsorship of the event. Gebru says DAIR will be freer to question the potential downsides of AI and will be unencumbered by the academic politics and pressure to publish that she says can complicate university research.

DAIR is currently a project of nonprofit Code for Science and Society but will later incorporate as a nonprofit in its own right, Gebru says. Her project has received grants totaling more than $3 million from the Ford, MacArthur, Rockefeller, and Open Society foundations, as well as the Kapor Center. Over time, she hopes to diversify DAIR’s financial support by taking on consulting work related to its research.
DAIR joins a recent flourishing of work and organizations taking a broader and critical view of AI technology. New nonprofits and university centers have sprung up to study and critique AI’s effects in and on the world, such as NYU’s AI Now Institute, the Algorithmic Justice League, and Data for Black Lives. Some researchers in AI labs also study the impacts and proper use of algorithms, and scholars from other fields such as law and sociology have turned their own critical eyes on AI.

#Intelligence_artificielle #Timnit_Gebru #Ethique

Timnit Gebru Is Building a Slow AI Movement - IEEE Spectrum
►https://spectrum.ieee.org/timnit-gebru-dair-ai-ethics

https://spectrum.ieee.org/media-library/portrait-of-a-woman-in-a-grey-sweater-and-cream-colored-scarf-sits-in-front-of-a-window.jpg?id=29614500&width=1200&coordinates=0%2C55%2C0%2C778&height=600

Timnit Gebru was a well-known scholar in the AI-ethics community long before she got fired by Google in December 2020—but that messy and dramatic incident brought a new level of attention to her work. Google apparently exiled Gebru from its AI ethics team (and subsequently fired the other leader of the team) in response to a paper about the dangers of the large language models that have become so important to the world’s biggest technology companies. The episode created a firestorm in the AI field.

But Gebru has made the most of the jarring opportunity. In December 2021, she announced the founding of a new organization, the Distributed AI Research Institute (DAIR), which is billed as “a space for independent, community-rooted AI research free from Big Tech’s pervasive influence.” Since then, Gebru has been staffing up. In February, AI and sociology researcher Alex Hanna joined as research director, departing from her Google job with a blistering resignation letter. Gebru and Hanna spoke with IEEE Spectrum about their plans for DAIR.

Alex Hanna: I’m not necessarily thinking about it from the perspective of the precautionary principle. I’m thinking of it more from the perspective of developing technology that works for people. A lot of the AI research that happens right now is AI for the value of AI itself. A lot of people are thinking about this body of tools known as AI and saying, “Well, everything looks like a nail, and we have this big hammer.”

We already know that deep learning has problems. These modes of research require organizations that can gather a lot of data, data that is often collected via ethically or legally questionable technologies, like surveilling people in nonconsensual ways. If we want to build technology that has meaningful community input, then we need to really think about what’s best. Maybe AI is not the answer for what some particular community needs.

It’s interesting to hear you talk about challenges with the data sets. Timnit, in your work on large language models you’ve called attention to problems with existing data sets, including embedded bias. The response I often hear is, essentially, “It’s just too hard to make data sets better.”

Gebru: If it’s just too hard to build a safe car, would we have cars out there? It goes back to what Alex was saying about a hammer. People think there’s just one way to do it. We’ve been trying to say, “Maybe there’s a different way we can think about this.” If you think [data-set curation] is a necessity, that means it’ll take more time and resources before you put something out there.

Hanna: This is a point we’ve made over and over. We’ve published on data-set practices and how many of these things go out with not enough attention paid to what’s in them. This data-hungry version of building models started with ImageNet, and it wasn’t until ImageNet was out for about 10 years that people started to dig in and say, “Wait, this [data set] is really problematic.”

Merci @fil pour le lien.

#Timnit_Gebru #Alex_Hanna #Intelligence_artificielle #Travail_de_la_donnée

Timnit Gebru was fired from Google — then the harassers arrived - The Verge
►https://www.theverge.com/22309962/timnit-gebru-google-harassment-campaign-jeff-dean

https://cdn.vox-cdn.com/thumbor/HvUYip6wBghIA9XUcZ4GwgKX4K8=/0x148:2050x1221/fit-in/1200x630/cdn.vox-cdn.com/uploads/chorus_asset/file/22346095/VRG_ILLO_4413_001.jpg

TimnitTimnit Gebru had expected her colleagues to rally around her when she was abruptly fired from Google on December 2nd. She was a well-respected AI ethics researcher, her termination as controversial as it was sudden. What she hadn’t anticipated was becoming a catalyst for labor activism in Silicon Valley — or the subject of a harassment campaign that surfaced alongside her supporters.

Her firing came weeks after Google managers asked her to retract a paper on the dangers of large language models, like the ones that power the company’s search engine. Gebru pushed back, saying the company needed to be more transparent about the publication process. Employees saw the termination as a blatant act of retaliation, and thousands of workers, researchers, and academics signed an open letter demanding an explanation.

In early December, as media attention mounted and tech workers across the country came to her defense, Gebru waited to see how Google would respond. What is this company capable of? she asked herself. She didn’t have to wait long to find out.
“Twitter enabled Gebru’s colleagues to stand up for her; it also made her vulnerable to a small but very active group of harassers”

On December 4th, Jeff Dean, Google’s head of AI, published his views on Gebru’s dismissal. He said she had resigned — a fact she openly disputed — and that her paper hadn’t met the bar for publication. His words lay the groundwork for a very different type of campaign, one which used the groundswell of support surrounding Gebru’s firing as a stand-in for the ills of cancel culture. Twitter enabled Gebru’s colleagues to stand up for her; it also made her vulnerable to a small but very active group of harassers.

Over the next two months, Pedro Domingos, a professor emeritus at the University of Washington, and Michael Lissack, a Wall Street whistleblower turned admitted harasser, promoted the narrative that Gebru’s work was “advocacy disguised as science.” They said she’d created a toxic environment at Google and was “obsessed with being a victim.” They wrote off her supporters as sycophants and “deranged activists.”

On Twitter, the campaign gained momentum with the help of anonymous accounts. Gebru suspected they were sock puppets, fake profiles created for the sole purpose of harassment, since many had popped up around her firing. Some claimed to work in tech, using phrases like “pro-DEI minority,” “She/them,” and “LBGTQ Latinx” in their bios. When Emily M. Bender, a computational linguistics professor at the University of Washington and Gebru’s co-author on the paper, reported one of these accounts for harassment, Domingos implied she was being prejudiced.
““He enables these people, and gets to distance himself because he’s not the one saying ‘go back to Africa’””

To Gebru, the harassment wouldn’t be possible without Jeff Dean, who stayed quiet about the campaign for months, despite being tagged in numerous posts. “He enables these people, and gets to distance himself because he’s not the one saying ‘go back to Africa,’” she says. “Pedro [Domingos] is smart enough not to say these types of things, too. They hide behind civility and enable the trolls. That’s how they get away with it.”

#Timnit_Gebru #Cyberharcèlement #Ethique #Intelligence_artificielle

Google Workers Say the Endless Wait to Unionize Big Tech Is Over

“You have a union when you say you have a union.”
▻https://inthesetimes.com/article/google-workers-alphabet-union-cwa

https://inthesetimes.com/imager/s3_us-east-1_amazonaws_com/in-these-times/GettyImages-1056086262_01c8ac7cc3b849e525e370bc51344120.jpg

The five most valu­able com­pa­nies in Amer­i­ca are all big tech com­pa­nies, and none of them are union­ized. Com­pound­ing this exis­ten­tial chal­lenge for orga­nized labor is the fact that the huge work forces of the com­pa­nies make union­iz­ing them seem an impos­si­bly large task. Now, one union has solved that prob­lem with a rev­o­lu­tion­ary approach: Just start.

(…)

Since the 2018 #Google walk­outs protest­ing sex­u­al harass­ment (and the sub­se­quent retal­i­a­tion against its orga­niz­ers), Google has been the most high pro­file hotbed of work­er orga­niz­ing among the big tech com­pa­nies — though all of that orga­niz­ing focused on spe­cif­ic issues as they arose, rather than on form­ing a union. Shaw began attend­ing events that employ­ees set up relat­ed to orga­niz­ing: a lun­cheon, a book club, a lec­ture. Even­tu­al­ly, he con­nect­ed with CWA staff and began actu­al labor orga­niz­ing in earnest. Last June, a group called Googlers Against Racism got more than 1,000 employ­ee sig­na­tures on a Cowork​er​.org peti­tion urg­ing the com­pa­ny to take a num­ber of steps to pro­mote diver­si­ty and end con­tracts with police. That group pro­vid­ed a pool of inter­est­ed activist work­ers that led direct­ly to dis­cus­sions about union­iz­ing, and to recruits for the union. Shaw says that the fir­ing last month of #Timnit_Gebru, an inter­nal crit­ic of the com­pa­ny, was ​“a real­ly big ral­ly­ing moment.”

#travail #syndicalisme

“I started crying”: Inside Timnit Gebru’s last days at Google | MIT Technology Review
▻https://www.technologyreview.com/2020/12/16/1014634/google-ai-ethics-lead-timnit-gebru-tells-story

https://wp.technologyreview.com/wp-content/uploads/2020/12/TimnitGebru_V1.jpg?resize=1200,600

By now, we’ve all heard some version of the story. On December 2, after a protracted disagreement over the release of a research paper, Google forced out its ethical AI co-lead, Timnit Gebru. The paper was on the risks of large language models, AI models trained on staggering amounts of text data, which are a line of research core to Google’s business. Gebru, a leading voice in AI ethics, was one of the only Black women at Google Research.

The move has since sparked a debate about growing corporate influence over AI, the long-standing lack of diversity in tech, and what it means to do meaningful AI ethics research. As of December 15, over 2,600 Google employees and 4,300 others in academia, industry, and civil society had signed a petition denouncing the dismissal of Gebru, calling it “unprecedented research censorship” and “an act of retaliation.”

The company’s star ethics researcher highlighted the risks of large language models, which are key to Google’s business.
Gebru is known for foundational work in revealing AI discrimination, developing methods for documenting and auditing AI models, and advocating for greater diversity in research. In 2016, she cofounded the nonprofit Black in AI, which has become a central resource for civil rights activists, labor organizers, and leading AI ethics researchers, cultivating and highlighting Black AI research talent.

Then in that document, I wrote that this has been extremely disrespectful to the Ethical AI team, and there needs to be a conversation, not just with Jeff and our team, and Megan and our team, but the whole of Research about respect for researchers and how to have these kinds of discussions. Nope. No engagement with that whatsoever.

I cried, by the way. When I had that first meeting, which was Thursday before Thanksgiving, a day before I was going to go on vacation—when Megan told us that you have to retract this paper, I started crying. I was so upset because I said, I’m so tired of constant fighting here. I thought that if I just ignored all of this DEI [diversity, equity, and inclusion] hypocrisy and other stuff, and I just focused on my work, then at least I could get my work done. And now you’re coming for my work. So I literally started crying.

You’ve mentioned that this is not just about you; it’s not just about Google. It’s a confluence of so many different issues. What does this particular experience say about tech companies’ influence on AI in general, and their capacity to actually do meaningful work in AI ethics?
You know, there were a number of people comparing Big Tech and Big Tobacco, and how they were censoring research even though they knew the issues for a while. I push back on the academia-versus-tech dichotomy, because they both have the same sort of very racist and sexist paradigm. The paradigm that you learn and take to Google or wherever starts in academia. And people move. They go to industry and then they go back to academia, or vice versa. They’re all friends; they are all going to the same conferences.

I don’t think the lesson is that there should be no AI ethics research in tech companies, but I think the lesson is that a) there needs to be a lot more independent research. We need to have more choices than just DARPA [the Defense Advanced Research Projects Agency] versus corporations. And b) there needs to be oversight of tech companies, obviously. At this point I just don’t understand how we can continue to think that they’re gonna self-regulate on DEI or ethics or whatever it is. They haven’t been doing the right thing, and they’re not going to do the right thing.

I think academic institutions and conferences need to rethink their relationships with big corporations and the amount of money they’re taking from them. Some people were even wondering, for instance, if some of these conferences should have a “no censorship” code of conduct or something like that. So I think that there is a lot that these conferences and academic institutions can do. There’s too much of an imbalance of power right now.

#Intelligence_artificielle #Timnit_Gebru #Google #Ethique

Google, IA et éthique : « départ » de Timnit Gebru, Sundar Pichai s’exprime
▻https://www.nextinpact.com/lebrief/45022/google-ia-et-ethique-imbroglio-sur-depart-timnit-gebru-sundar-pichai-sor

https://cdnx.nextinpact.com/data-next/images/bd/wide-linked-media/9495.jpg

Depuis une semaine, le licenciement par Google de la chercheuse Timnit Gebru fait couler beaucoup d’encre. Elle était chargée des questions d’éthique sur l’intelligence artificielle et l’une des rares afro-américaines spécialistes du sujet.

L’imbroglio commence dès les premiers jours quand elle explique sur Twitter que sa hiérarchie avait accepté sa démission… qu’elle affirme n’avoir jamais donnée. Cette situation arrive après que la scientifique se soit plainte que Google « réduise au silence les voix marginalisées » et lui demande de rétracter un article.

Une pétition a rapidement été mise en ligne sur Medium afin de demander des explications à l’entreprise, notamment sur les raisons de la censure. Actuellement, elle a été signée par plus de 6 000 personnes, dont 2 351 « googlers ». Sundar Pichai est finalement sorti du bois pour évoquer cette situation et promettre une enquête :

« Nous devons accepter la responsabilité du fait qu’une éminente dirigeante noire, dotée d’un immense talent, a malheureusement quitté Google. »

Marchant sur des œufs, il ajoute : « Nous devons évaluer les circonstances qui ont conduit au départ du Dr Gebru ». Dans son courrier, il parle bien de « départ » (departure en anglais), et ne prononce pas le mot licenciement. Pour lui, cette situation « a semé le doute et conduit certains membres de notre communauté, qui remettent en question leur place chez Google […] Je veux dire à quel point je suis désolé pour cela et j’accepte la responsabilité de travailler pour retrouver votre confiance ».

Sur Twitter, Timnit Gebru n’est pas convaincue et s’explique : « Il ne dit pas "Je suis désolé pour ce que nous lui avons fait et c’était mal" […] Donc je vois ça comme "Je suis désolé pour la façon dont ça s’est passé, mais je ne suis pas désolé pour ce que nous lui avons fait." ».

#Timnit_Gebru #Google #Intelligence_artificielle #Ethique

We read the paper that forced Timnit Gebru out of Google. Here’s what it says | MIT Technology Review
▻https://www.technologyreview.com/2020/12/04/1013294/google-ai-ethics-research-paper-forced-out-timnit-gebru/?truid=a497ecb44646822921c70e7e051f7f1a

https://wp.technologyreview.com/wp-content/uploads/2020/12/transglobal-copy-1-e1607136614490.jpg?resize=1200,600

The company’s star ethics researcher highlighted the risks of large language models, which are key to Google’s business.
by

Karen Hao archive page

December 4, 2020
Timnit Gebru
courtesy of Timnit Gebru

On the evening of Wednesday, December 2, Timnit Gebru, the co-lead of Google’s ethical AI team, announced via Twitter that the company had forced her out.

Gebru, a widely respected leader in AI ethics research, is known for coauthoring a groundbreaking paper that showed facial recognition to be less accurate at identifying women and people of color, which means its use can end up discriminating against them. She also cofounded the Black in AI affinity group, and champions diversity in the tech industry. The team she helped build at Google is one of the most diverse in AI, and includes many leading experts in their own right. Peers in the field envied it for producing critical work that often challenged mainstream AI practices.

A series of tweets, leaked emails, and media articles showed that Gebru’s exit was the culmination of a conflict over another paper she co-authored. Jeff Dean, the head of Google AI, told colleagues in an internal email (which he has since put online) that the paper “didn’t meet our bar for publication” and that Gebru had said she would resign unless Google met a number of conditions, which it was unwilling to meet. Gebru tweeted that she had asked to negotiate “a last date” for her employment after she got back from vacation. She was cut off from her corporate email account before her return.

Online, many other leaders in the field of AI ethics are arguing that the company pushed her out because of the inconvenient truths that she was uncovering about a core line of its research—and perhaps its bottom line. More than 1,400 Google staff and 1,900 other supporters have also signed a letter of protest.
Sign up for The Download - Your daily dose of what’s up in emerging technology
Stay updated on MIT Technology Review initiatives and events?
Yes
No

Many details of the exact sequence of events that led up to Gebru’s departure are not yet clear; both she and Google have declined to comment beyond their posts on social media. But MIT Technology Review obtained a copy of the research paper from one of the co-authors, Emily M. Bender, a professor of computational linguistics at the University of Washington. Though Bender asked us not to publish the paper itself because the authors didn’t want such an early draft circulating online, it gives some insight into the questions Gebru and her colleagues were raising about AI that might be causing Google concern.

Titled “On the Dangers of Stochastic Parrots: Can Language Models Be Too Big?” the paper lays out the risks of large language models—AIs trained on staggering amounts of text data. These have grown increasingly popular—and increasingly large—in the last three years. They are now extraordinarily good, under the right conditions, at producing what looks like convincing, meaningful new text—and sometimes at estimating meaning from language. But, says the introduction to the paper, “we ask whether enough thought has been put into the potential risks associated with developing them and strategies to mitigate these risks.”
The paper

The paper, which builds off the work of other researchers, presents the history of natural-language processing, an overview of four main risks of large language models, and suggestions for further research. Since the conflict with Google seems to be over the risks, we’ve focused on summarizing those here.
Environmental and financial costs

Training large AI models consumes a lot of computer processing power, and hence a lot of electricity. Gebru and her coauthors refer to a 2019 paper from Emma Strubell and her collaborators on the carbon emissions and financial costs of large language models. It found that their energy consumption and carbon footprint have been exploding since 2017, as models have been fed more and more data.

Strubell’s study found that one language model with a particular type of “neural architecture search” (NAS) method would have produced the equivalent of 626,155 pounds (284 metric tons) of carbon dioxide—about the lifetime output of five average American cars. A version of Google’s language model, BERT, which underpins the company’s search engine, produced 1,438 pounds of CO2 equivalent in Strubell’s estimate—nearly the same as a roundtrip flight between New York City and San Francisco.

Gebru’s draft paper points out that the sheer resources required to build and sustain such large AI models means they tend to benefit wealthy organizations, while climate change hits marginalized communities hardest. “It is past time for researchers to prioritize energy efficiency and cost to reduce negative environmental impact and inequitable access to resources,” they write.
Massive data, inscrutable models

Large language models are also trained on exponentially increasing amounts of text. This means researchers have sought to collect all the data they can from the internet, so there’s a risk that racist, sexist, and otherwise abusive language ends up in the training data.

An AI model taught to view racist language as normal is obviously bad. The researchers, though, point out a couple of more subtle problems. One is that shifts in language play an important role in social change; the MeToo and Black Lives Matter movements, for example, have tried to establish a new anti-sexist and anti-racist vocabulary. An AI model trained on vast swaths of the internet won’t be attuned to the nuances of this vocabulary and won’t produce or interpret language in line with these new cultural norms.

It will also fail to capture the language and the norms of countries and peoples that have less access to the internet and thus a smaller linguistic footprint online. The result is that AI-generated language will be homogenized, reflecting the practices of the richest countries and communities.

Moreover, because the training datasets are so large, it’s hard to audit them to check for these embedded biases. “A methodology that relies on datasets too large to document is therefore inherently risky,” the researchers conclude. “While documentation allows for potential accountability, [...] undocumented training data perpetuates harm without recourse.”
Research opportunity costs

The researchers summarize the third challenge as the risk of “misdirected research effort.” Though most AI researchers acknowledge that large language models don’t actually understand language and are merely excellent at manipulating it, Big Tech can make money from models that manipulate language more accurately, so it keeps investing in them. “This research effort brings with it an opportunity cost,” Gebru and her colleagues write. Not as much effort goes into working on AI models that might achieve understanding, or that achieve good results with smaller, more carefully curated datasets (and thus also use less energy).
Illusions of meaning

The final problem with large language models, the researchers say, is that because they’re so good at mimicking real human language, it’s easy to use them to fool people. There have been a few high-profile cases, such as the college student who churned out AI-generated self-help and productivity advice on a blog, which went viral.

The dangers are obvious: AI models could be used to generate misinformation about an election or the covid-19 pandemic, for instance. They can also go wrong inadvertently when used for machine translation. The researchers bring up an example: In 2017, Facebook mistranslated a Palestinian man’s post, which said “good morning” in Arabic, as “attack them” in Hebrew, leading to his arrest.
Why it matters

Gebru and Bender’s paper has six co-authors, four of whom are Google researchers. Bender asked to avoid disclosing their names for fear of repercussions. (Bender, by contrast, is a tenured professor: “I think this is underscoring the value of academic freedom,” she says.)

The paper’s goal, Bender says, was to take stock of the landscape of current research in natural-language processing. “We are working at a scale where the people building the things can’t actually get their arms around the data,” she said. “And because the upsides are so obvious, it’s particularly important to step back and ask ourselves, what are the possible downsides? … How do we get the benefits of this while mitigating the risk?”

In his internal email, Dean, the Google AI head, said one reason the paper “didn’t meet our bar” was that it “ignored too much relevant research.” Specifically, he said it didn’t mention more recent work on how to make large language models more energy-efficient and mitigate problems of bias.

However, the six collaborators drew on a wide breadth of scholarship. The paper’s citation list, with 128 references, is notably long. “It’s the sort of work that no individual or even pair of authors can pull off,” Bender said. “It really required this collaboration.”

The version of the paper we saw does also nod to several research efforts on reducing the size and computational costs of large language models, and on measuring the embedded bias of models. It argues, however, that these efforts have not been enough. “I’m very open to seeing what other references we ought to be including,” Bender said.

Nicolas Le Roux, a Google AI researcher in the Montreal office, later noted on Twitter that the reasoning in Dean’s email was unusual. “My submissions were always checked for disclosure of sensitive material, never for the quality of the literature review,” he said.

Now might be a good time to remind everyone that the easiest way to discriminate is to make stringent rules, then to decide when and for whom to enforce them.
My submissions were always checked for disclosure of sensitive material, never for the quality of the literature review.
— Nicolas Le Roux (@le_roux_nicolas) December 3, 2020

Dean’s email also says that Gebru and her colleagues gave Google AI only a day for an internal review of the paper before they submitted it to a conference for publication. He wrote that “our aim is to rival peer-reviewed journals in terms of the rigor and thoughtfulness in how we review research before publication.”

I understand the concern over Timnit’s resignation from Google. She’s done a great deal to move the field forward with her research. I wanted to share the email I sent to Google Research and some thoughts on our research process.▻https://t.co/djUGdYwNMb
— Jeff Dean (@🠡) (@JeffDean) December 4, 2020

Bender noted that even so, the conference would still put the paper through a substantial review process: “Scholarship is always a conversation and always a work in progress,” she said.

Others, including William Fitzgerald, a former Google PR manager, have further cast doubt on Dean’s claim:

This is such a lie. It was part of my job on the Google PR team to review these papers. Typically we got so many we didn’t review them in time or a researcher would just publish & we wouldn’t know until afterwards. We NEVER punished people for not doing proper process. ▻https://t.co/hNE7SOWSLS pic.twitter.com/Ic30sVgwtn
— William Fitzgerald (@william_fitz) December 4, 2020

Google pioneered much of the foundational research that has since led to the recent explosion in large language models. Google AI was the first to invent the Transformer language model in 2017 that serves as the basis for the company’s later model BERT, and OpenAI’s GPT-2 and GPT-3. BERT, as noted above, now also powers Google search, the company’s cash cow.

Bender worries that Google’s actions could create “a chilling effect” on future AI ethics research. Many of the top experts in AI ethics work at large tech companies because that is where the money is. “That has been beneficial in many ways,” she says. “But we end up with an ecosystem that maybe has incentives that are not the very best ones for the progress of science for the world.”

#Intelligence_artificielle #Google #Ethique #Timnit_Gebru