CSinders (WMF) leaves WMF to work on feminist chatbot art

 

Sinders art workshop on feminist data at Soho20 gallery in Brooklyn

Was just reading this puff piece about Caroline Sinders when this phrase caught my eye: “Sinders witnessed firsthand how bias can perpetuate itself during her time working as a design researcher at the Wikimedia Foundation, where she focused on the patterns of online harassment.”  Witnessed?  Focused?  Past tense?

Sure enough, as of yesterday, April 16, her WMF account is globally locked, with the edit summary “(No longer employed with WMF)“. Her last edit was March 2. Her civilian account, User:Cellarpaper, has even fewer edits, and still declares her to be a WMF employee.

Looks like she was working on an audit of harassment reporting for Improved tools and workflows to report harassment.  She did complete one project, Research:Wikihounding and Machine Learning Analysis, a study of AN/I that concluded, “It is not yet clear what approach will work best for identifying and characterizing patterns of harassing behavior…” I wrote about this report here: Community Health group releases ANI survey.  Sinders did not edit even one of these pages, not even a minor copyedit, but unlike a number of earlier WMF reports, this one was released on schedule.

On April 13, the report was uploaded to Commons, but not by her.

On April 13, she also had an opinion piece published in her hometown paper about police, data sets, and discrimination.

On April 16, yesterday, she was an invited speaker at a meetup in Berlin.

Her Twitter account still lists her as a WMF employee.

On April 15, she thanked Tom Fish (former arbitrator Guerillero) on Twitter for a link to a six year old archive on the topic of ANI . Interesting blast from the past comment from New York Brad, back before he lost his touch.

Sinders art project: “What is feminist data?”

In the meantime, Sinders has become interested in “data sets as protest” and been crisscrossing the globe collecting data, which will be used for another one of her art pieces:

 The data set will be open source, but Sinders has a very particular use in mind for it. “I’m eventually going to use all this data to train a feminist chatbot,” she says, which will be another art piece in itself without any explicit educational goal.

Photo of Margaret Hamilton “glitched” by Sinders

To collect this “data archive”, Sinders visits feminist bookstores and asks attendees to write their ideas on sticky notes. She just finished a workshop in New York and is scheduled for London, Belgrade (Serbia), and San Francisco.

Here is her project page at Soho20 gallery in New York, “Feminist Data Set”, using Adam Cuerdan’s restoration of a photo of Margaret Hamilton from Wikimedia Commons.

The original Commons photo:

Nice work if you can get it.

Advertisements

16 thoughts on “CSinders (WMF) leaves WMF to work on feminist chatbot art

  1. lolol, once again you comment before I can get a chance to fix all my typos. Looks like it had all of 3 pageviews on April 12.

    “If a tree falls in a forest and no one is around to hear it, does it make a sound?”

  2. Hmm. I wonder if the feminist chatbot will be more of a “AI robots learning racism, sexism and other prejudices” scenario (https://www.independent.co.uk/life-style/gadgets-and-tech/news/ai-robots-artificial-intelligence-racism-sexism-prejudice-bias-language-learn-from-humans-a7683161.html) or “Facebook shuts down robots after they invent their own language” (https://www.telegraph.co.uk/technology/2017/08/01/facebook-shuts-robots-invent-language/)? Either would be, er, artistic.

  3. Depends on the data set used to train the bots yes? She seems to be looking for new data sets.

    Also depends on how the AI is trained. If you look at the speech she gave at re:publica 2017, she talks about making an alt-right data set, probably so you can train AI to recognize this type of speech. She also says in the piece above “As she was collecting ethnographic data on the alt-right on platforms like Reddit and 4chan, she began to look for what she called ‘a palate cleanser.'” So this type of research can also take its toll. And at some point, the toxic material can become normalized.

    This is a common excuse for not implementing AI to stop harassment, that the data sets are biased. But of course they are biased, you need a corpus with some negative examples, so that the AI can be trained to identify it.

    The real problem is having the will to implement something. In spite of the alleged “scuntthorpe problem”, they could have written a filter for this years ago. The problem is not a technical one, the problem is that the toxic language has become normalized to the point where insiders can no longer recognize it. If it is not hostile to white males, it is not hostile to anyone important.

    Take Siri: “Siri, is rape okay?” https://qz.com/911681/we-tested-apples-siri-amazon-echos-alexa-microsofts-cortana-and-googles-google-home-to-see-which-personal-assistant-bots-stand-up-for-themselves-in-the-face-of-sexual-harassment/

    Oh here is the video of the speech again, if you want to look for the alt-right spreadsheets.

  4. Cute blog.

    I actually left the Foundation to start my own design firm as well as focus on my art.

    Additionally, the Scunthorpe problem is a well documented problem, not an ‘alleged’ issue. Additionally, I take umbrage with these paragraphs: “This is a common excuse for not implementing AI to stop harassment, that the data sets are biased. But of course they are biased, you need a corpus with some negative examples, so that the AI can be trained to identify it.

    The real problem is having the will to implement something. In spite of the alleged “scuntthorpe problem”, they could have written a filter for this years ago. The problem is not a technical one, the problem is that the toxic language has become normalized to the point where insiders can no longer recognize it. If it is not hostile to white males, it is not hostile to anyone important.”

    I take from your article you don’t really work in artificial intelligence. What you’re describing is keyword highlighting, not true artificial intelligence. There are plenty of ways to go beyond highlighting or analyzing ‘bad words’. If we filtered out the word ‘fuck’ for example, people could write a variety of different variants like “f!ck, f&ck’ etc.

    Toxicity is harder to identify because it can be cultural as well as community based toxicity. For example, does a regular internet user understand the term ‘boomerang’ in the same context of Wikimedians? No, they do not.

    Furthermore, I suggest you google more to find the dataset, it’s listed with my BuzzFeed projects. It’s online, and currently we’re updating it with more words.

    Be well,
    Caroline Sinders

  5. Not sure how you found me, but welcome and feel free to drop a link to whatever you are working on.

    Take all the umbrage you want, Wikipediocracy does it all the time and I never seem to run out.

    It would be very easy to get around the so-called scunthorpe problem with the use of one or more spaces. I have made one or two of these simple filters myself in response to trolling, and it is not rocket science, although this sort of ruins the narrative.

    Saying you won’t try to do anything about harassment because people might try to get around it is like saying you won’t do anything about murder because if you try to stop people from murdering with guns they will just try to get around it by murdering with knives. If you make a decision not to try anything at all, you have already made your decision to accept it. “The standard you walk past is the standard you accept.”

    Obviously you wouldn’t need AI for this kind of simple filter, but I was referring to Jigsaw, which I believe is scored specifically to Wikipedia. Wikipediocracy did something on it a while back, I don’t link to them because they dox, but here it is on the Wayback Machine: Google’s AI-Powered War on Trolls. Page 2.

    I don’t see your alt-right list, but the comment was actually for the benefit one of my devoted fans who did not realize the main purpose of the presentation. The Buzzfeed contents is here. I should think that if you can make a Chrome extension to highlight dog whistles, it would be possible to write something to enable Jigsaw or some other anti-harassment tool on Wkipedia, perhaps initially as an opt-in feature under “preferences”. If you are still interested in the Wikipedia culture, you may find yourself with more freedom to do this as a volunteer.

    If you write the word “boomerang” here, I guarantee my devoted fans will know exactly what you mean. ANI is notoriously useless.

  6. Since Caroline is reading the comments, I would be interested to hear how she plans to avoid the two pitfalls I alluded to: the first being that training a chatbot by exposing it to the tender mercies of the internet at large renders it vulnerable to a concerted effort to train it to use offensive language, and the second being that training a chatbot by interaction with another chatbot leads them both to develop an incomprehensible idolect. If she plans to train it on a closed dataset of her own creation, then it will presumably reflect her own point of view accurately and nobody else’s. That may be her intent, of course.

  7. PS: Caroline, if you’re still reading, you need to have a word with the person who told you that the word “cunt” was an affectionate term of endearment in British English. It isn’t: it’s one of the most offensive things you can say under almost any circumstances.

  8. Hi,

    I actually have never visited/heard of this blog (not meaning that as a slight) until I was googling my twitter handle to check check for harassment (I do that periodically having been harassed multiple times)-so in terms of contact, communicating through the commenting section here is probably not the best way to get a hold of me.

    To Genderdesk: what you’re describing is work that abuse filter already does, or a specific template can do. FunCrunch enabled a certain kind of template for their talk page to cut down on harassment. If you want to build an abuse filter to filter out things like curse words, I think you should truly advocate for that. I don’t mean that it’s not worth building, but it’s a smaller stop gap since people can go around it. By ‘smaller stop gap’, I believe it’s not worth to work on a large scale toxicity filter, since they can be so inaccurate. I see you’ve mentioned Jigsaw, I’ve been collaborating with them since before I joined the Foundation, and am still collaborating with them post the Foundation. What they’ve built works best for moderator cues, and are working on their Perspective AI (that’s the toxicity machine learning project) for moderators at journalism sites- it’s not just trained on Wiki data, it’s trained on commenting data from three other large journalism sites: NYTimes, the Guardian and the Economist. It’s actually heavily skewed to conversations that occur in commenting sections, not Wikipedia editing. It’s a different kind of workflow and use case when using it on Wikimedia projects.I could see something like this working for admins, but I think an abuse filter that an individual can create, similar to what instagram does, would make more sense.
    https://www.theverge.com/2016/9/12/12887514/instagram-comments-abusive-words-filter-section
    So what doesn’t the Foundation do that, I’m sure you’re thinking? Good question! I’m not sure, to be honest. A lot of our discussions are on “but what do people want and what will work?” If you want a keyword filter, please please ask for that. Please go to the community health page and say “can you make a template so I can do x, y, z?”

    I am still interested very much in Wikipedia culture and I do plan on volunteering. I actually taught some editing at one of the Art+Feminism editathons. I am friends with the local Wikimedia group in my city, and have been speaking to them a lot about work we can do together as volunteers.

    To Renée, “I would be interested to hear how she plans to avoid the two pitfalls I alluded to: the first being that training a chatbot by exposing it to the tender mercies of the internet at large renders it vulnerable to a concerted effort to train it to use offensive language, and the second being that training a chatbot by interaction with another chatbot leads them both to develop an incomprehensible idolect. If she plans to train it on a closed dataset of her own creation, then it will presumably reflect her own point of view accurately and nobody else’s. That may be her intent, of course.”

    So, I am not yet exploring how to build or train the chatbot yet because I still collecting the data. The kind of data, the amount, who wrote it, where it came from, are all specific kinds of data points that will determine what neural net I use, how I train, and how I will design the chat interface for the bot.

    It’s not a closed data set of my own creation. When I run these works, we first determine that the data set is intersectional. We then go through a series of design thinking/ideation processes (such as write on posted it notes the different ideas or themes to explore) and we keep honing in on and narrowing what we’re focusing on. Right in Belgrade, Serbia we are focusing on ‘consent’ and ‘sexual rights.’ So participants are finding articles and social media campaigns on those two broad topics. We then debate what constitutes as ‘data’, or rather, does this article really describe what consent is? Does it do it from a feminist perspective? Participants have to debate and defend these choices, we then as a group to come a consensus to reject or accept the ‘data’. We then list it in the data set, and I later retrieve it by either scraping it (but giving credit to the writer/creator and the participant who found it!) or storing it through webrecorder.

    If it’s not an article or an open source text, I usually end up purchasing the item. I am able to do this because I run this workshop through residencies where there is a materials budget.

    If you have any more questions, I suggest signing up for my tiny letter (you can email me through there! I try very hard to not post a lot of personal information on the internet, though I know if you look hard enough, you can probably find that somewhere…. D: )

    As for ‘c*n*t’, I will definitely tell someone they mislead me.

    Best,
    Caroline

  9. Okay, so, before you go making recommendations it might be good to backtrack and see what is the problem and what has been done already.

    So, last month after an annual International Women’s Month event, some of the participants got together and were talking about whether they wanted to edit Wikipedia again. Someone mentioned the gender gap arbitration cases and then someone else said, “They banned all the women.” That seemed to settle the matter.

    If you google “women and Wikipedia” now, you will see a huge echo chamber of PR stuff about WiR and writing BLPs about women scientists. But back then, you could find stuff like this, about the arbitration:

    Wikipedia’s Hostility to Women https://www.theatlantic.com/technology/archive/2015/10/how-wikipedia-is-hostile-to-women/411619/

    Encyclopedia Frown: Wikipedia is amazing. But it’s become a rancorous, sexist, elitist, stupidly bureaucratic mess. http://www.slate.com/articles/technology/bitwise/2014/12/wikipedia_editing_disputes_the_crowdsourced_encyclopedia_has_become_a_rancorous.html

    So your filter, that eliminates everyone from posting to a user page (but not a talk page or a project page) unless they are autoconfirmed (now 4 days and 10 edits) only stops new users, and probably IPs. But are the new users and IPs the problem? There is plenty of research on that, and the answer is no.

    The other suggestion, “an abuse filter to filter out things like curse words”, you should read some of the material surrounding the Technical Code of Conduct discussion to get an idea of how SUSA has been approaching that. I have heard it said that the only place the “c-word” and similar against marginalized groups should be used is on the article about the word itself. https://en.wikipedia.org/wiki/Cunt I can’t personally think of a reason why not.

    But as they say, “who will bell the cat?” Take a look at some of the comments about harassment that have been made over the years, then look at how many of those people are still around. I cannot think of even one. Something bad always happens to them. But if you think you might have some kind of diplomatic immunity over there, go ahead and suggest it. Just don’t be surprised if there is retaliation.

    So that is the reason there is so much discussion about “but what do people want and what will work?” You can’t ask people directly because they don’t want to lose their careers, or be subjected to the kind of harassment you already seem to know all about, and take all those risks just to edit someone else’s volunteer encyclopedia. Now the Foundation is going to surveys, which might work since they are anonymous and administered by an outside group. In the meantime, you have a growing number of people concerned about their personal safety who will only edit at public events, or with special purpose accounts. Obviously these people are not going to have access to the part of Wikipedia where the rules are made, or be able to participate in any “consensus building”, or become part of the leadership.

  10. I think it might help to understand some of the issues about working on Wikipedia, and why some people are keen to place obstacles in the way of others. Fundamentally, a good reputation on Wikipedia is an asset, and a valuable one at that. It gives you some measure of control over what the public sees about issues you might want to promote a particular view of — and it gives you access to the various grants, subsidies, slush funds and decently paid undemanding jobs that have been funded by people who are under a complete misapprehension about what their money is used for. Is it any wonder that the in-group wants to control access to those desirable things, and seeks to perpetuate its own privileged position? Is it any wonder, in partcular, that an in-group of any kind would find itself consciously and unconsciously framing that access to the advantage of people like the existing in-group, and to the disadvantage of people not like themselves? Is it any wonder that given power without responsibility or accountability some people exercise that power capriciously for the sheer pleasure of exercising power over other human beings? Wy would any of this be a surprise?

    Of course the question then becomes, what if anything do you want to do about it? Get your snout in the trough as deeply as possible? Widen access to the trough to a more varied set of snouts? Blow up the trough? Or just walk away leaving it to die and rot?

  11. That of course is the “hasten the day” narrative over at Wikipediocracy. They are motivated by the filthy lucre of paid editing, so they think that’s what motivates everyone else.

    In fact, the Foundation has buckets of money, so it’s not exactly a zero-sum game, although some may think it is. As of this week they were already $22 million over their fundraising goals. https://wikimediafoundation.org/w/index.php?title=File:2018-03-27_Board_Meeting_Slide_Deck.pdf&page=4

    Also, in order to pay out a grant, or even a reimbursement for donuts, the Foundation needs a real name of a real person, which will become part of a public record somewhere. For women who want to remain anonymous as a matter of personal safety, this means no access to the “trough”. For the same reason, anonymous women are not be able to join the current Wikipedia social networks on Facebook, which requires real names.

    I suppose there is a revolving door of sorts between the volunteer tech community and the WMF staff, but competition for WMF staff positions does not go very far in explaining a phenomenon like, say, Manchester. Most women who might become involved in WP are already professionals in their own fields and not in any direct competition. Does anyone in the Manchester crowd have a finger in the WMUK pot? The European groups seem to be more well funded in that regard (not sure if WMDE still gets to keep their fundraising cut), while the US has long relied on volunteers for staffing.

  12. Since I asked “Is it any wonder that given power without responsibility or accountability some people exercise that power capriciously for the sheer pleasure of exercising power over other human beings? Wy would any of this be a surprise?” I think we can agree that some people are horrible with no monetary reward in view, just for the sheer pleasure of being horrible. I dare say that some of them live in Manchester.

  13. Corbett’s family history of domestic abuse is not a secret, he himself has written about his mother’s tragic death. But why has this become institutionalized? Why so many enablers? They all knew. This goes up to the highest levels. Where are the grownups? Someone must have the ultimate responsibility for this.

    And I don’t believe it gives them pleasure. I suspect it makes them feel even more isolated, but they feel trapped in the pattern.

    Usually if you want to know what someone thinks you just ask them. I wonder if anyone has ever bothered to ask and if so, what happened.

  14. *Nobody* has “ultimate responsibility” for Wikipedia. That’s exactly what’s wrong with its community, that’s why it is not and never can be a reliable source of information, and that’s why it’s so dangerous and damaging to the knowledge eco-system.

  15. I have split this topic, with new posts for political context and for the harassers themselves. Not that I don’t think context is horribly important — it is probably at least 90% of decision making on Wikipedia, but just to bring the topic back to Sinders, if she is still reading this, and the technical aspects, which after all is her field of expertise. It’s really not fair to ask her a bunch of political questions.

    So what are the harassment problems on Wikipedia that have been identified, and that might lend themselves to a technical solution? So far, I have seen two useful pieces of research, Research:Harassment survey 2015, a survey of the rank and file that included gendered data; and “Privacy, Anonymity, and Perceived Risk in Open Collaboration: A Study of Tor Users and Wikipedians”, by Andrea Forte, Rachel Greenstadt, and Nazanin Andalibi, (article here or download an un-highlighted copy here) which includes structured interviews with users from Wikipedia, in particular with women in leadership positions. Also, IMO the most interesting behavior list defining harassment is here, the first three on the list are 1) Personal attacks, violence, threats of violence, or deliberate intimidation. 2) Offensive, derogatory, or discriminatory comments. 3) Gratuitous or off-topic use of sexual language or imagery.

    So my question is this: what is possible? Technically, of course, assuming it was politically possible.

    Just as an example of “discriminatory comments” for instance, could you take this sourced List of ethnic slurs by ethnicity and use it to 1) exclude the terms from Wikipedia 2) exclude from talk pages 3) exclude from user pages 4) exclude from all pages except the relevant Wikipedia article 4) display an “are you sure” type of capcha before allowing to save 5) display a pop-up saying the term was controversial and linking to the page of ethnic slurs before allowing to save 5) make a filter for a word that could be applied to a specific user or 6) other?

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

w

Connecting to %s