I am very excited to announce that a joint proposal w/ @AiEleuther @Mila_Quebec and @laion_ai to train Foundation Models on @ORNL's Summit supercomputer has won a 5.94M V100-hour INCITE grant.

The work we do at @AiEleuther is incredibly compute-intensive, but it is thanks to grants like this that we are able to continue to train and publicly release cutting edge research on large transformer models.

This was made possible by the work of a great many people: @QuentinAnthon15 @EricHallahan @AiEleuther;
@JJitsev Christoph Schuhmann Mehdi Cherti @laion_ai; @irinarish @introspection @MilaQuebec; Guillermo Cecchi @IBMResearch; @rioyokota @tokyotech_en; Sergey Panitkin @StabilityAI

It's often very challenging for academics and non-profit researchers to gain access to the large amount of compute required to do this kind of work, but with first the @BigscienceW and now this things are boding well for the future of open science large scale AI research.

INCITE 2023 Award

This plot from @OpenAI's Scaling Laws for Neural Language Models is widely referred to when discussing training dynamics, but did you know it has almost no bearing on what massive LM training curves look like? (1/12)

You can read this thread unrolled at: https://typefully.com/BlancheMinerva/mPialqw

Here are the same plot (as the one on the left) from GPT-NeoX 20B, for both training and validation loss. Why don't we see the same effect of a "burn-in" period? 🤔🤔🤔 (2/12)

I asked this same question in the #EleutherAI discord an hour ago and was blown away by what an eagle-eyed observer called thrasher pointed out: my plot starts too far to the right for the burn-in to show up! (3/12)

When looking at OpenAI's plot, it's easy to assume that it represents the training curve of large models. But 10^9 is "only" 1B: my model is 20B params and was trained for 400B tokens. The burn-in period ends at around the 20M token mark (4/12)

That's ~0.05% of the way through training! I logged my first loss value at 300 steps, thinking that was surely early enough to reveal anything interesting. But 300 steps of a model with a batch size of 3.1M is nearly 1B tokens: my logging can only capture the tail-end (5/12)

of the plot. The OpenAI plot shows two phase transitions, one just before 10^8 and one just before 10^9. The first phase transition occurs in the first 0.05% and the second in the first 0.25% of the training of my 20B parameter model. For Google's recent PaLM, (6/12)

those numbers drop to 0.025% and 0.125% respectively. More than 99.5% of training occurs in the "flatlined" regime after the second phase transition. Below are various evaluation benchmarks. Note that the *entire curve* is to the right of the second phase transition (7/12)

So clearly there's a lot of interesting stuff happening that OpenAI's plot doesn't tell us about.

Warning: these didn't go until the end of training. The final eval numbers for the model are:
Lambada: 0.720
HellaSwag: 0.535
PiQA: 0.779
Winogrande: 0.661

(8/12)

(I don't have MathQA and PubMedQA yet).

In my 3 month training run we blew past all the stuff it shows in two days! That early in training LLMs are still producing complete garbage; they don't have an idea of spelling let alone grammar or anything of substance.
(9/12)

WandB for GPT-NeoX: https://wandb.ai/eleutherai/gpt-thicc/reports/GPT-NeoX-20B-Pretraining--VmlldzoxMTg5MjY3

So what is going on during this rapid loss improvement if eval scores still suck? @AnthropicAI has been doing some research which may hold the answer: https://transformer-circuits.pub/2022/in-context-learning-and-induction-heads/index.html
They claim that few shot (my evals are all 0-shot) (10/12)

performance improves substantially. Additionally, they claim that this is where “induction heads” begin to appear. I haven’t looked into this personally yet, but it seems like a very interesting set of ideas. And it does make sense for 0-shot to develop after few-shot IMO (11/12)

As a reminder, intermediate checkpoints for GPT-NeoX 20B are available, DM me if you would like to experiment with them.

Also DM me if you want to experiment on GPT-NeoX but lack the compute! EleutherAI is happy to provide free compute to anyone doing research on it.

(12/12)

What do loss curves for LLMs look like?

Google decided that 137B and 280B weren't enough, so now they've gone and trained a 540B model.

https://ai.googleblog.com/2022/04/pathways-language-model-palm-scaling-to.html

Chinchilla is *hugely* punching above its weight here. Damn. 

@SashaMTL @TaliaRinger Hmmmm I coulda sworn I recently read something about how LLMs are Good for the Environment Actually (TM) because they're multitask models and one training run supports a lot of deployment, and yet here we are.

@SashaMTL @TaliaRinger Has anyone studied the value of drawing straight lines on log-log plots through three data points? We see stuff like this all the time, but using these plots to interpolate seems extremely suspicious to me. 

@SashaMTL @TaliaRinger It's cool to see Ben and @arankomatsuzaki's parallel layers get picked up outside of #EleutherAI!

"RoPE has been shown to have better performance on long sequence lengths" is a bit of a funny thing to read since @OfirPress's ALIBI seems to beat it on this specific task. 

Axillary z-loss shows up again. Has anyone experimented with this? I haven't, but it seems like it's gaining traction. 

This would be a god-send in a hypothetical world in which people have access to the checkpoints. Expect this to power some interesting papers coming out of Google about training dynamics in the near future though. 

Some interesting info contra GPT-NeoX and the Big Science Research workshop paper on training instability. Both papers claim spikes are caused due to bad data. @StasBekman do you recall ever running this experiment? I didn't, though now I'm interested in seeing if I can. 

@StasBekman Evaluating memorization is continuing to make it into the mainstream reporting on language models. 

@StasBekman Very cool to see them reproduce Carlini et al.'s equation so closely. The actual scaling law curves are usually pretty sensitive to the model choice, but here it's not. Given the architectural similarity, maybe this really is a big EleutherAI model?

https://arxiv.org/abs/2202.07646 

@StasBekman I'm mostly joking. That caught my attention, but drawing a conclusion like that at this stage would be highly premature. It would be very interesting if memorization scaling laws transferred though, especially across models trained on different datasets. 🤔🤔🤔

@StasBekman BIG Bench was designed to be very hard for current LLMs to solve, but it may last less time than intended... performance is clearly shifting towards the AI. I'm pretty curious where my task (indentify_math_theorems) falls on this plot, but I don't see the raw data anywhere. 

@StasBekman Check out the benchmark here: https://github.com/google/BIG-bench/

Yes, I know it's hard to use if your model isn't in TensorFlow. I would like to get that fixed but it's not my project. I might take a stab at integrating it with the EleutherAI LM Eval Harness again soon though.

@StasBekman Everything in @emilymbender's thread here is spot on. It's also worth pointing out some math people regularly slip up on: 30% of documents have names, 10% of names are in BookCorpus -&gt; 26.7% of names are not in BC.

https://twitter.com/emilymbender/status/1511123935085006848?s=20&t=NiG3T46EHkuiTg5NU4oROg

@StasBekman @emilymbender Yes I know those numbers don't add up, it's actually 11% in BC I just mistyped. If it were 10%, then 27% of names would not be in BC.

Big OOF

https://twitter.com/emilymbender/status/1511129363210670082?s=20&t=NiG3T46EHkuiTg5NU4oROg

Thoughts on Google's PaLM, a 540B parameter language model

A little over one year ago @CoreWeave approached #EleutherAI and asked how much compute we needed to train the largest open source language model in the world. Today, we are finally releasing the weights of a 20B parameter LM trained on the Pile.

@CoreWeave Tech report: http://eaidata.bmk.sh/data/GPT_NeoX_20B.pdf
Repo: https://github.com/EleutherAI/gpt-neox
Slim Weights: https://mystic.the-eye.eu/public/AI/models/GPT-NeoX-20B/slim_weights/
Full Weights: https://mystic.the-eye.eu/public/AI/models/GPT-NeoX-20B/full_weights/

@CoreWeave It has been a hell of a journey, trying to train large models during a global GPU shortage, and we are deeply indebted to @CoreWeave for working with us every step of the way to figure out how to source the compute we needed.

This has been the result of the work of countless people, but I especially want to should out @realmeatyhuman, our lead developer, and  @EricHallahan, my go-to man for things nobody wanted to do. That said, this was very much a group effort as the 17 names on the tech report and

over two dozen names on the GitHub repository shows. If I had thought ahead I would have collected everyone’s Twitter handles ahead of time, but 🤷‍♀️

This is far from the end of the line, both for this research project and for #EleutherAI in general. We have T0 style multitask finetunes and models finetuned on code coming soon™️, collaborations with @Mila_Quebec and @BigscienceW, and much more going on and coming soon.

As always, the best way to keep track of what’s going on in #EleutherAI or keep in touch with our brilliant researchers is to come hang out in our discord server. I know I’m biased, but it’s been a life changing experience for me. Come check it out: https://discord.gg/fR34TdvVtc

If you’re wondering what this “EleutherAI” thing is, I recommend checking out our one-year retrospective for a look at our history, our research, and some extremely niche memes 

https://blog.eleuther.ai/year-one/

GPT-NeoX 20B Release

Excited to share my newest paper, "Neural Language Models are Effective Plagiarists" with @EdwardRaffML. We took a dataset of CS 101 assignments and asked "can a language model do a good job solving these with minimal human intervention or knowledge?"

 https://arxiv.org/abs/2201.07406 

@EdwardRaffML There's been some very interesting work recently on solving college level assignments with transformers, but that work typically uses private models and more complicated pipelines. We wanted to focus on what was available to a random student with the internet, not an AI expert.

@EdwardRaffML If you're interested in my thoughts about training and evaluating LLMs after writing this paper, check out the thread here: https://twitter.com/BlancheMinerva/status/1484278387061493761?s=20

@EdwardRaffML To do that, we stuck with #EleutherAI's GPT-J, freely and publicly available at http://6b.eleuther.ai/. We used no prompting, no finetuning, and no tricks. 

We just entered the text of the assignment into the website several times and then tried to run the results in a Java compiler. We fixed minor errors that were solvable by reading the assignment text or by Googling the errors the code spit out, but that's it.

The solutions that GPT-J produced were often wrong. But they were also often correct. It only took a couple minutes to obtain a piece of code that correctly solved the problems we gave it, even with minor bug fixing and using a public demo API. 

The next question we turned to was whether these solutions would pass a plagiarism screening with the widely used MOSS software. We took a dataset of assignments that contained both original and plagiarized solutions to compare to.

In this image, different levels = different plagiarism techniques. We find that MOSS rates GPT-J's outputs as being as different or even more different from the original solution than the non-plagiarisms and the plagiarisms produced with the most advanced techniques. 

But even if GPT-J's solutions are different from the reference solution, these plagiarisms might be detected if they're too internally similar. To address this, we applying ISOMap to the pairwise MOSS similarity scores. We find quite low inter-cluster similarity for GPT-J's code

Black and green are original solutions, red and yellow are plagiarisms of the black dot specifically (yellow = more sophisticated plagiarism), and GPT-J is blue. Different shapes represent different assignments. 

We also looked at memorization: is this simply the result of the code being memorized by the transformer? We find that the answer is a resounding NO: no correct solutions are verbatim copies of text from the training data.

While examining prompting is outside the scope of this paper, we do take a look at one particularly noteworthy aspect: how specifying the programing language matters. Notice that GPT-J will sometimes give code in the wrong language, even when told what language to use. 

Our work leaves a lot of exciting avenues for future work including:
- using professors to rate the plagiarisms
- using actual students to find the solutions (In my defense, I know zero Java)
- using harder assignments (ours were limited by needing plag. data)

If you want to go read through the generations, you're more then welcome to! We included the full text of all 15 generations for all 7 assignments in the appendix. 

Neural Language Models are Effective Plagiarists

This is the only thread on NFTs you need to read. https://twitter.com/smdiehl/status/1445795667826208770

A lot of people retweeting this are talking about how "the community" is what makes this different. But having a vibrant community doesn't make something financially sound. World of Warcraft has (had?) a vibrant community, but that doesn't mean we should

conflate gold in WoW with real money.

People are also saying "but it's fun." That's totally fine. I spend money on things that are fun all the time. That's what money is for: buying utility. And if that's what your attitude is towards NFTs then great.

But NFTs are not:
- a career
- an investment vehicle
- ownership of an object, physical or virtual
- the future of the internet
- a way of life
- a social identity

And you see people saying stuff like this all the time. What worries me about NFTs is the cult surrounding it

If everyone's fine walking away in five years saying "that was a lot of fun and a worthwhile way to spend my money" then great. But the space is flooded with people who don't talk like that.

Here are some examples:

"NFTs... completely change the rules of ownership. Transactions in which ownership of something changes hands have usually depended on layers of middlemen to establish trust in the transaction, exchange contracts and ensure that money changes hands."
https://theconversation.com/nfts-are-much-bigger-than-an-art-fad-heres-how-they-could-change-the-world-159563

“Never before has a comedian asked for fans to invest,” Raybould says. “They’ve asked for fans to send them money, and they send them back a T-shirt or put their name in the credits [of a movie]. But do you want that, or do you want cold, hard cash?”

https://www.vox.com/the-goods/22700655/cryptocurrency-invest-nft-whiskey-playboy

Stella Contra NFTs

A couple corrections, resources, and parting thoughts on my thread about @StanfordAILab @StanfordHAI's Workshop https://twitter.com/BlancheMinerva/status/1429858134772834305

@StanfordAILab @StanfordHAI AI Ethics is not something invented in the past 10 years. It's something that tech companies have been finally compelled to pretend to care about in the past 10 years. A reading list of work that predates 2000.

https://twitter.com/BlancheMinerva/status/1429870544451670018?s=20

@StanfordAILab @StanfordHAI @jackclarkSF said his mention of mental health chatbots was inspired by WoeBot. The only study I found on WoeBot indicated it was on par with directing people to the NIMH website.

https://mental.jmir.org/2017/2/e19/

@StanfordAILab @StanfordHAI 1996: "Bias in computer systems" by Batya Friedman and Helen Nissenbaum performs case-studies of deployed technologies
https://dl.acm.org/doi/10.1145/230538.230561

@StanfordAILab @StanfordHAI Training GPT-3 is cheap, but public funding for AI is remarkably stingy compared to other sciences. If you can provide 5M in funding or access to 256 A100s with Infiniband interconnect I can release a 175B language model to the public this year. DM me. 

https://twitter.com/BlancheMinerva/status/1429863327191154689?s=20

@StanfordAILab @StanfordHAI I think that's everything I wanted to remark on / correct. If I missed anything or you feel I misrepresented something you said don't hesitate to reach out.

@StanfordAILab @StanfordHAI I criticized @jackclarkSF for a framing that excluded work by #EleutherAI and @BigscienceW, but I missed the beginning of his talk and didn't realize he was talking specifically about orgs who had already produced 175B+ models.

https://twitter.com/BlancheMinerva/status/1429861103069237255?s=20

@StanfordAILab @StanfordHAI To follow-up on my mention that people had already written compelling things about the workshop framing and naming, you can find some threads below

https://twitter.com/BlancheMinerva/status/1429859178504409088

@StanfordAILab @StanfordHAI 1992: "Human issues in the use of pattern recognition techniques" by Alan Dix looks at the human context and consequences of using neural networks to do pattern recognition

https://alandix.com/academic/papers/neuro92/neuro92.html

@StanfordAILab @StanfordHAI If you want to learn more about red teaming AI, come hang out with @aivillage_dc on discord or follow all-stars like @rharang @ErickGalinkin @moo_hax @ram_ssk @ruchowdh
https://twitter.com/BlancheMinerva/status/1429872388339113985?s=20

@StanfordAILab @StanfordHAI @jackclarkSF @BigscienceW I also missed @percyliang's opening address which lauded #EleutherAI and @BigscienceW quite heavily right before @jackclarkSF spoke, and is something I know he agrees with from public and private conversations.

@StanfordAILab @StanfordHAI @mer__edith on the politics of the workshop's framing
https://twitter.com/mer__edith/status/1427699822719651843

@StanfordAILab @StanfordHAI @emilymbender on content curation and issues with the concept of context-agnostic modeling. 
https://twitter.com/emilymbender/status/1429817716458672135

1980: "Do Artifacts Have Politics?" by Langdon Winner is a classic piece that connects technology with politics and social power, and identifies two distinct ways in which computer systems and technologies can be political.

https://www.cc.gatech.edu/~beki/cs4001/Winner.pdf

@StanfordAILab @StanfordHAI @MadamePratolung on the politics of language and labels
https://twitter.com/MadamePratolung/status/1428392092124987393?s=20

While this blog post isn't quite as old, 2008's "Gay marriage: the database engineering perspective" is essential reading for anyone designing systems for categorizing humans

https://web.archive.org/web/20100221005910/http://qntm.org/gay

See also the 2015 same-sex marriage update: https://qntm.org/support

1998: "An Ethics For The New Surveillance" by Gary T. Marx discusses the ethics of data collection and technological surveillance

https://web.mit.edu/gtmarx/www/ncolin5.html

1994: "Thinking through Technology: The Path between Engineering and Philosophy" by Carl Mitcham contrasts and synthesizes "the engineering approach" and "the humanities approach" to critical analysis of tech and develops criteria for evaluating tech

https://press.uchicago.edu/ucp/books/book/chicago/T/bo3683920.html

1984: "The Imperative of Responsibility: In Search of an Ethics for the Technological Age" by Hans Jonas is a critique of (then) modern technological utopianism grounded in "man’s duties toward himself, his posterity, and the environment."

https://press.uchicago.edu/ucp/books/book/chicago/I/bo5953283.html

Thoughts on Stanford's "Workshop on Foundation Models" (corrections and links)

Okay, time to live tweet my thoughts on @stanfordnlp @StanfordAILab's "Workshop on Foundation Models." A long thread.

First and foremost: please never use the phrase "foundational models" every again. It's a garbage name that people like @mmitchell_ai @emilymbender @mer__edith have criticized at length. I'll go find some of their comments and link to them later, but the short version is:

@mmitchell_ai @emilymbender @mer__edith 1. There is very little intellectually "foundational" about these models
2. It's not at all clear that GPT-3 and CLIP-DALL-E are the same kind of thing
3. The motivation for this relabeling appears to be entirely about political control over language

I missed @percyliang's intro talk, so I'll start with @jackclarkSF's

1. Jack says that everyone training or trying to train 100B+ language models are companies. This omits #EleutherAI and @BigscienceW from the narrative (note both groups were also excluded from the workshop)

2. Jack says that these models will obviously do good, but his examples are highly suspicious. He specifically positively raises examples like AI therapy chatbots which mental health experts are virtually universally against @MentalHealthAm

@MentalHealthAm 3. Jack is dead right about recommendation algorithms and their insidious impact on our behaviors.

4. I'm surprised by how critical he is willing to be about capital. Distinguishing between corporate and capital interests is an important and meaningful thing to do.

@MentalHealthAm We cannot forget that there are many people who are personally wealthy enough to fund the training of GPT-3. The CEO of any Fortune 500 company can do it without any meaningful impact on their lives.

If anyone reading this has 5M to spare, my DMs are open! I can train and release a completely open and free 200B language model with 5M in funding.

It's really funny to hear people talk about how GPT-3 is absurdly expensive. @mmitchell_ai compared it to @CERN a couple days ago.

@mmitchell_ai @CERN I fear that people without experience with large scale science are missing important context of scale. I am employed by a US Government contracting firm. We have a saying: a billion here, a billion there, and pretty soon you're talking about real money.

@mmitchell_ai @CERN My company has received over 100x the amount of money it would cost to train GPT-3 for AI research. The USG doesn't want a freely available GPT-3. If they did, it would exist.

Reminder that the US military spends more money in AI research than the entire private sector combined.

@mmitchell_ai @CERN It's also important to keep in mind that "freely available" is a complicated concept at scale. GPT-J (coming soon on @huggingface) is approximately the largest model that current tech can deploy on consumer GPUs.

https://github.com/huggingface/transformers/pull/13022

@mmitchell_ai @CERN @huggingface When I talk about making a 100B model or a 200B model "freely available" what I really mean is that the weights are published online. For you to actually use it, you'd need to take it to a cloud provider and pay them to use it. But at least @Microsoft wouldn't have a monopoly

Jumping ahead to the current panel because I have strong emotions about this:

Making these models publicly available is a prerequisite for auditing them. The current paradigm of private models is fundamentally at odds with auditing. And this is deliberate!

These models are very expensive and very profitable. The companies that train these models explicitly and deliberately censor research that critically examines large language models. This includes work like @mmitchell_ai @timnitGebru's Stochastic Parrots
https://www.technologyreview.com/2020/12/04/1013294/google-ai-ethics-research-paper-forced-out-timnit-gebru/

@mmitchell_ai @timnitGebru But I'm actually far more concerned about the censorship of security and interpretability research. A leaked internal email written by Carlini reveals that his recent paper was censored extensively by Google

https://www.reuters.com/article/us-alphabet-google-research-exclusive/exclusive-google-pledges-changes-to-research-oversight-after-internal-revolt-idUSKBN2AP1AC

@mmitchell_ai @timnitGebru Carlini's paper was about measuring memorization of training data. Forget about ethics and policy work for a second: if this kind of foundational research on how models work is censored, we have no hope of building a real understanding of how these models work let alone

meaningfully evaluate whether their use is a good idea. You can't talk about the ethics of technology if you don't know how the technology functions. And @GoogleAI doesn't want you to know how it functions.

Empowering this kind of work is explicitly a reason why #EleutherAI is working on building and releasing large language models: https://blog.eleuther.ai/why-release-a-large-language-model/
And I'm happy to say that we've already begun to see it
https://arxiv.org/abs/2107.06499
https://arxiv.org/abs/2105.05241
https://arxiv.org/abs/2107.07691

I missed the name of the woman who is currently speaking, but she is 100% spot on about these models already being deployed in ways that harm people and that people don't want.

These models are already being used to monitor political dissidents.

These models are already being used to spy on people.

These models are already being used to send people to jail.

If your understanding of the harms that these models do doesn't start with "these models are currently being used to violate human rights" and go downhill from there, you're quite poorly calibrated.

Police use shotspotter AIs put people in jail. But these algorithms don't work, or worse are openly fraudulent. In one example, an employee manually reclassified and then changed the recorded location of a “gunshot” when contacted by the police and asked to look for it.

When the defense filed a Frye motion, the prosecution withdrew all shotspotter-based evidence rather than defend it's scientific validity: https://www.vice.com/en/article/qj8xbq/police-are-telling-shotspotter-to-alter-evidence-from-gunshot-detecting-ai

Someone just said "who would have thought about these things 12 years ago."

I hate to break it to you, but computer security, social impacts of technology, and ethics were not invented in 2010. LOTS of people were thinking about this 12 years ago, and even many years prior

Have you never watched a sci-fi movie? Seriously?

Speaking of which, here's a great exposé on algorithms pitched as “moneyball meets the minority report” by people who somehow didn’t realize that the cops are the bad guys in that movie. 

https://projects.tampabay.com/projects/2020/investigations/police-pasco-sheriff-targeted/

This is a point I made on @mtlaiethics's panel recently:

https://twitter.com/mtlaiethics/status/1397595001698279429

@mtlaiethics @percyliang brings up "ethical red teaming." There's actually a whole community of "AI hackers," and at @defcon the AI security group @aivillage_dc recently ran the first "AI Ethics Bug Bounty" with @Twitter. @ErickGalinkin and @ruchowdh lead this effort

https://twitter.com/BlancheMinerva/status/1419791511051452418

I really need to write a blog post soon about my Rule 1 of technology: if it's the primary activity of the villian in a sci-fi book or movie, don't do it.

https://twitter.com/acidflask/status/1429883659058913288

@mtlaiethics @percyliang @defcon @aivillage_dc @Twitter @ErickGalinkin @ruchowdh I was in a rush and omitted @rharang from the list in this tweet. He was also a core person behind making this competition happen and is an awesome ML read teamer in his own right.

@mtlaiethics @percyliang @defcon @aivillage_dc @Twitter @ErickGalinkin @ruchowdh The panelists are correct that evaluation suits are insufficient to understand real-world deployment contexts. In other fields we do studies of deployed systems and publish them. Not so in ML, because companies won't let you. I've tried at my company and I know people who have

@mtlaiethics @percyliang @defcon @aivillage_dc @Twitter @ErickGalinkin @ruchowdh tried at @GoogleAI and @MSFTResearch.

If we want to make progress on this we need to figure out how to get the data out of companies and into the public eye. Because I promise you it exists. It's just not something we are allowed to talk about.

@mtlaiethics @percyliang @defcon @aivillage_dc @Twitter @ErickGalinkin @ruchowdh @GoogleAI @MSFTResearch Which reminds me I should probably make a disclaimer: My day job is doing work for @BoozAllen, a US military contractor commonly referred to as the largest private intelligence agency in the world. Booz and the US government in no way condone or support anything I say

and would probably specifically disown most of it. Booz does a lot of classified work, and nothing I say should be considered to be a comment on any non-public programs at Booz or the US government, be it classified or otherwise.

Basically everyone who work in cutting edge AI research is muzzled in one way or another, and I am no different. This is one of the major reasons to democratize public discussion IMO... I can say things about Stanford and Google that employees there can't, and other people can

say things about my employer that I can't. We must wear these muzzles to get access to technology and ideas that enable us to function as researchers. That's how the ML world works right now.

Anyways, back to the panel:

You cannot rely on the US government to regulate AI technology. For over a decade the reality of the world has been that people who are rich are above the law. You set aside funds to pay token judgements, and then you go profit off violating US Law

If the US government is going to regulate AI technology in more than in name, the very first thing that has to happen is that the *minimum* judgement against a company for violating US regulatory law is the total profits accrued by that company. And then you need to pile punitive

fines on top of that. If that doesn't happen, regulation is a joke. This isn't a uniquely ML problem, for example the Sackler family made 13 BILLION dollars creating the opioid crisis.

https://www.wsj.com/articles/sacklers-received-12-billion-to-13-billion-in-profits-from-oxycontin-maker-purdue-pharma-11570221797

I tried and failed to find a clip of this actually

https://twitter.com/voretaq7/status/1429881682245607436?s=20

It currently looks like they'll be paying approximately 4 billion dollars, and in exchange they'll get complete immunity for their blatant and deliberate illegal behavior that has killed tens of thousands of people

https://www.npr.org/2021/06/02/1002085031/sackler-family-empire-poised-to-win-immunity-from-opioid-lawsuits

Participatory approaches in AI are awesome, but they're not the solution to harm. Read this phenomenal paper about it: https://arxiv.org/abs/2007.02423

I also make remarks about this in my paper at a NeurIPS workshop last year with @wjscheirer: http://proceedings.mlr.press/v137/biderman20a

@wjscheirer It's interesting to hear alignment finally come out. I was wondering about this. I'm actually currently writing a position paper on the behalf of #EleutherAI about our attitudes about the release of technology.

I am hoping to have that out by the end of the month, though my migraines have already caused significant delays.

Anyways, I really want to see more communication and collaboration between what I'll call the "AI ethics" and the "AI alignment" communities. Neither community

is that positively disposed towards the other in my experience, and I think that's a major shame. This is something I am trying to work on within #EleutherAI, which is broadly speaking aligned to the alignment community rather than the ethics one.

It's cute to listen to people talk about "countries that don't have the same attitudes towards human rights" given the widespread abuse of human rights done by the US government.

LAPD Assistant Chief Horace Frank said “it is no secret” that the LAPD uses facial recognition, that he personally testified to that fact before the Police Commission a couple years ago, and that the more recent denials were "just mistakes."
https://www.latimes.com/california/story/2020-09-21/lapd-controversial-facial-recognition-software

SF banned the use of facial recognition technology for policing, so SF Cops pretend it’s okay to have other people use banned AI tech for them: https://www.sfchronicle.com/bayarea/article/Facial-recognition-tech-used-to-build-SFPD-gun-15595796.php

Going back to @shotspotter, no independent audit or study has ever supported it's claims. A MacArthur Center for Justice study found 86% of gunshot reports resulted in no evidence of a crime: https://endpolicesurveillance.com/

@shotspotter And yet it's deployed widespread in the US. My mom is a public defender in DC, and she says the majority of cases she sees have "evidence" provided by @shotspotter

@shotspotter Yes, China is a human rights abuser. But so is the US, both internally and externally. It's extremely hard to get meaningful data on how often the US military murders non-combatants abroad, but it looks like it's at least two per day on average: https://watson.brown.edu/costsofwar/files/cow/imce/papers/2020/Rising%20Civilian%20Death%20Toll%20in%20Afghanistan_Costs%20of%20War_Dec%207%202020.pdf

This is Merc. He’s really cute, but he doesn’t seem to want to let me tweet 

I'm getting a lot of messages and replies. I'm going to try to get to everyone, but it will take a while. I'm not ignoring you, and don't hesitate to nudge me again.

@shotspotter I like the question about how Yejin Choi got youtube data in violation of youtube's ToS. I'm really impressed that she was willing to straight up just answer that they violated Google's ToS. It's a lot more common to try to hide this fact behind weasel words.

@shotspotter In case anyone's curious, she's spot on about it being legal. Something widely ignored in AI research (including in the AI Ethics literature) is that ToS are basically pieces of paper. They are not legally binding, researchers *probably* have a pass to ignore them in the US

@shotspotter due to free use even when they do express actual legal rights, and website owners typically fundamentally don't  care.

Researchers are also very quick to ignore inconvenient truths about their data. People knew for years that ImageNet contained child pornography. I knew this in 2016 and I wasn't even an AI researcher in 2016. But until it was a liability to the authors, it was fine.

A current example is the "OpenSubtitles" dataset. The OpenSubtitles dataset claims to be a public dataset of scripts of movies. Despite what the paper and the original website say, it is in no way in compliance with US copyright law.

As far as I know, the Pile is the only paper that uses the dataset and admits that the website is a massive copyright violation. This dataset was actually the motivation for section 6.5 in the Pile paper, where we draw three senses in which

https://arxiv.org/abs/2101.00027

a dataset is (ethically) available for public use. As I said, it's probably fair use to use the data so legally it's not at issue (in the US at least). 

I would love to learn if I'm wrong about this. @mmitchell_ai @haydenfield @mer__edith @emilymbender do any of y'all know a paper or blog post that openly discusses the fact that some dataset authors lie about copyright compliance? With OpenSubtitles or anything else as examples?