1/68 🧵 AI thread part 3!

This time we look at the impact & future of AI:

- What are AGI (general AI) & superintelligence
- What is GPT missing to achieve AGI?
- Real-world impact of LLMs (does it change everything? will it steal your job?)

2/ And I still didn't get to AI existential risks (the risk AI cause the extinction of humanity). That will have to be for part 4!

Thread unrolled: https://typefully.com/norswap/GwP5n14

3/ I recommend checking Part 1, for background on how neural networks work:

https://twitter.com/norswap/status/1635021780816654337


4/ ... and part 2, which is specifically on GPT and other large language models (LLMs) and their intrinsic limitations (despite being super impressive):

https://twitter.com/norswap/status/1636389832279887872

5/ I'll harp some more on the limitations in this thread — in particular, why I don't think LLM can achieve AGI by themselves and new breakthroughs will be needed, probably even a new paradigm.

But before that, let's talk about what it means to be "intelligent".

6/ "Intelligence" is a poorly defined word. I'd argue that if you consider computational ability, computers have been more intelligent than us for a long time.

ChatGPT is undoubtedly more "intelligent" than many people for many common knowledge work / white-collar tasks.

7/ (In the sense that there are some tasks I'd rather give to ChatGPT than to some humans.)

8/ That's intelligence, then there's AGI, or artificial general intelligence. This just refers to an AI that is as good (or better) as a human on ALL tasks. Or maybe most tasks, or maybe just most intellectual tasks.

9/ A particularly important benchmark here is AI's ability to conduct original research and make new discoveries on its own.

Even more specifically, what happens when AI is able to improve itself (i.e. come up with better AIs or machine learning algorithms)?

10/ This leads us to "superintelligence", a state where AI is vastly more capable than us at everything, because it is able to improve itself rapidly.

11/ Is this even possible? Yes, I think it is — there are no impossibilities at least.

We can imagine the least possible effective way to do this: fully understand the human brain, and improve it.

12/ Because of the messy process of evolution, there is no way our current brain is the best possible thing.

13/ Also merely hooking a brain to a computer to perform on-demand calculations, and to a disk to dump memories should lead to very easy "intelligence" improvements.

14/ The whole debate, then, is (1) how close are we to AGI, and (2) what can/should we do about it?

15/ Within the AI X-risk (that's eXistential risk, to the human race) community, there are two camps: those who think the AI will get superintelligent all of a sudden ("fast takeoff") and those who think it will take more time, and we will have warning signs ("slow takeoff").

16/ (Importantly, this is a debate between those that already agree that AI poses extreme risks, and that this risk will come sooner (in the coming decades) rather than later.)

17/ If you want to read more about fast vs slow takeoff, the way to go is the debate between @ESYudkowsky and @paulfchristiano.

Christiano's article:
https://sideways-view.com/2018/02/24/takeoff-speeds/

Debate between them:
https://www.lesswrong.com/posts/vwLxd6hhFvPbvKmBH/

18/ See also @slatestarcodex take on all of it, w/ useful context:
https://astralcodexten.substack.com/p/yudkowsky-contra-christiano-on-ai

19/ We'll come back to existential risk in part 4. But for now, I want to return to GPT and its potential for achieving AGI.

20/ I believe that LLMs lack two fundamental capabilities to achieve AGI.

1. (easy?) The ability to do formal computation.

2. (hard) The ability to manipulate "concepts", i.e. examine and update the building blocks of its "reasoning" abilities.

21/ Let's tackle computation first, because it's probably a lot easier, though still challenging.

22/ GPT cannot in general compute. GPT3 cannot even get multiplications on large numbers right.

This is related to a point I make in part 2, that GPT can't really "reason" in general, because it can only learn "patterns" but not "meta-patterns".

https://twitter.com/norswap/status/1636389968846417920

23/ And as LLMs grow in size (= the neural networks contain more nodes), they are in fact more likely to fit the existing corpus more closely because this "literality" will be rewarded by their training.

24/ Here is some experimental data pointing in this direction, showing that as LLMs grow, they become less "truthful" because they reflect human misconceptions better.

https://arxiv.org/abs/2109.07958

25/ Another important limitation here is that every completion (for every word generated by GPT) uses up the same (bounded) amount of computation: the weighted sums are computed for the whole network and that is it. (If that's Chinese to you, see part 1!)

26/ Therefore, it's of course impossible for it to solve a problem where the best-known algorithms would use more computation than is available for the completions needed to type the answer!

27/ So how could LLMs acquire computational abilities?

One solution is to decompose. We humans can perform very large products — though not fast, by decomposing it into sums and smaller products (and maybe using a piece of paper!).

28/ If you could find a way to automatically get GPT to decompose its reasoning into smaller pieces, it could probably carry out fairly intricate computations — after all, everything can be reduced to simple arithmetic operation and/or logical gates!

29/ One possible recipe would as though you carried this interactive conversation with ChatGPT, but all done automatically:

1. Pose the problem
2. Ask ChatGPT to reason about how to solve to problem
3. Ask it to generate code to solve the problem.

30/

4. Ask it to reduce the code into a simpler "assembly code" form
5. Ask it to run the assembly code instruction by instruction while maintaining the value of the memory, registers, ...

31/ On the flip side, such a computation will carry an immense overhead compared to an equivalent computer program.

32/ Another issue is that you'd need to maintain a fairly large transcript for performing the computation. (And remember the input for every completion in ChatGPT is the whole transcript of the conversation!)

33/ This is just some speculation of course. Another solution would be to train another network to recognize things that look like computation requests, perform them outside of GPT and then inject the results into the transcript.

34/ This doesn't solve the (difficult by itself) problem of specifying the computation to run, so you'd still need ChatGPT for that.

35/ By the way, here's a good article by @TylerGlaiel illustrating how ChatGPT4 currently does on non-trivial (but also not *hard*) programming problems: https://tylerglaiel.substack.com/p/can-gpt-4-actually-write-code

36/ (LLMs can improve from here but for the reasons outlined above, I believe they can never really get good enough when you ask outside that is too much outside of its corpus.)

37/ Okay, so that was computation.

Let's get to the real hard cap on LLM intelligence, and that is its inability to manipulate "concepts": examine and update how it performs completions.

38/ Given a neural network, we currently can't interpret how it works! LLMs won't be able to develop this ability by themselves, as this motive is completely absent from their training corpus.

39/ Moreover, the only way an LLM has to communicate is to run the completion in the network — which will not give an accurate picture of its inner working.

40/ Why is this important? Concepts are how humans learn in the real of formal topics (math, science, engineering).

41/ It's relatively obvious to see that blind statistical emulation is bound to create errors when things reach a certain level of complexity. This is what we observe of LLMs today, and there is no reason to think this will change (as explained before).

42/ To avoid these errors, you need a conceptual understanding of the matter at hand. Even if these concepts are not couched in a formal language, they fulfill a similar role as formal axioms: they allow and disallow certain state of the world and certain chains of reasoning.

43/ I believe that until we are able to instruct AIs at a conceptual level, and until they are able to reason by manipulating such concepts, there is no chance that they will reach anything resembling general intelligence.

44/ Could we tweak LLMs to perform this conceptual reasoning? I'm out of my depth here but it seems fundamentally incompatible with their architecture.

45/ Maybe a breakthrough could come in a way to extract understanding from the neural network directly. It would be hard to validate this: humans can't interpret networks.

46/ Maybe we can find a way to skip validation, by plugging this directly with the rest of the system and observing the results?

This is not really satisfying: we still don't fully understand how things work, but at least it would maybe seem the AI understands concepts.

47/ That would be *very* unsatisfying from an AI safety perspective: such an AI could lie to us and we would have no way to tell. In fact, we can expect that it will have an explicit concept of "lying"

48/ Such a system would also need a way to update the neural network in response to new concepts.

49/ Speculating a lot, maybe a solution there is to feed it an explanation of the concepts, and then let it auto-generate (in a controlled manner) a bunch of examples which are then used to update the LLM incrementally (= jiggle its weights).

50/ This is speculation for illustration purposes. There might be a way to update LLM to manipulate concepts, but it seems really hard. We're fighting against the gravity of the statistical-pattern completion model!

51/ In fact, in my speculation, notice the two "magical black boxes": one that converts from a neural network to concepts, and one that converts from concept to training data to the LLM.

Any breakthrough here is predicated on solving the mismatch.

52/ It's also possible that I'm completely wrong and conceptual understanding is not required at all. It's my gut feeling supported by really abstract common-sense arguments. I can't deny there is a possibility it's wrong!

53/ I also want to make clear that I believe conceptual understanding is completely possible, it just probably requires a different architecture than LLMs (maybe to be used in conjunction with LLMs), or some fundamental change to the way we "do" LLMs.

54/ Ok. GPT won't take over the world. I believe LLMs will change the world though.

The analogy I keep coming to again and again are social networks. Unlike AI, they seemed relatively innocuous at first, but produced vast social changes.

54/ I think this is the nature of the changes AI will have.

On the good side, they can be used to automate a lot of really boring intellectual work.

On the bad side, they can be used for deepfakes, mass-manufactured subtle propaganda, and elaborate scams.

55/ This will be both slow and fast.

On the fast side, look at how fast Uber & co were to take root.  How it only took 10 years for smartphones to appear and everybody to be utterly dependent on them.

56/ On the other hand, relational databases have been invented in the 70s and you wouldn't believe the amount of paper forms and redundant information I have to file in in 2023.

58/ I think AI is not "priced in" yet. Its productization will have large and probably yet-unforeseen changes.

59/ On the other hand, we see people claiming that AI will change *everything* and that it's the only thing that matters anymore.

I disagree. Internet and smartphones changed things, but a lot of things remain unchanged.

60/ I can be sympathetic to the "it changes everything" point of view, but I think it's predicated on AIs vastly exceeding their current capabilities. I see LLMs and neural network improving still, but as explained in length, they have fundamental limitations.

61/ Will AI take your job? I like this framework: if your job is one of (1) novel, (2) unpredictable, (3) fragile (small errors matter a lot), then you're most likely safe.

https://blog.testdouble.com/posts/2023-03-14-how-to-tell-if-ai-threatens-your-job/

62/ I predict that something similar to what machine did to factory jobs will happen to low-skill intellectual work: slow displacement, with creation of jobs requiring new skills along the way.

63/ e.g., you're in "content marketing", I would be worried, though maybe not if you're involved in the creative direction there.

64/ Final thought. Wen AGI?

It's really hard if not impossible to tell, as it depends on breakthroughs we haven't had yet.

In theory, GPT could have been invented in 2005 (see this article), we just weren't looking in that direction back then!

https://dynomight.net/gpt-2/

65/ What's sure is that GPT has brought tons of attention and capital to the field. But this could be a trap. It's easy (and probably profitable) to build a product on top of LLMs.

66/ Greenfield AI research not building upon LLM will seem a lot riskier. It's still going to get funded more, but now everyone doing it will know they could get rich from a GPT hack instead...

67/ (This is something that plays out in blockchains, where infrastructure work is underfunded and understaffed compared to apps.)

68/ Next time (finally): thoughts on AI safety, and in particular AI existential risks. Some people that I respect a lot think that by 2050 there is a good chance that we won't exist as a species.

What to make of these claims?

Impact & Intelligence of GPT (What is it missing?)

1/63 🧵 Thread on AI — PART 2

This time we'll cover:
- How ChatGPT works (+ many good links)
- Its fundamental limitations: what can't it do?

Unrolled: https://typefully.com/norswap/gPRkWlT

2/ I ran out of space again 😅 So there will be a part 3 (and maybe a part 4) about:

- What are AGI (general AI) & superintelligence
- What is GPT missing to achieve AGI?
- How dangerous is superintelligence?

3/ I strongly recommend reading part 1 for a framework on what exactly neural networks do.

https://twitter.com/norswap/status/1635021780816654337

4/ As GPT 4 was just released, this is an interesting time to make this thread. I haven't read the whitepaper (I should), but it doesn't matter: I actually want to make very generic points what neural networks / LLM (large language models) are capable of.

5/ These points should hold no matter the model, unless the architecture changes in a truly fundamental way (to no longer be truly a neural network, but something more).

We'll speculate about such changes in part 3! (and maybe a bit in this one)

6/ So what sets ChatGPT apart from the networks we saw in part 1 (spam filtering and image-based generation / style transfer)?

Mostly, it's language. Because ChatGPT speaks, it seems intelligent. Or rather, it seems agentic.

7/ We'll discuss what "intelligence" ought to mean in part 3. But for now, let's see what's ChatGPT's got!

8/ GPT is an LLM or "large language model". That just means it's a really big neural network and that it was trained on a lot of input ("the internet") that you couldn't possibly label by hand.

9/ What is GPT's purpose? Quite simply, it's text completion. The model take some text (the "prompt") and generates a completion for it. Amazingly, it does this one. word. at. a. time.

10/ Is that it? Almost. ChatGPT is a model built on top of the GPT "base" model. The model is tweaked using RLHF (reinforcement learning from human feedback).

11/ The purpose of RLHF is to make ChatGPT more useful. The internet is full of people saying things that are incorrect, offensive, or generally not helpful — so pure completion might not be what you want! RLHF tweaks the AI to be a good assistant that follows your directions.

12/ @OpenAI explains RLHF in this article:

https://openai.com/research/instruction-following

14/ In a nutshell:

1. Take the base model.
2. Use a human to generate "good question/answer (QA) pairs" and use this to fine-tune the model to be more friendly.

15/

3. Get some QA pairs form the model, multiple per question (GPT has a random component, can output different answers for the same question), and have a human rank them.

16/

4. Use these rankings to train another neural network, the "reward model", whose role is to evaluate a QA pair. It's trying to mimic what the human ranker would do! (Because the human ranker doesn't scale to all possible QA pairs, but the model can.)

17/

5. Further fine-tune the improved base model from step 2 using the reward model. You now have ChatGPT!

18/ When I said that GPT was "just" a large neural network, of course that glosses over a ton of details. I don't think they matter too much for our discussion, but they are very interesting.

19/ The best technical-yet-approachable, explanation of how ChatGPT works is, according to me, this article by @stephen_wolfram

(At least up to "What Really Lets ChatGPT Work?" — after that it's a lot more speculative, though still interesting!)

https://writings.stephenwolfram.com/2023/02/what-is-chatgpt-doing-and-why-does-it-work/

20/ If you're looking for something more lightweight, try this:

https://www.jonstokes.com/p/chatgpt-explained-a-guide-for-normies

21/ If you're interested in RLHF specifically, this article dives into the details:

https://www.assemblyai.com/blog/how-chatgpt-actually-works/

22/ ChatGPT's "prompt" is the whole transcript of the conversation you've had with him.

OpenAI further adds a hidden message at the start of the conversation in order to prompt the AI to be extra helpful. This changes regularly, but here's what it was in January:

23/ This turns out to have a remarkable impact (there's a figure showing this in the OpenAI RLHF article above).

People have also figured out a bunch of other prompts you could give ChatGPT to be more generally useful, or better at this thing or that thing.

24/ These practices, plus the guidelines on how to write your questions/requests to optimize the results have come to be known as "prompt engineering".

25/ Here are some links I collected on prompt engineering:

- https://github.com/dair-ai/Prompt-Engineering-Guide
- https://prompts.chat/
- https://oneusefulthing.substack.com/p/how-to-use-chatgpt-to-boost-your

26/ People *love* to show how to get ChatGPT to ignore its hidden prompt (and some of its RLHF-derived inclinations), a practice known as "jailbreaking", but the phenomenon is really unsurprising if you understand ChatGPT capabilities ... which we're about to get into right now.

27/ Okay, back to the big picture stuff. What do the details of how ChatGPT work tell us about its capabilities?

28/ First off, what does GPT learn?

As a text completer, it makes statistical inferences on text.

Said otherwise, it learns "patterns" of text.

29/ What's really surprising is how elaborate these patterns can be. They're not simple mappings of the form "after these 10 words, the most likely are ...". Instead they're more like computer programs performing arbitrary learned computation.

30/ Again, I want to hammer that these "patterns" and "programs" are just weights in a neural net that does a bunch of weighted sums (see part 1!).

31/ The neural net cannot access/reason about these patterns (only use them to generate completions) and we humans can't currently understand or interpret them very well.

32/ Ted Chiang (who apparently is both a great author and a really clear thinker) wrote a great New Yorker op-ed that I want to use to illustrate some of these points.

https://archive.is/kEdyu

33/ Chiang frames ChatGPT as lossy compression: even if the model is large, it is a lot smaller than the web (even if you were to prune it of redundant information).

34/ Therefore, it cannot contain all factual information, and this explains its "hallucinations" (a technical term that refers to when LLM "make things up").

35/ On the other hand, he also points out how sometimes there are really good lossless compression techniques: for instance, you can compress all of the arithmetic text into a fundamental understanding of the arithmetic rules (i.e. a "program" that performs arithmetic!)

36/ Careful here though, as we are conflating a corpus of text and a set of facts!

37/ If there are facts or answers that most people get wrong on the internet, it is in theory more correct for the LLM to generate factually incorrect completions, unless we somehow manage to bias it away from these bad answers (this seems difficult in general).

38/ LLM make the same mistake as people. One cool illustration:

https://twitter.com/danrobinson/status/1635764590444449792

39/ Chiang is probably a little bit too bearish on the creative potential of GPT.

Because GPT learns a bunch of patterns, it is able to combine them in novel ways, even when the patterns or subject matters weren't seen together in the corpus.

40/ And it can be argued that most creative endeavors are just that: remixes.

I highly recommend watching the "Everything is a Remix" video, for a vivid illustration of this:
https://www.youtube.com/watch?v=nJPERZDfyWc

41/ There's also a new version of the series here: everythingisaremix.info/watch-the-series

42/ At the same time, it's true that ChatGPT is probably not able to come up with something *truly* novel (even if prompted), whether that's creative elements or original research.

43/ And these creative elements might not be most of our creative output, but they're really important, as they unlock an exponential amount of new combinations.

44/ Another point I want to make with respect to GPT's capabilities is that it is not able to "reason" in any meaningful way.

We've seen that LLMs are able to learn patterns. In theory, they could get good enough to perfectly learn the rules of arithmetic, logic, ...

45/ It can also combine these patterns. So, in theory, and if prompted intelligently, it should be able to derive new theories that are proven by the patterns it knows.

I think this runs into intrinsic limitations really quickly.

46/ Why? To derive new theories, it's usually necessary to assemble a bunch of new intermediate facts (lemmas, theorems).

Say GPT is able to derive the first layer of those. These are all novel, and hence "foreign", i.e. unlikely in text!

47/ The patterns that GPT learns are all subservient to its main goal: complete text.

Carrying out a reasoning using these new facts is bound to seem very unlikely as a text completion.

48/ There is a "gravity pull" where LLMs are bound to pull back completions towards things they have seen before.

Making correct inferences with respect to the new theory seems very unlikely, because it doesn't look like anything in the corpus.

49/ To even be able to reason using new theorems, GPT would need to learn "meta-patterns". Let me explain.

Basic patterns are mere word associations. Advanced patterns could be the rules of logic or arithmetic.

50/ Meta patterns would be patterns where, given new rules never seen before (which it derives from its existing patterns), it is able to reason using these new rules.

51/ You could argue: GPT can do that today! If I give it the rule of a simple but novel game, it is able to play!

That's true, but consider it has seen a lot of simple games and that maybe all it is doing here is applying rules in the "theory of simple games".

52/ In this context, rules (which are fairly simple themselves) are simply objects manipulated by this theory!

In other words, the "reasoning" necessary to carry out the rules ("the theory of simple games") is fairly common in the corpus and can thus be learned.

53/ It gets a lot more difficult when reasoning using the new theorems is a lot more foreign.

54/ Imagine a LLM whose corpus included things functions (f(x) = 3x + 2) but nothing about calculus.

Reasoning using derivatives and integrals are bound to look unlikely from a text completion perspective.

55/ So can LLMs learn meta-patterns? I might be wrong, but I would argue it is disincentivized by their very structure.

56/ By nature, what they will learn is how to apply existing theories because that's what they see in the corpus. They have no need to go to the meta level, it does not make them more accurate during training!

57/ Maybe if we found a way to "crank up the compression" we could force the model to go meta?

That seems difficult — the progress of LLMs so far has been due in great part to making them *larger*.

58/ And we can't try to compress some part of the "knowledge graph" of the LLM because there is no such thing!

There is just a bunch of weight and we don't generally which regions of the neural network correspond to which capabilities.

59/ ((I'm out of my depth here, I'm not sure that regions even match capabilities.

60/ Important to know that every neuron/node fires for *every* completion, however due to weights and the "activation function", a lot of them end up outputting zero or low value for any given completion. So... maybe?))

61/ Finally, as far as research is concerned, I'll point out that while it's generally "easy" to verify the validity of a reasoning that leads to a new result, the whole difficulty is finding the intuition that leads to the new reason in the first place.

62/ Can GPT serve as a good intuition pump for those? Maybe, but I doubt it: new results are often weird, and require out-of-the-box thinking. Exactly the opposite of something that tries to give you the most statistically significant answer!

63/ That's it for this thread!

Tune in for part 3, where we'll touch on what it means for an AI to be (super)intelligent, and what they'd need to get there!

ChatGPT and Its Limitations (AI Part 2)

1/57 🧵 Long overdue thread on AI — PART 1

We'll cover:
- What is artificial intelligence?
- What is machine learning? (+ example: spam filtering)
- The basics of neural networks
- How StableDiffusion & DALL-E work (high-level)

Unrolled: https://typefully.com/norswap/SUF6F2F

2/ I had to split this thread because it became too big :')

Next time I want to cover:
- How ChatGPT works (+ many good links)
- The fundamental limitations of neural networks
- AGI (general AI) & super AI: how close are we / what is missing
- How dangerous is super AI?

3/ Why this thread: too many bad takes on AI's relevance in the future. I wanted to write something I could point people to in future discussions on the topic :)

DISCLAIMER: This will probably contain some mistakes / simplifications / inaccuracies. Feel free to point them out!

4/ What even is artificial intelligence?

It covers a lot of things. In the 80s, a Makefile was considered AI (it's a rule-base "expert system"), so were logical programming languages like Prolog.

5/ A classical AI curriculum will walk you through fairly straightforward optimization techniques, including tree search + pruning and ordering heuristics (e.g. MinMax).

6/ Other technique includes:

- Linear Programming (LP
- Integer (Linear) Programming (I(L)P)

These help you find an optimal solution to a problem that can be defined as a set of linear (in)equalities.

7/ (The integer version restricts the variables to be integers.)

- Gradient descent — a way to find the minimum (or maximum) in a multi-variate function.

8/

- Lagrange multipliers —  a way to convert a constrained problem into a function, such that it can be solved via gradient descent.

9/ The gradient descent is particularly fundamental, and is an important component of the algorithm ("backpropagation") that helps train neural networks (more of this in a minute).

10/ Some links to familiarize yourself with gradient descent:
 - https://betterexplained.com/articles/vector-calculus-understanding-the-gradient/
- https://builtin.com/data-science/gradient-descent


11/ All of these techniques are well-suited for solving complete information games (e.g. chess) and optimization problems.

For example, check the Google HashCode competition problems, e.g. https://codingcompetitions.withgoogle.com/hashcode/round/00000000008f5ca9/00000000008f6f33)


12/ These optimization problems are also very common in the field of logistics: establishing schedules under a set of constraints / or preferences (e.g. to schedule exams or classes), vehicle routing, filling delivery trucks with packages Tetris-style, ...

13/ But people kind of take all of this for granted. While these problems are really difficult, machines have been better at them than us for a long time.

14/ In fact, machines' greatest asset is their ability to crunch a bunch of data/paths really fast. The "intelligence" we inject into these algorithms are heuristics and guide rails to get to a solution even faster.

15/ The reason why this is not so impressive to us is that it's based on *formal* reasoning. Rules are expressed in an unambiguous language that computers can understand and process.

They just run a guided search program that we wrote — and can reason about to some extent.

16/ The recent AI hype is all about machine learning, and in particular neural networks. Neural networks feel magical for numerous reasons.

17/ (1) We are not explicitly coding the rules or the heuristics that the machine follows. Instead, these are *learned* from a corpus of examples.

18/ (2) We can't really reason about what exactly the neural network has "learned". It's all encoded in a bunch of numbers ("weights") which are hard/impossible for humans to interpret. Much more on this in part 2.

19/ (3) Because they don't operate on formal data or rules, they're able to deal with ambiguous/fuzzy data, like human languages and pictures/art. These were things that were far outside the realm of machines, and that we thought were intrinsically human!

20/ Later we will look at what image generation models like DALL-E and StableDiffusion do. In part 2, we will look at large language models (LLM) like ChatGPT as well.

21/ But I'd like to start a bit earlier in machine learning history and work our way to the more recent findings. I hope it'll show that things are perhaps not as magical as they appear.

22/ One of the first big splashes of machine learning was bayesian filters. Most notably, this was used to filter spam.

23/ On the practical side of things, this was spearheaded by none other than @paulg: http://www.paulgraham.com/spam.html

25/ To train a bayesian spam filter, you start with a corpus of emails that have been annotated to be spam or not.

26/ From this, you can easily derive:

P(spam) = proportion of mails that are spam
P(viagra) = proportion of mails containing the word "viagra"
P(viagra | spam) = proportion of spams containing the word "viagra"

27/ From there you can combine these values to get P(spam | viagra).

Do this for enough relevant words and expressions that often appear in emails, combine all of them into a consolidated score, and you get a spam score that you can use to classify emails as spam or not!

28/ This is textbook machine learning: starting from a corpus of examples, we used statistical inferences to extract patterns allowing us to perform classification.

29/ From there, neural networks are only a step away!

From a high level, training a neural network entails tweaking a random computation until it gives you the results you want. For instance, accurate spam classification.

30/ How is the computation defined? It typically looks something like the picture below.

31/ Your input is cut into bits or small numbers and fed into each of the nodes (aka "neurons") in the "input layer". These are connected to all the nodes in a second "hidden" layer.

32/ Each edge between nodes of the two layers is associated with a "weight", such that the value of any node in the second layer is a weighted sum of the values of ALL the nodes in the input layer.

33/ Repeat this process for as many hidden layers as required, and finally for the output layer, which will give your results (e.g. a value between 0 and 1 representing a probability of an email being spam).

34/ You tweak this computation by progressively jiggling the weights around, improving the results as you go.

35/ The process/algorithm that causes the weight to change such that the result improves is called "backpropagation".  We won't enter into the details, but it suffices to say it is based on our old friend the gradient descent.

36/ Using this process, we can reconstruct a bayesian-style spam filter. The process is just less direct and more random.

37/ During training, the algorithm will "notice" that as it penalizes the word "viagra" the accuracy increases, and so you'll end up with sensibly the same result.

38/ (A point that will become crucial later: there is no such explicit "noticing", this is all mediated by pretty boring equations.)

39/ Of course, neural networks have many other applications. One that has been huge for more than a decade now is image classification: recognizing pictures of cats, human faces, etc, etc...

40/ The reason why neural networks work so well here is that, unlike text, the features of images are really hard to formalize.

41/ Better to let our random/self-correcting backpropagation algorithm figure out what the features of interest are directly, by operating directly on the pixels of the image!

42/ Using some visualization techniques, we can "peer" into the neurons of a layer to try to see what features they are computing.

The picture below is taken from an article by @stephen_wolfram which I will link to in part 2.

43/ So far, we're not stretching credulity too much. But what about DALL-E and StableDiffusion?

In this case, the network was trained on a corpus of pictures with textual descriptions. What was learned was an association between text and the features of images.

44/ Just like how a classifier develops an "understanding" of the features of a cat, StableDiffusion is able to generate pictures that possess these features.

45/ (Again: this "understanding" is not an explicit concept — you can't query the neural network for its understanding of the concept of a cat. You can only ask it to recognize or generate pictures of cats.)

46/ It all sounds easy when I said it like this, but there are a lot of pesky challenges. It's not that easy to learn what the features of a thing are.

47/ For instance, if your corpus only contains pictures of cows on a backdrop of grass, your neural network is unlikely to recognize a picture of a cow lounging on the beach as a cow.

48/ DALL-E, StableDiffusion, and ChatGPT work around these challenges by using the stupid amount of data that is available on the internet as a corpus.

49/ They still sometimes fail for basically this reason. It's really hard to generate things that are too much outside the box.

e.g. @afk0b has not really been able to generate a "plank anime waifu"

50/ Before we get to ChatGPT, let's quickly mention another cool neural network use: style transfers.

51/ Style transfer can be explained as basically learning a "recipe" for converting an image to a certain style. You could think of it as a series of actions to do in Photoshop, based on the content of the image to transform.

52/ e.g. "identify and preserve shapes outline"
e.g. "smudge the image using an airbrush"
...
(Again, none of that is explicit, it's just a bunch of weights + backpropagation.)

53/ Style transfers are an example of "generative adversarial networks". In this model, we actually train two networks. The network that actually performs the task, and a second "adversarial" network whose goal is to generate additional training data.

54/ The trick is that the adversarial network is trained to "fool" the main network (or in this case, make it produce bad outputs). This creates somewhat of an arm's race: when the main network is fooled, it improves to reduce the chances of this happening again.

55/ The adversarial network, on the other hand, has to become better at fooling the ever-improving main network.

56/ And that's it for today! Tune in sometime next week for part 2, where we will talk about ChatGPT, its capabilities, and speculate about what the future of AI might look like. (Are we all going to ☠️?)

An Overview of Artificial Intelligence and Machine Learning

1/54🧵 Thread on @VelodromeFi and why it's a superior system for incentivizing liquidity.

2/ Liquidity is super important for project tokens. A lack of liquidity makes the token price too volatile. It also prevents investors for exiting, which makes them wary to invest in the first place.

3/ (And more cynically, such a liquidity pool also doubles as another use case for the token, to prevent excessive selling.)

4/ The problem is that, for unpegged pairs, being a liquidity provider (LP) sucks, because of impermanent loss (IL). Price goes up? You would have been better holding and not LPing. Price goes down? Same thing.

5/ As an LP, you are the counterparty of adversely selected trades. You end up owning more tokens when price decreases, and less tokens when price increases.

6/ The IL numbers are not egregious, but can easily outpace fee APY. You can play with this app to get a sense: https://baller.netlify.app/

7/ For this reason, projects need to incentivize liquidity. Usually, this is done by bestowing tokens on LPs, to increase APY and make IL risk worthwhile.

8/ In this context, Velodrome and its predecessors/competitiors (@CurveFinance + @ConvexFinance, @solidlyexchange) offer two things:

1. A marketplace for liquidity: no need to roll you own pools & incentive logic, easy to advertise to prospective LPs

9/

2. The opportunity to purchase 1$ worth of incentive for less than 1$ worth of project token.

10/ How does that work? Every voting period (one week for Velodrome), token rewards ($VELO) are emitted and distributed amongst the LPs of the liquidity pools on velodrome.

11/ The amount of rewards per pool is decided by a vote. A pool receives an amount of reward proportional to its share of the total vote. To vote, users have to lock their $VELO (for up to 4 years, longer locks beget more voting power).

12/ Multiple strategies are possible. Projects can just acquire $VELO directly and lock it: this lets them direct a stream of future emissions to their pools. The point being for the price paid for $VELO will be less than the value of all future emissions.