Retrieval-Augmented Generation (RAG) is revolutionizing the field.

Introduced by Meta AI researchers in 2020, it's gaining attention for its remarkable advantages.

Let's dive into the world of RAG:

RAG, or Retrieval-Augmented Generation, is an innovative architecture designed to leverage the power of large language models (LLMs) while offering the flexibility to incorporate and update custom data as needed.

Unlike the traditional resource-intensive process of constructing bespoke language models or the need for frequent fine-tuning to adapt to evolving data, RAG presents a more streamlined and efficient approach for developers and businesses.

Pre-trained language models, as you might be aware, undergo training using vast amounts of unlabeled text data in a self-supervised manner. This extensive training equips them with a deep understanding of language, built upon statistical relationships in the text.

The key to these models' impressive capabilities lies in their parameters, which encapsulate knowledge that can be harnessed for a wide range of language-related tasks. This is commonly referred to as a "parameterized implicit knowledge base."

While the parameterized implicit knowledge base is indeed remarkable and enables the model to perform exceptionally well on various queries and tasks, it's not without its challenges.

One such challenge is the potential for errors and "hallucinations." These models can sometimes generate responses or information that isn't entirely accurate, highlighting the need for continued improvement and fine-tuning.

The development and adoption of architectures like RAG represent an exciting step forward in the field of Natural Language Processing, promising more efficient and flexible ways to work with language models while acknowledging the complexities they bring.

The Revolution of Retrieval-Augmented Generation

5 free Data Science courses from @Stanford:

▪️ Machine Learning
▪️ Statistical Learning
▪️ Reinforcement Learning
▪️ Computer vision
▪️ NLP

Read more 🧵👇

Machine Learning Full Course taught by @AndrewYNg

This course is an absolute legend.

If you ask someone who's already in ML, they likely say that they started their journey with this course.

http://cs229.stanford.edu/syllabus-autumn2018.html

Statistical Learning

Intro-level course in supervised learning, covering regression, classification, and unsupervised learning with tutorials and detailed sessions.

Emphasis is placed on intuitive explanations and uses R for computing.

https://online.stanford.edu/courses/sohs-ystatslearning-statistical-learning

CS234: Reinforcement Learning

Master the art of Reinforcement Learning (RL) with this comprehensive introduction. Explore core challenges, cutting-edge approaches, and powerful techniques like generalization and exploration.

http://web.stanford.edu/class/cs234/index.html

Computer vision with @drfeifei

A deep dive into the details of deep learning architectures with a focus on learning end-to-end models for these tasks, particularly image classification.

http://cs231n.stanford.edu/

Natural Language Processing with Deep Learning

The focus is on deep learning approaches: implementing, training, debugging, and extending neural network models for a variety of language understanding tasks.

https://www.youtube.com/playlist?list=PLoROMvodv4rOSH4v6133s9LFPRHjEmbmJ

5 Free Data Science Courses from Stanford

ML Research of this week:

▪️ AVFormer
▪️ Barkour
▪️ MatCha and DePlot
▪️ DIDACT
▪️ REVEAL
▪️ Improving mathematical reasoning
& more!

🧵

AVFormer: Injecting vision into frozen speech models for zero-shot AV-ASR

A simple method for augmenting existing large-scale audio-only models with visual information, at the same time performing lightweight domain adaptation.

https://twitter.com/GoogleAI/status/1664680207511322636?s=20

SQL-PaLM

An LLM-based tool adapted from PaLM-2 boasts SoTA performance in both in-context learning and fine-tuning settings.

The few-shot model outperforms the previous fine-tuned SoTA by a whopping 3.8% on the Spider benchmark.

https://twitter.com/omarsar0/status/1664441085693657088

High-resolution image reconstruction with latent diffusion models

This is a model that can read your mind!

https://twitter.com/leafs_s/status/1630906180381057024

Foundation models for reasoning on charts

MatCha is a pixels-to-text foundation model trained on two complementary tasks:

▪️ chart de-rendering
▪️ math reasoning

DePlot is a model built on top of MatCha for one-shot reasoning on charts via translation to tables.

https://twitter.com/hardy_qr/status/1662222363629588485

Saliency Cards

Researchers introduce saliency cards, a structured documentation of how saliency methods operate and their performance across a battery of evaluative metrics.

https://twitter.com/MIT_CSAIL/status/1664663370824392709

REVEAL

A visual-language model that learns to utilize a multi-source multi-modal “memory” to answer knowledge-intensive queries.

https://twitter.com/GoogleAI/status/1664348002780315649

Improving mathematical reasoning

Step-by-step verification beats outcome supervision for training models to tackle mathematical problems, according to recent research.

Plus, the complete dataset of 800K human feedback labels (PRM800K) is now available.

https://twitter.com/DrJimFan/status/1663972818160332800

DIDACT

It's a methodology for training large machine learning (ML) models for software development.
https://twitter.com/jacobaustin132/status/1663972128176128002/video/1

Barkour: Benchmarking animal-level agility with quadruped robots

The paper introduces the Barkour agility benchmark for quadruped robots, along with a Transformer generalist locomotion policy.
https://twitter.com/GoogleAI/status/1662145329180053506/video/1


AI regulations, risks, and safety are heating up 🔥

We've gathered crucial updates and opinions from the latest years.

Stay in the loop 🧵

An article by @TonyZador and @ylecun about AI taking over the world was published 4 years ago. Now more relevant than ever!

https://twitter.com/TheTuringPost/status/1638555260175228929

3 reasons why machines can become more intelligent than people are by @geoffreyhinton

https://twitter.com/TheTuringPost/status/1657034131719569412?s=20

Latest news in the field

https://twitter.com/TheTuringPost/status/1663499376759832577?s=20

How it all went in Congress by Sam Altman

https://twitter.com/TheTuringPost/status/1658826349879119872?s=20

A book about superintelligence published in 2019 by Nick Bostrom

https://twitter.com/TheTuringPost/status/1634601328218365953?s=20

A famous letter to pause AI experiments

https://twitter.com/TheTuringPost/status/1641462670426533890

Another letter about regulations

https://twitter.com/TheTuringPost/status/1663522228481212416?s=20

AI Regulations and Safety Updates

The latest news in the AI regulation field.

▪️ Governance of Superintelligence @sama, @gdb, @ilyasut
▪️ @OpenAI non-profit initiative
▪️ @DeepMind's contribution
▪️ @GoogleAI and EU AI Act
▪️ @MSFTResearch's Governing AI
▪️ @UNESCO and AI
and more!

🧵

Governance of Superintelligence

Sam Altman, Greg Brockman, and Ilya Sutskever set the definite topic for the last week with their paper Governance of Superintelligence.

“Now it is a good time to start thinking,” they said.

https://twitter.com/OpenAI/status/1660700692372410368

But, wait, does anybody knows what is AGI?

We don't even know what AGI actually is, so what are those future AI systems that are dramatically more capable?

https://twitter.com/SashaMTL/status/1663177667137380354?s=20

Mark Surman (@msurman) from the @Mozilla Foundation said to @lmatsakis from @semafor,

“AI has come into people's consciousness because of things like ChatGPT, but it's also just all around us and in everything. And in that sense, not much has changed, right?”

https://twitter.com/semafor/status/1662131227506405376?s=20

OpenAI non-profit initiative

OpenAI names it "Democratic Inputs to AI" and intends to award ten grants of $100,000 each for teams who propose their AI regulations.

https://twitter.com/OpenAI/status/1661811329957781504?s=20

DeepMind (@DeepMind) proposed the use of model evaluation and suggested an actual toolbox to address the extreme risks that arise from the misuse or misalignment of general-purpose models.

https://twitter.com/DeepMind/status/1661741186774884355?s=20

What does Google think about it?

From the tweet of @ThierryBreton, we learned that the EU agreed with Google CEO @SundarPichai to already develop an “AI Pact” on a voluntary basis ahead of the legal deadline of the AI regulation.

https://twitter.com/ThierryBreton/status/1661362714030092289?utm_source=www.turingpost.com&utm_medium=referral&utm_campaign=fod-4-the-governance-of-undefined-superintelligence-and-the-world-of-copilots

UNESCO and AI

UNESCO is actively developing policy guidelines on the use of generative AI in education and research, as well as frameworks of AI competencies for students and teachers in classrooms.

https://twitter.com/UNESCO/status/1662128339052646400

Liked this thread?

Subscribe to our weekly newsletter for free to receive these updates right into your inbox!

https://www.turingpost.com/p/fod4

Current AI Regulation Updates

Introducing our new newsletter series: "Froth on the Daydream" 🎉

We summarize over 100 newsletters weekly that cover AI, ML, and tech. Get up to speed fast!

Check out what we've already covered in this thread 👇

FOD#1: Google leak, Geoffrey Hinton quits abruptly, Joe Biden surprises AI meeting.

Read the full scoop at https://www.turingpost.com/p/froth1

FOD#2: latest on AI hype, Google I/O presentation, and updates from OpenAI, Anthropic, Meta, and more. Plus, we'll dive into the era of ephemeral software.

Check it out at https://www.turingpost.com/p/fod2

Get ready for FOD#3: "In Sam We Trust" and other epic tales of regulations, open-source vs. closed-source, and a new battle for better hardware!

Read now: https://www.turingpost.com/p/fod3

"Froth on the Daydream" Newsletter Series: AI and ML News Summarized

Meta AI recently open-sourced the code for the Massively Multilingual Speech (MMS) project.

It's a series of models for speech-to-text and text-to-speech for 1,100+ languages!

Here is how you can use it with several lines of code 👇🏼

Clone fairseq-py and install the latest version

Download MMS model

Choose one of the preferred models.

Prepare audio file

Requirements:

▪️ Your audio should be in .wav format
▪️ Your audio data should have a 16kHz sample rate

The second line of code is to satisfy these requirements.

Run Inference and transcribe your audio

To choose a language:

▪️ Go to https://github.com/facebookresearch/fairseq/tree/main/examples/mms#asr
▪️ Open Supported languages related to the model you've chosen
▪️ Replace --lang "eng" with the code of the chosen language

✅ Done!

https://twitter.com/ylecun/status/1660732998155640833?s=20

Open Source Speech-To-Text & Text-To-Speech Models for 1,100+ Languages

Have you checked our Twitter Library?

It's a collection of our most influential threads updated every week.

▪️ Open-source models
▪️ Free learning resources
▪️ Useful lists
▪️ & more!

https://www.turingpost.com/t/Twitter%20Library

“There is no question that machines will become smarter than humans—in all domains in which humans are smart—in the future,” says @ylecun.

“It’s a question of when and how, not a question of if.”

5 open-source datasets used to train LLMs

1. LAION-2B-en (text and image)
2. P3 (text, prompts)
3. VIMA dataset (text and image)
4. COYO-700M (text and image)
5. xP3 (text, prompts and code)

Links 🧵

LAION-2B-en (text and image)

https://laion.ai/blog/laion-5b/

P3 (text, prompts)

The Public Pool of Prompts relies on the Hugging Face Dataset library.

https://huggingface.co/datasets/bigscience/P3

xP3 (text and code)

A collection of prompts and datasets across 46 languages & 16 NLP tasks.

https://huggingface.co/datasets/bigscience/xP3

COYO-700M (text and image)

A large-scale dataset that contains 747M image-text pairs as well as many other meta-attributes to increase the usability to train various models.

https://github.com/kakaobrain/coyo-dataset

VIMA dataset (text and image)

https://vimalabs.github.io/