A complete data science roadmap for beginners🤖🚀

▶ Roadmap, tech-stack, learning resources, & projects to build a portfolio along with Timeline📅⏲

A thread🧵↓

📅 Step-by-step guide form start to finish data science until applying for a job📝

A Roadmap to follow:🛣↓

1. Python 
2. Math & statistics
3. SQL/No-SQL
4. Pandas, Matplotlib, Numpy
5. Excel & Tableau
6. Machine learning
7. Deep learning 
8. Deployment-AWS/GCP
9. Git/GitHub

1. Python

⁕ Level of skills: Basics to intermediate

⁕ Time allocation: 21-25 Days

⁕ Project Idea: Count down calculator

⁕ Learning outcome: Build python programming skills

⁕ FREE Learning resources:
🔗 https://www.w3schools.com/python/default.asp


2. Math & statistics

⁕ Topics to learn: linear algebra, stats, optimization, probability

⁕ Time allocation: 30 Days

⁕ Learning outcome: get your hands on key math concepts to learn data science and ML

⁕ FREE Learning resources:
https://www.khanacademy.org/math

3. SQL/No-SQL

⁕ Tools to learn: SQL, MySQL, MongoDB 

⁕ Time allocation: 20 Days

⁕ Project Ideas: Extract and analyze Employee data using SQL&MySQL

⁕ Learning outcome: Get your hands on RDBMS

⁕ FREE Learning resources:
 🔗 https://sqlbolt.com/

4. Pandas, Matplotlib, Numpy

⁕ Level of skills: UPTO intermediate

⁕ Time allocation: 30-45 Days

⁕ Project Ideas: Google play store data analysis using pandas, NumPy, and matplotlib

⁕ Learning outcome: Learn Key DS libraries

⁕ FREE Learning resources:
🔗 https://www.w3schools.com/python/matplotlib_intro.asp

5. Excel/Tableau

⁕ Tools to learn: Excel, power query, tableau, 

⁕ Time allocation: 20 Days 

⁕ Project Ideas: HR analytics dashboard

⁕ Learning outcome: Get your hands on Excel and tableau

⁕ FREE Learning resources:
🔗 https://www.youtube.com/c/ExcelTutorials/videos

6. Machine learning 

⁕ Alogorithms: Regression, classification, clustering

⁕ Time allocation: 30-35 Days

⁕ Project Ideas: Players auction price prediction

⁕ Learning outcome: Build regression and classification models

⁕ Learning resources:
🔗 https://developers.google.com/machine-learning/crash-course/ml-intro

7. Deep learning

⁕ Alogorithms: CNN, RNN, LSTM

⁕ Time allocation: 40-50 Days

⁕ Project Ideas: digit recognizer,  images classifier

⁕ Learning outcome: Build image classification models

⁕ FREE Learning resources:
🔗 https://www.freecodecamp.org/learn/machine-learning-with-python/

8. Deployment

⁕ Tools: Heroku, streamlit and AWS

⁕ Time allocation: 30 Days

⁕ Project Idea: deploy credit card default model on Heroku

⁕ Learning outcome: get hands-on deployment of ML projects

⁕ Learning resources:
🔗 https://www.coursera.org/learn/mlops-fundamentals

9. Git/GitHub

⁕ Skills: Get familiar with Github/ learn to use GIT commands

⁕ Time allocation: 10 Days

⁕ Learning outcome: learn to work with GitHub and work in group projects

⁕ Learning resources:
🔗 https://www.coursera.org/learn/introduction-git-github?irclickid=VLIXqcQX4xyIUq2WaWTSN2NBUkDwX8T0LSstxM0&irgwc=1&utm_medium=partners&utm_source=impact&utm_campaign=1193708&utm_content=b2c

Now to summerise everything you need to become a data scientist at ONE PLACE🤯

One of the best notebooks of everything you need to learn🔽

Have DATAI's notebooks as your go-to reference for anything data science

...and thank me later

https://www.kaggle.com/kanncaa1/code

You can read the unrolled version of this thread here: https://typefully.com/avikumart_/M3QcUgw

A complete data science roadmap

SQL is the most used database querying language but it's slow and inefficient

To query the database efficiently @esProc_SPL has developed a faster alternative to fetch data with fewer lines of code

Learn more about the same👇
https://blog.scudata.com/the-impasse-of-sql-performance-optimizing/

A major difference between SQL and SPL is that SPL uses a set of data types and operations to accelerate the performance of database querying.

SPL does not rely on relational databases or databases to fetch the data instead it uses a separate computing engine.

Let's look at the difference between SQL and SPL query to understand their respective syntax:

Traditional SQL statement👇

Here's how SPL is written in an ordered manner to efficiently fetch data from any databases👇

Learn more about SPL and its various features from source code

Don't forget to ⭐the repo for future reference👍👇
https://github.com/SPLWare/esProc

Accelerating Database Querying: Introducing SPL

Data storage and warehousing concepts to know as a data scientist👇

Data storage and warehousing are important components of modern data engineering.

With the exponential growth of data, it’s critical to have an efficient and scalable storage solution in place that can handle large amounts of data.

In addition, data warehousing enables organizations to store, organize, and analyze data from various sources in a centralized location, providing a more complete view of the organization’s data.

Let’s look at the tools and technologies to learn for data warehousing:

• Familiarity with data storage solutions such as Hadoop, NoSQL, and columnar databases

• Knowledge of data warehousing concepts and tools such as Amazon Redshift, Snowflake, and Google BigQuery

• Experience with data modeling and schema design for data warehouses

• Familiarity with distributed systems

Data Storage and Warehousing for Data Scientists

Course Alert⏰

A Crash Course in Data Science by Johns Hopkins Uni.

Features at a glance:

Flexible deadlines
Shareable Certificate
100% online
Coursera Labs

Start learning now and sign up to excel in your career👇⚡
https://www.datakwery.com/coursera/a-crash-course-in-data-science/?utm_source=avi&utm_medium=blog

WHAT YOU WILL LEARN

• Describe Data Science’s role in various contexts

• Understand how Statistics and Machine Learning affect Data Science

• Use the key terms used by data scientist

• Predict whether a Data Science project will be successful

Offered by Johns Hopkins University

The mission of Johns Hopkins University is to educate its students and cultivate their capacity for life-long learning, foster independent and original research, and bring the benefits of discovery to the world.

BONUS: Learn to build LLM powered app using langchain, openai and pinecone👇
https://www.analyticsvidhya.com/blog/2023/08/qa-applications/

Crash Course in Data Science

🤖📚 Beginner's Guide to Machine Learning 📚🤖

Machine learning can seem intimidating to novices.

Check out this thread that breaks down the basics into simple, easy-to-understand explanations

1/ What is machine learning?

Machine learning is a field of study where computers learn from data without being explicitly programmed.

It's about creating algorithms that can make predictions or take actions based on patterns and relationships in the data they are trained on.

2/ Mathematics and Statistics Fundamentals

Machine learning relies on mathematical and statistical concepts. Some key fundamentals include linear algebra (vectors, matrices), calculus (derivatives), and probability (random variables, distributions).

Understanding these concepts will help you grasp the underlying principles of machine learning algorithms.

3/ Machine Learning Concepts:

a) Supervised Learning:
In supervised learning, we provide the algorithm with labeled data (input-output pairs) to learn from. It aims to create a model that can predict outputs for new, unseen inputs.

b) Unsupervised Learning:
Unsupervised learning deals with unlabeled data. The goal is to discover hidden patterns, structures, or relationships within the data. It can be clustering data into groups or reducing the dimensionality of the data.

c) Reinforcement Learning:
Reinforcement learning involves an agent learning through interaction with an environment. It receives rewards or penalties based on its actions and learns to maximize the rewards. It's used in scenarios like game playing or robotics.

4/Libraries for ML:

a) scikit-learn:
A popular machine learning library in Python. It provides a wide range of algorithms for classification, regression, clustering, and more. It also offers helpful utilities for preprocessing data and model evaluation.

b) TensorFlow:
An open-source library developed by Google, TensorFlow is widely used for building and training deep learning models. It provides a flexible architecture and supports deployment on various platforms.

c) Keras:
Keras is a user-friendly deep-learning library that runs on top of TensorFlow. It simplifies the process of building neural networks and makes experimentation easier. It's great for beginners to dive into deep learning.

d) PyTorch:
Another popular deep learning framework, PyTorch offers dynamic computation graphs and a beginner-friendly interface. It's extensively used in academia and research due to its flexibility and ease of use.

5/ Deep Learning Concepts:

a) Convolutional Neural Networks: CNNs are specialized for image and video processing tasks. They use convolutional layers to automatically learn hierarchical representations from images, enabling them to recognize objects, features, or patterns.

b) Recurrent Neural Networks (RNNs):
RNNs are designed for sequential data, such as text or time series. They have memory to process and retain information about previous inputs, making them suitable for tasks like language modeling, speech recognition, and machine translation.

c) Transformers:
Transformers are a type of deep learning model that revolutionized natural language processing (NLP). They capture contextual relationships between words and excel in tasks like language translation, sentiment analysis, and text generation.

d) Transfer Learning:
Transfer learning involves leveraging pre-trained models on large datasets and adapting them to new, smaller datasets. It saves computational resources and training time, making it valuable when you have limited data.

5/ MLOps Tools:

a) MLflow: MLflow is an open-source platform that helps manage the machine learning lifecycle.

It offers components for tracking experiments, packaging models, and deploying them to production. It promotes reproducibility and collaboration.

b) Docker:

Docker is a popular tool for containerization. It allows you to package your machine learning application along with its dependencies, ensuring consistency across different environments and simplifying deployment.

c) CI/CD (Continuous Integration/Continuous Deployment):

CI/CD is a practice of automating the software development process. It involves building,

Beginner's Guide to Machine Learning

Data is the foundation of any ML/AI or data science project🤖

Without good data, it is impossible to train a machine learning model or build a data science product.

Here's why good data is important and how to get them👇

1/ If you have done any data science project then you must have noticed that data is paramount in your modeling and deployment.

Data is the fuel that drives machine learning models. Without data, machine learning models cannot learn or make predictions

2/ Data can be used to improve the accuracy and performance of machine learning models.

The more data you have, the better your models will be.💯🤖

3/ Data can be used to discover new insights and patterns.

By analyzing data, you can learn more about your customers, your products, and your business.

It also helps in making decisions about products and your business

4/ But here's the catch! Data can be expensive and time-consuming to collect.

This is where BrightData comes in. BrightData is a leading provider of structured public datasets.

Bright Data curates and source data from a variety of sources, including government websites, social media platforms, and financial institutions.

It offers a wide variety of datasets at the lowest cost and it perfectly fits any data projects you are working on

If you are working on an ML/AI or data science project, then you need to make sure that you have access to the right data.

Bright Data can help you get the accurate datasets you need at the lowest costs

Check out the link below to learn more👇
https://get.brightdata.com/webdatasets

BONUS: Learn to build LLM powered applications using LangChain, OpenAI and pinecone with complete code👇
https://www.analyticsvidhya.com/blog/2023/08/qa-applications/

The Importance of Good Data in ML/AI and Data Science

As a student academic writing is invaluable to becoming successful in your academic journey📝



Writing an essay takes hours of searching through countless papers and research articles



Introducing @sourcelyai  - an AI-powered tool to find sources for your
essays!👇


⚡http://sourcely.net  allows you to find, summarize, and format sources for your academic papers with ease!!



Instead of going through hours through google scholar databases and searching across the web "Sourcely" is the way to make your life easier!

How to use it?



👉Visit: http://sourcely.net


👉Paste your essay notes up to 300 chars or UNLIMITED with a pro plan
👉Click on "Find Sources" and Sourcely will give you a list of sources relevant to your essay notes🤯

👉Summarize papers quickly⚡



Sourcely does not only help you in finding the best papers for your essay but also helps you with quick summaries of papers so that you can save a tone of time👇

👉Download pdfs



It also allows you to download pdfs of academic papers you need for your research.



All these in a single tool🤯👇

Trusted by academics teams from top universities with pro plans find essays for UNLIMITED chars and excel in your academic journey


Check out the pricing page for more: https://sourcely.net/pricing

If you are a student or academician then this is the must-have tool in your toolbox! 📝



Go sign up using the below code to get SourcelyPRO 10% off😃
Code --> AV10
https://www.sourcely.net/pricing

Sourcely: Your AI-Powered Essay Writing Assistant

Documenting processes and guidelines can be tedious with boring screenshots and screen recordings😪

But there's a better way to create guidelines for your users or your team

Introducing @ScribeHow - an AI-powered tool to generate step-by-step guides (without tears)

"ScribeHow" is phenomenal and crafts impeccable auto-generated step-by-step guides using screen captures

It is a super easy-to-use Chrome extension that helps you to create a guide for any process and saves tons of time.

Download the ScribeHow now👇
 🔗 http://getscribe.how/chrome


Look at some benefits of ScribeHow:

• Create guides for any process.
• Customise your guides (editable).
• Save your 10x time with this extension.
• It can also work with Notion and can be exported too

Effortlessly capture any process with a simple Chrome extension

ScribeHow creates a step-by-step process within seconds for your users or your employees

Not only capturing process, but ScribeHow also allows you to edit and customize process flow, remove redundant information and add branding.

Let's look at the example of GitHub scribe on "How to get GitHub Notifications in Slack"

It's just as simple as it looks and you can share it with anyone at a click of a button.

Link to the scribe: http://bit.ly/3KLFO21

Recently, @ScribeHow also introduced Scribe AI

It can help you document your entire process flow using AI while embedding suitable scribes to any sub-processes👇
http://getscribe.how/chrome

Introducing ScribeHow: AI-Powered Step-by-Step Guides

Question and answering applications are the most useful applications of generative AI

Let's look at this most promising use-case of generative AI👇

What is a Q&A application using LLMs?

A question and answering (Q&A) application using large language models involves utilizing advanced natural language processing techniques, particularly those offered by large language models like GPT-3.5.

learn more: https://bit.ly/47H9DL6

These applications aim to automatically generate accurate and relevant answers to user-provided questions in a human-like manner.

This approach is flexible and far better than fine-tuning custom LLM because this is much more cost-effective than letter one.

Q&A application workflow:

1.  Document loaders: To load user documents for vectorization and storage purposes
2.  Text splitters: These are the document transformers that transform documents into fixed chunk lengths to store them efficiently

3.  Vector storage: Vector database integrations to store vector embeddings of the input texts
4.  Document retrieval: To retrieve texts based on user queries to the database. They use similarity search techniques to retrieve the same.

5.  Model output: Final model output to the user query generated from the input prompt of query and retrieved texts.

Depiction of an entire workflow👇

Advantages of Custom Q&A Applications Over a Model Fine-tuning

1.  Context-specific answers
2.  Adaptable to new input documents
3.  No need to fine-tune the model, which saves the cost of model training
4.  More accurate and specific answers rather than general answers

If you found this interesting then, check out the full article I wrote on Analytics Vidhya - India's leading Data Science and AI platform👇
https://www.analyticsvidhya.com/blog/2023/08/qa-applications/


Q&A Applications and Generative AI

Boosting algorithm clearly explained...

Boosting algorithms has always been a topic of discussion in the ML community due to their performance and generalization power🤖

let's deep dive into this exciting algo👇

What is boosting technique?

Boosting is a machine learning technique that uses a weak learner to fit the data and sequentially optimizes the error metric of each fit to reduce the error.

Base learners can be decision trees or any other weak learner suitable to the problem

It focuses on improving the model's performance by giving more weight to the misclassified samples in each iteration.

The primary idea behind boosting is to iteratively train new models that correct the errors of the previous ones.

Types of boosting algorithms

There are two major boosting algorithms 1) Adaptive boosting and 2) Gradient descent based boosting algos

Most widely used boosting algorithms:
1) Ada Boost
2) Gradient boosting
4) XGBoost
5) LightGBM
6) CatBoost

Code example of Ada boosting algorithm

Let's look the code example of ada boosting algorithm

There are other versions like "Hist Gradient Boosting" which fall under the type do gradient boosting algorithms.

It is used in datasets where samples are greater than 10,000 and features are mostly numerical.

Boosting techniques are famous in classical ML  due to their performance and feature reach libraries along with custom solutions available

Hyperparameters are also key factors in building the best predictive models, you can learn this from documentation
https://scikit-learn.org/stable/modules/classes.html#module-sklearn.ensemble

Check out some of my notebooks on ml modeling using these techniques👇
https://www.kaggle.com/avikumart/code

BONUS: Check out my latest article on custom Q&A application development using OpenAI, LangChain and Pinecone published on Analytics Vidhya👇
https://www.analyticsvidhya.com/blog/2023/08/qa-applications/