Generating reproducible outputs in OpenAI models.

A common challenge encountered when using GPT models in applications is the lack of determinism in certain situations.

If you seek deterministic outputs from GPT models, here's a useful trick to consider:

↓

The OpenAI completions API features a parameter called `seed` which enables the generation of consistent outputs for the same inputs.

When you set the `seed` to an integer, it should produce identical outputs for any request with the same seed value.

Seed works in the same manner as anywhere else in machine learning.

But in order for seed to generate same output among different requests, the inputs messages along with all other parameters must also be same.

Unlike many machine learning systems, the seed parameter here doesn't ensure identical output every time; it may only increase your odds.

Another consideration is the variation in the backend environment of OpenAI, which is beyond our control.

Whenever a request is sent to the completions API, we receive a response parameter known as `system_fingerprint`, which is an alphanumeric string like `fp_50f3bce4db`. 

This string represents the current backend configuration in the OpenAI environment.

If two consecutive requests return different fingerprint values, there is a possibility that the response may differ even if the input, seed, and all other parameters remain constant.

Although this doesn't mean that a different fingerprint would definitely give different output.

Here is a sample run without the seed parameter.

Here is a sample run with the seed parameter set to an integer.

If you aim for reproducible outputs, consider the seed, system_fingerprint, and input parameters.

Saving these values for later use can be beneficial.

Achieving Reproducible Outputs with OpenAI Models

Exciting news! 🚨
The ChatGPT API is now available!

• ChatGPT has been out for a while, accessible through web playground, and a plus subscription for priority access.

• Now, developers can leverage the API to build powerful applications.

Here's how to use it in Python ↓

• You can set your OpenAI API key using a  `.env` file.

• Then use the following code snippet to give it queries and get responses.

• Here's how the output would look like:

• If you have been a user of GPT-3 APIs, you might notice something different in the snippet.

• First, it is not under Completions API but rather under a new ChatCompletions API.

• Also there is a messages parameter which is what we have to use for handling conversations.

• There are three roles for messages.
- system
- user
- assistant

• The system role is used to define the base instructions to set the behaviour or personality of the bot.

• The user role is used to provide the prompt content from the user.

• The assistant role is used to provide the content for the assistant.

• Initially we can give a few sample content messages to set the initial conversation. As we have done in the snippet to make it a code assistant bot.

• As the conversation progresses, we can add the queries and responses to these messages to keep track of past conversation.

• The models of ChatGPT are not available under the renowned names but rather under their technical GPT-3.5 name.

• There are two models present:
-  `gpt-3.5-turbo`
- `gpt-3.5-turbo-0301`

• The pricing of this API is $0.002 per 1K tokens.

• If you know `davinci` which is the most advanced GPT-3 model, this API costs 10 times less than davinci does.

• It's price is equal to GPT-3 `curie` Model.

• Now that we have the ChatGPT as an API, the organizations that develop Chatbot applications would be the first to benefit.

• As ChatGPT would provide the human aspect to the conversations and make them more fluent!

Stay tuned for more information and tips!

Thanks!

ChatGPT API

How ChatGPT Works?

• Everyone knows about ChatGPT but not everyone is aware of how it works.

Here is an attempt at explanation ↓

[ In 10 Steps ]

• It is a large language model based which uses a technique called "transformer" to understand and generate human-like responses to text-based input.

• Transformer is a neural network architecture that excels at processing sequential data, such as text.

• The model is trained on a massive amount of text data from a wide range of sources, such as books, articles, and websites.

• These resources allow the model to learn patterns and relationships in language.

• The training data is preprocessed to create a corpus of text.

• This corpus is then used to train the model to predict the next word or sequence of words given the preceding text.

• During training, the model is presented with a sequence of words and their corresponding responses (next word).

• It's task is to find the next word for the given sequence.

• The model's predictions are compared to the actual next word in the sequence

• The model's parameters (weights and biases) are adjusted to minimize the difference between the predicted and actual values.

• This process is repeated many times, with different contexts and sequences.

• We repeat the process until the model's parameters are optimized to accurately predict the next word in a wide range of contexts.

• Once trained, the model can be used to generate new text by providing it with a prompt or starting sentence, and letting it generate the rest of the text based on its learned patterns.

• ChatGPT is fine-tuned on specific tasks or domains, such as question-answering or customer service, to improve its performance in those areas.

• When generating text, ChatGPT uses a beam search algorithm to generate a sequence of words.

• These are the words which are most likely to form a coherent and grammatically correct sentence given the starting prompt or context.

• To generate a response, the model takes in the input then generates a probability distribution over all possible responses.

• Then the response with the highest probability is selected as the model output.


This is a gist of the entire process.

Overall, ChatGPT is a complex and sophisticated language model and a lot more that goes into training.

Hope this helps!

How ChatGPT Works?

GPT-3 and ChatGPT

Although they work in quite a similar manner, they are different from each other in some aspects.

↓

• GPT-3 is a large language model which is trained on text data from diverse resources.

• GPT-3 can be fine-tuned for a wide variety of NLP tasks, like

Sentiment Analysis
Text Classification
Text Summarizing
Translation, etc.

• While ChatGPT is a specialized version of GPT-3 designed and trained on data for conversational human like responses.

• It has been customized to answer more specific scenarios common to conversational usecases.

• The biggest usecase is in chatbots.

• GPT-3 is much larger with 175 Billion paramerters.

• While ChatGPT is only 20 Billion parameters.

• So GPT-3 is capable to providing responses with more knowledge but in a conventional way.

• ChatGPT on the other hand has comparatively less knowledge but gives the answers in an engaging way and keeps track of the dialogues.

• I personally like GPT-3 very much merely because of the vast amount of enterprise use-cases that it can cater.

• However, the mainstream popularity of ChatGPT can't be ignored which largely due to the explanations and its human-like responses.

Neural Networks as Linear Combinations.

• NNs are multiple linear methods working in combination, but the power of non-linearity is what makes the difference.

• Let's see how many lines it'd take you to separate these two groups?

↓

• At first we can try to see how a single line does.

• This is something a logistic regression with linear function would do.

• As expected there isn't much that can be done with it.

• Without worrying about the activation function let's try using a NN.

• With one neuron in the hidden layer, it can be thought of as common logistic regression.

• Now we can increase the number of lines to solve the problem.

• Trying it with two lines now is expected to give a better solution.

• Much better than a single line but we're still misclassifying some of the orange observations as well.

• Let's check with NN having two neurons.

• As expected, not a perfect solution. What about three lines? Would that work?

• It could. Let's try. Seem possible.

• It is quite evident that we can do it with 3 neurons which is analogous to using 3 lines.

• So a neural network is a combination of linear methods BUT non-linearity is playing a key role here.

• All the outputs of the neurons are mapped to a continuous non-linear function that allows the complex learning and backpropagation to work.

• What would happen if we just apply the linear activation and not apply any activation at all?

• A combination of linear systems yields only another linear system,

• Without the non-linear activations the model would only try to fit a new single line to it, which we know won't work.

• We are trying to estimate an unknown function that can't always be approximated by simple linear functions,

• That is why we add non-linear transformations to the inputs.

If you're intrigued, do mess around with the architecture using the playground.

This helps in building good intuition.
http://playground.tensorflow.com

Who are the people in MLOps?

Data Science and MLOps are constantly evolving and have multiple roles and people.

Not only Data Scientists but many other people are also there in the ML Model lifecycle.

But who are they and what do they do?

↓

1. Subject Matter Experts

Data scientists may not have a deep understanding of the businesses and domains they work for.

SMEs help them frame the problem better by working alongside data scientists.

They have defined goals, business questions and KPIs.

2. Data Scientists

They do the analysis,  experimentations and build the ML models.

They are not always included in deploying or testing the model but the roles can overlap.

They handle the constant assessment of the model quality after deployment.

3. Software Engineers

It is important to have good software engineers working with data scientists so that larger systems can be built efficiently.

It is also important that the ML code and models deployment fit into the CI/CD pipeline of the existing software as well.

4. DevOps

DevOps have two primary roles in the ML models life cycle:

First, they conduct and build operational systems and create tests to ensure security, performance and availability of ML models.

Second, creating CI/CD pipelines is also generally their responsibility.

5. Model Risk Manager

MRM is important for regulatory compliance for larger industries.

They analyze the model outcomes and business goals that ML is supposed to solve.

They have to ensure that model poses no risk to the business in any way.

6. Machine Learning Architect

They are responsible for overall architecture and ensuring requirements of data needs.

ML architects ensure a flexible environment for pipelines.

They provide expertise in new technologies that may improve model performance in production.

Thanks for reading!

Checkout my newsletter!
underthecurve.substack.com

People of MLOps

Python Tip 🐍

If you want to check for membership in a collection, prefer Sets or Dicts over Lists or Tuples.

MLOps for Risk Mitigation

• In machine learning systems there are risks associated with an organization.

• Proper risk assessment and mitigation are important to be done.

• MLOps is very important for risk mitigation for the ML models.

• The cost-benefit evaluations should be performed whenever an ML system is to be deployed.

Risk Assessment

• The stakes for different projects could be different in machine learning.

• Models that are deployed for wide usage and outside of the organization tend to pose larger risks.

• For risk mitigation using MLOps following things should be analyzed properly:

1. Unavailability of the model for a given time.

2. Bad predictions from the model

3. Decrement in model accuracy or fairness over time.

4. Loss of skills needed for maintaining the model.

• Risk should be quantized by analyzing the probability and the impact of each event.

• Here's a matrix that might help:

Risk Mitigation

• ML models should not be deployed without proper MLOps infrastructure in place.

• Only in a production environment, the full assessment of the model can be done as any change in production can rapidly decrease the model's performance.

• The versions of the frameworks used in building and deployment of the model are also very important so the deployment versions should match up with the versions the model was verified on.

• Pushing the models into production is not the end of the cycle but rather a beginning.

• As monitoring the models and using the feedback to improve or reduce problems is also a critical aspect of the production cycle.

Next time we will take a look at the people associated with the MLOps.

I keep sharing such content so keep updated with my Twitter or subscribe to the weekly newsletter ↓

Thanks for reading!

Consider following @capeandcode.

https://www.getrevue.co/profile/rectifiedlearning/

MLOps and Risk Mitigation

Need for Machine Learning Pipelines

• Machine learning pipelines are crucial for sustainable machine learning systems.

• But what good do they bring? What are the benefits that we get from them?

Let's discuss ↓

0/7

Although building automated machine learning pipelines can lead to several little benefits, here are five of the major ones:

1. Easier Maintenance
2. Bug Prevention
3. Proper Trails
4. No Glue-Code
5. Time-Saving

1/7

1. Easier Maintenance

• Building pipelines require one time creation and then can be used for subsequent deployments.

• This allows for easy and consistent upgradation of models. Lets data scientists focus more on the experiments and improving the models.

2/7

2. Bug Prevention

• Machine Learning pipelines do everything with proper versioning and monitoring.

• The pipeline steps remain same until explicitly changed making sure no accidental updation takes place leading to untraceable bugs.

3/7

3. Proper Trails

• In many scenarios we may want to reproduce previous results which require accurate records.

• In pipelines all the data, hyper parameters, models can be traced to the details of the process they were created from which are hard to maintain explicitly.

4/7

4. No Glue-Code

• Glue code refers to the manual code required to connect steps in ML process flow.

• With standard pipelines, this code is reduced to standard and reusable code which connects the pipeline components.

5/7

5. Time-Saving

• Machine learning pipelines save a lot of time by making simpler process to update and reproduce models.

• They are essential in the projects that involve a customer base or is part of an application.

6/7

Machine Learning Pipelines are of paramount importance in the real world production applications.

Stay tuned for more on production pipelines.

7/7

I keep sharing such content so keep updated with my Twitter or subscribe to the weekly newsletter ↓

Thanks for reading!

Consider following @capeandcode.

https://www.getrevue.co/profile/rectifiedlearning/


"Learn SQL"

Great advice no doubt.

• But what topics to cover?
• Which SQL database to use?
• What resources to learn from?

Here's is a track you can follow ↓

1/22

Let's start with choosing the SQL database to learn.

• There are several of databases like Postgres, MS SQL server, MS Access, Oracle.

• But for learning purposes I'd suggest going with MySQL.

• For reasons that it is secure, free & open source and the support is great.

• Later on we can switch to other databases easily.

• Now what topic should we know in SQL? I knew the basics like select, where and joins but I couldn't get past them.

• So here are some topics and terms everyone should learn, starting with...

1. Theory

• DDL
• DML
• DCL
• TCL
• Databases
• Schemas
• Relationships

2. Basics

• Creating Databases
• Creating Tables
• Data Types
• Data Insertion
• Data Updation
• Data Deletion

3. Keys & Constraints

• Primary Key
• Candidate Key
• Super Key
• Foreign Key
• Default
• Unique
• Not null

4. Basics of Select Statements

• Select
• Aliases
• Where
• And
• Or
• Like-Not Like
• Between-And
• Is null-Is Not Null
• Distinct
• Order By
• Limit
• Exists-Not Exists
• Subqueries
• If
• Case

5. Aggregations

• Group By
• Having
• Count
• Sum
• Min & Max
• Avg
• Round
• If null
• Coalesce

6. Joins

• Inner Join
• Left Join
• Right Join
• Cross Join
• Self Join
• Union
• Union All

7. Window Functions

• Row number
• Partition
• Rank & Dense Rank
• Lag & Lead

8. Other Topics

• Stored Procedures
• User-defined Function
• Triggers

• From my perspective all of these topics are a must to be able to do substantial work with relational databases.

• Now that we know the topics, where to learn them from?

• If you're just a beginner, then you can start with crash courses to get a gist.

Like these two ↓

1. SQL from Freecodecamp

https://www.youtube.com/watch?v=HXV3zeQKqGY

2. Coding with Mosh

https://www.youtube.com/watch?v=7S_tz1z_5bA&t=3923s

• For advanced topics you might have to either get paid courses.

• Or you can learn from websites like W3 schools which have good explanation for each topic.
https://www.w3schools.com/sql/default.asp

• There's another great alternative, which is... Books! There are some great books you can pick up for learning.

Like these three ↓

1. Learning SQL by O'Reilly.

https://www.amazon.in/Learning-SQL-Generate-Manipulate-Greyscale/dp/935213978X/ref=sr_1_2?crid=1TT5A0X7WCO7J&keywords=learning+SQL&qid=1663820404&sprefix=learning+sq%2Caps%2C332&sr=8-2

2. SQL for Data Scientists by @BecomingDataSci

https://www.amazon.in/SQL-Data-Scientists-Beginner%E2%80%B2s-Building/dp/1119669367/ref=sr_1_2?keywords=sql+for+data+scientists&qid=1663819430&sprefix=sql+for+data+sc%2Caps%2C207&sr=8-2

3. Sam's Teach Yourself

https://www.amazon.in/Sams-Teach-Yourself-SQL-Minutes/dp/0672325675/ref=sr_1_3?crid=24XD2UVIO622G&keywords=teach+yourself+sql&qid=1663819341&sprefix=teach+yourself+sql%2Caps%2C226&sr=8-3

Once you have everything covered then you might want to get your hands dirty.

• One of the best ways is to download any open source database and try different concepts by yourself.

• Another good alternative is to visit Hackerrank and do their SQL track.

That's all for this one!

• SQL is a must-have skill these days and I hope everything above adds value to your learning.

• If you have other opinions or advice, do comment.

I keep sharing such content so keep updated with my Twitter or subscribe to the weekly newsletter ↓

Thanks for reading!

Consider following @capeandcode.

https://www.getrevue.co/profile/rectifiedlearning/