I signed up for ChatGPT+ ($20/mo) to try GPT-4. Can confirm the improvement over v3.5. I asked it one of the questions I ask job candidates for a role on our team: “Explain second order gradient based post training quantization algorithms”. GPT-3.5 didn’t produce a satisfactory answer - a couple of things were plain wrong, and it didn’t explain how second order gradients are actually used. Yesterday I tried the same question with GPT-4 and produced an answer good enough to pass a job interview. I was impressed. The answer was clear, correct, and had the right amount of detail.
Even more impressively, today I needed to write some code in C (I use Python daily and my C is rusty) to add asymmetrical rounding mode when quantizing FP32 to block floating point format (I already had the symmetrical rounding code). It involves some tricky bit-level manipulations, so I expected to spend at least a couple hours on writing it and testing it. So I decided to see if GPT-4 can help. It took 4 iterations of the prompt, but after only ~10 minutes I had the correct implementation in front of me. To be fair I spent another ~20 minutes visually inspecting the code, checking for mistakes (this would be much faster in Python). Then I spent another ~5 minute tweaking the prompt to describe a test, and yes, it produced the correct code for that too! It saved me at least an hour of time and mental effort.
Just wanted to say, what I’m feeling right now is a mixture of fear and awe for what’s coming. It’s like experiencing the explosion of WWW in the 90s, only compressed into a much shorter time scale. Many important things are going to change dramatically in the next few years.
@thanh-binh.to, This is how I use GPT to make code:
I typed into the command line: “environment arduino uno, code to create an array of 100, fill it with random numbers, and quick sort it”
This is what I got back from GPT …
Sure, here’s an example code that creates an array of 100 integers, fills it with random numbers, and then sorts it using the quick sort algorithm. This code assumes that you have already set up your Arduino Uno environment and connected it to a computer.
scss
// Define the size of the array
#define ARRAY_SIZE 100
// Function to swap two integers
void swap(int* a, int* b) {
int temp = *a;
*a = *b;
*b = temp;
}
// Function to partition the array for quick sort
int partition(int arr[], int low, int high) {
int pivot = arr[high]; // Choose the last element as the pivot
int i = (low - 1); // Initialize the index of the smaller element
for (int j = low; j <= high - 1; j++) {
if (arr[j] <= pivot) {
i++;
swap(&arr[i], &arr[j]);
}
}
swap(&arr[i + 1], &arr[high]);
return (i + 1);
}
// Function to implement quick sort
void quickSort(int arr[], int low, int high) {
if (low < high) {
int pi = partition(arr, low, high);
quickSort(arr, low, pi - 1);
quickSort(arr, pi + 1, high);
}
}
void setup() {
// Initialize the random seed
randomSeed(analogRead(0));
// Declare the array
int arr[ARRAY_SIZE];
// Fill the array with random numbers
for (int i = 0; i < ARRAY_SIZE; i++) {
arr[i] = random(100);
}
// Sort the array using quick sort
quickSort(arr, 0, ARRAY_SIZE - 1);
// Print the sorted array
for (int i = 0; i < ARRAY_SIZE; i++) {
Serial.println(arr[i]);
}
}
void loop() {
// Nothing to do here
}
In the setup() function, the code initializes the random seed, creates an array of 100 integers, fills it with random numbers using the random() function, and then sorts it using the quickSort() function. Finally, the code prints the sorted array using the Serial.println() function.
Note that this code is only meant to be run once, in the setup() function. The loop() function is left empty in this example code.
On casual inspection, the program appears to be correct and nicely structured.
It follows the usual conventions for the Arduino code environment.
Note that it is using the Arduino specific random function, with a specified max value, GPT selected 100 as the limit. Since I did not specify any particular range, GPT defaulted it to the only number mentioned in my initial specification. In an iterative program development such as @michaelklachko described, I would examine the code, note this detail, and add a random range specification to the list of parameters that I used. As I add constraints and hints to the specification, GPT makes these changes to the generated code.
I did not ask it to print out the results, GPT did that on its own.
Here is the new specification: “environment arduino uno, code to create an array of 100, fill it with random numbers ranging between 5 and 5000, and quick sort it”
Sure! Here’s an example code that should achieve what you’re looking for:
void swap(int arr[], int i, int j) {
int temp = arr[i];
arr[i] = arr[j];
arr[j] = temp;
}
int partition(int arr[], int low, int high) {
int pivot = arr[high];
int i = low - 1;
for (int j = low; j <= high - 1; j++) {
if (arr[j] < pivot) {
i++;
swap(arr, i, j);
}
}
swap(arr, i + 1, high);
return i + 1;
}
void quickSort(int arr[], int low, int high) {
if (low < high) {
int pi = partition(arr, low, high);
quickSort(arr, low, pi - 1);
quickSort(arr, pi + 1, high);
}
}
void setup() {
Serial.begin(9600);
randomSeed(analogRead(0));
int arr[100];
for (int i = 0; i < 100; i++) {
arr[i] = random(5, 5001);
}
quickSort(arr, 0, 99);
for (int i = 0; i < 100; i++) {
Serial.println(arr[i]);
}
}
void loop() {
// Nothing to do here
}
Just think of it as a human and describe the task exactly how you’d give to another capable human - explaining important attributes that need to be mentioned as well
well, GPT4 was supposed to be an incremental change anyhow We already have very similar systems just not as scaled as up OAI’s. GPT5 is the one supposed to be a true generational leap, leveraging many times the compute and hopefully true multimodality. But overall, GPT4’s performance while impressive isn’t really that surprising or unpredictable.
Just curious, where did you hear that? I can’t find any info about GPT-5.
I don’t know what it was supposed to be (I didn’t see any information about it until a couple of weeks ago). I agree that the change is incremental, however I feel like there’s a phase transition happening - it might be as much better than GPT-3 as how GPT-3 was better than GPT-2, but this time it might be good enough for many tasks where GPT-3 wasn’t. By “good enough” I mean human level performance. If GPT-5 provides a similar degree of incremental improvement, it might actually become better than humans on many tasks.
We already have very similar systems just not as scaled as up OAI’s. GPT5 is the one supposed to be a true generational leap, leveraging many times the compute and hopefully true multimodality. But overall, GPT4’s performance while impressive isn’t really that surprising or unpredictable.