Lost in Data #1: Coding with ChatGPT
Coding with an AI - Nice reads for the week - Open Data for an open world
Everybody is talking about ChatGPT for basic talks and QA. But as a developer, I wanted to see if it can code and how good it is at coding. I have been using AI assistants for a couple of years to speedup my coding, but those are limited to complete the current line (like Tab Nine), or to provide a generic code based on comments (like Github Copilot).
OpenAI (the company behind ChatGPT) created ChatGPT by collecting many QA from humans on the internet, then annotated, rated, and ordered them, so they can train a Machine Learning model to produce similar best responses to similar questions. This, of course, includes StackOverflow.
But it doesn’t end there: ChatGPT evolves and learns from its interactions with humans. That’s where the Reinforcement Learning tactics come in. Based on feedbacks from humans it interacts with, ChatGPT uses a rewards system to learn from its mistakes and successful responses. What we do when interacting with ChatGPT is to provide it with more data to improve its model.
That’s important, because it means that the current limitations of ChatGPT are “temporary” limitations.
Now let’s experiment and see what ChatGPT is capable to do with coding. I asked it to write some Python code to find the N-first prime numbers. It’s the kind of questions you typically ask a junior developer in a job interview:
Good! The model can write code, and it explained what it is doing, just like a junior developer.
Let’s level up. I then ask my developer candidate to improve the speed of his code (if possible):
Not bad at all! Things are getting very interesting now. First, the chatbot understands the notion of context: I didn’t have to explicitly talk about the problem we are currently pair-solving.
Second, it provided comments in its code! (much appreciated)
Finally, it added explanations before and after his code.
However, this time it made a mistake in the code:
Waw! Not only it acknowledged his mistake, said why his previous code was wrong, rewrote it with the correction, and explained what the addition does.
You’re hired. What is your expected salary?
…
Jokes aside, I don’t feel I’m in front of an AI, which is essentially just a mathematical model containing billions of parameters, and served by software.
Wait! Did it just pass the Turing test?
At least, this is the kind of interaction I have with peers.
Now, I don’t say that ChatGPT will replace developers. After all, my previous questions are trivial and not part of a much bigger actual project. But I can see it as my copilot developer if I provide more context. Instead of worrying about AI taking over developers’ jobs, we should see those smart helpers completely redefining how we solve problems.
I like the suggestions of ChatGPT compared to my two other AI-colleagues (aka TabNine and Copilot) because it provides context and explanations along with the code. So it’s not only about code generation but also about problem solving.
Even though you should never totally implement the solution suggested by ChatGPT, it is by far the best AI-coding assistant I interacted with. I’m very excited to see how it can be integrated with existing IDEs, and how this will change the way we code on a daily basis.
We’ll wait and see.
Until then, take care.
-- Hassen
Good Reads
OpenAI introduces ChatGPT. The blog explaining what is ChatGPT, and how it was created → https://openai.com/blog/chatgpt/
"Practically" building an image classifier, by Sairam Sundaresan
How open data helps Ukraine during the war → https://data.europa.eu/en/publications/country-insights/ukraine/how-open-data-helps-ukraine-during-war
Open Data
FIFA World Cup 2022 Tweets → https://www.kaggle.com/datasets/tirendazacademy/fifa-world-cup-2022-tweets
Satellite images updated hourly → https://scihub.copernicus.eu/
Biodiversity species coverage indicators → https://datacatalog.worldbank.org/search/dataset/0063384
That’s all for this newsletter folks. If you want to comment, please feel free to hit the reply button and give me your thoughts. I’ll reply as fast as possible.