Taking advantage of AI to help you write code

Hi, my name's Lachlan.

AI has exploded in popularity.

It's likely to play an increasingly important role in the way we design and develop software in the future.

AI is important.

Software development is fundamentally about transforming our ideas into instructions that a computer can follow, whether that's by writing assembly language, writing in a higher level programming language, or engineering prompts to have AI generate code for us.

Doing this effectively is a skill we develop and refine over time.

AI is here to turbocharge our capabilities, helping us to work smarter and faster and enhancing our creativity to solve complex problems.

As software developers, we need to embrace AI to understand it and to harness it to its full potential.

Let's dive right in and start by exploring the landscape of AI tools that are transforming the coding world.

First off, we have the conversational chatbots, including ChatGPT bing chat and Google Bard.

These can effortly generate snippets of code based on your prompt that you can copy and paste in your, into your editor.

They're good for iteratively refining the code based on your specifications.

Next up, we have extensions that integrate with your IDs like VScode.

GitHub copilot, Amazon Code Whisperer, tab nine and ask Cody.

These integrate seamlessly with your IDEs and provide enhanced autocomplete functionality.

They utilize context from your existing code and turn your comments into effective prompts.

It's like pair programming with someone who actually knows how to code.

How many of you are old enough to remember Clippy, Microsoft Word's AI assistant from 97?

Yeah, these ones are a bit more advanced than Clippy was.

Let's move on and take a look at how each of these tools generate code, in a somewhat real world scenario.

We'll be asking each to generate code based on a given prompt.

The goal here is to compare and contrast the outputs and see how they perform in a practical coding task.

So when thinking of a potential challenge I could give it, common ones like the Fibonacci sequence or the FizzBuzz program come to mind.

But as you may know, these are very well known.

And so I'm very likely to get pre canned answers out of the AI.

So I wanted to step it up just a little bit, while still keeping it reasonable for presentation.

So I came up with the fizzbonacci sequence.

So it's basically the fizzbonacci sequence combined with the rules for fizzbuzz.

So here's the prompt I'm going to feed to the AIs, more or less.

Create a generator function called fizzbonacci in TypeScript that accepts two parameters as the first two numbers in the sequence.

The output should follow the rules for fizzbuzz.

Then create an array of length 20 populated with the values from this generator.

First up, we have ChatGPT.

This is ChatGPT 4, by the way.

It looks pretty good at first glance.

It's got a loop, it's correctly calculating the Fibonacci sequence, it's applying the rules for fizzbuzz, and then, it's doing a loop to populate the array with numbers.

But when we look at the output It forgot to yield the first two values.

Somewhat forgivable mistake, but, you'll see this happens frequently.

Next up, we have Bing Chat.

Under the hood, this was based on ChatGPT4, so I expected similar results, and it more or less is.

It did also remember to yield the first two values, but it made a mistake in its first iteration, and just yielded fizzbuzz constantly.

Minor patch to that, and it gave me the correct output.

Next we have Google BARD.

This one's Google's own, completely different, not using ChatGPT.

That's, yeah, more than that.

So it looks similar to the original ChatGPT one.

It's chosen to hard code the the 20, limit into the generator function instead of doing that when populating the array.

Yeah, but when we look at the result, it's also forgotten to yield the first two values just the ChatGPT did originally.

Moving on to the IDE extensions, we'll start with GitHub Copilot I had, I was forced to modify the prompt a little bit just to explain the rules of fizzbuzz.

And I used the same prompt for the ones coming up as well in the IDE.

Sorry, so the actual generated function, calculates the Fibonacci sequence and it applies the fizzbuzz rules.

But, and it chose to start the rule, sorry, it chose to start the sequence from 1, 1.

And, when we look at the output, it also forgot to yield the first two values.

Common mistake, again.

Amazon CodeWhisperer.

This one just gave me the Fibonacci sequence.

Didn't even bother with its buzz rules even though I explained it.

A bit disappointing.

On the other hand, Tab9 literally just gave me the FizzBuzz sequence without bothering the Fibonacci.

And finally, AskCody.

This one had the worst UI out of all of them, but surprisingly, it gave me the most correct result.

So there was one bug that I had to fix, which was, populating the array.

Oh yeah, I just forgot to populate the array for me.

So anyway, I fixed that bug, and yeah, it worked.

Let's have a look at some of the benefits of conversational AI tools.

First up, these are great for ideation.

To help you to explore new ideas.

And they can often provide different perspectives, and possible implementations.

Not only does this save you time, it can also lead to innovative approaches you might not have considered.

They're great for generating boilerplate.

Typing out big chunks of code, or, copying and pasting it, is a lot easier, when just, you can generate it for you.

A lot of time saving there.

It excels at implementing well known or clearly described functions quickly.

So given a clear description or specification, it can generate that for you in your chosen programming language, again, saving you a lot of time and reducing errors.

Mostly.

Some AI tools can suggest improvements to your code.

If you pass your code to it, it can identify potential bugs and it can also suggest opportunities for optimization, and adherence to best practices.

So that can improve the quality of your code and potentially teach you a thing or two along the way.

In contrast, there are some benefits to AI extensions integrated into your IDEs.

These tools can supercharge your coding efficiency with predictive text capabilities.

They anticipate what you're likely to type next, offering more intelligent suggestions that align with your coding needs.

Yes, I will have a glass of water.

They can generate robust test suites based on your code and simple statements about expected functionality.

It's can help you catch bugs early, save time, and ensure your code is functioning as expected.

They have access to your entire codebase within Reason, giving them greater understanding of your project's context.

This leads to more relevant and precise suggestions, helping you to maintain consistency and adherence to your projects conventions and patterns.

However, there are some limitations and risks that we should consider.

As we've just seen, they're not perfect.

My previous demonstration, they, there's no guarantee that the code they generate, is going to be bug free, or that it'll even run.

So we're still ultimately responsible for the code we commit into our repos.

As developers, we need to, just like when we're copying and pasting from Stack Overflow, as we all do, we need to make sure that we review and understand what it's given us, and take responsibility for that.

We've got to verify that it does what we want it to do.

Some AI tools have limited context.

They're not going to know what you just discussed in your last meeting.

They're not going to be aware of the unique needs of your users.

And it may not be familiar with your company's particular coding standards or architecture.

You can help it by providing additional context in your prompts, but it's not a replacement for a deep understanding of your unique constraints and, and architecture and requirements.

It's a tool that can provide suggestions, but it's ultimately your knowledge and expertise that shapes those suggestions into a final product.

Using publicly available AI tools also carries the risk of revealing sensitive information and potentially violating your NDA.

Major companies like Apple, Amazon and Samsung have enacted policies prohibiting their employees from using AI tools in certain contexts to prevent leakage of their confidential information.

Simultaneously, other companies are exploring ways to tackle these, either by hosting their own AI tools internally, or by establishing contracts with AI companies like OpenAI, that stipulate how their data must be handled and disposed of.

Currently there's a class action lawsuit going on against GitHub, alleging that the co pilot, the GitHub co pilot, infringes upon copyright, because under certain circumstances it can generate code that very closely resembles existing code.

So while this and other related concerns are ultimately up to the lawyers to decide, and I am absolutely not a lawyer, beyond reporting this, I'm not going to comment on it.

It's critical to be aware.

That this is a potential issue that could come up, and just yeah, keep an eye on the situation.

Now we can go into the realm of prompt engineering.

This is the secret source for getting the most out of your AI.

This section is all about learning to communicate effectively with your AI tools, shaping the responses to our needs.

Let's start with the simplest form of communication, zero prompt, zero shot prompting.

This is basically, you just give it a simple question and you expect it to give it, a simple answer.

In this case, write a TypeScript function that takes two numbers and returns a sum.

That's a fairly simple request and we expect the AI's model to know how to handle this and give us an answer that works.

The next, type of engineering is few shot prompting.

This is where we provide within our prompt, some examples of how we want the response to appear, and then ask it to provide some more examples.

So in this case, we provided an example test case, and then just say, write more test case for different types of numbers.

And then it's followed that pattern and it's given me numbers for the response for zero, for floating points for whatever else has got there.

And with a bit more prompting, we could get it to handle edge cases, like not a number or infinity pretty easily.

Next up, chain of thought prompting.

This uses the power of reasoning to guide AI towards more accurate responses.

In its simplest form, zero shot COT, you just basically say, let's think this through step by step.

You're basically asking the AI to think about one step at a time, and generate a response according to that.

This can actually lead to more accurate responses than just jumping right to the answer at the end.

And with few shot COT, you provide examples demonstrating that reasoning process, and then it will also provide you an answer following that same reasoning process for the actual problem you want to solve.

So take an example, write a TypeScript function to calculate an array of integers, the average of integers, and then let's think this step by step.

That's enough to cause it to go step by step.

First it, just creates the function, then it adds the list of numbers together, then it takes the average, and whatever the final thing it does.

So basically, yeah, it's broken down into step by step to show what's going on.

And with more complex problems, not just calculating the average, this can be particularly useful.

Let's now take a look at building an actual application using ChatGPT.

So I wanted to figure out what to do.

So I went straight to ChatGPT and asked, and it gave me a list, things like the to do list, weather app, color palette, currency converter, good suggestions.

I wanted something that I could actually learn from.

So I decided to pick a calculator.

I wondered, how does a calculator work under the hood?

What are the algorithms that go on to, convert the input, into a format that can be evaluated, and then how does it evaluate the correct result?

So I asked from a high level, how does it work?

And it told me I need to handle the user input, the parsing, and the evaluation, and the result and in some cases storage, but I'm going to skip that bit.

So then I said, all right, I'm going to do a calculator app.

And I've told it I've got a basic React and TypeScript application.

And I said, just, okay, let's get started.

And it sat me out a calculator component.

And I put that in.

Then I asked it, I'm ready for the next step.

And it generated a number button for me.

It's a simple button with an on click handler, so I put that in.

Now I told it to include the number buttons in the calculator component with the digits zero to nine.

And it's generated an array for the numbers, and you'll notice it's put the numbers in the correct order for a calculator with 789 at the top, just in case you weren't aware, calculator buttons go opposite from phone buttons, and it, it knew that.

Then I asked it to create some operator buttons based on the, the number buttons I had, and to give me the basic operators, and it just generated, multiply, subtract, add and divide, whatever they are.

So I put that in.

Oh yes, I noticed a bug on a real calculator when display zero doesn't just concatenate the next number to the display.

It replaces the zero.

I wanted it to do that.

I told it, I explained what the bug is and it updated it to check if the current display value is zero.

And now I told it I needed an equals button to actually do it.

And it generated me one.

And, it was just using the JavaScript eval function.

So at this point, I had a somewhat functioning calculator app.

I had numbers, I had the operators, an equals button.

It would give me a result.

But I wanted to go a bit further.

I wanted to learn what's going on inside that eval function.

Can I actually write those algorithms myself?

Oh, before I get to that, I wanted a style sheet to make it look pretty.

I just told it to write one.

It knew what class names it had generated for my buttons and display output and stuff before.

So it generated me a somewhat reasonable style sheet.

This is what it looks like.

Your designer's job is safe, by the way.

ChatGPT is, not going to replace that just yet.

But yeah, then I was wondering, how does a calculator work?

Let's ask ChatGPT.

It told me about the Shunting Yard Algorithm and Reverse Polish Notation.

Have any of you heard of those two things before?

Nope?

Good.

We're going to explain it.

Reverse polish notation is a way of representing equations, basically in a way that can be processed with stack based processing.

Infix notation, this is what you're all familiar with, you put the operator between the numbers you want to operate on.

So in this case, 2 times 3, we get six, four times five, we get 20.

And then we add six and 20 to get 26.

With RPN, we instead do it this way.

We'd have the two numbers and then the operator we want to operate on them.

So we go two times three is six, four times five is 20 and then add six and 20 to get 26.

So how do you convert from that infix notation we're all familiar with to this reverse polish notation?

That's where the shunting yard algorithm comes in.

It's a method for pausing the input.

And this is the same algorithm that would be used by the, JavaScript parser.

And you type in your equation using infix notation.

It goes through it and it basically takes the algorithm, takes the numbers, and operators and converts them.

So imagine these lines are train trucks and the numbers and operators.

There are trains being shunted around the shunting yard.

So a number gets put straight into the output stack.

When it encounters an operator, it first goes into the operator stack.

Another number goes straight.

Then another operator, it checks the precedence of the operator in the operator stack.

If it's a higher precedence, it pops the operator stack and then puts the new operator in.

A number again.

And then when it's a lower precedence operator in the stack, it just pushes the new operator in on top of it.

And then when the operator, when the numbers are finished, it pops the operator stack in order and we end up with our final result.

So I asked ChatGPT for an implementation of the shunting yard algorithm and it gave me one.

At first glance it looked right, it was handling numbers, it was handling operators and even parentheses.

So I took this and I also asked it, now, I want a function to evaluate the result of RPN.

So I have my stack.

I want another function that can evaluate that.

And then let's just test the output.

So this was the input and then it gave me this as the output.

Doesn't quite look right, but let's see how we go.

It does four times five, then, adds three and 20, and then multiplies that by two, 46.

Okay, not quite what we wanted.

The bug, was that it wasn't checking the precedence of the operators in the queue first.

So I just explained that bug to chatGPT.

And that was enough for it to give me a revised version that fixed the bug.

I also needed a precedence function, that basically just returns a number indicating the precedence of the operator.

So I put that in, together, and then we see that it actually gives me the correct, result and that then evaluates correctly, as we expected it to earlier.

So let's take a look at the actual demo.

Okay.

So this is the calculator app.

There's my mouse.

Okay.

So we'll just try the original equation.

Two times three , plus four, times five.

I didn't input a keyboard thing, so it'd be harder, but anyway, and then equals, so 26.

I also got it to do some extra things with the help of ChatGPT.

When I input a number and then a parentheses, it puts an implicit multiplication in the end.

Oops.

Hang on.

Whatever.

Let's do clear entry there.

I wanted two.

And plus three.

8 times 5 should give me 40, that's correct.

Yeah, so this was a fun little thing to help to do.

I also handled, clear entry as well.

Sorry, just, Oh, and also if I do, I can change the operator, like a normal calculator.

There are some bugs in here, but, we'll ignore those for now.

But yeah, it works.

I only found the first bug in it last night when I was, rehearsing.

That's, whatever.

Yeah.

Okay, let's take a look, now that we've witnessed the capabilities of AI.

Let's look at some research into the effectiveness of AI tools, for software development.

Last year, on the GitHub blog post, from a Microsoft research team, they wanted to quantify, the developer productivity for developers using Copilot.

So it starts by considering what developer productivity means and how to most effectively measure it.

The first finding is that developer productivity goes beyond just speed.

Users reported improved job satisfaction, reduced frustration and preserved mental energy during repetitive tasks.

And, contributes to a more fulfilling and efficient work experience.

The survey found that when using GitHub Copilot, there was a significant and measurable improvements in perceived productivity, satisfaction and wellbeing, and efficiency and flow.

And they concluded, Oh wait, no, sorry.

Speed is important too, and to evaluate that, they conducted a controlled experiment, and they found that developers that use GitHub Copilot were able to complete, the task of writing a web server in JavaScript in less than half the time of those who weren't.

Sorry, they concluded by saying that it provides measurable benefits and that it's reflected in the feedback that they've heard from other companies using CoPilot.

But we shouldn't just take the word of Microsoft in this.

There are actually many more papers from independent researchers exploring this topic.

One of them I have here was a peer reviewed paper entitled The Effective Generative AI Tool Use by Students on their Computational Thinking Skills.

So this found that, tools like ChatGPT have the power to transform programming education by enhancing self efficacy and motivation.

And concluding that the use of ChatGPT in programming education statistically significantly increased students computational thinking skills.

So it definitely provides.

Some benefits, and there were a whole lot more papers.

I didn't have time to go through more.

But a lot of them reached very similar conclusions that AI does, help with education and it helps with, development.

That's what we're thinking to do.

So what we've looked at, we've looked at how to generate code with AI.

We've looked at the benefits of AI tools.

We've looked at how to, do prompt engineering and make, get the most out of your AI.

We've done an example of building an app with AI.

And we've looked at a small slice of the industry research that goes on.

If you haven't already, I strongly encourage you to, find out for yourself how AI can improve your workflow.

Thank you.

Happy to take any questions.