Generating application code from tests with AI

Part 5 of a practical example of using AI for coding

This is part 5 of a multi-part series:

• In Part 1, we discussed setup and best practices and answered the question: “Will AI replace software developers?”.

• In Part 2, we designed a solution and created a design document.

• In Part 3, we generated a sequence diagram and an OpenAPI specification.

• In Part 4, we generated tests. First.

• In Part 5, we’ll generate the application code.

Recap: URL Shortening Service, generated by AI

• We’re writing a solution for the URL Shortener Challenge.

• We’ve got a great design that we visualized with PlantUML and generated OpenAPI specifications for.

• ChatGPT generated some tests.

• Developers that use AI will replace those who don't  — It will take more time before AI completely replaces developers.

Generating an application from tests

If you’re not a fan of Test First development like me, you’re glad to finally be writing application code. You’re probably wondering why we didn’t just start there and ignore all this design and business needs nonsense. You might want to learn about how all this nonsense makes teams autonomous.

Boilerplate

Boilerplate first. The code base has to be set up. I asked ChatGPT to give me commands for the poetry stuff. I’ll summarize it here.

• Using poetry to manage dependencies, create a pyproject.toml file with poetry init, adding connexion and redis as a dependency.

• poetry add --dev the test dependencies like pytest, freezegun, and fakeredis.

• Paste tests into test_shorty.py

• Run the tests.

Of course, they fail; we have no application code.

Generate the code for real this time

To generate the code, I’m using CodeGPT plugin for PyCharm.

I select the code and use this prompt:

Write an implementation that satisfies these tests in the selected code. Here is the design doc:

{complete design document}

Note the “selected text” part. It uses whatever I have selected in the IDE as part of the input.

import hashlib
import random
import string
from datetime import timedelta
from flask import Flask, jsonify, redirect, request, url_for
from redis import Redis

app = Flask(__name__)
redis = Redis(host='redis', port=6379)

BASE_URL = "http://example.com/"
...

---

🛑 Stop generating!

It did a decent implementation, but it forgot one important piece:

Not quite. This uses connexion as specified in the specs.

After a quick and familiar apology:

My apologies for that mistake.

Here is the updated source code using connexion:

import hashlib
import random
import string
from datetime import timedelta
import connexion
from redis import Redis


app = connexion.FlaskApp(__name__, specification_dir="./")
app.add_api("openapi.yaml")


redis = Redis(host='redis', port=6379)

BASE_URL = "http://example.com/"

def generate_short_code(url: str) -> str:
    """
    Generate a short code for the given URL.
    """
    # Generate a unique hash for the URL using SHA1
    sha1_hash = hashlib.sha1(url.encode('utf-8')).hexdigest()
    # Convert the hash to base62
    base62_hash = encode_base_n(int(sha1_hash, 16), 62)
    # Truncate the hash to 7 characters to get the short code
    short_code = base62_hash[:7]
    return short_code

...etc etc

---

And just like that, it’s done. The code was close to correct without much cajoling because it had the tests and the design doc to work with.

It did need some corrections, though. The following prompts were followed by selecting the relevant part of the code and explaining the problem:

• The url_sha1 should be base62 encoded instead of hex encoded.

• base62 is not a real module. Use the existing encode_base_n function instead

• In the selected code, there is a lot of duplication. Be DRY.

  • That made the duplication worse. 🤦🏼

• Why is the selected code getting a TypeError('not all arguments converted during bytes formatting')

• How do I convert bytes to an int? (Yes, I was being lazy)

After many apologies, we got a good enough solution for me to take over with.

Make it work

The next part is not glamorous unless you like Test First development 🤩.

• Run the tests.

• Get an error.

• Fix the error by letting CodeWhisperer auto-complete your thoughts.

• Repeat until all the tests pass.

---

You can view the complete code along with:

• Design Document

• PlantUML diagram

• OpenAPI specification

• Tests

• Prompts

• A note from our AI partner

---

Conclusion

• Most of the time was spent designing a solution with ChatGPT.

• But that made everything else go faster.

• The design doc was fed into the prompt to generate tests, generating correct tests nearly immediately.

• Asking AI to generate code that satisfies a selected set of tests is freaky, but it works. It would not work as well if I didn’t supply it a design doc first.

• The moral of this really is:

  • Design First

  • API First

  • Test First

• Whether you get AI to help you with each stage or not, your results will be better, less error-prone, and faster.

• And doing it with AI is faster and fun but also frustrating.

Alternatives Considered

What if I skipped all this design, test, PlantUML, etc. stuff and just pasted the challenge as a prompt into ChatGPT 4?

• For a well know problem like a URL Shortener, it will give you working code right away.

• When I tried it, it generated a partial “example” Javascript solution using the Express framework and MongoDB for storage.

• I could tell it, “Write it in Python backed by redis.”

• Then I could ask for alternatives to generating short URLs other than random letter choices.

• And then I could ask it to write tests.

• I could walk through each minor change with it and eventually come to a similar solution as starting with the design first.

• Maybe I would have thought about traffic loads and other non-functional requirements. Maybe I would have thought of expiring URLs.

Here’s the thing, though.

It would have taken more time, I would have learned less, and the solution would have been less robust.

That might be okay when a quick and dirty solution is sufficient. My goal was to show you a practical example of how you can use generative AI to aid your work.

At a certain level, quick and dirty is not good enough.

By writing the design document first, we had thought through all the major details, so it was quick and easy when it came time to write code. Notice how this part of the article series was the shortest?

We could also have handed the design document to someone else, perhaps someone to develop the UI. Perhaps another AI. We would not have to walk them through every step to figure out what the software should do in order to generate the UI.

We’d start with the design doc and tell them to write code for it.

The design doc, the diagrams, and the API specification all contribute to making collaboration with other developers (human or AI) faster and more productive.

---

Next

All done. Thanks for the journey.

• Part 1: Setup and Process

• Part 2: Design

• Part 3: Sequence diagram and API specification

• Part 4 Generating tests

• Part 5: Generating the application and making it work ← YOU ARE HERE