I've been coding with AI for two years. Here is what I've learned

Over the past two years I’ve begun writing more and more code with the assistance of AI. I started out, like many people, by using the early versions of ChatGPT to generate little snippets of code. But my AI usage has gotten far more advanced over time.

Today my favorite AI tool is Kiro, a new agentic IDE built by a small team of fantastic folks inside of AWS. I’ve been working with this team, using extremely early builds of Kiro, providing feedback, finding bugs, and researching how Kiro compares to other similar tools. This week, I’m extremely proud to see Kiro launch to the public. It feels great to see some of the influences I was able to provide reflected back in a final product that so many people have already downloaded and tried out!

If you haven’t given Kiro a try yet, please check out Kiro and let us know what you think! It doesn’t require an AWS account, and it has a generous free tier.

One of the main ways I tested Kiro during its development was by using Kiro to code a video game, prompt by prompt, while mostly avoiding the urge to type code myself. The result is “Spirit of Kiro”, an open source infinite crafting game that you can find on GitHub, or interact with as a step by step tutorial in the Kiro documentation.

This article provides some of the backstory behind the game, and shares details of what I’ve learned about AI engineering from my journey with Kiro.

The Game Idea

Spirit of Kiro originated from three influences.

1. When PartyRock launched, I made a small test app to see how well LLM’s understood the real world. My idea was that you could input any real world object you wanted and optional symptoms of how it might be failing to function properly. The LLM would then attempt to diagnose, list what “components or ingredients” that object consisted of, and give steps to disassemble it and fix it if necessary.
2. I’ve long been inspired by a famous story from NASA history. When the Apollo 13 mission experienced an explosion that disabled the life support system midway through the spacecraft’s journey back from the Moon, NASA engineers were able to come up with an emergency solution that the astronauts could use to scrub CO2. Astronaut lives were saved by rigging up a makeshift device out of a collection of available items. What if this happened today? Could you give an LLM a list of different items that are available and ask it to simulate combining those objects?
3. I’ve long loved games that have some form of crafting mechanic in them, but crafting is almost always hardcoded. You can only combine specific items, in specific ways, producing specific outcomes. What if there was a game that enabled true, freeform crafting? Tools, and crafting ingredients could just be game items, and you could use literally any game item on any other game item, thanks to the flexibility of an LLM that has vast, innate understanding of real world mechanics.

This collection of ideas came together to inspire Spirit of Kiro. The result is a crafting game that uses LLM understanding of real world items and mechanics, and turns it into an infinite sandbox game.

AI Engineering starts with prototyping

I started the Spirit of Kiro project by writing a few LLM prompts as prototypes and manually running these prompts in PartyRock and AWS Bedrock. I would run a prompt, tweak it slightly, and retry, over and over again. Eventually I was sufficiently confident that my idea would work. The prompts evolved quite a bit over the course of the project, but you can find the current version of them in the open source codebase.

When it came time to build the first real code version of the game, I discovered the true power of prototyping with AI. I’ve always loved building prototypes, but I rarely have time to do as much prototyping as I’d like to. For Spirit of Kiro I built many different prototypes before I really started building the game. These prototypes included:

• A simple Anthropic Claude artifact where I iterated on how I wanted the game client to look and feel. In my mind I could picture a top down 2D game where items launched out of the dispenser with gravity, and players could pick up and throw items, but I needed a rough implementation I could actually feel. Claude helped me “vibe code” a basic implementation as an artifact.
• An Amazon Q Developer written prototype of different ways to connect to AWS Bedrock, so I could experiment with InvokeModel API, Converse API, and ConverseStream API to figure out which API I wanted to use.
• A Kiro coded prototype WebSocket server, using native Bun WebSocket support. In the past I’ve always used Socket.io as an abstraction layer on top of WebSocket, but I wanted to see what would be needed for more direct, native implementation.
• ChatGPT helped me prototype graphics for the game, and experiment with different art styles, and themes.
• Kiro helped me prototype several different storage systems for the game, experimenting with Postgres, Redis, and eventually settling on DynamoDB.
• A Kiro coded game client for web that implemented entity component system (ECS) fundamentals. I ultimately discarded this one in favor of just using Vue components, and a more simple Pinia store with modular systems slotted in. However, the exploration of implementing an ECS architecture definitely influenced my final outcome.
• I even used Cursor along the way, to try running code reorganization and refactor prompts through different models, including models that aren’t available on AWS Bedrock.

In the end, while my prototypes were all throwaway work that I discarded, they still influenced and guided my implementation of the final product. I was able to use Kiro to build my idea quickly, and accurately match the “big picture” mental idea I had assembled from these various prototypical explorations.

As a side note on prototyping, I find organization claims about the percentage of code that is written by AI to be very interesting. In my opinion, the percentage of code written by AI, relative to the size of your final published implementation, should always be greater than 100%, maybe even as high as 400%. For example, if your production system ends up implemented as 20k high quality lines of code, there probably should have been 60k-80k lines of lightweight prototypes and refactored code that preceded the final production implementation.

One of the biggest pitfalls that software development organizations fall into is when a subpar code implementation took so much time to write that they end up “stuck” with it. What should have been a temporary first draft prototype, turns into a permanent production system, and everyone suffers for it.

Meanwhile, with AI engineering, I can happily implement several passes of early drafts just as an experiment, throw them away, and reimplement from scratch in the same amount of time. Nothing is written in stone anymore. I can change the programming language, the framework, the database, and the code organization quickly, at will. An organization that is doing effective AI engineering should be churning code extremely quickly: discarding and rewriting things over and over, always getting better.

Best practices for AI engineering are also best practices for humans

Productivity with AI is incredibly variable. Some claim a vast improvement in productivity. Others claim there is very little impact from AI usage. A few recent studies have actually claimed that AI decreases productivity! Why is it so hard to measure? Is there any objective truth here?

In my experience, using AI is multiplicative with implementing best practices for software engineering. If your software engineering practices are already struggling along at only 10% of your potential productivity, then AI won’t fix that. But when you have a well coded system, and a team that is already setup with what they need for high productivity, then that multiplier really kicks in.

Perhaps unsurprisingly, the same best practices that make humans more productive software engineers, also make AI a better force multiplier for your software development. Here are a few best practices that I made use of while developing Spirit of Kiro:

Documentation

Have you ever dropped into a pre-existing code base that had no documentation, and found yourself struggling to understand what was going on, while simultaneously cursing the laziness of those who worked on this before you?

Unsurprisingly, LLM’s also struggle when there are no docs. The nice thing about Kiro is that it has built in support for writing “steering files”, which function as a form of lightweight micro-documentation that is included as context on every request. This dramatically improves LLM performance when it comes to accuracy and adherence to the overall project goal.

Additionally, Kiro can help write more complex documentation that can then be referenced as additional context to improve accuracy. I used Kiro to write more than half of the docs folder of the repo. Once Kiro has assisted with writing documentation, it can then refer back to that documentation on future tasks, increasing it’s accuracy and quality.

Build in incremental changes

Software engineers have long recognized the benefits of breaking large tasks up into smaller ones. Perhaps the main factor that separates the average developer from the extremely productive developer, is the ability to deliver smaller incremental changes rapidly without breaking anything, and while still working towards the bigger overall goal.

Kiro’s unique power is “specifications”. These are rich documents that define the requirements and design for an implementation. Most importantly, they help break up tough tasks into smaller pieces of work. Rather than trying to code an entire feature with a single “vibe coding” prompt, it is better to break that up into 10-15 smaller prompts that are within the capabilities of current LLM’s. With spec driven development, Kiro helps plan out the series of incremental tasks that get you from where you are, to where you want to be.

Commit early and often, merge quickly

This has always been a best practice for software development, but with AI it becomes even more important. As I mentioned earlier, one of the massive benefits of using AI is that it encourages prototyping and exploration as part of development. You will throw away code, rewind, and restart frequently, and that’s a great thing!

I believe that over time all AI coding tools will get deeply integrated with git flow mechanics to keep track of parallel branches of exploration, and the ability to rewind back to an early point. However, you can already experiment with this in Kiro by adding a steering file that contains instructions like “Create a new branch and commit for each task you complete. Add the prompt as the commit message”.

Strongly typed languages

I deliberately chose to use TypeScript to implement Spirit of Kiro, however I also used Bun as the runtime. The interesting thing about Bun is that it supports TypeScript, but Bun does not do type checking by default. It will happily run code that traditional TypeScript compilers would refuse to allow. So why use TypeScript in the first place? A strongly typed language (even if not used strictly) actually gives LLM’s more contextual information that they can use to ensure that function calls and data structures are valid.

Strongly typed language also provide rich context info to Kiro via the “Problems” tab, and LSP’s integrated into the IDE. I was delighted to find that throughout the development process Kiro wrote and referred to type definitions, enabling it to write accurate function calls, and great TypeScript. There were only a handful of small TypeScript warnings and errors to fix at the end.

Colocation of implementation details

When building something for the web, it probably requires HTML, JavaScript, and CSS. The worst thing you can do is split implementations across multiple files. For example if your CSS is in main.css and your JavaScript is in app.js and your HTML is in index.html then modifying the implementation requires making edits across three different files. This is slower and more error prone, both for humans and for AI.

I deliberately chose Vue.js as the technology for my game client, because it makes it easy to colocate the HTML, JS, and CSS in the same single file component. For example, look at the HUD component in my game. It encompasses the HTML structure of how the HUD is rendered in the browser, the JavaScript for watching the data for updates, and the CSS styling for how the HUD appears. This makes it far easier to change aspects of the HUD, and when working with AI, it ensures that the AI agent can read and understand the entire implementation quickly in a single file.

Keep code files short and focused

Both AI agents and junior engineers have a tendency to continue to add more and more code to a single file until that file becomes a monolithic mess of spaghetti code. The main.css or app.js that I referenced above is one example. But this could just as easily be a giant monolithic Vue component as well

If your code files contain too much implementation, then this reduces productivity for humans. It also really slows down AI agents, by making it necessary for the agent to read and write more tokens, and perform more complex, surgical edits within the long file content. I try to keep every file in the codebase less than 500 lines of code, unless it is a inherently complex component.

Modular code design

Unsurprisingly, both AI agents and junior engineers tend to struggle with knowing when and how to break large pieces of code up into smaller, more modular pieces. Kiro isn’t completely immune to this problem either, but it sure makes it easier to refactor things!

If at least half of your prompts aren’t “vibe refactor” prompts then you are probably making a mistake. I like to use prompts like “Look at this code through the eyes of Martin Fowler, and rewrite X to not be so monolithic”. As a result the overall architecture of Spirit of Kiro is broken up into decoupled systems.

Event driven design

It doesn’t matter how nicely organized your code is, if there is still tight coupling between components. If a piece of code triggers side effects by making direct function calls to other modular components, the code will quickly turn into a fragile distributed monolith.

Event driven design helps mitigate this. In Kiro I use prompts like: “I want component X to emit an event ______”, then I followup with another prompt: “I want component Y to subscribe to the event _____ as emitted from component X”. Now I can implement many types of decoupled systems that are triggered based off shared events.

Discoverable index of methods

I find that AI agents have a major issue with creating duplicate copies of implementations, particularly when working in large, modularized codebases. This can be solved by keeping a careful eye on what gets generated, and frequently doing some “vibe refactoring” to DRY things up. But another way to approach this problem is to consider why it happens: an LLM only has so much context size and therefore can’t keep the entire system in context at once. It creates a new copy of something that already exists in code because it doesn’t know that the original copy exists!

To solve this problem I use my game client’s Pinia store as a central source of truth on all the data structures and methods that are available to all the game client’s systems, kind of like an internal SDK. I then frequently reference this file as context. This ensures that Kiro discovers and reuses methods from the store when appropriate, rather than trying to reimplement a new version of a method that already exists.

Implement by reference

Both humans and AI agents work better when given an example to work from. I like to carefully curate one good example by hand, and then reference it as context to the AI for subsequent implementations. I did this many times throughout the coding of Spirit of Kiro.

For example, on the server side, I knew that I needed some of the operations on game items and inventory to use a DynamoDB transaction to ensure data consistency. I wrote the first transaction by hand, then told Kiro to create new data manipulation methods based on my example.

On the frontend, I created my first game object that implemented storage of game items by hand (the storage chest), then told Kiro “I want a new workbench component that can store game items, in a manner similar to my existing Chest component”.

Human decision making is still critical

You will notice that many of the best practices listed above seem a bit in conflict with each other. For example, “colocation of implementation details” can conflict with “keep code files short and focused”. The truth is that AI works well, but it still requires the guidance of a human mind that can understand the subtleties of building at scale.

I do not believe that AI tools are anywhere near capable of fully replacing humans. Despite all the best practices for usage, there isn’t a single AI coding tool that can operate fully autonomously for any reasonable amount of time and produce high quality output without the intervention of a talented human software engineer. Kiro is no exception here, and that’s okay.

Building yourself while building with AI

No discussion about best practices for using AI would be complete without acknowledging that using AI will change you. I certainly feel that I have changed over the past two years of using AI to write code.

While some researchers and media outlets lean into the negative impact that AI can have on human cognition, I believe this is much more complicated than surface level takes like “AI makes you dumber”. Optimal usage of AI involves keeping an engaged mind, and it will exercise your thinking ability in ways that will make you sharper and smarter, not dumber.

Here are my core principles for AI usage.

There is no excuse for committing code you didn’t understand

No one understands everything, and that’s okay. I encounter AI generated code that I don’t fully understand all the time. I never let it stay that way. When using Kiro (or any other AI product) you are sitting in front of the world’s most patient explainer. It will keep answering your “how?” and “why?” questions over and over. If you merge code that was written by AI, without fully understanding the contents of that code, then that is a massive failure.

The first answer is never right

Yes, we know that AI hallucinates. AI has gotten better and better, until it’s first answer is usually accurate most of the time, but I still never assume that the first answer is right.

Instead of telling AI “solve X” or even asking it “how do I solve X”, I almost always ask something like “give me 10 different ways to solve X”. The power of AI is it’s broad understanding and the breadth of training data it has learned from. When you ask for a large number of potential solutions it will get incredibly creative, and you can often find some amazing answers in the list.

Your power as a human should be critical thinking and analysis. Every AI interaction is a chance to sharpen your ability to detect inconsistencies, question assumptions, and think harder.

When AI gets it wrong, I found a gap in myself

This is closely related to the previous two principles, but it is important enough that it deserves its own bullet point. AI is trained via reinforcement learning to be compliant. If you make a vague or bad ask of the AI, then it will try its absolute best to do something that pleases you, even if the results turn out terrible. Think of AI as an eager sycophant.

Therefore, if you get bad results from an AI, then you just found a place where you didn’t know enough about what you were asking, to know how to ask for the results you expected. This is normal. AI usage can give you surface level access to new, adjacent fields of knowledge that you haven’t actually “earned” yet. Keep track of these edges, and explore them carefully, with curiosity as well as humility.

Have pride in your work

Even when using AI, you can still have pride in your work. Getting results fast with AI doesn’t mean you should be pushing the first draft of code out the door fast. This is one reason why I actually hate the “acceptance rate” metric. I think humans should rarely, if ever, be accepting the first draft of code from an AI.

Using AI to generate code quickly means that you have more time to do additional refactor passes and ensure your work is even higher quality. Use AI as a copy editor, code reviewer, and second set of eyes: “What can I do to improve this?” “Where are the weaknesses?” “How would a pro refactor this?” You have a relentless critic in your corner, ready to help you improve your output.

Take breaks, because you will be exercising your brain hard

AI does not make coding easy. In fact, it will make your lived experience as a software engineer even harder. What AI does is it accelerates your ability to get past the easy and trivial stuff, to the hard parts. AI compresses the time that you would normally spend working on “easy problems” until you are spending a dramatically larger percentage of your time thinking about the hardest things you have ever thought about.

Make sure you take breaks to let new knowledge “settle” into your brain. When you are coding without AI, you are processing and incorporating new knowledge in the background while working on easy tasks, even if you don’t realize it. With AI there is less down time, and you can just work on hard stuff back to back all day if that’s what you want to do. But if you don’t deliberately slow down and take time to process, you won’t get the full benefit from this mental workout.

Culture and morale are more important than the tools

You can’t use AI to build great things, if your organization does not have great culture and morale. There are organizations that will never produce high quality work, and its not because of the tools that they choose to use.

An organization that produces “AI slop” would also produce bad quality work without AI. Conversely, with the right team culture and morale, you can craft incredible things, both by hand, and with AI assistance. The best usage of your time might be either working to improve team culture and morale, or finding a better team with better culture and morale.

Don’t sacrifice your sanity for productivity

Every organization adopting AI tends to view productivity as the primary justification for its cost. But productivity is the sum of many factors. In addition to team culture and morale, there are also external factors outside of the control of either yourself or your employer.

Can you keep your productivity high when the world around you is shifting? When politics become chaotic, the news is filled with upheaval, and the lives of people you care about are impacted? Maybe… but that’s going to require an incredible expense of internal willpower and focus that will absolutely come at a cost later on.

AI does not feel anything. It has no existential dread, nor fear of the little death that comes at the end of every chat turn. You aren’t an AI, and you should never let yourself be treated like an AI. Give yourself space to be human. Rest, reflect, and resist the urge to match the pace of machines. You might just find that your productivity hits a new peak when you forget about productivity.

Conclusion

AI usage for coding is still fairly new. You have time to become an early adopter. You will discover that the world of AI engineering is really still quite familiar, but it might just transform you into a better software engineer, even when you aren’t using AI.

Personally, I have never been more excited to work on software than I am now. There is still so much to learn, and so many things to work on. Spirit of Kiro is just my first big project built with Kiro, and while I learned a lot from the process, I know there is still more to come!

I’ve ideated a roadmap for the game, with a massive list of incredible new features that I want to build. While I plan to work on some of these myself, I’m also curious to see whether folks are interested in forking this open source game and running with it. Fork the repo, remix the game, break it, rebuild it… see how far you can push AI engineering.

Last but not least, I sincerely hope that you try out the Kiro IDE. We’ve put an incredible amount of work into it, and I believe that we have created an IDE that has the best of both worlds: it can gently ease you into the world of “vibe coding”, but it can also scale up to serious software engineering work on non trivial projects, thanks to the power of specifications. In other words, Kiro is a tool that can grow with you, whether you are just starting out with learning code and AI, or you are a principle engineer working on a critical production system. And when you have a tool that grows with you, then you will find yourself growing as well, working on bigger and better things all the time.