Tag: llm

PIM Progress

~ David Cameron ~ Leave a comment

A week ago I started building my PIM and I have a status update to share:

In a little over a week, working a little in evenings, but mostly on weekends I have a pretty functional PIM.

In this age of vibe coding, this might seem a little slow, however this was built to be a well structured app. This was built to be a scalable, cloud hosted app that could run at zero cost, leaning on the AWS free tier.

How I built this

I built this the way that I would build an app manually, just that I got Claude to do the actual work for me.

  • carefully designed infrastructure, running on the platform of my choice (AWS)
  • Comprehensive automated tests (Unit, Integration and Functional tests)
  • CI/CD (Github)
  • Code reviews — i.e. getting the squishy human to review the code Claude wrote
  • Issue tracking (Linear)

Issue tracking, seriously?

Yep, I created tickets for almost everything I build before I built it. This might seem like overkill, but I found it helpful in three main ways.

Better discipline

I often have ideas of things I’d like to do, fix, or change in future while I’m mid-task. Writing these down as tickets means I don’t break my current flow of work, so it stops me churning without achieving anything. Often I’ll create a placeholder ticket, then fill it out properly later.

Able to work anywhere

Ticket writing also works well while I’m waiting for Claude to finish another task. I can create and edit tickets on my phone, so if something occurs to me when I’m not at my computer, I can capture it straight away instead of losing it.

Able to prioritise the work

Lastly, tickets make it much easier to prioritise. Often I’ll have an idea for a change that feels pretty important in the moment. In the cold hard light of day, it can become clear that it’s just not as urgent, and I can move it down the priority list.

What this looks like as a workflow

This mostly used a worklog workflow, which is similar to the process described here: https://nimbalyst.com/blog/claude-code-plan-mode/ — it argues for using Claude Code’s plan mode to agree on a concrete, file-specific approach before any edits happen, then keeping that plan as a persistent artifact you can check the implementation against.

The basic workflow is:

  1. Get Claude to create a plan that is committed to the repo. The plan should include a checklist.
  2. Work through the checklist implementing each step. Collect all prompts and responses in the worklog.
  3. When complete, raise a PR

This means the tickets can include a bit of detail on what you want to implement. Claude fleshes this out into a plan. You review the plan and confirm at each step.

Timeline

1-2 days

I had a login page that didn’t work. This might not seem much progress, however there was a lot behind that. This gave me:

  • the shell of a front end project, complete with unit tests
  • the shell of an api project, with a couple of endpoints implemented, complete with unit tests.
  • integration tests against the API
  • placeholders for actual pages to be built
  • a front end and API login process that worked on my machine
  • Fully built AWS infrastructure, including front end served through the CDN, api served by Lambda, data stored in DynamoDB
  • working CI/CD pipelines in github to build, unit test and deploy changes (including TF infrastructure)
  • an almost working site in AWS

This is the reverse of the way you’d build a vibe coded app: building the infrastructure first.

3-7 days – limited progress to work in evenings

I had a transaction listing page, which included:

  • fixing the login bug (turns out I was missing an egress rule in the Security Group)
  • A process to upload and process transactions from bank accounts
  • Storing the transactions
  • Classifying the transactions, including storing classifications for transactions
  • Classifying incoming transactions

8-9 days

I have a fully functioning dashboard, including:

  • I had fully completed the transaction classification process, including identifying transfers
  • able to reclassify transactions
  • dashboard summary of income, expenses
  • dashboard month by month comparison
  • switched to using QIF files rather than CSV as it meant supporting less file formats

Conclusion

After 9 days I had a well architected, scalable platform that would run at zero cost, fully built by Claude on pro plan.

This was possible because while Claude wrote the code, I designed the architecture and reviewed the implementation. There were multiple occasions where Claude took the wrong approach and needed to be corrected. For example, the initial handling of environments for terraform was poor, but easily corrected when identified.

I still need to work through automating the bank data import. At some point in the future I might create a mobile app for this. I might even open this up to a wider audience.

LLMs (AI) are changing the way code is written

~ David Cameron ~ Leave a comment

The way code is written is being fundamentally changed by LLMs. This is a freight train that you cannot stop. To quote someone else: If you’re not part of the steamroller, you’re part of the road.

I’ve been working in software for close to 30 years. In that time, I’ve written code in: Fortran, C, C++, Assembler, Matlab, VBScript, Visual Basic, Go, bash, powershell, batch files, C#, F#, Java, php, perl, python, javascript, typescript, objective C, SQL, Ruby and probably a few more I’ve forgotten.

I’ve written a lot of code, I’ve read a lot more.

I’ve never seen a change on the scale of what we’re seeing with AI.

Where we’ve been

For all of my career coding the process of creating or reading code hasn’t significantly changed. It’s a developer (or engineer) using a text editor to read and write code.

Sure, there have been advances. We’ve had the invention of IDEs, which have got progressively more capable. They’ve made it far easier to navigate and modify codebases. But when you get to the bottom of it, it’s still someone looking at a bunch of code in a text editor window.

The editors have changed (even emacs and vi/vim), but the process has been the same.

Not anymore.

What can LLMs do

LLMs are changing all this. Developers are still reading code (and likely a lot more code), but they’re writing less because LLMs are generating the code.

The reality is that LLMs are really very good at generating code. They get the syntax right, they structure the code reasonably well, they’re able to create good software. Really.

What they’re not as good at is judging what to build and whether this is the right thing to build. Humans are far better at this.

Good developers (or engineers) are needed to point the LLMs in the right direction and to review what they’ve generated.

LLMs are also quite good at reading code. They’re a great assistant to exploring an unfamiliar codebase.

What this looks like as a work pattern

Obviously this is a rapidly evolving area, but at the moment the current LLMs are good at doing well defined repetitive task. Working with LLMs today feels like managing a team of tireless and enthusiastic junior developers. They can move quickly, but they need extremely clear instructions. Without guidance they often go off in the wrong direction.

In practice, the workflow looks something like this:

  1. Break work into small, well-defined tasks.
  2. Send an agent (or coding assistant) to implement the task.
  3. Review the output carefully.
  4. Fix, refine, or sometimes discard the result entirely.

The key lesson is that smaller tasks produce better results.

Some real examples

I’m going to post a more extensive set of recommendations, but for now, some real examples of how I’ve found AI coding assistants helpful in the past few months (primarily working with a large and legacy codebase)::

  • refactoring code – it does a good job of completing clearly defined, repeatable tasks in the codebase
  • creating unit tests – a huge amount of the work of creating unit tests is setting up mocks. Claude does this well.
  • managing and creating functional tests. It can generate bdd descriptions from existing tests. It can convert tests from legacy frameworks to new ones. It can refactor tests to extract common code.
  • performing code reviews. You can get the AI to look for specific issues, beyond what you get from existing tools easily. For example, you can validate that the code uses our standards for front end styling
  • searching for dead code – taking the logs of production website usage, then getting Claude to find out if we have code that is not being used in production.
  • searching for new errors in logs – building a python script to query the error logs in production to see if there are any new errors

Summary

If you are building software and you’re not using AI tools, you need to start now.