From ebbd3a11cdfb6a88f7187e3f3cf6f65bdc488ea8 Mon Sep 17 00:00:00 2001 From: David Johnston Date: Wed, 27 May 2026 16:49:21 +1000 Subject: [PATCH 1/3] Draft --- .../posts/reasons_to_be_a_token_miser.mdx | 81 +++++++++++++++++++ ...ime_a_task_takes_changes_your_workflow.mdx | 38 +++++++++ 2 files changed, 119 insertions(+) create mode 100644 src/routes/posts/reasons_to_be_a_token_miser.mdx create mode 100644 src/routes/posts/the_time_a_task_takes_changes_your_workflow.mdx diff --git a/src/routes/posts/reasons_to_be_a_token_miser.mdx b/src/routes/posts/reasons_to_be_a_token_miser.mdx new file mode 100644 index 0000000..6b8666e --- /dev/null +++ b/src/routes/posts/reasons_to_be_a_token_miser.mdx @@ -0,0 +1,81 @@ +--- +meta: + title: Reasons to be a token miser + description: There is a school of thought that says 'Just throw money at the problem' - here's why you should be constantly optimising your token usage. + dateCreated: 2026-05-27 + + +tags: + - "ai" + +--- + + + +1. **Less tokens = faster results** + +There's a big difference between a workflow that takes a day to complete vs ten minutes to complete, vs one minute to complete, vs ten seconds to complete, vs a second to complete. + +If your prompt gives enough context to the LLM that it comes to an answer _without_ having to produce its own lengthy reasoning chains, chances are you get the answer quicker and stay focused on the task at hand. + + +2. **If a solution can be got to with less tokens, it's probably the better one** + +The observation I'll make here is that sometimes the LLM gets into a state where it goes around and round in circles in a reasoning chain, eventually it'll stop but it often it won't be the answer you need. + +I propose a cludgy aphorism here - 'If an LLM hasn't come to a solution by 80K tokens (or whatever number), chances are it's going to use another 200K tokens to get there, and chances are, that solution isn't going to be a good one'. + +3. **Positions yourself to avoid a rugpull** + +In many cases you can get away with just throwing a large model at a problem, coming back in an hour or so, and there will be _some kind_ of working solution. + +However, remember that currently AI use is subsidised by the AI providers - in some future such case it would be good to be able to move to a cheaper and/or self hosted provider. + + +## Practical Suggestions + +1. Make use of code generators + +For any coding work that is strictly derivative - that is - the content of the code can be trivially determined programatically - don't get AI to do this work, relegate that work to scripts. + +Instead of giving your LLM the instruction: + +``` +After creating the node in folder `src/nodes/`, add that node to the registry in `src/nodeRegistry.ts` +``` + +Give it the instruction: + +``` +After creating the node in folder `src/nodes/` run `bun generateNodeRegistry` +``` + +Even better, a hook could be used. + +2. Make use of codemods + +Let's say you you have some code that looks like this: + +```javascript +function foo() { + + return doX(); +} +``` + +And you are conducting a refactor where you now use `doX2()` which is an async function + +```javascript +//👇 +async function foo() { + + return doX2(); +} +``` + +This is a complicated task that, that will likely have cascades beyond just the initial file being changed. + +Don't just have AI come up with a plan and execute it. Have the AI create codemods - test those codemods, and execute that instead. + +A big advantage of codemods is that once they're written - they're repeatable. Chances, this change when merged will create all sorts of merge conflicts. The codemod can be used to transform the incompatable branch before doing the merge. + diff --git a/src/routes/posts/the_time_a_task_takes_changes_your_workflow.mdx b/src/routes/posts/the_time_a_task_takes_changes_your_workflow.mdx new file mode 100644 index 0000000..904c48c --- /dev/null +++ b/src/routes/posts/the_time_a_task_takes_changes_your_workflow.mdx @@ -0,0 +1,38 @@ +--- +meta: + title: The time required to wait for a job to complete - changes the workflow for how you do it + description: There's a big difference between a job that takes one hour, vs a job that takes one minute vs a job that takes less than a second + dateCreated: 2026-05-27 + + +tags: + - "software_engineering" + +--- + +See also: https://softwareengineering.stackexchange.com/questions/459194/is-there-a-term-pithy-law-that-represents-the-idea-that-the-time-it-takes-for-an + + +| Duration | Attention/Context | +|----------|-------------------| +| 1s | Unbroken attention, no context switch | +| 1-10s | Attention broken simply by boredom, potential to view another browser tab, check emails | +| 10s-1hr | Almost definite context switch to small tasks or non-productive tasks | +| 1hr+ | A context switch is required. At large waits, the context switch is actually less disruptive, because you now have time to get into that thing | + +Where I think this is most relevant when considering the time it takes for developer tooling to run - think running tests, linting either locally or in a build server. + +It can be tempting to say 'well what does it matter if the tests take fifteen minutes/two minutes/30 seconds to run - you'll waste more time waiting for the elevator/making coffee/participating the the company ping pong tournament.' + +But the point isn't the actual time spent, the point is that wait completely changes the way you do your work. + +## Practical suggestions + +- Adopt all the fast tool runners - bun, biome, typescript 7, etc. It's absolutely worth it. + +- Shard your test runs on the build server. + +- You can use tools like `lint-staged` to only run checks on a subset of files, but the problem with this is that your change might have broken something downstream from the file you are working on. + - Instead monorepo tools like NX and Turborepo have tooling to run commands only affected packages. If you make a change to package A and package B depends on it, then the tooling will run tests against both packages. + - There is similar tooling in the test runners. Bun has [`bun test --changed`](https://bun.com/blog/bun-v1.3.13#bun-test-changed) which works by using the import graph to determine which modules have changed. Be aware that if you were doing something with dynamic imports or file operations these might escape detection. + - [Vitest has also has a `--changed` flag.](https://vitest.dev/guide/cli.html#changed) From 153c9679a111149d1da5d5b5d8d572b9272cd66a Mon Sep 17 00:00:00 2001 From: David Johnston Date: Wed, 27 May 2026 16:51:30 +1000 Subject: [PATCH 2/3] Fix typos --- src/routes/posts/reasons_to_be_a_token_miser.mdx | 8 ++++---- .../posts/the_time_a_task_takes_changes_your_workflow.mdx | 6 +++--- 2 files changed, 7 insertions(+), 7 deletions(-) diff --git a/src/routes/posts/reasons_to_be_a_token_miser.mdx b/src/routes/posts/reasons_to_be_a_token_miser.mdx index 6b8666e..782d02c 100644 --- a/src/routes/posts/reasons_to_be_a_token_miser.mdx +++ b/src/routes/posts/reasons_to_be_a_token_miser.mdx @@ -21,7 +21,7 @@ If your prompt gives enough context to the LLM that it comes to an answer _witho 2. **If a solution can be got to with less tokens, it's probably the better one** -The observation I'll make here is that sometimes the LLM gets into a state where it goes around and round in circles in a reasoning chain, eventually it'll stop but it often it won't be the answer you need. +The observation I'll make here is that sometimes the LLM gets into a state where it goes around in circles in a reasoning chain, eventually it'll stop but often it won't be the answer you need. I propose a cludgy aphorism here - 'If an LLM hasn't come to a solution by 80K tokens (or whatever number), chances are it's going to use another 200K tokens to get there, and chances are, that solution isn't going to be a good one'. @@ -54,7 +54,7 @@ Even better, a hook could be used. 2. Make use of codemods -Let's say you you have some code that looks like this: +Let's say you have some code that looks like this: ```javascript function foo() { @@ -73,9 +73,9 @@ async function foo() { } ``` -This is a complicated task that, that will likely have cascades beyond just the initial file being changed. +This is a complicated task that will likely have cascades beyond just the initial file being changed. Don't just have AI come up with a plan and execute it. Have the AI create codemods - test those codemods, and execute that instead. -A big advantage of codemods is that once they're written - they're repeatable. Chances, this change when merged will create all sorts of merge conflicts. The codemod can be used to transform the incompatable branch before doing the merge. +A big advantage of codemods is that once they're written - they're repeatable. Chances are, this change when merged will create all sorts of merge conflicts. The codemod can be used to transform the incompatable branch before doing the merge. diff --git a/src/routes/posts/the_time_a_task_takes_changes_your_workflow.mdx b/src/routes/posts/the_time_a_task_takes_changes_your_workflow.mdx index 904c48c..06efb19 100644 --- a/src/routes/posts/the_time_a_task_takes_changes_your_workflow.mdx +++ b/src/routes/posts/the_time_a_task_takes_changes_your_workflow.mdx @@ -22,9 +22,9 @@ See also: https://softwareengineering.stackexchange.com/questions/459194/is-ther Where I think this is most relevant when considering the time it takes for developer tooling to run - think running tests, linting either locally or in a build server. -It can be tempting to say 'well what does it matter if the tests take fifteen minutes/two minutes/30 seconds to run - you'll waste more time waiting for the elevator/making coffee/participating the the company ping pong tournament.' +It can be tempting to say 'well what does it matter if the tests take fifteen minutes/two minutes/30 seconds to run - you'll waste more time waiting for the elevator/making coffee/participating in the company ping pong tournament.' -But the point isn't the actual time spent, the point is that wait completely changes the way you do your work. +But the point isn't the actual time spent, the point is that the wait completely changes the way you do your work. ## Practical suggestions @@ -35,4 +35,4 @@ But the point isn't the actual time spent, the point is that wait completely cha - You can use tools like `lint-staged` to only run checks on a subset of files, but the problem with this is that your change might have broken something downstream from the file you are working on. - Instead monorepo tools like NX and Turborepo have tooling to run commands only affected packages. If you make a change to package A and package B depends on it, then the tooling will run tests against both packages. - There is similar tooling in the test runners. Bun has [`bun test --changed`](https://bun.com/blog/bun-v1.3.13#bun-test-changed) which works by using the import graph to determine which modules have changed. Be aware that if you were doing something with dynamic imports or file operations these might escape detection. - - [Vitest has also has a `--changed` flag.](https://vitest.dev/guide/cli.html#changed) + - [Vitest also has a `--changed` flag.](https://vitest.dev/guide/cli.html#changed) From 7bc89a516d644db4db72e4acdd12a40e6e7049b7 Mon Sep 17 00:00:00 2001 From: David Johnston Date: Wed, 27 May 2026 17:07:11 +1000 Subject: [PATCH 3/3] Fixes --- next-env.d.ts | 2 +- .../posts/reasons_to_be_a_token_miser.mdx | 11 +++--- ...ime_a_task_takes_changes_your_workflow.mdx | 36 +++++++++++++++---- 3 files changed, 36 insertions(+), 13 deletions(-) diff --git a/next-env.d.ts b/next-env.d.ts index 9edff1c..c4b7818 100644 --- a/next-env.d.ts +++ b/next-env.d.ts @@ -1,6 +1,6 @@ /// /// -import "./.next/types/routes.d.ts"; +import "./.next/dev/types/routes.d.ts"; // NOTE: This file should not be edited // see https://nextjs.org/docs/app/api-reference/config/typescript for more information. diff --git a/src/routes/posts/reasons_to_be_a_token_miser.mdx b/src/routes/posts/reasons_to_be_a_token_miser.mdx index 782d02c..a0b5e9b 100644 --- a/src/routes/posts/reasons_to_be_a_token_miser.mdx +++ b/src/routes/posts/reasons_to_be_a_token_miser.mdx @@ -10,18 +10,19 @@ tags: --- +import { TextHighlight } from "@blacksheepcode/react-text-highlight"; 1. **Less tokens = faster results** -There's a big difference between a workflow that takes a day to complete vs ten minutes to complete, vs one minute to complete, vs ten seconds to complete, vs a second to complete. +See: The time required to wait for a job to complete - changes the workflow for how you do it

}>There's a big difference between a workflow that takes a day to complete vs ten minutes to complete, vs one minute to complete, vs ten seconds to complete, vs a second to complete. -If your prompt gives enough context to the LLM that it comes to an answer _without_ having to produce its own lengthy reasoning chains, chances are you get the answer quicker and stay focused on the task at hand. +If your prompt gives enough context to the LLM that it comes to an answer _without_ having to produce its own lengthy reasoning chains, then you'll get the answer quicker and stay focused on the task at hand. 2. **If a solution can be got to with less tokens, it's probably the better one** -The observation I'll make here is that sometimes the LLM gets into a state where it goes around in circles in a reasoning chain, eventually it'll stop but often it won't be the answer you need. +The observation I'll make here is that sometimes the LLM gets into a state where it goes around in circles in a reasoning chain, eventually it'll stop and often the solution won't be a good one. I propose a cludgy aphorism here - 'If an LLM hasn't come to a solution by 80K tokens (or whatever number), chances are it's going to use another 200K tokens to get there, and chances are, that solution isn't going to be a good one'. @@ -34,7 +35,7 @@ However, remember that currently AI use is subsidised by the AI providers - in s ## Practical Suggestions -1. Make use of code generators +1. **Make use of code generators** For any coding work that is strictly derivative - that is - the content of the code can be trivially determined programatically - don't get AI to do this work, relegate that work to scripts. @@ -52,7 +53,7 @@ After creating the node in folder `src/nodes/` run `bun generateNodeRegistry` Even better, a hook could be used. -2. Make use of codemods +2. **Make use of codemods** Let's say you have some code that looks like this: diff --git a/src/routes/posts/the_time_a_task_takes_changes_your_workflow.mdx b/src/routes/posts/the_time_a_task_takes_changes_your_workflow.mdx index 06efb19..29fde8b 100644 --- a/src/routes/posts/the_time_a_task_takes_changes_your_workflow.mdx +++ b/src/routes/posts/the_time_a_task_takes_changes_your_workflow.mdx @@ -10,15 +10,37 @@ tags: --- -See also: https://softwareengineering.stackexchange.com/questions/459194/is-there-a-term-pithy-law-that-represents-the-idea-that-the-time-it-takes-for-an +See also [this stack exchange question](https://softwareengineering.stackexchange.com/questions/459194/is-there-a-term-pithy-law-that-represents-the-idea-that-the-time-it-takes-for-an +) + + + + + + + + + + + + + + + + + + + + + + + + + + +
DurationAttention/Context
1sUnbroken attention, no context switch
1-10sAttention broken simply by boredom, potential to view another browser tab, check emails
10s-1hrAlmost definite context switch to small tasks or non-productive tasks
1hr+A context switch is required. At large waits, the context switch is actually less disruptive, because you now have time to get into that thing
-| Duration | Attention/Context | -|----------|-------------------| -| 1s | Unbroken attention, no context switch | -| 1-10s | Attention broken simply by boredom, potential to view another browser tab, check emails | -| 10s-1hr | Almost definite context switch to small tasks or non-productive tasks | -| 1hr+ | A context switch is required. At large waits, the context switch is actually less disruptive, because you now have time to get into that thing | Where I think this is most relevant when considering the time it takes for developer tooling to run - think running tests, linting either locally or in a build server.