I spent a few months building a personal work OS as a Claude plugin, and the most useful thing that came out of it was not the OS.
The thing I keep coming back to is this. Months in, I wrote a test for a feature I had just added. The test was checking whether a promotion backlog got reported the right way. It failed, but not on the thing I was checking. It failed because the skill under test, whose entire job is finding and removing stale knowledge, had been rewriting the knowledge base with no review step at all. Not in that release. In every release, going back to the first one.
Three of four runs did it. One of them rewrote fourteen files. I had read that skill dozens of times. I had reviewed it, edited it, and shipped it in every release since the first one. A single test run found in ten minutes what reading had missed for months.
That is the whole argument for measuring, and I would not have believed it as strongly if it had happened to somebody else’s code.
There is a related lesson that took longer to accept. Early on, a skill that was supposed to write two files kept writing one. I made the instruction more explicit. It still failed. I made it more explicit again and explained why it mattered, in the imperative, at the top of the step. Across five runs it produced the missing file under four different names, and once not at all.
Another skill was supposed to never do a specific destructive thing, and the instruction literally said never do this specific destructive thing. One run in three did it anyway.
So I stopped writing instructions and wrote a script. The script writes both files or neither. The 67% became 8 out of 8. The rule I took from it is one I now use everywhere: a step that has to happen every time, that nothing else will catch if it is skipped, belongs in code rather than in a prompt. Judgment stays with the model and with the person. Bookkeeping does not.
I also learned to publish the numbers that are still bad. One hand-off in the suite works 2 times in 5. It is documented at 2 out of 5 rather than reworded a fourth time until it sounds solved. I tried the rewording. It measured 0 out of 2 and I reverted it, because prose that does not move the number is churn dressed up as progress.
If any of that is interesting, the full thing is written up at Corp-OS: what it is, how it is put together, and the rest of what the evals found. The code and the architecture record are on GitHub.