Trust But Verify. Then Verify Again.

The Real Objection

The music world has a long list of concerns about AI, and I understand most of them: originality, attribution, copyright, and the very real fear that human musicianship will get flattened into content. I’m not waving any of that away.

But the concern I keep running into, because of how I am building dotBeat, is more immediate. AI can be wrong about music theory with total confidence, and those mistakes can quietly become part of the thing you are building.

The problem is how ordinary the wrong answer can look. It does not always arrive with caveats or visible uncertainty. Sometimes it just shows up as a tidy answer, written as if there is nothing to question.

The first time it really landed, I was asking how to play a slash chord on a standard re-entrant ukulele. That is not quite the same as constructing a chord on paper. A slash chord isn’t just about having the right notes. The note after the slash is supposed to be in the bass, and that gets complicated on a re-entrant ukulele.

The agent gave me a tidy answer anyway. It treated the ukulele as if the fretboard geography would obediently line up with the symbol, then started inventing facts to make the explanation work. At one point it told me the third fret of the C string was C, which is basic-fretboard wrong. Then it treated an Am7 shape as if that automatically put A in the bass, which misses the whole re-entrant problem. Then it offered a shape and named notes that were not the notes the frets actually produced. I had to call it out three times before it stopped trying to save the answer.

That was the moment the relationship changed. Not because AI became useless. It didn’t. But I understood what I was actually dealing with: a very fast research assistant with no reliable sense of where its own knowledge ends.

This is also where some people stop the conversation. For them, the problem is not that AI needs checking; the problem is that I am using AI to help build a music learning tool at all.

I understand the discomfort. If a tool is meant to teach beginners, the source of its musical information matters, and so do accuracy, judgment, and accountability.

But I don’t think “AI was involved” is the same thing as “AI is the authority.”

Where the Mistakes Hide

In web development, a wrong answer often reveals itself pretty quickly. The page throws an error, the layout looks wrong, or a button that worked yesterday stops doing anything useful. There is usually some visible consequence.

Music theory doesn’t always do that. A wrong note, a bad fretboard assumption, a mislabeled inversion, or a chord name that almost makes sense can sit there looking perfectly plausible. If I do not already know enough to catch it, it can go live on the site and stay there until someone who does know catches it later, which is an unpleasant thought when you are building something for beginners.

It also doesn’t only happen when I ask a direct theory question. Agents generate data. They write rules. They interpret existing data. They make decisions based on musical assumptions buried two or three layers down in the work.

Once AI becomes part of the build process, one bad assumption can get copied into a lot of places very quickly.

How I Check

When I’ve talked publicly about AI getting music theory wrong, I haven’t been surprised that people object to using AI this way. I get why it makes people uneasy. What has surprised me is how brutal some of the comments can be. A civil discussion is welcome, but it does make you think twice before posting anything AI-related.

I’m not handing an AI a blank website and saying, “Teach people music theory for me.” I’m asking questions, checking answers, trying things on an instrument, listening to theory lessons, finding contradictions, and figuring out why they exist. That feels like learning to me.

A lot of that learning has been audio: Music Student 101, Basics of Classical Harmony & Counterpoint by Seth Monahan, The Harmonious Blacksmith, and Basic Music Theory by Jonathan Harnum. I listen whenever I have a spare pocket of time, which has been excellent news for the dog because more theory means more walks.

Before AI, I could have misunderstood something in a book, found a bad answer on a forum, watched a YouTube video from someone confidently explaining it incorrectly, or simply remembered something wrong. The need to evaluate a source didn’t suddenly appear with AI.

What is different is the scale and the confidence. AI can produce a lot of information very quickly, and it can present a wrong answer in exactly the same tone as a correct one.

So my process now is basic, but strict: check the theory against reliable sources, and test it on the actual instrument.

If a chord is supposed to contain certain notes, I find them on the fretboard. If something sounds wrong when I play it, that counts. If two sources disagree, I don’t pick the answer I like better. I keep digging until I understand why they are different.

Sometimes the AI was wrong, sometimes I was wrong, and sometimes the simple question turned out not to be simple at all.

What It Taught Me

The strange part is that all this checking has made the tool more useful to me, not less.

The tool I learned not to trust blindly ended up teaching me the most, not because its answers were reliable, but because checking its answers forced me to actually learn the material.

Some of my most useful learning has started with an AI answer that made me think, “That cannot be right.” Then I had to find out why.

I can’t just ask, “What notes are in this chord?” anymore. I have to know why those notes are in the chord. I have to know enough to recognize when an answer doesn’t make sense.

Increasingly, when I ask an AI agent to change something on dotBeat, I’m not just checking whether the code works. I’m checking whether the musical assumption underneath the code is right.

That is a very different kind of testing, and I’m still anxious about it. I probably should be. But the answer, for me, is not to pretend AI was never involved. The answer is to be clear about the process, own the responsibility, and keep verifying.

If you spot something wrong, please tell me. That is not a disclaimer; it is a genuine request.