Eight thousand four hundred.

That’s how many MQL notifications hit our BDR team in a single window one Tuesday morning. None of them were real. No campaign went out. No list got imported. Nobody in marketing did anything at all.

The morning went about how you’d expect. Teams lights up around nine. A BDR manager asks what we launched. We didn’t launch anything. Somebody pulls the MQL count for the day and the line is vertical, which is the kind of chart that makes your stomach drop before your brain catches up.

Then we spent the rest of the day working backwards from a symptom, one guess at a time.

Working backwards is a terrible way to spend a day

If you’ve done this, you know the shape of it. You start at the thing that’s visibly wrong and try to reason your way upstream through a system with too many moving parts. Did a workflow fire? Did an integration push something? Did somebody import a list and forget to tell anyone? Each guess costs twenty minutes to rule out, and ruling something out doesn’t feel like progress even though it is.

We found it eventually. An implementation contractor had made a change in production that they weren’t authorized to make. It touched lead scoring. Every contact that got rescored crossed the MQL threshold, and every one of those crossings fired a notification.

The change itself took about four seconds.

Finding it took most of a day.

That gap is the whole story. Not the change, which was a mistake and mistakes happen. The gap. The hours between something breaking and anybody knowing why, which is time the BDR team spent chasing leads that weren’t leads.

And that’s the part that actually cost something. Pipeline recovers in a day. A BDR team that’s learned your MQL alerts are sometimes garbage takes considerably longer than that, and you pay it back in small increments for months. I’m still paying it back.

The whole time, HubSpot knew

Here’s where I have to be honest about my own part in this.

HubSpot has an audit log. Every one of those changes was in it. Timestamp, actor, email address, the object that changed. The information was not missing. It was not hard to get. There was no gap in the data at all.

It was sitting in a table nobody opens.

Which raises a question I’d invite you to sit with for a second, because it made me feel worse the longer I looked at it:

When did you last open your CRM audit log on a day when nothing was on fire?

For most of us the honest answer is never. We open the audit log after something breaks, to confirm what broke. It’s forensics. It’s the thing you go read to write the postmortem.

I don’t think that makes us bad at our jobs. I think it makes us normal. The log is designed for exactly the use we give it. It’s built so that after an event you can prove what happened and who did it, which is precisely right if you’re an auditor. It just isn’t built to tell you something is happening right now.

Nobody failed here. The tool is doing the job it was built for. It’s not the job I needed done.

Process is what already failed

The thing that changed how I thought about all of this wasn’t the incident. It was the third one.

April. Then May. Then a note from a June one on one where the contractor explained what the API request was and acknowledged they never got approval.

And we had tried process. After the first one we added an approval step. Ask before you touch production. It worked exactly as long as everybody remembered it existed, which turned out to be about six weeks. By the next incident somebody was in a hurry and the approval step was the thing that got skipped, because the approval step is always the thing that gets skipped.

A control that depends on the busiest person remembering it isn’t a control. It’s a hope.

So I stopped trying to prevent the change and started trying to shorten the gap.

It was never a monitoring problem

The reframe that unlocked it: the events are already recorded. I don’t need to detect anything. HubSpot detects it. What I need is triage.

An audit log gives you a record, a timestamp and an actor. What it never gives you is an answer to the only question you actually care about, which is whether this matters.

Here’s a real entry, lightly redacted:

category:        PROPERTY_VALUE
action:          UPDATE
targetObjectId:  40718322901
actingUser:      a****@partner-agency.example

Technically complete. Practically useless. Exactly the same shape whether somebody fixed a typo in a property description or rewired your scoring model. And there are hundreds of these a day in one HubSpot instance, which I know because I went and counted, and it was a higher number than I would have guessed.

A human reading hundreds of rows a day to find the two that matter is not a plan. It’s a slower way of finding out when the pipeline breaks.

So three boxes.

The audit log. You’re not building this part. Your platform already records the events.

A reasoning layer. One model call per entry. Summarize it in a sentence a human reads in two seconds. Rate it high or low against a rubric I wrote. Propose a specific next step.

A triage message. Only the risky ones, pushed into a place the team already looks.

That last one sounds like a detail and it’s the part most people get wrong. If the output lands somewhere new, it dies the same death the audit log did. Ours goes to a Teams channel we’re in all day. Yours might be Slack or email or a ticket queue. The rule is just: don’t make people go anywhere new.

I want to be specific about what the model is doing, because “I pointed AI at it” is doing a lot of work in most write-ups like this. I’m not using a model to detect the change. I’m using it to do the reading I don’t have time to do. The risk rubric is a named list I wrote by hand. Workflow edited or deleted. Workflow created by anyone outside the two named admins. Data pointed at a new external endpoint. Scoring or routing changed. A property definition or pipeline structure changed. Default to low.

I decided what risk means. The model applies it at a volume I can’t read.

The dumb one is the one that works

There are two flows, and the second one has no AI in it at all.

Flow one is what I just described. Flow two counts how many contacts entered the MQL lifecycle stage in the last ten minutes and yells if the number is too high. That’s it. No model, no audit log, no Enterprise tier required.

Flow one catches causes. Flow two catches symptoms.

The reason both exist is that flow one depends on a model correctly judging that a change was risky, and that’s a probabilistic call that will sometimes be wrong. Flow two doesn’t care what caused the spike. It knows what normal volume looks like and it complains when reality doesn’t match.

Put a model in the middle of a system and you’ve handed it a way to be wrong and quiet about it at the same time. So you put something deterministic next to it that fails differently.

If I could only have built one of them, I’d have built the dumb one. It would have caught the original incident on its own.

The column I built and never used

One more thing, because I’d rather tell you the rough part than pretend this is finished.

Every entry, high risk or low, gets written to a SharePoint list before it gets gated. That list is the best decision in the build. It’s queryable, so I can answer who changed what and when without scrolling a channel, and every row carries the model’s rating and its reasoning, which means I can go measure how often it was wrong.

I built a column for that. It’s called Feedback. Two options, Useful or Noise.

It’s blank on all eleven thousand rows.

I built the instrument and never picked it up. So when I tell you the false positive rate feels low, hear that word. Feels. I’m telling you how it seems, not what I measured.

That’s probably the most common failure shape with builds like this, and it isn’t a technical one. Standing up the monitoring is the fun part. Going back and reading it is the part that needs a calendar reminder.

The takeaway

The system that could have shortened my four hour day to a four minute one was already running. It had been running the entire time. Nothing was missing except a path from the place the event was recorded to the place I actually look.

That’s true of almost every platform in your stack right now.

So the question worth asking this week isn’t which governance tool to evaluate. It’s which of your systems would hurt the most if somebody changed it quietly, and whether anything in that system’s history has been read by a human in the last month.

Start there. And start with something you understand well, so you can tell instantly whether the thing got a rating right.

There’s a version of this as a talk at MOPZA26 (MOps-Apalooza Online, October 2026), and the full build notes are up: the flow structure action by action, the actual system prompt with its rubric, the SharePoint schema, and the seven things that ate real hours. If you build one of these, even a somewhat janky first version, I’d like to hear about it.