Who Changed That? Build Notes

This is the actual shape of what I built, written so you can rebuild it rather than copy it. It’s two Power Automate flows that write to the same Teams channel. Nothing here is HubSpot-specific except the first box. If your platform has an audit log and an API, the pattern holds.

Names, keys, IDs and internal references have been genericized. Anywhere you see {{DOUBLE_BRACES}}, that’s yours to fill in.

Contents

1. The shape
2. Flow 1: the change watchdog
3. The prompt
4. The ledger
5. Flow 2: the volume monitor
6. Gotchas that cost me real hours
7. Porting it to Salesforce

1. The shape

Three boxes.

  AUDIT LOG              REASONING LAYER            TRIAGE MESSAGE
  ---------              ---------------            --------------
  Platform already   ->  Summarize, rate,      ->   Post only the risky
  records the event      propose a next step        ones where people
                                                    already look

Everything below is detail on top of those three boxes.

Two flows sit under this, and the split is deliberate:

Flow 1 depends on a model correctly judging that a change was risky. That’s a probabilistic call and sometimes it will be wrong. Flow 2 doesn’t care what caused the spike, it only knows what normal looks like. A deterministic backstop under a probabilistic system.

If you only build one, build Flow 2. It’s simpler, it needs no audit log, and it would have caught the incident that started all this on its own.

2. Flow 1: the change watchdog

Scheduled cloud flow. Runs every 30 minutes.

Actions, in order

Recurrence                    every 30 minutes
Initialize variable           HubSpotKey      (String, secure)
Initialize variable           AzureKey        (String, secure)
Initialize variable           WindowStart     (String)
Initialize variable           AfterCursor     (String, empty)
Scope: TRY
  Compose                     BuildURI
  Do until (AfterCursor empty)
    HTTP                      GET audit log page
    Parse JSON                audit log response
    Apply to each (results)
      Condition               is this a workflow change?
        > HTTP                GET workflow name by ID
        > Compose             TargetName
      HTTP                    POST Azure OpenAI
      Parse JSON              model response
      SharePoint Create item  ledger row
      Condition               risk == HIGH
        > Teams               post to channel
    Set variable              AfterCursor = next page cursor
  Set variable                WindowStart = now
Scope: CATCH  (configure run after: has failed, timed out, skipped)
  Teams                       post failure notice

The window

The flow runs every 30 minutes and looks back 35. Every run overlaps the previous one by five minutes.

occurredAfter = addMinutes(utcNow(), -35)

Without the overlap you lose whatever lands in the seam between runs, and you lose it silently, which is the worst way to lose something. Dedupe on the audit log’s own event ID if the overlap bothers you. It never bothered me.

Building the URL

Do not build the URL inline in the HTTP action. Build it in a Compose first and reference the output. Power Automate throws unhelpful URI validation errors on inline expressions and will not tell you which character it disliked.

Compose: BuildURI

concat(
  'https://api.hubapi.com/account-info/v3/activity/audit-logs?occurredAfter=',
  variables('WindowStart'),
  if(empty(variables('AfterCursor')), '', concat('&after=', variables('AfterCursor')))
)

HTTP action

Method:  GET
URI:     @{outputs('BuildURI')}
Headers: Authorization: Bearer @{variables('HubSpotKey')}

Two things before you go try this:

• This endpoint is Enterprise tier. On Professional it isn’t there. I’d rather tell you now than have you lose an afternoon.
• The scope you need is account-info.security.read.

Use a Service Key, not a legacy Private App token. Service keys rotate without rebuilding the integration and they don’t die when somebody offboards.

Paging

Audit logs paginate. If you only take the first page you will miss changes, and you’ll miss them on exactly the busy days when you most needed the coverage.

The Do until condition is empty(variables('AfterCursor')). At the bottom of the loop, set AfterCursor from the response:

body('Parse_audit_log')?['paging']?['next']?['after']

If there’s no next page that expression returns null, the variable goes empty, and the loop exits. Set a loop limit (count 100, timeout PT1H) so a bad cursor can’t run forever.

Test this specifically. Force a window wide enough to produce more than one page and confirm the cursor actually advances. A loop that silently processes page one and exits looks identical to a loop that worked.

What comes back

Per entry: category, subCategory, action, objectId, objectType, actingUser (with email), occurredAt.

The categories I see in practice: CRM_OBJECT, CONTENT, DATA_PRIVACY_REQUEST, PROPERTY_VALUE, LIST, LOGIN, CRITICAL_ACTION, PROPERTY, APP_ACTIVITY, WORKFLOWS, CRM_OBJECT_ASSOCIATION.

Resolve the ID before the model sees it

This is the most reusable decision in the whole build.

An audit entry tells you an object changed. It gives you a twelve digit object ID. A twelve digit number is meaningless to a human and just as meaningless to a model. If you hand it to the model, the model will either ignore it or make something up.

So before anything reaches the reasoning layer: if the change touched a workflow, make a second call and resolve the ID into a real name.

Method:  GET
URI:     https://api.hubapi.com/automation/v3/workflows/@{items('Apply_to_each')?['objectId']}
Headers: Authorization: Bearer @{variables('HubSpotKey')}

Take body?['name'] and pass that into the prompt. The model never sees a number. It sees Lead Scoring v3.

The principle: look up anything you can look up. Every fact you hand a model deterministically is a fact it can’t get wrong. Most of the AI failures I’ve watched in ops come down to somebody asking a model to guess at something the system already knew.

The model call

Method:  POST
URI:     https://{{YOUR_RESOURCE}}.openai.azure.com/openai/deployments/{{YOUR_DEPLOYMENT}}/chat/completions?api-version={{API_VERSION}}
Headers: Content-Type: application/json
         api-key: @{variables('AzureKey')}

Note the header is api-key, not Authorization: Bearer. Azure OpenAI differs from the OpenAI API here and it will cost you twenty minutes if you don’t know.

You do not need a frontier model for this. It reads one structured record and answers three questions against a rubric you wrote. A small, cheap, fast deployment is the right call, and because the whole thing is an HTTP request, swapping the model later is a config change and not a rebuild. Worth doing on purpose. Your approved AI list next year probably looks different from this year’s.

Body:

{
  "messages": [
    { "role": "system", "content": "{{SYSTEM_PROMPT, see section 3}}" },
    { "role": "user",   "content": "{{THE AUDIT ENTRY AS JSON}}" }
  ],
  "temperature": 0,
  "response_format": { "type": "json_object" }
}

temperature: 0. You want the same entry to get the same rating every time. This is a classifier, not a brainstorm.

Error handling

Wrap the whole thing in a Scope named TRY. Add a second Scope named CATCH, configured to run after TRY has failed, timed out, or been skipped. CATCH posts to the same Teams channel.

Azure goes down sometimes. Models return malformed JSON sometimes. When that happens the Parse JSON step fails, the scope fails, and I get a message saying the watchdog errored and may have missed changes.

A silent watchdog is worse than no watchdog. With no watchdog you know you’re exposed. With a silent one you think you’re covered.

Go break it on purpose once. Put a bad key in, let the flow fail, and confirm the message actually reaches you. Untested error handling is a comment that says it’ll probably be fine.

3. The prompt

This is the part people ask about, and it’s the part that does the least magic. It’s a rubric. I decided what risk means. The model applies it at a volume I don’t have the bandwidth to read.

System prompt

You are a change triage assistant for a HubSpot CRM instance. You read a single
audit log entry and return a structured assessment. You do not have access to any
information beyond the entry provided. Never infer facts that are not in the entry.

Return ONLY valid JSON matching this schema, with no markdown fences and no
commentary:

{
  "summary":   "string, one plain-English sentence, max 25 words",
  "risk":      "HIGH" | "LOW",
  "triage":    "string, one specific action, max 30 words"
}

SUMMARY
Write one sentence a busy person understands in two seconds. Name the actor by
email, name the thing that changed by its name if a name was provided, and say what
happened. Do not include raw object IDs. Do not restate the category enums.

RISK
Default to LOW. Rate HIGH only if the entry matches one of these conditions:

  1. A workflow was edited, deleted, or deactivated.
  2. A workflow was created by anyone other than {{ADMIN_EMAIL_1}} or
     {{ADMIN_EMAIL_2}}.
  3. Data was pointed at a new external endpoint (webhook target changed, new
     integration authorized, export to an unfamiliar destination).
  4. Lead scoring or lead routing logic was changed.
  5. A property definition or a pipeline structure was changed. Property VALUE
     changes on individual records are LOW. Property DEFINITION changes are HIGH.
  6. The acting user's email domain is not {{YOUR_DOMAIN}}.

If none of the above match, the answer is LOW. Do not rate something HIGH because
it feels unusual. Volume of similar changes is not a risk factor. An entry you
cannot classify is LOW.

TRIAGE
Write the next action, not a category of action. "Investigate" is not an answer.
Name who to ask, or what to go look at, or both.

Good:  "Confirm with [email protected] whether the Lead Scoring v3 change was
        approved, then check MQL volume for the last two hours."
Bad:   "Review this change."

For LOW entries, triage should be "No action needed."

User message

Pass the entry as JSON with the resolved name already filled in:

{
  "occurredAt":   "2026-07-23T21:45:11.000Z",
  "actingUser":   "[email protected]",
  "category":     "WORKFLOWS",
  "subCategory":  "WORKFLOW_UPDATED",
  "action":       "UPDATE",
  "objectType":   "WORKFLOW",
  "objectName":   "Lead Scoring v3"
}

Why two risk levels and not five

Five levels means every rating becomes a debate with yourself and nobody trusts the middle three. Two levels forces the rubric to be explicit, which is the actual work. The question isn’t “how risky is this,” it’s “does this get a message or not.”

Tuning

The tuning surface is the rubric, not the model. When it rates something wrong, I don’t reword the prompt’s tone and I don’t swap the model. I add a line to the numbered list, or I add an explicit exclusion. Then the ledger tells me whether it stopped.

That’s the whole loop. It gets one wrong, I add a line, the list tells me if it worked.

4. The ledger

If I were starting over I’d build this first, before the alerting.

Every entry gets written to a SharePoint list before it’s gated. High and low, all of it. One row per change.

Teams is for the moment. The list is for everything that comes after it.

Three things it buys you:

• It’s queryable. You can answer who changed what and when without scrolling a channel.
• It measures the model. Every row carries the rating and the reasoning, so you can go count how often it got one wrong.
• It holds the disagreements. DecisionSource says AI on every row today. Sooner or later a person is going to overrule this thing, and I want that on the record.

A warning from experience: Feedback is the column that makes the second bullet real, and it is blank on every row in my list. I built the instrument and never picked it up. If you build this, put a recurring fifteen minutes on your calendar to go mark rows Useful or Noise. Otherwise your false positive rate is a feeling, not a number.

5. Flow 2: the volume monitor

Scheduled cloud flow. Every 10 minutes. No AI, no audit log, no Enterprise tier.

Recurrence                every 10 minutes
Initialize variable       Threshold (Integer) = {{YOUR_NUMBER}}
Compose                   WindowStartMs
HTTP                      POST contacts search
Parse JSON                response
Condition                 total > Threshold
  > Teams                 post to channel

Compose: WindowStartMs

The Search API wants epoch milliseconds.

div(sub(ticks(addMinutes(utcNow(), -10)), ticks('1970-01-01T00:00:00Z')), 10000)

HTTP

Method:  POST
URI:     https://api.hubapi.com/crm/v3/objects/contacts/search
Headers: Authorization: Bearer @{variables('HubSpotKey')}
         Content-Type: application/json

{
  "filterGroups": [{
    "filters": [{
      "propertyName": "{{HS_DATE_ENTERED_MQL_PROPERTY}}",
      "operator": "GTE",
      "value": "@{outputs('WindowStartMs')}"
    }]
  }],
  "limit": 1
}

limit: 1 on purpose. You want the total off the response, not the records. One row back, and the count is free.

The property name is hs_v2_date_entered_<lifecyclestage_id> and the ID is specific to your portal. Find yours by pulling the contact property list and looking for the hs_v2_date_entered_ prefix, or read it out of a contact record that recently became an MQL.

On the threshold: mine is 200, and 200 isn’t a magic number. It came from our baseline, not from anywhere clever. Pull your own MQL volume for a few normal weeks, find your ten minute peak, and set it comfortably above that. You’d rather start quiet and tighten than start noisy and get ignored.

6. Gotchas that cost me real hours

1. URI whitespace validation errors. Power Automate rejects a URL it doesn’t like and won’t tell you why. Build it in a Compose action first, then reference that output. Fixed it every time.

2. Parse JSON breaking on type mismatches. Real audit data has nulls where your sample data doesn’t. Generate the schema from a sample, then go make every field null-tolerant by hand: "type": ["string", "null"].

3. Conditions failing on type. A number arriving as a string does not compare the way you think it does. Wrap it: int(body('Parse_JSON')?['total']).

4. Teams actions corrupting after an upstream schema change. There is no fix. Delete the action and recreate it. I lost an hour repairing one before rebuilding it in ninety seconds.

5. The new designer removed “edit in advanced mode” for conditions. If you find an older tutorial that says otherwise, you’re not losing your mind. Use the fx expression tab on the operand instead.

6. Export a copy of your flow before you touch it. Power Automate will happily let you save your way into something that no longer runs, and the undo story is worse than you want it to be. It takes fifteen seconds and you’ll need it exactly once, which is enough.

7. Premium connector licensing. HTTP and the Azure OpenAI path need a premium Power Automate license. If you already have one this adds no new line item. If you don’t, price it before you build it.

The tool changes but the failure class doesn’t. Every low-code platform breaks the same three ways: strings that should be numbers, schemas that assume your sample data, and connectors that hold stale state. Build the URL in a variable first, tolerate nulls, rebuild rather than repair.

7. Porting it to Salesforce

I haven’t built this one. It’s a map, not a finished thing.

Four rows change. Three don’t. The three that don’t are the ones that took longest to get right.

Same story for a Marketo audit trail, for your CDP, for anything with an events endpoint. If your platform records who changed what, you can do this.

The question worth asking: which of our systems would hurt the most if somebody changed it quietly? Start there. Though if it’s a system you don’t know well, start somewhere you do, because you need to be able to tell instantly whether the thing got a rating right.

A note on what this is and isn’t

This detects. It does not remediate.

People ask why it doesn’t just undo the change. It knows what changed, it knows who did it, it could roll it back. Here’s why not: the system has no idea what anybody intended. Most of those changes are legitimate work by people doing their jobs. An automation that reverts production config based on a probabilistic risk rating is a far more dangerous thing than the problem I was trying to solve.

Detection that’s wrong costs me two minutes. Remediation that’s wrong costs me a workflow somebody spent a week building.

So it tells me, and I decide.

If you build one of these, even a somewhat janky first version, I’d like to hear about it. What you pointed it at, and whether it caught anything.

[email protected] · jimmypiraino.com · Stack & Signal