Skip to content

Work that comes back every week? No human needed. More on AI and automation

Automation

EspoCRM Webhooks in n8n: The Event Name, the Signature and the Stage That Did Not Exist

Part 3 of 6 in the series The Follow-Up Machine

A brushed metal block hovering above a metal plate that carries a matching recess, light passing through the gap.

This is part 3 of the follow-up machine series. Part 2 covered the architecture. This part covers “Followup 1: Register”, the workflow that starts watching a deal the moment it enters the Proposal or Negotiation stage in EspoCRM.

It is the smallest of the five workflows, 16 nodes. It also produced the most mistakes per node.

The shape of the workflow

Webhook
  → Inject secret (Set)
  → Verify signature (Code)
  → Valid? ──no──→ Respond 401
       └─yes──→ Respond 200
                  → Split batch
                  → Filter stage
                  → Fetch deal (HTTP) ──fail──→ end quietly
                  → Load cadence (Postgres)
                  → Inject deal fields (Set)
                  → Compute due dates (Code)
                  → Upsert watch + steps (Postgres)
                  → Log run (Postgres)

Two decisions shape everything after the signature check. The workflow answers EspoCRM before it does any work. And it tolerates losing the event entirely, because the hourly reconciliation in part 4 will register the deal anyway.

Mistake 1: an event name that is accepted and never fires

The first version subscribed to Opportunity.update.stage. EspoCRM accepted it, saved it and showed it in the webhook list. Then it never sent anything. Two stage changes in the CRM, zero deliveries, while the n8n endpoint answered test requests perfectly.

EspoCRM’s syntax for a field-specific event is {Entity}.fieldUpdate.{field}:

Opportunity.fieldUpdate.stage

Plain Opportunity.update also works, because the workflow filters by stage anyway. What does not work is a plausible-looking name in between, and nothing tells you so.

Mistake 2: choosing the webhook secret yourself

The setup notes said to pick a secret and enter it in EspoCRM. That is backwards. EspoCRM generates a secret key for each webhook and signs every delivery with it. You copy that key from the webhook’s detail view into n8n. A secret you invent never matches, and every delivery gets a 401.

Mistake 3: the signature format

The first signature check computed a base64 HMAC of the body and compared it to the header. The first real delivery failed it. The actual format is different:

  • The header value is base64.
  • Decoded, it is the webhook ID, a colon, then the HMAC-SHA256 of the raw body.
  • Newer EspoCRM versions send a Signature header with the HMAC as a hex string. The older X-Signature header carries the raw digest bytes.
What is inside the signature header?D4 / FOLLOW-UP MACHINEWhat is inside the signature header?Decode the wrapper first. Compare the digest against the exact body bytes. Watchful Loop: Register Header value: Base64 encodedSignature / X-SignatureBase64 decodewebhook ID:0x3aHMAC-SHA256digest of the raw bodyignored by this checkno security value herefirst colon; never part of the webhook IDSignaturenewer headerdigest as hex textX-Signatureolder headerdigest as raw bytesRaw body bytesWebhook: Raw Body ONSet: binary passthrough ONThe production check accepts both digest formats. Decoding Base64 does not verify the signature.
Base64 decoding reveals the webhook ID, the first colon separator and the digest. The digest is checked against the exact body bytes, accepting hex text or raw bytes.

The check that runs in production accepts both:

const crypto = require('crypto');
const item = $input.first();
const secret = item.json.webhook_secret;
const raw = Buffer.from(item.binary.data.data, 'base64'); // raw body from the Webhook node

const invalid = () => [{ json: { signature_valid: false } }];
const header = item.json.headers['signature'] ?? item.json.headers['x-signature'] ?? '';
const decoded = Buffer.from(String(header), 'base64');
const sep = decoded.indexOf(0x3a); // first ':' (webhook ids never contain one)
if (sep < 0) return invalid();
const provided = decoded.subarray(sep + 1);

const hmac = crypto.createHmac('sha256', secret).update(raw);
const rawDigest = hmac.digest();
const hexDigest = Buffer.from(rawDigest.toString('hex'));
const matches = (a, b) => a.length === b.length && crypto.timingSafeEqual(a, b);

if (!matches(provided, hexDigest) && !matches(provided, rawDigest)) return invalid();
return $input.all();

The webhook ID in front of the colon is split off and ignored. It carries no security value: the HMAC alone binds the body to the secret. The comparison uses timingSafeEqual, and checks lengths first because timingSafeEqual throws on buffers of different lengths.

Three settings are required for this node to work at all:

  1. Raw body ON in the Webhook node. The HMAC is over the exact bytes EspoCRM sent, not over n8n’s parsed JSON.
  2. Binary passthrough ON in the Set node in front. A Set node drops binary data by default, and the raw body is binary data.
  3. NODE_FUNCTION_ALLOW_BUILTIN=crypto on the task runner. Without it, require('crypto') fails in a Code node.

n8n 2.x and $env. The secret itself arrives through that Set node, from $env, because Code nodes in n8n 2.x run in a task runner that cannot read $env (see part 2).

An invalid signature routes to a Respond to Webhook node with status 401. It does not trigger an alert.

Anything public on the internet gets scanned, and alerting on scanners teaches you to ignore alerts.

Answer first, work later

The valid path answers 200 before it does anything else. EspoCRM retries a delivery that responds slowly, so a workflow that does its database work first and answers afterwards creates its own duplicate deliveries.

Answering first has a cost. If Postgres is down, the event is lost, because EspoCRM already has its 200 and will not send it again. That is accepted on purpose: the reconciliation pass in part 4 finds every in-scope deal without a watch within the hour.

The payload does not contain the deal

A field-update webhook carries the record ID and the attributes that changed. Usually that means id and stage, and not the deal name or the account. So after filtering by stage, the workflow fetches the deal:

GET {ESPO_BASE_URL}/api/v1/Opportunity/{id}?select=name,stage,accountName

The node is set to three tries, so one call and two retries. If all three fail, processing of that item ends without a warning. Reconciliation will pick the deal up.

One detail needed a redesign of the node chain. The first version combined the fetched deal with the cadence from Postgres using a Merge node, combined by position. That is fine for one deal. When one fetch in a multi-deal delivery fails, the positions shift, and deal N receives deal N+1’s name. The fix was to drop the positional merge and build each item with named-node references in a Set node:

name        = {{ $('Fetch deal').item.json.name }}
cadence     = {{ $('Filter stage').item.json.cadence }}

Paired items stay paired, even when some fail.

Mistake 4: a stage that does not exist

The stage filter kept deals in Proposal/Price Quote or Negotiation. Proposal/Price Quote is the stage name in SugarCRM and SuiteCRM. A stock EspoCRM calls it Proposal.

So every deal that entered the proposal stage, which is the exact case the machine was built for, was dropped by the filter. The same list fed the reconciliation query, so reconciliation missed them too. Nothing failed. Every run was green.

It survived five days of testing because the test deal sat in Negotiation, which was spelled correctly. The fix accepts both spellings:

// EspoCRM's default is 'Proposal'. 'Proposal/Price Quote' is the SugarCRM/SuiteCRM spelling.
export const STAGES_IN_SCOPE = ['Proposal', 'Proposal/Price Quote', 'Negotiation'] as const;

The lesson is not about EspoCRM.

A filter is a silent failure by design: its job is to drop things.

Test every value a filter is meant to keep, not only one of them.

The Postgres node and its parameter string

The last two nodes write to Postgres, and the n8n Postgres node has a trap in how it passes query parameters. The parameters are one string of comma-separated expressions, and the node splits that string itself.

Two consequences hit Register:

Bare literals are dropped. A parameter list like ='wf1', {{ $json.startedAt }}, ... loses the 'wf1', because only {{ }} segments become parameters. Every later parameter shifts left by one. The error was invalid input syntax for timestamptz: 3: the timestamp parameter had received the item count. The fix is to wrap literals in braces too, {{ 'wf1' }}.

null becomes the text 'null'. A JavaScript null in an expression resolves to the string null, and that string lands in the column. Register wrote it into the watch, and it surfaced later in Evaluate: the first dry-run task description read “TEST Followup machine (null)”. The fix is to send an empty string for absent values and turn it back into a real NULL in SQL with nullif(..., '').

The Postgres node parameter string. Wrap literals in {{ }}, send '' instead of null for absent values, pass free text as one JSON object.

There is a third trap in the same place, commas inside values. It hit the AI step first, so it is in part 5. The watch insert now passes its values as one JSON object and unpacks them in SQL, which avoids all three:

with p as (select $1::jsonb as j)
insert into watches (subject_type, subject_id, subject_url, subject_name, account_name, context, cadence)
select j ->> 'subject_type', j ->> 'subject_id', j ->> 'subject_url', j ->> 'subject_name',
       nullif(j ->> 'account_name', ''), nullif(j ->> 'context', ''), j ->> 'cadence'
from p
on conflict (subject_type, subject_id) where status = 'open' do nothing
returning id

What to take from this part

  • In EspoCRM, field events are Entity.fieldUpdate.field. A wrong name is accepted and silent.
  • EspoCRM generates the webhook secret. Copy it, do not choose it.
  • Verify the raw body, accept both signature headers, compare timing-safe.
  • Answer before you process, and make losing the event survivable.
  • Never merge by position when some items can fail.
  • Test every value a filter should keep.

Next: Part 4, Evaluate.