Apex Climate Solutions — As-built (technical)
Apex Climate Solutions is a demonstration automation system for a simulated HVAC (heating and cooling) service company, built on Make.com with three Postgres databases on Neon, Slack as the staff workspace, and eight single-call AI steps with fixed reply formats plus a model that indexes the company documents for search. This document records the live build as read on October 28, 2026, updated with the fixes made through November 2, 2026; where it differs from the project charter, the difference is listed in section 6. Before publishing, it was checked for accuracy and readability in separate review sessions that had no part in writing it.
Contents.
- At a glance: one request, start to finish
- When something fails
- 1. Architecture as delivered
- 2. Production scenarios
- 3. Data contracts between scenarios
- 4. Schema: tables and database functions
- 5. Tests and results
- 6. Charter versus built
- 7. Known limits
- Appendix A. Production scenarios, one block each
- Appendix B. Reference lists
- Appendix C. When something fails: full table
- Glossary
How to read it. The main body, At a glance, When something fails and sections 1 to 7, takes about 20 minutes; Appendix A holds one short block per production scenario, Appendix B the full reference lists and Appendix C the full failure table. Terms are defined in the Glossary at the end.
What is deliberately not here. Web addresses, account and resource identifiers, and where credentials are kept, all held in a private access register outside this document; the company, its customers, documents and messages are simulated.
At a glance: one request, start to finish
A customer's request passes about ten fixed steps and up to three AI calls before a person sees it, and a person makes the decisions that commit the company: dispatch, stock changes and public review replies. Three things reach a customer before a person sees the request: the phone agent's live conversation, which follows a set script to gather the caller's details; the WhatsApp acknowledgment; and the form confirmation email. After that, each request is recorded and checked by fixed rules; an unknown sender's message also passes the junk gate; real requests are rated and posted to the matching Slack channel for a person to triage. The steps below follow one WhatsApp message; phone calls and web forms take the same path after their first step.
- It arrives. Meta posts the message to a webhook. The scenario checks the message's signature, which proves it came from Meta, and refuses it if the check fails.
- It is recorded. Before any judgment, the message gets a database record. That record is also the duplicate check: if the same message arrives twice, the second copy stops here. It creates a correlation ID, one label that ties together every audit row about this request.
- Fixed rules screen it. The pre-filter stops a message with no sender, one that is empty or over 2,000 characters, one from a blocked sender, or a sender's 12th message within 2 minutes (a flood). Depending on the reason, a stopped message is either held for a person in the Slack channel #normal-triage or stopped with one alert.
- The customer is looked up by an exact match on their WhatsApp number, phone number or email. A known customer skips the next step.
- The junk gate (AI call 1) decides whether an unknown sender's message is a real service request. It calls something junk only when it is at least 80% sure. Junk is marked low-attention (probably not a service request) and goes no further, with a 120-character snippet and a link to its run. A daily digest at 08:00 lists every parked message in #normal-triage until a person ticks it reviewed. A message that uses one of a fixed list of emergency words (gas, smoke, burning or burned, fire, flames, propane, rotten eggs, carbon monoxide, sparks, an explosion, a boom, a bang or banging, sulfur, an electrical smell, scorched or a scorching smell, no heat or heat that has stopped working, and the Spanish words for these) is never parked, whatever the junk gate says: it carries on as real, and the audit log records the override.
- Extraction (AI call 2) pulls out the equipment, the symptoms and any name or address. A fixed database step then matches the equipment and suggests possible customers. A suggestion is never applied automatically.
- The urgency rater (AI call 3) rates the request from 1 (routine) to 5 (emergency). A fixed safety rule follows: on a phone call where the caller reports a gas or burning smell, the request is Level 5 whatever the AI said.
- A format check confirms every AI reply has the expected shape before anything is saved.
- The job is written. A repeat contact from the same person joins their open job instead of creating a second one, and joining can only raise the urgency, never lower it. The job gets its own Slack thread.
- It is posted for dispatch. Level 5 goes to the Slack channel #urgent, tagged "Uncorroborated" when no customer was matched. Levels 1 to 4 go to #normal-triage. The triage deadline is 15 minutes for Level 5 and 4 hours otherwise. A web-form customer also gets a confirmation email.
The other two parts. Intake and triage is one of three parts (C1). The knowledge base (C2) answers technicians' questions in the Slack channel #tech-help from company documents, citing them. Operations (C3) tracks parts and truck stock from technicians' Slack posts, tells a technician when a job is assigned, drafts replies to customer reviews for a person to approve, and runs a watchdog that flags neglected jobs. Each scenario runs when its trigger fires: a message arriving, a schedule, or another scenario calling it.
Where people decide
- The dispatcher reads each posted job, decides what its urgency means, and assigns a technician in Directus, the office screen. The urgency rating is advice only.
- A new customer starts as pending. The triager confirms them in Directus, and only then are they matched automatically to later messages.
- The technician confirms each job's status, and any parts used, by reacting in the job's Slack thread. Every job thread lives in the Slack channel #parts-used. Stock changes only after that confirmation.
- A reviewer approves every drafted reply to a customer review before anything is posted. Yelp replies are posted by hand.
- An administrator loads company documents by posting them in the private Slack channel #knowledge-docs. Nothing else is meant to change what the knowledge base contains; the loader refuses any request without Slack's signature.
When something fails
Most failures take a safe default or post where a person will see them, and the few that do not are named below as accepted limits. On October 31 a sweep of every production scenario for failures nobody would hear about found the gaps. The fixes were built the same day and passed independent tests on test copies, except three rare failure paths that were never forced to fail in testing: the low-attention digest failing to post one of its items, the phone-call branch of the intake alert for a failed signature step, and a form heartbeat checker alert while Slack is down. They are rare, off the normal path and low impact on customers or the business, so they are accepted limits.
Where alerts go. Operational alerts go to #system-health. Most failures that could cost a customer request go to #urgent, where the dispatcher is watching. One Slack connection carries every alert, so an email to the project owner backs up the dispatch posts, the low-attention digest, a form-heartbeat alert, the missed-call notice and the flood alert.
Main failure categories:
- Before a job exists: a failed intake secret read, signature step or parse, database read, hand-off, format check, job write or urgency rating posts the request to #urgent for a person. A failed junk gate or extraction lets the request carry on.
- AI calls in the knowledge base, stock, parts and review steps: one retry, then an alert, plus a "try again" where a technician is waiting.
- Forged or unsigned webhook requests: refused, answered "OK" and named in #system-health.
- Silent, accepted: a parts or review reply whose retries run out, and a few rare drops, tell nobody.
Appendix C holds the full failure table, the October 31 fixes, the accepted limits with what each costs, and the behavior chosen by design.
1. Architecture as delivered
27 production scenarios on Make.com carry three components; each runs when its trigger fires, and the C3 weekly report is built but not scheduled. Everything enters at a small number of edges, is handed between scenarios inside Make by subscenario call, reads and writes three Neon databases, and leaves as a Slack post, a WhatsApp reply or a confirmation email. AI is used wherever a step needs judgment: each AI step does one job, returns a fixed format, and is checked before anything uses it. Everything around those steps is fixed rules (filters, routers, database functions and schedules), and a person decides every dispatch, stock change and public reply. The hosted phone agent follows a set script to gather a caller's details.
Three fixed pipelines draw on shared stores, with AI only where judgment is needed
- Component or shared group
- Fixed rule
- AI call
- Store
C1 runs left to right as one request's path; the shared #tech-help sorter hands stock questions down to C3; each part uses the shared stores it needs.
How work enters
| Edge | What starts it | Scenario |
|---|---|---|
| WhatsApp message | Meta posts to a webhook | C1 WhatsApp intake |
| Phone call | Retell, the hosted voice agent, posts after the call is analyzed | C1 phone and form intake |
| Web form | A Google Apps Script posts each submission | C1 phone and form intake (same webhook). The script's daily check-in goes to its own webhook: C1 form heartbeat checker (added Oct 29) |
| Document upload | A custom Slack app forwards #knowledge-docs events to a webhook | C2 document loader |
| Dispatcher assigns or cancels a job | A Directus Flow posts to a webhook | C3 job assignment notice |
| Technician or reviewer posts or reacts in Slack | Slack listeners on a person's Slack user connection | #tech-help question sorter, C3 parts used and C3 parts confirm (technicians); C3 review approval (reviewers); C1 low-attention digest tick (dispatchers, added Oct 29) |
| Clock | Make schedules | C1 Meta token daily check, C1 silent-message daily digest, C1 low-attention digest (added Oct 29), C3 job watchdog, C3 weekly report (not scheduled), C1 form heartbeat watcher (daily, added Oct 29) |
| By hand | Run on demand with a mock review | C3 review drafter |
The WhatsApp, phone-and-form, job-assignment, document-loader and form-heartbeat webhooks check a signature or a shared secret before doing anything, and refuse and alert on a failure.
The three components
- C1, intake and triage. Both intake scenarios normalize their channel into one request shape and hand it to the request pipeline front. The front records the request (the duplicate check), runs the pre-filter, handles WhatsApp acknowledgments, then calls intake resolution (customer lookup, the junk gate, AI extraction, equipment and customer matching), then the urgency rater, then a fixed safety rule, a schema check, the job record, the job's Slack thread (open job thread) and the dispatch post. Form requests also get a confirmation email.
- C2, knowledge base. The document loader turns an uploaded PDF into searchable chunks and swaps the live version in one transaction (a set of database changes that all succeed or all fail together). The question answerer retrieves passages and returns an answer that must cite them. Questions reach it through the #tech-help question sorter, a shared step that C2 and C3 both use: it sends every technician question either to C3's stock lookup (nine fixed phrases, no AI) or to the question answerer.
- C3, operations watchdog and review monitor. Parts used reads a technician's parts report and posts a read-back; parts confirm, undo and discard acts on the reaction. The truck stock lookup answers stock questions. Job assignment notice and send assignment notice tag the technician when a dispatcher assigns a job; the technician's reaction updates the job. The review drafter drafts a reply to a review and review approval acts only on a person's reaction. The job watchdog finds neglected jobs, low stock and overdue items in one database call and posts a digest and reminders, using record sweep alert to count each reminder.
Shared stores
- Neon: a knowledge corpus database, an append-only audit log database and a CRM database (section 4). Directus, an admin screen, sits on the CRM for dispatchers.
- Make data stores: shared settings, including the Directus address used in record links.
- Slack: seven channels. #urgent, #normal-triage, #tech-help, #parts-used (one thread per job), #review-approvals, #system-health and the private #knowledge-docs.
The AI steps
| Step | Scenario | Model | Job |
|---|---|---|---|
| Junk gate | C1 junk gate | Anthropic Claude Haiku 5.5 | Sort a real service request from junk; junk only at confidence 0.8 or higher |
| Extraction | C1 intake resolution | Anthropic Claude Haiku 4.5 | Pull equipment, symptoms, name and address fragments as JSON |
| Urgency rater | C1 urgency rater | Anthropic Claude Opus 5.5, prompt v5 | Rate urgency 1 to 5 against a written rubric |
| Embeddings | C2 document loader, C2 question answerer | Google gemini-embedding-001, 1,536 dimensions | Turn each chunk and each question into an embedding, a list of numbers that lets text be searched by meaning |
| Grounded answer | C2 question answerer | Google Gemini 3.5 Flash Lite, prompt v5 | Answer only from retrieved passages, citing their IDs |
| Parts extraction | C3 parts used | Google Gemini 3.5 Flash Lite, prompt v2 | Read part, equipment type and quantity from a technician's post |
| Part lookup | C3 truck stock lookup | Google Gemini 3.5 Flash Lite, prompt v1 | Read which part a stock question asks about |
| Review draft | C3 review drafter | Google Gemini 3.5 Flash Lite, prompt v2 | Sentiment, draft reply and a note for the approver, in one call |
| Ranking | C3 job watchdog | Google Gemini 3.5 Flash Lite, prompt v1 | Rank flagged jobs, only when two or more are flagged |
The three Anthropic calls carry a prompt-cache marker. Structured replies in C1, C2 and the watchdog pass a code-fence strip (a step that removes stray formatting marks the AI sometimes adds) and a JSON parse before anything reads them. What happens when each step fails is in Appendix C.
2. Production scenarios
27 production scenarios: 13 in C1, 2 in C2, 8 in C3 and 4 shared helpers. Each runs when its trigger fires: a message arriving, a schedule, or another scenario calling it; the review drafter is started by hand, and the weekly report is built but not scheduled. Appendix B1 lists each one with its trigger, and Appendix A has one short block per scenario. (Four of the C1 scenarios were added on October 29: the low-attention digest and its tick listener, and the form heartbeat checker and watcher.)
The sorter and the three shared steps are labeled INFRA or Shared in Make because more than one component uses them.
3. Data contracts between scenarios
Every internal hand-off is a Make subscenario call with typed, named inputs, so the called scenario's input list is the contract; most callees also return a small set of named outputs that the caller routes on. Appendix B2 lists each call's inputs and outputs, as read from the live blueprints (a blueprint is the exported definition of a scenario: its steps and settings). A caller that waits does so until the callee finishes; the form confirmation email is the only call made without waiting.
Contract rules that hold across the table.
- One request shape for C1. WhatsApp, voice and form are reduced to the same input list before the pipeline front. Voice adds the two safety answers from the Retell call (gas or burning smell, water leak); WhatsApp adds message_type.
- The correlation ID is created at the duplicate-check insert and carried in every audit row, so one query rebuilds a request's whole path.
- AI outputs are parsed against a fixed shape before use. A reply that does not parse becomes a known failure value (rater_error, gate_failed, extraction_outcome) rather than reaching a database write.
- A 503 code and "try again" is the C2 and C3 callees' shared failure signal back to the sorter, which then posts nothing of its own. 503 is the standard web code for "service unavailable".
- test_prefix inputs on the shared steps let test runs label their posts; production calls leave them blank.
4. Schema: tables and database functions
Three separate Postgres databases on Neon, all in one US East region on Postgres 18, hold every record; most business rules that must hold no matter which scenario calls them live inside the database as stored functions. Read live on October 28, 2026. Row counts are demo data and change with testing.
Knowledge corpus database
Holds the three simulated company documents, split into chunks, each with an embedding (1,536 numbers per chunk, searched through the pgvector extension). No customer data.
| Table | Holds | Rows | Rule that matters |
|---|---|---|---|
| Documents | One row per company document | 3 | Type is diagnostic guide, permit code index or warranty policy |
| Loads | One load of a document | 7 (3 active, 4 superseded) | At most one active load per document |
| Chunks | Text chunks and their embeddings | 390 (209 live) | A saved view shows only chunks from active loads |
- The load function loads a new version of a document and makes it live in one transaction, so the old version stays live if anything fails. It refuses a future Effective date (Panama time), skips the load when the file is unchanged (same content fingerprint), and locks per document so two loads cannot race. Only the loader login can call it, and that login has no direct table rights.
- The search function returns the 1 to 20 live chunks closest in meaning to a question, filtered by document type, zone and effective date. It applies no similarity floor (a minimum closeness in meaning for a chunk to count); the 0.72 floor is applied in the question answerer scenario.
Audit log database
One table, the event log: one row for each main C1 step except the duplicate check (Appendix B6, item 3), plus selected C2 and C3 outcomes (section 6, item 4), about 1,666 rows today. Columns record time, a correlation ID that ties all rows of one request together, component, event type, decision, outcome, model, urgency level and a JSON payload. Senders are stored as a hash and no raw message text is written.
- Append-only by trigger. Two triggers refuse every update, delete and truncate on the table, for every login, including the table's owner. The Neon account holder can still switch the triggers off, so the log is protected against changed history but is not tamper-proof (section 7).
CRM database
Customers, jobs, stock and reviews, all mock data marked as such, plus Directus, an admin screen generated from the database for dispatchers.
| Area | Tables | Rule that matters |
|---|---|---|
| Intake | incoming messages, WhatsApp sessions, contact attempts, blocked senders | Channel plus the channel's own event ID is unique, which is the duplicate check. One open WhatsApp session per chat |
| Customers and jobs | customers, equipment, jobs, job threads | A new customer is created pending and is matched only once confirmed. One Slack thread per job. Status, urgency and permit values come from fixed lists |
| Stock | technicians, trucks, parts, truck stock, pending stock updates, pending part choices, usage log | Stock cannot go below zero. A usage row can be reversed once, by a negative row that names it |
| Reviews | reviews, pending review drafts | One draft per review; a review's platform plus its ID is unique, which is its duplicate check |
| Watchdog | digest state | A single row holding the last digest's fingerprint, so an unchanged digest is not reposted for 4 hours |
| History | row history | A copy of the old row on every update to customers and jobs. Append-only by trigger |
Functions. About 40 stored functions hold the business rules: the intake checks and job creation, stock changes that only the reporting technician can confirm, review drafts that two reactions cannot both claim, and the watchdog's findings. Appendix B3 lists them by purpose, with every table name.
Logins and least privilege
Each scenario connects with its own database login that can touch only what its job needs. The CRM has nine working logins: one for C1 intake, seven for C3 work (assignment, lookup, stock, review, sweep, watchdog and rollup) and one for Directus. No Make or Directus login holds delete or truncate on any business table; a customer is ended by setting status to declined. Directus may edit only listed columns (for jobs: status, assigned technician and last technician update). The audit log's write login is the one exception to least privilege: Neon made it a member of its admin group and that cannot be removed with SQL, so the append-only triggers are what protect the log (accepted demo limit, section 7).
5. Tests and results
Three AI steps were formally evaluated, and most main paths were tested on the live build (the watchdog on a database copy). The AI evaluations, with their pass rules, version histories and limits, are on a separate eval report page and are not repeated here. In short: the urgency rater scored 66 of 66 on its development set and 90 of 90 on new everyday messages; the junk gate marked 0 of 26 real requests junk, in 3 runs each; the grounded answer passed every required check on prompt v5.
Appendix B4 lists the 19 test areas.
What the tests do not cover. No live customer traffic has run through the system. Five AI steps have no formal evaluation: extraction, the review drafter, watchdog ranking, parts extraction and part lookup. Extraction is checked by the fixed resolution step after it, and every review draft waits for a person. Several fixes from the pre-demo review of every scenario and database function were built and run in the same session, so they are builder tests, not independent scores. The job watchdog's current version, changed on October 25 to 26, has not been tested in a Make run; its database logic was tested on a throwaway copy. The weekly report is not scheduled and was not re-tested after its review findings.
6. Charter versus built
The live build matches the charter's design in its structure and in most decisions checked. The October 28 read found 11 details where the record differed from the build, all now reconciled, and 7 behaviors the charter did not describe, 6 of them now resolved. "Charter" means the working charter unless the public charter is named.
Record differences (items 1 to 11). All eleven are reconciled in both charters, and Appendix B6 lists them. The three that mattered most: the record did not show that the Meta token check runs every 24 hours, and the public charter overstated both audit coverage and retries.
Behaviors the charter did not describe (items 12 to 18). All but item 15, whose effect is low, have been resolved: follow-ups to closed jobs (item 12), the document loader's sender check (item 16) and small gaps (items 13, 14, 17 and 18).
Behaviors seen in the live read that the charter did not describe
| # | Where | What the read showed | Status |
|---|---|---|---|
| 12 | C1 request pipeline front | The step that posts jobs to dispatch has no route for a message added to a recently closed job, and its name still says it has one | Resolved and independently re-checked on Oct 29, 2026: no route is needed. Since Oct 23, 2026, a closed job is never reopened, so a follow-up starts a new job and posts to dispatch through the normal Level routes. The grace labels left in the scenario are unused |
| 13 | C1 request pipeline front | Only the WhatsApp reply path checks that a message has at least one letter or digit. An emoji-only message can still reach the AI calls and create a job | Confirmed and fixed Oct 29, 2026: the database pre-filter now holds any message with no letter or digit on every channel. It goes to the pre-filter review post and creates no job. One emoji-only test per channel. Fixed and re-checked |
| 14 | C1 job-record function | Reusing a form customer compares email with case, while the customer lookup ignores case, so a differently capitalized address could create a second pending customer | Confirmed and fixed Oct 29, 2026: the reuse now ignores case, matching the lookup. A test copy reproduced the duplicate with the old version; the fixed version reused the customer. Fixed and re-checked |
| 15 | C2 search date | The search function dates "today" in GMT; the loader dates it in Panama time. The two disagree from 19:00 to midnight Panama | Low effect, since the loader refuses future dates |
| 16 | C2 document loader | This read did not find a signature or secret check on the loader's webhook, unlike the other three webhooks | Confirmed missing. A Slack signature check was added Oct 29, 2026: forged, unsigned and replayed requests refused, a real upload accepted. An independent check of the live version on Oct 29 got the same results |
| 17 | C3 job assignment notice | The "technician set" and "job cancelled" routes can both pass on one edit, which would post two notices | Fixed Oct 29, 2026: the technician route now also requires that the same edit does not cancel the job, so an edit that does both posts only the cancellation notice. Test copy run with three edits. Fixed and re-checked |
| 18 | Gemini calls | The question answerer, parts used, truck stock lookup and review drafter set no output length limit; only the watchdog does (1,500) | Fixed Oct 29, 2026: question answerer and review drafter 1,024 tokens, parts used and truck stock lookup 512, on both tries of each. The answerer's 22-case test set was re-run and scored in a separate session, and a separate session re-checked all four live |
7. Known limits
Every limit the charter and the pre-demo review accepted for the demo still stands except two fixed on Oct 29; the other 9 of the 11 accepted review limits, and both review-publishing items, were confirmed as described in the live read. "Confirmed live" means the October 28 read of the blueprints or database matched the accepted description.
Designed or declined, not built
| Item | As built | Status |
|---|---|---|
| Review intake from Google, Instagram and Yelp | No polling scenario exists. Reviews enter the drafter by hand as mock reviews, labeled as such | Design only. Confirmed live |
| Publishing approved replies to Google and Instagram | A tick replies that publishing is not connected and the draft stays waiting. Yelp is handed to a person | Design only. Confirmed live |
| Push to a field-service vendor (Jobber; Housecall Pro dropped) | No scenario exists | Design only |
| WhatsApp calling | Retell web calls are the only voice channel; no phone number is rented | Design only |
| Retention-locked audit copy (a write-once store) | Not built. The audit log is append-only by trigger, but the account holder can still change it | Design only. The log is not tamper-proof |
| Dead-letter queue with a replay path | Not built. A permanent send failure alerts #system-health and a person resends by hand from the job record | Declined for the demo |
| Detection that the Google Form trigger stopped | Built Oct 29 as its own daily check. The form script checks in daily with its entries from the last 48 hours. No check-in for over 26 hours, or an entry that never reached Apex, alerts #system-health | Built. Independent check passed Oct 29; the scheduled daily check-in ran on its own Oct 31 |
| Detection of a document revised outside the system | Not built. A file is seen only when posted in #knowledge-docs | Declined |
| Multi-region hosting and failover | Not built. Every store is in one Neon region | Declined |
| Changing a model by configuration | Not possible. The model name and prompt are typed into each AI step | Accepted; changing a model is a one-field edit per step |
Accepted demo limits. Eleven smaller limits from the pre-demo review were accepted for the demo, and this read confirmed most of them as described. Two examples: a resent parts report can deduct stock twice, and the "other truck" in a stock reply is the one with the most stock, not the nearest. Appendix B5 lists all eleven with their status.
Limits accepted after the silent-failure sweep. On October 31 the project owner accepted the limits found by the sweep of every production scenario, after the fixes that followed it. On November 1 they were re-sorted into 17 accepted limits, each with what it costs, and 6 behaviors kept by design; both tables are in Appendix C. The largest: one Slack connection carries every alert, nothing watches the monitors themselves, and a C3 reply whose retries run out tells nobody.
About 30 smaller review items were also accepted together on October 25, 2026.
Appendix A. Production scenarios, one block each
One block per production scenario, read from its live blueprint on Oct 28, 2026. "Break" means Make stores the failed run and retries it later, but only in a scenario that starts itself: in a scenario another scenario calls, Make stores no failed run, so a Break there retries nothing (tested Oct 29); "Ignore" and "Resume" mean the run carries on. Every alert named below goes to #system-health in the six-line format unless another channel is given.
C1 WhatsApp intake
- Trigger: webhook from Meta, up to 100 runs a minute; runs may overlap.
- Steps: answers Meta's setup handshake; drops sent, delivered and read receipts right after the trigger; reads Meta's app secret and checks the message signature; for a customer message, opens or reuses the WhatsApp session (open_or_get_whatsapp_session) and hands off to the request pipeline front, waiting for it.
- Returns to Meta: the default success reply, only after the pipeline finishes (accepted limit).
- Errors and alerts: Ignore after each alert. #urgent if the secret read, a signature step, the parse, the session or the hand-off fails; since Oct 31 that post names the sender and message, or carries the raw body when no sender can be read. #system-health for a refused signature, failed delivery, template notices and unrecognized events.
C1 phone and form intake
- Trigger: one webhook shared by Retell and the Google Form script; stored incomplete runs on.
- Steps: drops Retell's call-started and call-ended events; checks Retell's signature (with a 5-minute age limit) or the form's secret header; parses and answers right away; then routes four ways: a completed call with content is handed to the pipeline front with the two safety answers, a complete form is handed off, a silent call with a number posts a missed-call notice to #normal-triage, and an unrecognized request alerts. A silent call with no number stops with no post.
- Errors and alerts: Ignore on the secret reads, signature steps, parse and hand-offs. Since Oct 31, a secret-read, signature-step or parse failure posts the request to #urgent with the caller or form details; raw call data is never posted. #urgent alerts for hand-off failures. A failed missed-call notice is not retried: it emails the project owner and alerts #system-health with the caller's number and call id, then the run ends cleanly. An incomplete form response goes to the unrecognized-request alert, which cannot identify the customer (accepted limit).
C1 request pipeline front
- Trigger: called by the two intake scenarios; stored incomplete runs on, though Make stores none for a called run.
- Steps: duplicate-check insert (insert_inbound_event), which creates the correlation ID; a filter that stops a repeat delivery; pre-filter (prefilter_check); then one route per verdict. Sender flood (a sender's 12th message within 2 minutes, since Oct 31): one #urgent alert naming the sender, their latest message and their open job with a record link, or "none"; later messages from that sender are dropped. Held message: review post in #normal-triage. WhatsApp acknowledgments: counts recent replies from the audit log and replies by the message's place in the session, up to the ceiling; a message with no text gets one "please type it" reply per 2 hours. Pass: calls intake resolution, then the urgency rater; applies the voice safety floor (gas or burning smell forces Level 5); checks the schema; writes the job (create_job_record) and an audit row; calls open job thread; posts Level 5 to #urgent (tagged Uncorroborated with no matched customer) and Levels 1 to 4 to #normal-triage; for a form, calls the confirmation email without waiting.
- Errors and alerts: a rater error alerts #urgent and creates no job. Break, 3 tries 15 minutes apart, on the pre-filter review post, which cannot retry here because this scenario is always called; the intake's #urgent fallback fires at once instead and names the request (tested Oct 31); on the two dispatch posts, an email and a #system-health alert, then the run ends cleanly; Resume after audit, thread, email and settings failures; Ignore elsewhere. #urgent for read, hand-off, rater, schema and job-record failures; #system-health for send failures and audit-write failures. If the flood alert to #urgent fails, an email and a #system-health alert go out (Oct 31).
C1 intake resolution
- Trigger: called by the pipeline front.
- Steps: exact customer lookup (lookup_customer) and audit row; calls the junk gate and writes an audit row; a fixed emergency word check (emergency_keyword_hit) runs next, and a junk verdict on a message naming an emergency is overridden: one audit row, then it carries on to extraction; any other junk verdict marks the request low-attention, saves a 120-character snippet and the run link, and stops (if that save fails or changes no row, the request is posted to #normal-triage instead); otherwise the extraction call, fence strip, parse and audit row; then resolve_intake (equipment match and fuzzy customer suggestions) and audit row.
- AI: Anthropic Claude Haiku 4.5, 1,024-token limit, 10-second timeout, one attempt; cache marker set, prompt too short to cache.
- Errors and alerts: Resume throughout. A failed gate counts as real; a failed emergency word check counts as an emergency, so nothing is parked, and on a junk verdict alerts #system-health at most once an hour (a Make data store holds the last alert time); a failed extraction leaves every field unresolved. #urgent if the lookup or resolution fails; since Oct 31 those two alerts say to check Directus and create the job by hand, because a replay will not rebuild it. #system-health if an audit write fails.
C1 junk gate
- Trigger: called by intake resolution.
- Steps: a known customer returns "bypassed" with no model call; otherwise the model call, fence strip, first JSON object, parse, verdict. Junk only when the model says not real with confidence 0.8 or higher.
- AI: Anthropic Claude Haiku 5.5, prompt v2, 1,024-token limit, 10-second timeout; 1-hour prompt cache.
- Errors: any failure returns values that send the message on as real. No alert of its own.
C1 urgency rater
- Trigger: called by the pipeline front with the message and its resolved context.
- Steps: model call, fence strip, parse, return a whole number 1 to 5. Advisory only.
- AI: Anthropic Claude Opus 5.5, prompt v5, 1,024-token limit, 15-second timeout; 1-hour prompt cache.
- Errors: any failure returns rater_error true; the pipeline front alerts. No retry.
C1 form confirmation email
- Trigger: called by the pipeline front, without waiting; runs one at a time; stored incomplete runs on.
- Steps: only for a form request with an email and a new or bundled job; counts confirmations sent to the same hashed address in the last 30 minutes, and sends nothing if that check cannot be read; sends the email through Gmail if the count is zero; writes an audit row.
- Errors and alerts: temporary failures alert and Break, 3 tries 15 minutes apart; a permanent failure writes an audit row with the address redacted, alerts and stops.
C1 silent-message daily digest
- Trigger: every 24 hours.
- Steps: totals the day's silent messages (audit.silent_digest) and posts to #normal-triage only if there were any.
- Errors: a read failure alerts.
C1 low-attention digest
- Trigger: daily at 08:00 Panama (added Oct 29).
- Steps:
low_attention_digest_summarybuilds the header and posts it to #normal-triage, or a single none-waiting line. If anything is waiting,low_attention_digest_itemsreturns one line per message, oldest first, up to 50, each posted as a reply in the header thread and recorded withsave_low_attention_post. The database escapes each snippet so customer text cannot ping anyone or post a live link. No sender ID. - Errors: a failed read alerts #system-health. A failed header or item post is not retried: it emails the project owner and alerts #system-health, then the run ends cleanly (Oct 31). If Slack is down at 08:00, that day's digest is skipped and unticked items are listed again the next day. Runs no longer wait for one another, so one failure does not block later digests. A reply that is posted but not recorded alerts, and that message is listed again the next day.
C1 low-attention digest tick
- Trigger: Slack listener for reactions (added Oct 29). Only a white check mark in #normal-triage gets past the filter.
- Steps:
mark_low_attention_reviewedmarks the one message behind that reply reviewed, records who ticked it and erases its snippet, then confirms in the thread. A tick on any other message does nothing. - Errors: a failed mark alerts #system-health, and the message is listed again the next day.
C1 Meta token daily check
- Trigger: every 24 hours; run data is kept confidential for the whole scenario.
- Steps: asks Meta whether the WhatsApp access token is valid and when it expires; verdict ok, warn (7 days or fewer left) or failed.
- Alerts: warn, failed and any step failure alert; if Slack refuses the alert, Make emails the owner.
C1 form heartbeat checker
- Trigger: webhook for the Google Form script's daily check-in, which lists the form entries of the last 48 hours (added Oct 29).
- Steps: checks the secret header; parses the check-in; only a body whose source is the form heartbeat goes on; asks the CRM which listed entries never reached Apex (form_entries_missing). With nothing missing and the trigger present, it records the check-in and answers "OK". Otherwise it emails the project owner the missing entry ids, then alerts #system-health, and only then records the check-in and answers "OK" (Oct 31).
- Errors and alerts: if both the email and the alert fail, the check-in is not recorded, so the 09:00 watcher flags a missed check-in. A failed secret read, header read, parse or entry lookup alerts #system-health and still answers "OK". A wrong secret alerts and answers "Refused". A check-in from the wrong source gets "OK" and no instant alert; the watcher flags it within about 26 hours (accepted limit). If the answer step itself fails, nothing answers (accepted limit).
C1 form heartbeat watcher
- Trigger: daily at 09:00 Panama (added Oct 29).
- Steps: reads the last recorded check-in; if none arrived for over 26 hours, alerts #system-health, naming the window and asking a person to compare the form's responses against Directus.
- Errors and alerts: a failed read alerts. A failed alert Breaks, 3 tries 15 minutes apart, and the next day's run alerts again. The alert cannot name individual missing entries (accepted limit). Nothing watches this watcher itself (accepted limit).
C2 document loader
- Trigger: webhook fed by the project's own Slack app; answers Slack immediately, then works.
- Steps: for each file posted in #knowledge-docs: refuses non-PDFs; downloads it; extracts text; refuses a file with a page under 100 readable characters or more than 100,000 characters; reads the type, zone and effective date from the first line; splits into chunks of up to 1,800 characters plus the section path, refusing more than 100; one batch embedding call; corpus.load_and_activate; replies Uploaded, Updated or No change in the uploader's thread.
- AI: Google gemini-embedding-001, document mode, 1,536 dimensions, 120-second timeout.
- Errors and alerts: download, text and embedding steps wait 3 seconds and try once more, then alert; the split and the database load alert with no retry; replies to the uploader Break, 3 tries 15 minutes apart. A document with no file type is dropped with no reply (accepted limit).
C2 question answerer
- Trigger: called by the #tech-help sorter.
- Steps: embeds the question; retrieves passages (corpus.match_chunks) and keeps those at similarity 0.72 or higher; with none, replies "Nothing in the company documents answers this."; otherwise asks for a grounded JSON answer, strips fences and parses; two guards then block an answer that cites a passage it was not given, cites nothing, or contains a number not copied from a passage; posts the answer with a Sources list built from database rows, or a "could not verify" message listing the sections found.
- AI: Google gemini-embedding-001 in question mode; Google Gemini 3.5 Flash Lite, prompt v5, temperature 0, 1,024-token limit, 60-second timeout.
- Errors and alerts: embedding, retrieval and answer each retry once after 3 seconds, then alert and tell the technician to try again; replies Break, 3 tries 15 minutes apart, and if an answer post still fails, the sorter's alert and "try again" reply fire (tested Oct 31). A blocked answer writes an audit row.
#tech-help question sorter
- Trigger: Slack listener on #tech-help.
- Steps: passes only top-level messages from a person, with text and no Slack subtype, plus photo posts since Nov 1; a fixed rule sends any question containing one of nine stock phrases ("on my truck", "in my truck", "on the truck", "do I have any", "on hand", "in stock", "who has", "anyone have", "anybody got") to the truck stock lookup, and everything else to the question answerer, waiting for either.
- Errors and alerts: a failed hand-off alerts and posts one "try again" reply (Break, 3 tries 15 minutes apart). A photo with text is answered from the text, with a note that the photo was not read; a photo with no text gets a reply asking for the question as a new message (Nov 1).
Shared: open job thread
- Trigger: called by the pipeline front and the job watchdog.
- Steps: skips repeat contacts; for a new job, checks job_thread_exists, posts the job's first message in #parts-used and saves the thread.
- Errors and alerts: each failure alerts and returns a failure outcome; no retry. A missing thread is left for the watchdog to find. An unknown action is dropped with no alert (accepted limit).
Shared: send assignment notice
- Trigger: called by the job assignment notice and the job watchdog.
- Steps: reads the job (get_assignment_notice_context); skips a job with no technician or that is closed; tags the technician in the job's thread with the reaction key (🚗 on the way, 🏁 done) and saves the notice time (save_assignment_notice_ts).
- Errors and alerts: a failed read, missing Slack user, missing thread, failed post or failed save alerts. No retry. If the Directus address cannot be read, a yellow #system-health alert posts and the notice goes out with a plain, unclickable record reference (Oct 31).
Shared: record sweep alert
- Trigger: called by the job watchdog.
- Steps: one call to record_sweep_alert, which counts a reminder once.
- Errors and alerts: a failure alerts; the item is alerted again on the next run (over-alerting, not loss; accepted limit).
C3 job assignment notice
- Trigger: webhook from a Directus Flow when a dispatcher changes a job's status or technician.
- Steps: checks the shared secret header; if a technician was set, calls send assignment notice; if the job was canceled, posts "Cancelled by" in the job thread.
- Errors and alerts: every failure alerts and carries on; the cancellation post Breaks, 3 tries 15 minutes apart. If the Directus address cannot be read, a yellow #system-health alert posts and the run continues with a plain, unclickable record reference (Oct 31).
C3 parts used
- Trigger: Slack listener on #parts-used, up to 100 runs a minute; runs may overlap.
- Steps: first checks whether the message answers an earlier "which part?" question; otherwise extracts the part, equipment type and quantity, checks them (check_part_report), and posts a read-back asking for ✅ to confirm or ❌ to discard, saved as a pending update; an unclear part gets the three closest matches to choose from.
- AI: Google Gemini 3.5 Flash Lite, prompt v2, temperature 0, 512-token limit, 20-second timeout; one retry after 3 seconds.
- Errors and alerts: posts to the technician Break, 3 tries 15 minutes apart; if the retries run out, nobody is told and the technician sees no reply (accepted limit); database failures alert.
C3 parts confirm, undo and discard
- Trigger: Slack reactions in #parts-used: ✅, ❌, ↩️, 🚗, 🏁.
- Steps: finds which message was reacted to; ✅ confirms and deducts stock in one database step; ❌ discards and writes an audit row; ↩️ on a confirmation undoes it with a reversing row; 🚗 and 🏁 on an assignment notice update the job (update_job_status). Anyone but the reporting or assigned technician is told why nothing changed.
- Errors and alerts: each database write posts "trying again", waits 8 seconds and retries once, then edits the message with the result; posts Break, 3 tries 15 minutes apart, and if the retries run out nobody is told (accepted limit). Stock can be deducted while the "saved" reply fails, which leaves no undo anchor; the stock record is still correct (accepted limit). A reaction from anyone but the reporting technician changes nothing and gets a short refusal reply; only if that reply fails is nothing said. Unlisted reactions or outcomes are ignored (both by design).
C3 truck stock lookup
- Trigger: called by the sorter.
- Steps: extracts the part; checks the asker's truck (check_own_truck); if it has none, finds the truck with the most on hand (find_nearest_truck); replies with one of four answers.
- AI: Google Gemini 3.5 Flash Lite, prompt v1, temperature 0, 512-token limit, 60-second timeout; one retry after 3 seconds.
- Errors and alerts: a second failure alerts and hands "try again" back to the sorter. A negative stock quantity is dropped with no reply (accepted limit).
C3 review drafter
- Trigger: run by hand with a review; no scenario calls it.
- Steps: saves the review (save_review), stopping if a draft already exists; one AI call; posts the draft to #review-approvals, with a MOCK REVIEW banner when the review is a mock; saves the draft for 24 hours.
- AI: Google Gemini 3.5 Flash Lite, prompt v2, temperature 0.2, 1,024-token limit, 60-second timeout; one retry after 3 seconds.
- Errors and alerts: failures alert and return an outcome; the approval post Breaks, 3 tries 15 minutes apart, and if the retries run out nobody is told (accepted limit).
C3 review approval
- Trigger: Slack reactions ✅ or ❌ in #review-approvals.
- Steps: finds the draft (one retry after 3 seconds); a ✅ on Google or Instagram replies that publishing is not connected and changes nothing; otherwise claims the draft, marks it discarded or posted by a person (Yelp), writes an audit row and replies; if the final save fails, releases the claim.
- Errors and alerts: posts Break, 3 tries 15 minutes apart, and if the retries run out nobody is told (accepted limit); a failed audit write alerts and the run carries on. Unlisted reactions or outcomes are ignored (by design).
C3 job watchdog
- Trigger: every 30 minutes, 07:00 to 20:00, plus runs at 00:00 and 04:00.
- Steps: one database call (find_watchdog_findings) returns neglected jobs with reasons, low stock and due reminders, holding the digest in quiet hours and for 4 hours when unchanged; opens a job thread that was never opened and re-sends missed notices (a Slack thread deleted later is not re-created: accepted limit); if two or more jobs are flagged, asks the AI to rank them; build_watchdog_digest_text reconciles the ranking and the digest posts to #normal-triage; due reminders post, up to 7 per item, each counted by record sweep alert.
- AI: Google Gemini 3.5 Flash Lite, prompt v1, temperature 0.1, 1,500-token limit, 60-second timeout; one retry after 3 seconds.
- Errors and alerts: a failed database read alerts and retries the run once a minute later; an unusable ranking alerts and the digest posts unranked. If the Directus address cannot be read, a yellow #system-health alert posts and the run continues with plain, unclickable record references (Oct 31).
C3 weekly report to Google Sheets (not scheduled)
- Trigger: weekly, Monday 06:00.
- Steps: appends the week's audit rows (audit.weekly_rollup) to one tab; clears and rewrites the truck-stock tab (weekly_truck_stock_rollup).
- Errors and alerts: every step alerts with no retry; if the clear succeeds and the write fails, the stock tab is left empty.
Appendix B. Reference lists
The full lists behind sections 2, 3 and 4, for a maintainer. The main body states the rules they carry.
B1. Production scenarios
| Component | Scenario | Trigger | AI step |
|---|---|---|---|
| C1 | WhatsApp intake | Webhook (Meta), up to 100 runs a minute | None |
| C1 | Phone and form intake | Webhook (Retell and Google Form) | None |
| C1 | Request pipeline front | Called by the two intakes | None directly |
| C1 | Intake resolution | Called by the pipeline front | Extraction |
| C1 | Junk gate | Called by intake resolution | Junk gate |
| C1 | Urgency rater | Called by the pipeline front | Urgency rater |
| C1 | Form confirmation email | Called by the pipeline front | None |
| C1 | Silent-message daily digest | Every 24 hours | None |
| C1 | Meta token daily check | Every 24 hours | None |
| C1 | Low-attention digest (added Oct 29) | Daily at 08:00 | None |
| C1 | Low-attention digest tick (added Oct 29) | Slack listener | None |
| C1 | Form heartbeat checker (added Oct 29) | Webhook (the form script's daily check-in) | None |
| C1 | Form heartbeat watcher (added Oct 29) | Daily at 09:00 | None |
| C2 | Document loader | Webhook (own Slack app) | Embeddings |
| C2 | Question answerer | Called by the sorter | Embeddings, grounded answer |
| Shared | #tech-help question sorter | Slack listener | None (fixed phrase rule) |
| Shared | Open job thread | Called by the pipeline front and the watchdog | None |
| Shared | Send assignment notice | Called by the job assignment notice and the watchdog | None |
| Shared | Record sweep alert | Called by the watchdog | None |
| C3 | Job assignment notice | Webhook (Directus Flow) | None |
| C3 | Parts used | Slack listener, up to 100 runs a minute | Parts extraction |
| C3 | Parts confirm, undo and discard | Slack listener, up to 100 runs a minute | None |
| C3 | Truck stock lookup | Called by the sorter | Part lookup |
| C3 | Review drafter | Run by hand with a mock review | Review draft |
| C3 | Review approval | Slack listener | None |
| C3 | Job watchdog | Every 30 minutes, 07:00 to 20:00, plus 00:00 and 04:00 | Ranking |
| C3 | Weekly report to Google Sheets | Weekly, Monday 06:00 | None |
The Make team also holds 33 non-production scenarios (11 backups, 15 test copies and harnesses, 7 others), none scheduled; five test harnesses run only when started by hand. None of them is part of the delivered system.
B2. Data contracts between scenarios
| Called scenario | Called by | Inputs | Returns |
|---|---|---|---|
| Request pipeline front | WhatsApp intake; phone and form intake | channel, platform_event_id, sender_id, callback_contact, message_text, received_at, caller_name, service_address, email, gas_or_burning_smell, water_leak, message_type | Nothing; it is the end of the line |
| Intake resolution | Request pipeline front | channel, sender_id, message_text, correlation_id | stage, failed, customer_resolved, customer_id, junk_route, gate_failed, extraction_outcome, equipment type, make and model, symptoms, name and address fragments, customer_match_status, equipment_id, candidate_customer_ids |
| Junk gate | Intake resolution | message, customer_resolved | route, outcome, bypassed, gate_failed, confidence, raw_reply, model, stop_reason, token and cache counts, prompt_version |
| Urgency rater | Request pipeline front | message, context | urgency, rater_error, model, stop_reason, token and cache counts, raw_reply, prompt_version |
| Open job thread | Request pipeline front; job watchdog | job_id, action, urgency, channel, issue_text, correlation_id, test_prefix, record_link | outcome, thread_opened, thread_ts, channel_id |
| Form confirmation email | Request pipeline front (no wait) | channel, action, job_id, email, caller_name, correlation_id | outcome |
| Question answerer | #tech-help question sorter | question, channel_id, thread_ts | status_code, reply_text, finish_reason, model_version, token counts, error_message, prompt_version |
| Truck stock lookup | #tech-help question sorter | question, asker's Slack user, channel_id, thread_ts | status_code, reply_text, finish_reason, prompt_version |
| Send assignment notice | Job assignment notice; job watchdog | job_id, test_prefix | outcome, notice_ts |
| Record sweep alert | Job watchdog | kind (triage, review_draft, stock_update, stock_end_of_day, review_undrafted), item_id, alert_number, test_prefix | outcome (recorded, not_changed, record_failed) |
| Review drafter | Run by hand | platform, review_id, author, rating, text, is_mock, review date | outcome, draft_id, sentiment, detail |
B3. Database names: tables and functions
Knowledge corpus database. Tables: source_document (documents), ingest_batch (loads) and chunk (chunks), plus the view active_chunk. Live chunks per document: diagnostic guide 99, warranty policy 56, permit code index 54. Functions: corpus.load_and_activate (the load function) and corpus.match_chunks (the search function).
Audit log database. Table: audit.event (the event log). Reader functions: outbound_count and reply_window (the WhatsApp reply ceiling), silent_digest (the day's silent messages) and weekly_rollup (report rows without the payload). Besides the two append-only triggers, a third trigger copies the urgency level out of the payload on urgency rater rows so the weekly report can read it without the payload. C1 decision values in use: junk_gate, customer_lookup, extraction, resolution, urgency_rater, job_record, form_confirmation, low_attention and junk_gate_override.
CRM tables. Intake: inbound_events, whatsapp_sessions, job_contact_attempts, sender_blocklist. Customers and jobs: customers, equipment, jobs, job_threads. Stock: technicians, trucks, parts, truck_stock, pending_stock_updates, pending_part_choices, usage_log. Reviews: reviews, pending_review_drafts. Watchdog: watchdog_digest_state. History: row_history.
CRM functions by purpose.
- Intake: prefilter_check (no sender, length outside 1 to 2,000 characters, blocked sender, a sender's 12th message within 2 minutes, counting the current one; changed Oct 31); insert_inbound_event (the duplicate check); lookup_customer (exact match on WhatsApp number, phone digits or email); resolve_intake (equipment match, then fuzzy customer suggestions by name and address); create_job_record (runs with its owner's rights, locks per sender and customer, bundles repeat contacts onto an open job, only ever raises urgency, sets the triage deadline of 15 minutes for Level 5 and 4 hours otherwise); open_or_get_whatsapp_session and session_position (the 6-hour conversation window); phone_digits.
- Stock: check_own_truck and check_part_report (part search, plural-tolerant); save_part_choice and answer_part_choice; save_pending_update (4-hour expiry); confirm_stock_update, undo_stock_update and discard_stock_update (each refuses anyone but the reporting technician); find_nearest_truck (ranks other trucks by quantity on hand, not distance); weekly_truck_stock_rollup.
- Assignment and status: get_assignment_notice_context, save_assignment_notice_ts, update_job_status (only the assigned technician moves a job to in progress or completed).
- Reviews: save_review, save_review_draft, find_review_draft, claim_review_draft (one conditional update, so two reactions cannot both act), release_review_draft, finish_review_draft (refuses "published" for Yelp).
- Watchdog: find_watchdog_findings (wrapper with the 4-hour repeat hold) over find_watchdog_findings_at (testable with a supplied clock, handles quiet hours); build_watchdog_digest and build_watchdog_digest_text (strip code fences from the AI ranking, keep every flagged job, drop invented IDs); record_sweep_alert.
- Triggers: row history capture on customers and jobs; jobs_system_times sets assigned and completed times and clears a stale assignment notice; updated-time stamps on several tables.
B4. Tests by area
| Area | What was tested |
|---|---|
| C1 | Unknown-contact intake capture and routing on all three channels, plus failure cases |
| C1 | Voice intake end to end, Retell call through job record and urgency |
| C1 | Urgency rater test set and runs (with re-scoring) |
| C1 | Junk gate evaluation and the switch to Haiku 5.5 |
| C1 | Customer match status: zero, one and several candidates |
| C1 | Junk-gate bypass only for an exact-match known customer |
| C1 | The shared webhook's four routes and the silent no-number stop |
| C1 | Repeat contacts bundle into one untriaged job within 24 hours |
| C1 | Form confirmation email under a live temporary failure |
| C1 | A forced audit-write failure still produces a job and one alert |
| C1 | The advisory-only urgency rater never writes a calendar entry |
| C2 | Retrieval similarity threshold calibrated at 0.72 on 16 cases |
| C2 | Grounded-answer evaluation with an AI judge |
| C3 | Atomic truck-stock decrement, race and failed-write routes |
| C3 | Watchdog breach check flags every neglected job and fails loudly |
| C3 | Review approval: nothing publishes without a person approving |
| All | Estimated against measured run cost |
| All | Pre-demo review of all 23 production scenarios (as of Oct 28) and the database functions, with fixes |
| All | Re-scoring of the three AI evaluations in a separate session |
B5. Accepted demo limits from the pre-demo review
| Limit | Status on Oct 28 |
|---|---|
| WhatsApp intake holds Meta's "got it" reply until the pipeline finishes, so Meta may resend a slow message; the duplicate check makes a resend harmless | Confirmed live |
| Phone intake refuses a Retell request signed more than 5 minutes earlier, so a late redelivery or hand replay is refused | Confirmed live |
| A failed Level 5 dispatch post in a run started by an intake scenario is probably not retried (inferred, not proven) | Tested Oct 29: confirmed, nothing retried it. Fixed the same day: a failed dispatch post now emails the project owner and alerts #system-health |
| The form confirmation email has no test guard, so a test-harness run sends a real email | Confirmed live |
| Intake resolution records "flagged" even if saving the low-attention flag failed, and nothing reads the flag yet | Fixed Oct 29: a failed or zero-row save now posts the request to #normal-triage, then logs flag_write_failed. The daily low-attention digest reads the flag. Since Oct 30 the post goes out before the audit row, and a failed audit write alerts #system-health and carries on, so the run still returns normally. Independent re-checks passed Oct 30, including both Oct 30 changes |
| Parts used has no duplicate guard on the technician's message, so a resent Slack event can create a second read-back, and a tick on both would deduct twice | Confirmed live |
| The audit log's write login is in Neon's admin group, which SQL cannot remove; the triggers keep the log append-only | Confirmed live |
| Review approval has no named-approver list; anyone in #review-approvals can approve | Confirmed live |
| The weekly report has no schedule; a re-run would append overlapping rows | Confirmed |
| The truck stock lookup's "other truck" is the one with the most stock, not the nearest | Confirmed live. It is not the nearest truck |
| The WhatsApp reply ceiling is stated as 20 in 24 hours; the real maximum is 18 (6 replies, then one per 2 hours) | Confirmed live |
B6. Record differences, now reconciled (section 6, items 1 to 11)
| # | Area | The record says | The live build does | Reason |
|---|---|---|---|---|
| 1 | C1 Meta token daily check | Built, with no daily run recorded (the charter's list of what is built, designed only or declined) | Runs every 24 hours | Not recorded at the time. The project owner decided on Oct 28 to keep the daily run |
| 2 | Audit log columns | step_name, model, model_version, input_ref, output, outcome, latency_ms, created_at | occurred_at, component, event_type, actor, decision, outcome, model_ref, urgency_level and a JSON payload | Not recorded |
| 3 | C1 audit rows | A row after each main intake step, including the duplicate-check insert | No audit decision for the duplicate-check insert appears among the rows written; lookup, junk gate, extraction, resolution, rater and job record do | Not recorded. Seen in the audit data, not confirmed in the blueprint |
| 4 | Audit coverage outside C1 | Public charter: one audit row per pipeline step | C2 writes a row only when an answer is blocked; C3 writes rows for review outcomes and stock discards only | Not recorded |
| 5 | Retries on AI calls | Public charter: AI calls and reads get one quick retry | True for C2 and C3. The three C1 calls make one attempt and then take a safe default (treat as real, leave fields unresolved, or alert with no job) | The C1 failure rules are recorded in the working charter's model table; the public wording is broader than the build |
| 6 | Retries on outbound posts | Public charter: outbound messages retry three times, 15 minutes apart | The missed-call notice did until Oct 31; it now emails the project owner and alerts #system-health instead. C1 dispatch posts did not in practice, because no retry fires in a run another scenario starts; since Oct 29 a failed dispatch post sends an email and a #system-health alert instead. Most C2 and C3 replies to technicians and approvers retry three times, 15 minutes apart, and nobody is told if the retries run out | Not recorded |
| 7 | C2 chunk size | Chunks are at most 1,800 characters, each starting with its section path | Text is cut at 1,800 characters, then the section path is added in front, so a stored chunk can run longer | Not recorded |
| 8 | C2 similarity floor | Calibrated at 0.72 | Applied in the question answerer scenario. The database search function returns the top 1 to 20 chunks with no floor of its own | Location not recorded |
| 9 | Slack listeners | Five listeners, including the document upload | Four as of Oct 28. The document loader is reached through the project's own Slack app and a webhook | Recorded: Make's private-channel listener would not register (Oct 16 to 17) |
| 10 | CRM tables | 16 tables in the approved schema | 19. Added: pending_part_choices, sender_blocklist and watchdog_digest_state | watchdog_digest_state: the 4-hour repeat hold. The other two: not recorded in the sections read |
| 11 | C3 job watchdog schedule | The watchdog's build record and the pre-demo review's close-out note say every 15 minutes in the day window | Every 30 minutes, 07:00 to 20:00, plus runs at 00:00 and 04:00, matching the charter's list of what is built, designed only or declined. Make does not show the time zone of the windows | Recorded with the run-cost check; two older notes were not updated |
Appendix C. When something fails: full table
Most failures take a safe default or post where a person will see them, and the few that do not are named below as accepted limits. On October 31 a sweep of every production scenario for failures nobody would hear about found the gaps. The fixes were built the same day and passed independent tests on test copies, except three rare failure paths that were never forced to fail in testing: the low-attention digest failing to post one of its items, the phone-call branch of the intake alert for a failed signature step, and a form heartbeat checker alert while Slack is down. They are rare, off the normal path and low impact on customers or the business, so they are accepted limits. The principle: critical paths (a customer request reaches a person, an emergency reaches dispatch) are guarded, alerted and backed up, and rare, low-impact cases are accepted and recorded. On November 1 the project owner re-sorted the remaining gaps: the second table lists the accepted limits with what each one costs, and the third lists behavior chosen by design.
What the fixes changed:
- Intake, before the hand-off: if WhatsApp or phone-and-form intake cannot read its secret, run a signature step or parse a request, the request is posted to #urgent with the sender and message, or the caller or form details. Raw phone-call data is never posted.
- Missed-call notice: a failed notice emails the project owner and alerts #system-health with the caller's number and call id, instead of parking a retry that tells nobody.
- Sender flood: the rule now trips on a sender's 12th message within 2 minutes (it was the 11th within 5). The alert goes to #urgent and names the sender, their latest message and their open job with a record link, or "none". Later messages from that sender are still dropped.
- Low-attention digest: a failed post emails the project owner and alerts #system-health, then the run ends cleanly. One failure no longer holds up later digests.
- Form heartbeat: when form entries are missing, the alert and an email listing the missing entry ids go out before the check-in is recorded. If both fail, the check-in is not recorded, so the 09:00 watcher flags a missed check-in.
- Intake resolution: its two #urgent alerts now say to check Directus and create the job by hand, because a replay will not rebuild it.
- Directus address: if the assignment notice, the send-notice step or the watchdog cannot read it, a yellow #system-health alert posts and the run continues with a plain, unclickable record reference.
- #tech-help photo posts: a photo posted with text is now answered from the text, with a note that the photo was not read. A photo with no text gets a reply asking for the question as a new message.
Earlier fixes still stand. A real request marked junk is listed in the daily low-attention digest at 08:00 until a person ticks it reviewed, so it can wait up to about a day. A message that uses one of the emergency words is never parked; emergency wording that avoids the list is still parked, and the list is final. A failed junk-flag save posts the request to #normal-triage. The document loader has checked Slack's signature since October 29. A failed dispatch post is not retried, so it sends an email and a #system-health alert instead.
| What fails | What happens | Who sees it |
|---|---|---|
| Junk gate | The message is treated as real and carries on. The audit row is the only trace | No one (by design) |
| Saving the junk flag (Oct 29) | The request is posted for a person instead of being parked unseen | #normal-triage |
| Emergency word check (Oct 30) | The message is not parked and carries on as real. If the junk gate had called it junk, an alert says the check is broken, at most once an hour | #system-health |
| Extraction | Every field is left unresolved and the job carries on | The dispatcher, as blank fields on the job |
| Urgency rater | No job is created; the raw request is posted for a person | #urgent |
| Intake secret read, signature step or parse, before the hand-off (Oct 31) | The request is posted for a person with its sender and message, or the caller or form details. Raw call data is never posted. These alerts fire before the signature check, so the text is unchecked | #urgent |
| A database read, a hand-off between scenarios, the format check or the job write | The raw request is posted for a person. Intake resolution's alerts say to create the job by hand in Directus, because a replay will not rebuild it | #urgent |
| An audit-log write | The job is still created | #system-health |
| Pre-filter review post (a held message) | Not retried, because an intake scenario always starts the pipeline front (the shared intake step). The intake's fallback fires at once and names the request (tested Oct 31) | #urgent |
| Sender flood: a sender's 12th message within 2 minutes (Oct 31) | That message and later ones from the sender are dropped. One alert names the sender, their latest message and their open job with a record link | #urgent; if that post fails, an email to the project owner and #system-health |
| A dispatch post to Slack (#urgent or #normal-triage) | Not retried: Make keeps no failed run when an intake scenario starts the pipeline, so a retry setting there never fires (tested Oct 29). Instead, within seconds, an email goes out and an alert posts, and the run ends cleanly. The job is already saved; a person triages it from the job record | An email to the project owner, and #system-health |
| Missed-call notice (Oct 31) | Not retried. An email and an alert name the caller's number and call id, and the run ends cleanly | An email to the project owner, and #system-health |
| Form confirmation email | Retried 15 minutes later: the pipeline front starts it without waiting, so Make keeps the failed run (tested Oct 31). A permanent failure writes an audit row and alerts | #system-health |
| Low-attention digest post (Oct 31) | An email and an alert, then the run ends. If Slack is down at 08:00, that day's digest is skipped, and unticked items are listed again the next day | An email to the project owner, and #system-health |
| Form heartbeat finds missing entries (Oct 31) | An email listing the missing entry ids and an alert go out before the check-in is recorded. If both fail, no check-in is recorded, so the watcher flags it at 09:00 | An email to the project owner, and #system-health |
| Form heartbeat: no check-in for over 26 hours | The daily 09:00 watcher alerts. It cannot name the missing entries | #system-health |
| Reading the Directus address in the assignment notice, send-notice step or watchdog (Oct 31) | A yellow alert, and the run continues with a plain, unclickable record reference | #system-health |
| Any AI call in the knowledge base, stock, parts or review steps | One retry after a few seconds, then an alert, plus an apology or "try again" where a technician is waiting | The technician, and #system-health |
| An answer post in #tech-help (knowledge base or stock lookup) | The sorter's alert and its "try again" reply both fire (tested Oct 31) | The technician, and #system-health |
| A reply to a technician or approver in the parts and review steps | Retried 3 times, 15 minutes apart. If the retries run out, nobody is told; the technician just sees no reply | No one (accepted limit) |
| The watchdog's ranking | One retry, then the digest posts unranked with a note saying so | #system-health |
| A forged or unsigned request at the WhatsApp, phone-and-form, job-assignment or document-loader webhook | Refused, and answered "OK" so the sender does not keep retrying. A genuine request refused by mistake is therefore lost | #system-health, naming its source |
Accepted limits. Each is a known gap kept for the demo, with what it costs.
| Limit | What it costs |
|---|---|
| A record-sweep alert whose count fails to save | It posts again on the next run: over-alerting, not loss |
| The heartbeat watcher's missed check-in alert | It says entries are missing but cannot name which ones |
| A heartbeat from the wrong source | It gets "OK" and no instant alert. The watcher flags it within about 26 hours |
| The step that answers the form script fails | The script is not told the outcome |
| An incomplete Google Form response | It goes to the "unrecognized" alert, which cannot identify the customer. Every question is required, so this happens only if the form or its script is edited |
| One Slack connection carries every alert | If it fails, every Slack alert fails with it. Email backup covers the dispatch posts, the digest, the heartbeat, the missed-call notice and the flood alert |
| Nothing watches the monitors themselves | A monitor that stops is noticed only when someone looks. A real deployment would add an outside uptime check |
| Dormant grace routing in the pipeline front's dispatch router | Unused routing left in the build. No job can reach it since Oct 23 |
| C3 parts and review replies | Retried 3 times, 15 minutes apart. If the retries run out, nobody is told; the technician or approver sees no reply. In parts confirm, stock can be deducted while the "saved" reply fails: the stock record is correct, but no undo anchor is posted |
| Rare silent drops | A document with no file type, a negative stock quantity and an unknown action in open job thread are each dropped with no reply and no alert |
| Low-attention digest when Slack is down at 08:00 | That day's digest is skipped, with an email and a #system-health alert, and unticked items are listed again the next day: one late day |
| The #tech-help photo note, smaller gaps | It fires for any shared file, PDFs included. An upload with files but no Slack subtype takes the pre-fix path. Nobody is alerted when the note's own retries run out. An odd or missing note, rarely |
| A #tech-help photo post whose hand-off to the answerer fails | The technician gets the usual "try again" reply, then the photo note, which mentions an answer that does not exist: a confusing extra line |
| Some alerts show a double space after "What failed:" | Looks only |
| Paths closed without an independent test | Parts confirm and undo; the document loader's refusal replies, split-failure alert and forced AI retry; the assignment notice with no secret header, two simultaneous edits or a cleared technician. These rare paths are built but unproven |
| Three fixed failure paths never forced to fail in testing | The low-attention digest failing to post one of its items; the phone-call branch of the intake alert for a failed signature step; a form heartbeat checker alert while Slack is down. They are rare, off the normal path and low impact on customers or the business, so each is built but unproven on that one path |
| A job's Slack thread is deleted | Nothing re-creates it. The watchdog step that would do so was never built |
By design. Behavior chosen on purpose, not defects, with the reason for each.
| Behavior | Why |
|---|---|
| A delivery that fails the signature check is refused and answered "OK", so it is not resent. #system-health names the source | An attacker's request must not trigger retries. A genuine request refused by mistake is lost |
| If the junk gate fails, the message is treated as real and carries on. The audit row is the trace | A lost real request costs more than a junk message reaching a person |
| A stock reaction from anyone but the reporting technician changes nothing and gets a short refusal reply. Only if that reply fails is nothing said | Only the reporting technician may act, and nothing changed |
| New intake alerts for a failed secret read or signature step fire before the signature check, so while the secrets store is failing, unchecked text can reach #urgent | Loud beats silent while the check itself is broken |
| Unlisted reactions or outcomes in C3 parts and review are ignored | A stray emoji should not trigger anything |
| Once a sender trips the flood rule and #urgent is alerted, their further messages in that 2-minute window are dropped without another alert | One alert per flood is enough; repeats would flood #urgent |
Where alerts go. Operational alerts go to #system-health in one six-line format: what failed, what it affects, what happened, the next step and a link to the run. Most failures that could cost a customer request go to #urgent, where the dispatcher is watching. One Slack connection carries every alert, so when a dispatch post, the low-attention digest, a form-heartbeat alert, the missed-call notice or the flood alert fails, an email to the project owner backs it up, and that email does not depend on Slack.
Glossary
- Scenario: one Make.com workflow. Each has exactly one trigger, the event that starts it. In the database, "trigger" means something else: a rule that runs by itself whenever a row is added, changed or deleted.
- Subscenario call: one scenario starting another inside the same Make account and waiting for its answer, with typed inputs. This is how most hand-offs work here; no hand-off between scenarios goes over a public web address.
- Webhook: a web address that an outside service (Meta, Retell, Google, Directus, Slack) posts to, which starts a scenario.
- Stored function: a piece of logic that lives in the database and is called by name, so a rule is enforced in one place no matter which scenario calls it.
- Production: the live demo build, as opposed to test copies. It has never handled real customer traffic.
- #system-health: the Slack channel where operational failure alerts land (see When something fails).