Compare revised rules

Keep both definitions and compare the same inputs before changing your application.

Before replacing a model in your application, check what changes for the cases you already understand. Keep both model files and execute the same inputs against each. This guide performs that comparison from your client using the public execution API; it does not rely on stored models or receipt history.

Revise the definition you kept

Submit its content and questions as output, with the requested policy change:

jq -n --slurpfile model invoice-model.json \
  '{content: $model[0].content, output: $model[0].questions,
    prompt: "Increase the early-payment discount from 2% to 3%. Pay 97% of invoice_amount for approved invoices within the inclusive 10-calendar-day window. Keep the deadline, approval requirement and other outcomes unchanged."}' \
  > revision-request.json

curl --fail-with-body -sS https://api.aityx.ai/v1/systemtwo/models \
  -H "Authorization: Bearer $AITYX_API_KEY" \
  -H "Content-Type: application/json" \
  -d @revision-request.json > revision.json

jq '.model' revision.json > invoice-model-revised.json

Inspect both content (the executable rules) and questions (the answer contract). Send the returned complete model for execution; sending question text alone can generate rules again. AI revision uses System Two pricing. Keep the original file so your application can continue using it until you choose to switch.

Keep cases that reveal a mistake

Save these as invoice-cases.json. They include a qualifying invoice, the inclusive cutoff, late payments and an invoice that needs approval.

[
  {
    "name": "initial",
    "state": {
      "invoice_amount": 25000,
      "invoice_date": "2026-10-01",
      "payment_on": "2026-10-11",
      "approved": true
    }
  },
  {
    "name": "matching-json",
    "state": {
      "invoice_amount": 25000,
      "invoice_date": "2026-10-01",
      "payment_on": "2026-10-11",
      "approved": true
    }
  },
  {
    "name": "past-cutoff",
    "state": {
      "invoice_amount": 25000,
      "invoice_date": "2026-10-01",
      "payment_on": "2026-10-12",
      "approved": true
    }
  },
  {
    "name": "unapproved",
    "state": {
      "invoice_amount": 25000,
      "invoice_date": "2026-10-01",
      "payment_on": "2026-10-11",
      "approved": false
    }
  },
  {
    "name": "early-40000",
    "state": {
      "invoice_amount": 40000,
      "invoice_date": "2026-10-01",
      "payment_on": "2026-10-06",
      "approved": true
    }
  },
  {
    "name": "late-9000",
    "state": {
      "invoice_amount": 9000,
      "invoice_date": "2026-10-01",
      "payment_on": "2026-10-13",
      "approved": true
    }
  }
]

Compare both models from your client

Save this as compare-invoices.mjs and run node compare-invoices.mjs. Each execution sends the complete definition and matching JSON at System One pricing. Both sets of receipts are kept in your output file.

import { readFile, writeFile } from "node:fs/promises";

const read = async path => JSON.parse(await readFile(path, "utf8"));
const original = await read("invoice-model.json");
const revised = await read("invoice-model-revised.json");
const cases = await read("invoice-cases.json");
const values = answers => Object.fromEntries(Object.entries(answers).map(([key, a]) =>
  [key, a[a.type]]));

async function execute(model, state) {
  const response = await fetch("https://api.aityx.ai/v1/systemone", {
    method: "POST",
    headers: {
      Authorization: `Bearer ${process.env.AITYX_API_KEY}`,
      "Content-Type": "application/json",
    },
    body: JSON.stringify({ model, state }),
  });
  const result = await response.json();
  if (!response.ok) throw new Error(result.detail ?? `aityx ${response.status}`);
  return result;
}

const comparison = [];
for (const test of cases) {
  const before = await execute(original, test.state);
  const after = await execute(revised, test.state);
  const oldValues = values(before.answers);
  const newValues = values(after.answers);
  const changed = [...new Set([...Object.keys(oldValues), ...Object.keys(newValues)])]
    .some(key => oldValues[key] !== newValues[key]);
  comparison.push({ name: test.name, state: test.state, changed,
    before: oldValues, after: newValues,
    receipts: { before: before.receipt, after: after.receipt } });
}
await writeFile("invoice-comparison.json", JSON.stringify(comparison, null, 2));
console.table(comparison.map(({name, changed, before, after}) => ({
  case: name, changed, before: before.payable_amount, after: after.payable_amount,
})));

Under the written invoice policy, three payable amounts should change: the two qualifying $25,000 cases move from $24,500 to $24,250, and the early $40,000 case from $39,200 to $38,800. Late and unapproved cases should keep their original outcomes. Review any unexpected difference before switching.

Choose what your application submits

Deploy the revised model file when the comparison meets your expectations. Keep the original for rollback. Changing the model in your console session does not change the definition submitted by your application.

If a generated rule disagrees with the policy, include the policy clause and a failing case in your revision request. There is no server-side feedback history: keep that evidence with your own tests.