Accounting Policy Assistant

What to do

1. 1Set the objectiveDescribe the purpose, intended audience, and scope of the accounting policy decision.
2. 2Map current processMap how questions are received, policy documents are searched, and accounting treatments are drafted, reviewed, and approved. Identify who handles what and how the steps are connected.
3. 3Define the scopeThe agentic workflow retrieves policy documents, classifies questions, and drafts accounting treatments. Interpretation and sign-off remain with policy specialists. Posting bookings is outside the scope.
4. 4Set acceptance criteriaDefine what a good result looks like at each step. Require correct classification, current policy sources, traceable citations, and clear escalation where the policy is ambiguous or does not cover the question.
5. 5Choose a test caseUse an accounting question and approved policy documents that a policy specialist can check against an expected classification and accounting treatment.

Example

A user asks about the accounting treatment of a standard twelve-month property quota-share treaty. The question gets classified as Defined since there are existing policy documents for it. The task then is to find the current premium-allocation policy, prepare the accounting treatment with citations, and send it to the policy specialist for sign-off.

What to do

1. 1Define sources and ownersName the owner of the policy documents, supporting documents, and past decision records.
2. 2Prepare the dataKeep current and superseded policies distinguishable, and define how drafts are handled. Summarize each document's scope and create an index for retrieval.
3. 3Assign reference IDsGive each policy document, section, and decision record a stable ID, such as POL-05 §16 for a policy paragraph. Link these references to the document's metadata so citations identify the correct source and version.
4. 4Add input checksCheck that all relevant policy topics are covered by the input documents, with complete source references and clearly identified versions.

Example

The corpus contains POL-04 v2.1 as superseded and POL-04 v3.0 as the current version. A question about the risk adjustment must use v3.0 and state that v2.1 was found but not relied on. Removing the old paper makes retrieval simpler, but it would make comparisons across versions much harder down the road (e.g. What is the impact of the policy change?).

What to do

1. 1Break down the taskWork backwards from the desired output and identify the processing steps required to get there. Group activities that together produce one meaningful, intermediate result. For example, a set of relevant policy passages provides the basis for classifying the question and drafting a response.
2. 2Specify each stepDescribe what each step receives, what it should do, and what it passes on. For instance, specify how retrieved policies inform the classification and how cited findings become a proposed accounting treatment.
3. 3Define exception handlingSpecify when to request missing facts or refer a question to a policy specialist. Define how to present conflicting interpretations or a policy gap when a treatment cannot be proposed.
4. 4Define the outputDefine the output schema, including field names, data types, and required values. Specify how missing information is represented and provide a populated example. For the policy response, include fields for the question classification, proposed treatment, and policy citations.

Example

The workflow separates finding relevant policies from assessing the question and drafting a response. This lets you check the relevance of the retrieved passages and the reasoning behind the proposed treatment independently. The drafting step then brings those results together in a response with citations.

The final output could follow a schema like this:

{
  "classification": "defined | ambiguous | undefined",
  "proposed_treatment": "string or null",
  "evidence": [{"reference": "POL-05 §16", "source": "source link"}],
  "supported_interpretations": [],
  "escalation_summary": "string or null",
  "source_status_notice": "string or null",
  "review_status": "pending"
}

What to do

1. 1Assign each stepDecide whether code, an AI agent, or a person should carry out each step. Use code to retrieve documents by reference, agents to interpret policy passages, and a policy specialist to approve the treatment.
2. 2Implement each stepWrite the code and agent instructions required to process the step. You can use AI to assist you with this work. For the drafting step, supply the question, selected policy passages, and classification rules.
3. 3Connect the stepsPass each step's output to the steps that depend on it. Implement the exception handling and requests for human input. Carry the stable IDs throughout each step all the way into the accounting policy response.
4. 4Run the workflowProcess the sample inputs from start to finish. Inspect intermediate results and fix failures until the workflow produces an output that follows the schema.

Example

Implement retrieval so the drafting agent receives the selected policy passages with their reference IDs. This gives it the relevant context and the references needed for citations. Run the sample question through the connected steps to produce a first structured response for specialist review.

What to do

1. 1Identify possible failuresWalk through the existing workflow steps and identify errors that could compromise the final output. For accounting policy, consider incorrect policy versions, unresolved citations, and treatments that lack support.
2. 2Implement validationsImplement checks for those errors. The checks can be performed by code or agents. For instance, check required response fields and allowed classification values in code and use an agent to assess whether the cited passages support the treatment.
3. 3Raise issues for reviewFlag failed checks alongside the results and summarize them for the reviewer. Explain what failed, include the relevant evidence, and state what needs checking or correcting. For example, highlight an unresolved citation and include the proposed treatment.

Example

A proposed accounting treatment cites a policy paragraph that cannot be found. The citation check flags the reference alongside the treatment and adds the issue to the review summary. The specialist receives the response with the unresolved reference clearly identified for correction.

What to do

1. 1Collect representative casesGather realistic inputs covering common tasks, difficult cases, and known failures. Include cases where the workflow should identify missing information or raise an issue for review. Include questions with clear policy coverage, conflicting interpretations, and policy gaps.
2. 2Define expected resultsSpecify the expected results for each case before running the evaluation.
3. 3Set up gradingDefine how each result will be assessed against those expectations. Use code for exact comparisons and an agent or person with clear scoring criteria for judgments about content quality.
4. 4Run and compareRun the evaluation set and inspect failures to identify which steps need improvement. Record a baseline and repeat the evaluation after changes to compare performance.

Example

An evaluation case asks about a treatment the policy documents do not cover. The expected response is Undefined, with no proposed treatment and a request for specialist review.

What to do

1. 1Assign responsibilitiesDefine who uses the workflow, who reviews and approves results, and who maintains it.
2. 2Design the user interfaceGive users a clear way to provide inputs and inspect results.
3. 3Define the review processSpecify when review is required, what the reviewer must check, and how corrections and approvals are recorded.
4. 4Create audit logRecord the inputs, workflow version, outputs, and review decisions so each result can be traced back to how it was produced and approved.
5. 5Prepare for ongoing operationsDefine how the workflow will be monitored, maintained, and improved over time.

Example

A reviewer opens the proposed accounting treatment and checks the cited policy passages. They approve the treatment, make corrections, or request further information through the same interface. The audit log records their actions and final decision.