← Case studies
CASE STUDY · FINANCIAL SERVICES

Credit decision support: what the audit found, and how we fixed it

We ran our audit tool on a credit-decision assistant. This case study sets out what the audit found, what it means for a lender, and what our forward-deployed engineers did after the report.

CASE STUDY 01 FINANCIAL SERVICES EU AI ACT ANNEX III ISO/IEC 42001
  • Tier High-risk Annex III(5), essential services
  • AI components detected 2 One AI SDK, one vector store
  • Findings 2 Both medium
  • Annex III point 5 Essential services, including creditworthiness

At a glance

Sector
Financial services, consumer lending to natural persons, including creditworthiness assessment
System
A credit-decision assistant. It takes in applications, calls Google's AI SDK (google.genai) and connects to a Qdrant vector store.
EU AI Act scope*
Annex III, point 5: essential services, including creditworthiness. The audit tool places the use in the high-risk tier.
What was audited
The AI components, the risk tier, and the records, human oversight and documentation the use would need
What the team now has
A decision record with fixed fields, a technical file for each release, a tested stop path and a named monitoring owner
Stack detected
Google AI SDK (google.genai)
Qdrant (qdrant_client)

Context

A system that helps decide who gets a loan can change a person's life.

The EU AI Act places systems that assess creditworthiness under Annex III, point 5. That brings duties on data, records, human oversight and explanations (Articles 9 to 15, 26 and 86). Data protection law adds its own safeguards for automated decisions (GDPR Article 22).

Lenders and credit teams ask the same four questions:

  • Which AI models, SDKs and data stores does the system use?
  • Who checks a decline before the applicant hears it?
  • What do we keep, for how long, and who can see it?
  • What do we tell an applicant who asks why?

What we audited

Our audit tool reads the code.

It builds an inventory of AI components (SDKs, models, vector stores and prompt templates), places the use in an EU AI Act tier with the Annex III point that applies, and lists what the team must show for that tier. Each finding is tied to an article or control and given a severity. Each one goes to the team with the evidence behind it.

  • AI inventory

    Which AI SDKs, vector stores and prompt templates the system uses, and where they sit in the code.

  • Risk tier

    Whether the use is high-risk under Annex III, and which point applies.

  • Obligations to evidence

    Records of each decision (Article 12), human oversight (Article 14), technical documentation (Article 11 and Annex IV) and explanations (Article 86).

  • Next steps

    Items that sit outside the code, such as what runs in production, what data is stored and who can stop the system.

What the audit found

The two medium findings concern what data leaves the system and what is kept.

  • MEDIUMModel calls in the application code

    AI SDK detected: Google (google.genai)

    Why it mattersEach call sends application data to an outside provider. The decision record needs the model version, and the data sent needs a lawful basis.

  • MEDIUMThe memory module

    Vector store detected: Qdrant (qdrant_client)

    Why it mattersStored vectors can carry personal data. Article 10 data governance and GDPR apply, and retention has to be set.

  • TIERAudit summary

    Risk tier: high-risk, Annex III(5)

    Why it mattersBrings duties on data, records, human oversight and explanations (Articles 9 to 15, 26 and 86)

Open items

  • OPEN ITEM

    Next step: decision records

    Not visible in the code Article 12 needs an automatic log of each decision, with fixed fields

  • OPEN ITEM

    Next step: override and stop

    Reviewer workflow Article 14(4)(d) and (e): a person can override the output and stop the system

  • OPEN ITEM

    Next step: technical documentation

    Release process Annex IV documentation should match the version that runs

  • OPEN ITEM

    Next step: explanations for applicants

    Decision output Article 86 and GDPR Article 22 safeguards for automated decisions

The open items are the questions a lender is asked next: who decides, what is recorded, and what the applicant is told.

What we did next

After the report, our forward-deployed engineers either fixed the issues hands-on or advised and worked alongside the team to close them.

That is how we work. Each finding is triaged by severity with the team. An engineer then either fixes it hands-on, or advises and works with the team to close it. The team keeps the decisions that belong to a lender, such as the legal basis and the wording of an explanation. After the change, we run the audit tool again on the same system, and the re-audit report goes into the technical file.

From audit report to technical file A flow from the audit report, with findings ranked by severity, to a triage step the team and an engineer share. From triage the flow splits: engineers fix some findings hands-on, and for others they advise the team, which keeps the decisions on wording and legal basis. Both paths lead to a re-audit of the same system, and the re-audit report goes into the technical file. The two human steps are highlighted. FROM AUDIT TO REMEDIATION AUDITREPORTFindings withseverity and evidence HUMANTRIAGE BYSEVERITYTeam and engineeragree the order FIXHANDS-ONEngineers changecode and config HUMANADVISETHE TEAMTeam decides wordingand legal basis RE-AUDITSAME SYSTEMSame tool, rerunson the release TECHNICALFILERe-audit reportadded to the file Ranked Code fix Advise Checked From audit report to technical file A flow from the audit report, with findings ranked by severity, to a triage step the team and an engineer share. Engineers then fix some findings hands-on, and the team decides the wording and legal basis for the others. The findings are re-audited on the same system, and the re-audit report goes into the technical file. The two human steps are highlighted. AUDIT TOOLAUDIT REPORTFindings with severity and evidence TEAM AND ENGINEER (HUMAN)TRIAGE BY SEVERITYOrder agreed before any change ENGINEERSFIX HANDS-ONModel version, retention, deletion TEAM (HUMAN)ADVISE THE TEAMTeam decides wording and legal basis AUDIT TOOLRE-AUDITSame tool, same system TECHNICAL FILERE-AUDIT REPORT FILEDAdded to the file for the release Ranked by severity Code fix Or advise the team Both paths Checked
Each finding is triaged with the team, fixed or advised on, and re-audited before it goes into the technical file.
  • Fixed hands-on

    Our engineers put the model version into every decision record, so each decision can be traced to the model that made it. They also moved the Qdrant connection behind a configuration file, with a retention setting and a route for deleting records about a person.

  • Advised and worked alongside the team

    Our engineers worked with the team on the fields of the decision record, the override reason and the stop control. The team's risk and legal staff decided the wording of explanations and the legal basis for sending data to the provider.

  • What we would advise a lender

    • Record the model version with every decision.
    • Set a retention period for any vector store that holds personal data.
    • Require a reason for every override. A blank field is a gap.
    • Test the stop control before each release, and keep the test record.
    • Keep the technical file in step with the model version.

What the team now has

  • A decision record.

    Each decision writes fixed fields, including the model version and any override reason.

  • A retention rule for the vector store.

    Retention is set, and a route exists for deleting a person's records.

  • A technical file for each release.

    It matches the version that was tested, and a person signs it off.

  • A tested stop path.

    Reviewers can override a recommendation, record why, and stop the system.

  • Monitoring with a named owner.

    Drift, bias and error alerts go to a person, and each change follows the same loop.

How it is built

The sections below describe how the audited system is built and how we would govern it, using the components the audit detected.

Architecture

The system has two layers. The top layer is the decision flow. Data comes in from sources, passes through ingestion and quality gates, and reaches a feature store. The scoring service and the decision service then use those features. Loan officers work in the reviewer console.

The bottom layer is governance. It holds the model registry, the monitoring service, the event log store and the audit export. It watches the decision flow and keeps the record. It does not replace the checks in the flow itself.

How the credit system is built The diagram shows the decision flow across the top, from data sources through ingestion and quality gates, a feature store, the scoring service and the decision service to the reviewer console. A governance band runs underneath with the model registry, monitoring, the event log store and the audit export. Human steps are highlighted. DECISION FLOWDATA SOURCESApplications,bureau, accountsINGESTION ANDGATESSchema, rangeand freshnesschecksFEATURE STOREVersionedfeatures withlineageSCORINGSERVICEScore andreason codesDECISIONSERVICERules, routing,decision recordREVIEWERCONSOLE(HUMAN)Checks, override,stopFeeds inValidateddataFeaturevectorsScore andreasonsReferredcasesOverrideor stopGOVERNANCE PLANEMODELREGISTRYVersions, owners,approvalsMONITORINGDrift, bias anderror alertsEVENT LOGSTOREAppend-onlydecision recordsAUDIT EXPORTEvidence packper releaseApproved modelScores andinputsEachdecisionEach human actionExport onrequest How the credit system is built The diagram shows the decision flow across the top, from data sources through ingestion and quality gates, a feature store, the scoring service and the decision service to the reviewer console. A governance band runs underneath with the model registry, monitoring, the event log store and the audit export. Human steps are highlighted. DECISION FLOWDATA SOURCESApplications, bureau, accountsINGESTION AND GATESSchema, range and freshness checksFEATURE STOREVersioned features with lineageSCORING SERVICEScore and reason codesDECISION SERVICERules, routing, decision recordREVIEWER CONSOLE (HUMAN)Checks, override, stopFeeds inValidated dataFeature vectorsScore and reasonsReferred casesOverride or stopGOVERNANCE PLANEMODEL REGISTRYVersions, owners, approvalsMONITORINGDrift, bias and error alertsEVENT LOG STOREAppend-only decision recordsAUDIT EXPORTEvidence pack per release
The decision flow runs left to right. The governance band records every decision and watches the flow.

How a decision flows

An application is checked against basic rules. The model then scores it and gives the main reasons. An approval goes to the applicant, and the decision is recorded.

A recommended decline goes to a loan officer first. The officer can confirm the decline or overturn it. Either way, the applicant receives the decision and the reasons.

An applicant who disagrees can ask for a review. A reviewer who was not involved in the first decision looks at the case again. Every step writes to the event log.

How a decision flows, with human checks A swimlane diagram with five lanes: applicant, loan officer, scoring service, reviewer and record. An application is scored. Approvals go straight to the applicant. A recommended decline goes to a loan officer, who can confirm or overturn it. An applicant who disagrees can ask for review by a separate reviewer. Every step writes to the record. APPLICANTLOAN OFFICER(HUMAN)SCORINGSERVICEREVIEWER(HUMAN)RECORDSUBMITAPPLICATIONIncome,purpose,historySCORE ANDEXPLAINInputs checkedagainst rulesRECOMMENDATION?Approve ordeclineREVIEWDECLINECase andreasonsOVERRIDE?Confirm oroverturnRECEIVEDECISIONDecision andreasonsASK FORREVIEWApplicantdisagreesINDEPENDENTRE-REVIEWNot involved infirst decisionSTOREEXPLANATIONReasons andmain factorsWRITE EVENTLOGEvery step,with time stampSendapplicationScorereadyApproveDeclineCheckcaseConfirm declineOverturnto approveDisagreesRequest reviewRevised decisionSave reasonsLog decisionLog overrideLog outcome How a decision flows, with human checks A swimlane diagram with five lanes: applicant, loan officer, scoring service, reviewer and record. An application is scored. Approvals go straight to the applicant. A recommended decline goes to a loan officer, who can confirm or overturn it. An applicant who disagrees can ask for review by a separate reviewer. Every step writes to the record. APPLICANTSUBMIT APPLICATIONIncome, purpose, historySCORING SERVICESCORE AND EXPLAINInputs checked against rulesSCORING SERVICERECOMMENDATION?Approve or declineLOAN OFFICER (HUMAN)REVIEW DECLINECase and reasonsLOAN OFFICER (HUMAN)OVERRIDE?Confirm or overturnAPPLICANTRECEIVE DECISIONDecision and reasonsAPPLICANTASK FOR REVIEWApplicant disagreesREVIEWER (HUMAN)INDEPENDENT RE-REVIEWNot involved in first decisionRECORDSTORE EXPLANATIONReasons and main factorsRECORDWRITE EVENT LOGEvery step, with time stampSend applicationScore readyDeclineCheck caseConfirm declineOverturn toapproveDisagreesRequest reviewApproveRevised decision
Every recommended decline is checked by a person before it is sent. Each step is logged.

Human oversight and explanations

Article 14 asks that people can understand the system, notice when they rely on it too much, interpret its output, and override or stop it. Article 26(2) asks deployers to give oversight to people with the right competence, training and authority.

The reviewer console is built for this job. It shows the score, the main reasons, data quality flags and the model version. It records every override with a reason. It has a stop control, which is tested before handover.

Article 86 gives a person the right to clear and meaningful explanations when a decision based on a high-risk system's output significantly affects them. GDPR Article 22 adds safeguards for decisions based solely on automated processing. The build stores the reasons for each decision, so a plain explanation can be given on request. Wording is checked with your legal advisers.

Data and model governance

Article 10 asks for careful data governance. The build covers four things:

  • Lineage

    Every feature can be traced back to its source, and to each change made to it.

  • Quality gates

    Missing, out-of-range or stale data is flagged or stopped at ingestion.

  • Bias checks

    Outcomes are compared across groups where your data and legal basis allow it. Where they do not, the file records the gap and the reason.

  • Drift checks

    Inputs and scores are compared with a baseline before each release and during use.

Accuracy, robustness and security tests (Article 15) run before each release. The model registry records each version, its data, its owner and its approval.

Keeping the model under watch

How the model is watched and changed A loop of eight steps around a circle. Monitoring checks scores; an alert leads to a review by the owner; the review either closes or leads to a change request; the change is tested again, approved by a named person, released and logged, then returned to monitoring. Two steps are human. Monitor scoresDrift, bias and error checksAlert raisedDrift, bias or incidentReview (HUMAN)Owner triages the caseChange requestWhat changes and whyRevalidateTests, bias and drift checksApprove release (HUMAN)Named person signs offReleaseNew model versionLog changeEvent log and technical fileThreshold crossedTicket openedChange neededNo change neededTest the changeTest reportApprovedRecord releaseBack to watch How the model is watched and changed A loop of eight steps around a circle. Monitoring checks scores; an alert leads to a review by the owner; the review either closes or leads to a change request; the change is tested again, approved by a named person, released and logged, then returned to monitoring. Two steps are human. Monitor scoresDrift, bias and error checksAlert raisedDrift, bias or incidentReview (HUMAN)Owner triages the caseChange requestWhat changes and whyRevalidateTests, bias and drift checksApprove release (HUMAN)Named person signs offReleaseNew model versionLog changeEvent log and technical fileThreshold crossedTicket openedChange neededTest the changeTest reportApprovedRecord releaseNo changeneededBack to watch
Every change goes round the loop: tested again, approved by a person, and logged.

When monitoring shows drift, a bias alert or an incident, the case goes to review. If a change is needed, a change request follows. The change is tested again, approved by a named person and released. Every step is logged.

Logging and evidence trail

Article 12 requires high-risk systems to keep automatic logs. Article 26(6) asks deployers to keep the logs under their control for at least six months. Article 19 sets the same minimum for providers. Your policy can set a longer period, but the build never keeps logs for less than six months.

Each decision writes a record with these fields:

  • Time stamp and request ID
  • Model name and version
  • A reference to the input record, not the raw personal data
  • Score, reason codes and data quality flags
  • Decision and route (approved, declined or referred)
  • Pseudonymised reviewer ID, with any override and its reason
  • The explanation sent, and when it was sent
  • The release ID of the technical file the record belongs to

The technical file

What sits in each release’s technical file A tree with the technical file at the top and nine blocks below it. The blocks cover the general description, development record, data and governance, oversight and instructions, accuracy and testing, risk management, changes over time, event logs, and assessments. Two blocks are highlighted for human review. TECHNICAL FILEAnnex IV, one per releaseGENERAL DESCRIPTIONPurpose, provider, versionsDEVELOPMENT RECORDDesign choices and build stepsDATA AND GOVERNANCESources, lineage, bias checksOVERSIGHT AND INSTRUCTIONSOversight design, instructions for useHUMANACCURACY AND TESTINGTest reports and robustnessRISK MANAGEMENTRisk register and decisionsCHANGES OVER TIMEChange log per releaseEVENT LOGSFixed fields, kept six months or moreASSESSMENTSArticle 27 and DPIA recordsHUMAN What sits in each release’s technical file A tree with the technical file at the top and nine blocks below it. The blocks cover the general description, development record, data and governance, oversight and instructions, accuracy and testing, risk management, changes over time, event logs, and assessments. Two blocks are highlighted for human review. TECHNICAL FILEAnnex IV, one per releaseGENERAL DESCRIPTIONPurpose, provider, versionsDEVELOPMENT RECORDDesign choices and build stepsDATA AND GOVERNANCESources, lineage, bias checksOVERSIGHT AND INSTRUCTIONSOversight design,instructions for useHUMANACCURACY AND TESTINGTest reports and robustnessRISK MANAGEMENTRisk register and decisionsCHANGES OVER TIMEChange log per releaseEVENT LOGSFixed fields, keptsix months or moreASSESSMENTSArticle 27 and DPIA recordsHUMAN
The technical file is built per release. Most blocks come from the build’s own records.

The technical file is the document set that Annex IV lists for high-risk systems. The build generates most of it from the records above, so each release has a file that matches what was tested. A person reviews the draft before it is signed off.

Fundamental rights impact assessment and DPIA

Article 27 asks deployers of Annex III point 5 systems to assess the impact on fundamental rights before first use. The assessment describes the process the system supports, how often it runs and who is affected. It also covers the specific risks of harm, the human oversight measures and what happens if the risks appear.

A GDPR data protection impact assessment (DPIA, Article 35) covers the risks to personal data. Article 27(4) allows parts of a DPIA to count towards the impact assessment, where the DPIA already covers the same duties.

We design the two assessments together:

  • The data inventory and data flows feed the DPIA. The affected groups and possible harms feed the Article 27 assessment.
  • The oversight measures in the assessment match what the reviewer console does.
  • Validation test reports feed the risk sections of both documents.
  • At handover, both documents are final, signed off and linked to the release. Article 27(2) requires them to be updated when a change affects them.

EU AI Act obligations mapped to controls

From the Act’s duties to the controls and the evidence A map in three columns. The left column lists groups of articles, the middle column lists the control built for each group, and the right column lists the evidence each control produces. Four rows cover risk and quality, data, documentation and oversight, and deployer duties with explanations. ARTICLESCONTROL WE BUILDEVIDENCE PRODUCEDARTICLES 9 AND 17Risk and quality managementRISK GATERisk register checked at each releaseRISK FILEPer release, with QMS recordDesigned intoProducesARTICLE 10Data governanceDATA GATELineage, quality, bias, driftDATA SHEETData sheet and bias reportDesigned intoProducesARTICLES 11 TO 15Documentation, logs, oversight, accuracyREVIEWER CONSOLE (HUMAN)Fixed log fields; override with reasonsTECHNICAL FILEAnnex IV pack and six-month logsDesigned intoProducesARTICLES 26, 27, 86Deployer duties, assessment, explanationsand GDPR Articles 22 and 35ASSESSMENT WORKFLOW (HUMAN)FRIA, DPIA and explanation workflowASSESSMENT RECORDApproved FRIA and DPIA; explanation logDesigned intoProduces From the Act’s duties to the controls and the evidence A map in three columns. The left column lists groups of articles, the middle column lists the control built for each group, and the right column lists the evidence each control produces. Four rows cover risk and quality, data, documentation and oversight, and deployer duties with explanations. ARTICLES 9 AND 17Risk and quality managementDesigned intoRISK GATERisk register checked at each releaseProducesRISK FILEPer release, with QMS recordARTICLE 10Data governanceDesigned intoDATA GATELineage, quality, bias, driftProducesDATA SHEETData sheet and bias reportARTICLES 11 TO 15Documentation, logs, oversight, accuracyDesigned intoREVIEWER CONSOLE (HUMAN)Fixed log fields; override with reasonsProducesTECHNICAL FILEAnnex IV pack and six-month logsARTICLES 26, 27, 86Deployer duties, assessment, explanationsand GDPR Articles 22 and 35Designed intoASSESSMENT WORKFLOW (HUMAN)FRIA, DPIA and explanation workflowProducesASSESSMENT RECORDApproved FRIA and DPIA; explanation log
Each row links a group of duties to a control in the build and to the evidence the control produces.
Each obligation, the control we build for it, and the evidence it produces
ObligationControl we buildEvidence produced
Art 9 risk managementRisk register and release gateRisk file per release
Art 10 data governanceLineage, quality gates, bias and drift checksData sheet and bias report
Art 11 and Annex IV technical documentationDocumentation generated from the buildTechnical file per release
Art 12 record-keepingEvent log with fixed fieldsLog export and field schema
Art 13 transparency to deployersModel card and instructions for useInstructions for use
Art 14 human oversightReviewer console, override and stop controlOverride log and oversight design
Art 15 accuracy, robustness, cybersecurityTest suites, input checks, access controlsTest reports and access review
Art 17 quality management (providers)Change and document control, aligned to ISO/IEC 42001Change and document records
Art 26 deployer dutiesMonitoring, reviewer briefing, log retentionMonitoring reports and logs kept six months or more
Art 27 impact assessmentAssessment workflowApproved assessment and notification record
GDPR Articles 22 and 35Decision safeguards and DPIA workflowDPIA and explanation records
Art 86 explanationsExplanation built from stored reasonsExplanation record for each request

Which duties apply depends on whether you develop the system yourself (provider) or use it under your authority (deployer). We map the build to your role first.

Delivery plan

We work in five phases. Each phase ends with a review. Sprints run to two weeks.

  1. Discover

    Confirm the use case, the data sources and your role. Check the Annex III position.

  2. Design

    Agree the architecture, the oversight design and the scope of both assessments.

  3. Build

    Build the data pipelines, scoring service, reviewer console and event log. Draft the technical file as the build goes.

  4. Validate

    Run the tests and the bias and drift checks. Complete the impact assessment and DPIA drafts.

  5. Handover

    Transfer the pack, run the handover sessions and agree the monitoring owner, as set out in your contract.

Five phases from discovery to handover A timeline of five phases shown as bars in sequence: Discover, Design, Build, Validate and Handover. Under each phase is what the team produces. A strip under the Build phase notes short sprints. Bar lengths show order only, not time. DISCOVERUse case, data, roleUSE CASE BRIEFScope, role,Annex III checkReviewDESIGNArchitecture and controlsARCHITECTURE ANDCONTROLSOversight andassessment scopeReviewBUILDPipelines, service,console, logsWORKING BUILDPipelines, service,console, logsReviewVALIDATETests, bias and drift checksTEST PACKTests, reports,assessment draftsReviewHANDOVERContract scope and sessionsHANDOVER PACKPer contract; runbooks,sessionsShort sprints,two weeks each Five phases from discovery to handover A timeline of five phases shown as bars in sequence: Discover, Design, Build, Validate and Handover. Under each phase is what the team produces. A strip under the Build phase notes short sprints. Bar lengths show order only, not time. DISCOVERUse case, data, roleUSE CASE BRIEFScope, role, Annex III checkReviewDESIGNArchitecture and controlsARCHITECTURE AND CONTROLSOversight and assessment scopeReviewBUILDPipelines, service, console, logsWORKING BUILDPipelines, service, console, logsShort sprints of two weeksReviewVALIDATETests, bias and drift checksTEST PACKTests, reports, assessment draftsReviewHANDOVERContract scope and sessionsHANDOVER PACKPer contract; runbooks, sessions
Each phase ends with a review. Bar lengths show order only.

Handover pack

  • Code and infrastructure definitions for the scoring and decision service
  • The reviewer console and its user guide
  • The event log schema and export tool
  • The technical file for the release
  • The Article 27 assessment and DPIA, with the sign-off record
  • The monitoring runbook and alert rules
  • The model card and instructions for use
  • Briefing notes for reviewers

Questions teams ask us

No single build does that. It puts the controls and the evidence in place, which helps you prepare for the duties that apply. Your legal, risk and data teams still decide the use, the legal basis and the sign-off.
Both paths exist. Our forward-deployed engineers fix some issues hands-on. For others, they advise and work alongside your team, so the decisions stay with you.
It depends on whether you develop the system or use it under your authority. The duties differ, so we map the build to your role first.
Where Article 86 applies, an affected person can ask for one. The build stores the reasons, so a plain explanation can be given. Wording goes through your legal advisers.
At least six months for deployers under Article 26(6). Your policy can set a longer period.
Yes. Each change follows the loop in the monitoring and change loop diagram. Tests run again, a named person approves, and the technical file and event log are updated.

Talk through a system like this

Bring your scoring pipeline and your review process. We will run the same audit tool on it and walk through the findings, the fixes and the handover pack with you.

← All case studies

* EU AI Act scope: Annex III, point 5 (essential services, including creditworthiness assessment). The tier is the one the audit tool returned for this system.