QFlowLearn: how we built QTI 3 authoring

Open assessment: Secure assessment delivery

Secure assessment delivery: an institutional evidence guide

Evaluate high-stakes assessment delivery: worker/actor architecture, durable receipts, protected scoring, recovery, accommodations, and load-test evidence.

By Sam Ottenhoff Published Updated

Summary

Reliable high-stakes assessment delivery needs an authoritative record for each attempt, durable receipts for accepted work, protected scoring content, and a tested recovery policy. A worker/actor design separates stateless request handling, one stateful authority per attempt, and asynchronous processing. Test the complete workflow under synchronized load and failure conditions before launch.

Written for: assessment program owners, academic technology leaders, information security teams, assessment platform engineers, procurement and accessibility reviewers

Define what secure delivery means for your assessment

Secure online assessment delivery means that an eligible candidate receives the approved assessment, the system records the intended response and submission once, recovery does not change the outcome, accommodations work, and staff can investigate a disputed result. The controls you need depend on what the result determines, who takes the assessment, which systems supply identity and roster information, and how a candidate can recover from a failure.

Start by documenting the decision that the assessment supports. A formative course quiz, a placement assessment, and a licensure examination do not have the same risks or response requirements. Include the consequences of an incorrect score, an unavailable service, an unauthorized attempt, an accommodation failure, and a disputed result.

Question and Test Interoperability (QTI) 3 describes assessment content, tests, results, accessibility data, and some delivery behavior. It does not assign responsibility for candidate identity, access control, monitoring, incident response, or records retention. The assessment host and the institution must define those responsibilities.

Before a pilot, test an unauthorized launch, an item with protected answer information, a reconnect during an attempt, a duplicate submission, a stale browser tab, an accommodation change, and an incident-evidence request. Record the expected and actual result for each condition.

Separate the delivery responsibilities

An assessment can be structurally valid QTI and still fail operationally. Keep the boundary between portable content and delivery controls clear before you evaluate a product or design an integration.

ResponsibilityWhat to testEvidence to retain
Identity and eligibilityOnly an eligible candidate can begin the correct assessment under the intended policy.Launch, authorization, roster, and eligibility results without exposing secrets.
Released contentEvery candidate receives the approved item, asset, rule, and accommodation version.A version identifier, release approval, package digest or equivalent record, and correction history.
Attempt stateNavigation, answers, time, breaks, and review rules behave consistently after refresh, reconnect, and restore.Ordered attempt events, state transitions, and recovery outcomes.
Submission and scoringThe host accepts the intended response once, applies the expected policy, and scores against authoritative rules.Submission receipt, response state, score disposition, and scoring version.
Accessibility and accommodationsCandidate preferences, additional time, assistive technology, and accessible interactions work in the released assessment.Test scripts, environment details, results, exceptions, and remediation owners.
Incident reviewStaff can reconstruct a material event, communicate with candidates, correct a defect, and preserve the original result.Incident timeline, affected attempts, decision record, correction or rescoring history, and retention status.

The same organization can own more than one responsibility, but the operating agreement should name one accountable owner for each one. Do not use a product feature list to assign those duties.

Use a worker/actor architecture for live attempts

When a cohort starts together, launch requests can exhaust shared database connections or queue behind the same queries. At the deadline, synchronous scoring, reporting, and learning management system (LMS) calls can delay a submission acknowledgment. A candidate who sees a timeout may retry even though the first request committed. The service must resolve that ambiguity without losing or applying the work twice.

A worker/actor architecture assigns different responsibilities to three kinds of execution:

  • A stateless ingress worker authenticates the request, validates its envelope, and routes it to the correct attempt. It does not keep the authoritative attempt in process memory.
  • An attempt actor owns one attempt’s response state, event order, timing policy, and receipts. Every tab and retry for that attempt reaches the same authority. Separate attempts use separate actors.
  • An asynchronous queue worker processes committed work later: scoring finalization, reporting projections, exports, or LMS synchronization. It does not decide whether the candidate’s response was durably accepted.

An actor is an addressable unit of state and behavior. Cloudflare’s Durable Objects supply that model with attached strongly consistent storage. One object per attempt is a natural coordination boundary: a save, a reconnect, and a time extension all affect the same candidate record. Avoid routing every candidate’s responses through one course-wide object.

Why Cloudflare fits this workload

Workers can route requests to an attempt object, while storage and asynchronous services handle different parts of the assessment lifecycle. The following is a recommended allocation of responsibilities, not a claim that choosing these services proves assessment reliability.

ServiceAssessment responsibilityBoundary to preserve
WorkersAuthenticate, validate, route, and serve authorized candidate content.Process memory is not the durable attempt record.
Durable ObjectsCoordinate one attempt and commit its state, receipt, and pending downstream work.Keep slow external calls and bulk scoring outside the local commit.
R2Store released packages, assets, snapshots, and exports.Enforce immutable version keys and authorization in the application.
D1Store assignment metadata, relational audit/reporting views, and processing status.A reporting projection can lag the attempt authority.
QueuesBuffer and deliver committed jobs to asynchronous consumers.Consumers and downstream services must make repeated effects safe.
WorkflowsOrchestrate multi-step publication, remediation, and exports.Retried steps still need idempotent external effects.
PostgreSQL through HyperdriveSupport transactional authoring, review, and revision history when that system uses PostgreSQL.Connection pooling does not remove origin-database limits or replace attempt coordination.

This allocation can reduce contention by distributing live response writes across attempt objects and deferring shared reporting writes. That is an architectural inference to test. D1 processes each database’s queries one at a time; moving every response into one reporting database would retain a shared bottleneck. Hyperdrive pools PostgreSQL connections, which helps an authoring service reuse connections without putting its revision transactions on the candidate save path.

Cloudflare’s Durable Object rules explain storage and response gates, but asynchronous external calls can still interleave application work. Persist required state and recheck relevant policy; do not rely on a single-threaded object to make arbitrary multi-service operations atomic. An object is one addressable authority, not a copy of live attempt state in every edge location. Individual objects and their shared dependencies have platform limits.

A conventional service with transactional database writes can implement the same acceptance contract. Compare it with the actor design under the same workload before claiming lower latency or cost. The actor approach adds explicit event, projection, and recovery contracts that the team must operate.

Control identity and access

An assessment launch needs to establish the candidate, course or program context, assessment assignment, and authority to begin or resume an attempt. Test each role that can view, start, submit, score, override, rescore, or export an assessment.

Use least privilege. An instructor who can review results may not need access to item answer keys. A support representative who can diagnose a launch failure may not need to see candidate responses. A delivery service should reject an attempt request that does not match the authenticated candidate, released assessment, and current policy.

For Learning Tools Interoperability (LTI) integrations, record the launch configuration, service scopes, key rotation owner, and offboarding procedure. Test access after an enrollment change, course copy, role change, dropped candidate, and expired link. The LTI Advantage implementation guide provides the integration context; it does not replace your institution’s authorization tests.

Release an approved, traceable assessment

Treat release as a boundary. Before candidates begin, capture which test, items, assets, scoring rules, accommodations, and operational policy apply. Keep that record immutable for active attempts.

If an author corrects an item after release, create a new version. Do not silently replace the content seen by candidates who already started. Define when the correction applies, how staff identify affected attempts, and whether a rescoring process is necessary. Keep the original candidate response and outcome alongside the later decision.

QTI packages can move assessment content between systems. Validate the package, its asset paths, stylesheets, custom interactions, and declared relationships before release. The QTI validation guide explains why XML, schema, semantics, accessibility, rendering, scoring, and interoperability require separate results.

For object storage, retain a version-specific manifest and content digests, and prevent replacement of released keys. R2’s consistency contract supports immediate reads of committed writes and conditional creation. It does not make an ordinary object key immutable; overwrites and cached content need explicit policy.

Keep the authoritative state on the host

The candidate browser needs enough content and state to render an item and collect a response. Keep access policy, answer keys, scoring rules, deadline decisions, and the final attempt record on the host.

Define which event establishes an attempt, saves a response, accepts a submission, returns a score, or closes an attempt. Give each event a durable identifier. When a candidate retries after a timeout or reconnect, the service must distinguish a safe repeat from a different state change.

Test the following conditions with representative items and policies:

  1. Open the assessment in two tabs, then submit from the stale tab.
  2. Disconnect during a save, refresh the browser, and reconnect.
  3. Submit twice because the candidate did not receive a response.
  4. Submit at the deadline while many other candidates submit.
  5. Restart the browser after a saved response and restore the attempt.
  6. Correct an item or answer key after an attempt has started.
  7. Review a disputed score without changing the original response record.

Record the expected result before the test. A demonstration that shows a successful submission does not prove how the system behaves during a failure.

Commit before returning a receipt

Treat a receipt as evidence of a specific committed action. A response receipt confirms a saved response; a submission receipt confirms the accepted final answer set and submission state. Neither implies that scoring or LMS grade return has finished.

The acceptance contract should define this sequence:

  1. The browser journals an event with an attempt identity, session identity, sequence number, and stable idempotency key.
  2. The attempt authority checks authorization, ownership, policy, and event order. A repeated key returns the original receipt; a conflicting reuse must not overwrite the earlier event.
  3. The authority commits the accepted state, receipt, and durable intent for downstream processing together in its local transaction.
  4. The service returns the receipt. The browser verifies which event or event range it covers before pruning the journal.
  5. Background delivery projects the committed work into reporting and processing systems. Failed delivery remains pending for retry and operational review.

Record at least the attempt and released-version identity, accepted event identity, committed sequence, receipt identifier, server acceptance time, and resulting status. These are proposed evaluation fields, not a published product API. Test a lost acknowledgment by letting the server commit and then preventing its response from reaching the browser. The retry should recover the same accepted action.

Persisting downstream intent closes a concrete failure window: the attempt can commit just before the process fails to enqueue its scoring job. A durable outbox lets later execution recover that handoff. A background callback alone does not establish a recoverable record. Verify response-save and final-submit recovery independently, including failure of reporting storage after acceptance.

Make asynchronous effects duplicate-safe

Cloudflare Queues provides at-least-once delivery. It does not provide automatic application deduplication or exactly-once processing. Give each processing operation a stable identity, enforce uniqueness where results are stored, and use a downstream idempotency key when the destination supports one. Test interruption after an effect succeeds but before the consumer acknowledges the message.

Queue delivery order is not guaranteed. Consumers must use the committed event sequence or result version to reconcile ordering rather than assuming arrival order is authoritative.

Apply the same discipline to Workflows steps, which can retry. Track failed jobs, retry exhaustion, and dead-letter handling. Queue acceptance is a handoff, not proof that scoring, export, or grade return completed.

Final submission should commit promptly without waiting for those jobs. An adaptive item can still require trusted processing during the attempt to determine its next presentation; that work needs its own measured latency budget. Isolate bulk scoring, reports, exports, and LMS synchronization so their failures do not delay ordinary response receipts.

Preserve candidate access during recovery

Recovery has technical and academic consequences. A system may restore the last saved answer, accept a late submission under a policy, reopen an attempt, or require staff review. Define those outcomes in advance and show candidates what happened.

Test browser storage and server-side state separately. Browser storage can improve recovery from a connection loss, but the institution still needs a server-side record that can resolve duplicate requests, browser changes, and support cases. Test a new browser, a cleared browser profile, and a device change if your policy permits candidates to use them.

Give candidates a receipt after a material action such as final submission. Give staff an event trail that answers when the candidate started, saved, submitted, received a result, or encountered an error. Do not expose other candidates’ data, internal secrets, or answer information in either record.

An IndexedDB journal can retain unacknowledged work across a reload in the same browser profile. Show candidates whether work is recorded locally, confirmed by the server, or awaiting recovery. Reconnect must reconcile the server’s committed sequence and replay missing events in order; a gap or stale session must not silently replace newer work.

Browser storage has quota and persistence limits. Test unavailable storage, a cleared profile, and a failed local write. A journal cannot recover unsent work from a lost device or guarantee access to items that were never downloaded. Continuing a cached active attempt and starting a complete exam offline are different capabilities.

For a lab outage, rehearse a cohort losing connectivity, recording answers, selecting Finish, reloading, and reconnecting together. Preserve the captured Finish answer set and show when it still lacks a server receipt. Record which attempts recover automatically and which require a supervised reopen or workstation transfer. Assign staff responsibility for each unresolved attempt.

Keep timing and accommodation decisions on the server

The attempt authority should track start time, elapsed time, deadlines, pauses, and approved extensions. The browser displays that policy; its clock and a client-supplied timestamp cannot establish eligibility for late work. Record the policy revision applied to each material transition, including a staff extension or reopen.

Define what happens when Finish is selected before a deadline but reaches the server afterward. Preserve available work and evidence for the agreed adjudication process without silently awarding more time. Test deadline wake-up without another browser request, a simultaneous extension and submit, and replay from a stale tab. Recovery must preserve approved accommodations and create an attributable record of any policy change.

Durable Object alarms can wake the authority without a browser request. Alarm execution is at least once, so the handler must recheck the current deadline and make repeated closure safe. Deadline closure and retrying downstream work must not postpone one another.

Deliver accommodations in the released workflow

Accessibility and accommodations are delivery requirements. Test them in the same launch, navigation, save, review, and submission flow that candidates use. A component demonstration cannot establish that the complete assessment process works.

Test representative candidate preferences, additional time, breaks, keyboard operation, screen-reader name and state, error feedback, zoom, contrast, and media alternatives. QTI 3 includes accessibility and Personal Needs and Preferences structures, but the delivery host must apply them correctly. WCAG 2.2 also evaluates complete processes, not isolated screens.

Keep private accommodation information out of portable QTI content and limit access to the staff and services that require it. Document where each preference is stored, who can change it, and how a change affects an active attempt.

Plan for incidents and result challenges

Define the incident process before a high-consequence assessment. Name who detects a problem, who decides whether to pause or continue, who communicates with candidates, and who approves a correction or rescoring decision.

For every material incident, preserve the assessment version, candidate state, service status, staff actions, and final decision. Keep the original score when a later correction produces a different outcome. A candidate, reviewer, or auditor should be able to understand the event without reconstructing it from informal messages.

Test the process with a tabletop exercise and a representative recovery scenario. Include a content error, an integration outage, a candidate-specific accommodation issue, and a delayed submission. The goal is to verify responsibilities and evidence, not to simulate every possible attack.

Protect scoring content and state the evidence boundary

Candidate-safe delivery requires more than hiding an answer in the interface. Keep trusted scoring XML, answer mappings, and unreleased feedback behind server authorization. Score validated responses against the approved content version, and release review material under an explicit policy. A rescoring job should retain the original response, original result, corrected scoring version, reason, actor, and later result.

Longsight’s public qti3 code supplies inspectable item-level evidence. At the reviewed 0.13.4 commit, delivery-security tests check removal of protected scoring and feedback subtrees and explicit refusal of unsupported static delivery cases. Server-scoring tests check scoring against authoritative XML and ignoring forged outcome values supplied with responses. Feedback-release browser tests check a host-owned release gate; the host still protects source and outcome access.

The November 9 execution record reports 3,487 passing unit tests, three skipped external-certification tests, 664 passing browser tests, and two Firefox launch failures. The complete release gate did not pass. The implementation matrix separates these library capabilities from platform duties. None of those totals establishes durable host receipts, outage recovery, accessibility conformance, or production capacity.

QFlowLearn’s delivery documentation describes per-attempt authority and recovery behavior, but marks its public load-test summary as in progress. Product documentation is attributable implementation information, not a reproducible benchmark. Request the applicable evidence packet and test conditions before accepting its capacity claims. The vendor-neutral architecture above remains a recommendation to validate in the intended deployment.

Measure synchronized load and recovery

Publish results that another assessment team can reproduce. Compare implementations using the same item mix, payload sizes, candidate regions, timing policies, launch window, and submit window. Include cold and warm content access and degraded dependencies. Record the exact build and platform configuration; a test script by itself supplies no measured result.

Require a report with the following measurements:

  • Launch-to-ready and response/submit-to-durable-receipt latency at p50, p95, and p99, with timeout and error counts.
  • Total attempted, receipt-confirmed, rejected, recovered, and unresolved attempts, including candidates still missing a final receipt when observation ends.
  • Response integrity after duplicate, out-of-order, stale-tab, reload, and reconnect sequences; no average latency can establish that work survived.
  • Queue backlog and drain time, final-score latency, and LMS synchronization lag measured separately from submission acceptance.
  • Recovery time after a cohort outage, with missing work, manual interventions, and unresolved cases reported explicitly.
  • Measured requests, compute, storage operations, queue retries, retained data, and database use, with prices and assumptions dated for each cost comparison.

Set acceptance thresholds before running the test. Include reporting or LMS unavailability and a slow scoring consumer. Identify dependencies still required for first launch and policy changes; success during an established attempt does not prove that those operations survive the same outage. This guide presents no measured comparison that establishes lower cost or greater capacity for Cloudflare.

Request evidence with the right scope

Ask suppliers and internal teams for evidence that matches the claim. A certification record, a validation result, a load test, an accessibility report, and an incident exercise answer different questions.

Request the following materials for the exact product version and deployment model under review:

  • An architecture diagram showing identity, launch, delivery, scoring, storage, monitoring, and support boundaries.
  • A support matrix that distinguishes implemented, tested, partial, unsupported, and externally verified capabilities.
  • Representative QTI packages, expected scoring outcomes, and import or delivery results.
  • A release record that ties item revisions, assets, policy, and approval to the candidate experience.
  • Recovery test results for disconnects, duplicates, browser restarts, deadlines, and service restoration.
  • Accessibility test results for representative item families and candidate workflows.
  • Security and authorization test evidence, including administrator and support roles.
  • Incident, correction, rescoring, data-export, and retention procedures.

Verify current certification independently. 1EdTech’s QTI conformance and certification information identifies the official program; it does not establish that a different product, release, profile, or deployment is certified.

Set acceptance criteria before procurement or launch

Write acceptance criteria that name the content, candidates, policies, integrations, environments, and failure cases you will test. Use your institution’s real items and roles. Define who can approve an exception, how long it may remain open, and what happens if a material check fails.

Use the secure assessment delivery acceptance test plan to prepare the evaluation. For product-specific implementation and recovery details, review QFlowLearn’s assessment delivery documentation. Test any product against your institution’s own acceptance criteria.

Limits of this guidance

  • This guide is a technical and operational framework, not legal advice or a certification claim.
  • Security controls must match the assessment's purpose, candidate population, policy, and threat model. A control that is suitable for low-stakes practice may be unsuitable for a licensure or admissions decision.
  • A successful pilot does not prove resilience during production load, an incident, an accommodation change, or an attack.
  • QTI can describe assessment content and some delivery behavior. It does not replace institutional policy for identity, authorization, incident response, or records retention.
  • The worker/actor design is an architectural recommendation. Public qti3 tests cover item behavior; they do not qualify a delivery platform for production load or outage recovery.

Procurement checklist

  • Define the decision, candidate population, assessment consequences, threat model, and acceptable failure behavior.
  • Test identity, authorization, launch, candidate eligibility, and administrator access with representative roles.
  • Require an immutable record of the released assessment version, item assets, rules, and approval state.
  • Verify that scoring uses authoritative content and that candidate-facing delivery does not disclose protected answer information.
  • Test reconnect, duplicate submission, deadline, stale-tab, browser restart, and service-recovery behavior.
  • Require synchronized launch and submit results with receipt latency percentiles, unresolved attempts, recovery outcomes, and measured workload costs.
  • Test keyboard use, screen-reader output, time accommodations, breaks, and candidate preferences in the same delivery workflow.
  • Require a reviewable event record for submission, scoring, correction, rescoring, and result challenges.
  • Name incident owners, communication rules, evidence retention, data-export responsibilities, and recovery-test frequency.

Downloads

Related code

Standards and sources

  1. [1]Web Content Accessibility Guidelines 2.2. W3C requirements for complete web processes, including assessment workflows.
  2. [2]LTI Advantage implementation guide. 1EdTech guidance for launch, platform and tool responsibilities, and service access.
  3. [3]QTI accessibility resources. 1EdTech resources on accessibility support in QTI implementations.
  4. [4]QTI 3 Best Practices and Implementation Guide. 1EdTech guidance for QTI 3 tests, navigation, submission, accessibility, accommodations, packaging, and security considerations.
  5. [5]QTI conformance and certification. Current 1EdTech information about validation, conformance, and certification.
  6. [6]OWASP Authorization Cheat Sheet. General application-security guidance for authorization checks and least privilege.
  7. [7]Cloudflare Durable Objects and the actor model. Addressable stateful compute with attached strongly consistent storage.
  8. [8]Cloudflare Queues delivery guarantees. At-least-once delivery requires application idempotency for duplicate-safe effects.