Define what your institution is buying
Question and Test Interoperability (QTI) defines a format for exchanging assessment content. Your purchase can include an authoring tool, an item bank, a delivery service, or several connected systems. Before you write the request for proposal (RFP), identify which supplier is responsible for each part. The open assessment infrastructure guide explains those responsibilities.
Start with the workflows your institution must preserve. List your existing item banks, the people taking the assessments, testing conditions, integrations, and export destinations. Before reviewing supplier responses, mark each requirement as mandatory or optional.
Download the evaluation worksheet (plain text) and use one copy per supplier. Score technical requirements separately from price. Record any unmet mandatory requirement, even when a supplier offers a lower price.
Specify the QTI role, version, and features
1EdTech’s procurement guidance recommends specifying the application role, QTI version, certification level or profile, and required features. It also calls for a current certification registration number and representative sample content. Copy the exact profile terminology from the applicable certification documents when preparing your RFP.
Ask each supplier to complete a requirements table:
| Purchased function | Requirement to define | Evidence to request |
|---|
| Authoring | Required interactions, scoring rules, media, metadata, and review workflow | Editable examples, revision history, exported packages, and validation results |
| Item bank and test construction | Item reuse, identifiers, permissions, provenance, and test structure | A bank sample, an assembled test, an export inventory, and documented limits |
| Delivery | Supported content, accommodations, navigation, timing, and submission policy | Candidate demonstrations and recorded acceptance tests |
| Scoring and reporting | Expected outcomes, partial credit, manual scoring, corrections, and grade return | Known responses with expected scores, actual results, and correction records |
| Exchange | Required import sources and export destinations, including exact versions | Source packages, conversion diagnostics, and destination test results |
For every row, record the product version and whether the capability is available, partial, planned, or unsupported. Label planned work as such. If delivery or scoring is a separate service, request evidence for that service.
Verify the certification claim
Use the 1EdTech certification resources to find the applicable conformance documents and current directory listing. Save the listing URL, registration number, product name, role, QTI version, profile, and date checked. Ask the supplier to explain whether the listing covers the release you would buy.
If certification is mandatory and a supplier cannot provide it, record the requirement as unmet. If your institution accepts a promise of future certification, document who approved the exception, the deadline, and what happens if the supplier misses it.
Maintain separate records for certification, supplier tests, and tests your institution runs across systems. Your technical team can use the QTI validation guide to check the file format, question behavior, scoring, and transfer between systems.
Test import, editing, export, and delivery
Choose content that exercises your difficult cases: partial credit, multiple responses, mathematics, shared resources, local media, accommodations, and legacy imports where required. Use synthetic or approved test content without learner data. Agree on expected outcomes before the supplier runs the test.
- Inventory the source package, item identifiers, assets, scoring rules, and required metadata.
- Import it into the proposed product and save all diagnostics, including omitted or transformed content.
- Make the agreed edits in the authoring tool.
- Export the assessment package.
- Have your technical team run the agreed checks and record the tools and QTI versions used.
- Import the export into the intended destination and inspect the content and assets.
- Submit known responses and compare the scores and candidate experience with the expected results.
- Record every loss, workaround, and manual repair, including who must resolve it and whether your institution accepts it.
If the intended destination is unavailable, mark that test as not tested. Screenshots and imports back into the source product do not show whether the destination can use the content. Use the QTI migration guide to plan the inventory and compare the transferred content with the source.
Check what the supplier has tested
Ask the supplier to show that your questions, scoring rules, and accessibility settings work after a move to the system you plan to use. Ask what did not transfer correctly and whether your team will need extra tools or manual fixes.
Your technical team can use Longsight’s QTI 3 test results and known limits as an example when reviewing supplier evidence.
Test accessibility in the purchased workflows
Require an accessibility conformance report that names the product version, evaluated interfaces, known exceptions, and people responsible for fixes. Include authoring, candidate delivery, results, and exports in your review.
1EdTech’s accessibility guidance explains the responsibilities of authors and delivery systems. Test that each required accommodation survives import, editing, export, and delivery. Confirm that the candidate can use it during the assessment.
Use the accessible assessment procurement guide to define keyboard, assistive-technology, and content tests. Record the browsers and assistive technologies used, actual results, and the person responsible for each defect.
Require delivery and operational evidence
Define the results you expect when testing identity, access to released content, attempt recovery, duplicate submissions, deadlines, scoring corrections, and audit records. Test connection loss and recovery under your institution’s policy. The secure assessment delivery guide provides a detailed test plan.
For capacity claims, request the tested workload, number of concurrent users, test duration, environment, response times, failure rate, and recovery results. Before accepting the claim, check whether that workload represents your event.
For either hosted or campus-run systems, name who patches the service, restores backups, monitors failures, responds to incidents, and supports integrations. Record the data locations, access controls, retention requirements, and recovery objectives your institution needs.
Put acceptance and exit conditions in the agreement
Review these draft requirements with your procurement team. Before issuing the RFP, fill every bracketed field.
- Evidence: The supplier provides the evidence listed in [requirements table] for [product release and configuration] by [review date]. Each document or test result identifies its scope and known limitations.
- Acceptance: The institution accepts each mandatory requirement only when [named test] meets [expected result] in [evaluation environment]. [Decision owner] must approve each exception and record [deadline for fixes and retesting].
- Change review: A change to [required feature, integration, or deployment configuration] triggers [agreed regression tests] before institutional approval.
- Exit: By [transition milestone], the supplier provides [item content, local assets, required metadata, test definitions, results, and audit records] in [documented formats]. The institution checks the export through [destination import or independent inspection]. The agreement specifies [fees, support period, and deletion conditions].
Distinguish QTI package exports from operational records such as attempts, grades, and audit history. Define a usable export for each required record type. Document any restriction on content that the institution does not own or have permission to transfer.
Record the procurement decision
Attach the completed worksheet and links to the evidence to your evaluation record. Record who accepted each requirement, rejected it, or approved an exception. Set the next test date for each unresolved item.
If you are evaluating QFlowLearn, request its RFP packet and compare the responses with your requirements. For help defining your assessment requirements, contact Longsight.
