QTI validation: what each test proves and what it does not

Run separate validation checks

Teams often use “valid” to mean several different things. A document can be valid XML and still declare the wrong response type. It can pass a semantic validator and still fail in a destination player. It can render correctly with a mouse and remain unusable from a keyboard.

Name each test and store its result separately. Do not describe an item as “valid” without saying which checks passed.

Check XML well-formedness

An XML parser checks whether tags, attributes, entities, namespaces, and character encoding form a document it can read. Run this first because later checks need a parsed document.

It proves that the XML is parseable. It does not prove that the document is QTI, follows the QTI schema, or behaves as intended.

Report parser errors with a file, line, and column where the parser provides them. Reject external entity expansion and other XML features that the application does not need.

Check the QTI schema

XML Schema Definition (XSD) validation compares the document with the required QTI XML binding. It can find unknown elements, misplaced content, invalid attributes, namespace mistakes, and some datatype errors.

Pin the schema set and version used for the result. Do not fetch mutable schemas during a production validation run. Store or checksum the official files according to their license and distribution terms.

Schema validation does not answer the following questions:

• Do response identifiers resolve?
• Do cardinality and base types agree across the item?
• Does response processing produce the intended outcome?
• Does the receiving system support the interaction?
• Is the item accessible?
• Is the package safe to extract?

1EdTech links to its online QTI validator from the QTI conformance page. When its access terms and data handling fit your institution, use the service. Keep a local validation path for private content, automated builds, and reproducible results.

Check semantic consistency

Semantic validation operates across related parts of the item. It should check that:

• Each interaction references a declared response.
• Correct responses and default values match the declaration.
• Cardinality and base type fit the interaction.
• Identifiers are unique in their required scope.
• Processing variables exist and functions receive compatible values.
• Referenced outcomes and templates resolve.
• Choice and response limits are coherent.
• Asset and stylesheet references use allowed paths and schemes.

These checks need stable diagnostic codes. A code such as response.identifier.missing is easier to test, aggregate, and document than a prose message that changes on each release.

Check the supported profile

A system rarely implements every role and feature in a standard at the same depth. A profile test compares an item with the subset that an importer, authoring tool, player, or scoring service actually supports.

For every feature, use a status with a defined meaning:

• Implemented: public code handles the feature.
• Tested: automated tests cover the claimed behavior.
• Demonstrated: a public, repeatable example shows the behavior.
• Partial: public code implements only a named subset.
• Not implemented: the project states that it does not provide the feature.
• Not verified: public evidence is insufficient.
• Not applicable: the feature falls outside the component’s role.

Do not turn “parsed without error” into “supported.” Support should include the behaviors the user depends on.

Check processing and scoring

Run known responses against authoritative item XML and compare the complete outcome state. Include empty, correct, incorrect, partial, excessive, and restored responses.

For custom response processing, test every branch and boundary. Use deterministic fixtures so a change in the engine or converter creates a reviewable diff.

For high-stakes assessments, do not rely on the browser as the only scoring authority. The host system should define where it stores trusted item XML and attempt state, validates responses, and calculates final scores.

Check rendering and operation

Render representative items in a real browser. Confirm content order, controls, constraints, feedback, media, assets, and item styles.

Test with the following inputs and settings:

• Keyboard only
• Zoom and reflow
• High contrast or forced colors where relevant
• At least the screen reader and browser combinations in your institutional support policy
• Pointer and touch when the interaction depends on placement or movement
• Saved-state restoration
• Validation and error messages

A screenshot cannot prove keyboard, focus, name, role, state, or scoring behavior.

Check accessibility

The Web Content Accessibility Guidelines (WCAG) 2.2 provide testable web content criteria. Authoring tools also influence whether authors can produce and preserve accessible content, while delivery systems control the final candidate experience.

Combine automated checks with human review. Automated tools are useful for missing names, invalid relationships, contrast, and common markup failures. Human reviewers must evaluate prompt clarity, meaningful alternatives, reading order, interaction instructions, time limits, error recovery, and assistive-technology behavior.

For each test, record the item, renderer, browser, assistive technology, configuration, and success criteria.

Check package safety

Treat imported packages as untrusted input.

Validate the following package properties:

• Archive size, file count, and compression ratios
• Path traversal and absolute paths
• Duplicate or case-colliding names
• Allowed file types and content signatures
• External URLs and unsafe schemes
• Stylesheets and active content
• Custom or portable interaction code
• Immutable storage and authorization boundaries

A QTI schema does not describe all archive-level security risks.

Check interoperability

To test interoperability, export a clean package and import it into another relevant product. Deliver the item, submit known responses, and inspect the content and outcomes.

Record the following test details:

• Both product and version numbers
• QTI version and profile
• The exact fixture set
• Configuration and extensions
• Expected and observed results
• Material differences
• Test date

Record the products, versions, profiles, fixtures, and configurations used in the test. The result applies only to that tested setup.

Keep certification separate

1EdTech certification is a formal program with defined roles, tests, and current listings. Public code, internal tests, or a successful demo are not substitutes.

Verify a certification claim against 1EdTech’s current directory and note the certified product, version, role, and date. If the evidence is absent, use “not verified,” not an inference.

Put validation into the workflow

Run fast checks while an author edits, fuller checks before export, and the complete acceptance suite in continuous integration (CI) or release review. Store validation results with the item revision and tool version.

A useful diagnostic answers four questions:

1. What failed?
2. Where did it fail?
3. Why does it matter?
4. What can the author or engineer do next?

Show these diagnostics while authors edit so they can fix problems before release.