Open assessment: public evidence review
QTI 3 implementation matrix
This matrix reviews public evidence for LongsightGroup/qti3, an item-focused Question and Test Interoperability (QTI) project. It records what the repository implements, tests, leaves partial, or does not provide.
How to read the matrix
Automated tests show behavior inside the repository. They do not show that two independent products can exchange the same content. Certification requires a separate 1EdTech process. Longsight completed this review of release 0.9.8 at commit 5a230d9 on July 17, 2026. Links to the main branch may change after that date.
Status definitions
- Implemented
- Public code handles the named feature; test evidence was not required for this label.
- Tested
- Public automated tests cover the claimed behavior.
- Demonstrated
- A public, repeatable example shows the behavior.
- Partial
- Only the stated subset has evidence.
- Not implemented
- The project does not provide the feature.
- Not verified
- Public evidence is insufficient.
- Not applicable
- The feature belongs to the host rather than this item-level project.
Evidence matrix
| Area | Status | Public scope | Evidence | Last verified | Boundary |
|---|---|---|---|---|---|
| QTI 3 item XML parsing | Tested | The core parses item XML into typed models and reports structured diagnostics. | Core package and tests | This status applies to the repository’s item profile, not every QTI test or results document. | |
| XSD schema validation | Not implemented | The qti3 runtime does not run XSD validation. | README non-goals | Use the official validator or a pinned local schema set as a separate gate. | |
| Semantic item validation | Tested | Declarations, response types, references, constraints, and processing diagnostics have automated tests. | Core source | Semantic checks do not establish schema validity or destination support. | |
| Standard item interactions | Tested | The public README lists fixtures, conformance checks, accessibility metadata, and browser tests for the current target interaction set. | Question-type support matrix | The project defines “supported” for item-level behavior. Host application behavior remains separate. | |
| Browser item rendering | Tested | A native custom-element player renders one item and includes unit and browser test suites. | Player package | The player is not a complete assessment runner or candidate application. | |
| Response processing and scoring | Tested | Core application programming interfaces (APIs) process responses, score fixtures, and serialize item session state. | Core package | Host systems own authoritative storage, attempt policy, authorization, and final grade return. | |
| Candidate-safe delivery XML | Tested | The core exposes preparation and submission APIs with automated coverage. | Secure-delivery implementation article | These APIs do not provide authentication, proctoring, network security, or a full delivery service. | |
| Saved item state | Tested | Public core and player contracts serialize and restore item responses and state. | Repository README | A host owns persistence, retention, concurrency, and recovery policy. | |
| Accessibility contracts | Partial | The repository includes keyboard, focus, accessible-name, validation-message, and proof metadata tests. | Accessibility package | Public automated evidence does not prove conformance with the Web Content Accessibility Guidelines (WCAG) or usability with every assistive-technology combination. | |
| Personal Needs and Preferences (PNP) data | Tested | A package parses, normalizes, validates, and resolves host-provided QTI 3 PNP data. | PNP package | Identity, consent, storage, authorization, service access, and institutional accommodation policy are not implemented. | |
| QTI-shaped authoring output | Tested | The writer package creates item XML and package output through typed APIs and tests. | Writer package | A library is not a complete authoring product, review workflow, or content-governance system. | |
| QTI 1.2 and 2.x migration | Partial | The migrator provides public source and tests for named QTI 1.2 and 2.x package and item mappings. | Migrator package | Public evidence does not cover every vendor extension, malformed legacy export, or destination behavior. | |
| Package loading and inspection | Partial | Tooling can inspect packages and load item references useful for item-focused validation. | Command-line interface (CLI) package | The project does not claim a full assessment-test delivery implementation. | |
| Portable Custom Interaction (PCI) host contract | Tested | The player parses PCI launch data and exposes a host mount and state contract with tests. | Portable custom interaction tests | The host still owns execution isolation, code trust, accessibility, assets, and product integration. | |
| Full assessment-test runner | Not implemented | The repository lists test navigation, timing policy, sections, and the complete candidate runner as non-goals. | README non-goals | Host products provide the surrounding assessment application. | |
| Shared stimulus delivery | Not implemented | The repository states that shared stimulus delivery is outside its current item-focused role. | README non-goals | Fixtures or package references do not turn the player into a stimulus delivery service. | |
| QTI Results Reporting | Not implemented | Item outcome state exists, but public evidence does not show a QTI Results Reporting implementation. | Repository README | Internal outcome serialization is different from implementing the standard reporting role. | |
| Learning Tools Interoperability, roster, identity, and gradebook | Not applicable | The item library deliberately leaves LMS shells, rostering, identity, analytics, and gradebook integration to hosts. | Project scope | Evaluate these contracts in the host product and institutional environment. | |
| Independent cross-product interoperability | Not verified | The repository has conformance fixtures and external-content tests, but this review found no current public report naming independent product versions and complete round-trip results. | Conformance package | Conformance fixtures are useful implementation evidence, not independent interoperability proof. | |
| 1EdTech QTI certification | Not verified | This review does not claim current certification for qti3 or QFlow. | 1EdTech conformance and certification | Certification requires a current official listing for a named product, version, and role. |
Limits of this guidance
- This page reviews source code and tests. It does not report independent interoperability testing.
- Automated accessibility evidence does not establish WCAG conformance for a complete candidate experience.
- Longsight does not claim current 1EdTech certification for qti3 or QFlow on this page.
- Product behavior outside public repositories needs a separate review in a named product build.
Missing public test results
- A versioned cross-product test report with named independent implementations and representative fixtures.
- Manual keyboard and assistive-technology results for each high-risk interaction family.
- A package-level corpus report covering malformed input, archive safety, asset preservation, and migration exceptions.
- Current certification records if Longsight completes the official program for a named product and role.