Education, training, and assessment software
Online Assessment Platform Requirements Checklist
A practical requirements framework for training providers, certification bodies, schools, employers, and professional organizations building or selecting an assessment platform.
Published by Kennedy Gichobi · Expert-reviewed by Kennedy Gichobi · Published · 3238 words
Define the decision the assessment must support
An online assessment may provide low-stakes practice, determine course completion, award a professional credential, screen a job applicant, document workforce competence, place a learner into a level, or contribute evidence to a regulated decision. Those purposes require different assurance, accessibility, review, retention, and operational controls. Begin with the decision and its consequences rather than starting with a timed quiz screen.
Document who creates the assessment, who approves it, who is eligible, who takes it, who may supervise it, who scores it, who reviews exceptions, and who relies on the result. Define whether an attempt produces feedback, a grade, pass or fail, a proficiency level, a recommendation, a credential, or only one piece of a wider human decision. Qualified education, psychometric, legal, accessibility, privacy, and subject-matter professionals should determine policy; the software should implement the approved policy consistently and preserve evidence.
Set measurable outcomes such as reducing manual assembly, shortening result turnaround, supporting approved accommodations, improving item-quality analysis, making appeals traceable, transferring assessments between compatible tools, or issuing verifiable credentials. “Prevent all cheating” is not a credible product requirement. A useful integrity objective names the threat, consequence, permitted controls, evidence, false-positive tolerance, and human response.
Model assessment records separately
Represent organization, program, subject, competency, blueprint, item, stimulus, rubric, item version, form, section, delivery window, candidate, enrollment, eligibility, accommodation, attempt, response, score, review, incident, decision, appeal, credential, and audit event as related records. Avoid placing the entire examination inside one mutable document. An item can appear in several forms, a candidate can have several attempts, and a corrected score must not overwrite the original decision without history.
Use stable identifiers. Titles, names, course codes, and credential labels change. Preserve external identifiers for learning platforms, student systems, identity providers, testing partners, and credential wallets. Separate a person from their registrations and attempts so account changes do not rewrite historical evidence.
Define lifecycle states for every consequential object. An item might move through draft, technical review, content review, accessibility review, approved, active, suspended, retired, and archived. An assessment form may be assembled, reviewed, published, scheduled, closed, rescored, and withdrawn. State transitions need an authorized actor, reason, timestamp, and immutable reference to the version affected.
Govern the item bank as controlled intellectual property
Capture item type, prompt, stimulus, response options, correct or expected response, scoring rule, rationale, feedback, subject, objective, competency, difficulty estimate, cognitive level, language, accessibility information, author, owner, source, license, status, exposure, and version. Keep authoring metadata apart from candidate-facing content. Never send correct answers, internal tags, or hidden scoring logic to the browser before they are required.
Define creation, import, cloning, review, approval, translation, revision, suspension, retirement, and deletion. A material edit to a live item should create a new version rather than changing what previous candidates saw. Preserve the item version, order, option order, stimulus, presentation settings, and scoring rule associated with each delivered attempt.
Protect question banks according to their value and exposure risk. Limit bulk exports, watermark accountable review copies where appropriate, monitor unusual access, rotate forms, and revoke unnecessary author privileges. Do not make the candidate application download an entire item bank. Security controls should not prevent legitimate accessibility review, independent quality assurance, or a contractual data export.
Build assessments from an approved blueprint
An assessment blueprint connects the decision to content domains, competencies, item counts, weights, difficulty, item types, duration, and other constraints. It makes form assembly explainable. Define whether forms are fixed, randomly assembled, staged, adaptive, or selected by an administrator. Specify required and optional pools, exclusions, enemy items that must not appear together, stimulus grouping, language rules, and exposure limits.
Validate every generated form before publication. Confirm item counts, score totals, domain coverage, time expectation, accessibility review, dependencies, answer availability, randomization rules, and prohibited combinations. Freeze the approved form or the exact assembly rules and seed needed to reconstruct it. A later bank update must not silently alter a scheduled examination.
Computer-adaptive testing, equating, standard setting, and defensible score interpretation require specialist expertise and adequate data. Do not infer that adding a difficulty field creates a valid adaptive test. If adaptive delivery is required, document the measurement model, starting rule, item-selection algorithm, exposure control, stopping rule, scoring, calibration process, simulation evidence, monitoring, and fallback behavior.
Specify the candidate journey end to end
Map invitation or discovery, account creation, eligibility, registration, payment if applicable, scheduling, identity steps, system check, accommodations, instructions, launch, consent where required, assessment, breaks, submission, provisional status, scoring, result release, review, appeal, credential, and support. Include expired invitations, duplicate registrations, payment reversal, missing eligibility, late arrival, device failure, interrupted connectivity, and a candidate who needs help without exposing examination content.
Use clear status and next actions. “Pending” is not enough when a candidate needs to know whether payment, eligibility evidence, manual review, scheduling, scoring, or an appeal is pending. State the responsible party, expected timing, safe contact channel, and what the candidate should do.
Avoid unnecessary friction. Require a system check only for capabilities actually needed. Let candidates practice the exact interaction patterns without exposing live items. Save instructions in an accessible form that can be revisited. Make support available before a timed session rather than revealing a phone number only after launch fails.
Make assessment delivery resilient
Define supported browsers, devices, network conditions, session duration, autosave behavior, navigation, review flags, section locks, breaks, permitted resources, submission, and recovery. Decide whether a candidate may revisit items, skip, change answers, or submit early. Show time and progress without creating avoidable distraction. Warn before consequential navigation or final submission, then confirm receipt with a durable attempt identifier.
Save responses incrementally using idempotent operations. Record the candidate action time and server receipt time where latency matters. Make retries safe so a slow network cannot create two attempts or replace a newer answer with an older one. The client should show whether an answer is saved, queued, or failed; a spinning icon is not adequate evidence.
Define recovery for refresh, browser crash, expired authentication, sleep, power loss, connectivity change, server deployment, provider outage, and an administrator ending a window. Preserve accepted responses and remaining-time policy. Record interruptions separately from misconduct. Test recovery under throttled and unstable networks instead of assuming autosave works because it succeeds locally.
Treat time rules as policy
Define whether time applies to the whole attempt, sections, or individual activities; whether instructions count; how breaks work; and which clock is authoritative. Record start, pause, resume, extension, expiration, submission, and relevant reasons. Do not trust a candidate device clock for enforcement. Display time consistently, but make the trusted service responsible for consequential expiration and submission decisions.
Accommodated time should derive from an approved rule or explicit duration without disclosing a disability to people who only need to operate the session. Define late start, approved pause, emergency interruption, server outage, and resumed-attempt behavior. An administrator changing time after launch should require authority, reason, confirmation, and an audit event visible to the appropriate reviewer.
Test boundary seconds, daylight-saving transitions, time zones, device sleep, multiple tabs, delayed requests, reconnects, and simultaneous administrative action. A timer animation can drift; the server-side decision and reconstructable event history should control the outcome. Include these cases in automated tests and in a supervised rehearsal using the same production timing architecture.
Build accessibility and accommodations into the assessment
Use WCAG 2.2 as a shared technical baseline while qualified professionals determine applicable policy and legal obligations. Test the complete candidate journey, not only the public registration page. Candidate-facing items, timers, dialogs, navigation, equations, tables, code, media, drag interactions, status updates, error recovery, review screens, and results must work with representative assistive technologies and user settings.
Provide semantic structure, keyboard operation, visible focus, sufficient contrast, zoom and reflow, text alternatives, captions and transcripts, clear labels, understandable errors, status announcements, alternatives to dragging, and accessible authentication. Time limits require special attention: implement approved warnings, extensions, pauses, or alternatives without making candidates repeatedly disclose personal information.
Model accommodations as authorized delivery settings, not free-text notes broadcast to proctors and support staff. Settings might govern time, breaks, presentation, language, input method, reader support, alternate format, or location. Preserve approval, effective period, assessment scope, confidentiality, and the exact settings used. An accommodation must not change the construct being measured unless the responsible professional intentionally approves that effect.
1EdTech’s QTI 3 materials include accommodation and accessibility capabilities, but a standards-compliant package does not guarantee that authored content or the receiving interface is accessible. Test representative imported items, platform controls, assistive technology, and the organization’s operating process together.
Choose identity assurance based on risk
Account login, identity proofing, authentication, eligibility, and session monitoring solve different problems. Decide what confidence the assessment decision requires and what evidence is proportionate. A practice quiz may need only a learner account; a consequential license examination may require approved proofing and supervised controls. Avoid collecting government documents or biometrics merely because a vendor offers the feature.
NIST SP 800-63-4 provides a risk-based framework for identity proofing, authentication, and federation. It is guidance for U.S. government digital systems, not an automatic requirement for every education provider, but its separation of assurance decisions is useful. Document the selected assurance, threats, user population, recovery process, privacy impact, accessibility, redress, and fallback.
Support secure account recovery without letting support staff bypass assessment policy informally. Keep identity evidence access restricted and retention bounded. If an external identity or proctoring provider is used, verify permitted use, jurisdictions, subprocessors, deletion, incident handling, accessibility, failure modes, human review, and how candidates without supported devices complete an equivalent approved path.
Set honest boundaries for proctoring and integrity controls
Integrity controls may include honor statements, randomized forms, restricted windows, supervised locations, human proctors, secure browsers, environment checks, plagiarism review, similarity analysis, response-pattern analysis, or follow-up examination. Each control has limitations, accessibility effects, privacy costs, support demands, and false-positive risk. No camera, browser lock, or automated score proves intent by itself.
Define prohibited conduct, permitted materials and devices, notice, consent where applicable, data collected, decision use, retention, reviewer qualification, candidate response, and appeal. Separate a technical event from an allegation and a confirmed finding. Loss of focus, background sound, face-detection failure, unusual response time, or network change can have innocent explanations.
Use automation to surface evidence for authorized human review when appropriate, not to make unexplained high-impact decisions. Preserve the relevant policy version, event, recording segment or artifact where lawful, reviewer notes, candidate explanation, outcome, and retention schedule. Provide a usable alternative when a control excludes a candidate or fails for reasons outside their control.
Make scoring deterministic and reviewable
Define points, weights, partial credit, penalties, rounding, unanswered treatment, section rules, pass standards, rubrics, manual scoring, moderation, and result precision. Version scoring rules. Preserve raw responses, item-level outcomes, transformations, manual judgments, adjustments, and the final decision. A displayed percentage should be reproducible from retained evidence.
Separate automatic scoring from result release. Some attempts may require manual scoring, integrity review, invalid-item treatment, or moderation. Show candidates a status that does not imply a final decision prematurely. Restrict answer keys and reviewer-only feedback while allowing authorized scorers the context they need.
For constructed responses, use double marking, adjudication, sampling, or calibration where approved by policy. Record the rubric version, scorer, scores, comments, conflicts, and resolution. If AI assists classification or feedback, define its limited role, evaluation evidence, human oversight, sensitive-data handling, drift monitoring, and a non-AI path. Generative output should not silently become a consequential score.
Handle item corrections and rescoring explicitly
An item may be ambiguous, miskeyed, inaccessible, exposed, mistranslated, or technically broken. Define who may suspend it, how affected attempts are found, whether scoring excludes or credits it, who approves rescoring, and how candidates are notified. Preserve both original and revised results with the rule, population, reason, approver, and time.
Recalculate through a controlled job that is repeatable and reconciled. Preview affected counts and score changes before commitment. Prevent duplicate credential issuance or contradictory notifications. If a revised result changes eligibility or invalidates a credential, route the case through approved policy rather than silently changing a dashboard.
Measure item performance carefully. Difficulty, discrimination, option behavior, completion time, omission, and exposure can inform review, but small or nonrepresentative samples may mislead. Protect candidate privacy and avoid treating group differences as proof of candidate deficiency. Qualified specialists should interpret results and approve retirement or revision decisions.
Design results, feedback, appeals, and corrections together
Specify what candidates, instructors, employers, regulators, and administrators can see and when. Results may include completion, scale score, proficiency, domain performance, attempt history, feedback, next eligibility date, or credential status. Avoid releasing item content or answer keys when that undermines the program. Explain scores in language appropriate to their validated use.
Define result holds, release schedules, corrected results, downloads, verification, and notification. Notifications should reveal minimal sensitive information and direct the recipient to authenticated content. Track delivery without treating email receipt as proof that a person reviewed the result.
An appeal workflow should state grounds, deadline, required information, evidence access, reviewer independence, communications, decision, correction, and further review. Preserve the assessment and policy versions that applied. Give candidates a clear way to correct identity or administrative data separately from challenging an academic or certification decision.
Issue credentials that can be verified and maintained
Define the achievement, issuer, recipient binding, criteria, evidence reference, issue date, expiration, renewal, status, revocation, replacement, and verification experience. Decide whether the organization needs a printable certificate, a public verification page, a wallet-compatible digital credential, or several representations. A PDF with an editable name is not strong verification.
Open Badges 3.0 represents achievements as digitally signed credentials compatible with the W3C Verifiable Credentials Data Model 2.0. It can support portable, information-rich credentials, but implementation still requires issuer governance, key management, privacy decisions, status or revocation behavior, recipient support, and conformance testing. Confirm the exact version and whether a vendor is certified for the required role rather than accepting a generic “supports badges” claim.
Keep credential issuance idempotent. A repeated completion event must not create several live credentials. Preserve the result and policy decision that authorized issuance while minimizing public disclosure. Plan signer-key rotation, compromised keys, issuer name changes, expired credentials, corrected recipient information, platform migration, and verification after the original application is replaced.
Specify interoperability with evidence
1EdTech Question and Test Interoperability enables exchange of items, tests, and results among compatible authoring, banking, delivery, and scoring systems. QTI 3 adds web-oriented presentation, accommodation support, and capabilities for adaptive and technology-enhanced assessment. “QTI compatible” remains too vague for procurement. State the exact version, conformance profile, item types, extensions, metadata, scoring, results, packaging, import, export, and round-trip expectations.
Build a representative interoperability test pack. Include stimuli, media, mathematics, tables, alternate content, multiple languages, accommodations, feedback, partial credit, rubrics, custom interactions where permitted, and edge cases. Import it into the intended receiving product, deliver and score it, export it again, and compare meaning rather than only schema validity. Record unsupported features and an exit strategy.
For LMS, student information, HR, identity, payment, proctoring, notification, analytics, and credential integrations, define identifiers, source of truth, version, authentication, permissions, mapping, freshness, retries, idempotency, reconciliation, support, and cost. Prototype the least certain exchange before dependent workflows are built.
Protect candidate data and assessment content
Inventory identity, contact, eligibility, payment references, accommodations, responses, scores, recordings, device or network signals, support messages, appeals, credentials, and derived analytics. For every element, define purpose, authority, collection, visibility, sharing, decision use, retention, correction, export, and deletion. Assessment content and candidate information may need different access and retention policies.
Enforce organization, program, assessment, role, assignment, and record-level authorization on trusted services. Apply it to APIs, search, exports, reports, background jobs, caches, logs, notifications, and support tools—not only visible navigation. A training provider administering programs for several clients must not allow one client to infer another client’s candidates, items, results, or counts.
Use separate environments, managed secrets, encryption, controlled uploads, dependency management, reviewed deployment, backups, restoration tests, monitoring, incident response, and vulnerability handling. NIST’s Secure Software Development Framework provides practices that can be integrated into the chosen lifecycle. Match assurance work to impact instead of treating a vulnerability scan as proof of safe scoring and authorization logic.
Make reports operational and statistically honest
Define each metric’s population, grain, formula, exclusions, timezone, refresh, owner, privacy threshold, and drill-down. Operational measures may include registration completion, launch failures, interruptions, scoring queue age, manual-review workload, appeals, integration exceptions, and credential delivery. Assessment measures may include score distribution and item statistics when valid for the sample and intended interpretation.
Do not rank instructors, candidates, locations, or groups using unstable or context-free measures. Suppress or aggregate small groups where disclosure is possible. Separate exploratory analytics from validated decision rules. Show data freshness and version so a report created before rescoring is not mistaken for the current result.
Make exports authorized, traceable, and usable. Include stable identifiers, definitions, timestamps, versions, and relevant status. Test spreadsheet formula injection defenses, large exports, cancelled jobs, expired links, and permission changes during generation. Avoid permanent public download URLs.
Plan migration, operation, and ownership
Profile programs, users, eligibility, item banks, media, forms, attempts, responses, scores, accommodations, incidents, appeals, and credentials before promising migration. Identify unsupported item types, malformed packages, duplicate people, missing identifiers, obsolete answer keys, and records whose legal or operational meaning is unclear. Rehearse conversion and reconcile counts, relationships, score totals, assets, and representative candidate histories.
Define support levels for enrollment deadlines and live assessment windows. Build operational views for stuck attempts, failed autosaves, scoring queues, time adjustments, provider outages, integration errors, and credential failures. Give support staff safe diagnostic information without exposing answer keys or unrelated candidate data. Require stronger authority for score, eligibility, accommodation, and identity changes than for ordinary account help.
Set recovery objectives and rehearse restore. Verify item versions, attempt state, responses, timers, permissions, scores, and issuance idempotency after recovery. The organization should control or be able to transfer repositories, domains, cloud resources, signing keys, identity configuration, storage, integrations, deployment pipelines, backups, monitoring, exports, and documentation.
Use this checklist before approving an assessment platform
Confirm that the proposal defines the decision and stakes; separates people, registrations, attempts, responses, scores, and credentials; versions item and scoring content; assembles forms from an approved blueprint; supports governed authoring; maps the full candidate journey; saves responses safely; implements time and accommodations as policy; tests accessibility; selects proportionate identity and integrity controls; separates events from findings; makes scoring reproducible; supports rescoring and appeals; issues maintainable credentials; proves QTI interoperability with representative content; reconciles integrations; protects candidate and item data; reports honestly; restores safely; and preserves ownership.
Then test one difficult examination day: a candidate receives an approved time extension, the identity provider is slow, an imported item loses alternate content, the connection drops after an answer change, a proctoring signal is inconclusive, one item is later miskeyed, manual scoring disagrees, the result changes after controlled rescoring, an appeal is filed, and credential issuance retries twice. A strong design explains authority, version, evidence, accessibility, state, retry, candidate communication, reconciliation, correction, and review at every step.
If the broader question is whether to buy, integrate, or build, first review the custom LMS development cost guide. Then share your programs, candidate groups, assessment stakes, item types, blueprint, scoring, accommodations, identity approach, integrity policy, credentials, integrations, volumes, migration, and current failures through the project questionnaire so discovery can define an appropriate assessment architecture.
Authoritative references
Related software planning guides
- Learning Management System Delivery Timeline for Training Providers
- How Training Providers Should Evaluate an LMS
- Custom LMS Development Cost: A Guide for Training Providers