SCORE-AI

SCORE-AI explores how an AI capability assessment can reflect the work a person is actually expected to do. The underlying ROSAI framework treats AI readiness as role-oriented: the useful combination of product, engineering, data, transformation, design and leadership capabilities changes with the role.

Claims such as “AI-ready” or “AI expert” are easy to make and hard to interpret. A generic score hides the fact that an engineer, product manager, transformation leader and business head may need very different combinations of capabilities. The product challenge was to provide a structured, useful signal without presenting it as a certification, hiring recommendation or prediction of job performance.

If a person defines their role mix first, a structured assessment can weight AI capabilities against that context and produce a more useful development profile than a single role-agnostic test score.

1. Define the role before measuring capability

The participant allocates their role across six areas: Product & Business, Engineering, Data & ML, Transformation, Design & UX, and Leadership & Governance. The mix sets the reference point and changes the importance of individual capabilities.

2. Assess judgement as well as familiarity

The assessment takes about 15 minutes and includes 18 questions. It begins with role mix and responsibility level, then covers usage, judgement, experience, evidence and impact. It concludes with three short role-adaptive scenarios.

3. Separate deterministic structure from AI evaluation

Structured scoring is deterministic. The AI evaluator grades only the three scenario responses against a defined rubric. Results retain the grader and framework versions so existing results are not silently rescored under a new version.

4. Treat evidence as context, not proof

Participants can declare products, repositories, demos, architecture documentation, case studies, measurable outcomes and other work. The system reports Evidence Strength as the availability of distinct, self-declared sources; it does not independently verify claims and does not change the capability score.

5. Return three measures together

The result presents a 0–100 role-adjusted ROSAI Score, Evidence Strength and Assessment Confidence. These are deliberately separate measures so a strong capability signal is not confused with evidence availability or response consistency.

SCORE-AI is the first working implementation of ROSAI — Role-Oriented Skills Assessment for AI. It maps seven dimensions against role context: AI Fluency, Applied AI Judgement, Build & Execution, Evaluation & Reliability, Data & ML Capability, Responsible AI & Governance, and Business & Organizational Impact. Score bands run from Emerging through Leading, while serious gaps in a core capability can limit the overall score.

Assessment design requires restraint. A score must be interpretable without overstating what it can establish. The product separates declared evidence from independently verified evidence, keeps confidence distinct from capability, applies role-adjusted weighting, and states clear limits: it is not a certification, a psychometrically validated instrument, a hiring recommendation, a population percentile or a detector of AI-generated writing.

The public experience delivers a private, no-sign-up assessment and a personal development scorecard. Participants receive role-adjusted capability context, an inventory of declared evidence availability and an assessment-confidence indicator. Share links and evidence-detail inclusion remain user-controlled, while research, grader-improvement and future-training permissions are optional.

When an AI assessment produces a number, the surrounding product design matters as much as the number itself. Clear definitions, transparent inputs, separate measures, versioned evaluation and explicit limits make the result more useful and reduce the risk of a score being used as a shortcut for a decision it cannot support.