Skills Checkoffs

The Skills Checkoffs feature allows students to submit video recordings of clinical skills demonstrations for AI-assisted evaluation and educator review.

Overview

Skills checkoffs are video-based assessments that enable students to demonstrate clinical competencies through recorded performances. The system uses AI vision technology to analyze videos and provide initial grading, which educators can then review, adjust, and finalize.

Key Features

AI-Powered Video Analysis

• Automated Grading: AI analyzes video content (both visual and audio) to evaluate each step of a skill
• Evidence-Based Scoring: AI provides detailed evidence for each score, citing specific observations from the video
• Timestamp Tracking: AI identifies when each step occurs in the video with start and end timestamps
• Confidence Levels: AI indicates confidence levels (high/low) based on video quality and clarity

Flexible Grading Systems

Skills checkoffs support multiple grading approaches:

• Satisfactory/Unsatisfactory: Binary pass/fail scoring (0 or 1)
• Likert Scale: 1-5 point rating scale
• Custom Scoring: Custom rubrics with defined score ranges
• Score Types: Pass/Fail or Percentage-based final scores

Video Submission

• Direct Upload: Students can upload video files directly through the web interface
• Drag and Drop: Easy file upload with drag-and-drop support
• Video Validation: Automatic validation of video format and duration
• Progress Tracking: Real-time upload progress indicators
• Processing Status: Live status updates during AI processing

Workflow

For Administrators and Preceptors

Creating Skills Checkoff Templates

1. Navigate to Templates page
2. Click "Create Skills Checkoff Template"
3. Fill in template details:
   • Name: Descriptive name for the skill (e.g., "Hand Hygiene")
   • Description: Detailed description of what the skill entails
   • Grading Type: Select from Satisfactory/Unsatisfactory, Likert Scale, or Custom Scoring
   • Score Type: Choose Pass/Fail or Percentage
4. Add steps to the template:
   • Step Name: Name of each step in the skill sequence
   • Step Description: Detailed instructions for what should be demonstrated
   • Step Order: Sequence number for the step
   • Max Score: Maximum points for this step (based on grading type)
   • Optional: Link to curriculum objectives
5. Save the template

Assigning Skills Checkoffs

1. Navigate to Assign Evaluations page
2. Select the Skills Checkoffs tab
3. Choose one or more skills checkoff templates
4. Select target students (individual or by classroom)
5. Choose a classroom (optional but recommended)
6. Set a due date
7. Click "Assign" to create assignments

Bulk Assignment Tips:

• Select multiple templates to assign multiple skills at once
• Use classroom selection to assign to all students in a classroom
• Set appropriate due dates to manage student workload

Reviewing Submissions

1. Navigate to My Evaluations page
2. Find submissions in the Skills Checkoffs section
3. Click on a submission to open the review interface
4. Review the video and AI analysis:
   • Watch the video with synchronized step timestamps
   • Review AI scores and evidence for each step
   • Check AI confidence levels
5. Adjust scores as needed:
   • Override AI scores for any step
   • Add educator notes for clarification
   • Modify step-level feedback
6. Provide overall feedback:
   • Add general educator feedback
   • Choose whether to show AI feedback to students
7. Finalize the submission:
   • Review all scores and feedback
   • Click "Finalize" to lock in the grades
   • Final score is automatically calculated

Review Features:

• Video Player: Full-featured video player with playback controls
• Step Navigation: Jump directly to specific steps using timestamps
• Side-by-Side Comparison: View AI scores alongside educator scores
• Evidence Review: Read detailed AI evidence for each step
• Score Override: Modify any AI score with justification
• Feedback Toggle: Control whether students see AI feedback

For Students

Viewing Assignments

1. Navigate to My Evaluations page
2. Find your skills checkoff assignments in the Skills Checkoffs section
3. Each assignment shows:
   • Template name and description
   • Due date
   • Completion status
   • Number of steps required

Submitting a Video

1. Click on an assignment to open the submission page
2. Review the template details:
   • Read the skill description
   • Review all steps you need to demonstrate
   • Note any specific requirements
3. Prepare your video:
   • Show your face at the start of the video to verify your identity
   • Ensure good lighting and audio quality
   • Practice the skill before recording
   • Record in a quiet, appropriate environment
4. Upload your video:
   • Click "Choose File" or drag and drop your video
   • Supported formats: MP4, MOV, AVI, WebM
   • Wait for upload to complete
5. Monitor processing:
   • Video uploads to secure storage
   • AI processing begins automatically
   • Status updates show progress (Pending → Processing → Completed)
   • Processing typically takes 2-5 minutes

Video Recording Tips:

• Show your face at the start of the video to verify your identity
• Record in landscape orientation for best results
• Ensure hands and equipment are clearly visible
• Speak clearly and narrate your actions
• Follow the step sequence from the template
• Keep the video focused on the skill demonstration

Viewing Results

1. After submission, the status will update to "Completed" when AI processing finishes
2. Click on your submission to view results:
   • Watch your video with step timestamps
   • See AI scores for each step (if enabled by educator)
   • Read AI evidence and feedback
   • View educator feedback (after review)
   • See final score (after educator finalization)

Student View Features:

• Video Playback: Review your performance
• Step Breakdown: See how each step was evaluated
• Feedback Review: Read detailed feedback from AI and educators
• Score Summary: View overall performance score

Template Management

Template Structure

Each skills checkoff template consists of:

• Template Metadata:

  • Name and description
  • Grading type and score type
  • Institution association
  • Creation and modification dates

• Steps:

  • Ordered sequence of skill components
  • Each step has a name, description, and max score
  • Steps can be linked to curriculum objectives
  • Steps are evaluated independently

Grading Types Explained

Satisfactory/Unsatisfactory:

• Binary scoring: 0 (Unsatisfactory) or 1 (Satisfactory)
• Simple pass/fail evaluation
• Best for skills with clear right/wrong outcomes

Likert Scale:

• 1-5 point scale
• Allows for nuanced evaluation
• Useful for skills with varying quality levels

Custom Scoring:

• Uses custom scoring rubrics
• Flexible score ranges defined per rubric
• Allows institution-specific grading scales

Template Best Practices

1. Clear Step Descriptions: Write detailed, specific instructions for each step
2. Logical Sequencing: Order steps in the natural sequence of the skill
3. Appropriate Granularity: Break skills into meaningful, evaluable steps
4. Consistent Naming: Use clear, consistent naming conventions
5. Curriculum Alignment: Link steps to curriculum objectives when relevant

AI Processing

How AI Evaluation Works

1. Video Upload: Video is securely uploaded to cloud storage
2. File Processing: Video is processed and made available to AI
3. Frame Analysis: AI analyzes video frames and audio
4. Step Identification: AI identifies when each step occurs
5. Evaluation: AI evaluates each step against the template criteria
6. Evidence Generation: AI generates detailed evidence for each score
7. Score Assignment: AI assigns scores based on observations

AI Evidence Quality

AI provides evidence that includes:

• Visual Observations: What was seen in the video
• Audio Observations: What was heard (speech, sounds)
• Timing Information: When actions occurred
• Quality Assessment: Confidence in the evaluation
• Specific Citations: References to exact moments in the video

Confidence Levels

• High Confidence: Video/audio quality is sufficient for accurate grading
• Low Confidence: Required actions or speech cannot be clearly evaluated

Submission Statuses

• Pending: Video uploaded, awaiting AI processing
• Processing: AI is analyzing the video
• Completed: AI processing finished, awaiting educator review
• Failed: Processing encountered an error (contact support if this occurs)

Scoring and Finalization

Score Calculation

• Step Scores: Each step receives an individual score
• Total Score: Sum of all step scores divided by maximum possible points
• Final Score: Percentage (0-100) or Pass/Fail based on template settings
• Passing Threshold: Determined by template configuration

Educator Review Process

1. Initial Review: Educator reviews AI scores and evidence
2. Score Adjustment: Educator can override any AI score
3. Feedback Addition: Educator adds notes and overall feedback
4. AI Feedback Control: Educator decides if students see AI feedback
5. Finalization: Educator finalizes the submission, locking scores

Finalization

Once finalized:

• Scores cannot be changed (contact admin for exceptions)
• Students can view their final results
• Submission appears in reports and analytics
• Assignment is marked as completed

Best Practices

For Administrators

1. Template Design: Create clear, comprehensive templates with detailed step descriptions
2. Assignment Timing: Assign skills checkoffs with adequate time for practice and submission
3. Review Promptness: Review and finalize submissions in a timely manner
4. Feedback Quality: Provide constructive, specific feedback to help students improve
5. AI Integration: Use AI scores as a starting point, but always review and adjust as needed

For Students

1. Preparation: Review the template steps before recording
2. Practice: Practice the skill before recording your submission
3. Identity Verification: Show your face at the start of the video to verify your identity
4. Environment: Record in a quiet, well-lit environment
5. Video Quality: Ensure hands, equipment, and actions are clearly visible
6. Narration: Verbally describe your actions as you perform them
7. Review: Watch your video before submitting to ensure quality

For Educators

1. Thorough Review: Always review AI scores rather than accepting them blindly
2. Evidence Consideration: Read AI evidence carefully to understand the evaluation
3. Fair Assessment: Adjust scores when AI evaluation seems incorrect
4. Constructive Feedback: Provide specific, actionable feedback
5. Timely Finalization: Finalize submissions promptly to provide timely feedback

Troubleshooting

Common Issues

Video Upload Fails:

• Check file format (MP4, MOV, AVI, WebM supported)
• Ensure file size is reasonable (typically under 500MB)
• Check internet connection stability
• Try a different browser

Processing Takes Too Long:

• Processing typically takes 2-5 minutes
• Longer videos may take more time
• If processing exceeds 10 minutes, contact support

AI Scores Seem Incorrect:

• Review the video and AI evidence carefully
• Check if video quality affected AI analysis
• Override scores as needed during review
• Low confidence scores may indicate video quality issues

Cannot View Submission:

• Ensure you have proper permissions
• Check if submission is still processing
• Contact administrator if access issues persist

Video Won't Play:

• Check browser compatibility
• Ensure video format is supported
• Try refreshing the page
• Contact support if issue persists

Privacy and Security

• Secure Storage: All videos are stored in secure cloud storage
• Access Control: Only authorized users can view submissions
• Institution Isolation: Data is isolated by institution
• Retention Policy: Videos are retained according to institution policies
• FERPA Compliance: System designed to comply with educational privacy regulations

Related Features

• Templates: Create and manage skills checkoff templates
• Assign Evaluations: Assign skills checkoffs to students
• My Evaluations: View and manage assignments and submissions
• Reports: Generate reports on skills checkoff performance
• Classrooms: Organize students and assignments by classroom

Advanced Features

Curriculum Objective Mapping

• Link template steps to curriculum objectives
• Track student progress toward program outcomes
• Generate reports on objective coverage

Custom Scoring Rubrics

• Create institution-specific scoring rubrics
• Define custom score ranges and criteria
• Reuse rubrics across multiple templates

Bulk Operations

• Assign multiple skills checkoffs at once
• Review multiple submissions efficiently
• Export submission data for analysis

Analytics and Reporting

• Track completion rates
• Analyze average scores
• Identify common areas for improvement
• Generate performance reports

Support

For additional help or to report issues:

• Check the main Help documentation
• Contact your institution administrator
• Reach out to technical support