Performance Assessment
A core part of SHAREing's vision is performance analysis, with the aim of building a performance assessment service. This page hosts materials and resources on performance methodologies and tools for Research Technical Professionals (RTPs) to begin producing performance assessments.
SHAREing guides
SHAREing produces guides on performance assessments and related topics which we make publically available. We have a living document which works as a guidebook on our experiences of building a performance assessment service. Benchmarking guidelines have also been produced to support developers in building effective benchmarks of their codes.
Performance Assessment repository
The assessment-template repository contains the templates for performing a performance assessment of newly submitted software using the above form. These templates are used by assessors to evaluate the software across the assessment stages. Currently the pre-assessment and high-level assessment stages are templated, and work is ongoing on the methodology for lower-level assessment stages.
The high-level analysis is now at a stage in which it can be implemented and refined via user feedback. To get this process started we have created a template high-level report to allow digital Research Technical Professionals (dRTPs) to have a go at running some high-level assessments.
The GitHub repository hosting the templates, Python scripts, and dependencies is linked below, along with the templates directly. Feedback for the notebook is welcomed via GitHub issues, or via merge requests with proposed features.
Performance assessment templates repository
Repository containing the templates and Python scripts for data processing and graph generation
Go to repository
Pre-assessment template
Template used by assessors to evaluate submitted code before undertaking an assessment
Go to template
High-level assessment template
Template used by assessors to produce the high-level assessment
Go to templateExample Performance Assessments
Below are links to reports of completed performance assessments. These reports are using SHAREing's performance methodology which is in a very early stage of development.