NKLP Functional Skills Group.
Minutes from Meeting at Redruth Technology College Monday 20th October 2008: 1.30-3.00pm
Agenda:
Minutes of last meeting
NKLP – Functional Skills website – updates
QCA – Assessment Principles for Pilot Programme
Registration of cohorts
Training days for your diary
AOB
Subject Team time
Minutes:
Minutes of Last meeting: Some discussion about item 4: size of cohorts? Advice from FS Directors is to keep groups small and manageable, but cohorts ultimately at the discretion of schools and leaders. Item 5: Who’s working with which awarding bodies – discussion – many FS Reps ‘looking around’ not strictly considering working with their current awarding bodies – will keep Paul informed of ultimate decisions.
NKLP – Paul gives update of FS website – will work with Sharon Calloway to make the site more user friendly.
QCA – Assessment Principles passed around to subject reps, to read and reflect and pass-on to school departments. May feature as a future group activity with FS Advisors.
Registration of cohorts: Again issue of cohort sizes discussed. Close links / communication with Exam Officers discussed. Concerns from Helen at Camborne about letter from WJEC English ‘capping’ their FS schools registration – but since overcome because =e of our Gateway 2 status.
Training days for your diary: Staff to book the CCED FS days – to organise bookings, cover, expenses with their schools. All invoices to John Mitchell, business manager, Pool B&E College reference – Diploma Lines.
AOB
Subject Team time: Staff met as FS Subject Reps groups to discuss current progress, training, experiences, etc.
Wednesday, 22 October 2008
Thursday, 16 October 2008
FS English Pilot Assessment Principles 2008-09
Final FS assessment principles: second year of the pilot programme (QCA) v1.0 Functional skills
English
Assessment arrangements and principles for pilot
QCA review of assessment materials:
Awarding bodies should develop assessment materials using the principles provided in this document.
The assessment principles stipulate the awarding body design and delivery arrangements for functional skills qualifications in the second year of the pilot programme (September 2008 – September 2009). All assessments delivered from January 2009 must comply with these principles.
The assessment principles are subject to change during the FS pilot phase.
Teaching and learning:
Curriculum teaching and learning must cover the skills standards for English in their entirety at a given level. Awarding body guidance to centres on the delivery of the qualifications should emphasise this coverage requirement.
Section references:
Unless otherwise stated, the bracketed section references within this document relate to ‘The statutory regulation of external qualifications in England, Wales and Northern Ireland 2004’ document (Ref: QCA/04/1293).
Final FS assessment principles: second year of the pilot programme (QCA) v1.0
Design principles for the assessment of functional skills
English
Entry level to level 2
Purpose:
The principles in this document provide the design framework for assessment of pilot qualifications in functional skills. These principles have been produced to define the parameters within which assessment models and materials should be developed for functional skills English pilot qualifications.
Assessment model:
1. The assessment must be designed to measure application of skills for a purpose within a "real life" context. These contexts should not appear contrived for the purposes of assessment.
2. The assessment can be entirely task-based, or a combination of task/s with test-style items. Refer to principle 3 for permissible proportions.
Assessment instrument definitions
Task-based assessment:
Functional skills tasks are assessments that present [a] scenario/s within regular, 'real-life' contexts, which candidates engage with in order to resolve a situation or produce an effective and appropriate response or outcome. These assessment activities must reflect real-life situations and provide opportunities for candidates to apply and transfer their knowledge and skills for a legitimate purpose.
In line with the FS standards' level differentiation factors, task-based assessment requires candidates to identify and solve problems by selecting from, making choices about, and utilising different elements within an assessment scenario.
At L1 and L2, task-based assessment should include multi-step and multiple component activities, drawing upon a range of skills and requiring choice, selection and evaluation in line with the requirements of the FS standards.
Task-based assessment should support the application and demonstration of skills and competencies that are appropriate to their situation.
Test-based assessment:
Functional skills test-based assessments include more specifically focussed objective test questions and assessment items, using fixed responses, multiple choice items, or short answer responses.
This type of assessment may use more disjointed or self-contained items in order to isolate particular skills and their performance, ensure sufficient coverage of the standards, or to corroborate achievement elsewhere, but should be conscious of overarching functional skills design aims to provide engaging and realistic skills-application opportunities.
3. Where test-style items are used they must be valid (i.e. to assess understanding and knowledge rather than problem-solving and practical application of skills).
Final FS assessment principles: second year of the pilot programme (QCA) v1.0
In developing assessments that use both task and test-style items awarding bodies must ensure that:
• the allocation of marks to those standards assessed through test-style items is not disproportionate to the way the standards are specified in full;
• test style items are in appropriate balance to items which assess problem-solving and practical application of skills
As guidance, standards assessed through test-style items should contribute no more than 25% of the total marks available.
4. Assessment of functional skills must support a holistic view of skills proficiency in relation to the functional skills standards at the specific level. It must not be atomistic. All mark schemes should be explicitly referenced to the skill standards but must not place undue weighting on individual standards nor make them individually conclusive.
5. Assessments must include details of how assessment questions and anticipated outcomes relate to the attribution of results for candidate performance; this must be mapped to the requirements of the FS standards.
6. The requirements of the assessment must be specified externally (i.e. by the awarding body).
7. Assessment items may be externally set by an awarding body or requirements may be externally set and provide for internally contextualised task-based assessments. These parameters apply at all levels (entry level – level 2).
8. Where assessment is proposed that is not 100% externally set and marked, awarding bodies must provide proposed arrangements for:
• quality assurance;
• standardisation / moderation within and across centres and examiners;
• external verification.
Where an assessment includes witness statements, or similar internal assessment components, the requirements of the assessment and standard must be explicit. It must include details regarding the assessment context and expectations in terms of candidate performance at a given level. ABs are responsible for providing the documentation that supports these requirements and maintains auditable evidence of performance. ABs will need to make clear how they manage and audit these processes, including details of their qualitative review capability.
9. Drafting of assessment materials should minimise any later need to make reasonable adjustments for disabled candidates (s14) and where this is not the case ensure reasonable adjustments can be made available (s15). In the very last resort, component exemptions should be offered where barriers to disabled candidates remain (DCSF regulations Sept 2007).
10. Reasonable adjustments should reflect the candidate’s usual methods of working but they should not give the candidate an unfair advantage compared to candidates not using reasonable adjustments, or invalidate the assessment objectives set out in the specification (s16).
Final FS assessment principles: second year of the pilot programme (QCA) v1.0
11. Awarding bodies must review all assessment materials to ensure they afford fair and equal access in terms of disability, gender, race, age, sexual orientation and religion/belief.
Levels:
12. The assessment must be designed to assess a particular, single level of the FS standards. All elements of the assessment (e.g. source materials, questions, presentation, language etc) must be appropriate to the stated level.
Where an assessment is able to provide sufficient evidence of achievement at the level below the stated level, an 'exceptional' award may be made at this level. This type of award may be possible due to the nature of the FS standards, structured as progressive levels that subsume one another. However, partial achievement at a level does not attest to achievement at the level below; clear evidence of competence against the skill standards at a given level is required for an award.
Coverage:
13. Assessment must cover the requirements of the English skill standards in speaking and listening; reading; and writing; and must avoid assessment of skills and knowledge outside the scope of these standards.
14. Awarding bodies must not combine assessment of more than one of the three functional skills of English, mathematics and ICT within a single assessment.
15. All language used in assessment material must be simple and direct, unless intentionally and appropriately assessing comprehension at the given level. All materials must be free from covert and overt discrimination, and bias (s38). (These requirements do not proscribe the use of source materials that are incorporated to assess functional skills relating to learner choice, selection and judgement). As a general principle, the level of understanding demanded by the assessment that is not expressly contained within the English standards should be at the level below that which is being assessed. This principle should apply to any content or information included in assessments.
16. The level of reading required must not exceed the level being assessed. The demand placed on the reading aspect of the assessment must take account of both the level and duration of the assessment.
17. Assessment within the qualification must cover each of the skills areas identified by the English standards as well as being referenced to the four elements of level differentiation: complexity; familiarity; technical demand; and independence. Each skill area has equal weighting for assessment purposes. These are:
• Speaking and listening;
• Reading;
• Writing.
18. Awarding bodies can develop separate assessments for each of the three skill areas (i.e. Speaking and listening; Reading; Writing) or produce integrated assessment materials covering two or more skill areas. All
Final FS assessment principles: second year of the pilot programme (QCA) v1.0
assessments must require candidates to achieve each skill area at the level being assessed and compensation across skill areas is not permitted.
19. The assessment should be designed to incorporate the different elements of the FS skill standards (skill standards, coverage and level differentiation factors) in integrated scenarios that reflect 'real life' circumstances.
20. At each level, the assessment must be based on all of the ‘skill standards’ (FS standards English document v0.6: left-hand column) and all underpinning ‘coverage’ items (FS standards English v0.6 document: right-hand column).
The assessment can sample from the ranges within the skill standard statements and ranges in the coverage statements that comprise them. The assessment must focus on functionality and the application of skills as articulated by the skill standards.
N.B. Reading and writing assessments at all of the levels must include texts suitable for the level and relating to real-life contexts.
21. Where the assessment samples from particular ranges within the FS standards, awarding bodies are responsible for making judgements relating to this sampling. These judgements must take account of how the level of difficulty and demand, and therefore the qualification level, are influenced by these ranges (for example, the candidate's proficiency with multiple texts, sources, formats, and media at L2) as well as the assessment’s intended purpose as a measure of a candidate’s overall English competence as specified by the standards at the level being assessed.
22. Assessment requirements and mark schemes must clearly show how the assessment model and each assessment based on this model relate to the requirements of the functional skills English standards.
23. Assessment must acknowledge the requirement to achieve each of the skill areas.
Assessment conditions:
24. All assessment, including that of speaking and listening, must be undertaken in controlled, supervised conditions. For speaking and listening, this means that the candidate must be made aware that their contribution is for the purposes of assessment and that the assessment must be supervised and outcomes recorded.
25. Awarding bodies may provide source materials to be used by candidates during controlled assessments.
26. Pre-release materials are permitted. Awarding bodies must provide clear guidance in relation to pre-release materials and the requirements for the use of these materials prior to the controlled assessment.
Pre-release materials should be distributed to candidates no more than 3-4 weeks of teaching and learning time prior to the summative assessment. In certain learning contexts where the contact time is very limited, ABs may request an extension to allow equivalent time for learner engagement with materials. Pre-release materials can provide candidates with sufficient time to understand the nature of particular assessment contexts or scenarios but are
Final FS assessment principles: second year of the pilot programme (QCA) v1.0
not an assessed component. ABs should ensure that the requirements of the assessment are not apparent to candidates, or others, through their pre-release materials.
27. Access to dictionaries, reference materials, support devices and 'help' functions is not currently stipulated. Access is determined by the awarding body in line with their assessment design and should accord with the assessment objectives (Assessments should acknowledge the degree of accuracy required, and the level of credit, or marks, available in line with this availability as part of their design, and the instrument should assess the effectiveness of their use. Adequate centre and candidate guidance is essential). Awarding bodies will need to give consideration to the implications for management, access, security and reliability.
28. Assessment is of the candidate’s own ability to reach an outcome by independent application of skills. For this reason, candidates must not be given help by others (e.g. centre staff, external invigilators, peers) in order to meet the requirements of the assessment.
29. Awarding bodies must provide a model and requirements to centres for the assessment of speaking and listening. QCA will require details from awarding bodies relating to the quality assurance arrangements for this aspect of assessment. In particular awarding bodies will need to clearly identify how the assessment models meet the requirements for a summative assessment of speaking and listening for functional skills qualifications. (Refer to principle six for further detail)
30. Reasonable adjustments should be made available to candidates with disabilities (s15). For each of these awarding bodies must publish to centres its procedures and share relevant data with QCA on request (s 17, s18, s20)
31. Special considerations should be made available to candidates who suffer temporary illness, injury or indisposition. For each of these awarding bodies must publish to centres its procedures and share relevant data with QCA on request (s 17, s18, s20)
Duration:
32. The duration of the speaking and listening component leading to a qualification is not specified by the regulator for the pilot. Awarding bodies must provide a description and rationale for their assessment approach and its duration. Where this is not time limited, awarding bodies will need to clearly identify the conditions under which the summative assessment of speaking and listening skills takes place.
33. The duration of assessment for the reading and writing components leading to a qualification should be no longer than five hours in its entirety, unless reasonable adjustments are put in place for a disabled candidate. The expectation is that at levels 1 and 2 the assessment of these skills will be approximately 2 hours but that at entry level greater flexibility may be required.
34. Where the assessment model includes pre-release of source material/s, awarding bodies must provide a description and rationale for their assessment approach and procedures relating to assessment materials that are to be considered outside of this duration. Where pre-release materials are
Final FS assessment principles: second year of the pilot programme (QCA) v1.0
used, awarding bodies will need to clearly stipulate for centres how the use of this material relates to the summative assessment.
35. Where the assessment model includes pre-release of source material, the duration of supervised assessment should not assume the degree to which candidates have engaged with the material and should provide adequate reading time. ABs should consider the volume of pre-release materials in relation to the duration of their summative assessment in terms of overall burden.
36. Awarding bodies are required to offer a minimum of two assessment opportunities in the second year of the pilot programme. Awarding bodies should be able to demonstrate progress in their ability to deliver FS assessments more flexibly to meet the needs of candidates and centres. The number of assessment opportunities for speaking and listening is not specified by the regulator for the pilots but awarding bodies will need to consider manageability for centres and clearly specify what is required in the guidance for centres.
English
Assessment arrangements and principles for pilot
QCA review of assessment materials:
Awarding bodies should develop assessment materials using the principles provided in this document.
The assessment principles stipulate the awarding body design and delivery arrangements for functional skills qualifications in the second year of the pilot programme (September 2008 – September 2009). All assessments delivered from January 2009 must comply with these principles.
The assessment principles are subject to change during the FS pilot phase.
Teaching and learning:
Curriculum teaching and learning must cover the skills standards for English in their entirety at a given level. Awarding body guidance to centres on the delivery of the qualifications should emphasise this coverage requirement.
Section references:
Unless otherwise stated, the bracketed section references within this document relate to ‘The statutory regulation of external qualifications in England, Wales and Northern Ireland 2004’ document (Ref: QCA/04/1293).
Final FS assessment principles: second year of the pilot programme (QCA) v1.0
Design principles for the assessment of functional skills
English
Entry level to level 2
Purpose:
The principles in this document provide the design framework for assessment of pilot qualifications in functional skills. These principles have been produced to define the parameters within which assessment models and materials should be developed for functional skills English pilot qualifications.
Assessment model:
1. The assessment must be designed to measure application of skills for a purpose within a "real life" context. These contexts should not appear contrived for the purposes of assessment.
2. The assessment can be entirely task-based, or a combination of task/s with test-style items. Refer to principle 3 for permissible proportions.
Assessment instrument definitions
Task-based assessment:
Functional skills tasks are assessments that present [a] scenario/s within regular, 'real-life' contexts, which candidates engage with in order to resolve a situation or produce an effective and appropriate response or outcome. These assessment activities must reflect real-life situations and provide opportunities for candidates to apply and transfer their knowledge and skills for a legitimate purpose.
In line with the FS standards' level differentiation factors, task-based assessment requires candidates to identify and solve problems by selecting from, making choices about, and utilising different elements within an assessment scenario.
At L1 and L2, task-based assessment should include multi-step and multiple component activities, drawing upon a range of skills and requiring choice, selection and evaluation in line with the requirements of the FS standards.
Task-based assessment should support the application and demonstration of skills and competencies that are appropriate to their situation.
Test-based assessment:
Functional skills test-based assessments include more specifically focussed objective test questions and assessment items, using fixed responses, multiple choice items, or short answer responses.
This type of assessment may use more disjointed or self-contained items in order to isolate particular skills and their performance, ensure sufficient coverage of the standards, or to corroborate achievement elsewhere, but should be conscious of overarching functional skills design aims to provide engaging and realistic skills-application opportunities.
3. Where test-style items are used they must be valid (i.e. to assess understanding and knowledge rather than problem-solving and practical application of skills).
Final FS assessment principles: second year of the pilot programme (QCA) v1.0
In developing assessments that use both task and test-style items awarding bodies must ensure that:
• the allocation of marks to those standards assessed through test-style items is not disproportionate to the way the standards are specified in full;
• test style items are in appropriate balance to items which assess problem-solving and practical application of skills
As guidance, standards assessed through test-style items should contribute no more than 25% of the total marks available.
4. Assessment of functional skills must support a holistic view of skills proficiency in relation to the functional skills standards at the specific level. It must not be atomistic. All mark schemes should be explicitly referenced to the skill standards but must not place undue weighting on individual standards nor make them individually conclusive.
5. Assessments must include details of how assessment questions and anticipated outcomes relate to the attribution of results for candidate performance; this must be mapped to the requirements of the FS standards.
6. The requirements of the assessment must be specified externally (i.e. by the awarding body).
7. Assessment items may be externally set by an awarding body or requirements may be externally set and provide for internally contextualised task-based assessments. These parameters apply at all levels (entry level – level 2).
8. Where assessment is proposed that is not 100% externally set and marked, awarding bodies must provide proposed arrangements for:
• quality assurance;
• standardisation / moderation within and across centres and examiners;
• external verification.
Where an assessment includes witness statements, or similar internal assessment components, the requirements of the assessment and standard must be explicit. It must include details regarding the assessment context and expectations in terms of candidate performance at a given level. ABs are responsible for providing the documentation that supports these requirements and maintains auditable evidence of performance. ABs will need to make clear how they manage and audit these processes, including details of their qualitative review capability.
9. Drafting of assessment materials should minimise any later need to make reasonable adjustments for disabled candidates (s14) and where this is not the case ensure reasonable adjustments can be made available (s15). In the very last resort, component exemptions should be offered where barriers to disabled candidates remain (DCSF regulations Sept 2007).
10. Reasonable adjustments should reflect the candidate’s usual methods of working but they should not give the candidate an unfair advantage compared to candidates not using reasonable adjustments, or invalidate the assessment objectives set out in the specification (s16).
Final FS assessment principles: second year of the pilot programme (QCA) v1.0
11. Awarding bodies must review all assessment materials to ensure they afford fair and equal access in terms of disability, gender, race, age, sexual orientation and religion/belief.
Levels:
12. The assessment must be designed to assess a particular, single level of the FS standards. All elements of the assessment (e.g. source materials, questions, presentation, language etc) must be appropriate to the stated level.
Where an assessment is able to provide sufficient evidence of achievement at the level below the stated level, an 'exceptional' award may be made at this level. This type of award may be possible due to the nature of the FS standards, structured as progressive levels that subsume one another. However, partial achievement at a level does not attest to achievement at the level below; clear evidence of competence against the skill standards at a given level is required for an award.
Coverage:
13. Assessment must cover the requirements of the English skill standards in speaking and listening; reading; and writing; and must avoid assessment of skills and knowledge outside the scope of these standards.
14. Awarding bodies must not combine assessment of more than one of the three functional skills of English, mathematics and ICT within a single assessment.
15. All language used in assessment material must be simple and direct, unless intentionally and appropriately assessing comprehension at the given level. All materials must be free from covert and overt discrimination, and bias (s38). (These requirements do not proscribe the use of source materials that are incorporated to assess functional skills relating to learner choice, selection and judgement). As a general principle, the level of understanding demanded by the assessment that is not expressly contained within the English standards should be at the level below that which is being assessed. This principle should apply to any content or information included in assessments.
16. The level of reading required must not exceed the level being assessed. The demand placed on the reading aspect of the assessment must take account of both the level and duration of the assessment.
17. Assessment within the qualification must cover each of the skills areas identified by the English standards as well as being referenced to the four elements of level differentiation: complexity; familiarity; technical demand; and independence. Each skill area has equal weighting for assessment purposes. These are:
• Speaking and listening;
• Reading;
• Writing.
18. Awarding bodies can develop separate assessments for each of the three skill areas (i.e. Speaking and listening; Reading; Writing) or produce integrated assessment materials covering two or more skill areas. All
Final FS assessment principles: second year of the pilot programme (QCA) v1.0
assessments must require candidates to achieve each skill area at the level being assessed and compensation across skill areas is not permitted.
19. The assessment should be designed to incorporate the different elements of the FS skill standards (skill standards, coverage and level differentiation factors) in integrated scenarios that reflect 'real life' circumstances.
20. At each level, the assessment must be based on all of the ‘skill standards’ (FS standards English document v0.6: left-hand column) and all underpinning ‘coverage’ items (FS standards English v0.6 document: right-hand column).
The assessment can sample from the ranges within the skill standard statements and ranges in the coverage statements that comprise them. The assessment must focus on functionality and the application of skills as articulated by the skill standards.
N.B. Reading and writing assessments at all of the levels must include texts suitable for the level and relating to real-life contexts.
21. Where the assessment samples from particular ranges within the FS standards, awarding bodies are responsible for making judgements relating to this sampling. These judgements must take account of how the level of difficulty and demand, and therefore the qualification level, are influenced by these ranges (for example, the candidate's proficiency with multiple texts, sources, formats, and media at L2) as well as the assessment’s intended purpose as a measure of a candidate’s overall English competence as specified by the standards at the level being assessed.
22. Assessment requirements and mark schemes must clearly show how the assessment model and each assessment based on this model relate to the requirements of the functional skills English standards.
23. Assessment must acknowledge the requirement to achieve each of the skill areas.
Assessment conditions:
24. All assessment, including that of speaking and listening, must be undertaken in controlled, supervised conditions. For speaking and listening, this means that the candidate must be made aware that their contribution is for the purposes of assessment and that the assessment must be supervised and outcomes recorded.
25. Awarding bodies may provide source materials to be used by candidates during controlled assessments.
26. Pre-release materials are permitted. Awarding bodies must provide clear guidance in relation to pre-release materials and the requirements for the use of these materials prior to the controlled assessment.
Pre-release materials should be distributed to candidates no more than 3-4 weeks of teaching and learning time prior to the summative assessment. In certain learning contexts where the contact time is very limited, ABs may request an extension to allow equivalent time for learner engagement with materials. Pre-release materials can provide candidates with sufficient time to understand the nature of particular assessment contexts or scenarios but are
Final FS assessment principles: second year of the pilot programme (QCA) v1.0
not an assessed component. ABs should ensure that the requirements of the assessment are not apparent to candidates, or others, through their pre-release materials.
27. Access to dictionaries, reference materials, support devices and 'help' functions is not currently stipulated. Access is determined by the awarding body in line with their assessment design and should accord with the assessment objectives (Assessments should acknowledge the degree of accuracy required, and the level of credit, or marks, available in line with this availability as part of their design, and the instrument should assess the effectiveness of their use. Adequate centre and candidate guidance is essential). Awarding bodies will need to give consideration to the implications for management, access, security and reliability.
28. Assessment is of the candidate’s own ability to reach an outcome by independent application of skills. For this reason, candidates must not be given help by others (e.g. centre staff, external invigilators, peers) in order to meet the requirements of the assessment.
29. Awarding bodies must provide a model and requirements to centres for the assessment of speaking and listening. QCA will require details from awarding bodies relating to the quality assurance arrangements for this aspect of assessment. In particular awarding bodies will need to clearly identify how the assessment models meet the requirements for a summative assessment of speaking and listening for functional skills qualifications. (Refer to principle six for further detail)
30. Reasonable adjustments should be made available to candidates with disabilities (s15). For each of these awarding bodies must publish to centres its procedures and share relevant data with QCA on request (s 17, s18, s20)
31. Special considerations should be made available to candidates who suffer temporary illness, injury or indisposition. For each of these awarding bodies must publish to centres its procedures and share relevant data with QCA on request (s 17, s18, s20)
Duration:
32. The duration of the speaking and listening component leading to a qualification is not specified by the regulator for the pilot. Awarding bodies must provide a description and rationale for their assessment approach and its duration. Where this is not time limited, awarding bodies will need to clearly identify the conditions under which the summative assessment of speaking and listening skills takes place.
33. The duration of assessment for the reading and writing components leading to a qualification should be no longer than five hours in its entirety, unless reasonable adjustments are put in place for a disabled candidate. The expectation is that at levels 1 and 2 the assessment of these skills will be approximately 2 hours but that at entry level greater flexibility may be required.
34. Where the assessment model includes pre-release of source material/s, awarding bodies must provide a description and rationale for their assessment approach and procedures relating to assessment materials that are to be considered outside of this duration. Where pre-release materials are
Final FS assessment principles: second year of the pilot programme (QCA) v1.0
used, awarding bodies will need to clearly stipulate for centres how the use of this material relates to the summative assessment.
35. Where the assessment model includes pre-release of source material, the duration of supervised assessment should not assume the degree to which candidates have engaged with the material and should provide adequate reading time. ABs should consider the volume of pre-release materials in relation to the duration of their summative assessment in terms of overall burden.
36. Awarding bodies are required to offer a minimum of two assessment opportunities in the second year of the pilot programme. Awarding bodies should be able to demonstrate progress in their ability to deliver FS assessments more flexibly to meet the needs of candidates and centres. The number of assessment opportunities for speaking and listening is not specified by the regulator for the pilots but awarding bodies will need to consider manageability for centres and clearly specify what is required in the guidance for centres.
FS Mathematics - Assessment Principles for Pilot Project 2008-09
Functional skills
mathematics
FS assessment principles for the pilot programme
QCA review of assessment materials:
Awarding bodies should develop assessment materials using the principles provided in this document.
The assessment principles stipulate the awarding body design and delivery arrangements for functional skills qualifications in the second year of the pilot programme (Sept 2008 – Sept 2009). All assessments delivered from January 2009 must comply with these principles.
The assessment principles are subject to change during the FS pilot phase.
Teaching and learning:
Curriculum teaching and learning must cover the skills standards for mathematics in their entirety at a given level. Awarding body guidance to centres on the delivery of the qualifications should emphasise this coverage requirement.
Section references:
Unless otherwise stated, the bracketed section references within this document relate to ‘The statutory regulation of external qualifications in England, Wales and Northern Ireland 2004’ document (Ref: QCA/04/1293).
Final FS assessment principles: second year of the pilot programme (QCA) v1.0
Design principles for the assessment of functional skills
mathematics
Entry level to level 2
Purpose:
The principles in this document provide the design framework for assessment of pilot qualifications in functional skills. These principles have been produced to define the parameters within which assessment models and materials should be developed for functional skills mathematics pilot qualifications.
Assessment model:
1. The assessment must be designed to measure application of skills for a purpose within a ‘real life’ context. These contexts should not appear contrived for the purposes of assessment.
2. The assessment can be entirely task-based, or a combination of task/s with test-style items. Refer to principle 3 for permissible proportions.
Assessment instrument definitions
Task-based assessment:
Functional skills tasks are assessments that present [a] scenario/s within regular, 'real-life' contexts, which candidates engage with in order to resolve a situation or produce an effective and appropriate response or outcome. These assessment activities must reflect real-life situations and provide opportunities for candidates to apply and transfer their knowledge and skills for a legitimate purpose.
In line with the FS standards' level differentiation factors, task-based assessment requires candidates to identify and solve problems by selecting from, making choices about, and utilising different elements within an assessment scenario. At L1 and L2, task-based assessment should include multi-step and multiple component activities, drawing upon a range of skills and requiring choice, selection and evaluation in line with the requirements of the FS standards.
Task-based assessment should support the application and demonstration of skills and competencies that are appropriate to their situation.
Test-based assessment:
Functional skills test-based assessments include more specifically focussed objective test questions and assessment items, using fixed responses, multiple choice items, or short answer responses.
This type of assessment may use more disjointed or self-contained items in order to isolate particular skills and their performance, ensure sufficient coverage of the standards, or to corroborate achievement elsewhere, but should be conscious of overarching functional skills design aims to provide engaging and realistic skills-application opportunities.
3. Where test-style items are used they must be valid (i.e. to assess understanding and knowledge rather than problem-solving and practical application of skills).
In developing assessments that use both task and test-style items awarding bodies must ensure that:
Final FS assessment principles: second year of the pilot programme (QCA) v1.0
• the allocation of marks to those standards assessed through test-style items is not disproportionate to the way the standards are specified in full;
• test style items are in appropriate balance to items which assess problem-solving and practical application of skills
As guidance, standards assessed through test-style items should contribute no more than 25% of the total marks available.
4. Assessment of functional skills must support a holistic view of skills proficiency in relation to the functional skills standards at the specific level. It must not be atomistic. All mark schemes should be explicitly referenced to the skill standards but must not place undue weighting on individual standards nor make them individually conclusive.
5. Assessments must include details of how assessment questions and anticipated outcomes relate to the attribution of results for candidate performance; this must be mapped to the requirements of the FS standards.
6. The requirements of the assessment must be specified externally (i.e. by the awarding body).
7. Assessment items may be externally set by an awarding body or requirements may be externally set and provide for internally contextualised task-based assessments. These parameters apply at all levels (entry level – level 2).
8. Where assessment is proposed that is not 100% externally set and marked, awarding bodies must provide their arrangements for:
• quality assurance;
• standardisation / moderation within and across centres and examiners;
• external verification.
Where an assessment includes witness statements, or similar assessment components, the requirements of the assessment and standard must be explicit. It must include clear details regarding the assessment context and expectations in terms of candidate performance at a given level. ABs are responsible for providing the documentation that supports these requirements and maintains auditable evidence of performance. ABs will need to make clear how they manage and audit these processes, including details of their qualitative review capability.
9. Drafting of assessment materials should minimise any later need to make reasonable adjustments for disabled candidates (s14) and where this is not the case ensure reasonable adjustments can be made available (s15). In the very last resort, component exemptions should be offered where barriers to disabled candidates remain (DCSF regulations Sept 2007).
10. Reasonable adjustments should reflect the candidate’s usual methods of working but they should not give the candidate an unfair advantage compared to candidates not using reasonable adjustments, or invalidate the assessment objectives set out in the specification (s16).
11. Awarding bodies must review all assessment materials to ensure they afford fair and equal access in terms of disability, gender, race, age, sexual orientation and religion/belief.
Final FS assessment principles: second year of the pilot programme (QCA) v1.0
Levels:
12. The assessment must be designed to assess a particular, single level of the FS standards. All elements of the assessment (e.g. source materials, questions, presentation, language etc.) must be appropriate to the stated level.
Where an assessment is able to provide sufficient evidence of achievement at the level below the stated level, an 'exceptional' award may be made at this level. This type of award may be possible due to the nature of the FS standards, structured as progressive levels that subsume one another. However, partial achievement at a level does not attest to achievement at the level below; clear evidence of competence against the standards at a given level is required for an award.
Coverage:
13. Assessment must cover the requirements of the functional skills mathematics standards at a given level, and must avoid assessment of skills and knowledge outside the scope of these standards.
14. Awarding bodies must not combine assessment of more than one of the three functional skills of English, mathematics and ICT within a single assessment.
15. All language used in assessment material must be simple, direct, free from bias (s38) and covert or overt discrimination in either wording or context. Only information relevant to the assessment should be included. (This requirement does not proscribe the use of source materials that are incorporated to assess functional skills relating to learner choice, selection and judgement). As a general principle, the level of understanding demanded by the assessment that is not expressly contained within the mathematics standards should be at the level below that which is being assessed. This principle should apply to any content or information included in assessments.
16. The level of reading required in an assessment should be at the level below that being assessed and must not exceed the level being assessed. The demand placed on the reading aspect of the assessment must take account of both the level and duration of the assessment.
17. Assessment within the qualification must cover each of the process skills (representing, analysing and interpreting) and their key attributes identified within the FS mathematics standards (pages 4-5), as well as being referenced to the four elements of level differentiation: complexity; familiarity; technical demand; and independence (pages 6-7).
N.B. The process skills and their key attributes are included in Annex 1.
18. Process skills pervade functional skills activities. There should be opportunities to demonstrate each of the process skills (Representing; Analysing; and Interpreting) in equal or balanced proportions across a given assessment, although this balance may vary between individual questions or items.
Final FS assessment principles: second year of the pilot programme (QCA) v1.0
19. The assessment should measure the effective application of process skills and performance at the given level in purposeful contexts. The assessments should be designed to incorporate the different elements of the skills standards (process skills, performance and level differentiation factors) in integrated scenarios that reflect ‘real-life’ circumstances.
20. All assessments must require candidates to demonstrate each of the process skills and all of the performance requirements at the level being assessed.
21. At each level, assessment can be based on a sample of the ‘coverage/range’ items (FS standards mathematics v0.6 document: right-hand column) but all assessments must provide candidates with the opportunity to demonstrate a breadth of mathematical skills and include the assessment of number, geometry and statistics.
22. Where the assessment samples from the ‘coverage/range’ at a given level, awarding bodies are responsible for making judgements relating to coverage. These judgements must take account of the assessment’s intended purpose as a measure of a candidate’s functional mathematics competence as specified by the FS mathematics standards at the level being assessed.
23. Assessments should allow candidates to demonstrate their proficiency at the given level by utilising content that is explicit in the FS standards 'coverage and range' or equivalent mathematical skills and techniques as cited in the FS standards document. Where these techniques are sampled, ABs must make clear their approach to coverage and rationale in terms of sufficiency at the level. ABs must ensure that sampling does not undermine the measure of functional mathematics competence at the level or make assessment content predictable.
24. Assessment requirements and mark schemes must clearly show how the assessment model and each assessment based on this model relate to the requirements of the functional skills mathematics standards.
25. Assessment must acknowledge the requirement to achieve a level of proficiency at the level of the standard in accordance with the ‘performance’ standards as well as the overarching process skills (see Annex 1).
Assessment conditions:
26. All assessment must be undertaken in controlled, supervised conditions.
27. Awarding bodies may provide source materials to be used by candidates during controlled assessments.
28. Pre-release materials are permitted. Awarding bodies must provide clear guidance in relation to pre-release materials and the requirements for the use of these materials prior to the controlled assessment.
Pre-release materials should be distributed to candidates no more than 3-4 weeks of teaching and learning time prior to the summative assessment. In certain learning contexts where contact time is very limited, ABs may request an extension to allow equivalent time for learner engagement with materials. Pre-release materials can provide candidates with sufficient time to understand the nature of particular assessment contexts or scenarios but not
Final FS assessment principles: second year of the pilot programme (QCA) v1.0
as an assessed component. ABs should ensure that the requirements of an assessment are not apparent to candidates, or others, through their pre-release materials.
29. Access to calculators, reference materials, support devices and 'help' functions is not currently stipulated. Access is determined by the awarding body in line with their assessment design and should accord with the assessment objectives (for example, if estimation or mental arithmetic is being tested calculators should not be available for use. As part of their design, assessments should acknowledge the level of complication presented, the degree of accuracy required, and the level of credit, or marks, available in line with this availability, and assess the effectiveness of their use. Adequate centre and candidate guidance is essential). Awarding bodies will need to give consideration to the implications for security and reliability.
30. Assessment is of the candidate’s own ability to reach an outcome by independent application of skills. For this reason, candidates must not be given help by others (e.g. centre staff, external invigilators, peers) in order to meet the requirements of the assessment.
31. Reasonable adjustments should be made available to candidates with disabilities (s15). For each of these awarding bodies must publish to centres its procedures and share relevant data with QCA on request (s 17, s18, s20).
32. Special considerations should be made available to candidates who suffer temporary illness, injury or indisposition. For each of these, awarding bodies must publish to centres its procedures and share relevant data with QCA on request (s 17, s18, s20).
Duration:
33. The duration of assessment leading to a qualification should be no longer than three hours in its entirety, unless reasonable adjustments are put in place for a disabled candidate. The expectation is that at levels 1 and 2 the assessment of these skills will be approximately 2 hours but that at entry level greater flexibility may be required.
34. Where the assessment model includes pre-release of source material/s, awarding bodies must provide a description and rationale for their assessment approach and procedures relating to assessment materials that are to be considered outside of this duration. Where pre-release materials are used, awarding bodies will need to clearly stipulate for centres how the use of this material relates to the summative assessment.
35. Awarding bodies are required to offer a minimum of two assessment opportunities in the second year of the pilot programme. Awarding bodies should be able to demonstrate progress in their ability to deliver FS assessments more flexibly to meet the needs of candidates and centres. Awarding bodies will need to consider manageability for centres and clearly specify what is required in the guidance for centres.
Final FS assessment principles: second year of the pilot programme (QCA) v1.0
Annex 1: Process skills and their key attributes
Process skills:
The process skills are the learner’s ability to understand and make sense of mathematical information, to use and process that information, to interpret and analyse the results of their activity and to present this to others.
Process skills’ key attributes:
Representing
Making sense of situations and representing them
Analysing
Processing and using the mathematics
Interpreting
Interpreting and communicating the results of the analysis
A learner can:
• recognise that a situation has aspects that can be represented using mathematics
• make an initial model of a situation using suitable forms of representation
• decide on the methods, operations and tools, including ICT, to use in a situation
• select the mathematical information to use
A learner can:
• use appropriate mathematical procedures
• examine patterns and relationships
• change values and assumptions or adjust relationships to see the effects on answers in the model
• find results and solutions
A learner can:
• interpret results and solutions
• draw conclusions in the light of the situation
• consider the appropriateness and accuracy of the results and conclusions
• choose appropriate language and forms of presentation to communicate results and conclusions
Final FS assessment principles: second year of the pilot programme (QCA) v1.0
mathematics
FS assessment principles for the pilot programme
QCA review of assessment materials:
Awarding bodies should develop assessment materials using the principles provided in this document.
The assessment principles stipulate the awarding body design and delivery arrangements for functional skills qualifications in the second year of the pilot programme (Sept 2008 – Sept 2009). All assessments delivered from January 2009 must comply with these principles.
The assessment principles are subject to change during the FS pilot phase.
Teaching and learning:
Curriculum teaching and learning must cover the skills standards for mathematics in their entirety at a given level. Awarding body guidance to centres on the delivery of the qualifications should emphasise this coverage requirement.
Section references:
Unless otherwise stated, the bracketed section references within this document relate to ‘The statutory regulation of external qualifications in England, Wales and Northern Ireland 2004’ document (Ref: QCA/04/1293).
Final FS assessment principles: second year of the pilot programme (QCA) v1.0
Design principles for the assessment of functional skills
mathematics
Entry level to level 2
Purpose:
The principles in this document provide the design framework for assessment of pilot qualifications in functional skills. These principles have been produced to define the parameters within which assessment models and materials should be developed for functional skills mathematics pilot qualifications.
Assessment model:
1. The assessment must be designed to measure application of skills for a purpose within a ‘real life’ context. These contexts should not appear contrived for the purposes of assessment.
2. The assessment can be entirely task-based, or a combination of task/s with test-style items. Refer to principle 3 for permissible proportions.
Assessment instrument definitions
Task-based assessment:
Functional skills tasks are assessments that present [a] scenario/s within regular, 'real-life' contexts, which candidates engage with in order to resolve a situation or produce an effective and appropriate response or outcome. These assessment activities must reflect real-life situations and provide opportunities for candidates to apply and transfer their knowledge and skills for a legitimate purpose.
In line with the FS standards' level differentiation factors, task-based assessment requires candidates to identify and solve problems by selecting from, making choices about, and utilising different elements within an assessment scenario. At L1 and L2, task-based assessment should include multi-step and multiple component activities, drawing upon a range of skills and requiring choice, selection and evaluation in line with the requirements of the FS standards.
Task-based assessment should support the application and demonstration of skills and competencies that are appropriate to their situation.
Test-based assessment:
Functional skills test-based assessments include more specifically focussed objective test questions and assessment items, using fixed responses, multiple choice items, or short answer responses.
This type of assessment may use more disjointed or self-contained items in order to isolate particular skills and their performance, ensure sufficient coverage of the standards, or to corroborate achievement elsewhere, but should be conscious of overarching functional skills design aims to provide engaging and realistic skills-application opportunities.
3. Where test-style items are used they must be valid (i.e. to assess understanding and knowledge rather than problem-solving and practical application of skills).
In developing assessments that use both task and test-style items awarding bodies must ensure that:
Final FS assessment principles: second year of the pilot programme (QCA) v1.0
• the allocation of marks to those standards assessed through test-style items is not disproportionate to the way the standards are specified in full;
• test style items are in appropriate balance to items which assess problem-solving and practical application of skills
As guidance, standards assessed through test-style items should contribute no more than 25% of the total marks available.
4. Assessment of functional skills must support a holistic view of skills proficiency in relation to the functional skills standards at the specific level. It must not be atomistic. All mark schemes should be explicitly referenced to the skill standards but must not place undue weighting on individual standards nor make them individually conclusive.
5. Assessments must include details of how assessment questions and anticipated outcomes relate to the attribution of results for candidate performance; this must be mapped to the requirements of the FS standards.
6. The requirements of the assessment must be specified externally (i.e. by the awarding body).
7. Assessment items may be externally set by an awarding body or requirements may be externally set and provide for internally contextualised task-based assessments. These parameters apply at all levels (entry level – level 2).
8. Where assessment is proposed that is not 100% externally set and marked, awarding bodies must provide their arrangements for:
• quality assurance;
• standardisation / moderation within and across centres and examiners;
• external verification.
Where an assessment includes witness statements, or similar assessment components, the requirements of the assessment and standard must be explicit. It must include clear details regarding the assessment context and expectations in terms of candidate performance at a given level. ABs are responsible for providing the documentation that supports these requirements and maintains auditable evidence of performance. ABs will need to make clear how they manage and audit these processes, including details of their qualitative review capability.
9. Drafting of assessment materials should minimise any later need to make reasonable adjustments for disabled candidates (s14) and where this is not the case ensure reasonable adjustments can be made available (s15). In the very last resort, component exemptions should be offered where barriers to disabled candidates remain (DCSF regulations Sept 2007).
10. Reasonable adjustments should reflect the candidate’s usual methods of working but they should not give the candidate an unfair advantage compared to candidates not using reasonable adjustments, or invalidate the assessment objectives set out in the specification (s16).
11. Awarding bodies must review all assessment materials to ensure they afford fair and equal access in terms of disability, gender, race, age, sexual orientation and religion/belief.
Final FS assessment principles: second year of the pilot programme (QCA) v1.0
Levels:
12. The assessment must be designed to assess a particular, single level of the FS standards. All elements of the assessment (e.g. source materials, questions, presentation, language etc.) must be appropriate to the stated level.
Where an assessment is able to provide sufficient evidence of achievement at the level below the stated level, an 'exceptional' award may be made at this level. This type of award may be possible due to the nature of the FS standards, structured as progressive levels that subsume one another. However, partial achievement at a level does not attest to achievement at the level below; clear evidence of competence against the standards at a given level is required for an award.
Coverage:
13. Assessment must cover the requirements of the functional skills mathematics standards at a given level, and must avoid assessment of skills and knowledge outside the scope of these standards.
14. Awarding bodies must not combine assessment of more than one of the three functional skills of English, mathematics and ICT within a single assessment.
15. All language used in assessment material must be simple, direct, free from bias (s38) and covert or overt discrimination in either wording or context. Only information relevant to the assessment should be included. (This requirement does not proscribe the use of source materials that are incorporated to assess functional skills relating to learner choice, selection and judgement). As a general principle, the level of understanding demanded by the assessment that is not expressly contained within the mathematics standards should be at the level below that which is being assessed. This principle should apply to any content or information included in assessments.
16. The level of reading required in an assessment should be at the level below that being assessed and must not exceed the level being assessed. The demand placed on the reading aspect of the assessment must take account of both the level and duration of the assessment.
17. Assessment within the qualification must cover each of the process skills (representing, analysing and interpreting) and their key attributes identified within the FS mathematics standards (pages 4-5), as well as being referenced to the four elements of level differentiation: complexity; familiarity; technical demand; and independence (pages 6-7).
N.B. The process skills and their key attributes are included in Annex 1.
18. Process skills pervade functional skills activities. There should be opportunities to demonstrate each of the process skills (Representing; Analysing; and Interpreting) in equal or balanced proportions across a given assessment, although this balance may vary between individual questions or items.
Final FS assessment principles: second year of the pilot programme (QCA) v1.0
19. The assessment should measure the effective application of process skills and performance at the given level in purposeful contexts. The assessments should be designed to incorporate the different elements of the skills standards (process skills, performance and level differentiation factors) in integrated scenarios that reflect ‘real-life’ circumstances.
20. All assessments must require candidates to demonstrate each of the process skills and all of the performance requirements at the level being assessed.
21. At each level, assessment can be based on a sample of the ‘coverage/range’ items (FS standards mathematics v0.6 document: right-hand column) but all assessments must provide candidates with the opportunity to demonstrate a breadth of mathematical skills and include the assessment of number, geometry and statistics.
22. Where the assessment samples from the ‘coverage/range’ at a given level, awarding bodies are responsible for making judgements relating to coverage. These judgements must take account of the assessment’s intended purpose as a measure of a candidate’s functional mathematics competence as specified by the FS mathematics standards at the level being assessed.
23. Assessments should allow candidates to demonstrate their proficiency at the given level by utilising content that is explicit in the FS standards 'coverage and range' or equivalent mathematical skills and techniques as cited in the FS standards document. Where these techniques are sampled, ABs must make clear their approach to coverage and rationale in terms of sufficiency at the level. ABs must ensure that sampling does not undermine the measure of functional mathematics competence at the level or make assessment content predictable.
24. Assessment requirements and mark schemes must clearly show how the assessment model and each assessment based on this model relate to the requirements of the functional skills mathematics standards.
25. Assessment must acknowledge the requirement to achieve a level of proficiency at the level of the standard in accordance with the ‘performance’ standards as well as the overarching process skills (see Annex 1).
Assessment conditions:
26. All assessment must be undertaken in controlled, supervised conditions.
27. Awarding bodies may provide source materials to be used by candidates during controlled assessments.
28. Pre-release materials are permitted. Awarding bodies must provide clear guidance in relation to pre-release materials and the requirements for the use of these materials prior to the controlled assessment.
Pre-release materials should be distributed to candidates no more than 3-4 weeks of teaching and learning time prior to the summative assessment. In certain learning contexts where contact time is very limited, ABs may request an extension to allow equivalent time for learner engagement with materials. Pre-release materials can provide candidates with sufficient time to understand the nature of particular assessment contexts or scenarios but not
Final FS assessment principles: second year of the pilot programme (QCA) v1.0
as an assessed component. ABs should ensure that the requirements of an assessment are not apparent to candidates, or others, through their pre-release materials.
29. Access to calculators, reference materials, support devices and 'help' functions is not currently stipulated. Access is determined by the awarding body in line with their assessment design and should accord with the assessment objectives (for example, if estimation or mental arithmetic is being tested calculators should not be available for use. As part of their design, assessments should acknowledge the level of complication presented, the degree of accuracy required, and the level of credit, or marks, available in line with this availability, and assess the effectiveness of their use. Adequate centre and candidate guidance is essential). Awarding bodies will need to give consideration to the implications for security and reliability.
30. Assessment is of the candidate’s own ability to reach an outcome by independent application of skills. For this reason, candidates must not be given help by others (e.g. centre staff, external invigilators, peers) in order to meet the requirements of the assessment.
31. Reasonable adjustments should be made available to candidates with disabilities (s15). For each of these awarding bodies must publish to centres its procedures and share relevant data with QCA on request (s 17, s18, s20).
32. Special considerations should be made available to candidates who suffer temporary illness, injury or indisposition. For each of these, awarding bodies must publish to centres its procedures and share relevant data with QCA on request (s 17, s18, s20).
Duration:
33. The duration of assessment leading to a qualification should be no longer than three hours in its entirety, unless reasonable adjustments are put in place for a disabled candidate. The expectation is that at levels 1 and 2 the assessment of these skills will be approximately 2 hours but that at entry level greater flexibility may be required.
34. Where the assessment model includes pre-release of source material/s, awarding bodies must provide a description and rationale for their assessment approach and procedures relating to assessment materials that are to be considered outside of this duration. Where pre-release materials are used, awarding bodies will need to clearly stipulate for centres how the use of this material relates to the summative assessment.
35. Awarding bodies are required to offer a minimum of two assessment opportunities in the second year of the pilot programme. Awarding bodies should be able to demonstrate progress in their ability to deliver FS assessments more flexibly to meet the needs of candidates and centres. Awarding bodies will need to consider manageability for centres and clearly specify what is required in the guidance for centres.
Final FS assessment principles: second year of the pilot programme (QCA) v1.0
Annex 1: Process skills and their key attributes
Process skills:
The process skills are the learner’s ability to understand and make sense of mathematical information, to use and process that information, to interpret and analyse the results of their activity and to present this to others.
Process skills’ key attributes:
Representing
Making sense of situations and representing them
Analysing
Processing and using the mathematics
Interpreting
Interpreting and communicating the results of the analysis
A learner can:
• recognise that a situation has aspects that can be represented using mathematics
• make an initial model of a situation using suitable forms of representation
• decide on the methods, operations and tools, including ICT, to use in a situation
• select the mathematical information to use
A learner can:
• use appropriate mathematical procedures
• examine patterns and relationships
• change values and assumptions or adjust relationships to see the effects on answers in the model
• find results and solutions
A learner can:
• interpret results and solutions
• draw conclusions in the light of the situation
• consider the appropriateness and accuracy of the results and conclusions
• choose appropriate language and forms of presentation to communicate results and conclusions
Final FS assessment principles: second year of the pilot programme (QCA) v1.0
FS ICT Assessment Principles 2008-09
Final FS assessment principles: second year of the pilot programme (QCA) v1.0 Functional skills
Information and communication technology (ICT)
Assessment arrangements and principles for pilot
QCA review of assessment materials:
Awarding bodies should develop assessment materials using the principles provided in this document.
The assessment principles stipulate the awarding body design and delivery arrangements for functional skills qualifications in the second year of the pilot programme (September 2008 – September 2009). All assessments delivered from January 2009 must comply with these principles.
The assessment principles are subject to change during the FS pilot phase.
Teaching and learning:
Curriculum teaching and learning must cover the skills standards for ICT in their entirety at a given level. Awarding body guidance to centres on the delivery of the qualifications should emphasise this coverage requirement.
Section references:
Unless otherwise stated, the bracketed section references within this document relate to ‘The statutory regulation of external qualifications in England, Wales and Northern Ireland 2004’ document (Ref: QCA/04/1293).
Final FS assessment principles: second year of the pilot programme (QCA) v1.0
Design principles for the assessment of functional skills information and communication technology (ICT)
Entry level to level 2
Purpose:
The principles in this document provide the design framework for assessment of pilot qualifications in functional skills. These principles have been produced to define the parameters within which assessment models and materials should be developed for functional skills ICT pilot qualifications.
Assessment model:
1. The assessment must be designed to measure application of skills for a purpose within a "real life" context. These contexts should not appear contrived for the purposes of assessment.
2. The assessment can be entirely task-based, or a combination of (a) task/s with test-style items. Refer to principle 3 for permissible proportions.
Assessment instrument definitions
Task-based assessment:
Functional skills tasks are assessments that present [a] scenario/s within regular, 'real-life' contexts, which candidates engage with in order to resolve a situation or produce an effective and appropriate response or outcome. These assessment activities must reflect real-life situations and provide opportunities for candidates to apply and transfer their knowledge and skills for a legitimate purpose.
In line with the FS standards' level differentiation factors, task-based assessment requires candidates to identify and solve problems by selecting from, making choices about, and utilising different elements within an assessment scenario.
At L1 and L2, task-based assessment should include multi-step and multiple component activities, drawing upon a range of skills and requiring choice, selection and evaluation in line with the requirements of the FS standards.
Task-based assessment should support the application and demonstration of skills and competencies that are appropriate to their situation. In ICT, this must be practically assessed through the relevant ICT equipment.
Test-based assessment:
Functional skills test-based assessments include more specifically focussed objective test questions and assessment items, using fixed responses, multiple choice items, or short answer responses.
This type of assessment may use more disjointed or self-contained items in order to isolate particular skills and their performance, ensure sufficient coverage of the standards, or to corroborate achievement elsewhere, but should be conscious of overarching functional skills design aims to provide engaging and realistic skills-application opportunities.
3. Where test-style items are used they must be valid (i.e. to assess understanding and knowledge rather than problem-solving and practical application of skills).
In developing assessments that use both task and test-style items awarding bodies must ensure that:
Final FS assessment principles: second year of the pilot programme (QCA) v1.0
• the allocation of marks to those standards assessed through test-style items is not disproportionate to the way the standards are specified in full;
• test style items are in appropriate balance to items which assess problem-solving and practical application of skills
As guidance, standards assessed through test-style items should contribute no more than 25% of the total marks available.
4. Assessment of functional skills must support a holistic view of skills proficiency in relation to the functional skills standards at the specific level. It must not be atomistic. All mark schemes should be explicitly referenced to the skill standards but must not place undue weighting on individual standards nor make them individually conclusive.
5. Assessments must include details of how assessment questions and anticipated outcomes relate to the attribution of results for candidate performance; this must be mapped to the requirements of the FS standards.
6. The requirements of the assessment must be specified externally (i.e. by the awarding body).
7. Assessment items may be externally set by an awarding body or requirements may be externally set and provide for internally contextualised task-based assessments. These parameters apply at all levels (entry level – level 2).
8. Where assessment is proposed that is not 100% externally set and marked, awarding bodies must provide proposed arrangements for:
• quality assurance;
• standardisation / moderation within and across centres and examiners;
• external verification
Where an assessment includes witness statements, or similar internal assessment components, the requirements of the assessment and standard must be explicit. It must include details regarding the assessment context and expectations in terms of candidate performance at a given level. ABs are responsible for providing the documentation that supports these requirements and maintains auditable evidence of performance. ABs will need to make clear how they manage and audit these processes, including details of their qualitative review capability.
9. Drafting of assessment materials should minimise any later need to make reasonable adjustments for disabled candidates (s14) and where this is not the case ensure reasonable adjustments can be made available (s15). In the very last resort, component exemptions should be offered where barriers to disabled candidates remain (DCSF regulations Sept 2007).
10. Reasonable adjustments should reflect the candidate’s usual methods of working but they should not give the candidate an unfair advantage compared to candidates not using reasonable adjustments, or invalidate the assessment objectives set out in the specification (s16).
11. Awarding bodies must review all assessment materials to ensure they afford fair and equal access in terms of disability, gender, race, age, sexual orientation and religion/belief.
Levels:
Final FS assessment principles: second year of the pilot programme (QCA) v1.0
12. The assessment can be designed to assess a particular, single level of the FS standards. All elements of the assessment (e.g. source materials, questions, presentation, language etc) must be appropriate to the stated level.
Where an assessment is able to provide sufficient evidence of achievement at the level below the stated level, an 'exceptional' award may be made at this level. This type of award may be possible due to the nature of the FS standards, structured as progressive levels that subsume one another. However, partial achievement at a level does not attest to achievement at the level below; clear evidence of competence against the standards at a given level is required for an award.
Coverage:
13. Assessment must cover the requirements of the ICT skill standards and must avoid assessment of skills and knowledge outside the scope of these standards.
14. Awarding bodies must not combine assessment of more than one of the three functional skills of English, mathematics and ICT within a single assessment.
15. All language used in assessment material must be simple, direct, free from bias (s38) and covert or overt discrimination in either wording or context. Only information relevant to the assessment should be included. (This should not proscribe the use of source materials that are incorporated to assess functional skills relating to learner choice, selection and judgement). As a general principle, the level of understanding demanded by the assessment that is not expressly contained in the ICT standards should be at the level below that which is being assessed. This principle should apply to any content or information included in assessments.
16. The level of reading required in an assessment should be at the level below that being assessed and must not exceed the level being assessed. The demand placed on the reading aspect of the assessment must take account of both the level and duration of the assessment.
17. Assessment must cover each of the three skills areas identified by the ICT standards. Assessment mark schemes should accord with the following skills weighting ranges:
• Use ICT systems (15-30%);
• Find and select information (10-25%);
• Develop, present and communicate information (min. of 50%)
N.B. The mark scheme allocation will need to be determined as a percentage of total marks.
18. There should be opportunities to demonstrate each of the skill areas across a given assessment although this may vary between individual questions and items. These skill areas should be integrated where possible in assessments.
19. The assessment should measure the effective application of skills at the given level in purposeful contexts. The assessment must be designed to incorporate the different elements of the FS standards (skill standards,
Final FS assessment principles: second year of the pilot programme (QCA) v1.0
coverage and level differentiation factors) in integrated scenarios that reflect 'real-life' circumstances. The demonstration of the applied skills must be practically assessed through information and communication technology (ICT).
20. At each level, assessment can be based on either:
• all of the ‘skill standards’ (FS standards ICT document v0.6: left-hand column) and a sample of the underpinning ‘coverage’ items (FS standards ICT v0.6 document: middle column), or
• all of the ‘skill standards’ and all of the related ‘coverage’.
21. Where the assessment samples from the coverage and range column, awarding bodies are responsible for making judgements relating to this coverage. These judgements must take account of the assessment’s intended purpose as a measure of a candidate’s overall ICT competence as specified by the standards at the level being assessed.
22. Assessments should allow candidates to demonstrate their proficiency at the given level by utilising content that is explicit within the FS standards coverage and range. Where these techniques are sampled, ABs must make clear their approach to coverage and rationale in terms of sufficiency at the level. ABs must ensure that sampling does not undermine the measure of competence at the level or make sampled assessment content predictable.
23. Assessment requirements and mark schemes must clearly show how the assessment model and each assessment based on this model relate to the requirements of the functional skills ICT standards.
Assessment conditions:
24. All assessment must be undertaken in controlled, supervised conditions.
25. Awarding bodies may provide source materials to be used by candidates during controlled assessments.
26. Pre-release materials are permitted. Awarding bodies must provide clear guidance in relation to pre-release materials and the requirements for the use of these materials prior to the controlled assessment.
Pre-release materials should be distributed to candidates no more than 3-4 weeks of teaching and learning time prior to the summative assessment. In certain learning contexts where contact time is very limited, ABs may request an extension to allow equivalent time for learner engagement with materials. Pre-release materials can provide candidates with sufficient time to understand the nature of particular assessment contexts or scenarios but are not an assessed component. ABs should ensure that the requirements of an assessment are not apparent to candidates, or others, through their pre-release materials.
27. Access to reference materials, support devices and 'help' functions is not currently stipulated. Access is determined by the awarding body in line with their assessment design and should accord with the assessment objectives. (Assessments should acknowledge the degree of accuracy required, and the level of credit, or marks, available in line with this availability. The instrument should also assess the effectiveness of their use).
Final FS assessment principles: second year of the pilot programme (QCA) v1.0
28. Internet and e-mail are mandatory skills that must be assessed in all assessments at entry 3 and above (where necessary, this can be completed within a closed/secure electronic system or by simulation). The approach to this assessment will be determined by individual awarding bodies in line with their assessment design. Awarding bodies will need to give consideration to the implications for management, access, security and reliability (also applies to 26).
29. Awarding bodies should provide their proposals for how electronic access to external sources will be managed and their plans for how this will be monitored and evaluated.
30. Assessment is of the candidate’s own ability to solve a problem or reach an outcome by independent application of skills. For this reason, candidates must not be given help by others (e.g. centre staff, external invigilators, peers) in order to meet the requirements of the assessment.
31. Reasonable adjustments should be made available to candidates with disabilities (s15). For each of these, awarding bodies must publish to centres its procedures and share relevant data with QCA on request (s 17, s18, s20)
32. Special considerations should be made available to candidates who suffer temporary illness, injury or indisposition. For each of these, awarding bodies must publish to centres its procedures and share relevant data with QCA on request (s 17, s18, s20)
Duration:
33. The duration of assessment leading to a qualification should be no longer than five hours in its entirety, unless reasonable adjustments are put in place for a disabled candidate. The expectation is that at levels 1 and 2 the assessment of these skills will be approximately 2 hours but that at entry level greater flexibility may be required.
34. Where the assessment model includes pre-release of source material/s, awarding bodies must provide a description and rationale for their assessment approach and procedures relating to assessment materials that are to be considered outside of this duration. Where pre-release materials are used, awarding bodies will need to clearly stipulate for centres how the use of this material relates to the summative assessment.
35. Where the assessment includes pre-release of source material, the duration of the summative assessment should not assume the degree to which candidates have engaged with the material and should provide adequate reading time. ABs should consider the volume of pre-release materials in relation to the duration of their summative assessment in terms of overall burden.
36. Awarding bodies may provide source materials to be used by candidates during controlled assessments.
37. Awarding bodies are required to offer a minimum of two assessment opportunities in the second year of the pilot programme. Awarding bodies should be able to demonstrate progress in their ability to deliver FS assessments more flexibly to meet the needs of candidates and centres. Awarding bodies will need to consider manageability for centres and clearly specify what is required in the guidance for centres.
Final FS assessment principles: second year of the pilot programme (QCA) v1.0
Information and communication technology (ICT)
Assessment arrangements and principles for pilot
QCA review of assessment materials:
Awarding bodies should develop assessment materials using the principles provided in this document.
The assessment principles stipulate the awarding body design and delivery arrangements for functional skills qualifications in the second year of the pilot programme (September 2008 – September 2009). All assessments delivered from January 2009 must comply with these principles.
The assessment principles are subject to change during the FS pilot phase.
Teaching and learning:
Curriculum teaching and learning must cover the skills standards for ICT in their entirety at a given level. Awarding body guidance to centres on the delivery of the qualifications should emphasise this coverage requirement.
Section references:
Unless otherwise stated, the bracketed section references within this document relate to ‘The statutory regulation of external qualifications in England, Wales and Northern Ireland 2004’ document (Ref: QCA/04/1293).
Final FS assessment principles: second year of the pilot programme (QCA) v1.0
Design principles for the assessment of functional skills information and communication technology (ICT)
Entry level to level 2
Purpose:
The principles in this document provide the design framework for assessment of pilot qualifications in functional skills. These principles have been produced to define the parameters within which assessment models and materials should be developed for functional skills ICT pilot qualifications.
Assessment model:
1. The assessment must be designed to measure application of skills for a purpose within a "real life" context. These contexts should not appear contrived for the purposes of assessment.
2. The assessment can be entirely task-based, or a combination of (a) task/s with test-style items. Refer to principle 3 for permissible proportions.
Assessment instrument definitions
Task-based assessment:
Functional skills tasks are assessments that present [a] scenario/s within regular, 'real-life' contexts, which candidates engage with in order to resolve a situation or produce an effective and appropriate response or outcome. These assessment activities must reflect real-life situations and provide opportunities for candidates to apply and transfer their knowledge and skills for a legitimate purpose.
In line with the FS standards' level differentiation factors, task-based assessment requires candidates to identify and solve problems by selecting from, making choices about, and utilising different elements within an assessment scenario.
At L1 and L2, task-based assessment should include multi-step and multiple component activities, drawing upon a range of skills and requiring choice, selection and evaluation in line with the requirements of the FS standards.
Task-based assessment should support the application and demonstration of skills and competencies that are appropriate to their situation. In ICT, this must be practically assessed through the relevant ICT equipment.
Test-based assessment:
Functional skills test-based assessments include more specifically focussed objective test questions and assessment items, using fixed responses, multiple choice items, or short answer responses.
This type of assessment may use more disjointed or self-contained items in order to isolate particular skills and their performance, ensure sufficient coverage of the standards, or to corroborate achievement elsewhere, but should be conscious of overarching functional skills design aims to provide engaging and realistic skills-application opportunities.
3. Where test-style items are used they must be valid (i.e. to assess understanding and knowledge rather than problem-solving and practical application of skills).
In developing assessments that use both task and test-style items awarding bodies must ensure that:
Final FS assessment principles: second year of the pilot programme (QCA) v1.0
• the allocation of marks to those standards assessed through test-style items is not disproportionate to the way the standards are specified in full;
• test style items are in appropriate balance to items which assess problem-solving and practical application of skills
As guidance, standards assessed through test-style items should contribute no more than 25% of the total marks available.
4. Assessment of functional skills must support a holistic view of skills proficiency in relation to the functional skills standards at the specific level. It must not be atomistic. All mark schemes should be explicitly referenced to the skill standards but must not place undue weighting on individual standards nor make them individually conclusive.
5. Assessments must include details of how assessment questions and anticipated outcomes relate to the attribution of results for candidate performance; this must be mapped to the requirements of the FS standards.
6. The requirements of the assessment must be specified externally (i.e. by the awarding body).
7. Assessment items may be externally set by an awarding body or requirements may be externally set and provide for internally contextualised task-based assessments. These parameters apply at all levels (entry level – level 2).
8. Where assessment is proposed that is not 100% externally set and marked, awarding bodies must provide proposed arrangements for:
• quality assurance;
• standardisation / moderation within and across centres and examiners;
• external verification
Where an assessment includes witness statements, or similar internal assessment components, the requirements of the assessment and standard must be explicit. It must include details regarding the assessment context and expectations in terms of candidate performance at a given level. ABs are responsible for providing the documentation that supports these requirements and maintains auditable evidence of performance. ABs will need to make clear how they manage and audit these processes, including details of their qualitative review capability.
9. Drafting of assessment materials should minimise any later need to make reasonable adjustments for disabled candidates (s14) and where this is not the case ensure reasonable adjustments can be made available (s15). In the very last resort, component exemptions should be offered where barriers to disabled candidates remain (DCSF regulations Sept 2007).
10. Reasonable adjustments should reflect the candidate’s usual methods of working but they should not give the candidate an unfair advantage compared to candidates not using reasonable adjustments, or invalidate the assessment objectives set out in the specification (s16).
11. Awarding bodies must review all assessment materials to ensure they afford fair and equal access in terms of disability, gender, race, age, sexual orientation and religion/belief.
Levels:
Final FS assessment principles: second year of the pilot programme (QCA) v1.0
12. The assessment can be designed to assess a particular, single level of the FS standards. All elements of the assessment (e.g. source materials, questions, presentation, language etc) must be appropriate to the stated level.
Where an assessment is able to provide sufficient evidence of achievement at the level below the stated level, an 'exceptional' award may be made at this level. This type of award may be possible due to the nature of the FS standards, structured as progressive levels that subsume one another. However, partial achievement at a level does not attest to achievement at the level below; clear evidence of competence against the standards at a given level is required for an award.
Coverage:
13. Assessment must cover the requirements of the ICT skill standards and must avoid assessment of skills and knowledge outside the scope of these standards.
14. Awarding bodies must not combine assessment of more than one of the three functional skills of English, mathematics and ICT within a single assessment.
15. All language used in assessment material must be simple, direct, free from bias (s38) and covert or overt discrimination in either wording or context. Only information relevant to the assessment should be included. (This should not proscribe the use of source materials that are incorporated to assess functional skills relating to learner choice, selection and judgement). As a general principle, the level of understanding demanded by the assessment that is not expressly contained in the ICT standards should be at the level below that which is being assessed. This principle should apply to any content or information included in assessments.
16. The level of reading required in an assessment should be at the level below that being assessed and must not exceed the level being assessed. The demand placed on the reading aspect of the assessment must take account of both the level and duration of the assessment.
17. Assessment must cover each of the three skills areas identified by the ICT standards. Assessment mark schemes should accord with the following skills weighting ranges:
• Use ICT systems (15-30%);
• Find and select information (10-25%);
• Develop, present and communicate information (min. of 50%)
N.B. The mark scheme allocation will need to be determined as a percentage of total marks.
18. There should be opportunities to demonstrate each of the skill areas across a given assessment although this may vary between individual questions and items. These skill areas should be integrated where possible in assessments.
19. The assessment should measure the effective application of skills at the given level in purposeful contexts. The assessment must be designed to incorporate the different elements of the FS standards (skill standards,
Final FS assessment principles: second year of the pilot programme (QCA) v1.0
coverage and level differentiation factors) in integrated scenarios that reflect 'real-life' circumstances. The demonstration of the applied skills must be practically assessed through information and communication technology (ICT).
20. At each level, assessment can be based on either:
• all of the ‘skill standards’ (FS standards ICT document v0.6: left-hand column) and a sample of the underpinning ‘coverage’ items (FS standards ICT v0.6 document: middle column), or
• all of the ‘skill standards’ and all of the related ‘coverage’.
21. Where the assessment samples from the coverage and range column, awarding bodies are responsible for making judgements relating to this coverage. These judgements must take account of the assessment’s intended purpose as a measure of a candidate’s overall ICT competence as specified by the standards at the level being assessed.
22. Assessments should allow candidates to demonstrate their proficiency at the given level by utilising content that is explicit within the FS standards coverage and range. Where these techniques are sampled, ABs must make clear their approach to coverage and rationale in terms of sufficiency at the level. ABs must ensure that sampling does not undermine the measure of competence at the level or make sampled assessment content predictable.
23. Assessment requirements and mark schemes must clearly show how the assessment model and each assessment based on this model relate to the requirements of the functional skills ICT standards.
Assessment conditions:
24. All assessment must be undertaken in controlled, supervised conditions.
25. Awarding bodies may provide source materials to be used by candidates during controlled assessments.
26. Pre-release materials are permitted. Awarding bodies must provide clear guidance in relation to pre-release materials and the requirements for the use of these materials prior to the controlled assessment.
Pre-release materials should be distributed to candidates no more than 3-4 weeks of teaching and learning time prior to the summative assessment. In certain learning contexts where contact time is very limited, ABs may request an extension to allow equivalent time for learner engagement with materials. Pre-release materials can provide candidates with sufficient time to understand the nature of particular assessment contexts or scenarios but are not an assessed component. ABs should ensure that the requirements of an assessment are not apparent to candidates, or others, through their pre-release materials.
27. Access to reference materials, support devices and 'help' functions is not currently stipulated. Access is determined by the awarding body in line with their assessment design and should accord with the assessment objectives. (Assessments should acknowledge the degree of accuracy required, and the level of credit, or marks, available in line with this availability. The instrument should also assess the effectiveness of their use).
Final FS assessment principles: second year of the pilot programme (QCA) v1.0
28. Internet and e-mail are mandatory skills that must be assessed in all assessments at entry 3 and above (where necessary, this can be completed within a closed/secure electronic system or by simulation). The approach to this assessment will be determined by individual awarding bodies in line with their assessment design. Awarding bodies will need to give consideration to the implications for management, access, security and reliability (also applies to 26).
29. Awarding bodies should provide their proposals for how electronic access to external sources will be managed and their plans for how this will be monitored and evaluated.
30. Assessment is of the candidate’s own ability to solve a problem or reach an outcome by independent application of skills. For this reason, candidates must not be given help by others (e.g. centre staff, external invigilators, peers) in order to meet the requirements of the assessment.
31. Reasonable adjustments should be made available to candidates with disabilities (s15). For each of these, awarding bodies must publish to centres its procedures and share relevant data with QCA on request (s 17, s18, s20)
32. Special considerations should be made available to candidates who suffer temporary illness, injury or indisposition. For each of these, awarding bodies must publish to centres its procedures and share relevant data with QCA on request (s 17, s18, s20)
Duration:
33. The duration of assessment leading to a qualification should be no longer than five hours in its entirety, unless reasonable adjustments are put in place for a disabled candidate. The expectation is that at levels 1 and 2 the assessment of these skills will be approximately 2 hours but that at entry level greater flexibility may be required.
34. Where the assessment model includes pre-release of source material/s, awarding bodies must provide a description and rationale for their assessment approach and procedures relating to assessment materials that are to be considered outside of this duration. Where pre-release materials are used, awarding bodies will need to clearly stipulate for centres how the use of this material relates to the summative assessment.
35. Where the assessment includes pre-release of source material, the duration of the summative assessment should not assume the degree to which candidates have engaged with the material and should provide adequate reading time. ABs should consider the volume of pre-release materials in relation to the duration of their summative assessment in terms of overall burden.
36. Awarding bodies may provide source materials to be used by candidates during controlled assessments.
37. Awarding bodies are required to offer a minimum of two assessment opportunities in the second year of the pilot programme. Awarding bodies should be able to demonstrate progress in their ability to deliver FS assessments more flexibly to meet the needs of candidates and centres. Awarding bodies will need to consider manageability for centres and clearly specify what is required in the guidance for centres.
Final FS assessment principles: second year of the pilot programme (QCA) v1.0
Subscribe to:
Posts (Atom)