Marking system

Hello,

I recently failed CP 2 exams by 2 marks. I made a request for SAR and just got my marks.

I was surprised to note that one marker 1 gave me 10 marks and marker 2 gave me 5 on the same question.

I fail to understand the rationale behind the wide discrepancy in the marking. Can any tutor or marker shed some light please?

Rgds

 

There is a level of subjectivity in the marking. These aren’t exactly multiple choice questions.

What one marker considers a distinct valid point another may not. Therefore the marks different examiners give will differ, in some cases substantially so.

I believe they aim for markers to be within 10 marks of each other overall.

If you think this is unfair, as your result depends on which markers you get not just your answers, you’re correct.

 

A couple of things to bear in mind before being too harsh:
- There are well over 50 markers involved in the process. Some have been marking for years, some are new. There will be differences.
- As the previous poster said - this isn't multiple choice, or straight bookwork. There are certain marks on the schedule which are a definite right or wrong, but there are a lot more that are around how well things are communicated, how clear the language is, how well the candidate has shown that they understand what's going on. etc. It's really difficult to pin this down in some cases.
- As an example, a candidate might misunderstand a key point, and take their model in a completely different direction. The marker still needs to follow that though, and mark for what they've done, even though much of it may not even be on the marking schedule.

That said, a lot of effort is put in to try to get marking consistent, and this has been improving in recent years. All the markers mark the same 6 scripts up front, with more senior markers reviewing how they're marking and checking up on consistency throughout. There is also a thorough third review / moderation process, where all scripts that meet certain conditions (yours would be included, being within 2 marks of a pass) will be third marked by a senior marker. This probably happens in around 20-25% of cases.

So rather than being 'unfair', it's as fair as we can make it. There's been a constant evolution in process, trying to make things more consistent and more fair. If you have any suggestions on how the process could be improved, I'd be happy to pass this on to the chief examiner for the subject.

 

I meant to say that some amount of unfairness is impossible to eliminate in these types of exam (as is the case for life in general!), and so students should make peace with this. I haven't marked myself but I know people who have and so have some insight into the process. This wasn't meant to be a slight on the markers or examiners!

 

One issue is that it is not valid to have your script reviewed because you think it was marked wrong. This is because the process means that any close scripts are marked by a senior marker.

However, the current generation of candidates have come from a school system where systematic remark applications are made in an effort to change results where marks are close to a grade boundary. The IFoA system effectively does the remark before the results are published but this is not well understood by candidates.

 

The issue that i would have with the script review process is that it's effectively one person marking your script and giving you a pass or fail. The IFoA could reduce costs if all scripts were marking by one individual - but this would be seen as unfair - hence they have two and take the average.

Two independent blind markers could on average give you a pass, and then "senior marker" could then give you a fail. And the fail is what you are given. In fact the average of all 3 markers could be a pass, but it's irrelevant as they just give you the "senior marker" grade.

Would you consider that a fair system when so much of it is subjective?

 

Yes, I would. If you're either a clear pass or a clear fail, the two initial markers will bring that out. If you're a borderline candidate, your fate then relies on a marker who's been involved with the subject for years (13 in my case, and I'm not the longest-serving), has marked 1,000s of papers, and knows what a pass looks like and what a fail looks like. This helps to ensure consistency over time as well as within the bounds of any one session.
Yes - there are cases where the average is a pass, and the final review is a fail. There are even cases where both initial markers give a pass, and the final review is a fail. However, out of 1,000+ candidates per session, there are maybe 5-10 that fall into those categories, and they're looked at thoroughly, and have to be justified clearly. They're also looked over again by the Chief Examiner as a fourth review.
To approach this from another point of view - while the pass mark is usually 60, it's not hard to get a few extra marks to get 65 (just read the examiners' report carefully and do what it says!). Then the potential unfairness disappears. Put a bit more effort in and stand out from the crowd. Own your destiny, rather than bemoaning your fate.

 

You say it might only happen to 5/10 per sitting - but if you were to falling to that 5/10 on more than one sitting you are going to lose faith in the system. You state that it's looked over again by the Chief Examiner as a 4th review - does this individual provide a mark, and if so why isn't it provided in the SAR - is this "review" available to a student under the SAR request?

1. I've had a SAR show me having a 22.5% difference between marker 1 and marker 2 - one a clear pass, one a clear fail - and the final mark was a fail. No explanation given.

2. I've had another SAR (different subject) - difference between marker 1 and marker 2 was 16% - and script review failed me by 1 mark. I'm not sure how someone can say that a subject paper is a fail by 1% when another independent blind marker says it's a clear pass.

In both instances above one marker mark was over 60%.

3. Same subject as 2 above, pass mark was 55, marker 1 gave me 65.5, marker 2 gave me 52, average of these is 58.75, rounded down to 58 (Pass) - goes to script review and i'm given 48.5 - no explanation - it's lower than the other 2. But 3 independent blind markers on average give me 55.33 (65.5+52+48.5)/3, which round down is 55 which is a pass. But i get a fail.

I've paid for tutorials twice for the said subject, but no matter what i do something is found that gives me a fail. The examiners report states that other alternative solutions will gain marks were appropriate. But they don't provide an alternative - so who actually knows what is alternative.

Industry norm of 10 mark difference gets mention when you try raise the issues with the IFoA, would a system like the following not be fairer?

10 marks or less difference between marker 1 and 2 - goes to senior marker
15 marks difference between marker 1 and 2 - two senior markers must review
20 marks or more difference between marker 1 and 2 - script must be remarked by 2 new independent markers and 2 senior markers.

Finally - You can own your own destiny when you can control the results. When your results are based on someone else opinion, you have no control over this.

 

Lots of great fair points from everyone. I'd agree with above that you have no control on the final results. However, I see Nick's point that you should work hard in order to secure a pass with margin for human error. Once at least one examiner provides a pass I believe that you will be reviewed by the senior examiner. It's a lot of work to correct the CP2 again and I'd guess it's messy with spreadsheet and document. It's best not to concentrate on the marking process but rather work on some of the guaranteed marks such as next steps. It's better for your mental health in the end, I have found. I've vented a few times about it. My advice is go again, prepare better this time by practicing papers and get someone to correct them objectively. April is just around the corner so if you get started now you'll be ahead of alot of students.

 

Sorry just realized this is a CP2 thread - i had the "SAR request" thread open in another tab - if a mod could move it - it would be great as none of my negative experiences lie with CP2.

Edit by Admin: The two threads sort of overlap now, it'll be tricky to move the comments around coherently. Probably best left as they are rather than try and combine them

 

Marking is always subjective. Would you prefer the GCSE/A level system (at least for Maths) which is roughly:

1) Each question is marked by one examiner (some boards use marking by question rather than script)
2) Results are published
3) Candidates can request remarks, if so a senior examiner remarks. They are only allowed to change the mark if the original mark was unreasonable, ie they can't change it just because they would give it a different mark if the original mark was a valid exercise of academic judgement.
4) Candidate get remark, they can appeal to the Chair of the qualification if they still don't agree but changes are rare at that stage.

 

As per the IFoA - the accepted industry norm is differences of 10 marks. I've personally experience difference up to 22.5% and there is no explanation given. This time round there is 17% difference between a marker and the final grade. There should be no actuarial exam that is subject to that level of subjectivity from a marker. There is a breakdown somewhere in the marking system for differences of 22.5% to emerge. And i think if there is that level coming through the script should be marked a 4th time.

As it's not my final exam i can't get counselling for the exam - yes i can get someone to objectively review my script - but if they agree with marker 1 - i'm no better off - in fact i'm in a worse position as i won't know what to work on.

I'm not based in the UK so no nothing about their exams or appeal process. But under that approach above - every single student exam result is based off 1 person marking them. Under the IFoA approach, there are a 3 different situations from:

• average of 2 markers if both score over the pass mark or
• 1 senior script reviewer or
• 1 senior script reviewer followed by a chief examiner review (although i can't recall seeing this stated in policy?)

It's not beyond the laws of probability that 2 independent senior markers could have difference of opinion as to what is a pass and what is a fail?

How many times would you sit an exam and put up with large discrepancies before you might think it's not fair?