Suspicious CA3 pass mark/technical difficulties

Hi,

I sat this exam for the first time in this format (3rd time in total), and I was obviously disappointed to not see my name on the pass list on Thursday night.

I was even more disappointed when I saw my exam results letter and achieved a mark of 62. This standard would have been good enough to pass the March/April sitting as the pass mark then was 60. So why is my standard of communication no longer considered to be good enough?

Also, in the office I work in, nine students sat the August exam, four of which experienced technical difficulties with the CA3 app (despite there being no issues with their test presentations), these technical difficulties included:

• Having to redo their presentation hours after they believed they were finished, and given less than 30 minutes to do it after finding this out.
• Having to do their exam on movie maker, and app they would have been unfamiliar with.

All 4 of these students passed with a mark of 65 - bang on the pass mark. My actuarial brain tells me that this is no coincidence, and there is a clear correlation between those having technical difficulties and achieving exactly the pass mark. Whilst I do have sympathy and agree that these candidates should be compensated for the inconvenience caused by the IFoA, I can't help but think that those who did not have any technical difficulties with the app are having to achieve a higher than normal standard to achieve a pass, in order for the pass rates across sittings to remain stable. This evidence just makes me very suspicious, and I just don't see how this exam could have been marked consistently across the board.

For info, the pass rate for the August exam was 41.8% (296/708), and the pass rate for the March/April exam was 39.9% (135/338).

I guess the counter argument is that the August exam was easier than the March/April exam. I did not sit the exam in March/April as I was sitting SA2 just 2 weeks after the earliest of these dates! However, I spoke to a couple of colleagues who had sat both papers, who said that the August paper was easier.

I have emailed Education Services on this, making the same points I have made above. I will provide an update should they respond. I have also asked for a 'Subject Access Request' to get information on the marks awarded for my exam (doing this as one student alarmingly found out that the IFoA where not following the marking procedures set out in the student handbook). I am also debating whether or not to appeal this result, but I do get the feeling that I will be passing the IFoA £200 for nothing.

Let me know if you have any thoughts.

 

Hi Gavin
this was my fourth attempt and I failed with the mark of 62, same exactly as you did. The application was working well for me.

 

Hi Gavin, the situation you describe doesn't look good at all. I am encouraging people to get in touch with the student consultative forum (SCF) about all their CA3 issues and also cc the President and Chief Exec of IFoA. Actually I think, given the volume of complaints and concerns about this exam, that there should be an emergency extraordinary meeting between the SCF and the usual people, plus Chief Exec & Presidents of this profession, to focus on CA3 issues exclusively before Christmas.

For me it looks like the CA3 exam is in crisis and needs sorting out before next year's sittings. I'll get my result in February and if I fail then I'll be very nervous about doing a resit unless the profession has actioned the complaints from the SCF going back to June 2014. Also no doubt I'll be submitting a SAR also. I had averaged 61.5 on my presentation in the previous incarnation of this exam 4 years ago yet was failed with an FB (they refused to release written exam mark under SAR in those days), so it's interesting to see a pass mark of 60 going on this year...

If you dig into the notes from the SCF meetings, for one of the ST exams there was a spreadsheet blunder by the IFoA and they had to reverse passes and fails for some people 6 weeks after the results came out. I am just citing this example so that people realise mistakes have been made in the past, and so it's only proper we contact them to review the situation if we don't think things look right for whatever reason. Also I'd exhaust every possible complaint process & data protection rights option before handing over £200 to appeal a grade...

 

Hi Gavin,

This was my second attempt and I got exactly 62! A colleague of mine passed with exactly 65. Weird! I have also raised a SAR.

 

Hi aditya,

Do you know if your colleague experienced any difficulties with the CA3 app?
Thanks
Gavin

 

Interestingly, they have not publish the examiners' report yet. Is there a connection?

 

Hi,
I have received the following update from the Faculty:


As you were advised this has been passed to the online team on Monday and it is at the moment with the Online Education manager.
I am afraid I do not have an update on the outcome however I am happy to put a reminder on and chase again towards the of the week if you have not heard anything.

 

As I remember, in one of the tutorial videos in the IFoA app for this exam they say that there is no competition between students, meaning that if all of the students will achieve pass standard then all will pass.
Does this mean that the pass mark should be the same between the papers, and not variable like we currently see (60 in March 2016 vs 65 in August 2016)? It looks like they adjust the pass mark to the point where a share of passes is about 40%, which contradicts their own statement.

 

I see the examiner's reports are now available. I do see a few things I could have done better, but overall I believe I communicated the majority of the key messages. So struggling to see how I have failed. Will be interesting to see what my exam counselling report says on the matter.

 

I see they asked in the August 2016 Q1 for a graph. This is very unfair given the time this absorbs & that it's meant to be a written exam. Fair enough to expect graphs in the presentation but there are obvious problems when they ask for it in the written question.

No acted prep
Acted assignments & mock papers don't ask for graphs in Q1, so you've not had a chance to practice a written question with a graph in it. None of the marking schedules from acted had any marks for a graph.

This is one example where acted should stand up for its students to the IFoA and explain none of us are coached to do this. Therefore the preparation by acted falls short.

Graphs take time
After you've spent time deciding what graph to do, making it, labelling it... even inserting the graph in a word document is tricky compared to powerpoint when you've not had practice. Plus you have to manually type the data into a spreadsheet in order to make a graph: we are not given a .csv data file. This eats into time even more. I bet many people struggled to finish this exam on time & do their communication skills justice... if you're rushing to the bell and don't have time to review properly what you've written then there's no way you can find time to identify possible jargon and re-write to make it better etc.

In my view they should add half an hour extra for this exam if they're asking for graphs.

Word count
Another problem is how strict they will then be when it comes to the word count? Q1 asks for 600 words (more than what acted tend to ask for). Do the labels, title and any comments on the graph itself count in the word count? We don't know, because we've never been prepared for this. That could make a crucial difference between pass and fail. Acted marking schedules gave 5/5 if you were within 50 words of what was asked, or 0/5 if not. If they take your uploaded word file and simply do a word count then any graph pasted in as a picture would not have the graph words counted? If I had sat this and got 62 I would enquire specifically on this point for starters.

If one dataset is not graphed and you talk about it then you get your word count up, which may mean you reach 600 words, but they could say this is waffle and you should have just graphed it. If you made a really good graph that captured both data sets then you've got less to write about and may fail to reach the 600 words...

In my view the graph request is very unfair and significantly eats into the time available to write 600 words.

No wonder the examiners' report said people did better in the presentation. Even if the presentation requires 2-3 graphs then it's just bullet points for the rest of it usually. This means 2 hours is fair enough. Yet in this they need a graph and 600 words in 1 hr 45 - not fair!

 

This line in the examiner's report is laughable:
"There was no evidence of candidates running out of time and therefore not completing their answer."

Do students need to submit something left mid-sentence without remembering to put yours sincerely and name at the end for them to think they allocate enough time?
I think this means we should tell the IFoA directly if time is a serious issue as otherwise they assume all is fine.

I see that in the March 2016 paper, where you needed only 60 to pass, that no graph was required. 650 words in March vs 600 in August. Graphs take way more time than 50 words!!! So unfair.

 

Plus their graph in the examiners' report doesn't have a title.

More importantly - they're saying they weren't so keen on some graphs compared to others! So some people could have failed this exam because of their choice of graph in Q1?!!? It's meant to be a written question!

How many marks were there for the graph part then... any ideas acted?

 

Yeah I am not sure how the IFoA could know if we were tight for time or not. I had finished my initial draft with only 2 minutes to spare to upload it. I had no time to review. I can't remember what the notes said exactly, but they suggested a fair amount of time to 'Review'. I never had a chance to review, to remove jargon, or re-write sentences for it to read a little better.

I was actually well over the word count (700+). I also decided to do a bar chart, picking 4 different contract amounts to show the different expenses under A, B and C. In my opinion, this is equally as good as the line graph shown, but I suspect I could be marked down as it is not in the 'model' solution.

 

Very intriguing. Am just curious if anyone got a pass grades above 65%.

 

Pass mark up 5% and asking for a graph would make the August exam harder. There's no use publishing a pass mark if people's raw marks are adjusted or scaled. I'd be sure to ask of the mark breakdown with and without the overall category's mark.

 

It would also be fun to ask if the pass mark were 60 instead of 65 then how many more people would have passed?

 

I am sure it would be a fair amount above the 40% pass rate, and therefore the IFoA would fail to meet there budget from resit fees. From the resits alone, the IFoA will receive £179,220 (412 fails * £435). On top of that you will also have those sitting it for the 1st time.

 

Sorry, I forgot to add. I chased the IFoA on Monday about getting a response to the email I had sent them. I was told that I would receive something towards the end of this week from the Online Education team. I won't hold my breath. Email below


I am afraid I do not have an answer for you as yet however I have spoken to the managers and they have advised me that you will hear from us by the end of the week so please bear with us a little bit longer.

 

There is supposed to be a student consultative forum meeting on November 18th. There are many new complaints regarding CA3 since the last meeting in June, as well as the unaddressed concerns from previous meetings going back to June 2014. Notes are published from these meetings on the IFoA website
https://www.actuaries.org.uk/studying/student-consultative-forum

I'll be very interested to see how many points will be raised, what the response will be and who will action...

 

Indeed that's a lot of money and my simple question is: how much of that goes back to help retakers pass this exam?

 

I didn't realise there was a meeting on the 18th November. One of our student reps usually sends an email in advance for us to pass comments, which I would for this.