Nowadays, the trend is to use computer-based tests to provide “a rich view of the candidates’ abilities“. However, the quality of these tests varies dramatically, and the hiring managers may not be aware of the typical problems with the tests.
Key Problems with the Screening Tests
In reality, the tests measure
- the state of your mind – the more tired you are, the lower the score
- your familiarity with the question types – this is especially a problem for trick questions, such as finding patterns – if you have not encountered a particular pattern before, you will require extra time
- only after then your cognitive and other abilities.
In other words, the result of the standardized tests only shows how good you are at passing standardized tests. It does not show motivation, persistence, creativity, loyalty, risk taking and other key success criteria.
In addition, if you take the test repeatedly you are unlikely to achieve the same score – the test consistency is extremely poor even on the international tests such as GRE.
Finally, most these tests have been designed for native English speakers and, for example, analogies tests can be most subtle for even native English speakers.
Tips for the Hiring Manager
- Does the test assess the skills relevant for the job and are the results meaningful? For example, if you cannot use a calculator (unlike on the job) and the candidate is asked to manually perform complex divisions, do the results convey any relevant information?
- Is cheating (i.e. using a calculator or googling for solutions) acceptable or not? Some hiring managers consider it just fine since it reflects the workplace reality. Most frequent questions with complete solutions are available on the internet already but if you have limited time, it is less of a problem. Also proctored testing with two cameras (one from your PC and one from your cell phone) may work.
- You should aim at a pass rate of usually 70% and judging someone if he passed above 80 or 90 is highly unreliable. The more so that standardized tests look for standard people, and you are looking for unique, rare snowflakes.
- Whenever the test result is higher than a threshold but not good enough offer a retake – two or three times
- There are some employment agencies which offer you access to already “screened” top candidates for 20% of their annual salary. I strongly advise against using these services for their screening may be flawed and you can do the same or better yourself.
Tips for the Candidates
- Familiarize yourself with the question types.
- Take as many practice tests as possible. On Udemy you can get practice tests for most providers.
- If your dream employer uses tests and is unable to consider how unreliable they may be, look elsewhere.
Typical Screening Test Types
When hiring a new employee, it’s always useful to obtain a view of any candidate from as many angles as possible.
Type 1 – Cognitive Tests
One of the most used cognitive tests is from Criteria Corporation.
Sample Questions from What to Expect on the Criteria Cognitive Aptitude Test (CCAT) | Criteria Corp:
The problems with these tests are:
- For the Verbal questions, if you are not a native English speaker, the differences may be most minute
- For the Math questions, if you must perform complex multiplications or division on paper, without a calculator, then the test does not evaluate anything close to the job reality
- For the Spatial questions, if you have not encountered the pattern before, you are unlikely to answer the question correctly in the limited given time
In any case, most tests do not penalize you for a wrong answer, they count only correct answers. Thus, if you are not familiar with the particular question, just guess and move on.
Preparation Materials:
- JobFlare for Job Search | Play Games. Get Hired.
- Criteria Cognitive Aptitude Test (CCAT) – 6 Practice Tests | Udemy or 6 CCAT Real Tests – 2021 | Criteria Cognitive Aptitude Test | Udemy – note that the difficulty of Udemy tests varies significantly.
The employers look either at the number of correct answers or at its corresponding percentile. See this sample report along with the suggested score ranges by job position.
According to jobtestprep.com, the mean CCAT score is around 24.2. A score of 35 and above places the candidate in the top 10% of the distribution, and a score of 40 equates to the 96th percentile. - Practice Aptitude Tests: 1000s Of FREE Practice Questions
Type 2 – English spoken test
When hiring a non-English speaker, it’s principal to ensure he/she can understand spoken English and that he/she communicates clearly.
There are multiple tests for this, most with some problems – see We tested AI interview tools. Here’s what we found. | MIT Technology Review.
Type 3 – Coding tests
The coding tests are of two types – trick coding tests (e.g., use dynamic programming) and standard coding tests (e.g., implement a REST service).
For the trick coding tests, there are not that many of them – most of them have been already asked and their solution can be find on the Internet. The best preparation for them is to subscribe to Daily coding problem.
Type 4 – Personality Insights Tests
These tests aim at identifying career-related personality traits, such as assertiveness, competitiveness, cooperativeness.
Some firms have gone so far that they defined the minimum percentages on each of these traits for each role. In my view, that’s a deep misunderstanding since these tests can be gamed and your team benefits from as many traits as possible.
Examples from JobFlare 4.7 Has Arrived! | JobFlare Now Offers Personality Tests In-App:
Type 5 – Free-form Tests
These tests are most useful. You set up a problem, ask an open-ended question, and see how the candidate approaches the problem. What frameworks, examples, analogies does he/she use, how much creativity and risk-taking is present and so on.