How we change what others think, feel, believe and do

Psychometric tests

Disciplines > Human Resources > Selection > Psychometric tests

Description | Development | Discussion | See also

Description

Psychometric tests are used, as the name suggests, to measure some psychological aspect of the person. Most commonly in selection, this includes personality, ability and motivation.

Occupational tests are used to measure maximum intellectual performance in terms of attainment, ability and aptitude, or typical behavior in terms of motivation and temperament.

Typical tests identify the direction of interests and can be used to suggest types of jobs associated with these areas.

Attainment tests

Attainment tests are used to assess the level of achievement in a particular area, such as in high school examinations.

Aptitude tests

Aptitude tests assess potential in some target area, seeking to discover possible future capability. This is as opposed to ability tests, which seek current capability. They can be used to measure specific aptitudes or collective traits (eg. technical, verbal, numerical).

Intelligence tests

General intelligence tests include cognitive studies that focus on information processing and organization of knowledge.

These tests are often made up of batteries of sub-tests that each test a narrow range, such as arithmetic reasoning, verbal intelligence, etc.

Intelligence tests may measure two factors:

• Fluid ability: applying reasoning skills to novel situations (decreases with age).
• Crystallized ability: using culturally specific component (increases with age).

Intelligence is not normally distributed. At the bottom end, scores are tightly grouped, suggesting strong general factor. At the high end, scores show more independence between sub-tests, indicating specific intelligences.

Development

The overall approach to developing psychometric tests is to generate a large number of sample items, give them to a set of people and then keep only those that differentiate.

Maximum-performance questions are selected based on target-related factors. Questions here are based on right-wrong difference.

Typical-performance questions are selected based on personality, mood, attitude, temperament. Questions here are based on identifying differences in selected factors.

There are five methods of construction, as below.

Criterion-keyed

Criterion-keyed tests focuses on an external domain or criterion. Thus for interest inventory, criteria are interested related to specific occupational group.

They could be used in change to identify those who lack flexibility.

Example: MMPI

This method is criticized as having an atheoretical basis where selection of items based on empirical data on ability to differentiate. It addresses similarities and differences, not why these are so. The domain of test may be limited: for example, ‘mania’ in MMPI has only one criterion scale. The more specific a measure, the more limited it is by its generalizability. There can also be problems when moved from one context to another, especially across cultures.

Factor analytic

This identifies items that load onto one factor and not onto another. It has the advantage that scores always have the same meaning.

Development of the test seeks strong correlation between the item and factor.

Example: Cattell’s 16PF. He listed all personality traits he could find and gave tests to heterogeneous groups of adults. Then he used factor analysis to develop theory of structure and relationships (not for data reduction). This has since been correlated with 50 different occupations.

A key in doing this is the size of the sample group. The larger the group, the lower the standard error.

Item Response Theory has been devised to help test-developers assess the nature of differences.

Item analytic

This is a very simple method which correlates each item with the overall test. It is useful for eliminating unsatisfactory items prior to using factor analysis. This is useful in developing longer tests by eliminating weaker items.

There is a need to be careful here:

• Domain definition: e.g. avoid investigating trust by asking person if trustworthy.
• Bloated specifics: repeated coverage of same item leads to apparent high reliability.
• Transportability: these tests often based on social and other domain-specific values.

Thurston scales

These are widely used, particularly in assessing attitude. They identify statements concerning attitude, then assess relevance of these with a panel of experts.

Items are chosen on the standard deviation of rating given by experts (ie. Those they mostly agree on).

A high level of values tend to be in results, hence transportability is issue.

Guttman scales

These are less widely used.

Items in this method are sorted in terms of difficulty or intensity.
Problems include getting graduation, every item must correlate with total score. which needs many items and large samples.

Discussion

Bright people will tend to do well on many different types of test as such tests have a high correlation with intelligence. This reduces the value of the test in differentiating individuals.

Factors affecting test experience

Factors affecting test experience include:

• Test: pre-test information, type of test, language, instructions, structure, medium, timescales
• Person: experience, confidence, emotion, motivation, memory, culture
• Environment: Light, heat, humidity, noise, distractions, test administrator
• Computers: affect both developers and test-takers.
• Time: affects stress, ability to complete, alertness (time of day).  The test itself may also age, esp. when ‘semantically laden’.
• Test-taker:
  • Alpha ability: improves as a results of the test, which teaches them things.
  • Beta ability: improves management ability (eg. managing time, rtfq).
• Attention: to test taker (Hawthorne effect).

Criticisms and hazards

Criticisms, hazards and potential problems with psychometric test include:

• Inadequate definition of concept to be measured.
• Bias (undesirable) in differentiation (desirable) between test takers. Eg. gender bias.
• Poor application of tools, eg. inadequate job analysis, wrong usage of tools.
• Words defined differently by developers (eg. extravert, innovator), causing confusion.
• Misinterpretation of results by users.
• Not reading the test manual properly (which tells how/where it is to be used).

See also

Preferences, Social Research, Intelligence testing