Standards

© 2004 Arbeitskreis Assessment Center e.V.

Anlagen

German Standards for Assessment Center Operations 2004 (PDF DIN Format)167.67 KB

German Standards for Assessment Center Operations 2004 (PDF US-Letter format)161.82 KB

Notes:

These AC Standards are also available in a printer-friendly PDF version  (DIN-format, US-letter-format).
Download from our website at http://www.arbeitskreis-ac.de.

Many thanks to the following persons for their assistance in translating our standards:
Petra Claus, Stefan Höft, Diana E. Krause, Elmar Lammerskitten, Christof Obermann, David Purdy, Sabine Thies

Preliminary remarks:

• Introduction
• Objectives of the Arbeitskreis Assessment Center e.V.
• Objectives of the AC Standards
• Authors
• Structure of the Standards

The Standards in detail:
Standard 1: Clarification of Objectives and Integration
Standard 2: Job Analysis and Analysis of Task Requirements
Standard 3: Development of Exercises
Standard 4: Observation and Assessment
Standard 5: Selection and Preparation of Assessors
Standard 6: Pre-Selection and Preparation of Potential Participants
Standard 7: Preparation and Execution
Standard 8: Feedback and Follow-up Measures
Standard 9: Evaluation

Introduction:

Founded in the 1970s, “Arbeitskreis Assessment Center e.V.” is a forum for personnel selection and personnel development. It is an association of german-speaking experts working in industrial and service organizations with about fifty members and task force affiliates exchanging their experiences in personnel assessment and development.

Objectives of the Arbeitskreis Assessment Center e.V

• Development and enhancement of methods for qualitative personnel work, particularly personnel selection and development. This includes the development of quality standards for assessment centers, optimization and systematization of personnel selection, and personnel development instruments.
• Exchange of experiences referring to the implementation and execution of Assessment Centers and to the development and implementation of personnel development measures.
• Development and discussion of overall concepts in personnel development

Objectives of the AC standards:

The Quality Standards of Assessment Center Techniques were developed in 1992. Since 2002 the Quality standards have been revised and actualized by a task force in order to:

• Create a current basis for appropriate AC practices.
• Check the quality of offers for company use in order to identify unqualified offers.
• Facilitate transparency and clarity for decision makers and AC users.
• Further establish the acceptance of the AC method.

Members of the task force had been:
Jürgen Böhme, Reinhard Diesner, Ralph Glodek, Stefan Höft, Elmar Lammerskitten, Rainer Neubauer, Christof Obermann, Renate von Rüden

In February 2004, the association Arbeitskreis Assessment Center ratified the standards, which were then published. The copyright of the standards belongs to the Arbeitskreis Assessment Center e.V.

Structure of the Standards:

The task force drafted nine standards referring to the process of AC construction and implementation. First, the core principle of each standard is formulated by describing the content and benefit of each standard. This is followed by a discussion of best practices regarding the implementation of each standard. Finally, the “offences catalogue” marks procedures that are not acceptable for use in the AC.

1. Clarification of the Objectives and Integration

Principle:

Before developing and implementing an AC, the goals and general conditions of the application, as well as consequences for the participants, have to be communicated and clarified.

Benefit:

A very common, but misleading approach to AC practices is to say: "Let's develop an AC, and then many aspects of the process will become clearer by themselves". However, the benefits diminish when participants or other involved parties express resistance (e.g., junior staff might perceive the participation as punishment or middle management may fear losing their influence in personnel decisions, which they used to make on their own). Only by offering a suitable introduction can the desired benefits be achieved. One result of the clarification process could even be to abandon the AC implementation.

Implementation:

• What are the desired outcomes for the organization? These might include higher levels of commitment among key employees, a decrease in “departmental egos” or standardized screening of high potential candidates.
• Who is the internal client of the application? How can it be guaranteed that the thought-leaders of the organization (the “power promoters”) support this application?
• What influence shall the institution of the AC exert upon the organization's culture? What message is being sent?
• How should the necessary framework be established: goals, resources, budget, time horizon of institution
• How will the AC be integrated into the existing structure? That is, how much importance will be ascribed to the AC in comparison to the supervisor's opinion? What are the consequences for the participants?
• When using an internal selection AC, there are always losers. How should this problem be dealt with?
• What are the follow-up measures for the AC? What happens with the data? What validity is an AC result supposed to have? Which steps are planned with regard to placement and promotional decisions?

Offences:

• “Bluff packages”: Using the label “orientation workshops,” when the real intention is internal review and personnel selection. Conversely, using the label “internal selection AC” while not serving any selection purpose at all.
• Instituting an AC without involving human resources processes and procedures.
• Ambiguous positioning of the AC to avoid addressing the diminished role of line managers in regard to executive selection.
• Outsourcing all parts of the process to an internal or external consultant without ensuring the organization's commitment to final outcomes.
• Exaggerated clarification of objectives and integration ("Ask and integrate anybody").

2. Job Analysis and Analysis of Job Requirements

Principle:

Only a specific analysis of concrete job requirements allows a useful appraisal of aptitude.

Benefit:

An AC examines the fit between the person and the job. Prior to an assessment, a job analysis must be conducted. The outcome of this phase is a job requirement profile, which specifies the aspects that are crucial to job success. Detailed information is collected during this analysis, which describes the target dimensions. Furthermore, the job analysis serves as a basis for the following steps in AC construction.

Implementation:

Job analysis and analysis of job requirements serve the following functions:

• Documentation of relevant aptitudes and registration of observable, effective behaviors in these situations,
• Identification of knowledge, skills, abilities and other relevant characteristics that are necessary for job success (KSAOs),
• Establishment of minimum standards for the successful accomplishment of critical situations.

To fully ascertain all relevant aspects of the job, the analysis employs multiple methods of data analysis with different conceptual approaches, perspectives and ideas of man.
The following framework must be considered:

• The point of reference for the analysis is the concrete task in the specific organization.
• Participation of people who are actively involved in the development of the actual goal setting (decision makers, employee)
• Integration of defined business objectives and existing organization-specific personnel concepts and human resource development concepts.
• In addition to documenting current job requirements, future job requirements must be considered, as well. The profile of job requirements is detailed by specific behavioral indicators.

The profile of job requirements is detailed by specific behavioral indicators.

Offences:

• Abdication of job analysis and analysis of job requirements.
• Simple adoption of existing job requirements for other target groups or existing catalogs of job requirements from external consultants or other organizations.
• Collection of general dimensions without specificity.
• Partial collection of only past-related or future-related criteria.
• Using only a list of general ability characteristics.
• Exclusive use of only one method of analysis supporting only one diagnostical approach (test/simulation/interview).

3. Development of Exercises

Principle:

An AC consists of job-related simulations.

Benefit:

Whether a certain behavior is suitable or not depends on the general framework of task situations. Therefore, behavior can only be observed and assessed realistically in a situational context . In order to draw up an aptitude prognosis relating to a specific target position, the task and work situations must be re-enacted as realistically as possible. AC exercises simulate work situations consisting of job performance indices that are important for job success.

Implementation:

• Procedures must be based on the outcomes of job analysis and analysis of job requirements. They must allow for the observation of a broad range of behaviors and offer multiple observation opportunities for each job requirement.
• At least three different types of work situations must be simulated in an AC.
• Material for exercises and instructions for participants must provide a clear statement of the goals and expected outcomes of the exercise.
• Each job requirement must be recorded in at least two exercises (principle of redundancy).
• If role players are deployed, there must be precise role player instructions. These instructions must ensure a standardized level of difficulty, while still allowing for situation-specific responses, based on each candidate's approach. Furthermore, separate role player training is necessary.
• If role players are deployed, there must be precise role player instructions. These instructions must ensure a standardized level of difficulty, while still allowing for situation-specific responses, based on each candidate's approach. Furthermore, separate role player training is necessary.
• Non-simulations (tests, interviews) are only integrated in an AC if it is impossible to record job requirements sufficiently by simulations.
• Before adapting newly developed exercises, they must be pretested (using suitable persons) regarding their efficacy.

Offences:

• The use of exercises which do not reflect the job requirements of the target position or which do not correspond to the reality of the target position (e.g. “off-the-shelf” in-basket exercises of a generic nature), or which involve games of questionable ethics (e.g. games such as “NASA-exercise” or “survival-training,” etc.), or which use opinion surveys as behavioral samples.
• Ill-considered use of tests, computer simulations, or “off-the-shelf” AC-exercises.
• Overemphasis of certain kinds of exercises for economic reasons (e.g., group discussions).
• Use of procedures which only call for an expression of behavior intentions, e.g., 'how would you behave if', although direct simulations make sense and are feasible.
• Specification of an observation characteristic that is not adequately observable in the specific exercise, e.g. observation of the criteria 'learning ability' in a group discussion.
• Instructions for participants that include behavioral instructions (e.g. 'behave reluctantly') - so that the simulation becomes a spectacle.

4. Observation and Assessment

Principle:

Systematic observation of behavior is the basis of aptitude diagnosis.

Benefit:

An AC is characterized by the predominant application of work simulations. The recorded behavior of the participants is the central data pool, on which the assessors' exchange of impressions and conclusions is based. These documented observations serve as the fundamental basis for decisions regarding aptitude diagnosis and the identification of the profile of strengths and weaknesses for each participant. To ensure reliable and valid diagnoses, the application of a job requirement-related observational system is essential.

Implementation:

• Observations must be related to the job requirements. Job requirements are concretized by operationalizations for every exercise.
• The observational system must allow for a coherent connection between the observation phase and the assessment phase. Proper documentation of observations and assessment must also be guaranteed.
• ssignment plans must clearly regulate which requirement is measured in which exercise (job requirement-exercise matrix) and which assessees are observed by which assessors (observational rotation plan).
• Each behavior shown by each assessee must be recorded by at least two independent assessors in every exercise. Additionally, each job requirement must be observed in at least two exercises (principle of redundancy).
• To avoid assessment mistakes caused by overload and complexity, at most five (preferably three) job requirements shall be recorded.
• Each assessor must rate his/her specific behavior observations immediately after every exercise.
• Data integration to obtain an overall assessment rating (OAR) must be carried out promptly after the AC, in order to resolve discrepancies.
• The combination of data is carried out in a predetermined and clearly defined procedure that is consistent for all participants.

Offences:

• Instead of concrete operationalizations, abstract or vague words are used, which do not specify behavior-related interpretation.
• Indistinct, “ad hoc” systems of assessment, which do not provide a regulated process of observation and assessment.
• Assessment is effected independently of the observation by the use of undifferentiated general assessments ("good", "bad") which can only be traced back to general impressions.
• The assessors individual judgment is distorted by information sharing or exertion of influence (e.g. by observers of higher rank) prior the independent assessment.
• The intensity of the discussion varies among the assessees. Sometimes the first participants are discussed in more detail while decisions about the later participants are made hastily and without adequate discussion.
• Performance of other assessees serves as a judgment basis for a single assessee.

5. Selection and Preparation of Assessors

Principle:

Well-prepared assessors, who adequately represent the organization, are most suitable for making informed and valid decisions.

Benefit:

In the context of the AC, the assessors decide on the careers of others, so it is essential that they have sufficient knowledge of the target position, as well as the selection method and its implementation. Assessor training is an integral part of the AC program. It must enable the assessors to make valid diagnoses and informed decisions. Thus, the assessors will be able to recognize and consider risks and potential sources of mistakes during the AC procedure. The pre-selection and preparation of assessors can substantially influence the acceptance of an AC program, particularly the acceptance of its results.

Implementation:

• Assessors from internal departments must be functioning in a position that is at least one level above the target position.
• When selecting a group of assessors, it must be considered whether they adequately represent the organization (their experience in the organization and their professional background in reference to the target position).
• Depending on organizational policy and need, external consultants may complement the group of assessors.
• A well-balanced composition of experienced and new assessors is advisable.
• Every assessor who attends an AC for the first time must successfully complete assessor training that includes the following topics:
  • Information about aptitude diagnostics
  • Description of procedures
  • Review of job requirements and target position
  • The observational and assessment process
  • Separation of observation and assessment
  • Sources of mistakes in observation and assessment
  • Presentation of contents/objectives/process of the integration discussion
  • Feedback training
  • Reflection about responsibility and consequences of the assessor role
• Additional training is required when the contents of the AC change, new target groups are identified, or there are long periods between assessor assignments.

Offences:

• An assessor sends an unsuitable colleague as a substitute.
• An assessor is forced to attend an AC.
• Assessors are deployed without receiving sufficient training prior to the AC.
• Predominantly external experts or administrators select the new management trainees.
• In-house ACs: An assessor is a participant's direct supervisor.

6. Pre-Selection and Preparation of Potential Participants

Principle:

Systematic pre-selection and available advance information are the basis of the success of AC participants and the AC as a whole.

Benefit:

Pre-selection and preparation influence acceptance of the AC in the whole organization. Therefore, selection criteria must be announced prior to the AC and must be consistently followed for all participants. Information about the basic target, procedures, and risks of the AC must be provided to potential participants so that they are able to make an informed decision about whether to participate or not. Additional information about exercises and useful strategies of preparation must be provided to balance out the participants' different knowledge backgrounds and to make it possible to benefit from the AC experience. All arrangements also aim at preventing negative emotions ("being a looser") resulting from negative AC outcomes.

Implementation:

• Participation in the AC can be bound to formal criteria related to previous experiences (e.g., prior job positions, preparation workshops).
• All well-founded aptitude diagnostic procedures (psychometric test procedures, pre-interviews, etc.) may serve as methods of pre-selection if they refer to the determined job requirements of the target position.
• Only candidates with realistic chances of success should participate in an AC.
• Advance information should be given in a standardized way, e.g., in written form, to ensure the same level of information is given to all participants.
• The preparation for the AC should be supported by the particular supervisor. It includes competence-oriented coaching (training of skills), as opposed to strategies of deception (training of behavior patterns).

Offences:

• Pre-selection is effected arbitrarily and not systematically.
• Supervisor's nomination does not refer to stated criteria (e.g., wants to get rid of a disagreeable staff member, doesn't nominate his/her high performers, nominates the best specialist for a supervisor position etc.).
• The supervisor nominates his or her employee for the AC only to avoid giving negative feedback himself/herself .
• Self-nominated participants do not receive preparation and are thus disadvantaged compared to other participants.
• No information or incorrect information is provided about the goal (e.g., selection AC instead of pretended development AC) or about chances and risks of participation (e.g., implications of AC results on career development).
• No clear target requirements are provided for the exercises, or observation categories are kept secret

7. Preparation and Execution

Principle:

Good planning and presentation guarantee a transparent and target-oriented application of the AC technique.

Benefit:

An AC is a complex and dynamic process, and its procedures must be made clear and transparent. A responsible moderator and appropriate organizational resources enable all persons involved to completely concentrate on their tasks. Professional planning and implementation of the AC serve as a basis for fairness and respect to all parties involved.

Implementation:

• Planning of fixed dates, rooms, and involved groups of people is essential.
• The event takes place in premises that allow an undisturbed and confidential procedure.
• A differentiated and clearly arranged schedule allows all participants to discern when, where, and with whom (participants and assessors) exercises take place. The schedule should allow for a comparable sequence of exercises in order to avoid bias caused by sequencing effects.
• Only capable and trained people are used as role players.
• The moderator must ensure that time schedules and standards are maintained during processes of observation and the integration discussion. The moderator must be qualified for his/her tasks and must be in a position to maintain assessors' compliance to standards.
• Participants' waiting time must be minimized.
• The procedure must ensure exercise security, e.g., exercise material must be collected at the end of each exercise.
• At the beginning of the AC, information about the procedure and the rules is provided.
• Before the exercise starts, participants are informed about the criteria to be observed (only when methods have a developmental character).

Offences:

• The absence of a moderator for the AC.
• Assistants or observers are used as moderators.
• The moderator only reads the instructions and does not control the quality of the observation and judgment process.
• Spontaneous recruitment of participants, observers, or role players.
• Excessive demands on persons involved caused by unrealistically tight time schedules.
• Failure to plan for analysis time subsequent to every exercise.
• For some participants the exercise takes place in the morning, and for others the same exercise takes place in the evening.
• The integration discussion is conducted under time pressure.
• The AC takes place in public premises, which causes unavoidable disturbances.

8. Feedback and Follow-up Measures

Principle:

Every AC participant has the right to receive individual feedback in order to be able to understand the AC results and to be able to learn from it. After the AC, concrete follow-up measures must be derived and implemented.

Benefit:

Within the framework of the feedback process, every assessee must be able to receive essential information about his or her individual results and the behaviors underlying the decision. Only through this procedure can a candidate use the observations in terms of further development of competencies. If no or insufficient feedback is given, an AC will have limited use for the organization. High-quality feedback enhances the transparency of the method, which leads to higher acceptance by all parties involved.

Implementation:

• Feedback is given immediately after the AC to every candidate. Feedback is given to participants on a voluntary basis.
• Feedback is transmitted to a participant in a interview in a confidential framework.
• The procedure is supported by appropriate actions and is oriented toward the individual case.
• Feedback on concrete behavior is based solely upon observations made during the exercises and the outcomes of the integration discussion. The essential contents of the feedback conference are personal strengths and weaknesses referring to the job requirements. If the AC serves to make concrete decisions, the feedback conference should review the overall decision (OAR) and concrete recommendations for personnel development.
• Decisions based on AC outcomes must be quickly agreed to, documented, and communicated.
• At a later time, it is helpful to provide a summary of the essential topics to the participants.
• Data confidentiality and data protection must be maintained by all involved parties.
• After the execution of an internal AC, a concrete action plan of personal development measures is determined for each individual participant. This action plan is oriented to the job requirements of the present or future position. Its implementation is monitored regularly.

Offences:

• One or all participants do not receive feedback.
• Feedback is not behavior-based. Instead, sweeping statements are given (e.g., “You have a lack of social competence.”), or statements are made that are not based on demonstrated behavior.
• The assessors are systematically taken out of the feedback process, while other people (e.g. moderators) provide the feedback.
• Developmental recommendations are given which do not match with the participant's profile or are not related to the target position. (e.g. leadership workshop for specialists).
• Measures are neither planned nor implemented.

9. Evaluation

Principle:

Regular tests of reliability, validity, and quality control ensure that the objectives of the AC are achieved.

Benefit:

Every AC is a remarkable investment. Particularly newly delevolped ACs are nothing more than an agglomeration of hypotheses. Statistical tests of reliability, criterion validity, and construct validity ensure that the methods are constantly improved and legitimize the investments. Jedes AC stellt eine erhebliche Investition dar. Insbesondere ein neu entwickeltes Assessment Center ist aber zunächst nur ein Hypothesengebäude über den Zusammenhang zwischen Anforderungskriterien, AC-Komponenten und Bewährungskriterien in der Praxis. Die Güteprüfung sichert diese Zusammenhänge empirisch ab und sorgt dafür, dass das Verfahren ständig verbessert und so der notwendige Aufwand legitimiert wird.

Implementation:

• Reasons for an empirical quality control include the following:
  • First-time introduction of an AC to an organization
  • Adaptation of a given AC to a new target group
  • Adaptation of a given AC to sustainable modifications in an organization
  • Substantial changes in the AC procedure and/or AC material
• Also if the procedure has not changed, tests of reliability, validity criteria, and quality control must be repeated at least every two to five years. For evaluation, the following aspects are relevant:
  • Internal structure of the procedure: Procedure-relevant mistakes and restraints in the existing AC procedure must be identified and corrected early:
    • Adequate degree of difficulty of the particular exercises and dimensions
    • Satisfactory contribution of each exercise and dimension to the overall judgment
    • Exercises and dimensions must differentiate appropriately between participants
  • Quality of prognosis: Testing of prediction quality answers the question of whether the right candidates had been chosen or not. Therefore the correct criteria must be chosen:
    • Good predictive quality of potential/aptitude statements in terms of occupational success criteria
    • Tracing back the personal development recommendations that resulted from the AC
  • Acceptance and fairness: Acceptance constitutes the basic requirement for long-term application of the AC method and helps to create a positive image of the organization from the job applicants' point of view:
    • Fairness of the procedure for specific groups (e.g., repeated attendance by participants from different work areas and disciplines)
    • Acceptance of the procedure by participants, assessors, and other legitimately interested persons

Offences:

• Introduction and implementation of a new AC without testing the inner structure of the procedure.
• Instead of using empirical quality tests, confirmation is simply reached via acceptance or unsystematically sampled positive feedback.
• Quality control is solely carried out by the person/institution who/which developed the AC.
• Selecting the success criteria based on easy data availability.