Don't put your clients in a box-

Lead them through Self-Discovery

The MBTI® instrument has come under considerable amounts of criticism recently in the popular press. For those of us who have trained people to use this instrument and have found it useful, this can be painful. We realize that the criticism often comes from wanting the instrument to do things it isn’t designed to do.

I’ve been teaching people to use Psychological Type, Temperament/Essential Motivators and other models of individual differences since 1976 and conducted MBTI® Certification Programs for 17 years. At first, we just took the instrument results as the indication of one’s type pattern. Then we learned that this often wasn’t as accurate as we thought it was. This inaccuracy is in part the basis for many of the critiques. The critics say they take the ‘test’ one time and it comes out different another time. The MBTI® Manual contains data that indicates that the agreement with instrument results ranges from about 60% -80% of the people agreeing with all four preferences ‘measured’ by the instrument. This really isn’t bad since as the Manual points out, chance would predict likelihood of this fit to be 6.25%. But the critiques don’t understand this. And in a way they are right. To base important life decisions such as career choice on something with an error rate that big is not a good thing.

There is room for good critique. One of the main problems is that the proponents as well as the critics are confusing the MBTI® instrument with typologies in general. The critics may fail to realize that their resistance may be more to down side of using typologies than to the instrument. They seem to ignore that there is a downside to trait models as well even though there is a lot of data supporting them. As Jennifer Selby Long points out, the trait models also have down sides because they present certain qualities as negative and ‘less than’ rather than taking a neutral stance. With type theory these same qualities are shown to be positive aspects of one’s personality. In the end, people want to be seen in a positive light so no wonder the lay public likes type. In my view, I think there are several polarities that we can address that could move the discussion forward to help us reap the rewards of good critique. It wasn’t until I read the second blog post by Adam Grant that some of the polarities became clearer. In the end, I have to agree with him that we might benefit from resolving our scientific disputes through critical experiments with a mediator. It would be challenging, but worthwhile.

In truth personality instruments do work, but not all by themselves and not for some of the purposes the critics want them too. So here are some points to consider that apply not only to the MBTI® instrument, but to all typology instruments.

Objective vs subjective realities. I think that the critics’ mindsets come from a perspective that the only reality is objective, ‘measurable’ reality. Yet there is ample support for the influence of our mental models on our behavior. These live in our subjective realities. In my experience most of the proponents of using type models in their growth and change work seem to be insisting that these other realities are more important so they don’t put much stock in research that is objective in nature. Both the objective data gathering approach and the attention to subjective realities are important. And if we don’t realize this, energies and resources will be spent in arguments that can’t be won.

Instruments are not assessments. Before I discuss the idea of different mindsets, I want to point out a distinction between assessment and instrumentation. An instrument is just the questions or the tasks the individual is asked to do that results in a score or output of some kind. An assessment, historically, involves clinical judgment that factors in many more data points along with the instrument results. The MBTI® instrument is just that—an instrument. For typology instruments to be useful, they need to be used as part of an assessment process than takes into account many variables as well as part of an ‘application’ area. There are so many things that can make instrument results inaccurate, and therefore invalid and unreliable. The mere complexity of what it means to be human includes variables such as the mind set of the individual, how the instrument is framed to them, the proposed uses of it such as on the job where there is fear of bias versus with a coach where there is safety, the context such as work or individual interest, and it goes on and on.

Types are patterns that are stable throughout our lifetime AND we are dynamic so detecting these patterns is not easy. In the InterStrength CORE Approach™ we talk about how we have a Core Self that is there from birth and yet at the same time we have a Contextual Self that is very adaptable and a Developed Self that reflects the influences of both nature and nurture. So which Self is responding to an instrument? How do we ensure that personality type instruments are used in a way that helps us get at the Constant Core, yet makes space for the other selves?

An Integral Approach to Assessment. Some time ago, I participated in a webinar by Dr. Zachary Stein in which he presented an Integral view of assessment. Dr. Stein described an assessment method that can be applied to personality type as we take into account the many variables that greatly influence the accuracy of identifying one’s typology. Using the AQAL Matrix from the work of Ken Wilber, he described some aspects we need to attend to in our assessment methods. This matrix describes the various domains of what it means to be human. It can provide us with a map to understand some of the polarities in the arguments and perhaps help us move to even better approaches to understanding differences.

The top two quadrants apply to the individual and the bottom two apply to the collective, or groups. The left two quadrants apply to our internal, subjective sense of self and relationships, and the right two apply to the external, objective aspects of being human.

Dr. Stein pointed out that traditional measurement ignores the upper left and the lower left quadrants. He described aspects of assessment that go with each quadrant. I’ve added my comments relative to personality type assessment using tools like the MBTI® instrument.

INDIVIDUAL

Upper Left, often referred to as the “I” quadrant:

• What aspect of the individual meaning making are we accessing? What information are we missing?
• What does the instrument or the process mean to the individual? 

This would be influenced by type as well as level of development and other factors. This is subjective and therefore variable with different individuals. It has to do with how the client interprets and makes meaning of the instrument questions. How does the instrument affect the client? What are his or her feelings about the situation; hopes, dreams, or fears about how the information will be used; and so on? In addition to some of these individual and subjective aspects of the instrument taking situations, we need to realize that when people take a type instrument, they are answering questions about how they subjectively value different aspects. We don’t know their frames of reference or mind sets in response to the items and the instrument-taking situation. This introduces so many variables that are not taken into account with such wide variability in frame setting from none to extensive. These same variables are at play with trait-based instruments, but since results are presented quantitatively, the consequences are not so noticeable.

Upper Right, often referred to as the “It” quadrant:

• Does the assessment method involve observable behaviors and not just self-report that derives from subjective meaning making?
• How objective is the instrument being used?

How is its reliability (repeatability) affected by the contexts the individual is in?

This is objective behavior and the domain of validity and reliability. Can we have confidence in the scores? Are the responses to similar items consistent? What are the scores? Most personality instruments are judged on this basis. However, they ignore that the instrument results are not actual behaviors, but self- reports of preferences, which are really Upper Left and therefore subjective. To expect a direct correlation between behavior and self-report is unrealistic. There are many influences on our behaviors, including adaptation and development. The critics ignore these other influences and expect more than the instrument is designed to deliver. No wonder the instruments aren’t working the way they are expected to in research or just plain test-retest for the individuals.

COLLECTIVE (the group)

Lower Left, often referred to as the “We” quadrant:

• How do the various cultural contexts and relationships influence the self-report?
• How will they be treated when the information is shared?

How will the results be perceived in these cultures? What biases are present that can influence how someone responds and interprets descriptions?

The modal type of the culture can have an effect as well as other cultural factors. This is where the influence of cultural meanings and values comes into play and these influences greatly influence how the individual responds. There is also the issue of how the results will be used in the culture of the organization. If certain type preferences are given privilege in that culture, they will likely be seen as less desirable than others and maybe it is in the best interest of the individual to not respond from her or his core self. Relationship and cultural variables are major sources of reliability and validity variables. And the critics ignore that best practices with type models and instrument control for this. With good typology instruments and descriptions there is a lot of effort put into developing the items and descriptors to counter the effects of social desirability.

Lower Right, often referred to as the “Its” quadrant:

• How will the results be used? 
• Do these uses fit the intended uses? 

How do these uses influence the meaning made by the individual in responding? What are the influences of the setting? If the assessment is done in a work context, is there time for adequate self-reflection? Does the individual think of their job duties rather than how they operate outside those described behaviors? This is the influence of the fit between the instrument and its function. How useful will it be for the applications areas? In the arena of personality types, this becomes one of the biggest issues because while type indicates some likely talents, it doesn’t predict skill so to use it for selection or promotion is not appropriate. Yet the critics don’t get this point.

So you see, personality instruments are difficult to construct and administer in such a way that all of the areas are covered. As Dr. Stein points out, Integral psychometrics are ones that do not violate the integrity of the individuals, they adhere to the objective qualities of good instrument construction, they are used ethically, and they are shown to be useful in the systems they are being used in. Single instruments do not do the whole job. Assessment methods based on sentence completion might hold some hope, but this is expensive.

No wonder instruments don’t work…by themselves. As noted, one of the reasons personality instruments don’t work is that we are expecting an instrument to do the job of assessment. Assessment includes a much broader look than just the data points from a a single instrument. In the world of tests and measures, it includes clinical judgment. The instruments have to be part of an assessment process, not just isolated as ‘tests.’ And yet in most environments and cultures, we look to ‘tests’ to give us answers. Often the professional capable of doing an assessment is either not present and a ‘test’ is used from the web. In the world of type, the individual being assessed as also qualified to provide input into the clinical judgment. However, unless there are full narrative reports provided and encouragement to explore, this is not enough clinical judgment to make it a good idea to not have a professional involved. And even when a professional is engaged, organizations reduce the time they are given to do their job well. In addition many instrument certifications don’t provide a method to help the professional do the assessment aspect and may ignore the contracting skills needed to get the time and frame setting that will ensure a job well done. Then we wonder why the instruments come under attack!

It is up to the practitioner to get the knowledge base, skills, and tools to facilitate the assessment process. For typology, this is not likely to be the focus of brief, publisher-run certification programs, although ethics are covered. Some of the best sources of getting the knowledge base comes through organizations like the Association for Psychological Type International through their publications, training, and ethics statement.

It has been my life work to write material and create products and training that furthers this purpose, so I hope you explore the resources on this site. The InterStrength CORE™ Approach is an Integral approach to personality assessment and covers all the quadrants of the Integral Matrix. It provides tools, processes, and materials to engage clients in a self-discovery process that realigns them to their CORE and that helps round out the assessment process.

Myers-Briggs Type Indicator, Myers-Briggs, MBTI and MBTI Logo are trademarks or registered trademarks of the MBTI® Trust, Inc., in the United States and other countries.