Decision Making Competence Study

Decision-Making Study 1: Mark Ashton Smith & Matthew Checkley, 2012. In this study we construct a principle component analysis model of decision making competence.

Demographics

• There were 143 respondents for survey 1
• A majority of respondents were males (75%).
• The top five responding nationalities were: USA, UK, Australia, Canada, Germany.
• Most respondents were in the broad categories of ‘business’, ‘engineering’ (including IT). Many respondents were also students.
• Most respondents had bachelors or graduate degrees, but there was a wide distribution.

71 individuals from the first set of respondents took the follow-up survey.

Correlations between three decision making indices & life satisfaction

We used three measures of decision-making competence:

• DOI-N. A subset (12) of the items of the original Decision Outcomes Inventory (DOI) (Bruine de Bruin, Parker & Fischhoff, 2007) – labelled ‘DOI-N’ for ‘negative’ since the outcomes in this test are all negative.
• DOI-P. An analogous DOI-like test designed for achieving positive outcomes rather than avoiding negative outcomes – labelled ‘DOI-P’. This could be understood as capitalizing on opportunities.
• SE Self-Report.  A 6-item self-report for self-efficacy (see below).

For overall life satisfaction we used a short classic self-report test:

• Life Satisfaction. The ‘Satisfaction With Life Scale (SWLS) (Diener et al., 1985).

This data can be interpreted as follows:

• Both DOI-N and DOI-P are  positively correlated with the SE Self-Report, but not Life Satisfaction (although DOI-P is nearing significance with Life Satisfaction, as one might expect if one is capitalizing on opportunities).
• There is no correlation between the DOI-N and DOI-P. There are different abilities associated with avoiding errors (DOI-N) and capitalizing on opportunities (DOI-P). This is an important result because decision making competence does not only consist in avoiding mistakes, but also pro-actively achieving objectives. This needs to be measured too.
• SE Self-Report scores are strongly correlated with Life Satisfaction scores. This is interesting because it shows a strong link between perceptions of decision-making competence (two items on this test) and life satisfaction. The self-report items are shown below.

Self-Efficacy Self-Report

We created a new ‘Self-Efficacy Self-Report’ that was found to correlate highly with the SWLS (life satisfaction). This test has good internal reliability (Cronbach’s alpha = 0.87). This test shows a close link between perceived decision-making ability (items 5 and 6, and self-efficacy.

1. I feel overwhelmed. There aren’t enough hours in the day to stay on top of things. (Reverse question)

2. I have responsibilities and goals that demand creative and strategic thought, but I’m not getting to them. (Reverse question)

3. I have too many day to day demands and the backlog keeps growing. I can’t get any ‘real’ work done. (Reverse question)

4. I often experience stress, frustration or anger at not being able to stay on track with my goals and values. (Reverse question)

5. In consideration of the major life changing decisions I have made over the past 5 years, I believe the decisions I have made have been…

6. In consideration of the many day to day decisions I have made over the past 6 months, I believe that the decisions I have made have been…

Question item correlations with DOI-P, DOI-N, SE Self-Report and Life Satisfaction

A table showing whether there are significant correlations between the question items in the survey and the  indices can be downloaded here. In this table, red indicates highly significant, yellow indicates significant, and pink indicates nearing significance. Items with ‘inverse’ are inverse correlations. Items with a circled ‘r’ are scored in reverse.

In general there is a different pattern of results for DOI-P and DOI-N, with overlap of a few items.  The table below shows the items that are correlated with both DOI-N and DOI-P, those that are correlated with only DOI-N and those that are correlated with only DOI-P (inverse correlations are also shown).

click to enlarge

Items associated with both DOI-P and DOI-N

From this data you can see that items linked to both decision-making measures (positive and negative) are the following:

Thinking of scenarios

I try to anticipate different contingencies when I decide on a course of action, ensuring that I have flexible ways of dealing with them.

Learning from the past

I don’t often repeat past mistakes.

Self-control

I am good at saying ‘not now’ to things that are less important to my plans.
I don’t use the simple rule that I will take action based on what is the ‘latest and the loudest’
I don’t often procrastinate (leaving assignments to the last minute).

Grit

If I encounter obstacles while trying to implement a solution, I don’t often give up.

Life Goals

I feel that my life is characterized by long-term goals to which I am fully committed.

Items independently linked to DOI-N and DOI-P

It is clear from the data that there is a different pattern of results for DOI-N and DOI-P.

For example, items relating to carefulness, checking and rational analysis are associated with DOI-N while big picture thinking, leadership and dealing with pressure are associated with DOI-P.

Principle Components Analysis

We conducted a Principle Components analysis on the data and derived a 20 factor solution that accounted for approximately 80% of the variance. The factors were then interpreted based on item semantics.

A correlation matrix with these factors and their significance values is shown below. Red shading indicates labelled factors associated with DOI-N while blue shading indicates factors associated with DOI-P.  The red boxes indicate where the correlations are negative. Note the main difference between DOI-N and DOI-P.

• DOI-N is positively linked with Analytic-Process and Self-Management – and negatively with Feeling-Risk-taking (and Impulsiveness-Distractibility, marginally).
• DOI-P is positively linked with Instinctive-Robust-Results-Focused.

Both DOI-N and DOI-P are positively linked with Commitment.

Multiple Regression Analysis

We then conducted a multiple regression analysis using these factors as predictor variables, and scores on DOI-P and DOI-N as the dependent measures.

This graph below shows the relative weightings of the factors that predicted DOI-N, using a forwards stepwise method. The amount of variance explained by the model varied from 0.37 to 0.53 depending on the regression method.

Commitment is measured by items like:

• I make some decisions after an ‘incubation’ period of several days  until I feel the the time is right to commit to a decision.
• I can be quite ruthless in eliminating what is unnecessary when I have a clear goal to achieve.
• Once I have committed to a decision, I rarely go back on it.

Self-management is measured by items like:

• I determine priority levels for my different jobs in the week, and then work on the task that has the highest priority first.
• I prefer situations where I have to depend on my own ability and judgement, not on someone else’s.
• In making a decision I am always clear about the goal I want to attain and what results I want from my actions to attain my goal.

Analytic-Process is measured by items like:

• When making a decision, I consider various options in terms of a specified goal.
• When I make decisions I spend time weighing up the pros and cons of different options available to me.
• I reflect on the outcomes of my decisions and note whether they met my expectations or not.
• When I make an important decision, I try to reflect on any assumptions I am making that feed into it.
• I think in detail about different options available to me before making the decision.

This graph below shows the relative weightings of the factors that predicted DOI-P, using a forwards stepwise method. The amount of variance explained by the model varied from 0.14 to 0.39 depending on the regression method.

Instinctive-Robust-Result does not involve weighing up pros and cons of different options or reflecting on how things might have been otherwise (as Analytic-Process does). This factor is measured by items such as:

• I am good at knowing when to base decisions on my gut feelings and when to make more thought-out decisions.
• My ability to solve problems is not affected much by stress.
• I am good at making decisions under pressure.
• Getting what you want is not much due to luck.
• I enjoy being in a position of leadership.

Adding Demographic Variables to the Regression Analysis

Adding demographic variables – Gender, Age, Occupation Level and Educational Level –  to the regression analysis suggests the following:

• None of the demographic variables are predictive of DOI-N.
• Occupation Level is predictive of DOI-P. This makes sense if you consider that with better occupations, there are more opportunities to capitalize on.

The variance explained by Occupati0n Level, Instinctive-Robust-Results-focused, and Commitment is 0.21 to 0.38 depending on the regression method.

Summary

The results taken altogether indicate that:

• Commitment is essential to effective decision making
• Rational analysis and evaluation, self-management and self-control are useful for risk-avoidance (DOI-N) decision-making.
• Being more stress-robust, results oriented and instinctive (rather than analytic) is useful for ‘realizing-opportunities’ (DOI-P) type decision making.
• Occupation level enhances the ability to capitalize on opportunities.

[spacer]