changingminds.org

How we change what others think, feel, believe and do

| Menu | Quick | Books | Share | Search | Settings |

Experimental Effect

 

Explanations > Social ResearchStatistical principles > Experimental Effect

Description | Example | Discussion | See also

 

Description

The effect, or effect size, is an indication of the practical importance of an experimental result.

In essence, 'effect' is the gap between two measures, although it must be measured with a statistical value. A big effect means the two measures are very different, not just 'different' (which is what 'statistically significant' means).

Experimenters hence seek not only statistical significance but also a large effect. Sadly, they do not always find both in the same place.

Measures

The most common measure of effect is the Pearson correlation, r.

r = SQRT( SSM / SST)

Where SSM is the between-groups sum of the squares, and SST is the total sum of squares. 

A slightly more complex measure used to reduce bias due to sampling (as opposed to using a population) is omega, w. This is calculated as:

w = SQRT( (MSM - MSR) / (MSM + ((n-1) x MSR)) )

Where MSX is the mean sum of the squares (SS/df), with M being between groups and R being within groups. 'df' is the degrees of freedom.

Omega is typically used for ANOVA effect calculation (where MS is already used in the F-ratio).

Effect size

Cohen (1988) gives the rules of thumb for effect size for r (or omega):

  • Small effect: r = 0.1 (the effect explains 1% of the variance)
  • Medium effect: r = 0.3 (the effect explains 9% of variance)
  • Large effect: r = 0.5 (the effect explains 25% of variance)

Note that r is non-linear, and doubling it does not double the effect size.

Example

A measure of men and women in a study shows a statistically significant difference in their body mass index (BMI). r is calculated as 0.05 which is not that big an effect and considered not worth reporting.

However, when the data is segmented based on age, it is found that r is 0.43. The statistical significance is not as high, but the effect is much greater than for gender difference and considered to be carried forward as the main findings of the report.

Discussion

An experiment can report a statistically significant result, but this just says that the experiment has shown that there is a difference between two conditions, not that it is 'significant' in the usual sense of meaning 'big' and 'important' or otherwise earth-shattering in any way.

It is important in reporting experiments to indicate the effect as well as the significance. The American Psychological Association (APA) now recommends that all experimental reports include an indication of effect.

See also

Confidence, Two error types, Experimental power

 

Site Menu

| Home | Top | Quick Links | Settings |

Main sections: | Disciplines | Techniques | Principles | Explanations | Theories |

Other sections: | Blog! | Quotes | Guest articles | Analysis | Books | Help |

More pages: | Contact | Caveat | About | Students | Webmasters | Awards | Guestbook | Feedback | Sitemap | Changes |

Settings: | Computer layout | Mobile layout | Small font | Medium font | Large font | Translate |

 

You can buy books here

More Kindle books:

And the big
paperback book


Look inside

 

Please help and share:

 

Quick links

Disciplines

* Argument
* Brand management
* Change Management
* Coaching
* Communication
* Counseling
* Game Design
* Human Resources
* Job-finding
* Leadership
* Marketing
* Politics
* Propaganda
* Rhetoric
* Negotiation
* Psychoanalysis
* Sales
* Sociology
* Storytelling
* Teaching
* Warfare
* Workplace design

Techniques

* Assertiveness
* Body language
* Change techniques
* Closing techniques
* Conversation
* Confidence tricks
* Conversion
* Creative techniques
* General techniques
* Happiness
* Hypnotism
* Interrogation
* Language
* Listening
* Negotiation tactics
* Objection handling
* Propaganda
* Problem-solving
* Public speaking
* Questioning
* Using repetition
* Resisting persuasion
* Self-development
* Sequential requests
* Storytelling
* Stress Management
* Tipping
* Using humor
* Willpower

Principles

* Principles

Explanations

* Behaviors
* Beliefs
* Brain stuff
* Conditioning
* Coping Mechanisms
* Critical Theory
* Culture
* Decisions
* Emotions
* Evolution
* Gender
* Games
* Groups
* Habit
* Identity
* Learning
* Meaning
* Memory
* Motivation
* Models
* Needs
* Personality
* Power
* Preferences
* Research
* Relationships
* SIFT Model
* Social Research
* Stress
* Trust
* Values

Theories

* Alphabetic list
* Theory types

And

About
Guest Articles
Blog!
Books
Changes
Contact
Guestbook
Quotes
Students
Webmasters

 

| Home | Top | Menu | Quick Links |

© Changing Works 2002-
Massive Content — Maximum Speed

Site Menu

| Home | Top | Quick Links | Settings |

Main sections: | Disciplines | Techniques | Principles | Explanations | Theories |

Other sections: | Blog! | Quotes | Guest articles | Analysis | Books | Help |

More pages: | Contact | Caveat | About | Students | Webmasters | Awards | Guestbook | Feedback | Sitemap | Changes |

Settings: | Computer layout | Mobile layout | Small font | Medium font | Large font | Translate |

 

 

Please help and share:

 

Quick links

Disciplines

* Argument
* Brand management
* Change Management
* Coaching
* Communication
* Counseling
* Game Design
* Human Resources
* Job-finding
* Leadership
* Marketing
* Politics
* Propaganda
* Rhetoric
* Negotiation
* Psychoanalysis
* Sales
* Sociology
* Storytelling
* Teaching
* Warfare
* Workplace design

Techniques

* Assertiveness
* Body language
* Change techniques
* Closing techniques
* Conversation
* Confidence tricks
* Conversion
* Creative techniques
* General techniques
* Happiness
* Hypnotism
* Interrogation
* Language
* Listening
* Negotiation tactics
* Objection handling
* Propaganda
* Problem-solving
* Public speaking
* Questioning
* Using repetition
* Resisting persuasion
* Self-development
* Sequential requests
* Storytelling
* Stress Management
* Tipping
* Using humor
* Willpower

Principles

+ Principles

Explanations

* Behaviors
* Beliefs
* Brain stuff
* Conditioning
* Coping Mechanisms
* Critical Theory
* Culture
* Decisions
* Emotions
* Evolution
* Gender
* Games
* Groups
* Habit
* Identity
* Learning
* Meaning
* Memory
* Motivation
* Models
* Needs
* Personality
* Power
* Preferences
* Research
* Relationships
* SIFT Model
* Social Research
* Stress
* Trust
* Values

Theories

* Alphabetic list
* Theory types

And

About
Guest Articles
Blog!
Books
Changes
Contact
Guestbook
Quotes
Students
Webmasters

 

| Home | Top | Menu | Quick Links |

© Changing Works 2002-
Massive Content — Maximum Speed