changingminds.org

How we change what others think, feel, believe and do

| Menu | Quick | Books | Share | Search | Settings |

Experimental design principles

 

Explanations > Social Research > Design > Experinental design principles
Bias minimization | Sampling | Randomization | Noise reduction | Comparison | Groups | Control group | Pre-test and Post-tests| Falsification | See also

 

In designing a social research experiment there are a number of considerations and principles that are worth taking into account that help to create valid results. The conclusions that you can draw from your analysis will depend strongly on the principles you use.

Bias minimization

Researchers can easily introduce bias into results by accidental, subconscious or otherwise reduce the validity of the methods used. Many of the methods used seek to reduce this bias.

The subjects of research may also introduce their own bias, for example through Social Desirability Bias. What is said to them and what they understand may also need careful consideration, although this may raise ethical concerns.

Sampling

It is seldom possible to research an entire population, so a sample must be selected to represent the population. After the experiment, statistical analysis can be used to enable generalization back to the population.

There are many sampling methods that can be used and the appropriate method needs to be carefully selected.

Ideally, sampling is done randomly. In practice, samples are often selected based more on convenience and what is feasible. This need not invalidate experiments, but the analysis must be done carefully and conclusions drawn with care also.

Randomization

If a random sample can be selected, then statistics can be used to generalize, drawing conclusions about the entire population. Randomization is thus at the heart of a true experiment. Randomization also helps remove bias, where choices by the researcher, subject or others may lead to invalid results.

Selection and assignment

Randomization starts with selection of target (random selection). It may also apply in deciding what treatments are applied to which subjects, and when (random assignment). Where there are multiple groups, pre-selected subjects may be randomly assigned to groups.

Random assignment is sometimes difficult, for example in a study of two school classes, in which natural groups have to be used.

Probabilistic equivalence

Randomization creates probabilistic equivalence, where multiple test groups are declared statistically equivalent (two groups of people are never equal in all respects).

A comparison of pre-tests (for example with a t-test) will tell you how different groups in parallel experiments are. Typically, a 95% equivalence (alpha <= 0.05) is sought. This is particularly important when groups are not randomly assigned and probabilistic equivalence cannot be claimed.

Noise reduction

In electronics theory, a signal is made up of the original signal plus added unwanted noise, such as hissing on a radio. FM and DAB are methods to remove electronic noise and improve the 'signal to noise ratio'. The greater the signal-to-noise, the greater the effect.

There are two ways of improving the signal-to-noise ratio: increasing the signal or decreasing the noise. In social research, factorial design experiments increase the signal, whilst covariance or blocking designs aim to decrease noise.

In analysis, the test statistic may be based on the signal-to-noise principle (eg. the F-ratio).

Comparison

Comparison is a common way of eliminating noise. If two experiments are the same in all respects except for changes in target variables, then a comparison (effectively subtracting one from another) allows the 'signal' variables to be highlighted.

Groups

Experiments often use multiple groups, with each group forming a mini-experiment on its own, and with the findings from each group then being compared for further comparison, analysis and conclusion.

There are two main ways groups are used:

  • Similar people in each group, but with different treatments, for example in cluster sampling.
  • Different types of people in each group, but with the same treatment applied to each, for example in stratified sampling.

The same group may also be given the same treatment, but at different times or in different contexts, to see if they have changed or if they are affected by contextual factors (the context thus becomes part of the treatment).

Control group

A control group is one which includes the same type of people (preferably randomly assigned) as those in the treatment group.

The control group is a useful way of eliminating extraneous variables and reducing noise.

Pre-test and Post-test

Experiments that apply some kind of treatment can apply a test or measurement in two places: before the treatment, which is a pre-test and after the treatment, which is a post-test.

This allows for identifying change caused by the treatment by comparing before and after results from a similar test (for example a change in skill, attitude, etc.).

A post-test only may be used sometimes, for example in a 'taste-test', where the subject is asked about their experience in the treatment.

The term 'pre-test' is also used to describe a trial of a test before it is applied 'for real'. 

Falsification

Showing something to be true does not make it always true, even though you may repeat the experiment many times. If, however, you can show that it cannot be false, then you need go no further.

Thus, if you can show that the null hypothesis is false, then you have also proven that the primary hypothesis is true.

See also

Proving truth, Sampling terminology

 

Site Menu

| Home | Top | Quick Links | Settings |

| Disciplines | Techniques | Principles | Explanations | Theories |

| Blog! | Quotes | Guest articles | Analysis | Books | Links | Help |

| Contact | Caveat | About | Students | Webmasters | Awards | Guestbook | Feedback | Sitemap | Changes |

| Settings: | Computer layout | Mobile layout | Small font | Medium font | Large font |

 

You can buy books here

More Kindle books:

And the big
paperback book


Look inside

 

Please help and share:

 

Quick links

Disciplines

* Argument
Brand management
* Change Management
Coaching
+ Communication
Counseling
+ Game Design
+ Human Resources
+ Job-finding
* Leadership
+ Marketing
Politics
+ Propaganda
+ Rhetoric
* Negotiation
* Psychoanalysis
* Sales
Sociology
+ Storytelling
+ Teaching
* Warfare
Workplace design

Techniques

+ Assertiveness
* Body language
* Change techniques
* Closing techniques
+ Conversation
Confidence tricks
* Conversion
* Creative techniques
* General techniques
+ Happiness
+ Hypnotism
+ Interrogation
* Language
+ Listening
* Negotiation tactics
* Objection handling
+ Propaganda
* Problem-solving
* Public speaking
+ Questioning
+ Using repetition
* Resisting persuasion
+ Self-development
+ Sequential requests
Stress Management
* Tipping
Using humor
* Willpower

Principles

+ Principles

Explanations

* Behaviors
+ Beliefs
* Brain stuff
Conditioning
+ Coping Mechanisms
+ Critical Theory
+ Culture
+ Decisions
* Emotions
+ Evolution
Gender
+ Games
Groups
+ Identity
+ Learning
+ Meaning
Memory
+ Motivation
+ Models
* Needs
+ Personality
+ Power
* Preferences
+ Research
+ Relationships
+ SIFT Model
+ Social Research
Stress
+ Trust
+ Values

Theories

* Alphabetic list
* Theory types

And

- About
- Guest Articles
- Blog!
- Books
- Changes
- Contact
- Guestbook
- Links
- Quotes
- Students
- Webmasters

 

| Home | Top | Menu | Quick Links |

Changing Minds 2002-2014
Massive Content -- Maximum Speed