Operant Conditioning

Scientific psychology in the 1930s and 1940s was dominated by

behaviorism, particularly a form of behaviorism inspired by Pavlov’s classical
conditioning. The dominant figure in behaviorism at the time was a man named
Clark Hull, who developed an elaborate theory of the relationship between stimuli
and responses. Ultimately, Hull’s approach failed, despite a remarkable amount
of research and elaborate theoretical efforts. Today most graduate students in
psychology would probably not recognize Clark Hall’s name.
In the 1950s and 1960s, a researcher named B. F. Skinner came to
dominate experimental psychology. Skinner was brilliant, a prolific researcher,
and an eloquent spokesman for a new idea about learning now known as
operant or instrumental conditioning. Skinner was also known as a radical
behaviorist, because of his extreme rejection of mental processes as having any
use in psychological theory. Although Skinner’s radical behaviorism has been
rejected by the great majority of today’s psychologists, his extensively developed
ideas about trial-and-error learning–operant conditioning–remain among the most
important discoveries of modern psychology.

Classical conditioning versus operant conditioning

One of the significant limitations of classical conditioning was that it could not
explain new responses, that is, responses not produced naturally by some
innately salient stimuli. Consequently, classical conditioning could explain why
we salivate or blink or feel fear or get excited, but it could not explain in a very
straightforward way how we learn language or do math or ride a bike. It could not
explain how we do anything new. It was these complex behaviors that intrigued
Skinner.

Thorndike’s Law of Effect

The starting point for Skinner’s theory was an idea developed in the 1890s by
psychologist E. L. Thorndike. Thorndike had conducted studies in which he put a
cat in a cage that he had designed: his “puzzle box.” The cage had a variety of
ropes and levers that the cat could manipulate to allow it to escape from the
cage. Thorndike would put food outside of the cage as a motivation for the
animal to escape. He then watched the behavior of the animal and recorded the
amount of time it took for the animal to escape from the cage based on pushing
certain levers and ropes.
What he observed was that the animal was motivated to move around the
cage a great deal by its interest in getting the food, but it showed few signs of
insight or strategic planning. However, If the levers and ropes were placed in
such a way that the animal could inadvertently brush up against them, eventually
it would do what was necessary to escape from the cage and get the food.
 When Thorndyke placed the cat back in the cage, it did not simply go back to
random behaviors, but instead tended to return to places in the cage where it had
been just before escaped and to repeat behaviors that it had shown just before it
escaped. Some of these places and behaviors were relevant (e.g., pushing some
lever), and some were not (e.g., meowing at some time); however if the
behaviors could be associated with escaping the box, they were more likely to be
repeated. Over many trials, the animal became increasingly efficient at doing just
what was necessary to escape.
Thorndike called this behavior in response to successful escape the law of
effect. In simple terms, the law of effect states that an organism will repeat
behaviors that are associated with successful outcomes and it will avoid
behaviors that are associated with undesirable outcomes.
Skinner’s Use of the Law of Effect
Skinner developed the law of effect in ways that Thorndike could never have
imagined. The law of effect is important because it clearly states the basic idea
that the consequences of behavior influence the likelihood that behaviors recur.
Quite simply, if behavior can be followed by something good or something bad,
the probability of the behaviors being repeated depends on these outcomes.
Skinner believed that the mechanisms of learning are the same for all
species. This conviction let him to study animals in hopes that he could discover
the components of learning with organisms simpler than humans, including
pigeons and laboratory rats.

Shaping behavior
The starting point for much of the work in operant conditioning is a process
called shaping. Recall that, in classical conditioning, the response must already
exist for the associative learning to be meaningful. In operant conditioning, the
target behavior does not need to exist to begin with, except to a very rough
approximation. Then, a series of rewards (or punishments) can lead to a
behavior that is almost entirely new for the animal.
To make sense of shaping, let’s use an example. Imagine you want to teach
your dog to roll over when you say “roll over”. First of course your dog must be
physically able to roll over, but that is fairly likely. However, it may never have
actually rolled over in its entire life. If you simply wait until the dog spontaneously
rolls over and then say, ”roll over,” you will probably never teach the dog that
particular trick. Instead, the best way to teach the dog to rollover is to teach it in
what are called ”successive approximations” to rolling over. So, with dog treats in
your hand, you say “roll over” and reward the dog if it even dips its shoulders--a
behavior that is likely to occur quite spontaneously. After a few treats for the
should dipping behavior, the dog is likely to dip its shoulders again when you say
“roll over.”
Now you change your standards: Only if the dog lays on the ground will I you
give it a treat. It is already dipping its shoulders—half way to laying down—so
eventually it lays down and gets a treat. You reward laying down several times
after you say “roll over.” The next step will be to have the dog lay down and roll to
its side before it gets a treat. Finally, perhaps after many treats for various
successive approximations, it will need to roll onto its back and onto the other
side before you will give it a treat—each time in response to “roll over.”
This idea that behaviors can be shaped by successive approximations,
starting with behaviors that are only vaguely like the final behavior, is the key to
understanding learning through operant conditioning. Pavlov was able to show
how new triggers could initiate an existing response, but this could only explain a
limited amount of human behavior. Through exploiting the law the effect, Skinner
was able to show how new behaviors could emerge. But Skinner’s idea was
more profound than that. Instead of relying on simple association of two stimuli,
Skinner proposed that some learning occurs through an animal behaving and
“noticing” (perhaps subconsciously) the consequences of its own behavior.
Skiinner made an important distinction between two different kinds of
consequences: positive consequences and negative consequences. In Skinners
terminology, a reinforcer is an outcome that leads to an increase in some
behavior. For example, If a child says “Can I have some candy, please?” and the
parent gives the child candy, then the child is more likely to say “Can I have
some candy, please?” the next time he wants candy. Why? Because the candy
itself was a positive outcome of asking. Somewhere in the child’s mind, he
registers the success as a consequence of the request. Similarly, if the child
screams and you give it candy, the child will be more likely to scream the next
time she wants candy. Why? Because candy was a positive outcome of
screaming the last time he wanted candy. You have reinforced screaming for
candy—silly you!
In Skinners terminology, punisher is an outcome that leads to the decrease in
some behavior. For instance, if a child screams that she wants candy, and you
give her time-out and no candy, then she is less likely to scream the next time
she wants candy. Why? Because the screaming led to an undesirable outcome.
The child may not know how to get candy from you, but screaming doesn’t seem
like a good strategy. The punisher decreases the likelihood of repeating this
behavior.
The Skinner Box
Much of the work on operant conditioning has been done on research
animals not on children or human adults. Skinner created in mechanized testing
environment called a skinner box. A Skinner box contains mechanisms that allow
the animal to make responses. These mechanisms include levers that a rat might
press or buttons that a pigeon can push with a peck of its beak. The Skinner box
also contains trays for delivering rewards. Decades before the development of
computers, Skinner created highly mechanized labs where rats, pigeons or other
test animals could be shaped to engage in specific behaviors to receive food or
drink or to avoid shocks or other unpleasant experiences. He could vary the
number of presses of a lever needed to produce a food pellet and he could
manipulate how reliably the food was produced when the desired behavior has
been shown. Through hundreds of experiments, Skinner and his students and
colleagues traced the progress of operant learning under a huge variety of
scientifically varied conditions.
Cognition and Operant Conditining
Modern psychologists believe that many of Skinners findings are
fundamental for understanding human behavior, particularly unconscious
behavior. If we are correct, every minute of every day some part of your mind is
registering the consequences of something you do and increasing the likelihood
that you will repeat that behavior if the consequences are good and decreasing
the likelihood if the consequences are bad. This type of registering
consequences of behavior is now called “statistical learning”—changing
probability of certain actions based on their consequences.
Operant conditioning does not require conscious awareness, though it occurs
whether we are conscious of it or not. Right now, your mind is recording the
consequences of all sorts of actions. The positive or negative consequences of
moving your hand in a particular way to manipulate your computer are being
recorded. The positive or negative consequences of your deciding to study now
instead of going out with friends are being recorded. The positive or negative
consequences of your smiling at that person over there are being recorded. And
each time you engage in behavior the probability that a behavior will be repeated
under similar circumstances in the future is influenced by what happens when
you do it now. You are continually being shaped by your experiences, whether
you know it or not.