EXAMPLES OF SOME PROBE PROCEDURES
THINK ALOUDS TO ASSESS COMPREHENSION
(Wade, 1990)
1. An excellent way to obtain information about both the individual's
product and the performance process.
2. Think Alouds are individuals' verbal self-reports about
thinking processes.
3. Allow us to obtain information re: how they attempt
to construct
meaning from text.
4. The general process of "think alouds" are:
-- Examiner provides a task and asks the individual to
say
aloud everything that comes to mind as they are performing it.
-- Only indirect cues used to elicit information when necessary,
"Can you tell me more"?
-- The remarks are recorded on a recorder and the nonverbals
are also jotted down
-- When used to assess comprehension, the examiner usually
has students think aloud after reading short segments of passage.
5. For Wade's application, it is important that the reading passages
are selected/written so the readers cannot know for sure what
the topic is until they have read the last segment.
6. Readers must generate hypotheses during their think alouds
about the text's meaning from the clues in each text segment.
7. Wade has found that there are descriptive categories of
comprehenders
A. THE GOOD COMPREHENDER
– interactive reader who constructs meaning and
monitors comprehension.
– tends to draw on background knowledge
– Makes reasonable inferences about the passages
– recognizes when information is needed to confirm
hypotheses
– Abandons ideas inconsistent with further passages
but constructs another that is consistent
B. THE NON-RISK TAKER
– A bottom-up processor
– Takes passive role by failing to go beyond the text
to develop hypotheses.
– May look for clues from the examiner not the text
– May frequently respond "I don't know" or may repeat
words or phrases verbatim
– When they develop a hypothesis it is often given in
a questioning manner.
C. THE NON-INTEGRATOR
– Drawing on text clues and prior knowledge, new
hypotheses are developed for every segment of the text
– These typically never relate to the previous hypotheses
or to information presented earlier in the text.
– Appears a curious mixture of top-down/bottom-up
processing.
D. THE SCHEMA IMPOSER
– a top-down processor who holds an initial hypotheses
despite incoming information that conflicts with that schema.
– Appears unaware of alternative hypotheses
E. THE STORY TELLER
– Extreme examples of top-down processors
– Draw far more on prior knowledge or experience than on
information stated in the text
– Many seem to identify strongly with a character and makes
causal inferences based on what they would do.
Wade's Procedure for a Comprehension Think Aloud
(Wade, 1990:445)
I. PREPARING THE TEXT
Choose a short passage (expository or narrative) written
to meet the following criteria:
1. Text should be from 80 to
200 words in length,
depending on the reader's age and reading ability.
2. The text should be new to
the reader, but on a topic
that is familiar to him or her. (Determine by means
of interview or questionnaire prior to this assessment).
3. The text should be at the
reader's instructional level,
which can be determined by use of an informal
reading inventory.
4. Topic sentence should appear
last, the passage
should be untitled.
5. The text should be divided
into segments of 1 to
4 sentences each.
II. ADMINISTERING THE THINK
ALOUD PROCEDURE
1. Tell the reader that he or
she will be reading a
story in short segments of one or more sentences.
2. Tell the reader that after
reading each section, he or
she will be asked to tell what the story is about.
3. Have the student read a segment
aloud. After each
segment is read, ask the reader to tell what is happening,
followed by nondirective probes questions as necessary.
The questions should encourage the reader to generate
hypotheses (what do you think this is about?) and to
describe what he or she based the hypotheses on (what
clues in the story helped you?)
4. Continue procedure until
the entire passage is read.
Then ask the reader to retell the entire passage in his
or her own words (The reader may reread the story first).
5. The examiner might also ask
reader to find the most
important sentence(s).
6. The sessions should be tape
recorded and transcribed.
Observations should also be recorded.
III. ANALYZING THE RESULTS
Ask the following questions when analyzing the transcript:
-- Does the reader generate hypotheses? How confident
of
them is he/she?
-- Does he/she support hypotheses with information from
the passage?
-- What information from the text does the reader use?
-- Does he/she relate material in the text to background
knowledge and experience?
-- Does reader integrate new information with the schema
already activated?
-- What does the reader do if there is information that
conflicts with this schema?
-- At what point does the reader recognize what the story is
about?
-- How does the reader deal with unfamiliar words?
PRE READING PLAN (PREP)
(Langer, 1982)
This technique is designed to aid interventionists in determining
where mediation should occur.
– Determine prior
knowledge that the student possesses
about the topic and how this knowledge is organized.
– Determines the language
the student uses to express knowledge
about a topic.
– Allows the interventionist
to determine how much additional
background information must be taught.
– Organized initially around
following questions:
a. Tell me everything you think of when you
hear.....
b. What made you think of............
c. Do you want to add to or change your first
response....
– Questions allow
the following from student:
a. Student free associates; provides access to
prior
knowledge.
b. Student reflects on thought processes; provides
access to organization of knowledge.
c. Student reformulates and refines responses;
provides access to their revision ability.
– If the student has
a lot of knowledge then the following
responses are expected:
1. Provides supra-ordinate concepts (higher class
categories)
2. Provides definitions with precision
3. Provides analogies as a substitution or comparison
for
literal concept
4. Provides linkages that connect one concept with
another
– If the student has
some
knowledge then the following responses
are expected:
1. Provides examples that are in the appropriate
class and
fairly specific
2. Provides attributes that are subordinate to
larger
Concept
3. Provides defining characteristics that provide
information on one major aspect of the concept
– If the student has
little
knowledge then the following
responses are expected:
1. Provides associations that are tangential but have
some
cognitive links
2. Provides morphemes
3. Utilizes a similar phonetic unit comparison (Sounds
alike)
4. Provides first hand experiences that are tangential
responses based on experience
5. Provides no apparent knowledge
– It must be remembered
that lack of knowledge may be
due to a number of factors; not just disorder or deficit.
– These behaviors
are not only diagnostic in nature, they
may also reflect strategies that may be needed at a
higher stage of knowledge acquisition during instruction
(target is always toward the highest level responses)
CLOZE PROCEDURES
These procedures require that a well-written piece of text is given to the student with every nth word deleted. The student then reads the text aloud and supplies the missing words so that meaning is sustained.
Across all areas of applied linguistics, this is perhaps the most widely-used
test of meaning-making. There are several uses of these types of
procedures:
– To determine whether reading material is
within the child's reading ability
– Assessment of a student's ability to do
"on-line
comprehending" so that
the context itself can provide
missing words to sustain meaningfulness.
There is much debate on several issues using cloze techniques:
– Whether to use written or oral cloze techniques
– Whether to delete every nth (random) word
or not
– If it is a random deletion, how often should
they be deleted?
– Deletion is non-random, how are the words
selected
– In scoring should you employ "Exact replacement"
or "not exact
replacements"
– Should the student be given an overview
of the story before reading it or not?
Of course, if you want to use a particular procedure developed and researched, you must use the strategies employed by those researcher/developers.
If not, the answers to these questions actually depend on the questions
asked and the purposes for the cloze procedures. Remember, you choose
procedures from the structured probe technology to answer specific questions
or hypotheses. Some direction (based on my experience):
– Oral or written cloze
*
If you are interested in actual literacy processing, it may be best to
use written cloze procedures since deriving meaning from visual
text is what reading is about.
*
However, you can use oral cloze to determine how well the child is
doing "on-going comprehending" of literary language.
– Random or non-random deletion
*
To determine whether the student can use the context in general,
a random deletion procedure seems more appropriate. That gives
you an opportunity to see how the student can weave together all
the potential cueing systems to negotiate meaning.
*
If you have a hypothesis about a particular cueing system, however,
then you can delete only words that relate to that system
(syntactic/semantic/graphophonemic).
– With random deletion, how often do
you delete?
*
Many procedures have their own instructions in this
*
Research tends to show, however, that anything more frequent
than every 5th word becomes too disruptive to the meaning-making
process. Oller and others suggest that every 7th word tends to yield
the best reliability and validity.
– Exact replacement or Not exact replacement
*
This again, depends on the purpose and procedure. For instance,
Pikulski and Tobin, (1982) require "exact replacement" with their
"readability procedure" While Kemp (1987) allows a "not exact
replacement" and provides some general "rules of thumb".
*
Otherwise, in keeping with the spirit of meaning-making and
on-going comprehending, it is probably best to use "not exact
replacement" so long as the terms are synonymous with the
target word or if the replacement maintains the general meaning
of the passage.
– Should the student be given an overview
of the passage at first.
*
Unless otherwise indicated, giving the child an opportunity to
use "initiation strategies" wherein they have a general understanding
and anticipation about the actual text certainly is consistent with
authentic meaning-making through literacy and should be done.
A few specific applications:
– Pikulski & Tobin, 1982
*
Use a written cloze to determine appropriate reading level
– Take the text in question, leave out every fifth word
– The student reads the text and when he gets to the
deleted word, he provides the best guess of the word
the author actually used.
– Only exact replacements are scored as correct. Not
even
synonyms that maintain the meaning are correct.
– If a student can fill in 40 to 60 percent of the blanks with
the same word as the author, the text is within the student's
instructional reading level.
– Kemp (1987)
*
Uses a written cloze procedure and creating blanks "that are
balanced" and that most parts of speech are sampled.
*
When using "not exact replacement" (NER), he suggests the
following criteria (percentage of good "NER":
70%
= a "marginal level of comprehending"
70% - 80% = an "instructional
level"
>85%
= an "independent level"