Conducting needs analyses

wish

Needs analyses depend on our learners already knowing, or our ability to discover, their needs for English language training. There a number of ways of going about conducting a needs analysis but before we look at them, let's get the trees out of the way of the wood. Which of the following do you think are true? Click here when you have an answer.
Needs analysis:

is different from testing
always includes some form of diagnostic testing
should include some form of assessment of learning style
refers to people's need for the language only
can be conducted without consulting the learner(s)
is a straightforward process involving canvassing our students' views
must always be done before a course is planned

Comments:

Yes. Needs analyses may include some form of testing but it is not the same thing as testing.
No. Some analyses don't have to have a test involved in them. For example, if you are constructing a course for very elementary learners, a diagnostic test may not provide any useful data.
Maybe. See the criticisms of learning-style theories for more. Some do, some don't.
No. People's needs for and wishes concerning particular materials, groupings and procedures may also be considered.
Yes, oddly. There are numerous occasions when the learners' employers, schools, sponsors etc. know what the needs are for their clients.
No. This is a complicated area because we are relying quite heavily on introspection.
Should, perhaps, but in the real world it's not always possible. If a course is well described, learners are often able to self-select, depending on their own perceptions of what they need without any needs analysis.

A little psychology

The usual way of conducting a needs analysis is some form of questioning. However, even simple questions such as What area of English is most important to you? are not as simple and may not be as useful as they look.
Think for a moment about why this might be the case and then click here for some commentary.

The question requires a level of introspection which not all learners have.
The respondent needs some meta-language (such as study skills, reading skills, tense structures, functions, notions etc.) to produce a helpful response.
The respondents may actually write down what they think are important skills that they have heard about rather than what is important to them.
Respondents may not know what is important to them.
Respondents may give the answer they think the questioner wants.

The Johari window

The Johari window was invented over 60 years ago by Joseph Luft and Harrington Ingham (hence its name). It was originally designed to help people try to understand themselves and it looks a bit like this:

johari

The four central areas are described by the 'Things ...' sentences surrounding them.
For example, the Facade is the area of things that I know about myself but keep hidden from others. The Blind Spot refers to things that others see in me which I know nothing about. Insight is our ability to draw on our unconscious understanding to realise something about ourselves.
Originally, the technique was used for people to select 6 adjectives to describe themselves and then their peers were asked to select 6 which described the subject. Putting these adjectives into the grid, it is claimed, helps people to see themselves as others see them and be better able to use insight and introspection.
For our purposes here it stands as a reminder that however carefully a questionnaire or interview process is designed, it ultimately depends on people knowing about themselves and, as the window demonstrates, few of us really do.

Conducting a needs analysis

Before we start a needs analysis we need to do one on ourselves:

What do we want to know?
How are we going to discover it?

What

There is a wide range of things that we might want to know, of course. Among them are:

setting: in what settings (work, school, university, social encounters, with native speakers, dealing with officialdom etc.) does the subject need to use English?
skills: do the learners need to deploy all four skills or are some of them more important than others?
sub-skills: of the important ones, are there particular subskills (such as writing e-mails, giving oral presentations etc.) that the learners need?
accuracy levels: is it important that the learners focus on producing accurate language or is basic communicative competence the aim?
functions: are there specific functions (such as asking for permission, inviting, offering, declining etc.) which are particularly important in the setting described?
notions: are there particular notions (such as arrangement of objects, degree, motion etc.) which the learners need to be able to handle successfully?
registers: are there particular registers (academia, engineering, the military, air transport, tourism etc.) in which the learners will have to use English and will need the lexical and structural means to do so?
learning styles: are there particularly common learning styles in the group? See the guide to learning styles for ways of gathering these data. If you do gather this information, what are you going to do with it and do you trust it?

You may have thought of others. Too often, as soon as we start to construct a needs analysis, what we need to know becomes clear. It should, of course, be clear before we start.

Clearly, the nature of the course will determine what data we want to gather. It may be quite narrowly focused if, e.g. we are dealing with an ESP (English for Special Purposes) course such as one targeted at a particular register and set of skills, or an EAP (English for Academic Purposes) in which we also know the register and the skills people need (essay writing, seminar presentations etc.).
If, on the other hand, we are planning a General English course (sometimes called ENAP, English for No Apparent Purpose), we may need to gather much wider data concerning the learners' views of their own strengths, weaknesses and needs.

How

There are really only two main ways:

On paper: questionnaires, written prose responses from learners and so on.
Face-to-face: interviews, focus groups, discussions etc.

Both of these have pros and cons:

In writing		Face-to-face
For	Against	For	Against
can be cheaply administered at a distance	is impersonal and often not monitored	is personal and can be carefully monitored	may be expensive in terms of time and travel
is fixed and reliable: everyone answers the same questions	is inflexible: later questions can't be premised on earlier responses	is flexible and can be altered to suit responses received	becomes unreliable with too much alteration
can be carefully constructed and designed	errors in the questions cannot be eradicated	can be altered easily if a question is flawed	relies on a subjective judgement of what is said
results can be carefully, statistically analysed	little data concerning strength of respondents' feelings	judgement about strength of feelings can be made	it is difficult reliably to collate and analyse results

It is possible in some circumstances to use both approaches and that can avoid the disadvantages of either.

Constructing questionnaires

Whichever approach is used (and the most common is a written questionnaire) the same considerations apply to how questions are constructed. On this depends absolutely the quality of the data we can gather.

There are a number of key considerations.

Can you involve the whole group or do you need to select a sample to represent it? If so, how will you ensure they really are representative?
How are you going to keep records? (This is especially important for face-to-face encounters.)
Are you asking only the questions you need to ask? Don't muddy the data and irritate people by asking questions which have no bearing on the planning. If you can't or don't want to do anything with the answers, don't ask the questions.
Are respondents likely to interpret your questions unpredictably? Are they unambiguous?
Are your questions comprehensible, given the levels of the learners? If necessary, can you ask them in the learners' first language(s)?
Have you thought about how you will analyse the data? If, e.g., the response option is yes/no then you will only know how many or what percent of your sample answered yes/no, not how strongly they felt.
Are your questions and response choices neutral?
Have you considered the order and grouping of questions? Previous questions may bias later answers.
Does each question relate to only one topic? Obviously, if you ask Do you enjoy reading and listening lessons? Yes / No then the results will be useless.
If you have a multiple response item such as At work / At school / In social settings / With officials etc. (Tick all that apply) have you covered all the possibilities or do you need a hard-to-analyse Other ________ category?
If the questionnaire is not being administered by you, have you written clear instructions about how it should be administered?

Design

Questionnaires are an indirect method of collecting data; they are substitutes for face-to-face interaction with respondents.
(Lee, 2005:760)

A matrix

This is a popular and efficient way of designing a questionnaire. For example

Tick one box for each question only.

	Not important	Fairly important	Important	Very important
Speaking
Listening
Reading
Writing
Learning new words
Grammar
Working in pairs

Note that this matrix has an even number of response possibilities. That stops people always picking the middle one.
The disadvantage is that all questions have to be phrased in a way that makes the responses appropriate. You can't ask Do you enjoy ...?

Closed questions

There are three sorts:

Alternate response questions such as:
Put a tick by your choice:
I enjoy working with a partner: Yes ___ No ___
Homework is helpful: True ___ False ___
Multiple-choice items. The matrix above is an example but all the choices are the same. If you want different possibilities for the responses, you need to re-format the questionnaire to allow this. That means having a series of questions with different possible responses for each. It's flexible and precise but imposes more of a strain on the respondents.
If you want to make sure the response possibilities are truly random, put them in alphabetical order.

Scales
In these questions, respondents must order a number of items in terms of, say, importance or enjoyability. For example
Number the following. 1 = the least important. 10 = the most important

No	Activity or skill
	Speaking with a partner
	Pronunciation practice
	Having lots of homework
	Studying grammar
	Learning the words I need for work
	Improving my writing
etc.

Open questions

With smaller groups it is practical to use open-ended questions which allow for unconstrained responses. They have the advantage that the respondent can be much more precise about the answer. The disadvantage is that the results are much harder to analyse statistically and may require reliability-reducing interpretation.
For example
Please write your answers or finish the sentences in the left-hand column.

No	Questions or sentence to finish	Answer or sentence completion
1	Where do you use your English mostly?
2	Which is your strongest skill in English?
3	In ten years' time I want to be using my English for ...
4	The thing I most like about learning English is ...
etc.

Open- vs. closed-questions

Here's a summary of the advantages and disadvantages of the two types of question (from Lee, 2005:769).

Question Type	Advantages	Limitations
Closed-ended	Easier and quicker to answer More likely to get answers about sensitive topics Easier to code and analyse statistically Easier to compare different respondents’ answers Easier to replicate	Frustration without desired answer Confusing if many response choices are offered Misinterpretation of a question without notice Simplistic responses to complex issues Blurred distinctions between respondents’ answers
Open-ended	Opportunity for respondents to give their opinion Unanticipated findings to be discovered Adequate for complex issues Creativity, self-expression, and richness of detail are permitted Respondents’ logic, thinking processes, and frames of reference are revealed	Different degrees of detail and irrelevance in answers Difficulty with response coding Difficulty with comparison and statistical analysis A greater amount of respondent time, thought, and effort is necessary Requires space for answers

Analysis

However you construct the test, you will need to analyse the results somehow. There are two things to remember even if your maths isn't too good.

Convert language to numbers. For example, if you have a Not important / Fairly important / Important / Very important scale, assign 0 to Not important up to 4 for Important. That way, it's easy to construct a graph or other kind of visual representation like a pie chart.
Understand the difference between Mean and Mode. For example
The numbers are 2, 4, 3, 4, 2, 4, 4, 3, 4, 3
The mean is the average: 3.3 (i.e., the total of all the numbers divided by 10 (the number of responses))
The mode is 4 (i.e., the most common response)
These tell you different things. The mean gives you a way to see how the hypothetically typical learner responded (although, in fact, nobody scored anything as 3.3). The mode tells you how popular a response was to each item (how people really voted).

Testing

Now that you have carried out a needs analysis, it is logical to go to the next step and find out how good the learners already are at the target language systems and skills which have emerged. To do that, you'll need a valid diagnostic test. Go to the guide to testing and evaluation to find out how to make one.

More help

There is a guide on this site, more or less a mini-course, in how to do classroom research (linked below) in a way that provides objective, usable data. You are advised to look there before you construct a needs analysis for the first time.

Related guides
testing and evaluation	the obvious place to go
doing classroom research	this is a long guide, almost a mini-course, in how to elicit usable data
example	for an example of a basic needs analysis form in PDF format
learning-styles	for more on what they are and the problems with the theories

Reference:
Lee, SH, 2005, Constructing Effective Questionnaires. Available from http://www.davidlewisphd.com/courses/EDD8006/fall11/2006-Lee.pdf