Understanding Computer User Frustration: 
Measuring and Modeling the Disruption from Poor Designs 
 
Katie Bessiere
1
, Irina Ceaparu
2
, Jonathan Lazar
3
, John Robinson
1
 and Ben Shneiderman
2
 
 
1
Department of Sociology 
University of Maryland, College Park, Maryland 20742 
 
2
Department of Computer Science, Human-Computer Interaction Laboratory,  
Institute for Advanced Computer Studies & Institute for Systems Research 
University of Maryland, College Park, Maryland 20742 
 
3
Department of Computer and Information Sciences & 
Center for Applied Information Technology 
Towson University, Towson, Maryland, 21252  
 
 (kbessiere@socy.umd.edu, irina@cs.umd.edu, jlazar@towson.edu, robinson@socy.umd.edu, 
ben@cs.umd.edu) 
 
 
Draft: September 21, 2002 
 
 
 
Abstract 
  
User frustration in the use of information and computing technology is a pervasive and persistent 
problem.  When computers crash, network congestion causes delays, and poor user interfaces 
trigger confusion there are dramatic consequences for individuals, organizations, and society. 
These frustrations not only cause personal dissatisfaction and loss of self-efficacy, but may 
disrupt workplaces, slow learning, and reduce participation in local and national communities. 
We propose a Computing Frustration Model with incident specific and individual variables to 
guide research. Our study of 108 computer users shows high levels of frustration and loss of 1/3 
to 1/2 of time spent.   The importance of the users' goals and the severity of the disruption were 
correlated with frustration. Those who had strong self-efficacy, indicated by a willingness to 
stick to solving the problem, reported lower levels of frustration.  Implications for users, 
software developers, managers, and policymakers are discussed. 
 
Keywords: user frustration, user interface design, training, helpdesk, computer experience, 
computer anxiety
 2
Introduction 
 
Everyone is familiar with computer problems and the ensuing frustration that results when, yet again, 
your program crashes with no warning, taking the last thirty minutes of work with it.  Frustration is a 
common theme among computer users.  As technology rapidly advances, we the users must deal with the 
ensuing error messages that invariably result, as well as annoying delays, incompatible files, and 
indecipherable menus.  
These challenges are well-known by individual users, but less is known about the workplace 
consequences of these frustrating experiences. How much time is lost on a daily basis as we struggle with 
our machines?  How do these experiences affect our mood, our days, our being?  What role does our prior 
experience with technology play?  
 The challenge of the ?digital divide,? a gap between those who have access to and can effectively 
use information technology and those who do not, was originally identified by the U.S. Department of 
Commerce (U.S. Department of Commerce, 1999) A recent report released by the U.S. Department of 
Commerce, A Nation Online: How Americans Are Expanding Their Use of the Internet, claiming that the 
digital divide gap is disappearing, has resulted in funding being cut for two digital opportunity programs: 
The U.S. Department of Education?s Community Technology Centers Program (CTC) and the U.S. 
Department of Commerce?s Technology Opportunities Program (TOP).  Both of these programs were 
designed to create and improve technology access and use in under-served and low-income communities.  
A policy brief released by the Benton Foundation (Dickard 2002) demonstrates that these programs were 
vital to the reduction in the technology gap existent in America today.  In addition, a reexamination of the 
same data by the Consumer Federation of America (Cooper 2002) claims that the perception that the 
?digital divide is disappearing? is wrong, and that the data actually show the opposite.  
We believe that user frustration is a significant issue that is closely tied to the digital divide.  
Even if universal access to technology is attained, users will still struggle with using the technology 
(Kling, 2000).  For the effective use of technology, careful attention must be given to documentation, 
tutorials, training, online user assistance, and helpdesk support (Lazar and Norcio, 2001).  Kling (2000) 
recognizes that easy-to-use interfaces, user support, technical skills, and a network of people who can 
help, are part of the social access to technology, as opposed to technological access. Even with the most 
up-to-date hardware, software and network connections, users may still find poorly-designed technology 
hard to use (Kraut, Scherlis, Mukhopadhyay, Manning, & Kiesler, 1996). Simply providing the 
technology to those who cannot afford it, will not ensure that people can actually use it, and will therefore 
not solve the digital divide (Kling, 2000). 
For the effective, on-going use of information technology, free hardware and software alone are 
not sufficient to allow users to become productive and satisfied. Systems must be designed for ease of 
use. For all users to be effective in their tasks, there must be support for users in many forms, including 
documentation, tutorials, training, online user assistance, and helpdesk support . The use of well-
designed, easy to use software, along with sufficient support and training, can make a measurable impact 
on the lives of users. The community networking and software project developed at MIT for the residents 
of Camfield Estates, a low-income housing community in Roxbury, MA, is a good example of using 
technological resources to improve the economic situations and overall lives of people (Pinkett, 2002). 
Successes can only occur when users have interfaces that are not frustrating, and support to help them 
utilize the technology effectively. 
 This study examines the factors that moderate the experience of frustration in computer usage. In 
this study, the computer frustrations of 108 users are examined through the use of modified time diaries. 
Individuals? prior experiences, psychological characteristics, level of computer experience, and social 
system are all examined to determine how they influence the frustrations that users face with their 
computers. In addition, factors such as the importance of the task that was interrupted, the frequency of 
occurrence (both of same and different frustrations), and the amount of time or work lost as a result of the 
problem are also examined to determine how they affect the experience of frustration as well. The 
existing psychological literature on frustration provides a foundation for the examination of the frustration 
 3
process in computer use. Through the Computing Frustration Model presented in this paper, the factors 
correlated with frustration are examined. The goal is to help elucidate the nature of the frustration 
experience with computer use. Implications for numerous stakeholders, including users, managers, 
software designers, and policymakers, are discussed.  
 
 
Frustration 
 
A review of the psychological literature reveals diverse definitions of frustration.  Sigmund Freud 
introduced frustration as a concept with external and internal aspects and related it to goal attainment.  
Frustration occurs when there is an inhibiting condition, which interferes with or stops the realization of a 
goal.  All action has a purpose or goal whether explicit or implicit, and any interruption to the completion 
of an action or task can cause frustration.  For Freud, frustration included both external barriers to goal 
attainment and internal obstacles blocking satisfaction (Freud 1921).  This concept of frustration as a 
duality is also discussed in the analysis of frustration as both cause and effect (Britt and Janus 1940).  As 
a cause, frustration is an external event, acting as a stimulus to an individual and eliciting an emotional 
reaction.  The emotional response, in this case, is the effect. The individual is aroused by this external 
cause, and a response is often directed towards the environment (such as yelling, hitting, or anger). 
The level of frustration experienced by an individual clearly can differ, depending on the 
circumstances surrounding the frustrating experience, and on the individual involved.  One major factor in 
goal formation and achievement is goal commitment, which refers to the determination to try for and 
persist in the achievement of a goal (Campion and Lord 1982). Research on goal theory indicates that 
goal commitment has a strong relationship to performance and is related to two factors:  the importance of 
the task or outcome and the belief that the goal can be accomplished (Locke and Latham 2002). 
Individuals will have a high commitment to a goal when the goal is important to them and they believe 
that the goal can be attained (Locke 1996). The importance of the goal to the individuals, in addition to 
the strength of the desire to obtain the goal (Dollard, Doob et al. 1939), will affect the level of goal-
commitment as well as the strength of the subsequent reaction to the interruption.  Self-efficacy, the belief 
in one?s personal capabilities, can also affect goal commitment (Locke and Latham 1990). The belief 
about how well a task can be performed (self-efficacy) when it involves setbacks, obstacles, or failures 
may also affect how committed individuals are to that goal (Bandura 1986).   
 
 
User Frustration with Computers 
 
There are many possible causes of user frustration with computer technology. For instance, a software 
application may crash, an error message may be unclear, or an interface can be confusing (Preece, Rogers, 
and Sharp, 2002). Users can lose work and waste time. In the context of the social psychological research 
literature, frustration occurs when users cannot attain their task goals. The causes of the problem could 
include poor interface design, computer hardware or software failure, or even the users? lack of 
knowledge regarding the computer, but the result is the same: users cannot complete their tasks, and may 
have an emotional response (frustration!) to the inability to attain their goal. This is especially true as the 
Internet changes the nature of the computer user population, to include many more non-technical people 
and people with little computer experience (Shneiderman, 2000; Cummings and Kraut, 2002). 
  
The amount of frustration that users face can differ, depending on a number of factors, such as the 
time lost, the importance of the goal, the user?s self-efficacy, and the user?s computer experience. The 
Computing Frustration Model examines what specific aspects and qualities lead to user frustration, with 
the end goal being to determine ways of ameliorating the frustration. Based on the frustration literature, 
goal-attainment theory, and the literature on computer attitudes and anxiety (Bessiere, Ceaparu, Lazar, 
Robinson, and Shneiderman, 2003), we propose a Computing Frustration Model (Figure 1). 
 4 F
i
gur
e
 1
:
 C
o
m
put
e
r
 
Frust
r
at
ion M
o
de
l 
                                      
 
Go
al In
ter
r
u
p
t
io
n
 
Pr
oducti
v
e
 Pr
ogre
s
s
 
Frustr
ation
!
! 
St
re
n
g
t
h
 of De
si
re
/
 
A
n
tici
p
a
tio
n Ex
pe
cta
t
io
ns
 
S
e
ve
r
ity
 o
f
 
I
n
te
r
r
uptio
n
/
 
T
i
me
 L
o
ss 
L
e
ve
l of 
G
o
a
l
  
C
o
m
m
i
tm
en
t/
 S
e
lf 
E
ffi
c
a
c
y
/ I
m
p
ort
a
n
c
e 
Co
m
pute
r
 
A
n
x
i
e
t
y
/
A
ttitu
de
s
/
 
Ex
p
er
i
e
n
c
e/
 S
e
lf
 E
f
f
i
c
a
c
y
M
ood
/ Ps
y
c
h
o
logi
c
a
l
 
F
acto
r
s 
C
u
ltu
ra
l/
S
o
c
i
eta
l
 I
n
flu
e
n
c
e 
/ L
e
ar
ni
ng
 
In ci d e n t   Factors 
Individual Factors
 5
Frustration theory indicates that it is the interruption of a goal or task that causes individuals to become 
frustrated.  There are various factors that can then subsequently affect the level of frustration experienced.  
These can fall into two categories:  the incident and individual factors. 
 
Incident Factors 
The incident factors that affect the level of frustration experienced by end users include the level of goal 
commitment, the severity of the interruption, and the strength of the desire to obtain the goal.  Goal theory 
tells us that experience, self-efficacy, and the importance of the goal all affect the commitment to the goal 
or task.  When the goal interruption occurs, the level of goal commitment, measured in terms of the 
importance of the task to the user, will affect the amount of frustration experienced by individuals 
directly.  Severity of interruption is measured as the amount of time it took to fix the problem combined 
with the amount of time lost due to the problem.  The strength of desire for the goal is also affiliated with 
how important the goal was, so importance is also used here as a proxy for strength of desire.   
 
Individual Factors 
Individual factors affecting the strength of the frustration include computer experience variables, mood 
and other psychological factors, and the cultural and societal influences upon the individual.  Measures of 
mood include satisfaction with life, how often subjects get upset over things, and general mood.  Also of 
interest are how the frustrating incident affected the users? day, and how frustrated users were, overall, 
after the session.  Computer variables are separated into computer experience/self-efficacy and computer 
anxiety/attitudes.  Computer experience can be measured as years of computer use, hours of computer use 
per week, and a subjective measure of experience ? also effective as a measure of computer self-efficacy.  
Additional measures of computer self-efficacy include confidence about their ability to fix problems, how 
much users persevere when encountering a problem on the computer, and how much users thought about 
unresolved computer problems after being unable to fix them.  Computer anxiety is measured with two 
questions, one on level of comfort with the computer and one on how subjects react to a problem with a 
computer.  Cultural and societal influences are not measured, as they are expected to be a constant.  Our 
sample is taken from an American university setting in which it is expected that most respondents will be 
American.  While user frustration as occurs in different cultures is certainly an interesting question to 
pursue, it lies outside the scope of the current study.  
 
 
Research Methodology 
 
An attractive way to study the incidence of frustration is a modified time diary, rather than a survey.  
Survey questions often ask respondents to estimate the answer to the question from memory, which can 
often lead to inflated or incorrect answers.  Because it is important to ascertain exactly what it is that users 
encounter as they work on their everyday tasks, in addition to the time lost due to these experiences, a 
modified time diary was chosen as the best way to obtain the data.  Time diaries minimize the reporting 
burden on the respondents by allowing them to record their time use immediately after it occurs, instead 
of attempting to remember an aggregate amount of information at a later date.  In addition, it is possible 
to capture the session length and the exact amount of time lost due to frustrating experiences by using this 
modified version of the time diaries, information that may be lost or incorrect if it were asked in a survey 
format.   
In order to obtain data reflecting typical computer usage, we asked subjects to work on the 
computer for a minimum of one hour, on tasks of their own choosing.  Because self-set goals are more 
meaningful to individuals, they may be better than assigned goals that may be unclear or be rejected 
(Locke, Shaw, Saari, & Latham, 1981). Subjects worked on tasks of their own choosing as opposed to 
assigned tasks. In order to be able to collect data on tasks that are important to the individuals, it was 
important that the tasks were not pre-assigned.  Prior to the session, subjects filled out a short online pre-
session questionnaire assessing demographic information, computer experience and attitudes, and mood.  
 6
After completing the one-hour long data collection period, subjects then returned to the website and filled 
out a 5 question post-session survey.  A typical user in this study would: 
 
1) Go to the website and fill out the pre-session questionnaire 
2) Begin the work session and work for a minimum of an hour, filling out paper frustration 
reports when encountering anything frustrating (the modified time diary)  
3) Immediately following the one-hour session, the user would return to the website to fill out 
the post-session survey 
4) After filling out the post-session survey, the user would enter the paper frustration reports 
into an online database. 
 
The pre-session survey [Appendix A] asked respondents about demographic information, 
computer experience and attitudes, level of computer anxiety, and mood.  Previous research indicates that 
level of computer experience or perception of computer self-efficacy can affect subsequent user behavior 
(Murphy, Coover et al. 1989; Brosnan 1998). Questions were chosen after a careful review of previous 
research on the Computer Aptitude Scale, assessing computer attitudes, computer anxiety/confidence, and 
computer liking (Loyd & Gressard 1984; Nash & Moroz 1997).  Stemming from this, it was hypothesized 
that prior experience and level of perceived knowledge will affect individuals? level of frustration as well.  
To assess the overall state of the individuals, we included three questions dealing with overall life 
satisfaction, general mood, and how often they get upset over things.  The post session survey [Appendix 
B] consisted of five questions to assess mood after the session, how frustrated overall the individuals were 
after the session, how these frustrations affect the rest of the day, and the frequency and typical nature of 
the frustrating experiences during the session. 
To minimize the amount of additional time to fill out the forms, subjects filled out a one-page 
time diary for each incident that occurred in the one-hour long data collection period. By filling out a 
paper time diary form, rather than an online version, the amount of added frustration with the computer 
was minimized, and was therefore only based on the user?s work on the computer. When the hour was 
completed, subjects returned to the website, filled out the post session form, and then entered their 
frustration reports into the database via a form on the website. There were two stages of data collection: 
subjects performing self-reported diaries, and subjects observing other users. Subjects were enlisted from 
students at both the University of Maryland and Towson University.  For the self-reported diaries, 
students completed a minimum of one hour of data collection and filled out incident reports for their daily 
computer use. For the observations, students were also asked to observe another person not in the class 
and gather data on this individual. Subjects for the observations were enlisted by the students and 
followed the same methodology as the self reports.  Instead of filling out the frustration experience report 
forms themselves, however, the observers filled them out, asking the subjects to talk out loud and share 
their experience with the observer.   
Our first analysis of the data (Ceaparu, Lazar, Bessiere, Robinson, and Shneiderman, 2002) 
examined the frequency, cause and the level of severity of frustrating experiences.  The three task 
applications that were the cause of the most frustrating experiences (N=373) were web browsing (122 
frustrating experiences), email (49 frustrating experiences), and word processing (44 frustrating 
experiences).  The specific causes of frustration most often cited were error messages (35), timed 
out/dropped/refused connections (32), freezes (24), long download time (23), and missing/hard-to-find 
features (23).   Most subjects indicated that the frustration experience had happened before (277), as 
frequently as several times a month (40), week (54), or even several times a day (60). 
The most disturbing result in our previous analysis was with the amount of time lost due to 
frustrating experiences.  One third to one half of the time spent in front of the computer was lost due to 
frustrating experiences when looking at both the time it took to fix the problem and any additional time 
that was lost due to the problem.  When only looking at the time actually lost spent fixing the problem, 
we found that between one quarter and one third of the time in front of the computer was spent fixing the 
 7
problem.  This is clearly a large amount of time lost due to problems on the computer, and this time lost 
has a value both in productivity and in monetary terms.   
 
 
Results 
  
We analyzed both incident level frustration, measured as level of frustration per incident, and session 
frustration, measured as overall frustration after the session, in an effort to determine the factors that are 
indeed correlated with level of frustration.  Individual level frustration factors examined are demographic 
factors, computer experience, computer anxiety, computer self-efficacy, and mood.   In addition, 
measures of overall frustration after the session, mood after the session, and the effect of the session on 
the individuals? day were analyzed. 
 
Demographic Information 
Out of 112 total subjects, 4 were discarded due to technical problems with the database.  There were a 
total of 372 frustrating experiences reported.  The remaining 108 subjects in the study were approximately 
equal male/female, and were composed mainly of college undergraduates (75.9%).  As a result, exactly 
half of the subjects were under the age of 22.  The remaining half ranged from age 22 to 80.  Respondents 
also reported a high level of self-reported perceived computer experience, and 40% of the respondents 
reported either being a computer professional or student (see Table 1.1). 
 
Table 1.1  Perceived Computer Experience 
 
Experience Level:   N=108, mean = 6.88, sd = 1.88 
 Frequency Percentage 
1 1 0.9 
2 2 1.9 
3 4 3.7 
4 8 7.4 
5 6 5.6 
6 15 13.9 
7 26 24.1 
8 22 20.4 
9 24 22.2 
Total: 108 100 
 
Age is negatively correlated with experience in our study (r = -.278, p<.001) indicating that younger users 
seem to have more experience with computers, an expected result. 
 
Incident Frustration Level 
Incident frustration level is measured on a scale of 1 (not very frustrating) to 9 (very frustrating) for each 
incident occurring in the study.   The frequency table shows that two-thirds of the incidents resulted in 
high levels of frustration, 7-9 (see Table 1.2). 
 
 
 
 
 
 
 
 8
Table 1.2  Incident Frustration Level 
 
Frustration Level:   N=372, mean = 6.74, sd = 2.13 
 Frequency Percentage 
1 6 1.6 
2 16 4.3 
3 19 5.1 
4 22 5.9 
5 27 7.3 
6 45 12.1 
7 67 18 
8 81 21.8 
9 89 23.9 
Total: 372 100 
 
 
Incident frustration level is quite significantly skewed towards the high end, indicating that users are often 
very frustrated by problems that they encounter with their computer. 75.8% of the incidents reported in 
the sessions resulted in higher than neutral frustration.  In addition, almost 50% of the incidents (45.7%) 
resulted in a frustration score of 8 or 9, the highest levels of frustration possible. 
 
In addition to frustration level, we asked subjects to record their feelings after the incident, in an effort to 
determine what kind of reaction was elicited from them after a problem with the computer (see Table 
1.3). 
 
Table 1.3  Post-Experience Feeling 
 
Feeling:   N=370 
 Frequency Percentage 
Angry at the Computer 155 41.9 
Angry at Yourself 15 4.0 
Determined to Fix it 84 22.7 
Helpless/Resigned 45 12.2 
Other 71 19.2 
Total: 370 100 
 
Here we see that 42% of the users in the study have a resultant feeling of being mad at the computer after 
a frustrating experience.  Only 4% of the users reported being angry at themselves, but 12% of them 
reported feeling helpless or resigned, indicating that some users do in fact experience a sense of loss of 
control when faced with computer problems.  On the other hand, 23% of the users were determined to fix 
the problem, which may be mediated by the high level of experience reported by the subjects. 
 
Session Frustration Level 
After the session was completed, subjects answered a few questions designed to measure how the 
frustrations that they experienced affected them overall.  Overall frustration about the session, how it 
affected their day, their mood after the session were all measured, as well as a question on whether they 
experienced more or less frustrating incidents in the study, as compared to a typical day (see Table 1.4).   
 
 9
 
Table 1.4 Post-Session Variables 
 
 
Overall 
Frustration 
(N=108) 
Affected Day 
(N=108) 
Pre Mood 
(N=108) 
Post Mood 
(N=108) 
More or Less 
Frustrations 
(N=108) 
Mean 5.87 3.87 6.1 5.66 4.54 
Standard 
Deviation 
1.89 1.95 1.45 1.59 1.84 
 
Overall, it does not appear that the session as a whole produced much of an overall affect on the 
individuals.  It would appear that the frustrating incidents encountered were about average for these 
subjects.  The score for whether they encountered more or less frustrating experiences in an average day 
was just about 5.  The mean for overall frustration was much lower than the mean for incident frustration, 
indicating that the individual incidents as a whole did not produce a feeling of high overall frustration.  
Subjects reported a low mean score for whether or not the incidents taken together affected the rest of 
their day, as well.  Mood scores went down slightly on average after the session, from 6.1 to 5.7.   
 
Correlations 
To determine the factors that influence both incident specific frustration and the overall effects of 
frustration in computer use, we ran correlations with time, computer anxiety, computer self-efficacy, 
mood, and importance variables (See Table 1.5). 
 
 
Table 1.5  Frustration Scores and Correlates 
 
 Incident Variable Session Variables 
 
Frustration 
(N=372) 
Overall 
Frustration 
(N=108) 
Post-Mood 
(N=108) 
Affect Day 
(N=108) 
 r r r r 
Time Variables 
Time Lost  
(Incident) 
.316***    
Time Fix 
(Incident) 
.231***    
Computer Years -.059 -2.00*  -.261** 
Hours per Week -.122* -1.75  -.091 
Time to Fix  
(Total) 
 .124 .007 .250** 
Time Lost 
(Total) 
 .062 -.014 .146 
 10
Computer Anxiety Variables 
Anxiety .029 -.106 .344*** -.171 
Comfort .100
?
 .012 .274** -.164 
Computer Self-Efficacy Variables 
Experience -.020 -.151 .315*** -.239* 
Ability to Fix -.019 -.157 .286** -.282** 
Unresolved -.058 -.008 .122 .047 
Stick with it -.139** -.327*** .437*** -.245** 
Mood Variables 
Life Satisfaction -.095 -.178 .185 -.166 
PreMood .116* .005 .336*** -.009 
Upset Often -.027 .150 .362*** .155 
Mood Swing -.085 -.321***  -.241* 
Importance 
Importance .237***    
Avg. Importance  .085 .110 .136 
* = p<.05  ** = p<.01  *** = p<.001   ?
 
= p=.055 
 
 
Incident Frustration Factors 
Incident frustration in our subjects has a high positive correlation with the amount of time it took to fix 
the problem, the amount of time or work lost due to the problem, and the importance of the task.  
Computer experience does not seem to be a factor in the amount of frustration experienced, although 
there is a slight significant negative correlation between the number of hours worked per week and the 
frustration level, indicating that the number of hours worked per week could lessen the amount of 
frustration experienced.  Level of comfort with the computer, while not significant, has a p value of .055.  
In addition, whether or not the subjects would stick with a problem encountered on the computer is 
negatively correlated with the level of frustration experienced, with a p value of <.01.  Mood before the 
session, on the other hand, is significant at p=.05, and has a positive relationship with frustration level. 
 
Session Frustration Factors 
Overall frustration level after the session on the computer is highly correlated (p<.001) with whether 
users experienced more or less frustrations on an average day, whether they would stick with a problem 
until it was fixed, and also with the difference in pre and post session mood.  A strong positive correlation 
exists between the average amounts of frustrations as compared to the current session.  Negative 
relationships are seen between overall frustration and the stick with it and mood swing variables.  A small 
but significant negative relationship is also found between the number of years the individual has been 
using a computer and level of overall frustration. 
 How the frustrations experienced were expected to affect the day of the individual is positively 
correlated with the total amount of time it took to fix all the problems experienced, and whether they 
experienced more or less frustrations on average.  There is a negative correlation between the effect on 
the day and number of years of computer use, level of perceived experience, perceived ability to fix 
problems on a computer, willingness to stick with the problem until it is solved, and the mood swing 
between the start and end of the session. 
 The individuals mood after the session has a high positive correlation (p<.001) with level of 
computer anxiety, perceived level of experience, mood before the session, and whether the individual gets 
upset often.  Smaller positive relationships (p<.01) exist with level of comfort with the computer and 
perceived ability to fix problems on the computer. 
 11
 
 
Discussion 
 
The Computing Frustration Model hypothesizes that frustration is affected by importance of the task, 
previous experience, the severity of the interruption, level of computer anxiety, computer self-efficacy, 
and psychological or mood factors. 
 
Importance of Goal (I1) 
The importance of the task has a high positive correlation with incident specific frustration levels, 
indicating that the importance of each individual task has a strong effect on the level of frustration 
experienced because of that incident.  However, the average importance level of all the tasks combined 
was not correlated with any of the post-session outcome variables.  Overall frustration, post-session 
mood, and the affect of the day do not appear to be correlated with the average importance of the tasks 
performed.  This indicates that task importance affects the immediate reaction to the interruption but does 
not linger on as time goes on. 
 
Severity of Interruption (I2) 
The severity of the interruption, measured as the amount of time it took to fix the problem and the amount 
of time or work lost as a result of the problem, is highly correlated with the incident frustration level, but 
not with the post-session outcome measures.  Both of these measures are positively correlated with 
frustration level, indicating that as the amount of time lost increases, so does the level of frustration.  It is 
also possible that as frustration increases due to time loss, the increase in frustration causes the individual 
to become less able to fix the problem, which would then lead to an increase in time lost.  This cascade 
effect could obviously become quite problematic as the individual encounters more frustrating 
experiences and loses more and more time. 
Total time lost only appears to be significantly related to whether or not the frustrations affect the 
subject?s day.  Here as well there is a positive association, indicating that as the amount of time to fix the 
problem goes up, the person?s day is affected more and more.  However, the amount of time lost as a 
result of the problem is not significant here for any of the post-session variables.   
 
Strength of Desire/Importance(I3) 
As mentioned above, the importance of the task has a high positive correlation with incident level 
frustration level.  This is what we expected to find, given that goal theory indicates that the importance of 
the task is vital to goal commitment, and that frustration theory indicates that it is an interruption on the 
path to a goal that causes frustration.  It makes sense, then, that the more important the task or goal, the 
more frustrated a person would get on an individual task basis when they could not complete the task.   
 
Experience(I4) 
We measured level of computer experience in two different ways, both an objective measure of 
experience and a subjective measure of experience.  In terms of incident level frustrations, it does not 
appear that either measure of experience has a significant effect on the level of frustration experienced.  
There is a slight negative relationship observed due to the number of hours worked per week, indicating 
that possibly the more a person works on a computer during the week the less frustrated they are when 
they encounter problems.  It is possible that this is due to a habit effect, where frustrations are so common 
that the subjects fail to register them as very frustrating when they occur.  It is also possible that the 
skewed distribution of experience in our subjects could be affecting the relationship here.   
 The number of years of computer use also has an expected significant negative relationship with 
the session variables of overall frustration and the effect on the day.  Apparently here as well, years of 
experience is related to a decrease in the effects of frustrating experiences on the individual.  For the 
session level variables, level of perceived experience is also correlated with mood after the session and 
 12
the effect on the day.  As expected, there is a negative relationship with the effect on the day and a 
positive correlation between perceived level of experience and mood.  As the perceived level of 
experience goes up, so does the mood after the session. This would make sense given the previous 
findings that the more experience you have the less it might affect your day and the less overall frustration 
you experience.  Our subjects were for the most part highly experienced, and we might anticipate more 
severe negative effects on users with less experience. 
 
Computer Anxiety/Attitudes (I4) 
Surprisingly, only mood after the session was correlated with either anxiety or level of comfort with the 
computer.  However, we suspect that this is due to the relationship between experience and computer self-
efficacy with these variables.  Both of these variables do correlate with the mood after the session, 
however.  The more comfortable people are with the computer and the less anxious they are, the better 
their mood after the session.   
 
Computer Self-Efficacy (I4) 
As expected, computer self-efficacy does correlate with frustration levels.  The most interesting of these 
variables is the ?stick with it? variable, which asked respondents how likely they would be to stick with a 
problem on the computer until it was solved.  As the likelihood increases that they would stick with the 
problem until a solution is found, level of frustration (both incident and session based) decreases, there is 
less of an effect on the day, and the mood after the session increases.  This suggests that people who enjoy 
solving problems or are more stubborn are less likely to become frustrated by problems on the computer.  
Perceived level of experience and perceived ability to fix a problem on the computer are positively related 
to post-session mood, and negatively related to the effect on the day.  In other words, as computer self-
efficacy increases, the frustration levels go down and the mood afterwards is better. 
 
Mood Variables (I5) 
 Mood does not appear to have a strong affect on either the incident or session frustration level.  Mood 
prior to the session appears to affect individual level frustration levels, but not overall frustration.  When 
looking at the mood swing that occurs before and after the session, however, we find that there is a strong 
negative correlation between this difference and overall frustration.  This could indicate that as overall 
frustration level goes up, the difference in mood goes from positive (post mood being higher than pre 
mood) to negative.  This holds true for the effect of the frustrations on the day as well.  This is what we 
would expect to find with regard to frustration and the change in mood. 
  
Cultural/Societal Influence (I6) 
Cultural factors were not measured in this experience, due to the nature of our sample.  While the 
question of whether international and cultural differences affect the experience of frustration is an 
interesting one, our sample was drawn from an American university and was expected to be fairly 
homogenous in terms of cultural differences.  Because the sample was relatively small, we felt that not 
enough cultural diversity would be found and as such did not measure cultural differences in this study.  
However, we do believe that this is an important variable, and when extrapolating beyond Americans as a 
population should be examined in greater detail. 
 
 
 13
 
 
 
 
 
 
Conclusion 
 
From this data, the severity of the interruption and the importance of the task combine to form the greatest 
influence on incident frustration levels.  On the other hand, computer self-efficacy has the greatest effect 
on overall frustration levels.  Experience, computer anxiety, and mood also all combine to influence the 
level of overall frustration, mood, and affect the day of the individual as well.  It appears that incident 
level factors correlate more with incident level frustration, and that individual level factors correlate more 
with the overall outcome variables.  In other words, the circumstances surrounding the actual frustrating 
incident have a greater effect on the resultant frustration due to that one incident, and other factors such as 
mood, experience, computer anxiety, or computer self-efficacy do not have as much of an effect.  
However, when we look at the overall experience of frustration, and how it affects the mood and day of 
the individual, we see that these variables become very significant to the outcome of the frustrating 
experiences.  In addition, incident specific variables lose their importance.  Even a variable such as total 
time lost, which we would expect to be correlated with the post-session outcome variables, is not 
significant.  Perhaps incidents in and of themselves cause a fleeting but very strong annoyance, but 
overall frustration with the events in the session stays more constant.  Because it is more constant, it is 
correlated with other global variables on an individual basis.  Each incident by itself causes a brief flare-
up in frustration, but adds to the users experience and levels of self-efficacy, which in turn affect global 
frustration levels.   
 
  
Implications of this Study 
 
For Users:  While individual frustrating experiences are annoying and cause a brief flare-up in 
frustration, it is rather the individual factors that are of the most concern.   Previous experience, attitudes 
toward the computer, and computer self-efficacy all have an effect on the experience of users with the 
computer.  In order to have the best experience, it appears that a positive attitude towards the computer 
and development of skills is essential.  Users should make an effort to receive training on their computer 
systems and software.  In addition, the type of training received should best match the task needs, e.g., 
training related to web browsing, an exploratory environment, should receive training using the 
exploratory model (Lazar and Norcio, 2003). An increase in knowledge and experience can only serve to 
enhance the computer attitudes of the individuals. Since computers are increasingly being used for 
personal and entertainment-related tasks, rather than workplace-related tasks, the users become more 
responsible for ensuring the success of their tasks (Cummings and Kraut, 2002). While individual 
frustrations may continue to occur, the increase in experience and computer self-efficacy and the positive 
tilt in their attitudes towards the computer will minimize the effect of these frustrating experiences as a 
whole. 
 
For Software Developers:  Software developers should emphasize strategies for reducing the frequency of 
user frustration.  More reliable software, better user interfaces, clearer instructions, and improved tutorials 
could reduce and prevent problems.  For instance, subjects cited error messages as major sources of 
frustration. Although the problems of poorly-worded error messages are well-established, unfortunately, 
error messages are still negative, unclear, and do not actually help users respond to errors (Shneiderman, 
1998; Lazar and Huang, 2003). Some error messages are well-known, such as the ?fatal exception,? also 
known as the ?Blue Screen of Death.?  A recent e-mail sent by one of the authors of this paper was 
 14
returned, with the resulting error message stating that ?[servername] does not like recipient.? Such an 
error message cannot actually assist users in responding to errors. Another issue is that many developers 
design technology for the future, thinking that ?this application will be great when users get faster 
download speeds and larger monitors and storage devices? While the field of information technology is 
inherently ?future-oriented? (Kling, 1999), the effort should be made to develop computer systems that 
users can use effectively now, given the technology that users currently have.  
 
For Managers:  While it is hard to extrapolate from our sample to the business population, there are some 
clear possible effects.  First, each frustrating experience will cost the company in minutes and work lost.  
Second, the aggregate effect on the individual is to lower the mood and increase the likelihood of having 
a bad day, which can adversely affect work performance as well.  Clearly, it is a good idea for businesses 
to invest in both the technology and the training of their staff, in the technology in order to improve 
productivity.  Future study on workers should help to bring forward these issues with a more appropriate 
sample. 
 
For Policy:  If previous experience shapes attitudes towards the computer, then the development of 
technological skills is essential to the elimination of user frustration. Purchasing hardware and software 
simply is not sufficient to enable effective use of computer technology.  Policy makers should not 
pronounce the digital divide ?solved? simply because people have access to computers, but rather, should 
promote programs that provide training to underserved populations. If frustration is such a large part of 
the user experience, and experience and perceived efficacy can help eliminate this frustration, then it is 
vitally important that the low-income and disadvantaged communities receive as much help in this arena 
as possible. Unfortunately, support and training have never been a priority for government funding in the 
United States. The Universal Service Program, sponsored by the U.S. Federal Government (also known 
as the E-Rate), helps to provide funding to connect schools and libraries to the Internet (U.S. Congress, 
1996).  However, the law specifically excludes expenses for training users (Lazar and Norcio, 2001). 
Many government programs also prohibit the use of funds for training or any non-equipment related 
expenses (General Accounting Office, 1998). Training and exposure are clearly vitally important to the 
user experience.  It is quite possible that new users, who lack knowledge, training, and access to 
helpdesks will simply never advance their skills.  
 
 
Future Research 
 
The data tells an intriguing story, indicating that frustrating experiences each have a strong but fleeting 
instantaneous effect.  However, the total effect of frustrating experiences has the strongest implications 
for users.  This theory should be validated with further data collections, which we are pursuing.  In 
addition, different pools of subjects should be examined in order to verify whether this pattern continues 
beyond the student population.  Examining incident and individual factors, to determine its relationship to 
frustration levels will also further the knowledge of what factors can be altered in order to ameliorate the 
user experience.  While it is clear that experience is an extremely important factor, further examination of 
other variables will elucidate how to decrease user frustration without having to wait until the users 
become experts.   
 Cultural and societal variables were not examined here, but an examination of various cultures 
and their responses to and experience of frustration would also be extremely interesting.  Data from this 
line of study could serve to show how different coping strategies are employed by users to deal with 
frustration.   
 Information technology is continuously advancing and that user frustration is unlikely to be 
eliminated completely.  However, examining the factors behind frustration could influence 
documentation, training, design of systems and software, helpdesk services, and personal strategies for 
coping.   
 15
 
 
Acknowledgements 
We appreciate partial support from National Science Foundation grant for Information Technology 
Research (#0086143) Understanding the Social Impact of the Internet: A Multifaceted Multidisciplinary 
Approach.  We appreciate the devoted efforts of Prof. Shirley Anne Becker formerly of the Florida 
Institute of Technology and her students Ali Al-Badi and Madhan Thirukonda in preparing the website 
for data entry, and Kent McKay of Adonix Systems for help in transferring and converting the system for 
use at the University of Maryland. We also thank Cheryl Schroeder-Thomas for comments on an earlier 
draft. 
 16
References 
  
Bandura, A. (1986). Social Foundation of Thought and Action: A Social-Cognitive Theory. Englewood 
Cliffs, NJ, Prentice Hall. 
  
Berkowitz, L. (1978). "Whatever happened to the frustration aggression hypothesis?" American 
Behavioral Scientist 21(5): 691-708. 
  
Bessiere, K., Ceaparu, K., Lazar, J., Robinson, J., Shneiderman, B. (2003, in press). Social and 
Psychological Influences on Computer User Frustration. J. Newhagen (ed.), NAME OF BOOK HERE. 
Mahwah, NJ: Lawrence Erlbaum Associates.  
 
Britt, S. H. and S. Q. Janus (1940). "Criteria of Frustration." The Psychological Review 47(6): 451-469. 
  
Brosnan, M. J. (1998). "The Impact of computer anxiety and self-efficacy upon performance." Journal of 
Computer Assisted Learning 14: 223-234. 
  
Campion, M. and R. Lord (1982). "A Control Systems Conceptualization of the Goal-Setting and 
Changing Process." Organizational Behavior and Human Performance 30: 265-287. 
 
Ceaparu, I., J. Lazar, K. Bessiere, J. Robinson, and B. Shneiderman (2002). ?Determining Causes and 
Severity of End-User Frustration.? Paper under review. 
 
Cooper, M. N. (2002).  ?Does the Digital Divide Still Exist?  Bush Administration Shrugs, But Evidence 
Says ?Yes?.? URL: http://www.consumerfed.org/DigitalDivideReport20020530.pdf 
 
Cummings, J., and Kraut, R. (2002). Domesticating Computers and the Internet. The Information Society, 
18(3), 221-231. 
 
Dickard, N. (2002).  ?Federal Retrenchment on the Digital Divide: Potential National Impact.? URL: 
http://www.benton.org/policybriefs/brief01.html 
 
Dollard, J., Doob, L. W., Miller, N. E., Mowrer, O. H., & Sears, R. R. (1939). Frustration and aggression. 
New Haven, CT: Yale University Press. 
  
Freud, S. (1921). Types of Onset and Neurosis. The Standard Edition of the Complete Psychological 
Works of Sigmund Freud. J. Strachey. London, Hogarth Press. 12: 227-230. 
 
Kling, R. (2000). Learning About Information Technologies and Social Change: The Contribution of 
Social Informatics. The Information Society, 16 (3), 217-232. 
 
Kling, R. (ed.) (1999). Computerization and Controversy: Value Conflicts and Social Choices. 
Brooks/Cole Publishing Company.  
 
Kraut, R., Scherlis, W., Mukhopadhyay, T., Manning, J., Kiesler, S. (1996). The homenet field trial of 
residential internet services. Communications of the ACM, 39(12), 55-63.  
 
Lazar, J. and Huang, Y. (2003, in press). ?Improved Error Messages for Web Browsers: An Exploratory 
Experiment?. In J. Ratner (ed.) Human Factors and Web Development, 2nd Edition, Mahwah, NJ: 
Lawrence Erlbaum Associates, 167-182. 
 
 17
Lazar, J. and Norcio, A. (2001) ?Service-Research: Community Partnerships for Research and Training? 
Journal of Informatics Education and Research, 2(3), 21-25 
 
Lazar, J., and Norcio, A. (2003, in press). ?Training Novice Users in Developing Strategies for 
Responding to Errors When Browsing the Web.? International Journal of Human-Computer Interaction. 
 
Locke, E. A., Shaw, K. N., Saari, L. M., & Latham, G. P. (1981). Goal setting and task performance: 
1969-1980. Psychological Bulletin, 90, 125-152. 
 
Locke, E. A. (1996). "Motivation Through Conscious Goal Setting." Applied Preventative Psychology 5: 
117-124. 
  
Locke, E. A. and G. P. Latham (1990). A Theory of Goal Setting and Task Performance. Englewood 
Cliffs, NJ, Prentice Hall. 
  
Locke, E. A. and G. P. Latham (2002). "Building a Practically Useful Theory of Goal Setting and Task 
Motivation: A 35-Year Odyssey." received from author. 
 
Loyd, B. & Gressard, C. (1984). Reliability and factorial validity of computer attitude scales. Educational 
and Psychological Measurement, 44, 501-505. 
  
Murphy, C., D. Coover, et al. (1989). "Development and Validation of the Computer Self-Efficacy 
Scale." Educational and Psychological Measurement 49: 893-899. 
 
Nash, J.B. and P.A. Moroz (1997).  ?An Examination of the Factor Structures of the Computer Attitude 
Scale.? Journal of Educational Computing Research. 17(4): 341-356. 
 
Pinkett, R. (2002). Camfield Estates-MIT Creating Community Connections Project: High Technology in 
a Low-to-Moderate Income Community. In J. Lazar (ed.). Managing IT/Community Partnerships in the 
21st Century. Hershey, PA: Idea Group Publishing, 221-246. 
 
Preece, J., Rogers, Y., and Sharp, H. (2002). Interaction Design: Beyond Human-Computer Interaction. 
New York: John Wiley & Sons. 
 
Shneiderman, B. (2000). Universal Usability: Pushing Human-Computer Interaction Research to 
Empower Every Citizen. Communications of the ACM, 43(5), 84-91.  
Shneiderman, B. (1998). Designing the User Interface: Strategies for Effective Human/Computer 
Interaction. Reading, MA: Addison-Wesley. 
  
United States Congress. (1996). Telecommunications Act of 1996 (P.L. 104-104) . Washington, D.C.: 
U.S. Government Printing Office. 
 
United States Department of Commerce (1999). Falling Through the Net: Defining the Digital Divide. 
Available at: <http://www.ntia.doc.gov/ntiahome/fttn99/contents.html> 
 
United States General Accounting Office. (1998). School technology: Five school districts' experiences in 
funding technology programs (GAO/HEHS-98-35). Washington, D.C.: United States General Accounting 
Office 
 18Appendix
 A. Pre-
Sessio
n
 S
u
rv
ey
 
 
Nam
e
:
  
_____
_______
___
_________
 
Em
a
i
l:
  ___
___
_________
______
______
_
_____ 
Cla
s
s
:
 _____
______
________ 
 
S
e
c
t
io
n
 
I:
 D
e
mo
g
r
a
p
h
i
c
 In
fo
rma
t
io
n
 
 1.   Age:
   
______
____ 
2.   G
e
nder
:
  F    
M 
3. 
Educati
o
n:   
___ Hig
h
 Schoo
l Grad
uat
e
 
___ Fre
s
h/S
oph 
in Col
l
eg
e
 
___ 
J
r
.
/
Sr
. in Co
ll
eg
e 
___ Col
l
eg
e Gradu
a
t
e
 
___ Adv
a
nced Deg
r
ee 
 
4. 
Employ
men
t: (
P
l
e
ase
 
cho
o
se ONE) 
__ K
-
12 Studen
t
 
__ Pro
f
ess
i
on
al 
(D
octo
r/L
a
w
y
er/e
tc
) 
__ Serv
ic
e/Cus
t
om
er Supp
ort 
__ Hom
e
m
a
k
e
r 
__ Stud
ent -
 Com
puter 
Scien
ce 
__ A
cadem
ic/E
duca
t
or 
__ C
l
e
r
ic
al
/A
dm
inis
tra
tiv
e
 
__ S
e
l
f
-
E
m
p
loy
e
d 
__ Stud
ent 
? Othe
r 
__ Com
puter 
T
e
chni
ca
l/Eng
ine
e
r
i
ng
 
__ Sal
e
s/
Ma
rk
eting
 
__ Unem
ploy
ed, 
look
ing
 for w
o
rk
 
__ Execu
tiv
e/
 
Manag
e
r
i
a
l
 
__ Othe
r 
T
e
c
hni
cal
/Eng
ine
e
ring
 
__ Trad
esm
a
n/Cra
f
t
s
m
a
n 
__ Ret
i
re
d 
__ Othe
r_____
______
____
_________
______
______
_
_________
______
______
_
_________
______
______
_
_________
____ 
 
Sect
ion 
II
: Comput
er Ex
per
i
enc
e
 and A
t
t
i
t
udes
 
 1. 
How m
a
n
y
 
y
ears hav
e
 y
ou 
been u
s
ing
 a de
sk
top or 
lap
t
op com
pute
r
 fo
r hom
e or 
work
 use?  ___
____ 
2. 
How m
a
n
y
 hours pe
r week
 do y
ou use a d
e
sk
top o
r
 l
a
p
t
op com
pute
r
?  _
_____ 
3. 
What
 ty
pe of
 Oper
at
ing
 Sy
stem
 is in
sta
l
l
e
d on 
the
 com
pute
r
 th
at y
ou a
r
e c
u
rr
en
tly
 using
?
 
___ DOS 
___ Ma
cOS 10 
___ Ma
cOS Pr
ior 
___ W
i
ndows 9
5
 
___ W
i
ndows 
X
P
 
___ W
i
ndows N
T
 
___ W
i
ndows 9
8
 
___ W
i
ndows 
ME 
___ W
i
ndows 2
000 
___ Unix
/Lin
ux 
 
4. 
What
 ty
pe of
 app
l
i
cat
ion
s
 a
nd prog
ram
s
 do y
ou ty
pica
l
l
y
 use? (
c
he
ck
 all 
tha
t
 app
l
y
)
 
___ Em
ail 
___ Othe
r I
n
terne
t
 Use 
___ Graph
i
c D
e
sig
n
 Prog
ra
m
s
 
___ Chat
/I
nstant
 Me
ssag
i
n
g
 
___ Wo
rd Pr
oce
ssi
ng
 
___ Prog
ram
m
i
n
g
 T
ool
s 
___ We
b Brows
i
ng
 
___ Spr
eadsh
ee
t Prog
ram
 (
E
xcel
) 
___ Dat
a
bas
e
 
T
o
o
l
s 
___ Pre
s
en
ta
tion 
Tool
s 
(power
poin
t
) 
___ Othe
r (
p
le
ase 
exp
l
ai
n) 
_________
______
______
_
_________
______
______
_
 
 
5. 
How m
a
n
y
 
y
ears hav
e
 y
ou 
been u
s
ing
 the
 wor
l
d wid
e
 
web? _____
______ 
6. 
How m
a
n
y
 hours pe
r week
 do y
ou spend o
n
l
i
ne? P
l
e
a
s
e
 ind
i
ca
te 
the 
am
ount of 
ti
m
e
 that y
ou are 
ac
tua
lly
 usi
n
g
 the com
puter wh
i
l
e
 on
li
ne, 
not s
i
m
p
ly
 the am
ount of 
ti
m
e
 
y
ou are con
n
ec
ted 
to 
th
e in
ter
n
et.
   ____
_____ 
7. 
Do y
ou curre
nt
ly
 ___ Hav
e
 a perm
anen
t conn
ec
tion
 to 
the 
int
e
rn
et 
OR
 ___ d
i
a
l
 i
n
 
throug
h a m
odem
 
 198. 
Wher
e i
s
 th
e com
pute
r
 y
ou are u
s
ing
 now?  _
_
 Hom
e
   __Work
  __L
ibr
a
ry
   __Co
m
puter Lab   __O
the
r
  
9. 
Whic
h of 
the
 fo
llow
i
ng
 do 
y
ou do when enco
unt
er
ing
 a prob
lem
 on the 
com
puter 
or app
l
i
c
a
tion
 th
at y
ou ar
e 
using
?
 
___ t
r
y
 to f
i
x i
t
 on m
y
 own
 
___ con
s
ul
t a m
a
nual
/
he
lp 
tuto
ri
al 
___ Ask
 help d
e
sk
/con
sul
t
a
n
t 
for h
e
lp  
___ Ask
 a fr
iend
/r
el
at
iv
e 
for h
e
lp 
___ Giv
e
 up or
 le
av
e it 
unsolv
e
d 
 
  
Sect
ion I
II: 
For th
e 
f
o
l
l
o
w
ing ques
t
i
o
ns, p
l
eas
e
 cho
o
s
e th
e numb
er th
at b
e
st 
c
o
rresp
o
nds 
to your 
fe
el
in
gs 
 
1.
  Com
puters m
a
k
e
 
m
e
 fee
l:   
V
e
r
y
 Un
c
o
mf
o
r
t
a
b
l
e
 
1   
2   3   4   5 
   6   7  
 8    9   
 
Very Comf
ort
abl
e 
 2.
  Whe
n
 y
ou run 
into
 a p
r
o
b
lem
 on the 
com
puter o
r
 an
 appl
ica
t
i
on y
ou ar
e us
ing
,
 do y
ou fee
l
: 
   
Anxi
ou
s 
  1   2
   3   4  
  5   6
   7    8   9  
 
Rel
a
xed
/I
ndi
ff
e
r
ent
 
 3.
  Whe
n
 y
ou encoun
te
r a p
r
obl
em
 on the com
pute
r
 or
 
an app
l
i
cat
ion y
ou a
r
e u
s
in
g
,
 how do y
ou feel 
abou
t y
our ab
il
ity
 to 
fi
x i
t
? 
   
He
l
p
le
s
s
  
 1   2   
3   4   5  
  
6    7   8    
9  
C
onf
ide
n
t 
I ca
n fi
x i
t
 
 4.
  How expe
ri
enced
 do y
ou thi
n
k
 y
ou are when 
it 
co
m
e
s to using
 a com
pute
r
? 
   
Very Inexp
er
ienc
ed
  1   
2   
3   4   5    6
   7   8  
 9  
Ver
y
 Exper
i
enc
e
d 
 5.
  Whe
n
 th
ere 
is 
a pr
obl
em
 with a
 com
puter 
tha
t
 I
 can'
t
 im
m
e
diately
 solv
e, I
 would st
ick
 with 
it 
unt
il I
 hav
e
 t
h
e answe
r
. 
   
Stron
g
ly D
i
sag
r
ee
 
 1   2   3
 
  4    5    6  
 7   8   
9  
St
rong
ly Agr
ee 
 6.
 I
f
 a probl
em
 is le
ft u
n
re
s
o
lv
ed on a 
com
puter, I
 would con
t
i
nue 
to t
h
ink
 abou
t 
it
 aft
e
rwa
r
d. 
   
Stron
g
ly D
i
sag
r
ee
 
 1   2   3
 
  4   5    6   
 7   8   
9  
St
rong
ly Agr
ee 
 7.
 Rig
h
t now, how s
a
t
i
s
f
ie
d
 with y
our 
li
fe a
r
e y
ou? 
   
Very U
n
sa
t
i
sf
ied
 
 1   2   
3   
4    5   6   7
    8    9
  
V
e
r
y
 Sa
tis
fi
ed 
 8.
 How oft
e
n do y
ou g
e
t up
set ov
er 
th
ing
s
?    
N
o
t V
e
r
y
 O
f
ten
  1  
 2    3   
4
   5   6   7   
8    9    
V
e
ry O
f
t
e
n 
 9.
 Rig
h
t now, m
y
 
m
ood is: 
  
Very Unha
ppy
  1
   2    3 
  
4   5   6   7 
  8    9  
V
e
ry Ha
ppy
 
   
 20Appendix
 B. Pos
t
-
S
ession 
Surv
ey
 
 
 
Nam
e
:
  
_____
_______
___
_________
_ 
Em
a
i
l:
  __
___
_________
______
______
_
___ 
Clas
s: _
______
______
___
 
 Pleas
e
 c
i
rc
le 
the nu
mber 
t
h
at bes
t
 co
rre
sponds 
to y
o
ur 
f
e
el
ings. 
 1. 
Rig
h
t now, m
y
 
m
ood is: 
 
Very Unhap
py  
  1
   2   3  
  
4   5  6   7  
 8   9    
V
e
ry Ha
ppy
 
 
2. 
We a
s
k
e
d y
ou to re
cord
 y
our f
r
us
tra
t
ing
 exper
i
en
ces. 
Ov
erall, h
o
w fr
ust
r
a
t
ed 
are 
y
ou afte
r th
ese 
expe
ri
enc
e
s
?
 
 
Not Frus
tra
t
ed
 at A
l
l
 
   1  2
   3   4    5  
 6   7   
8   9    
V
e
ry Fru
s
tr
ate
d
 
 
3. 
H
o
w
 w
ill 
the 
fru
st
ra
tio
ns t
h
at y
ou expe
ri
enc
e
d a
f
fe
ct 
th
e res
t
 o
f
 y
our day
?
 
 
Not at A
l
l
    1  2  
 3   4   
5   
6    7   8   9
    
Ver
y
 Much
 
 4. 
A
r
e th
e in
cid
e
nt
s t
h
at 
occu
r
r
ed w
h
ile y
ou w
e
re r
eco
rdi
n
g
 y
our experi
ence
s ty
pi
cal
 of y
our ev
ery
d
ay
 com
puter
 expe
rie
n
ce? 
 
_____  Yes 
            
    ____
__  No 
 5. 
I
n
 
g
e
neral, do
 y
ou expe
rie
n
ce m
o
re or 
les
s
 f
r
us
tra
t
ing
 incide
nts
 whi
l
e us
ing
 a com
pute
r
 on a
n
 av
erag
e day
?
 
 
Less
     
1   2   3 
  4   5   
6    
7   8  9    
Mor
e