1 | Minnesota Tablet Study
Minnesota Tablet Usability Study Report
May 2015
Background
The Minnesota Comprehensive Assessments (MCA) are currently administered using desktop
and laptop computers. With the recent rise in the use of tablet computers, the Minnesota
Department of Education (MDE) will allow students to test on tablets beginning in the 2015-
2016 school year. MDE and Pearson conducted a usability study to evaluate whether there are
features of the test items or of the tablets that might disadvantage students testing on tablets.
Previous research on the use of tablets for testing suggests three major differences between
desktop/laptop computers and tablets that may pose challenges for students using tablets to
respond to long reading questions and technology-enhanced reading, math and science items.
1. Screen size
• The reduced screen size of tablets compared to desktop/laptop computers often
requires additional scrolling and paging through long reading passages, or
difficulty interacting with technology-enhanced reading, math, and science
items.
2. Touch-screen keyboards rather than physical keyboards
• Tablet computers offer no resting state for fingers when using the touch-screen
keyboard, thereby limiting one’s ability to use “keyboarding” skills.
• Touch-screen keyboards “pop-up” and cover screen real estate while in use,
making the effective screen size even smaller.
3. Lack of a mouse
• Fingers are less precise than a mouse when used as a pointer.
• Without a mouse, it is more difficult to place the cursor at the desired point for
text revision.
Previous Pearson research on student experiences with tablets is found at the following
address: http://researchnetwork.pearson.com/tablets
Purpose
Because none of the tablet research to date has been conducted specifically using Minnesota
students and Minnesota test content, MDE commissioned a study. The purpose of the
Minnesota Student Tablet Study was to capture and summarize student experiences taking
Minnesota test questions on a tablet. The results of the study would inform implementation
decisions regarding allowable devices for testing. The study was aimed at 4
th
and 9
th
graders.
Method
A brief training was initially provided by Pearson staff for district and school technology staff
and administrators. Using standardized scripts and materials, participating students were
administered a set of exemplar MCA test items using a tablet and then responded to a brief
survey about their experience. The test items were not scored.
The districts / schools were
advised that they had the option to use external keyboards. Pearson did not collect that detail
from the participating districts or schools as it was not defined initially as a key requirement.
The items presented to the students included long reading passages, and technology-enhanced
items from reading, math, and science. Items that were selected for the study were those that
required the student to interact with the questions by scrolling, using the touch-screen
keyboard, and using their finger for response selections.
2 | Minnesota Tablet Study
Students were only permitted to use tablets that met the requirements of the TestNav 8
testing platform. The table below lists the features of the eligible tablets. All participating
districts/schools were responsible for providing the tablets for student test-taking.
Tablet Requirements for Study Participation
Tablet Type Screen Size Minimum Operating System
iPad 9.7 inches iOS 7 and iOS 8
Android 10.1 inches Android Lollipop
Participants
Participation in the study was voluntary. The Minnesota District or School Consent Form was
provided to each participating institution and was returned signed by one or more
administrative staff members. The district (or school) level consent form indicates that
notification for student participation was handled by each district/school. MDE was provided
the consent form prior to Pearson distributing for the study. Signed forms were received from
the following districts: Minnetonka, Wadena-Deer Creek, Wayzata, Win-E-Mac.
A total of 110 students representing 5 schools and 4 districts participated in the tablet study.
Twenty-two (20%) were from grade 4 and eighty-eight (80%) were from grade 9. Of the
participants, 55% were female and 45% were male. The sample was composed predominately
of white students (85%), followed by Asian (6%), Hispanic (6%), black (2%) and Indian (1%)
students.
Results
A summary of responses to all survey questions can be found in the appendix of this
document. Key findings are summarized below:
Previous Tablet Use
• 58% of students reported using tablets at home for fun
• 75% of students reported using tablets for school work
• 74% of students reported having an arrangement with their school that allows them to
take tablets home
• 82% of students reported having used a tablet to take a test or quiz at school
Re
sponses to Tablet Administration of MCA Items
• 88% of students said something unexpected happened never or rarely when clicking on
something in the test
• 78% said they could see everything they needed on the test most or all of the time
• 89% of students reported zooming in never or rarely
• 9% said zooming made it more difficult to answer the question
• 70% used the onscreen keyboard only
• 91% reported that it was easy or very easy to use the keyboard
• 5% of students thought scrolling made answering the items very hard and an additional
35% thought it was somewhat hard
• 74% reported having no problem seeing the animations and simulations
• 81% had no problems using the tablet to answer math questions
3 | Minnesota Tablet Study
Discussion/Conclusion
The results from this small-scale study suggest that the participants were relatively adept at
using tablet computers both at home and at school. In addition, the majority of students
surveyed had little difficulty using the tablet to take the test and respond to the items.
However, a non-trivial proportion of students reported problems with scrolling and seeing
animations/simulations.
Limitations
The results of this study should be interpreted with caution for several reasons. First, the
sample used is not of sufficient size or representativeness to permit drawing strong inferences
about the full population of Minnesota students. The sample size did not permit the
disaggregating of results to investigate patterns among various subgroups. In addition, the
participants comprised a convenience sample, including only those schools who volunteered to
participate. It is possible, if not likely, that the participating districts are the most ardent users
of tablets. It is unclear at this point whether the results would generalize to the state as a
whole, or to students/schools that are not as adept with using tablets. Furthermore, only
students in two grades were surveyed and the sample was heavily weighted toward 9
th
graders, with roughly four times as many 9
th
graders as 4
th
graders. It is not known whether
the results observed here will generalize to other grade levels. Additionally, it is not known
whether students who did not use an external keyboard chose not to, or were not given that
option by their school or district. Finally, students may not have been motivated to take the
survey seriously, which might limit the validity of the results.
4 | Minnesota Tablet Study
Appendix: Survey Responses for All Questions
The tables below provide the number and percentage of students who selected each option for
each survey question. Note that percentages will add up to 100 (within rounding) in cases
where only one choice was permitted. However, for multiple response questions (i.e., check all
that apply), the percentages may exceed 100 since multiple selections were possible.
6. Which of the following do you regularly use at home for fun (for example:
playing games, talking with friends, etc.)? (check all that apply)
Type
Number
Percentage
Desktop computer
24
21.82%
Laptop computer
43
39.09%
Touch-screen tablet (for example: iPad, Kindle Fire, Galaxy Tab,
Nexus 7, etc.)
64
58.18%
Smart phone (a phone with a touch screen and/or keyboard)
85
77.27%
None of the above
3
2.73%
7. Which of the following do you regularly use for school work (for example:
homework, classroom work, research, etc.)? (Check all that apply)
Type
Number
Percentage
Desktop computer
39
35.45%
Laptop computer
45
40.91%
Touch-screen tablet (for example: iPad, Kindle Fire, Galaxy Tab,
Nexus 7, etc.)
83
75.45%
Smart phone (a phone with a touch screen and/or keyboard)
25
22.73%
None of the above
5
4.55%
8. In which classes in school do you use a tablet? (Check all that apply)
Type
Number
Percentage
English/Language Arts class
75
68.18%
Math class
88
80.00%
Science class
83
75.45%
Social Studies class
54
49.09%
Music or Art class
17
15.45%
I do not use a tablet in school
7
6.36%
9. Which of the following best describes the tablet you use for school work?
Type
Number
Percentage
The tablet is assigned to me at the beginning of the year and I can
take it home with me.
81
73.64%
The tablet is assigned to me at the beginning of the year and it
stays at school.
3
2.73%
The tablet is checked out to me for certain classes or activities but
returned at the end of the class or activity.
16
14.55%
The tablet I use for school is one that I personally own.
2
1.82%
I do not use a tablet for school work.
8
7.27%
5 | Minnesota Tablet Study
10. For what types of school activities do you use your tablet? (Check all that apply)
Type
Number
Percentage
Research/looking things up
92
83.64%
Creating projects
68
61.82%
Taking pictures
49
44.55%
Watching videos
53
48.18%
Checking grades
81
73.64%
Doing homework
83
75.45%
E-mailing or messaging the teacher
57
51.82%
E-mailing or messaging other students
30
27.27%
Reading books or stories
47
42.73%
Writing essays or papers
70
63.64%
Group projects
62
56.36%
None of the above
6
5.45%
11. How many text messages do you usually send each day?
Type
Number
Percentage
I do not send text messages
18
16.36%
Fewer than 10 text messages
25
22.73%
Between 10 and 50 text messages
35
31.82%
Between 50 and 100 text messages
16
14.55%
More than 100 text messages
16
14.55%
12. Which of the following have you EVER used to take a test or quiz at school?
(Check all that apply)
Type
Number
Percentage
Desktop computer
87
79.09%
Laptop computer
69
62.73%
Touch-screen tablet (for example: iPad, Kindle Fire, Galaxy Tab,
Nexus 7, etc.)
90
81.82%
Smart phone (a phone with a touch screen and/or keyboard)
9
8.18%
None of the above
1
0.91%
13. When taking the test how often did something unexpected (something other
than what you meant to do) happen when you clicked or tapped on part of a test
question?
Type
Number
Percentage
Never (0 times)
39
35.45%
Rarely (1-2 times)
49
44.55%
Sometimes (3-5 times)
18
16.36%
Often (more than 5 times)
4
3.64%
14. When taking the test could you see everything on the screen at the same time
that you need to answer the questions on the test (the story, the question itself, the
place where you put your answer, etc.)?
Type
Number
Percentage
No, I could not ever see everything I needed
4
3.64%
Yes, I could see what I needed some of the time
21
19.09%
Yes, I could see what I needed most of the time
49
44.55%
Yes, I could see what I needed all of the time
36
32.73%
6 | Minnesota Tablet Study
15. How often did you have to make the parts of the test item larger (zoom in) or
smaller (zoom out) in order to answer the question?
Type
Number
Percentage
Never (0 times)
63
57.27%
Rarely (1-2 times)
36
32.73%
Sometimes (3-5 times)
9
8.18%
Often (more than 5 times)
2
1.82%
16. On which tests did you have to make images larger (zoom in) or smaller (zoom
out)? (Check all that apply)
Type
Number
Percentage
Reading
21
19.09%
Science
20
18.18%
Math
16
14.55%
No subject areas
70
63.64%
17. In your opinion, did the process of making the test larger (zoom in) or smaller
(zoom out) seem to make it harder to answer the test item?
Type
Number
Percentage
Yes
10
9.09%
No
31
28.18%
I am not sure
19
17.27%
I did not have to make any test items larger or smaller
50
45.45%
18. In your opinion, did it seem like you had to spend more time to answer the test
items that needed to be made larger (zoom in) or smaller (zoom out)?
Type
Number
Percentage
Yes
18
16.36%
No
28
25.45%
I am not sure
15
13.64%
I did not have to make any test items larger or smaller
48
43.64%
Missing response
1
0.91%
19. Did you use the onscreen keyboard or external keyboard?
Type
Number
Percentage
Onscreen only
77
70.00%
External keyboard
11
10.00%
I used both
22
20.00%
20. How easy or hard was it to use the keyboard to take your test?
Type
Number
Percentage
Very Easy
34
30.91%
Easy
66
60.00%
Hard
10
9.09%
Very Hard
0
0.00%
7 | Minnesota Tablet Study
21. Did scrolling through the reading passages make it hard to answer the test
questions?
Type
Number
Percentage
Scrolling did not make it hard to answer the reading questions
66
60.00%
Scrolling made it somewhat hard to answer the reading questions
38
34.55%
Scrolling made it very hard to answer the reading questions
6
5.45%
22. On the Science test did you have any problems seeing the animations and
simulations clearly enough to answer the questions?
Type
Number
Percentage
I did not have any problems seeing the animations and simulations
81
73.64%
I had some problems seeing the animations and simulations
25
22.73%
I had a lot of problems seeing the animations and simulations
3
2.73%
Missing response
1
0.91%
23. On the Math test did you have any problems using the tablet to answer the
questions?
Type
Number
Percentage
I did not have any problems using the tablet to answer the math
questions
89
80.91%
I had some problems using the tablet to answer the math questions
17
15.45%
I had a lot of problems using the tablet to answer the math
questions
2
1.82%
Missing response
2
1.82%
