See also EED Usability Testing for a general usability testing process overview

Participant Notes

Tasks

  • Develop initial questionnaire user-ee5fe  
  • Develop interview schedule user-ee5fe  
  • Recording waiver form? user-c2913  
  • Identify individuals who will be tested. Patrick Quinn  
  • Design/define user tasks for users to complete. user-ee5fe Patrick Quinn  
  • Determine if any A/B design comps could/should be used Patrick Quinn  
  • Logistics planning Dana Shum  

Questions to Answer

  • What are the key difficulties that people encounter when visiting the site?
  • Is the "Download All" paradigm sufficient , or is a positive selection (shopping cart) mechanism necessary?
  • How important is discoverability and relevance, given the way people typically search?
  • At what point do the map and timeline become useful / at what point are they in the way?

Agenda

LP DAAC Usability Testing Agenda

Participants

In addition to the planned LPDAAC trip, the following people volunteered to participate in a usability study during the Earthdata Search webinar. See attached spreadsheet for additional information such as name and affiliation.

Tentative Interview Schedule

  • 0:00 - 0:10: Subject setup, introductions, pre-interview Q&A
  • 0:10 - 0:30: EDSC-specific participant tasks
  • 0:30 - 0:45: Day-to-day task walkthrough (how does the participant use software in their day-to-day activities–EDSC or not)
  • 0:45 - 0:50: Follow-up Q&A
  • 0:50 - 1:00: Tear-down, prep for next participant

We need to remain flexible here and swap out some of this for either A) more Q&A and interview-type questions or B) more software and EDSC-specific tasks and observation based on what the day-to-day of each participant looks like.

User Study Script

  1. Welcome participant, introduce Jeff and Patrick (have packet ready), survey the office for study locations and/or ask the participant where they'd like us and where they'd be most comfortable.
  2. GTM setup and recording waiver information and signing. Answer any questions or concerns about the recording and use.
  3. Review initial questionnaire with participant and ask if they have anything else to add. If the participant has not completed the survey--review the questions with them before the tasks start.
  4. Task Introduction:
    1. During this session, I would like you to think aloud as you work to complete the tasks. I will not be able to offer any suggestions or hints, but from time to time, I may ask you to clarify what you have said or ask you for information on what you were looking for or what you expect to have happen.
    2. Test Participant’s Role:
      1. Today I am going to be asking you to complete some tasks on Earthdata Search and tell me how easy or difficult it was to find the information.
      2. There is no right or wrong answer. If you have any questions, comments or areas of confusion while you are working, please let me know.
      3. If you ever feel that you are lost or cannot complete a task with the information that you have been given, please let me know. I will ask you what you might do in a real-world setting and then either put you on the right track or move you on to the next scenario.
      4. As you use the site, please do so as you would at home or your office and try to work through the tasks based on what you see on screen.
      5. I may ask you other questions as we go and we will have wrap up questions at the end.
      6. http://www.usability.gov/how-to-and-tools/resources/templates/introduction-to-testing-with-moderator-interaction.html
  5. Tasks:
    1. Find a data collection hosted at LPDAAC that has map imagery and view the latest imagery over the continental US
    2. Find and download all Landsat 8 images from January of this year which cover Sioux Falls and have no clouds
    3. Find an example of ASTER and Landsat 7 data collected within an hour of one another over Sioux Falls
    4. Use the client to find and retrieve data which may be relevant to your work
    5. As an alternative, show us your workflow from another application (one used more frequently than Earthdata Search)
  6. Follow up Q&A:
    1. Discussion on overall task completion--what went well, what didn't.
    2. What seemed to be the major hang-up points for you while trying to complete the tasks?
    3. What feature or features made it easier for you to complete tasks (or even sub-tasks)?
    4. Do you have any other questions or comments for us before we finish?
  7. Stop/end all recording and verify that GTM and any other media has been saved. Log out of GTM.
  8. Make sure participant has recording waiver and contact information.
  9. Say thanks and smile :)

Initial Questionnaire

  1. What is your position title and role within the Earthdata community?
  2. Do you currently use (or have you used) Earthdata Search (https://search.earthdata.nasa.gov/) in your current job capacity?
  3. Do you currently use (or have you used) any other search, discovery or visualization tools in your current job capacity (this could be an EOSDIS application, DAAC application or other 3rd-party application)? If yes, please list the applications.
  4. If you answered yes to (2) or (3), please estimate how many hours each week you spend with these applications.
  5. What is your largest pain-point when using search and discovery applications?
  6. What would be your top feature request for the application you use the most in your day-to-day tasks?

Software Tasks

  • Self-directed. sdfds

Webinar Top Feature Requests / Improvements

We asked webinar attendees which features they'd most likely to see. These are the answers:

  1. Event Related Searches (33 people)
  2. Parameter Level Facets (8 people)
  3. Improved Relevancy Rankings (7 people)
  4. Better OPeNDAP Tie-Ins (4 people)
  5. (Write-in) More flexible subsetting (1 person)

Note: The vast majority of attendees identified themselves as "Researcher/Decision Support- I need the data for my research/application(s)"

Barriers To Use

We asked webinar attendees what would prevent them from using Earthdata Search today. Note that some of the answers seem to treat the question as hypothetical, i.e. if suchandsuch were a problem I wouldn't use Earthdata Search, regardless of whether suchandsuch actually is a problem. These are the answers, in no particular order:

  • Time from discovery to data access
  • Interpretation of data (4 people said this. Unclear as to what this means, though)
  • Temporal resolution vs spatial resolution
  • Understanding the data itself
  • Network congestion
  • If the website is slow to respond to changes

  • I will definitely need to practice so that I understand what we just discussed.

  • I followed Patrick, but I coulnd see any snowcover data in January 2016? (Note: This is potentially a relevancy issue with several very similar-looking collections, only some of which have visualizations)

  • How to download subsetting for multiple sites?

  • Data formats (3 people said this)

  • and where is the multiband radar imagery??  Oh yeah...NASA still does not have radar satellites

  • access to free tools for processing digital

Note: The vast majority of attendees identified themselves as "Researcher/Decision Support- I need the data for my research/application(s)"

Related Tickets

Key Summary T Created Updated Due Assignee Reporter P Status Resolution
Refresh

 

 

  • No labels