Study Case : Style Theory, Item Review
Background and strategic fit
- Currently, our users need to fill in item reviews after they’ve returned the item, before they can book their next box. There are room for improvement in timing and the survey itself. The timing blocks the user from going to their next step.
- The survey asks a lot of information other than in the fit category - that we end up not using, because the information is in free text and unstructured format. It’s hard for us to go through all the input data for useful information. Additionally, because it’s in free text
format, someone needs to monitor and approves the reviews. We don’t have capacity for that and so all of those reviews end up not being used.
- Currently, our users need to fill in item reviews after they’ve returned the item, before they can book their next box. There are room for improvement in timing and the survey itself. The timing blocks the user from going to their next step.
- The survey asks a lot of information other than in the fit category - that we end up not using, because the information is in free text and unstructured format. It’s hard for us to go through all the input data for useful information. Additionally, because it’s in free text
format, someone needs to monitor and approves the reviews. We don’t have capacity for that and so all of those reviews end up not being used.
Goal
- Get useful data from reviews For Data and Merchandise Team
- Make review easier to complete for users
- Get useful data from reviews For Data and Merchandise Team
- Make review easier to complete for users
Success Metrics
- Time to completion
- “Honesty” ratio to come back to pre-revamp levels
- Questions for Data to Answer
- Time to completion
- “Honesty” ratio to come back to pre-revamp levels
- Questions for Data to Answer
Research
We research best practices and found some best practices for item review such as google play store and other app.We also research for the rating scale at first the idea was to have 3 points scale back then but after researching I believe that 5 points scale works better for our item review.
These are the keys take away and the summary of the review scale :
https://measuringu.com/three-points/
- More scale points increase reliability. Not much has changed in the 40–50 years since Nunnally’s work. Despite changes in survey formats (especially web surveys), the literature overwhelmingly shows that as you increase the number of scale points you increase reliability. The reliability will increase most when going from three to five points and less is gained as you exceed seven or eleven points.
- Three-point scales are not reliable. Across multiple studies, using only three points was shown not only to have lower reliability, but the reliability wasn’t even adequate, and in some studies, even when averaging across items, too much is lost. The low reliability is a consequence of using a coarse scale to represent a continuum. By forcing respondents to choose too few categories, it introduces more error in responding and thus makes responses less consistent in the same study (internal reliability) and over time (test-retest reliability).
- There’s a loss of intensity and validity with three-point scales. Using only three points loses all information about intensity, or the strength of people’s opinion. Not everyone feels equally favorably or unfavorably toward a brand, interface, or experience. All but one of the 12 studies we examined recommended using more than three points in a rating scale. In the one study that did suggest three-point scales were enough, it averaged the results across multiple items, not using a single item.
- Two- and three-point scales are perceived as quicker and easier. In one study, participants rated two- and three-point scales as being quicker and easier to respond to. This lends credence to the idea that shorter scales require less mental effort to respond to. If speed and the perception of ease is paramount (especially if using multiple items) a researcher may decide that two or three points are good enough. But know that while respondents may save time, it may stifle their ability to actually express their opinions. Or, if you look at the scale as a user interface, you’re preferring a faster and easier but less effective scale.
-Two and three points are insufficient to express feelings adequately. While two- and three-point scales may be perceived as faster and easier to respond to, participants overwhelmingly felt they were inadequate in allowing them to express their feelings. We’ll attempt to replicate these findings and measure actual response time (as opposed to perception of speed) in an upcoming article.
- The three-point scale superiority is a myth. If you’re concerned about putting a burden on your respondent by including more points, this analysis suggests participants don’t find scales with more than three points necessarily more difficult. In fact, in one study, scales with five, seven, and ten points were rated as EASIER to use than two- and three-point scales.
User Flow
