Ravelry accessibility symptom survey methods
Many people have expressed concerns about data collection and usage plans around my survey and I’m going to try to speak to some of those issues here. For now, I plan to keep the survey open through mid morning Saturday April 10 (I’m in MDT). If you want more information on accessibility problems with ravelry, see this blog post.
While I am not by training a researcher that does work on human subjects, I do have a PhD in nuclear astrophysics and have plenty of experience drawing conclusions with biased datasets. I have personally had migraines from using both Merino and Classic skins, that started while using Ravelry and diminished (but did not immediately go away) after leaving the site. I am only a hobbyist fiber artist and do not rely on any income from my work with fiber.
I will mostly focus on statistical analyses of either all people affected or looking at groups of people with more common effects, but also plan to report out a summary of effects reported in “other” that don’t fit a named category in the survey responses.
To avoid privacy issues, I am not collecting any identifying information like names, usernames, or emails. In principle, this does mean that someone could respond multiple times, but I do have plans to do cross checks for data integrity. I can check for things like identical responses with near matching timestamps. I also plan to check that my control group (unaffected users) looks like what one would expect given what other data we have on typical ravelry users and estimates of how many people in a given age group wear glasses vs contacts vs neither. If things don’t look like one might expect, I’ll plan to dig deeper into these issues.
The sample in this survey is, as expected, biased. People more strongly affected by ravelry are more likely to be following the issue, and more likely to respond to a survey on the issue, so I have no plans to compare something like the number of affected responses to the number of unaffected responses. However, I will assume that people that are still invested in this issue likely interacted with ravelry at some point during the first month after launch. Someone captured the active users in the last 30 days one month after the redesign, and that figure is 894,175, which is comparable to their own reported active users per month statistic of 1 million.
I also plan to look at distributions within affected groups, seeing if people that suffer a particular effect are more likely to wear glasses or not, have issues with optical illusions, etc. This dataset will require care in the conclusions that can be drawn from it, but I expect there is useful information in the data.
