Technology

I don't like the kind of A/B testing that large corporations do, but I'm not so certain that this particular user experiment Reddit is running would qualify for rigorous IRB review in most academic settings.

Firstly, let's talk about consent. An IRB can make a determination to waive the requirement to obtain informed consent for research. The IRB must find and document:

(i) The research involves no more than minimal risk to the subjects;
(ii) The research could not practicably be carried out without the requested waiver or alteration;
(iii) If the research involves using identifiable private information or identifiable biospecimens, the research could not practicably be carried out without using such information or biospecimens in an identifiable format;
(iv) The waiver or alteration will not adversely affect the rights and welfare of the subjects; and
(v) Whenever appropriate, the subjects or legally authorized representatives will be provided with additional pertinent information after participation.

Secondly, some kinds of research can be waived from documenting informed consent:

(i) That the only record linking the subject and the research would be the informed consent form and the principal risk would be potential harm resulting from a breach of confidentiality. Each subject (or legally authorized representative) will be asked whether the subject wants documentation linking the subject with the research, and the subject’s wishes will govern;
(ii) That the research presents no more than minimal risk of harm to subjects and involves no procedures for which written consent is normally required outside of the research context; or
(iii) If the subjects or legally authorized representatives are members of a distinct cultural group or community in which signing forms is not the norm, that the research presents no more than minimal risk of harm to subjects and provided there is an appropriate alternative mechanism for documenting that informed consent was obtained.

Insofar as the kind of UI/UX A/B testing which is employed poses minimal risk to the participant and the waiving of the need to obtain informed consent has no adverse effect on the participant, an IRB is likely to make a determination that consent can be waived. It would not surprise me that Universities themselves utilize UI/UX A/B testing for their own websites, both external and internal facing, for improvement. I doubt many would explicitly file a with their IRB to conduct such an experiment, but some may reach out to inquire if there are explicit concerns.

However, at a level even above informed consent, is a question of whether the research is actually subjected to IRB review to begin with. There is a classification of research that is exempt from human subjects review, and some kinds of research do qualify for human subjects review exemptions.

For UI/UX A/B testing, this particular section will have some application, considering a lot of UI/UX A/B testing only cares for aggregate responses to human behavior:

(2) Research that only includes interactions involving educational tests (cognitive, diagnostic, aptitude, achievement), survey procedures, interview procedures, or observation of public behavior (including visual or auditory recording) if at least one of the following criteria is met:

(i) The information obtained is recorded by the investigator in such a manner that the identity of the human subjects cannot readily be ascertained, directly or through identifiers linked to the subjects;
(ii) Any disclosure of the human subjects’ responses outside the research would not reasonably place the subjects at risk of criminal or civil liability or be damaging to the subjects’ financial standing, employability, educational advancement, or reputation; or
(iii) The information obtained is recorded by the investigator in such a manner that the identity of the human subjects can readily be ascertained, directly or through identifiers linked to the subjects, and an IRB conducts a limited IRB review to make the determination required by §46.111(a)(7).

Now, one can make the argument that the kind of profile information websites like Reddit have on you is identifying. In practice, "personally identifying information" has specific definitions and the information that Reddit has on you is unlikely to satisfy that criteria.

Finally, this particular set of charts is a helpful reference for whether research even qualifies for IRB/human subjects review to begin with, and walks through the decision points. Notice that one specifically starts with a question of whether the research contributes to generalizable knowledge...

In short, if you ran the exact same experiment Reddit is running in an University setting for an university website (for example, testing whether making visitors create an university website account [????] to view an article published by the University press versus not having to do so makes a difference in engagement metrics), I doubt you would be run out of the University by the IRB. Perhaps the IRB might have a stern word if you failed to check-in with them prior, but even so I'm not confident that will be the case (I'm more inclined to believe the IRB member who you first reach out to will kindly tell you to sod off).

Now with ALL of that said, I still dislike the fact businesses run these experiments. It's definitely not ethical in the sense that businesses should not be aggressively using its everyday users as guinea pigs for their experiments, but just merely being a shitty thing to do is not sufficient by itself to merit the full IRB process.