The FTC and Independent Reverse Engineering

This dataset measures how often the Federal Trade Commission relies on the work of independent researchers when regulating consumer privacy and security.

We manually analyzed all public FTC actions related to consumer privacy and security between Jan. 1, 2017 and July 15, 2024, a total of 102 FTC cases and 332 individual counts (see our methodology for more details).

The dataset accompanies a law review article by Michael A. Specter and Andy Sellars, forthcoming in 2025. We are making this dataset available ahead of publication in order to crowdsource bugs and receive feedback from the community.

For details on the analysis of software security as a credence good, see Specter’s dissertation, Chapter 2.

Summary Statistics

We examined 102 cases with a total of 322 different “counts.” Of these, 283 counts required some form of external investigation.

In 26.5% of these counts, the FTC learned about the issue thanks to some form of independent research. Note that this is a low bar: 33.7% of cases have at least one count that is based on independent research.

Where is independent research most effective?

Category	Counts based on independent research	Counts that required some investigation	% of counts informed by independent research
§ 5 deception – privacy claims	17	42	40.5%
§ 5 deception – security claims	16	37	43.2%
§ 5 unfairness – data collection/use/disclosure that harmed consumer	8	15	53.3%
§ 5 deception – “deceptive failure to disclose” data collection, use or disclosure	8	12	66.7%
Children’s Online Privacy Protection Act	6	15	40%
§ 5 unfairness – cybersecurity standards	5	16	31.3%

Who are these independent researchers?

Category	Counts	Example
Journalists at media organizations	53	The Markup on Meta Pixel tracking
Unaffiliated researchers	19	Jonathan Leitschuh on Zoom bypassing MacOS safeguards
Advocacy organizations	12	Campaign for Commercial Free Childhood on Amazon Echo
Cybersecurity vendors	10	Kryptowire on BLU Products sending data overseas
Institutional academics	7	Reardon et al. on OpenX bypassing Android permissions
Other software vendors	3	Wladimir Palant (AdBlock Plus) on Avast collecting browser data
Trade organizations	2	BBB’s Children’s Advertising Review Unit on COPPA violations

Dataset

Below is our raw dataset. The same information is available in raw CSV form: actions.csv claims.csv.

Methodology

Dataset Collection

We look here at public cases brought by the FTC in matters related to consumer privacy and security. The FTC maintains a library of all public cases and proceedings, and tags actions “Privacy and Security” if they relate to those issues. We include a small number of privacy and security cases that are tagged with related labels, including cases tagged as COPPA actions, and cases labeled “Consumer Privacy.”

We include all cases we could find in the FTC’s library with these tags that had an initial filing (either a complaint before the FTC or a complaint in a federal district court) between 1 January 2017 and 15 July 2024. In the event the action has an amended complaint(s), we look at the most recent complaint.

At the end of each complaint, the FTC articulates discrete counts, or specific alleged violations of different laws. We include each of those counts in the dataset, unless the count is an alleged violation of a state law (This occurs when the FTC brings an action with a state Attorney General as co-plaintiff.) For each count, we scrutinize the complaint and conduct web searches and searches of news databases to determine how the FTC became aware of the facts that led to the respondent or defendant’s alleged liability. We describe below how we classify our findings, and provide our count-by-count justification in the dataset.

For those where independent research was involved, we also label the researcher by the type of research party they are, which we describe below.

Codeing Methodology

Cause of action

We categorize each claim based on the law alleged to be violated, with two main exceptions:

• For claims alleging a violation of a prior FTC consent order, such actions are labeled “Violation of Earlier FTC Order”
• For violations of “Section 5,” the general statute the empowers the FTC to pursue claims for unfair or deceptive practices (15 U.S.C. § 45), we identify whether the FTC alleges a “deceptive” trade practice or an “unfair” trade practice, and we further label the general category of unfair or deceptive behavior:
  • For deception claims:
    • Deceptive commercial claims – false or misleading information about the commercial aspects of a transaction, including payment and financing terms.
    • Deceptive failure to disclose data collection, use, or disclosure – statements by vendors that omit material information about how the company collects, uses, or discloses consumer information.
    • Deceptive statement re: COPPA or HIPAA compliance – false or misleading statements about the vendor’s compliance with the Children’s Online Privacy Protection Act or the Health Insurance Portability and Accountability Act.
    • Means or instrumentalities to engage in deception – providing information or material that allowed others to make false or misleading commercial statements.
    • Privacy claims – false or misleading claims about the vendor’s collection, use, or disclosure of information about consumers.
    • Security claims – false or misleading claims about the safeguards the vendor has adopted to prevent the unwanted collection, use, or disclosure of consumer information.
    • US/EU Privacy Shield – false or misleading claims about the vendor’s compliance with the US/EU Privacy Shield program, the legal agreement in effect from 2016 to 2020 that allowed for cross-border transfer of personal information between the European Union and United States. (This program superseded the previous US/EU “Safe Harbor,” and now has been superseded by the EU–US Data Privacy Framework.)
  • For unfairness claims:
    • Billing practices – procedures for billing customers that cause an injury to consumers.
    • Collection, use, or disclosure of data that causes consumer harm – the vendor collects, uses, or discloses data that causes an injury to consumers. These claims are often based on the use or disclosure of especially sensitive consumer information, such as consumer health information.
    • Cybersecurity standards – adoption of cybersecurity procedures or standards that are so poor (or entirely absent) that they cause an injury to consumers.
    • Data retention standards – adoption of standards on the access, use, or retention of consumer data that are so poor (or entirely absent) that they cause an injury to consumers.
    • Failure to obtain Affirmative Express Consent prior to disclosing sensitive information – failure to adhere to the FTC’s standards requiring companies to obtain the user’s affirmative express consent prior to disclosing certain types of sensitive information.
    • Injurious product – offering a product that is known to cause consumer injury.
    • Intentional degrading of security – intentionally degrading the security a consumer has adopted with respect to their information.
    • Outsourcing privacy obligations – attempts to wholly outsource the vendor’s obligation to safeguard a consumer’s privacy or security.
    • Retroactive policy change – attempts to retroactively alter a vendor’s privacy policy or similar statement, without providing notice and obtaining consent from affected consumers.
    • Use of known inaccurate data or failure to honor consumer choices – intentionally departing from a consumer’s stated preferences for data collection, use, or disclosure, or using information about a consumer that is known to be inaccurate.

The FTC’s discovery of the issue

We define “independent research” as research that was conducted (1) by a private party; (2) not at the request of the software vendor or the government; and (3) with the apparent objective of sharing the findings with a regulator, the general public, or some portion of the general public.

We label the FTC’s discovery of the facts that inform their alleged violation as follows:

• Directly from independent research – we are able to conclude from the available facts that the FTC learned of the issue from independent research. This can be shown by:
  • Statements by the FTC in the complaint, press release, or contemporaneous statement from an FTC Commissioner that states that the issue was discovered through independent research.
  • The FTC in a complaint, press release, or contemporaneous statement from an FTC Commissioner cites a third-party “FTC complaint” as the basis of discovery of the claim, and the third-party filing cites independent research as the basis for discovering the issue.
• Likely from independent research – it appears that independent research was the most likely source of the claim, although the FTC does not state so specifically. This is shown by:
  • The FTC in the complaint, press release, or contemporaneous statement from an FTC commissioner cites to generally to “the press” or similarly as the source of discovery of the issue, provided we can also identify a contemporaneous news report that claims to have revealed the issue.
  • The FTC notes the approximate date of discovery of the issue, and around that time there was independent research that claims to be the basis of discovering the issue.
  • The complaint discloses that an independent researcher contacted the vendor directly about the issue, and the vendor in turn told the FTC or the general public about the issue.
  • The FTC does not specifically credit a third-party “FTC complaint” as the instigation of the investigation, but there is a publicly-known third-party complaint that cites independent research as the basis of discovery of the issue that predates any apparent investigation by the FTC into the same issue.
  • There was prominent prior news coverage about an independent researcher’s discovery of the issue that predates any apparent investigation by the FTC into the same issue.
• Indirectly related to independent research – the FTC did not directly discover the issue from independent research, but there is a strong nexus between independent research and the means the FTC used to discover the issue. (To be clear, claims labeled under this category are not considered discovery due to independent research in the summary statistics above, but are mentioned instead to show other ways in which independent research plays an important role in this ecosystem.) This can include:
  • General news or academic reporting about a vendor, which then prompted the FTC to conduct its own investigation and discover the specific issue.
  • The issue was initially discovered by a malicious actor, by an independent researcher brought the malicious actor’s behavior to the public’s or FTC’s attention.
  • There was prominent prior news covered about the specific industry or type of behavior that became the basis of the claim, and the FTC’s investigation appears to have shortly followed after such prominent reporting.
• Unlikely independent research – it appears that the FTC learned about the issue through another means of discovery other than independent research. This is shown by:
  • The FTC notes the approximate date of discovery of the issue, and around that time there was a source other than independent research that claims to be the basis of discovering the issue.
  • The issue appears to be only discoverable through investigation of a vendor’s internal administrative practices or other information that would be inaccessible to independent researchers.
  • The context around the complaint suggests that the FTC learned of the issue directly from injured consumers.
  • The FTC does not specifically credit a third-party “FTC complaint” as the instigation of the investigation, but there is a publicly-known third-party complaint that cites a source other than independent research as the basis of discovery of the issue.
  • There was prominent prior news coverage about the discovery of the issue through a means other than independent research that predates any apparent investigation by the FTC into the same issue.
• Not independent research – we are able to conclude from the available facts that the FTC learned of the issue through a means other than independent research. This can be shown by:
  • Statements by the FTC in the complaint, press release, or contemporaneous statement from an FTC Commissioner that states that the issue was discovered through a means other than independent research.
  • The FTC in a complaint, press release, or contemporaneous statement from an FTC Commissioner cites a third-party “FTC complaint” as the basis of discovery of the claim, and the third-party filing cites a means other than independent research as the basis for discovering the issue.
• Uncertain – we are unable to ascertain how the FTC discovered the information, and think it equally plausible that the discovery came from independent research or not.

The type of independent researcher

We categorize independent researchers as follows:

• Advocacy Organization – the independent researcher’s primary occupation or affiliation seems to be at an organization dedicated to advocating for law and policy reform in privacy, security, or another policy area.
• Cybersecurity Vendor – the independent researcher’s primary occupation appears to be at a company that primarily provides cybersecurity technology or consultation services to other vendors.
• Institutional Academic – the independent researcher’s primary occupation appears to be a professor, instructor, researcher, or other staff member at an institute of higher education.
• Journalist at Media Organization – the independent researcher appears to be a professional journalist working at an organization dedicated to informing the public or members of the public on matters of general public concern.
• Other Software Vendor – the independent researcher’s primary occupation appears to be at a company that provides software and software services, in areas other than cybersecurity. (Those who provide software or software services primarily in cybersecurity are labeled as a “Cybersecurity Vendor.”)
• Trade Organization – the independent researcher’s primary occupation appears to be at an organization that
• Unaffiliated Researcher – the independent researcher appears to be a researcher who operates without any affiliation with a software company, institute of higher education, or media organization.
• Unknown – we are unable to identify the independent researcher’s primary occupation or affiliation, or confirm that they appear to operate without any such affiliation.

Want to cite this dataset?

@online{specterFTCIndependentReverse,
  title = {The {{FTC}} and {{Independent Reverse Engineering Dataset}}},
  author = {Specter, Michael and Sellars, Andrew},
  url = {https://ftcreverse.engineering/},
  urldate = {2025-01-10},
}