Characterisation and analysis of key studies used to restrict substances under REACH

Understanding how scientific studies are used in regulatory risk assessments is important since it influences the outcome of an assessment, and thus the level of protection of human health and the environment. Within the REACH legislation (Registration, Evaluation, Authorisation and Restriction of Chemicals, EC Nr. 1907/2006) hazard information on clearly defined (eco-)toxicological endpoints are submitted for the registration of substances, and this information is intended for concluding on potential hazards and risk as well as subsequent risk management measures such as restrictions. The present study aimed to (1) characterise key studies used by the European Chemicals Agency’s Committee for Risk Assessment to restrict hazardous substances; (2) analyse if the REACH registration database provided the key studies used in these restrictions, and (3) investigate potential expert disagreements related to the use of non-standard studies in the restrictions. Our analysis showed that 58% of the 53 scrutinised key studies were non-standard studies, all available by paying a fee or through open access. Sixteen (30%) of the key studies were consulted from external sources outside the REACH registration database by the Committee for Risk Assessment. Only one study of the 16 external key studies was a standard study. Further, 9% (5/53) of the key studies used by the Committee for Risk Assessment were inaccessible to third parties, all were standard studies. The uses of non-standard studies were (unsuccessfully) challenged for five substances during the public consultation. These results suggest that non-standard studies contributed to the identification and management of substances of concern, that the REACH registration database may not be sufficient for the identification and management of uncontrolled hazards and risks, and that the transparency of the decisions made by the Committee for Risk Assessment was partially hampered due to the use of standard studies inaccessible to third parties.


Background
The main objectives of the REACH regulation (Registration, Evaluation, Authorisation and Restriction of Chemicals EC Nr. 1907Nr. /2006 are to protect human health and the environment from unacceptable risks posed by hazardous chemicals while ensuring the competitiveness of the EU chemicals industry [50]. Since 2010, the European Chemicals Agency (ECHA) has processed 36 restrictions whose monetised net health benefits are estimated to be 400 million euros, and the reduced environmental and human health exposure of hazardous substances is estimated to be 95 000 tonnes per year [23,27]. Nonetheless, the REACH restriction system is currently subject to revision [47]. So far, the number of restrictions has not reflected the original expectations, and the identification procedure of relevant candidates for restriction needs to be improved [49]. The availability of adequate hazard information is a crucial element in initiating the Open Access *Correspondence: Flora.Borchert@aces.su.se restriction procedure. Generally, one or more so-called key studies on a substance's hazard(s) are used as justification to restrict a substance [44]. Although all available evidence should be included when developing a restriction proposal (REACH Annex XV), information from the REACH registration database (i.e. the dossiers) is intended for consideration as the primary source [44,50]. However, there were flaws detected in the current REACH registration system which might hamper a proper identification and implementation of chemical risk management measures [16,56,58].
Any company producing and/or importing chemicals at or above one tonner per year must register their substances under REACH. The required hazard information is fed into the REACH registration database by the companies (registrants) as study summaries (REACH Title II Article 10). The full study reports are owned by the registrants and can be requested by a Member State Competent Authority (MSCA), or ECHA, who are responsible for preparing a restriction dossier in case an uncontrolled risk to human health and/or the environment was identified and risk management measures in form of a restriction are deemed necessary. However, using study summaries instead of full study reports can have implications for the risk assessment. Study summaries submitted by the registrants were shown to be inconsistent or incomplete concerning the reporting of information about the study design, results, and interpretation of the results [59]. Also, the evaluation and reporting of the reliability and relevance of key studies in study summaries lacked structure and transparency [56]. Moreover, an examination of the availability of registration data revealed that more than half of the high-tonnage substances (i.e. produced or imported at or above 1000 tonnes per year) lacked the required information for one or more endpoints [16]. ECHA acknowledged that the "non-compliance of registration dossiers is hampering progress" to adequately address substances of concern. For 1262 of the approximately 19 000 substances registered in 2018, new data need to be generated, or existing data need to be evaluated in more detail, to enable a decision on risk management [28]. The incompleteness of REACH registration dossiers raises the overall question if the dossiers are useful and used to identify and restrict uncontrolled hazards and risks posed by hazardous substances. Analysing the key studies used in the REACH restriction process is vital to answering this question. Lack of information about chemicals, or lack of transparency of the available information, can have consequences for the implementation of risk management measures, such as restrictions. In return, a lack of appropriate risk management measures can have consequences for the protection of human health and the environment.
(Eco-)toxicity studies conducted according to internationally standardised and validated test guidelines, provided by for example the OECD or ISO, and approved by the European Commission, are so-called standard studies and usually submitted by the registrant [2, 43,56,57] to comply with the legal information requirements according to REACH Article 13 (3). In case required information is already available, the registrant shall use this information instead of conducting a new standard study. The advantage of standardised test methods is that they, if followed, ensure a high level of reliability and transparency, as well as enable a comparison across substances, test organisms, test designs, and endpoints. Conducting a standard study is costly and therefore mainly carried out by, or on the behalf of, the chemical producer or importer. Further, the REACH regulation requires the registrants to only submit (robust) study summaries (REACH Article 10 (a)(vi) and (vii)), however, the original study report is not required to be published and made available for third party evaluation. Studies that do not follow a standardised guideline, i.e. non-standard studies, may lack in reporting of methods and results or may use alternative test organisms, endpoints, and test designs. This may result in disagreements regarding their reliability and relevance, and thus usefulness for regulatory assessment [5,66]. However, for some chemicals, nonstandard studies have investigated and reported effects at lower concentrations than available standard studies, indicating that they may be more sensitive in terms of detecting specific effects. Examples of this include the endocrine active chemical bisphenol A [15,68,79] or the neurodevelopmental toxic flame retardant decabromodiphenyl ether (decaBDE, [23]). Further, REACH requires the registrants to apply the Klimisch method for evaluating the reliability of a study, hence, applying reliability criteria and concluding wheater a study is reliable without restriction (category 1), reliable with restrictions (category 2), not reliable (category 3), or not assignable (category 4) [63]. Studies fulfilling the two highest categories are normally accepted for regulatory use.
Consequently, the present study aimed to (1) characterise key studies used by the European Chemicals Agency's Committee for Risk Assessment (RAC) to restrict hazardous substances; (2) analyse if the REACH registration database provided the key studies used in these restrictions, and (3) investigate potential disagreements between experts related to the use of non-standard studies in the restrictions. The overall aim of this study is to better understand the use of scientific evidence in regulatory decision-making for the protection of human health and the environment.

Selection of REACH restrictions and key studies
First, the REACH restrictions were retrieved from ECHA's database Registry of restriction intentions until outcome (https:// echa. europa. eu/ regis try-of-restr iction-inten tions) on February 19th, 2021. We selected all restrictions in which RAC supported the proposed restriction in their opinion, engaged in the hazard assessment and discussion, i.e. in contrast to only referring to other expert groups' opinion (e.g. EFSA, IARC, US EPA) or other regulations (e.g. CLP), and provided and/or supported one or more key studies on a substance's (eco-) toxicological hazard(s) that served as the basis for the restriction. Eighteen RAC opinions were selected based on these criteria ( Fig. 1 and Additional file 1: Table S2; the de-selected cases, including justifications, can be found in Additional file 1: Table S1). It should be noted that the decisions on restrictions are made by the European Commission, but the RAC opinions can be considered as pre-decisions since they provide the basis for the final decision [42].
Key study for this analysis was defined as an (eco-)toxicity study that was used to calculate Derived No Effect Levels (DNEL), Derived Minimum Effect Levels (DMEL), Predicted No Effect Concentrations (PNEC), or another type of (eco)toxicological safety concentration, or considered most relevant by RAC for the justification of a restriction in their opinion's conclusion and provision of key elements underpinning the RAC conclusion(s) on a substance's hazard(s), if applicable. The term key study in this study is hence applied slightly differently than in the REACH registration dossiers. In the REACH registration database, the registrants categorise studies using a dropdown menu, into key, supportive, weight of evidence, or disregarded [22]. The same clarity could not be found in the RAC opinions.

Description of REACH restrictions and characterisation of key studies
The selected restrictions were described using the following information: • Dossier Submitter, i.e. the MSCA or ECHA, of the initial restriction proposal, and publication year of the RAC opinion. • Scope of the restriction. Information on the substance's main use(s) that was aimed to be restricted. • Hazard-based or risk-based assessment. For the hazard-based assessments, RAC evaluated and restricted the substance using the information in the hazard assessment only. This type of assessment is also referred to as the "generic approach to risk management" in the Chemicals Strategy for Sustainability by the European Commission [50]. For the risk-based assessments, RAC evaluated and supported a restriction using calculated risk characterisation ratios based on hazard safety levels and exposure concentrations. • Protection goal. Whether the restriction aimed to protect human health (HH), the environment (ENV), or a combination of the two. • Identified effect of concern. The (eco-)toxicological effect of concern. • Registered production/import volume. With increasing production/import volume of a substance comes increased test requirements. The extent of information available from the REACH registration database can influence the identification and management of uncontrolled risks. Information was collected from the registration dossiers, the Brief Profiles, or Substance Infocards [31]. For substance groups, a combined tonnage range of the group members was indicated, if possible.
In case the RAC opinion did not provide sufficient information for our analysis, the restriction dossier submitted by the Dossier Submitter, i.e. the Member State or ECHA, was consulted. Restriction dossiers are available from the same database as the RAC opinions (see above). Since documents varied in structure and format, any additional part of the RAC opinion or the restriction dossier that was relevant for the scope of the present study was read carefully, if necessary.
Overall, 53 key studies were identified in the selected RAC opinions (Additional file 1: Table S3). The key studies were characterised according to the following aspects: A study was categorised as standard if this was articulated by RAC, stated in the study's protocol, and/or if a protocol equivalent to a guideline was followed, i.e. studies that followed a standardised protocol and investigated one or more non-standard endpoints in addition to the mandatory ones, or followed a protocol that was not yet officially a guideline by the time of study performance. All other studies were categorised as non-standard studies. • Reliability evaluation. A study's assigned reliability category [63] was extracted from the RAC background document and restriction dossier (Annex XV dossier) which were carefully read and screened with the search terms Klimisch, reliable, and reliability to find relevant information. If no information was available in these documents, the REACH registration dossier of the respective substance was consulted, if available. • Accessibility. The key studies were categorised as published when they were accessible via online literature search. Studies that were not possible to get access to because they were not openly published were categorised as not accessible. • Funding source. In the light of distributing monetary resources for the generation of knowledge that is (possibly) used for the risk assessment and management of harmful substances, the information about the funding of a study was extracted from the study itself and categorised as government, industry, or academia. To avoid errors, ambiguous funding sources were researched online to enable categorisation. • Authors' affiliation. Information on who is producing the data that are used to manage uncontrolled risk can indicate if a regulation provides the data needed. Companies spend a noticeabe amount of money and effort to comply with the information require-ments for registration, but it is unknown if and to what extent those submitted data are used to manage uncontrolled risks by ECHA. The authors' affiliation was retrieved from the study itself and categorised as government, industry, or academia. Governmental agencies or local ministries were considered as government. Any affiliation to a chemicals producer, contract laboratory, or consulting company was marked as industry. Universities or independent research institutes were marked as academic. Unclear affiliations were researched online to enable categorisation. It was further analysed if the key studies used by RAC were already identified and included in the initial restriction proposal by the Dossier Submitter, or if RAC identified and suggested one or more different key studies themselves in their opinion.

Analysis of the REACH registration database and comparison to key studies used in the restrictions
The REACH registration database was accessed between February 20th, 2021 to June 11th, 2021 to identify the key studies used in registration dossiers. In cases where a group of substances was restricted, e.g. formaldehyde and formaldehyde releasers, the registration dossier of the leading component for which a hazard assessment was performed, i.e. formaldehyde in this example, was consulted. In cases where ECHA listed specific group members of a restriction, i.e. the four phthalates, each key study was compared to the respective registration dossier. For substances and substance groups for which no registration dossier was available, a comparison was not possible. When the key study used by RAC equaled the key study used in the REACH registration database it was categorised as same, and when it differed from the key study it was categorised as other. Key studies in the restrictions that were not included in the registration dossiers, were categorised as external.

Investigation of potential expert disagreements related to the use of non-standard studies in the restrictions
While standard studies usually are considered suitable for regulatory assessments by default, the criteria for evaluating non-standard studies are less clear and may open up a scope of different views on their suitability. We investigated if this was the case by examining (1) the RAC opinions, (2) the RAC meeting minutes, and (3) the document Comments on Annex XV report, including the documents therein. We searched these documents for the surnames of the first authors and the year for the nonstandard key studies. If the surnames contained non-English characters, alternative spelling was also used (e.g. o instead of ø). For scanned documents, that could not be searched automatically, a manual search was performed. It should be noted that the Comments on the Annex XV report only contain non-confidential comments from the public consultation.

Description of the restrictions
In total, the analysed restrictions were proposed by MSCAs from eight different EU nations and ECHA itself, where ECHA was involved most often in proposing a restriction (8/18), followed by France and Denmark (3/18, each). The RAC opinions on the restriction proposals of the 18 examined substances were published between 2011 and 2020 (Additional file 1: Table S2). In total, twelve restrictions (67%) aimed to protect human health and five restrictions (28%) the environment. One case (decaBDE) aimed at both human health and environmental protection. Risk-based restrictions primarily aimed to protect human health (9/10). Hazard-based restrictions aimed to protect the environment (4/5). Groups of substances were covered by half (9/18) of the analysed restrictions. The restriction on TDFAs in spray products in combination with organic solvents was the only restriction that addressed adverse mixture effects (lung injuries). Most substances were registered for hightonnage volumes up to 100 000 and 1 000 000 tonnes per year (tpa) ( Fig. 2; Additional file 1: Table S2). Methanol was registered for up to 100 000 000 tpa, while NPEOs, DIBP, and BBP had the lowest registered volume with up to 10 tpa.
For five analysed restrictions, chromium VI compounds, dimethyl fumarate, inorganic ammonium salts, intentionally added microplastics, and PAHs, no registration dossiers were available. Dimethyl fumarate is a biocide and thus not subject to registration under REACH. The restriction of inorganic ammonium salts covers a group of several substances but does not list individual group members. Chromium VI is formed during leather chrome tanning procedures where chromium III is used, and only a Substance Infocard is available. Microplastics are a group of different types of crude oil-based synthetic polymers and polymers are exempted from registration in REACH. The eight PAHs covered by the restriction have no registration dossiers since they are not intentionally produced but part of crude oil products. It should be noted that REACH restrictions are often considered as a 'safety net' for managing uncontrolled risks that have not been or cannot be managed by other REACH processes, such as substance evaluation or authorisation [32]. Further, in contrast to the registration requirements, a restriction does not require a minimum tonnage placed on the market or imported, and may cover polymers like microplastics.

Characterisation of the key studies used for restrictions
Overall, 53 key studies were identified and analysed (Additional file 1: Table S3). The analysed key studies were published between 1959 and 2016. The identified toxicological effects of concern for the restrictions covered lung injuries, irritation and sensitisation of the respiratory tract, eyes and skin, hepatic toxicity, as well as carcinogenicity, developmental (neuro-)toxicity, systemic toxicity, and toxicity to reproduction. Toxicity to the aquatic, sediment and soil environment, and endocrine disruption in fish were the identified ecotoxicological effects of concern (Additional file 1: Table S2). Four studies were human studies [7, 12, 62, 80] (Additional file 1: Table S3). Details on the endpoints considered most relevant by RAC are displayed in Additional file 1: Table S3.
The majority of key studies were published in scientific journals or made publicly available by the company (48/53, 91%) while five studies were not accessible (e.g. confidential business property, 9%; Table 1). All of the inaccessible studies were standard studies, and all four key studies used to restrict calcium cyanamide were not accessible (Additional file 1: Table S3). In total, 31 of the key studies were non-standard studies (58%; Table 1). Seven restrictions used only non-standard studies as key studies, another seven restrictions used only standard studies as key studies, and the remaining four used both standard and non-standard studies. For more details on the key studies, see supplementary information.
The majority of the studies had at least one author affiliated with academia (32/53, 60%), followed by industry (20/53, 38%), and governmental institutions (15/53, 28%) (Additional file 1: Table S3). Note that overlaps are possible since a study can have authors affiliated with several different sectors. For standard studies, 67% (14/21) of the studies had at least one author affiliated with the industry. In contrast, for non-standard studies, 77% (24/31) of the studies had at least one author affiliated with academia. Governmental institutions were identified as the main funding source (43%, 23/53), followed by industrial funds (23%, 12/53). The funding source was not specified in 17 of the 53 studies (32%). In 12 of these studies, authors were affiliated with academia, in four studies they were affiliated with the industry, and in six studies they were affiliated with government (again, overlaps are possible).
For 16 standard (16/21, 76%) and 12 non-standard studies (12/31, 39%), a Klimisch category for the reliability evaluation was proposed in the registration dossier, or by the Dossier Submitter in the restriction proposal. Most of the evaluated standard studies were assigned as reliable without restrictions (13/16, 81%). The rest were assigned as reliable with restrictions. Among the evaluated nonstandard studies, the majority (10/12, 83%) was assigned as reliable with restrictions, and only one was assigned as reliable without restrictions ([64], used to restrict the phthalate DBP). One non-standard study ( [81], used to restrict decaBDE) was assigned as not reliable by the registrant. In their opinion, RAC did not comment on the reliability evaluation provided in the registration database or by the Dossier Submitter. Instead, they provided arguments why a study was suitable for risk assessment. For a more detailed description see "Expert disagreements related to the use of non-standard studies in the restrictions" section.
In general, if a reliability evaluation was available in the REACH registration database, no indications were found that neither RAC nor the Dossier Submitter re-evaluated the study. If no reliability evaluation was available in the REACH registration database, the Dossier Submitter or RAC briefly described why a key study was considered of sufficient reliability, assigned Klimisch categories themselves, or did not comment on the reliability at all. The Dossier Submitter's evaluation was usually adopted by RAC without any further notice.
In approximately half of the analysed restrictions, RAC agreed with the Dossier Submitter's choice of key studies. However, in eight restrictions (NMP, calcium cyanamide, methanol, DMF, lead, microplastics, NP/NPEOs, bisphenol A), RAC replaced, added, or made a sub-selection of the proposed key studies. For details on the rationales behind RAC's decisions, see supplementary results to "Characterisation of the key studies used for restrictions" section.

Comparison between key studies submitted to the REACH registration database and the key studies used in the REACH restrictions
Fourteen of the 53 key studies (26%) underlying the REACH restrictions were also used as key studies in the REACH registration database, i.e. the registration dossiers ( Fig. 3; Additional file 1: Table S3). Most of them were standard studies (13/14) performed by authors affiliated with the industry (12/14). Three of the 53 key studies (6%) were used in the registration database but not as key studies, and 16 (30%) were consulted from sources other than the registration database. Fourteen of these 16 external studies were non-standard studies. For 12 of the external key studies, used to restrict NP and NPEOs in textiles, and lead in gunshot over wetland, the adverse effect served as the basis for the restriction (i.e. endocrine disruption in fish, and toxicity to birds; Additional file 1: Table S3), was not covered by the respective registration dossier. For 20 of the 53 key studies (38%) registration dossiers were not available for comparison (see "Description of the restrictions" section, Fig. 3). For decaBDE, restricted due to developmental neurotoxicity and PBT/vPvB properties, the non-standard study by Viberg et al. [81] was marked as disregarded in the registration dossier and assigned as not reliable. However, the same study was used as key study in the restriction dossier. The registrant argued that Viberg et al. [81] was "not suitable for use in human risk assessment" because of an invalid experimental design [38]. It was not clear if the study was included in the registration dossier before or after the regulatory discussions had taken place.
In the case of the four phthalates, restricted due to toxicity to reproduction, the non-standard study by Lee et al. ([64], assessed as reliable without restrictions by the registrant) was added as the key study to the registration dossier of DBP only after the first restriction proposal was published in 2011 (see endpoint summary of toxicity to reproduction of DBP; [43]). RAC expressed support for the Lowest Observed Adverse Effect Level (LOAEL) of 2 mg/kg bw/day from the Lee study also in the new restriction proposal in 2017 [28]. The standard study by Aso et al. [8] was used as additional information in the registration dossier for the endpoint developmental toxicity [44]. The non-standard study by Ahmad et al.
[4], used to restrict BBP, was not included in the registration dossier of BBP. The standard studies by Tyl et al. [78] and Nagao et al. [69], also used to substantiate the restriction of BBP, were marked as key to the registration dossier and assessed as reliable without restrictions.
For formaldehyde, restricted due to carcinogenicity, the non-standard study by Rusch et al. [77] was included in the registration dossier as additional information instead of the key study. It was assigned as reliable with restrictions with the remark "study well documented, meets generally accepted scientific principles, acceptable for assessment" [39].
TDFAs were restricted due to lung injuries when used in combination with organic solvents in spray products for consumer products. A registration dossier was only available for triethoxy silane (CAS 51851-37-7). However, no study on inhalation effects in combination with organic solvents was submitted. Instead, an oral standard study testing the substance alone was submitted and used by the registrant to extrapolate for inhalation effects for workers and the general population. The extrapolated DNEL of 0.29 mg/m 3 was in a similar concentration range as the calculated DNEL of 0.21 mg/m 3 by RAC using the non-standard study by Nørgaard et al. [26,40,70].

Expert disagreements related to the use of non-standard studies in the restrictions
While there were no reports in the RAC meeting minutes of discussions about the overall choice of a non-standard key study, five of the non-standard key studies were commented on during the public consultation (5/31, 16%). Raised objections by third parties were responded to by the Dossier Submitter and RAC, and were summarised by RAC in the final opinion document for four of these five studies.
The study by Kilo et al. [62] was used as the key study to restrict the solvent DMF (N,N-dimethylformamide) due to hepatotoxic effects and the cause of alcohol intolerance. Based on epidemiological data from workers, the researcher concluded that "long-term exposure to DMF […] does not result in any adverse liver effects", but the investigated DMF levels "still elicit certain alcohol intolerance reactions" [62]. Although Kilo et al. [62] is part of the registration dossier (assessed as reliable with restrictions, "study well documented, meets generally accepted scientific principles, acceptable for assessment" [41]), industry representatives questioned its use as a key study in the assessment used for restriction. The study was considered "not useful" for DNEL derivation due to drawbacks in method quality, and using alcohol intolerance as the key endpoint for DNEL calculation was argued not be appropriate because of individual liver enzyme activities and drinking behaviour among populations (e.g. differences in European and Asian populations). Based on a provided meta-analysis, it was further claimed that the current long-term inhalation DNEL (occupational exposure limits) of 15 mg/m 3 is not related to risks of liver harm [46,53,54]. RAC acknowledged the deficiencies in the adequacy of the study as well as the Lowest Observed Adverse Effect Concentration (LOAEC) of ≥ 20 mg/ m 3 from the meta-analysis but encountered difficulties in setting a No Observed Adverse Effect Concentration (NOAEC) based on the provided data. They decided on a DNEL for hepatic effects in humans based on Kilo et al. [62] of 6 mg/m 3 , arguing that this value is supported by a non-human animal study that observed developmental toxicity after inhalation [35,51]. The study by Hellwig et al. [51] is a standard study assessed as reliable without restrictions that was used as a key study as well. Those criticising the use of Kilo et al. [62] are, as far as we can see, not the registrants since none of the listed members of the Industrievereinigung Chemiefaster e.V. [55] are listed as registrants in the registration dossier of DMF [41].
The developmental neurotoxicity study by Viberg et al. [81] was used as the key study for the restriction of decaBDE. This study was already at the centre of several regulatory discussions (e.g. EU RAR, US EPA, Environment Canada). During the public consultation, an anonymous company commented that the study by Viberg et al. [81] suffers from deficiencies in experimental design and statistical analysis, causing an increase in false-positive results. Further, they argue that the Dossier Submitter should use two standard studies by Biesemeier et al. [17,18], that were performed to address the findings reported in Viberg et al. [81] and other studies, as key evidence [6]. Biesemeier et al. [17,18] conclude that decaBDE is not a developmental neurotoxicant. The Dossier Submitter replied by listing the limitations they and other agencies had identified in Biesemeier et al. [17,18], and listing the studies and risk assessments supporting the use of Viberg et al. [81] as key evidence. RAC supported the Dossier Submitter in their reply and the final opinion, referring to other studies supporting the findings of Viberg et al. [81] and other European agencies' decisions [23].
The four phthalates were identified as endocrine disruptors and the study by Lee et al. [64] was used as the key study for DBP. This study was not included in the registration dossier of DBP. During the public consultation, the European Council for Plasticisers and Intermediates (ECPI), part of the European Chemical Industry Council (Cefic), argued that the inclusion of the magnitudes lower DNEL of DBP into the overall risk assessment and combined Risk Characterisation Ratios calculations is a conservative judgement that "needs to be acknowledged and justified and possibly even refined. " They claim that the observations in Lee et al. [64] "are inconsistent with the larger body of literature on DBP questioning the reliability of this study as the key study". Further, investigated effects "were seen at low incidence" and analysed anti-androgenic endpoints are possibly "not relevant to the common MOA" (Mode of Action), and "only statistically significant at the highest dose tested" [45]. The Dossier Submitter replied"The commenter accuses the Dossier Submitter of obscuring and misreporting of data in the restriction proposal. The Dossier Submitter rejects these allegations and considers them baseless…", and continues "In contrast with the view of the commenter, the Dossier Submitter considers it reasonable to regard the observed mammary gland effects as anti-androgenic". They referred to the restriction dossier with a presentation of an OECD study and further scientific studies that support the anti-androgenic mechanism of action of DBP in male mammary gland as observed in Lee et al. [64], and added that "The commenter did not provide any evidence on the contrary" [27]. RAC supported the Dossier Submitter in their reply and did not address the discussion in their final opinion [28].
The study by Bennett et al. [12] was used to restrict methanol because it documents severe poisoning incidents in humans. The Methanol REACH Consortium, an industry expert group on behalf of ERM Consulting (https:// www. reach centr um. eu/ about/), argued that Bennett et al. [12] misinterpreted their own results, i.e. that "the death was a result of ethanol poisoning that was misdiagnosed as methanol poisoning", and that "the reported blood methanol concentrations of the victims are inconsistent with the reported exposures" [73]. RAC commented that nonetheless clear methanol poisoning symptoms were observed and that "[a]bsence of detectable methanol in the blood does not exclude methanol poisoning, since there was a 12-h latent period before the symptoms occurred, during which methanol was metabolised to formic acid (formic acid was not measured). Ethanol ingestion that led to high blood ethanol level could occur several hours after methanol ingestion, when formic acid was already formed, and could be a contributing factor to methanol toxicity (e.g. contributing to acidosis and CNS depression" [24]. RAC summarised the discussion in their opinion and the meeting minutes, and decided to use a minimum methanol oral dose for severe ocular toxicity of 0.26 g/kg bw based on Bennett et al. [12] instead of 0.66 g/kg bw as suggested by the Methanol REACH Consortium. Further, RAC disagreed with the Dossier Submitter's choice of the endpoint. The Dossier Submitter used the minimum acute oral lethal dose of 0.3 g/kg bw, but RAC considered severe ocular toxicity as the most critical non-lethal adverse effect [25]. In the restriction on lead used in gunshots over wetlands, only the key study by Bellrose [11] was discussed during RAC meetings and commented on during the public consultation. Bellrose et al. (1959) estimated the annual mortality of waterfowl caused by the ingestion of lead, supported by experimental data. During the public consultation, an individual commenter presented a recalculation of the data from Bellrose et al. (1953) using "more modern statistical techniques" and concluded that the results by Bellrose et al. (1953) are "likely to be reliable" [29]. The study providing the re-calculations, however, could not be found in the open literature. RAC initially questioned the reliability of the results presented by Bellrose et al. [11] but shifted to support them after the re-calculation [30,33].

Discussion
Science is the basis for risk assessments and the subsequent risk management measures in chemical regulation. Still, only a few scientific studies on the role of science in decision-making in this area have been performed. From an EU perspective, the possibility to restrict the use of chemicals that give rise to unacceptable risks is a key tool for the protection of human health and the environment. From this study, the following main findings will be discussed:

Non-standard studies contribute to the risk management under REACH restrictions
An important difference between standard and nonstandard test methods is that non-standard (eco)toxicity studies can incorporate alternative endpoints, test species, and/or test designs, which in some cases may be more sensitive to detect a relevant effect. A more sensitive study results in a lower effect value and is hence more protective when used as a key study in decisionmaking. One example of this is the rapid test development within research on endocrine disruption, and subsequent use of these studies in risk assessments [13,52]. Still, this use has not been straightforward. The use of non-standard endpoints has been subject to lengthy discussions and disagreements among stakeholders. For example, the RAC-supported study by Viberg et al. [81] investigated neurodevelopmental effects and, in contrast to the less sensitive standard studies, observed abnormal behaviour in adult mice exposed to decaBDE during their neonatal life. Similarly, the endocrine-disrupting effects of bisphenol A were detected at lower exposure levels in non-standard studies that tested, e.g. reduced sperm production or developmental neurotoxicity, compared to standard studies [14,15]. However, quantifying more sensitive endpoints in a standardised way can be challenging due to the physiological complexity of an organism, in particular hormonal interactions, neurodevelopment, and behaviour [1, 19,61], or the test design itself (e.g. omics approaches). This can have implications for the reproducibility of an endpoint, and in return implications for a study's reliability. Using potentially less sensitive but feasible endpoints can be considered a compromise when identifying and managing hazardous chemicals. Besides the importance for regulatory decisions, nonstandard studies can contribute to the development of guidelines for standard studies. One example of this is the vitellogenin (VTG) endpoint used for screening of endocrine active substances in fish in the OECD test guidelines 230 and 229. Both test guidelines acknowledge that the experimental design is not capable of distinguishing direct endocrine effects from hepatotoxic or other non-endocrine effects. This is due to VTG production being stimulated in the liver by endogenous oestrogen in female oviparous vertebrates [71,72]. Also here, nonstandard studies are ongoing to address the problem, e.g. screening for potential endocrine biomarkers through analysis of gene expression profiles [10,74]. Another role of non-standardised research is to identify new or additional types of hazards, i.e. those not required for testing by the REACH regulation. From this study, examples include the identification of endocrine disruptors to fish as a key endpoint in the restrictions of NPEOs in textiles or the injuries on the lung from TDFAs in spray products in mixture with organic solvents.
Where stated, governmental and/or industrial institutions were the main funding sources of the characterised key studies. Most of the non-standard studies' authors were affiliated with academia (77%), and most of the standard studies' authors were affiliated with industry (67%). This is not surprising since chemical producers and importers usually generate standardised data for the registration of the chemicals they produce or import, and governmental institutions are funding research projects to generate new knowledge on the hazards and risks of chemicals. Based on these results, it can be hypothesised that academic researchers contribute reliable and relevant information to the risk management (restriction) of harmful substances under REACH.
Almost half of the analysed RAC opinions (7/18) relied on non-standard key studies only, and non-standard studies comprised 58% of the 53 analysed key studies in the REACH restrictions, of which the authors were mainly associated with academia, as expected. Similar numbers for the REACH registration database are lacking but since the majority of key studies used in the restrictions that were not included in the registration database were non-standard studies, we hypothesise that the number of non-standard studies in the REACH registration database is lower compared to REACH restrictions. This hypothesis is also supported by an analysis of the test requirements in the REACH guidance documents where internationally recognised guidelines are recommended [20,21]. The incentives to include nonstandard studies with low effect values in REACH registration dossiers may also be low for the registrants. A study by Tarazona et al. [76] showed that, depending on the endpoint, in 22% to 36% of the analysed ecotoxicity studies in the REACH registration database no information on the experimental methods or potential standard testing equivalent methods were provided. These results cannot be directly compared to the present study since the present study is limited to key studies, but it provides a guiding number. Further research could analyse the whole set of studies included in a restriction dossier/RAC opinion, the evaluation thereof, and the decision-making behind the selection of a key study.
In the new Chemical Strategy for Sustainability, the European Commission declared to strengthen the use of academic studies [50]. Ensuring that (eco)-toxicity studies are evaluated according to the same reliability and relevance criteria regardless of study type would be one important action [67]. Improving the reporting of nonstandard studies is also imperative for their improved regulatory use since only a rigorous description of e.g. experimental and statistical methods, chemical analysis, and results enable an evaluation of a study's reliability and relevance [3,58]. The GLP system was therefore invented for studies used for regulatory purposes, which is often implemented when performing standard studies. Since non-standard studies do not follow a standardised protocol or systematic study documentation according to GLP, their reliability (and relevance) evaluation tends to be more time-demanding.
Overall, there were few reported expert disagreements related to the applicability of non-standard studies in the studied RAC opinions. Among the analysed key studies, industry representatives challenged, unsuccessfully, the use of non-standard studies for decaBDE, DBP, DMF, and methanol claiming low reliability and/or relevance. Arguments addressing the reliability or the relevance of a study were often not clearly separated from each other by RAC or during public consultation. In decisionmaking, experts may use different evidence because the protection goal of the risk assessment differs, they may have different access to data (partly time-dependent), they can evaluate the reliability or relevance in different ways, and/or interpret data differently [15,75]. Further, a higher number of key studies was used to restrict substances of environmental concerns than of human health concerns. Three of the five restrictions addressing environmental concerns had a higher number of key studies, which were mostly non-standard studies. The highest number of key studies was used in the RAC opinions on lead used in gunshot over wetland due to bird toxicity (six), NP and NPEOs in textiles due to endocrine disruption in fish (six), and intentionally added microplastics to articles due to aquatic toxicity and persistency (15). In comparison, only one out of twelve restrictions addressing human health concerns (i.e. the restriction of phthalates) had a comparable amount of key studies (six) which were mainly non-standard studies, too (Additional file 1: Table S3). This suggests that, first, restricting substances due to environmental concerns might require more evidence than for human health, and second, that non-standard studies could be considered important for identifying hazards and risks of emerging pollutants (e.g. microplastics), on wildlife (e.g. birds), and effects that are not systematically screened for under REACH (e.g. endocrine disruptors). Further studies could establish whether this is an actual pattern.

The REACH registration database does not contain all data relevant to the restriction of hazardous chemicals
Approximately one-third of all key studies in the restrictions did not originate from the REACH registration database. The hazards that were addressed by these key studies were lung toxicity of a chemical mixture (TDFAs in spray products containing organic solvent), endocrine disruption in fish and rats (NP and NPEOs, DBP), ocular toxicity (methanol), and bird mortality (lead). Under REACH, it is not systematically screened for endocrine effects, nor does it require any testing on mixtures. Further, for most of the external key studies (12/16), no information was provided in the REACH registration database for the adverse effect that served as the basis for the restriction. Those were exclusively ecotoxicological effects (endocrine disruption in fish, toxicity to birds) which imply that the REACH registration database provides incomplete data to adequately protect the environment from hazardous substances. This is supported by the findings that on average 61% of the information submitted to the REACH registration database for substances produced or imported into the EU at or above 1000 tonnes per year on ecotoxicological endpoints was "non-compliant" [16]. Further, in the restrictions of methanol, NP and NPEOs, and lead, RAC added external key studies to the ones already suggested by the Dossier Submitter since no adequate information was available from the REACH registration database. However, for the restrictions of DMF and bisphenol A, RAC consulted key studies from the REACH registration database that were not used by the Dossier Submitter.
The extent to which external key studies are needed to restrict harmful substances may also depend on the production/import volume of a specific substance since the required hazard information increases with higher tonnage produced/imported, i.e. it could be hypothesised that external studies are used more frequently to restrict low-tonnage substances. External key studies were used to restrict two low-tonnage substances (TDFAs, NP and NPEOs), two high-tonnage substances (methanol, lead), and one group of substances with a large range of 1 to 100 000 tpa (Fig. 2). Similarly, it could be hypothesised that non-standard studies are used more frequently to restrict low-tonnage substances and substances not registered in REACH when less or no information is available in the registration database. Three low-tonnage substances were restricted, of which non-standard studies were used to restrict TDFAs and decaBDE, while NP and NPEOs were restricted using a mix of standard and (mainly) non-standard studies (Additional file 1: Table S3). Standard studies were used to restrict two non-registered substances (chromium VI, DMFu), and non-standard studies were used to restrict another two non-registered substances (inorganic ammonium salts, PAHs). A mix of standard and (mainly) non-standard studies was used to restrict the non-registered microplastics (Additional file 1: Table S3). Thus, the current data do not allow for an interpretation of clear patterns. An in-depth analysis of the full dataset of the restrictions could address the relationship between the production/ import volume of a substance and the type of data used for restriction.
In REACH registrations, applicants are required to make use of all available studies of sufficient reliability and relevance. This means that applicants should search the peer-reviewed literature [21]. If and how this search is performed is not publicly reported, which makes it impossible to know if the difference between the use of key studies in restrictions and registrations is due to a difference in the evaluation of reliability and relevance of the studies, or if studies were considered at all. Regardless, it can be concluded that the REACH registration database does not contain all studies applicable for the assessment of a specific substance. This may have consequences for human health and the environment since the identification of hazards are fundamental for risk management measures such as restrictions.

The transparency of REACH restrictions is partially hampered by the use of key studies that are inaccessible to third parties
The vast majority (91%) of the analysed key studies used to restrict uncontrolled risks under REACH were accessible, i.e. they were published. In terms of transparency, this is commendable. Nonetheless, there was a partial lack of transparency due to the use of inaccessible studies. In the present analysis, five of the 53 key studies were inaccessible (9%), which were all standard studies and mainly used to restrict the fertiliser calcium cyanamide (4/5) due to environmental concerns. It remains unknown if the use of inaccessible data negatively influenced the risk management decision by RAC to sufficiently protect human health and the environment from chemical risk. A lack of transparency for the general public may not be recognised by those who have direct access to data, or can easily request them. The new EU Transparency Regulation that entered into force in March 2021 requires full disclosure of studies and any scientific data that are used for risk assessment related to food and feed safety [49]. According to the Chemical Strategy for Sustainability, the European Commission aims to extend this to other pieces of chemical legislation [50]. Hence, the REACH regulation could benefit from that since transparency enhances the credibility and acceptance of regulatory procedures and decision-making overall [9,60,65].
There was also a lack of transparency and clearness in how the RAC opinions were written, for example, identifying key studies was challenging due to the lack of explicit expression of such, especially with respect to environmental concerns, as well as a lack of clarity of reliability and relevance evaluations, and discussions about the adequacy of studies. The REACH restrictions process would gain more credibility and user-friendliness from improved structure, clearness, and transparency on these aspects.
It needs to be noted that the analysis of key studies used by RAC to restrict substances under REACH in the present investigation is limited to the selected 18 cases out of the 34 RAC opinions (Fig. 1). In these 18 cases, the selection of key studies was made within the REACH restriction process. To receive a broader view on the choice of scientific studies in the decision-making for chemicals, further research could address the use and characterisation of key studies within other risk managing procedures under the REACH regulation as well as within other regulations, such as the CLP, plant protection product, biocidal, or cosmetic product regulations.

Conclusions
Regulatory risk assessment is complex and an efficient restriction system needs to be flexible enough to handle different types of substances or substance groups, as well as different types and amounts of data. In the current study, the use of key studies for 18 REACH restrictions was analysed. Non-standard studies comprised 58% of the studies, and thus contributed to the risk management under REACH. Most of the studies consulted from registration dossiers to inform the restriction procedure were standard studies which is not surprising considering that standard studies are usually submitted for substance registration to comply with the regulatory requirements. Bearing in mind that all inaccessible key studies (9%) were standard studies presents a need for enhanced transparency, similar to the new EU Transparency Regulation for food laws where all the submitted data are now publicly available. Sixteen (30%) of all key studies in the REACH restrictions were not consulted from the REACH registration database which implies that the registration data are not sufficient for proper identification of hazardous substances and uncontrolled risks for these substances. Almost all of the external key studies, i.e. studies not included in the REACH registration database, were non-standard studies. Regarding their applicability for a restriction, experts rarely disagreed in the analysed cases. In cases where experts disagreed, RAC supported the choice of a non-standard key study in the restriction proposal.
Based on the results from this study, we recommend that registrants strive toward making their registration data publicly available, conducting the update of their registration dossiers systematically and regularly, and including reliable and relevant non-standard data in their registration dossiers. For RAC, ECHA and the MSCAs, we recommend striving towards a transparent and structured reporting of evidence and justifications for the selection of key studies to support decision-making. We understand that there are limited resources, but we also recommend actively engaging in the identification of uncontrolled risks that goes beyond the usage of REACH registration data, and initiating dialogue with researchers on the regulatory use of their studies. Further, we recommend the European Commission to promote the use of all available evidence in the assessment and management of chemicals by: • developing guidance for regulators and registrants on how to report the screening for "all available evidence" that should be performed when registering substances to the REACH registration database, • demanding a frequent, traceable, and published update of registration dossiers to ensure that the most recent knowledge of substances is available to registrants and authorities for risk identification and management, • revising the guidance on how to evaluate the reliability and relevance of (key) studies for use in the REACH registration database as well as the REACH restrictions, • together with scientific journals, developing and implementing reporting criteria for (eco)toxicity studies that promote reliable and reproducible studies that apply to the regulatory requirements, • ensuring that EU-funded projects report studies that comply with regulatory requirements, and • promoting unrestricted access to data used as the basis for risk assessment and management decisions.