• All opinions expressed are those of the presenter and do not represent any organization.

• Some slides need to be scrolled down to see the full content.

• FDA: Statistical Software Clarifying Statement

“FDA does not require use of any specific software for statistical analyses, and statistical software is not explicitly discussed in Title 21 of the Code of Federal Regulations [e.g., in 21CFR part 11]. However, the software package(s) used for statistical analyses should be fully documented in the submission, including version and build identification.”

As an organization, we need to ensure compliance and reduce the risk of using R and R packages in regulatory deliverables.

• R is widely used in clinical trial study design.
  • Following Lachin and Foulkes (1986)
  • gsDesign: an R package for group sequential design under proportional hazards.
  • simtrial, gsDesign2, and gsdmvn: experimental R packages for group sequential design under non-proportional hazards
  • Bookdown: https://keaven.github.io/gsd-deming/
• R is flexible for handling novel missing data approaches following the estimand framework in ICH E9 (R1)
  • Recurrent event data: Gao et al. (2017) Diao et al. (2020)
  • Robustness: Liu et al. (2021)
• R is used in Bayesian statistics.
  • stan and network meta-analysis for drug reimbursement analysis
• R is widely used in visualization
  • SafetyGraphics
  • forestly: Interactive forest plot for DMC safety monitoring in clinical trials

• Clinical study report is a key deliverable for clinical trials to regulatory agencies. (e.g., FDA, CFDA)
  • ICH E3 Structure and Content of Clinical Study Reports.
• We try to fill in gaps to streamline workflow using R for clinical trial development:
  • To develop, validate, and deliver analysis results.
  • To submit analysis results to regulatory agencies in eCTD format.
• Focus on table, listing, figure (TLFs) delivered in RTF/Microsoft Word format.
  • In the pharmaceutical industry, RTF/Microsoft Word play a central role in preparing clinical study reports.
  • Different organizations can have different table standards.

• The R submission working group is a cross-industry pharma working group focusing on improving practices of R-based clinical trial regulatory submission.
  • members and affiliations
• Mission: Easier R-based clinical trial regulatory submissions
• pilot 1 - common analyses
  • 2021.11 Initial submission to FDA.
  • 2022.12 Received FDA response
  • 2022.02 Revised submission to FDA
  • 2022.03 Received final FDA response.

Challenges

• How to submit internally developed (proprietary) R packages?
• How to follow ICH and/or FDA guidances in preparing eCTD package?
• How to enhance reproducibility from FDA reviewer’s perspective?

Assumptions

• Focus on analysis and reporting given available ADaM datasets.

• Open source pilot eCTD submission package to FDA
  • Proprietary R package
  • R scripts for analysis
  • Analysis data reviewer guide (ADRG)
  • etc.
• Development: https://github.com/RConsortium/submissions-pilot1
• eCTD package: https://github.com/RConsortium/submissions-pilot1-to-fda
• R consortium announcement

Initial Submission

• “Using R version 4.1.1, FDA was able to run the submitted code and confirm the applicant’s tables and the submitted figure in report-tlf pdf file.”
• “Using FDA developed code, a statistical analyst was able to independently generate tables using the submitted data.”
• Initial submission FDA response

Revised Submission

• FDA agrees that the initial phase of the R Pilot submission has been completed.
• Revised submission FDA response

• Pilot R Shiny app submission to FDA.
• Seek pilot submission opportunities to other regulator agencies.
• Pilot submission with advanced analysis methods (e.g., study design, missing data, Bayesian, etc).

• FDA Study Data Technical Conformance Guide:

“Sponsors should provide the software programs used to create all ADaM datasets and generate tables and figures associated with primary and secondary efficacy analyses. Furthermore, sponsors should submit software programs used to generate additional information included in Section 14 CLINICAL STUDIES of the Prescribing Information (PI)26 if applicable. The specific software utilized should be specified in the ADRG. The main purpose of requesting the submission of these programs is to understand the process by which the variables for the respective analyses were created and to confirm the analysis algorithms. Sponsors should submit software programs in ASCII text format; however, executable file extensions should not be used.”

Although FDA did not expect submitted R code is executable, sponsor shall enhance reproducibility.

• Fixed R version: e.g., R 4.1.0
• Fixed R package snapshot date: e.g., 2021-08-31
  • RStudio Public Package Manager
  • CRAN Time Machine by MRAN
• Flexibility of input and output path:
  • Define path as parameter.
• eCTD deliverables dry-run in Windows environment
• Provide steps to follow in ADRG appendix

We share the same philosophy described in Section 1.1 of R Packages book and quote here.

• “Anything that can be automated, should be automated.”
• “Do as little as possible by hand. Do as much as possible with functions.”
• “The goal is to spend your time thinking about what you want to do rather than thinking about the minutiae of package structure.”

Tools:

• r2rtf: create production-ready tables and figures in RTF format.
• pkglite: represent and exchange R package source code as text files.
• cleanslate (under internal validation): create portable R environments.

Bookdown: https://r4csr.org/