In this workshop, we have three parts:

• Delivering TLFs in CSR
  • Session 1 (45 min)
  • Break and/or Exercise (20 min)
  • Session 2 (45 min)
  • Break (10 min)
• Clinical trial project
  • Session 3 (30 min)
  • Break (10 min)
• eCTD submission package
  • Session 4 (40 min)
  • Q&A (10 min)

• All opinions expressed are those of the presenter.

• Some slides need to be scrolled down to see the full content.

• Learning how to create TLFs for clinical study reports.
• Learning how to organize a clinical development project.
• Learning how to prepare eCTD submission package to FDA.

The toolchain, process, and formats may be different in different organizations. We only provide a recommended way to address them.

• Contributors of the training materials.
  • Please consider submitting issues or PR in the github repo.
• R Consortium
  • R validation hub
  • Submission working group
  • R Tables for Regulatory Submission (RTRS) working group
  • Please consider to join and contribute to a working group.

In this workshop, we assume you have some R programming experience and clinical development knowledge.

• Data manipulation: tidyverse, dplyr, tidyr, ggplot2 etc.
• ADaM data: adsl, adae etc.

install.packages(c(
  "tidyverse", # Data manipulation
  "r2rtf",     # TLF generation
  "pkglite"    # eCTD submission
))

• Training Material: https://r4csr.org/
• Demo project:
  • Analysis project: https://github.com/elong0527/esubdemo
  • eCTD submission package: https://github.com/elong0527/ectddemo
• R Consortium FDA submission pilot:
  • https://github.com/RConsortium/submissions-pilot1
• During the workshop, we will use esubdemo project.
  • esubdemo is shared in Posit Cloud (https://posit.cloud/)
  • You can also clone the esubdemo project to your local RStudio IDE.

As an organization, we need to ensure compliance and reduce the risk of using R and R packages in regulatory deliverables.

• R is widely used in clinical trial study design.
  • Following Lachin and Foulkes (1986)
  • gsDesign: an R package for group sequential design under proportional hazards.
  • simtrial, gsDesign2, and gsdmvn: experimental R packages for group sequential design under non-proportional hazards
  • Bookdown: https://keaven.github.io/gsd-deming/
• R is flexible for handling novel missing data approaches following the estimand framework in ICH E9 (R1)
  • Recurrent event data: Gao et al. (2017) Diao et al. (2020)
  • Robustness: Liu et al. (2021)
• R is used in Bayesian statistics.
  • stan and network meta-analysis for drug reimbursement analysis
• R is widely used in visualization
  • metalite.table1: Interactive demographic table.
  • forestly: Interactive forest plot for DMC safety monitoring in clinical trials

We share the same philosophy described in Section 1.1 of the R Packages book and quote here.

• “Anything that can be automated, should be automated.”
• “Do as little as possible by hand. Do as much as possible with functions.”
• “The goal is to spend your time thinking about what you want to do rather than thinking about the minutiae of package structure.”

The ICH E3: structure and content of clinical study reports provides guidance to assist sponsors in the development of a CSR.

In a CSR, most of TLFs are located in

• Section 10: Study patients
• Section 11: Efficacy evaluation
• Section 12: Safety evaluation
• Section 14: Tables, Figures and Graphs referred to but not included in the text
• Section 16: Appendices

• Public available CDISC pilot study data.

• The dataset structure follows the CDISC Analysis Data Model (ADaM).

• tidyverse: a collection of R packages to simplify the workflow to manipulate, visualize and analyze data in R.

• r2rtf: an R package to create production-ready tables and figures in RTF format.

In the pharmaceutical industry, RTF/Microsoft Word play a central role in preparing clinical study reports

• ICH E3 Structure and Content of Clinical Study Reports

Different organizations can have different table standards

• E.g., Table layout, Font size, Border type, Footnote, Data source

• r2rtf is an R package to create production-ready tables and figures in RTF format.

r2rtf is designed to:

• provide simple “verb” functions that correspond to each component of a table, to help you translate a data frame to a table in RTF format;
• enable pipes (%>%);
• only focus on the table format.
  • Data manipulation and analysis should be handled by other R packages. (e.g., tidyverse)

Before creating an RTF table, we need to:

• Figure out table layout.

• Split the layout into small tasks in the form of a computer program.

• Execute the program.

r2rtf is designed to enable pipes (%>%).

head(iris) %>%
  rtf_body() %>%           # Step 1 Add table attributes
  rtf_encode() %>%         # Step 2 Convert attributes to RTF encode
  write_rtf("minimal.rtf") # Step 3 Write to a .rtf file

r2rtf package provides the flexibility to customize table appearance for

• Table component: title, column header, footnote, etc.
• Table cell style: size, border type, color, font size, text color, alignment, etc.
• Flexible control: the specification of the cell style can be row or column vectorized.
• Complicated format: pagination, section grouping, multiple table concatenations, etc.

r2rtf package also provides the flexibility to convert figures in RTF format.

r2rtf provides simple “verb” functions that correspond to each component of a table, to help you translate data frame to tables in RTF file.

https://r4csr.org/tlf-disposition.html

https://r4csr.org/tlf-population.html

https://r4csr.org/tlf-baseline.html

https://r4csr.org/tlf-efficacy-ancova.html

https://r4csr.org/tlf-ae-summary.html

https://r4csr.org/tlf-ae-specific.html

https://r4csr.org/tlf-assemble.html

In a late-stage clinical trial, the number of A&R deliverables can easily be in the hundreds.

For an organization, it is also common to have multiple ongoing clinical trials in a clinical program.

In this part, let’s consider how to organize a clinical trial project as an A&R lead.

Our primary focus is creating a standard R package structure to organize the project, with 4 goals in mind:

• Consistency
• Reproducibility
• Automation
• Compliance

https://r4csr.org/project-folder.html

• Setting up for success
  • Teamwork
  • Architecture
  • Boundaries
  • Community
• The SDLC
  • Planning
  • Development
  • Validation
  • Operation

https://r4csr.org/project-management.html

• FDA: Statistical Software Clarifying Statement

“FDA does not require use of any specific software for statistical analyses, and statistical software is not explicitly discussed in Title 21 of the Code of Federal Regulations [e.g., in 21CFR part 11]. However, the software package(s) used for statistical analyses should be fully documented in the submission, including version and build identification.”

• The R submission working group is a cross-industry pharma working group focusing on improving practices of R-based clinical trial regulatory submission.
  • members and affiliations
• Mission: Easier R-based clinical trial regulatory submissions
• pilot 1 - common analyses
  • 2021.11 Initial submission to FDA.
  • 2022.12 Received FDA response
  • 2022.02 Revised submission to FDA
  • 2022.03 Received final FDA response.

Challenges

• How to submit internally developed (proprietary) R packages?
• How to follow ICH and/or FDA guidances in preparing eCTD package?
• How to enhance reproducibility from FDA reviewer’s perspective?

Assumptions

• Focus on analysis and reporting given available ADaM datasets.

• Open source pilot eCTD submission package to FDA
  • Proprietary R package
  • R scripts for analysis
  • Analysis data reviewer guide (ADRG)
  • etc.
• Development: https://github.com/RConsortium/submissions-pilot1
• eCTD package: https://github.com/RConsortium/submissions-pilot1-to-fda
• R consortium announcement

Initial Submission

• “Using R version 4.1.1, FDA was able to run the submitted code and confirm the applicant’s tables and the submitted figure in report-tlf pdf file.”
• “Using FDA developed code, a statistical analyst was able to independently generate tables using the submitted data.”
• Initial submission FDA response

Revised Submission

• FDA agrees that the initial phase of the R Pilot submission has been completed.
• Revised submission FDA response

• Pilot R Shiny app submission to FDA.
• Seek pilot submission opportunities to other regulator agencies.
• Pilot submission with advanced analysis methods (e.g., study design, missing data, Bayesian, etc).

We will discuss strategies to prepare proprietary R packages and analysis code into proper formats for submission:

• The whole game
• Practical considerations
• Prepare R packages using pkglite
• Prepare analysis programs
• Update ADRG and ARM

https://r4csr.org/submission-package.html

We will also give recommendations to make the R code running environment reproducible for dry run tests and reviews using cleanslate.

https://r4csr.org/submission-environment.html

• To provide a tool for packing and restoring R packages as plaintext assets that are easy to store, transfer, and review.
• To provide a grammar for specifying the file packing scope that is functional, precise, and extendable.
• To provide a standard for exchanging the packed asset that is unambiguous, human-friendly, and machine-readable.

library("pkglite")

"/path/to/pkg/" %>%
  collate(file_ectd()) %>%
  pack()

pack(
  "/path/to/pkg1/" %>% collate(file_ectd()),
  "/path/to/pkg2/" %>% collate(file_ectd()),
  output = "/path/to/pkglite.txt"
)

"/path/to/pkglite.txt" %>%
  unpack(output = "/path/to/output/")

File specifications offer flexibility and brevity in specifying the files to include.

File collections contain the evaluation results of file specifications for packing.

library("pkglite")

"/path/to/pkg" %>%
  collate(file_root_core(), file_r(), file_auto("inst/"))

pkglite.txt follows the standard Debian Control File (DCF) format used by Debian, R, and RStudio IDE, to be both machine-readable and human-readable.

# Generated by pkglite: do not edit by hand
# Use pkglite::unpack() to restore the packages

Package: pkg1
File: DESCRIPTION
Format: text
Content:
  Package: pkg1
  Type: Package
  Title: Example Package One
  Version: 0.1.0