September 2022



In this workshop, we have three parts:

  • Delivering TLFs in CSR
    • Session 1 (45 min)
    • Break and/or Exercise (20 min)
    • Session 2 (45 min)
    • Break (10 min)
  • Clinical trial project
    • Session 3 (30 min)
    • Break (10 min)
  • eCTD submission package
    • Session 4 (40 min)
    • Q&A (10 min)


  • All opinions expressed are those of the presenter.

  • Some slides need to be scrolled down to see the full content.

Training Objective

  • Learning how to create TLFs for clinical study reports.
  • Learning how to organize a clinical development project.
  • Learning how to prepare eCTD submission package to FDA.

The toolchain, process, and formats may be different in different organizations. We only provide a recommended way to address them.


  • Contributors of the training materials.
    • Please consider submitting issues or PR in the github repo.
  • R Consortium
    • R validation hub
    • Submission working group
    • R Tables for Regulatory Submission (RTRS) working group
    • Please consider to join and contribute to a working group.


In this workshop, we assume you have some R programming experience and clinical development knowledge.

  • Data manipulation: tidyverse, dplyr, tidyr, ggplot2 etc.
  • ADaM data: adsl, adae etc.
  "tidyverse", # Data manipulation
  "r2rtf",     # TLF generation
  "pkglite"    # eCTD submission



As an organization, we need to ensure compliance and reduce the risk of using R and R packages in regulatory deliverables.

  • R is widely used in clinical trial study design.
  • R is flexible for handling novel missing data approaches following the estimand framework in ICH E9 (R1)
    • Recurrent event data: Gao et al. (2017) Diao et al. (2020)
    • Robustness: Liu et al. (2021)
  • R is used in Bayesian statistics.
    • stan and network meta-analysis for drug reimbursement analysis
  • R is widely used in visualization
    • metalite.table1: Interactive demographic table.
    • forestly: Interactive forest plot for DMC safety monitoring in clinical trials


We share the same philosophy described in Section 1.1 of R Packages book and quote here.

  • “Anything that can be automated, should be automated.”
  • “Do as little as possible by hand. Do as much as possible with functions.”
  • “The goal is to spend your time thinking about what you want to do rather than thinking about the minutiae of package structure.”

Delivering TLFs in CSR

ICH E3 Guidance

The ICH E3: structure and content of clinical study reports provides guidance to assist sponsors in the development of a CSR.

In a CSR, most of TLFs are located in

  • Section 10: Study patients
  • Section 11: Efficacy evaluation
  • Section 12: Safety evaluation
  • Section 14: Tables, Figures and Graphs referred to but not included in the text
  • Section 16: Appendices



  • tidyverse: a collection of R packages to simplify the workflow to manipulate, visualize and analyze data in R.

  • r2rtf: an R package to create production-ready tables and figures in RTF format.

r2rtf Introduction


In the pharmaceutical industry, RTF/Microsoft Word play a central role in preparing clinical study reports

Different organizations can have different table standards

  • E.g., Table layout, Font size, Border type, Footnote, Data source

  • r2rtf is an R package to create production-ready tables and figures in RTF format.

r2rtf is designed to:

  • provide simple “verb” functions that correspond to each component of a table, to help you translate a data frame to a table in RTF format;
  • enable pipes (%>%);
  • only focus on the table format.
    • Data manipulation and analysis should be handled by other R packages. (e.g., tidyverse)


Before creating an RTF table, we need to:

  • Figure out table layout.

  • Split the layout into small tasks in the form of a computer program.

  • Execute the program.

Minimal Example

r2rtf is designed to enable pipes (%>%).

head(iris) %>%
  rtf_body() %>%           # Step 1 Add table attributes
  rtf_encode() %>%         # Step 2 Convert attributes to RTF encode
  write_rtf("minimal.rtf") # Step 3 Write to a .rtf file

Package Overview

r2rtf package provides the flexibility to customize table appearance for

  • Table component: title, column header, footnote, etc.
  • Table cell style: size, border type, color, font size, text color, alignment, etc.
  • Flexible control: the specification of the cell style can be row or column vectorized.
  • Complicated format: pagination, section grouping, multiple table concatenations, etc.

r2rtf package also provides the flexibility to convert figures in RTF format.

Function Summary

r2rtf provides simple “verb” functions that correspond to each component of a table, to help you translate data frame to tables in RTF file.

Functions Purpose Optional/required
rtf_page_header() add page header optional
rtf_title() add title optional
rtf_subline() add subject line optional
rtf_colheader() add column header optional
rtf_body() add table body required
rtf_footnote() add footnote optional
rtf_source() add data source optional
rtf_page_footer() add page footer optional
rtf_encode() convert table into rtf code required
write_rtf() write rtf code into .rtf file required

Function Illustration

Break and/or Exercise (20 min)

CSR Examples

Disposition Table

Analysis Population

Baseline Characteristics

Efficacy Table

AE Summary Table

Specific AE Table

Assemble TLFs

Break (10 min)

Clinical Trial Project


In a late-stage clinical trial, the number of A&R deliverables can easily be in the hundreds.

For an organization, it is also common to have multiple ongoing clinical trials in a clinical program.

In this part, let’s consider how to organize a clinical trial project as an A&R lead.

Folder Structure

Our primary focus is creating a standard R package structure to organize the project, with 4 goals in mind:

  • Consistency
  • Reproducibility
  • Automation
  • Compliance

Project Management

  • Setting up for success
    • Teamwork
    • Architecture
    • Boundaries
    • Community
  • The SDLC
    • Planning
    • Development
    • Validation
    • Operation

Break (10 min)

eCTD Submission

Clarification from FDA

“FDA does not require use of any specific software for statistical analyses, and statistical software is not explicitly discussed in Title 21 of the Code of Federal Regulations [e.g., in 21CFR part 11]. However, the software package(s) used for statistical analyses should be fully documented in the submission, including version and build identification.”

R Consortium Pilot Submission Working group

  • The R submission working group is a cross-industry pharma working group focusing on improving practices of R-based clinical trial regulatory submission.
  • Mission: Easier R-based clinical trial regulatory submissions
  • pilot 1 - common analyses
    • 2021.11 Initial submission to FDA.
    • 2022.12 Received FDA response
    • 2022.02 Revised submission to FDA
    • 2022.03 Received final FDA response.

Challenges and Assumptions


  • How to submit internally developed (proprietary) R packages?
  • How to follow ICH and/or FDA guidances in preparing eCTD package?
  • How to enhance reproducibility from FDA reviewer’s perspective?


  • Focus on analysis and reporting given available ADaM datasets.


FDA Response

Initial Submission

  • “Using R version 4.1.1, FDA was able to run the submitted code and confirm the applicant’s tables and the submitted figure in report-tlf pdf file.”
  • “Using FDA developed code, a statistical analyst was able to independently generate tables using the submitted data.”
  • Initial submission FDA response

Revised Submission

Future Work

  • Pilot R Shiny app submission to FDA.
  • Seek pilot submission opportunities to other regulator agencies.
  • Pilot submission with advanced analysis methods (e.g., study design, missing data, Bayesian, etc).

Submission Package

We will discuss strategies to prepare proprietary R packages and analysis code into proper formats for submission:

  • The whole game
  • Practical considerations
  • Prepare R packages using pkglite
  • Prepare analysis programs
  • Update ADRG and ARM

Running Environment


pkglite: Compact Package Representations

  • To provide a tool for packing and restoring R packages as plaintext assets that are easy to store, transfer, and review.
  • To provide a grammar for specifying the file packing scope that is functional, precise, and extendable.
  • To provide a standard for exchanging the packed asset that is unambiguous, human-friendly, and machine-readable.

Pipe-friendly Workflow


"/path/to/pkg/" %>%
  collate(file_ectd()) %>%

  "/path/to/pkg1/" %>% collate(file_ectd()),
  "/path/to/pkg2/" %>% collate(file_ectd()),
  output = "/path/to/pkglite.txt"

"/path/to/pkglite.txt" %>%
  unpack(output = "/path/to/output/")

File Specifications + File Collections

File specifications offer flexibility and brevity in specifying the files to include.

File specification type Functions
Manual discovery file_spec()
Automatic discovery file_auto()
Common patterns file_root_core(), file_r(), file_man(), file_src(), file_vignettes(), …
Default sets file_default(), file_ectd()

File collections contain the evaluation results of file specifications for packing.


"/path/to/pkg" %>%
  collate(file_root_core(), file_r(), file_auto("inst/"))


pkglite.txt follows the standard Debian Control File (DCF) format used by Debian, R, and RStudio IDE, to be both machine-readable and human-readable.

# Generated by pkglite: do not edit by hand
# Use pkglite::unpack() to restore the packages

Package: pkg1
Format: text
  Package: pkg1
  Type: Package
  Title: Example Package One
  Version: 0.1.0



Diao, Guoqing, Guanghan F Liu, Donglin Zeng, Yilong Zhang, Gregory Golm, Joseph F Heyse, and Joseph G Ibrahim. 2020. “Efficient Multiple Imputation for Sensitivity Analysis of Recurrent Events Data with Informative Censoring.” Statistics in Biopharmaceutical Research, 1–9.

Gao, Fei, Guanghan F Liu, Donglin Zeng, Lei Xu, Bridget Lin, Guoqing Diao, Gregory Golm, Joseph F Heyse, and Joseph G Ibrahim. 2017. “Control-Based Imputation for Sensitivity Analyses in Informative Censoring for Recurrent Event Data.” Pharmaceutical Statistics 16 (6): 424–32.

Lachin, John M, and Mary A Foulkes. 1986. “Evaluation of Sample Size and Power for Analyses of Survival with Allowance for Nonuniform Patient Entry, Losses to Follow-up, Noncompliance, and Stratification.” Biometrics, 507–19.

Liu, Siyi, Shu Yang, Yilong Zhang, et al. 2021. “Multiply Robust Estimators in Longitudinal Studies with Missing Data Under Control-Based Imputation.” arXiv Preprint arXiv:2112.06000.