RSS-NET analysis of simulated GWAS summary statistics and B cell regulatory network
Xiang Zhu
2019-08-18
Last updated: 2021-06-06
Checks: 7 0
Knit directory: rss-net/
This reproducible R Markdown analysis was created with workflowr (version 1.6.2). The Checks tab describes the reproducibility checks that were applied when the results were created. The Past versions tab lists the development history.
Great! Since the R Markdown file has been committed to the Git repository, you know the exact version of the code that produced these results.
Great job! The global environment was empty. Objects defined in the global environment can affect the analysis in your R Markdown file in unknown ways. For reproduciblity it’s best to always run the code in an empty environment.
The command set.seed(20190823) was run prior to running the code in the R Markdown file. Setting a seed ensures that any results that rely on randomness, e.g. subsampling or permutations, are reproducible.
Great job! Recording the operating system, R version, and package versions is critical for reproducibility.
Nice! There were no cached chunks for this analysis, so you can be confident that you successfully produced the results during this run.
Great job! Using relative paths to the files within your workflowr project makes it easier to run your code on other machines.
Great! You are using Git for version control. Tracking code development and connecting the code version to the results is critical for reproducibility.
The results in this page were generated with repository version 6a224d7. See the Past versions tab to see a history of the changes made to the R Markdown and HTML files.
Note that you need to be careful to ensure that all relevant files for the analysis have been committed to Git prior to generating the results (you can use wflow_publish or wflow_git_commit). workflowr only checks the R Markdown file, but you know if there are other scripts or data files that it depends on. Below is the status of the Git repository when the results were generated:
Ignored files:
Ignored: .Rhistory
Ignored: .Rproj.user/
Ignored: examples/ibd2015_nkcell/ibd2015_Primary_Natural_Killer_cells_from_peripheral_blood_NK_out_66.mat
Ignored: examples/ibd2015_nkcell/randinit_results/
Ignored: examples/ibd2015_nkcell/results/
Ignored: examples/wtccc_bcell/results/
Unstaged changes:
Modified: rmd/ibd2015_nkcell.Rmd
Note that any generated files, e.g. HTML, png, CSS, etc., are not included in this status report because it is ok for generated content to have uncommitted changes.
These are the previous versions of the repository in which changes were made to the R Markdown (rmd/wtccc_bcell.Rmd) and HTML (docs/wtccc_bcell.html) files. If you’ve configured a remote Git repository (see ?wflow_git_remote), click on the hyperlinks in the table below to view the files as they were in that past version.
| File | Version | Author | Date | Message |
|---|---|---|---|---|
| Rmd | 6a224d7 | xiangzhu | 2021-06-06 | wflow_publish(“rmd/wtccc_bcell.Rmd”) |
| html | 4b3b74f | xiangzhu | 2021-06-06 | Build site. |
| Rmd | 7ee878f | xiangzhu | 2021-06-06 | wflow_publish(“rmd/wtccc_bcell.Rmd”) |
| html | efd45b7 | xiangzhu | 2021-06-06 | Build site. |
| html | ce99060 | xiangzhu | 2020-12-02 | Build site. |
| html | a7ed39f | xiangzhu | 2020-11-30 | Build site. |
| html | 6ac4388 | xiangzhu | 2020-06-29 | Build site. |
| Rmd | ddaf878 | xiangzhu | 2020-06-29 | wflow_publish(“rmd/wtccc_bcell.Rmd”) |
| html | 93ce892 | xiangzhu | 2020-06-29 | Build site. |
| html | 260f36c | xiangzhu | 2020-03-15 | Build site. |
| Rmd | baf07ec | xiangzhu | 2020-03-15 | wflow_publish(“rmd/wtccc_bcell.Rmd”) |
| html | a424171 | xiangzhu | 2020-03-08 | Build site. |
| html | 5c5e8e1 | xiangzhu | 2020-03-08 | Build site. |
| html | 31f9064 | xiangzhu | 2020-03-06 | Build site. |
| Rmd | 9469f2d | xiangzhu | 2020-03-06 | wflow_publish(“rmd/wtccc_bcell.Rmd”) |
| html | f2c772c | xiangzhu | 2020-03-06 | Build site. |
| Rmd | 26bfe19 | xiangzhu | 2020-03-06 | wflow_publish(“rmd/wtccc_bcell.Rmd”) |
| html | 991b323 | xiangzhu | 2020-03-06 | Build site. |
| Rmd | e3daa54 | xiangzhu | 2020-03-06 | wflow_publish(“rmd/wtccc_bcell.Rmd”) |
| html | 3703a3d | xiangzhu | 2020-03-06 | Build site. |
| Rmd | 4de49b7 | xiangzhu | 2020-03-06 | wflow_publish(“rmd/wtccc_bcell.Rmd”) |
| html | 5467783 | xiangzhu | 2020-03-06 | Build site. |
| Rmd | 9729b44 | xiangzhu | 2020-03-06 | wflow_publish(“rmd/wtccc_bcell.Rmd”) |
| html | c2ee73a | xiangzhu | 2020-03-06 | Build site. |
| Rmd | b109ab5 | xiangzhu | 2020-03-06 | wflow_publish(“rmd/wtccc_bcell.Rmd”) |
Overview
Here we describe an end-to-end RSS-NET analysis of two synthetic datasets that are simulated from real genotypes of 348965 genome-wide SNPs from 1458 individuals in the UK Blood Service Control Group (Wellcome Trust Case Control Consortium, 2007) and a gene regulatory network inferred for B cells (ENCODE Project Consortium, 2012). This example gives a quick view of how RSS-NET works.
To reproduce results of this example, please use scripts in the directory wtccc_bcell/, and follow the step-by-step guide below. Before running any script in wtccc_bcell/, please install RSS-NET.
With the software installed and the input data downloaded, one should be able to run this example by simply typing the following line in shell:
$ sbatch run_simulation.sbatchIf a different directory is used to store the software, input data and/or output data, please modify file paths in the given scripts accordingly.
Step-by-step illustration
Download data files
All data files required to run this example are freely available at Zenodo 
. Please contact me (xiangzhu[at]psu.edu) if you have any trouble accessing these files. After a complete download, you should see the following files.
$ tree .
.
├── m0_data.mat
├── m0_true_gene.mat
├── m1_data.mat
├── m1_true_gene.mat
├── Primary_B_cells_from_peripheral_blood_gene2gene.mat
├── wtccc_bcell_data.md5
├── wtccc_bcell_results.zip
├── WTCCC_Primary_B_cells_from_peripheral_blood_snp2gene_cis.mat
├── WTCCC_Primary_B_cells_from_peripheral_blood_snp2net.mat
├── WTCCC_snp2gene.mat
└── wtccc_tidy_snp2gene_ncbi35togrch37.mat
0 directories, 11 filesTo help readers confirm if they use the same files as we do, we report 128-bit MD5 hashes of all files in wtccc_bcell_data.md5.
To help readers confirm if they can reproduce results of this example, we also provide the results (wtccc_bcell_results.zip) in the same Zenodo deposit.
For simplicity and generality, we introduce the following short-hand notations.
gwas = 'WTCCC';
net = 'Primary_B_cells_from_peripheral_blood'; 1. m*_data.mat: simulated GWAS summary statistics and reference LD estimates
Here we simulate two synthetic datasets from WTCCC genotypes and B cell network: one (m0_data.mat) based on RSS-NET baseline model \(M_0\), and the other (m1_data.mat) based on RSS-NET enrichment model \(M_1\). We match m*_data.mat by the number of trait-associated SNPs and the proportion of phenotypic variation explained by all SNPs. By this design, m*_data.mat have the same amount of genetic signals, and they only differ in how genetic signals are distributed: for the enrichment dataset m1_data.mat genetic associations are enriched in the B cell network; for the baseline dataset m0_data.mat genetic associations are randomly distributed across the genome.
$ md5sum m*_data.mat
5d652796b4cc358843920b1edf2819fc m0_data.mat
996eae1410f89607f1debefb31ec7811 m1_data.mat
$ du -sh m*_data.mat
3.0G m0_data.mat
3.0G m1_data.matLet’s look at the contents of m*_data.mat.
>> sumstat=matfile('m0_data.mat');
>> sumstat
matlab.io.MatFile
Properties:
Properties.Source: 'm0_data.mat'
Properties.Writable: false
SiRiS: [22x1 cell]
betahat: [22x1 cell]
chr: [22x1 cell]
pos: [22x1 cell]
se: [22x1 cell]GWAS summary statistics and LD estimates are stored as cell arrays. RSS-NET only uses the following variables:
betahat{j,1}, single-SNP effect size estimates of all SNPs on chromosomej;se{j,1}, standard errors ofbetahat{j, 1};chr{j,1}andpos{j, 1}, physical positions of these SNPs (GRCh37 build);SiRiS{j,1}, a sparse matrix, defined asrepmat((1./se),1,p) .* R .* repmat((1./se)',p,1), whereRis the estimated LD matrix of thesepSNPs.
2. ${gwas}_snp2gene.mat: physical distance between SNPs and genes
This file contains the physical distance between each GWAS SNP and each protein-coding gene, within 1 Mb. This file corresponds to \({\bf G}_j\) in the RSS-NET model.
$ md5sum WTCCC_snp2gene.mat
50da1887601df3a0914380e2ea9b1be7 WTCCC_snp2gene.mat
$ du -sh WTCCC_snp2gene.mat
645M WTCCC_snp2gene.matIn this example, there are 18334 genes and 348965 SNPs.
>> snp2gene=matfile('WTCCC_snp2gene.mat');
>> snp2gene
matlab.io.MatFile
Properties:
Properties.Source: 'WTCCC_snp2gene.mat'
Properties.Writable: false
chr: [348965x1 double]
colid: [41897736x1 int32]
ncbi35_pos: [348965x1 int32]
numgene: [1x1 int32]
numsnp: [1x1 int32]
pos: [348965x1 int32]
rowid: [41897736x1 int32]
val: [41897736x1 double]
>> [snp2gene.numgene snp2gene.numsnp]
18334 348965The SNP-to-gene distance information is captured by a three-column matrix [colid rowid val]. For example, the distance between gene 7 and SNP 161 is 2855196 bps.
>> colid=snp2gene.colid; rowid=snp2gene.rowid; val=snp2gene.val;
>> [colid(6688) rowid(6688) val(6688)]
7 161 28551963. ${net}_gene2gene.mat: gene regulatory network
This file contains information of gene-to-gene connections in a given regulatory network.
$ md5sum Primary_B_cells_from_peripheral_blood_gene2gene.mat
0098dd6d85762b18d5d594da6c3cfe93 Primary_B_cells_from_peripheral_blood_gene2gene.mat
$ du -sh Primary_B_cells_from_peripheral_blood_gene2gene.mat
1.8M Primary_B_cells_from_peripheral_blood_gene2gene.mat>> gene2gene=matfile('Primary_B_cells_from_peripheral_blood_gene2gene.mat');
>> gene2gene
matlab.io.MatFile
Properties:
Properties.Source: 'Primary_B_cells_from_peripheral_blood_gene2gene.mat'
Properties.Writable: false
colid: [110062x1 int32]
numgene: [1x1 int32]
rowid: [110062x1 int32]
val: [110062x1 double]For implementation convenience, this file contains the trivial case where each gene is mapped to itself with val=1.
>> colid=gene2gene.colid; rowid=gene2gene.rowid; val=gene2gene.val;
>> [gene2gene.numgene sum(colid==rowid) unique(val(colid==rowid))]
18334 18334 1For a given network, transcription factors (TFs) are stored in rowid and target genes (TGs) are stored in colid. In this example there are 3018 TGs and 436 TFs. Among these TFs and TGs, there are 91728 edges. The edge weights range from 0.61 to 1. These TF-to-TG connections and edge weights correspond to \(\{{\bf T}_g,v_{gt}\}\) in the RSS-NET model.
>> [length(unique(colid(colid ~= rowid))) length(unique(rowid(colid ~= rowid)))]
3018 436
>> [length(colid(colid ~= rowid)) length(rowid(colid ~= rowid))]
91728 91728
>> val_tftg = val(colid ~= rowid);
>> [min(val_tftg) quantile(val_tftg, 0.25) median(val_tftg) quantile(val_tftg, 0.75) max(val_tftg)]
0.6101 0.6278 0.6515 0.6885 1.0000 4. ${gwas}_${net}_snp2net.mat: SNP-to-network proximity annotation
This file contains binary annotations whether a SNP is “near” the given network, that is, within 100 kb of any network element (TF, TG or associated regulatory elements).
$ md5sum WTCCC_Primary_B_cells_from_peripheral_blood_snp2net.mat
701264f811c0f5edd582ee3eec5cfd15 WTCCC_Primary_B_cells_from_peripheral_blood_snp2net.mat
$ du -sh WTCCC_Primary_B_cells_from_peripheral_blood_snp2net.mat
9.4M WTCCC_Primary_B_cells_from_peripheral_blood_snp2net.matIn this example, there are 348965 GWAS SNPs and 121308 of them are near the B cell network (i.e. val=1).
>> snp2net=matfile('WTCCC_Primary_B_cells_from_peripheral_blood_snp2net.mat');
>> snp2net
matlab.io.MatFile
Properties:
Properties.Source: 'WTCCC_Primary_B_cells_from_peripheral_blood_snp2net.mat'
Properties.Writable: false
chr: [348965x1 double]
ncbi35_pos: [348965x1 int32]
pos: [348965x1 int32]
snpid: [348965x1 int32]
val: [348965x1 double]
window: [1x1 double]
>> [length(snp2net.val) sum(snp2net.val) snp2net.window]
348965 121308 100000
>> unique(snp2net.val)'
0 15. ${gwas}_${net}_snp2gene_cis.mat: SNP-to-gene cis regulation
This file contains the SNP-to-gene cis regulation scores. (For simplicity we do not use context-matching cis eQTL studies in our simulations.) This file corresponds to \((c_{jg}-1)\) in the RSS-NET model.
$ md5sum WTCCC_Primary_B_cells_from_peripheral_blood_snp2gene_cis.mat
b77eebd576eee6d19ac3c8ed94681f00 WTCCC_Primary_B_cells_from_peripheral_blood_snp2gene_cis.mat
$ du -sh WTCCC_Primary_B_cells_from_peripheral_blood_snp2gene_cis.mat
4.5M WTCCC_Primary_B_cells_from_peripheral_blood_snp2gene_cis.mat>> snp2gene_cis = matfile('WTCCC_Primary_B_cells_from_peripheral_blood_snp2gene_cis.mat');
>> snp2gene_cis
matlab.io.MatFile
Properties:
Properties.Source: 'WTCCC_Primary_B_cells_from_peripheral_blood_snp2gene_cis.mat'
Properties.Writable: false
colid: [291181x1 int32]
l0: [1x1 double]
l1: [1x1 double]
numgene: [1x1 int32]
numsnp: [1x1 int32]
rowid: [291181x1 int32]
val: [291181x1 double]
window: [1x1 double]In this example, there are 291181 SNP-gene pairs with cis regulation scores available, consisting of 111097 SNPs and 5345 genes. The cis regulation scores (val) range from 0.02 to 0.90.
>> [snp2gene_cis.numsnp length(unique(snp2gene_cis.rowid)) snp2gene_cis.numgene length(unique(snp2gene_cis.colid))]
348965 111097 18334 5345
>> val=snp2gene_cis.val;
>> [min(val) quantile(val, 0.25) median(val) quantile(val, 0.75) max(val)]
0.0202 0.1615 0.2661 0.3864 0.8963Run RSS-NET analysis
To facilitate running RSS-NET on real data, we provide a generic script simulation_template.m. For the present example, the RSS-NET analysis of two synthetic datasets is implemented by run_simulation.m and run_simulation.sbatch.
Please see scripts in wtccc_bcell/ for full details. Below we only highlight the important parts.
1. Specify input data files
First we need to tell RSS-NET where to find all required input files, by creating the following structure array data. Please ensure that you save all downloaded files in dat_path.
data.sumstats_file = strcat(dat_path,trial_name,'_data.mat');
data.snp2net_file = strcat(dat_path,gwas_name,'_',net_name,'_snp2net.mat');
data.snp2gene_file = strcat(dat_path,gwas_name,'_snp2gene.mat');
data.gene2gene_file = strcat(dat_path,net_name,'_gene2gene.mat');
data.snp2gene_cis_file = strcat(dat_path,gwas_name,'_',net_name,'_snp2gene_cis.mat');2. Specify a hyper-parameter grid
Next we need feed a hyper-parameter grid to RSS-NET, by creating the following structure array hyper. The total computational cost of RSS-NET is proportional to the grid size. Hence, if you want to finish running this example faster, you can always reduce the grid size.
hyper.theta0 = true_theta0;
hyper.sigma0 = true_sigma0;
hyper.theta = [0 ((true_theta-0.5):0.25:(true_theta+0.5))];
hyper.sigma = [0 ((true_sigma-0.5):0.25:(true_sigma+0.5))];3. Initialize variational parameters
We can provide initial values for variational parameters in RSS-NET as follows. If we do not provide initial values, RSS-NET will use a random initialization.
options.alpha = alpha0;
options.mu = mu0;4. Fit the RSS-NET model
We use rss_net.m to fit the RSS-NET model on a given dataset. The model fitting results consist of variational lower bounds logw and variational parameter estimates [alpha mu s] for the given hyper-parameter grid.
[logw,alpha,mu,s,param] = rss_net(data,hyper,options);5. Summarize network-level enrichments
To assess whether a regulatory network is enriched for genetic associations with a trait, we evaluate a Bayes factor (BF) comparing the baseline model (\(M_0:\theta=0~\text{and}~\sigma^2=0\)) in RSS-NET with the enrichment model (\(M_1:\theta>0~\text{or}~\sigma^2>0\)). The BF computation is implemented in calc_bf.m.
log10_bf = calc_bf(logw,param);6. Summarize gene-level associations
To summarize association between a locus and a trait, we compute \(P_1^{\sf net}\), the posterior probability that at least one SNP \(j\) in the locus is associated with the trait (\(\beta_j\neq 0\)): \[
P_1^{\sf net}=1-\Pr(\beta_j=0,~\forall j\in\text{locus}~|~\text{data},M_1).
\] As in Zhu et al (2021), a locus is defined as the transcribed region of a gene plus 100 kb upstream and downstream. The locus definition is provided in wtccc_tidy_snp2gene_ncbi35togrch37.mat. The \(P_1^{\sf net}\) computation is implemented in compute_pip.m.
seg_path = strcat(dat_path,'wtccc_tidy_snp2gene_ncbi35togrch37.mat');
p1_net = compute_pip(seg_path,logw,alpha);7. Submit jobs
After understanding the basic workflow of RSS-NET, we submit two jobs of running RSS-NET on the synthetic datasets m*_data.mat. To this end, we write a simple sbatch script run_simulation.sbatch, and submit it to a cluster with Slurm available.
$ squeue -u xiangzhu
JOBID PARTITION NAME USER ST TIME NODES NODELIST(REASON)
66258089_0 broadwl run_simu xiangzhu R 2:48:17 1 midway2-0009
66258089_1 broadwl run_simu xiangzhu R 2:48:17 1 midway2-0011For each job, we request 1 node with 8 CPUs and 32 Gb total memory. The memory utilized by these two jobs is around 16 Gb each. The running time of these two jobs is around 4 hours each.
8. Understand analysis results
Running run_simulation.sbatch yields two MAT-files m*_results.mat.
>> m0_res=matfile('m0_results.mat');
>> m1_res=matfile('m1_results.mat');As shown below, RSS-NET yields a small BF (\(0.13\)) for the baseline dataset m0_data.mat and a large BF (\(5.85\times 10^{55}\)) for the enrichment dataset m1_data.mat.
>> [m0_res.log10_bf m1_res.log10_bf]
-0.8917 55.7671We also compare gene-level associations inferred from RSS-NET (m*_res.pp) with ground truth (m*_true_gene.mat). RSS-NET shows a higher statistical power in identifying gene-level associations on the enrichment dataset m1_data.mat (\(\text{AUC}=0.78\)) than the baseline dataset m0_data.mat (\(\text{AUC}=0.60\)). This is because RSS-NET automatically exploits the underlying network enrichment in m1_data.mat when assessing genetic associations.
>> m1_gene=matfile('m1_true_gene.mat');
>> m1_mdl = fitglm(m1_res.pp,m1_gene.true_gene,'Distribution','binomial','Link','logit');
>> [X,Y,T,m1_auc] = perfcurve(m1_gene.true_gene,m1_mdl.Fitted.Probability,1);
>> m1_auc
0.7767
>> m0_gene=matfile('m0_true_gene.mat');
>> m0_mdl = fitglm(m0_res.pp,m0_gene.true_gene,'Distribution','binomial','Link','logit');
>> [X,Y,T,m0_auc] = perfcurve(m0_gene.true_gene,m0_mdl.Fitted.Probability,1);
>> m0_auc
0.5954
devtools::session_info()─ Session info ───────────────────────────────────────────────────────────────
setting value
version R version 4.1.0 (2021-05-18)
os Ubuntu 20.04.2 LTS
system x86_64, linux-gnu
ui X11
language (EN)
collate en_US.UTF-8
ctype en_US.UTF-8
tz America/New_York
date 2021-06-06
─ Packages ───────────────────────────────────────────────────────────────────
package * version date lib source
bslib 0.2.5.1 2021-05-18 [3] CRAN (R 4.1.0)
cachem 1.0.5 2021-05-15 [3] CRAN (R 4.0.5)
callr 3.7.0 2021-04-20 [3] CRAN (R 4.0.5)
cli 2.5.0 2021-04-26 [3] CRAN (R 4.0.5)
crayon 1.4.1 2021-02-08 [3] CRAN (R 4.0.3)
desc 1.3.0 2021-03-05 [3] CRAN (R 4.0.4)
devtools 2.4.1 2021-05-05 [3] CRAN (R 4.0.5)
digest 0.6.27 2020-10-24 [3] CRAN (R 4.0.3)
ellipsis 0.3.2 2021-04-29 [3] CRAN (R 4.0.5)
evaluate 0.14 2019-05-28 [3] CRAN (R 4.0.0)
fansi 0.5.0 2021-05-25 [3] CRAN (R 4.1.0)
fastmap 1.1.0 2021-01-25 [3] CRAN (R 4.0.3)
fs 1.5.0 2020-07-31 [3] CRAN (R 4.0.2)
git2r 0.28.0 2021-01-10 [1] CRAN (R 4.1.0)
glue 1.4.2 2020-08-27 [3] CRAN (R 4.0.2)
htmltools 0.5.1.1 2021-01-22 [3] CRAN (R 4.0.3)
httpuv 1.6.1 2021-05-07 [3] CRAN (R 4.0.5)
jquerylib 0.1.4 2021-04-26 [3] CRAN (R 4.0.5)
jsonlite 1.7.2 2020-12-09 [3] CRAN (R 4.0.3)
knitr 1.33 2021-04-24 [3] CRAN (R 4.0.5)
later 1.2.0 2021-04-23 [3] CRAN (R 4.0.5)
lifecycle 1.0.0 2021-02-15 [3] CRAN (R 4.0.4)
magrittr 2.0.1 2020-11-17 [3] CRAN (R 4.0.3)
memoise 2.0.0 2021-01-26 [3] CRAN (R 4.0.3)
pillar 1.6.1 2021-05-16 [3] CRAN (R 4.0.5)
pkgbuild 1.2.0 2020-12-15 [3] CRAN (R 4.0.3)
pkgconfig 2.0.3 2019-09-22 [3] CRAN (R 4.0.0)
pkgload 1.2.1 2021-04-06 [3] CRAN (R 4.0.5)
prettyunits 1.1.1 2020-01-24 [3] CRAN (R 4.0.0)
processx 3.5.2 2021-04-30 [3] CRAN (R 4.0.5)
promises 1.2.0.1 2021-02-11 [3] CRAN (R 4.0.3)
ps 1.6.0 2021-02-28 [3] CRAN (R 4.0.4)
purrr 0.3.4 2020-04-17 [3] CRAN (R 4.0.0)
R6 2.5.0 2020-10-28 [3] CRAN (R 4.0.3)
Rcpp 1.0.6 2021-01-15 [3] CRAN (R 4.0.5)
remotes 2.4.0 2021-06-02 [3] CRAN (R 4.1.0)
rlang 0.4.11 2021-04-30 [3] CRAN (R 4.0.5)
rmarkdown 2.8 2021-05-07 [3] CRAN (R 4.0.5)
rprojroot 2.0.2 2020-11-15 [3] CRAN (R 4.0.3)
sass 0.4.0 2021-05-12 [3] CRAN (R 4.0.5)
sessioninfo 1.1.1 2018-11-05 [3] CRAN (R 4.0.0)
stringi 1.6.2 2021-05-17 [3] CRAN (R 4.0.5)
stringr 1.4.0 2019-02-10 [3] CRAN (R 4.0.0)
testthat 3.0.2 2021-02-14 [3] CRAN (R 4.0.4)
tibble 3.1.2 2021-05-16 [3] CRAN (R 4.0.5)
usethis 2.0.1 2021-02-10 [3] CRAN (R 4.0.3)
utf8 1.2.1 2021-03-12 [3] CRAN (R 4.0.4)
vctrs 0.3.8 2021-04-29 [3] CRAN (R 4.0.5)
whisker 0.4 2019-08-28 [3] CRAN (R 4.0.0)
withr 2.4.2 2021-04-18 [3] CRAN (R 4.0.5)
workflowr * 1.6.2 2020-04-30 [1] CRAN (R 4.1.0)
xfun 0.23 2021-05-15 [3] CRAN (R 4.0.5)
yaml 2.2.1 2020-02-01 [3] CRAN (R 4.0.0)
[1] /home/maimaizhu/R/x86_64-pc-linux-gnu-library/4.1
[2] /usr/local/lib/R/site-library
[3] /usr/lib/R/site-library
[4] /usr/lib/R/library