--- title: "HyperCom Overview" author: "Justin K. Moy & Pawel F. Przytycki" output: rmarkdown::html_vignette: fig_width: 6 fig_height: 4 vignette: > %\VignetteIndexEntry{HyperCom Overview} %\VignetteEngine{knitr::rmarkdown} %\VignetteEncoding{UTF-8} --- #### Last Updated: `r Sys.Date()` This tutorial runs HyperCom on 1,000 mouse heart cells and shows how to: 1. run the whole analysis with one call, 2. read the ranked ligand-receptor pairs, 3. score and plot the cells that send and receive a signal. The last section runs each step on its own, for those who want to see how HyperCom works. ```{r setup, message=FALSE, warning=FALSE} library(HyperCom) library(dplyr) library(ggplot2) library(ggridges) library(Matrix) library(tidyr) ``` ## 1. Check your input HyperCom needs three inputs. Most failed runs come from one of them, so check them first. | Input | What it must be | |---|---| | **Counts** | A genes x cells matrix (sparse or dense). Row names are gene symbols; column names are cell names. Raw or normalized, **never scaled**: HyperCom only uses whether each gene is above 0 in each cell. | | **Cell metadata** | A data frame with one row per cell; row names are the cell names. A **sample** column (library or replicate) and, optionally, a **group** column (condition). | | **Ligand-receptor database** | `cellchat.hs.lrdb` for human (genes like `TGFB1`) or `cellchat.mm.lrdb` for mouse (`Tgfb1`). | Also: - **Cell names must be unique.** When you merge samples, add the sample name to each barcode. - **Cells are compared only within their sample**, so each sample needs at least 2 cells that express a ligand or receptor gene. - Cells in the counts but not in the metadata are left out, with a warning. ### The example data A 1,000-cell subset of the heart failure data of [Alexanian et al., 2021](https://doi.org/10.1038/s41586-021-03674-1), with log-normalized counts. It comes with the package as `alexanian2021.counts` and `alexanian2021.metadata`. ```{r} counts <- alexanian2021.counts dim(counts) head(rownames(counts)) head(colnames(counts)) ``` ```{r} cell.meta <- alexanian2021.metadata head(cell.meta) table(cell.meta$sample, cell.meta$condition) ``` ```{r} ggplot(cell.meta, aes(x=UMAP_1, y=UMAP_2, colour=celltype)) + geom_point(size=0.5) + theme_classic() ``` The data is mouse, so we use the mouse database. Each row is one interaction: its name, the ligand, then the receptor genes (`NA` when the receptor has fewer subunits). ```{r} lrdb <- cellchat.mm.lrdb dim(lrdb) head(lrdb, 2) ``` ## 2. Run HyperCom One call runs the whole analysis. - `parallel=TRUE` uses one thread per physical core (`threads` sets the number; the default is 1 thread). - `significance=TRUE` adds p-values; `seed` makes them reproducible. ```{r} outdir <- file.path(tempdir(), "overview") res <- HyperCom(lrdb=cellchat.mm.lrdb, counts=alexanian2021.counts, metadata=alexanian2021.metadata, sample="sample", group="condition", significance=TRUE, seed=1, parallel=TRUE, outdir=outdir, verbose=FALSE) ``` `res` holds the results, and they are also saved to `outdir`: | File | Contents | In `res` | |---|---|---| | `0.counts.rds` | counts of the ligand and receptor genes | | | `1.lrs.csv` | interactions whose genes are all expressed | `res$lrs` | | `2.hyperedges.csv` | hyperedges of the hypergraph | | | `3.adj.rds` | adjacency matrix | `res$adj` | | `4.infMat.rds` | influence matrix | `res$infMat` | | `5.metadata.csv` | metadata of cells and genes | `res$metadata` | | `6.priority/priority..csv` | ranked pairs, one file per weight | `res$priority` | Read the CSV files with `read.csv` and the `.rds` files with `readRDS` (after `library(HyperCom)`). ```{r} list.files(outdir, recursive=TRUE) ``` ### From a Seurat object or SingleCellExperiment Same arguments, but HyperCom reads the counts and metadata from the object. `sample`, `group` and `keep` name columns of its cell metadata (`seurat[[]]` or `colData(sce)`). ```{r, eval=FALSE} res <- HyperCom_seurat(seurat=seurat_object, lrdb=cellchat.mm.lrdb, sample="sample", group="condition", parallel=TRUE, outdir="overview") res <- HyperCom_sce(sce=sce_object, lrdb=cellchat.mm.lrdb, sample="sample", group="condition", parallel=TRUE, outdir="overview") ``` - **`HyperCom_seurat`** reads the `counts` layer of the `RNA` assay by default (change with `assay` and `layer`). Works with Seurat v5 and older. Needs the SeuratObject package. - **`HyperCom_sce`** reads the `counts` assay by default (change with `assay`). Row names must be gene symbols. Data stored on disk (DelayedArray, HDF5Array) is read only for the database genes. Needs the SingleCellExperiment package from Bioconductor. ## 3. Ranked ligand-receptor pairs `res$priority` has one table per weight, sorted from the highest priority. A pair ranks high when the cells that express its genes: - are a large fraction of all cells, and each of its genes is expressed in many cells, - send or receive the signal strongly, - split evenly into senders and receivers, - and, with `group`, concentrate in some groups. With `significance=TRUE`, the tables also have p-values: - **p:** the fraction of 1,000 background pairs of random genes that rank at least as high. It moves in steps of 0.001, and 0 means none did. Background pairs that get no priority are left out, with a warning. - **p.adj:** p with the Benjamini-Hochberg adjustment. ```{r} head(res$priority[["1"]][, c("interaction", "priority", "p", "p.adj")]) ``` `weights` sets how much the fraction of cells counts. The default, 1, weighs it like the other terms; smaller values favor pairs specific to fewer cells. Several weights give one table each (see `prioritize_lr` below). ## 4. Score cells Thbs1-Sdc4 is the top pair. `score_hypercom` scores every cell for it: - **`sender`, `receiver`:** from 0 to 1, how strongly the cell sends or receives the signal, compared with the other cells of its sample. - **`dif`** = receiver - sender; **`avg`** is their mean; **`max`** the larger of the two. - **`hypercom`:** one score per cell, negative for senders and positive for receivers. - **NaN:** the cell has no path to the pair's genes. ```{r} Thbs1_Sdc4 <- score_hypercom(infMat=res$infMat, metadata=res$metadata, ligand="Thbs1", receptors="Sdc4") head(Thbs1_Sdc4[, c("cell", "sender", "receiver", "max", "hypercom")]) ``` How the scores are made: each pair of cells i and j of a sample gets a score for how strongly cell i reaches the ligand and the receptor reaches cell j, relative to how strongly i reaches j. These scores are percent-ranked together. A cell's `sender` score is its average rank as cell i, and its `receiver` score its average rank as cell j. `hypercom` is the sign of `dif` times the percent rank of its size among the cells, times `avg`. On the UMAP, senders are blue and receivers orange. Cells that express no ligand or receptor gene are not in the hypergraph and stay grey. ```{r} umap <- data.frame(cell=rownames(cell.meta), UMAP_1=cell.meta$UMAP_1, UMAP_2=cell.meta$UMAP_2) umap <- left_join(umap, Thbs1_Sdc4[, c("cell", "hypercom")], by=join_by(cell)) ggplot(umap, aes(x=UMAP_1, y=UMAP_2, colour=hypercom)) + geom_point(size=0.5) + scale_colour_gradient2(low="#0074a7", mid="grey90", high="#e69c25", na.value="grey70") + ggtitle("Thbs1-Sdc4") + theme_classic() ``` Split by condition, TAC has higher densities of both senders and receivers. ```{r, warning=FALSE, message=FALSE} Thbs1_Sdc4 <- left_join(Thbs1_Sdc4, res$metadata, by=join_by(cell)) melt.Thbs1_Sdc4 <- pivot_longer(dplyr::select(Thbs1_Sdc4, c(cell, group, sender, receiver)), !c(cell, group), names_to = "type", values_to = "score") ggplot(melt.Thbs1_Sdc4, aes(x = score, y = type, fill=group)) + geom_density_ridges(alpha=0.4) + xlim(0.25, 1) + xlab("Score") + ylab("Score Type") + ggtitle("Thbs1-Sdc4") + theme_classic() ``` For a receptor complex, give all its subunits; their scores are averaged. ```{r} Tgfb1_Tgfbr <- score_hypercom(infMat=res$infMat, metadata=res$metadata, ligand="Tgfb1", receptors=c("Tgfbr1", "Tgfbr2")) head(Tgfb1_Tgfbr[, c("cell", "sender", "receiver", "hypercom")]) ``` ## 5. Large data ### Run time and memory Time and memory depend on the number of cells **per sample** (they grow with its square), so more samples of the same size cost little. The data of this tutorial takes seconds. One sample, with p-values, 8 cores: | Cells in the sample | Time | Peak memory | |---|---|---| | 1,000 | 6 s | 0.5 GiB | | 3,000 | 42 s | 0.8 GiB | | 10,000 | 8 min | 2.4 GiB | | 30,000 | 1.3 h | 15 GiB | | 60,000 | 5.9 h | 57 GiB | Time also grows with the number of ligand-receptor pairs expressed (about 700 here). P-values took about 40% of the time at every size from 3,000 cells up (about 60% as long without them); with fewer pairs, they are a larger share. Measured with `HyperCom_seurat()` on one sample of a Xenium 5K human ovarian cancer slide, the CellChat human database (about 700 pairs expressed), `threads=8` on an Intel Xeon Gold 6426Y node; median of 5 runs (3 at 60,000 cells). By default the influence matrix is never built in full, so its memory grows with the number of cells, not its square. `infmat.method="lu"` builds it in full and gives exactly the numbers of HyperCom 0.0.0.9000 (the default agrees with them to about 1e-15), but needs two matrices of all cells and genes squared. ### Runs that stop Scoring the pairs takes almost all the time: hours on large data. As it goes, HyperCom saves the scored pairs in `/6.priority/cache`. **If a run stops** (cluster time limit, crash, closed R session), **run the same call again.** The earlier steps repeat (seconds) and scoring continues from the cache, losing at most the last batch of pairs. You will see a message like `Continuing from the cache in overview/6.priority/cache: 812 of 2042 pairs already scored`. - **Same results** as a run that never stopped, on the same type of computer. Another CPU can change the last digits (about 1e-13), so on a cluster ask for the same node type. Without `seed`, the run keeps the background pairs it drew before the stop. - **The cache is deleted** once the priority files are written. - **A cache from other inputs** (other counts, metadata or `infmat.method`) is not used. It is moved to `cache.old`, with a warning; rename it back to continue the run it belongs to. - **If the cache cannot be written** (e.g. a full disk), scoring goes on without it. `cache=FALSE` turns it off. - `prioritize_lr` keeps its cache in `/cache`, and none with `outdir=NULL`. On a cluster, a job can therefore ask for less time than the whole run. Submit the same script again until it finishes: ```{r, eval=FALSE} res <- HyperCom(lrdb=cellchat.mm.lrdb, counts=counts, metadata=cell.meta, sample="sample", group="condition", significance=TRUE, seed=1, parallel=TRUE, outdir="overview") ``` ## 6. Step by step: how HyperCom works This section runs each step of `HyperCom` on its own, with `outfile=NULL` so nothing is written. ### Filter the counts To save memory, keep only the genes of the database that are expressed (above 0). The matrix is also transposed to cells x genes. ```{r} lr.counts <- filter_counts(counts=counts, lrdb=lrdb, outfile=NULL) dim(lr.counts) ``` Then keep only the interactions whose genes are all expressed. ```{r} lrs <- generate_lrs_table(counts=lr.counts, lrdb=lrdb, outfile=NULL) dim(lrs) ``` ### Build the hypergraph A hyperedge links any number of nodes. HyperCom makes two types: 1. **expression edges:** a ligand or receptor gene and the cells that express it, 2. **interaction edges:** the genes of one ligand-receptor interaction. They are stored as a data frame with an edge and a node column (a node belongs to that edge), each edge with weight 1. ```{r} hyperedges <- generate_hyperedges(counts=lr.counts, lrs=lrs, outfile=NULL) head(hyperedges, 2) ``` The adjacency matrix has the cells, ligand genes and receptor genes as nodes; each entry is the number of hyperedges two nodes share. It is not stored in full: `adj` keeps which nodes are in which hyperedge, which is much smaller, and works like a matrix (`dim(adj)`, `adj[i, j]`, `as.matrix(adj)` for the whole matrix). ```{r} adj <- generate_adjacency_matrix(hyperedges=hyperedges, outfile=NULL) dim(adj) adj[c("Thbs1", "Sdc4"), 1:3] ``` ### Compute the influence matrix A random walk with restart on the adjacency matrix gives the influence matrix: entry (i, j) is the share of a walk started at node i that ends at node j, so each row sums to 1. At every step the walk returns to its start with the restart probability `r`, so about `r` of each node's influence stays on itself and the rest spreads over the nodes it shares hyperedges with. `generate_infMat` takes `r` (default 0.5); `HyperCom` always uses 0.5. On this example the ranking hardly depends on it: `r` = 0.1 and 0.8 give the same top 20 pairs as 0.5. Like `adj`, `infMat` works like a matrix without being stored in full. ```{r} infMat <- generate_infMat(adj=adj, r=0.5, outfile=NULL) dim(infMat) infMat[c("Thbs1", "Sdc4"), 1:3] ``` ### Prepare the metadata The matrices now have genes as well as cells, so the metadata gets a row per gene too, with `"gene"` in each column. To save memory, only the cell, sample and group columns are kept; `keep` adds others. ```{r} node.meta <- generate_metadata(infMat=infMat, metadata=cell.meta, sample="sample", group="condition", keep=NULL, outfile=NULL) head(node.meta) ``` ### Rank the pairs `prioritize_lr` ranks the pairs (section 3). The priority is the weighted geometric mean of five terms of the cells that express the pair's genes: their fraction of all cells, the smallest fraction of cells expressing one of its genes, the mean of each cell's larger score as sender or receiver, how evenly they split into senders and receivers, and how much they concentrate in some groups. `weights` is the weight of the first term; here two weights give two tables. ```{r} priority <- prioritize_lr(lrs=lrs, adj=adj, infMat=infMat, metadata=node.meta, weights=c(1, 0.5), parallel=TRUE, outdir=NULL, verbose=FALSE) head(priority[["1"]]) ``` ## Citation ```{r} citation("HyperCom") ``` ```{r} sessionInfo() ```