Faster background screening on big samples (13% less time per screening batch on one sample of 30,000 cells), same results.
Faster scoring and screening (whole prioritize_lr with p-values 1.11-1.29x faster on 3,000-20,000 cells, 1.5x on one sample of
30,000 cells), same priorities, p-values and order on the bundled data.
The same results as 0.0.0.9000, much faster and with far less memory. On the bundled data every step, the cell scores, the
priorities and the seeded p-values are identical to 0.0.0.9000's. Two exceptions: the default influence matrix method "woodbury"
differs from 0.0.0.9000's by about 1e-15 (infmat.method="lu" gives its numbers exactly; on the bundled data the priorities and
p-values are the same either way), and pairs whose cells are perfectly balanced over groups get priority 0 where 0.0.0.9000 gave NaN.
HyperCom() and HyperCom_seurat() return, invisibly, a list of their results instead of 0: priority (one dataframe per weight),
metadata, infMat, adj and lrs, the objects also written to outdir (the files are as before). Results need not be read back:
res <- HyperCom(...), then res$priority[["1"]] or score_hypercom(res$infMat, res$metadata, "Thbs1", "Sdc4").message() instead of print(), so suppressMessages() or verbose=FALSE silences it: what each step
found, the pairs scored about every 5 s with the time left, and the top pair of each weight. The per-pair score lines are gone; the
scores are in the returned tables and the CSV files.dim,
dimnames, rownames, x[i, j] and as.matrix(); code that used them as base matrices (rowSums(adj), heatmaps) needs
as.matrix() first. .rds files of them need library(HyperCom) to be read. Plain matrices, e.g. files of earlier versions, work
everywhere as before.parallel=TRUE uses one thread per physical core and threads sets
the number. The cl argument of prioritize_lr, HyperCom and HyperCom_seurat is gone: remove cl= from calls (it now stops
with "unused argument"), and in calls by position drop the value given for cl, which would otherwise be read as outdir.generate_cell_info,
auto_grid_mem_gb, available_memory_gb, prioritize_lr(cores=, infMat.cell=, cell.info=), method="auto"). Its version is
higher, so update.packages() moves spatial (v1-fast) installs to 0.2.0.HyperCom_sce() runs the pipeline on a SingleCellExperiment, as HyperCom_seurat() does on a Seurat object: one assay
(assay="counts" by default) and the cell metadata (colData); sample, group and keep name columns of colData. Assays held
on disk (DelayedArray, HDF5Array) are read for the genes of lrdb only. Needs SummarizedExperiment (Bioconductor).prioritize_lr, HyperCom, HyperCom_seurat: threads (number of threads; below 1 stops, more than the available cores warns and
uses them all), seed (set before drawing the background pairs, so p-values are reproducible; ignored without significance; the session's
random numbers are restored afterwards) and verbose, after outdir.score_hypercom: parallel and threads as prioritize_lr (one thread by default). On 20,000 cells one pair took 4.6 s on 1 thread
and 0.9 s on 16. Scores are the same at any number of threads.generate_infMat: method ("woodbury", the default, "lu" or "chol"), hyperedges and threads; HyperCom and
HyperCom_seurat: infmat.method. On a plain adjacency matrix given without its hyperedges the default is "lu".prioritize_lr, HyperCom, HyperCom_seurat: cache (default TRUE). With an output directory, the pairs scored so far are
kept in <outdir>/cache (<outdir>/6.priority/cache for HyperCom) while pairs are scored, and the folder is removed once the
priority files are written. A run that stops (time limit of a cluster job, crash) continues where it stopped when the same call
is run again, losing at most the last batch of pairs, with the same results as a run that never stopped on the same type of
computer (another CPU can change the last digits, about 1e-13); without seed it keeps
the background pairs drawn before the stop. A cache made from other inputs is not used and is moved to cache.old, with a
warning; a cache that cannot be written is skipped, with a warning. The cache holds 5
numbers per scored pair and 10 per screened background pair (about 200 KB for the bundled data with significance) and the names
of the cells (0.8 MB at 20,000 cells); on 20,000
cells its writes took 0.1 s of a 77 s scoring step, with the same peak memory.prioritize_lr documents its p-values: the fraction of 1000 background pairs scoring at least as high, in steps of 0.001; 0 means
none did.On 20,000 synthetic cells in 3 samples (16 threads, a tenth of the CellChat pairs, no p-values) the whole pipeline takes about 1.3 min and peaks at 3.5 GB; 0.0.0.9000 ran out of memory at this size. 50,000 cells take about 10 min and 17 GB. The bundled data take seconds.
significance, the 1000 background pairs are screened instead of scored: one pass over a background pair's cell grids bounds
its priority without ranking the cells, and only the few whose bounds hold a real pair's priority (about 1%) are scored in full.
The p-values are the same as with every background pair scored. On 16 threads a run with p-values took 1.7 s instead of 2.9 s on
the bundled data (12 s instead of 20 s on 1 thread), 31 s instead of 90 s on 3,000 cells, 85 s instead of 240 s on 5,000 cells
(one sample) and 8.4 min instead of 23 min on 20,000 cells in 3 samples, with no more memory. A stopped run keeps the screened
bounds in its cache too."woodbury" 0.4 s at 20,000 cells, "chol" 46 s with one dense node x node matrix, "lu" 105-121 s with
two (0.0.0.9000 held five). "chol" and "woodbury" differ from "lu" by about 1e-15 relative, which can reorder exactly tied
grid entries: on 20,000 cells the pairs keep their order and priorities move by at most 1.3e-13 ("woodbury") and 1.9e-14
("chol").generate_infMat saves the influence matrix uncompressed (gzip saved 7% of the file and took 50 times longer than writing it).Mistakes that ran to meaningless results, or failed late with unclear messages, now stop at once with a message that says what to fix. Results of valid inputs are unchanged.
HyperCom checks lrdb, the sample / group / keep columns, weights and infmat.method before any step runs (a misspelt
column used to fail after the influence matrix; lrdb and counts swapped took 27 s and 6 GB), and stops when no interaction of
lrdb has all its genes expressed, e.g. a human lrdb on mouse counts (it used to finish with an empty priority table).filter_counts stops on counts with genes as column names (transposed) and on negative values (scaled data, which ran as if every
value were expression).generate_metadata stops when a cell of the graph has no sample or group (they were turned into genes), when a sample or group is
named "gene", when a sample has fewer than 2 cells in the graph (its cells get no scores, and with significance every p-value
became NA), and when no node is a row name of the metadata; sample and group must be one column name each.generate_hyperedges stops on cells sharing a barcode (they were mangled with make.names and merged or lost).score_hypercom stops when given only one of ligand and receptors (a ligand alone gave the network score, receptors alone
all-zero scores), when ligand names more than one gene and when receptors is empty; score_hypercom and prioritize_lr name
genes missing from the graph instead of failing with a subscript error.prioritize_lr stops on weights that are not numbers of at least 0 (negative weights gave infinite priorities) or that would share
a name and an output file, on a threads that is not one number, and on an outdir that is not one string (or NULL).HyperCom leaves out cells of the counts that are not in the metadata, with a warning (they used to become gene nodes and be
drawn as background genes, which skewed the p-values), stops when no cell is, and warns when outdir already holds priority
files of other weights, which it keeps. filter_counts warns about gene names used by several rows (the first row is kept).stringsAsFactors=TRUE) are read by name; the adjacency matrix was
indexed by factor codes and gave wrong priorities without a warning, and an all-NA receptor column dropped every interaction.node / edge columns by name, so results no longer depend on the session's
collation.generate_metadata: a single keep column keeps its name; kept columns named cell, sample or group are renamed cell.1, ...
instead of breaking the join; only gene nodes are labelled "gene" (cells keep their missing keep values); factor keep columns
get a "gene" level instead of NA, also when kept twice; dates and other classed columns are kept as strings.generate_hyperedges takes the interaction names from the first column, whatever its name.filter_counts ignores missing counts in the gene means (it stopped with "subscript out of bounds") and works with a single gene;
generate_lrs_table keeps its columns when no interaction is present.HyperCom no longer changes the working directory; it writes into outdir, created recursively. prioritize_lr creates nested
output directories. With the cluster gone, it no longer stops a cluster it was given or defaults to an SGE qrsh one.HyperCom_seurat reads the layer asked for with SeuratObject 4 as well (its GetAssayData ignored layer and returned data).library(HyperCom) loads Matrix, so the bundled sparse counts can be subset straight away.HyperCom_seurat needs
it). Minimum versions: dplyr 1.1.0, parallelly 1.23.0 (older versions failed with cryptic errors).base-R branch. DESCRIPTION describes the method and has URL and BugReports; citation("HyperCom")
gives the bioRxiv preprint. The bundled datasets are documented.vignettes/HyperCom_Overview.Rmd, R Markdown instead of Quarto), so R CMD build renders it and R CMD check runs it; it needs
knitr, rmarkdown, ggplot2 and ggridges (Suggests). The rendered HTML and the outdated PDF are no longer in the repository.filter_counts returns a cell x gene matrix; generate_metadata returns a dataframe; the lrdb's first column may have any
name; vignette HyperCom_seurat example fixed.src/score_grids.cpp): an exact integer sort instead of percent_rank on n x n values; means summed in long double in
0.0.0.9000's order, so scores are bit for bit; about 10 bytes per within-sample entry for the grids in flight. Grids too big for the
cache are scored one at a time by all threads, with 8 bytes per entry (6-byte keys) and value buckets of about equal counts (bounds
from a histogram of a sample of the grid).generate_infMat: "lu" has LAPACK's dgesv (the routine solve() calls) factor the system in place; "chol" is HyperCom
0.1.0's Cholesky inverse (dpotrf + dpotri), bit for bit; "woodbury" uses the Woodbury identity on adj = H H' (H the node x
hyperedge incidence) and checks exactly that given hyperedges make the adjacency. The package links R's LAPACK (on Windows through
an import library for Rlapack.dll) and is compiled as C++17 on every R version.generate_adjacency_matrix keeps the incidence H; as.matrix(adj) accumulates the upper triangle in C++ (0.0.0.9000's matrix
exactly, 4x faster than xtabs + tcrossprod). A hypercom_influence keeps H, the row sums, delta and the hyperedge x node
matrix Z; its entries equal the dense "woodbury" matrix's bit for bit.R/preprocess.R, hypergraph.R, influence.R, metadata.R, score.R, prioritize.R,
pipeline.R, utils.R) instead of one R/functions.R; the code itself is unchanged.::: to the package's own functions; weighted.mean is called as stats::weighted.mean.Rcpp::unwindProtect (src/r_alloc.h), so an allocation error unwinds the
C++ stack (destructors run, inputs are unprotected) before R's error is raised. The grid kernel packs its 6-byte keys byte by
byte, so big grids rank the same on big-endian machines (a 6-byte memcpy of a uint64_t kept the wrong bytes there).