BOLDNODE (No-API
Offline Data
Explorer) is a sister package to
BOLDconnectR that provides local access to BOLD data
packages through DuckDB, enabling exploration and analysis of public
BOLD data without API limitations. BOLDNODE therefore,
offers an offline solution for working with large scale BOLD datasets
locally. Please refer to the package documentation https://github.com/sameerpadhye/BOLDNODE for more
details about the package.
This vignette demonstrates a complete workflow integrating the
search, collect, get_ and bcdm_to
functions in BOLDNODE for efficient and scalable
exploration of BOLD data packages.
As a case study, we use Canadian Cerambycidae (longhorn beetles) to illustrate how these functions can be combined to search, retrieve, and transform the BCDM data.
BIN × province occurrence matrix, calculate
the species richness estimates,pairwise beta diversity using the
vegan package and visualize the species X province
association via Non Metric Multidimensional Scaling (nMDS) using
vegan and ggplot2.DNAStringSet
object, calculate nucleotide base proportions using the
Biostrings package, align the sequences using
Muscle and visualize the alignment using ape
and phangorn.sf object and visualize their geographic distribution using
ggplot2 package.BOLDNODE and other
packagesThe latest (in development) version of BOLDNODE can be
installed via pak. Packages like Biostrings
require BiocManager since they are hosted by
Bioconductor.
BOLD data packages can be downloaded in the parquet
format from the BOLD website (https://boldsystems.org/data/data-packages/). Historical
Data of quarterly BOLD snapshots (each assigned a stable DOI to support
reproducible research) can also be accessed by scrolling down the same
webpage.
The path where the file is downloaded is saved as a variable that can
be used as the input for the search function.
bold_parquet_search queries the parquet data package
using user-defined search terms such as COI-5P for
marker type, Cerambycidae for taxonomic group, and
Canada for geographic region.
# Search the BOLD dataset
cerambycidae_search <- bold_parquet_search(
input.parquet = parquet_file,
taxonomy = "Cerambycidae",
geography = "Canada",
marker = "COI-5P"
)The total number of records is shown in the console.
#> The search has 3330 records in the dataset
The search result is then collected into memory (i.e., downloaded in
the R session) as a data frame using bold_search_collect
(Please note that only first 100 results and a few columns are shown
here).
get_concise_summary provides a summary of the search
results, including total records, unique species, BINs, countries,
institutes, markers, and marker length range.
get_bin_consensus computes the consensus taxonomic
identifications for each BIN in the Cerambycidae dataset. The function
works upwards through taxonomic ranks (i.e., from subspecies to kingdom)
to find the lowest concordant identification that satisfies the provided
threshold (Please note that only first 50 results are shown
here).
# Get BIN consensus with strict consensus threshold (1.0)
bin_consensus <- get_bin_consensus(
cerambycidae_search,
threshold = 1.0,
min.ids = 1
)
# View consensus at different taxonomic ranks
table(bin_consensus$concordant_rank)
#>
#> family genus species subfamily subspecies
#> 1 29 271 2 8
# Examine the results
DT::datatable(head(bin_consensus, 50),
options = list(
pageLength = 10,
scrollX = TRUE
)
)bcdm_to_occmatrix creates a site-by-taxon
occurrence matrix from the full Cerambycidae dataset, using
province.state as the geographic grouping variable and
species as the taxonomic rank (Please note only 20
species are shown here).
# Generate occurrence matrix at species level, grouped by province/state
occ_matrix <- bcdm_to_occmatrix(
cerambycidae_search,
kingdom = "Animalia",
taxon.rank = "species",
site.cat = "province.state"
)
# Visualize the output
DT::datatable(
head(occ_matrix, 20),
options = list(
pageLength = 10,
scrollX = TRUE
)
)The resulting occurrence matrix can be directly used as input
data for many functions in widely used packages like
vegan.
vegan)library(vegan)
species_richness_est <- poolaccum(occ_matrix)
species_richness_est
#> N S Chao Jackknife 1 Jackknife 2 Bootstrap
#> 3 114.70 509.2237 180.2000 210.6517 144.3219
#> 4 139.83 423.3368 223.1925 271.1842 176.4379
#> 5 159.58 421.0545 256.2280 315.6145 201.5049
#> 6 182.71 427.9597 290.7933 357.6440 229.5597
#> 7 203.13 435.3686 322.3843 396.9474 254.6864
#> 8 217.81 441.1566 343.1625 421.1004 272.0712
#> 9 236.93 443.3329 369.9878 451.1742 294.7678
#> 10 255.09 450.3986 394.0590 476.9186 315.8369
#> 11 273.11 469.0037 419.3100 506.0896 337.1291
#> 12 287.00 485.0655 438.2500 528.0227 353.3101Species based richness estimates
vegan)# Calculate pairwise beta diversity
beta_diversity <- vegdist(occ_matrix, method = "bray") |> round(2)
# Visualize
DT::datatable(
data.frame(as.matrix(beta_diversity)),
options = list(
pageLength = 10,
scrollX = TRUE
)
)Pairwise Bray Curtis dissimilarity between the provinces
vegan & ggplot2)library(ggrepel)
# Using the occurrence matrix generated above for nMDS
nmds <- metaMDS(
occ_matrix,
distance = "bray",
k = 2,
trymax = 100,
trace = FALSE
)
# Site scores
sites <- as.data.frame(scores(nmds, display = "sites"))
sites$Site <- rownames(sites)
# Species scores
species <- as.data.frame(scores(nmds, display = "species"))
species$Species <- rownames(species)
# Keep only the 40 species furthest from the origin
species$dist <- sqrt(species$NMDS1^2 + species$NMDS2^2)
species20 <- species[order(species$dist, decreasing = TRUE), ][1:40, ]
# Plot
ggplot() +
geom_point(
data = sites,
aes(NMDS1, NMDS2),
size = 4
) +
geom_text(
data = sites,
aes(NMDS1, NMDS2, label = Site),
size = 4,
alpha = 0.6,
vjust = -0.4
) +
geom_text_repel(
data = species20,
aes(NMDS1, NMDS2, label = Species),
colour = "red",
fontface = "bold",
size = 3,
max.overlaps = Inf
) +
coord_equal() +
theme_bw() +
labs(
title = "nMDS (Bray-Curtis)",
x = "NMDS1",
y = "NMDS2"
)
A nMDS plot showing the associations between cerambycid species and provinces (Only 40 species shown here for representation)
get_bin_reps sub-samples the search results by BIN and
selects representative records according to the provided criteria,
yielding a representative sample for downstream analyses. As we saw
above, some BINs include multiple species, therefore we will select
representatives for all BIN-taxon combinations. Here, we select up to
three records per BIN, prioritizing 658-bp sequences from specimens
identified via morphological examination. With
enforce.scientific = TRUE, interim or provisional names are
ignored as distinct taxa (e.g. “Sternidius sp. A” is treated the same as
“Sternidius”).
# Sample 3 records per BIN-taxon combination
bin_reps <- get_bin_reps(
bold.search.res = cerambycidae_search,
Nreps = 3,
by.tax = TRUE,
enforce.scientific = TRUE,
criteria = list(
seq_length = 658,
id_method = "Morphology",
vouchered = TRUE
)
)
# Compare sample to full dataset
n_bins <- length(unique(bin_reps$bin_uri))
cat(
"Sampled", nrow(bin_reps), "representatives from", n_bins, "BINs",
"(out of", tot_records, "total records)."
)
#> Sampled 1020 representatives from 311 BINs (out of 3330 total records).The actual number of sampled records varies according to BIN size and taxonomy, as we can see in the collected results (Please note only 50 records are shown here). In the case of BINs with multiple species, further analysis of barcode sequences often reveals small but consistent differences between them.
bcdm_to_dnastringset converts the search results into a
DNAStringSet object. First,
bold_parquet_search is used to filter the COI-5P
Clytus records from Canada. Additional search parameters are
put in to return only full length barcodes having very low (or none)
ambiguous bases (Only top three results displayed)
(Please note: The bcdm_to_dnastringset
function requires Biostrings to be installed
before use).
library(Biostrings)
# Filter for genus Clytus
clytus_search <- bold_parquet_search(
input.parquet = parquet_file,
taxonomy = "Clytus",
geography = "Canada",
marker = "COI-5P",
basecount = 658,
ambi.base.cutoff = "<1%"
)
# Collect the filtered data
clytus_data <- bcdm_to_dnastringset(clytus_search,
cols_for_seq_names = c("processid", "bin_uri", "species")
)
head(clytus_data, 3)
#> DNAStringSet object of length 3:
#> width seq names
#> [1] 658 GACCCTTTACTTTGTATTTGGAG...CAGTTCTATATCAACATCTATTT ASALC013-12|BOLD:...
#> [2] 658 GACCCTTTACTTTGTATTTGGAG...CAGTTCTATATCAACATCTATTT ASALC197-12|BOLD:...
#> [3] 658 AACCCTTTACTTCGTCTTTGGGG...CAATTCTATATCAACATCTATTT ASALC224-12|BOLD:...The DNAStringSet object can then be used in
various downstream analyses using packages like
Biostrings, muscle, ape
etc.
Biostrings)muscle and Biostrings)library(Biostrings)
library(muscle)
alignment_muscle <- muscle(clytus_data)
#>
#> MUSCLE v3.8.31 by Robert C. Edgar
#>
#> http://www.drive5.com/muscle
#> This software is donated to the public domain.
#> Please cite: Edgar, R.C. Nucleic Acids Res 32(5), 1792-97.
#>
#> file9149555b46f4 87 seqs, max length 658, avg length 658
#> 1034 MB(6%)00:00:00 Iter 1 0.03% K-mer dist pass 11034 MB(6%)00:00:00 Iter 1 13.09% K-mer dist pass 11034 MB(6%)00:00:00 Iter 1 26.15% K-mer dist pass 11034 MB(6%)00:00:00 Iter 1 39.21% K-mer dist pass 11034 MB(6%)00:00:00 Iter 1 52.27% K-mer dist pass 11034 MB(6%)00:00:00 Iter 1 65.33% K-mer dist pass 11034 MB(6%)00:00:00 Iter 1 78.40% K-mer dist pass 11034 MB(6%)00:00:00 Iter 1 91.46% K-mer dist pass 11034 MB(6%)00:00:00 Iter 1 100.00% K-mer dist pass 1
#> 1034 MB(6%)00:00:00 Iter 1 0.03% K-mer dist pass 21034 MB(6%)00:00:00 Iter 1 13.09% K-mer dist pass 21034 MB(6%)00:00:00 Iter 1 26.15% K-mer dist pass 21034 MB(6%)00:00:00 Iter 1 39.21% K-mer dist pass 21034 MB(6%)00:00:00 Iter 1 52.27% K-mer dist pass 21034 MB(6%)00:00:00 Iter 1 65.33% K-mer dist pass 21034 MB(6%)00:00:00 Iter 1 78.40% K-mer dist pass 21034 MB(6%)00:00:00 Iter 1 91.46% K-mer dist pass 21034 MB(6%)00:00:00 Iter 1 100.00% K-mer dist pass 2
#> 1034 MB(6%)00:00:00 Iter 1 1.16% Align node 1035 MB(6%)00:00:00 Iter 1 2.33% Align node1035 MB(6%)00:00:00 Iter 1 3.49% Align node1035 MB(6%)00:00:00 Iter 1 4.65% Align node1036 MB(6%)00:00:00 Iter 1 5.81% Align node1036 MB(6%)00:00:00 Iter 1 6.98% Align node1036 MB(6%)00:00:00 Iter 1 8.14% Align node1036 MB(6%)00:00:00 Iter 1 9.30% Align node1037 MB(6%)00:00:00 Iter 1 10.47% Align node1037 MB(6%)00:00:00 Iter 1 11.63% Align node1037 MB(6%)00:00:00 Iter 1 12.79% Align node1037 MB(6%)00:00:00 Iter 1 13.95% Align node1038 MB(6%)00:00:00 Iter 1 15.12% Align node1038 MB(6%)00:00:00 Iter 1 16.28% Align node1038 MB(6%)00:00:00 Iter 1 17.44% Align node1039 MB(6%)00:00:00 Iter 1 18.60% Align node1039 MB(6%)00:00:00 Iter 1 19.77% Align node1039 MB(6%)00:00:00 Iter 1 20.93% Align node1040 MB(6%)00:00:00 Iter 1 22.09% Align node1040 MB(6%)00:00:00 Iter 1 23.26% Align node1040 MB(6%)00:00:00 Iter 1 24.42% Align node1041 MB(6%)00:00:00 Iter 1 25.58% Align node1041 MB(6%)00:00:00 Iter 1 26.74% Align node1041 MB(6%)00:00:00 Iter 1 27.91% Align node1042 MB(6%)00:00:00 Iter 1 29.07% Align node1042 MB(6%)00:00:00 Iter 1 30.23% Align node1042 MB(6%)00:00:00 Iter 1 31.40% Align node1043 MB(6%)00:00:00 Iter 1 32.56% Align node1043 MB(6%)00:00:00 Iter 1 33.72% Align node1043 MB(6%)00:00:00 Iter 1 34.88% Align node1043 MB(6%)00:00:00 Iter 1 36.05% Align node1044 MB(6%)00:00:00 Iter 1 37.21% Align node1044 MB(6%)00:00:00 Iter 1 38.37% Align node1044 MB(6%)00:00:00 Iter 1 39.53% Align node1045 MB(6%)00:00:00 Iter 1 40.70% Align node1045 MB(6%)00:00:00 Iter 1 41.86% Align node1046 MB(6%)00:00:00 Iter 1 43.02% Align node1046 MB(6%)00:00:00 Iter 1 44.19% Align node1046 MB(6%)00:00:00 Iter 1 45.35% Align node1047 MB(6%)00:00:00 Iter 1 46.51% Align node1047 MB(6%)00:00:00 Iter 1 47.67% Align node1047 MB(6%)00:00:00 Iter 1 48.84% Align node1048 MB(6%)00:00:00 Iter 1 50.00% Align node1048 MB(6%)00:00:00 Iter 1 51.16% Align node1049 MB(6%)00:00:00 Iter 1 52.33% Align node1049 MB(6%)00:00:00 Iter 1 53.49% Align node1049 MB(6%)00:00:00 Iter 1 54.65% Align node1049 MB(6%)00:00:00 Iter 1 55.81% Align node1049 MB(6%)00:00:00 Iter 1 56.98% Align node1049 MB(6%)00:00:00 Iter 1 58.14% Align node1049 MB(6%)00:00:00 Iter 1 59.30% Align node1049 MB(6%)00:00:00 Iter 1 60.47% Align node1050 MB(6%)00:00:00 Iter 1 61.63% Align node1050 MB(6%)00:00:00 Iter 1 62.79% Align node1050 MB(6%)00:00:00 Iter 1 63.95% Align node1050 MB(6%)00:00:00 Iter 1 65.12% Align node1050 MB(6%)00:00:00 Iter 1 66.28% Align node1051 MB(6%)00:00:00 Iter 1 67.44% Align node1051 MB(6%)00:00:00 Iter 1 68.60% Align node1051 MB(6%)00:00:00 Iter 1 69.77% Align node1051 MB(6%)00:00:00 Iter 1 70.93% Align node1051 MB(6%)00:00:00 Iter 1 72.09% Align node1052 MB(6%)00:00:00 Iter 1 73.26% Align node1052 MB(6%)00:00:00 Iter 1 74.42% Align node1052 MB(6%)00:00:00 Iter 1 75.58% Align node1052 MB(6%)00:00:00 Iter 1 76.74% Align node1052 MB(6%)00:00:00 Iter 1 77.91% Align node1053 MB(6%)00:00:00 Iter 1 79.07% Align node1053 MB(6%)00:00:00 Iter 1 80.23% Align node1054 MB(6%)00:00:00 Iter 1 81.40% Align node1054 MB(6%)00:00:00 Iter 1 82.56% Align node1054 MB(6%)00:00:00 Iter 1 83.72% Align node1055 MB(6%)00:00:00 Iter 1 84.88% Align node1055 MB(6%)00:00:00 Iter 1 86.05% Align node1056 MB(6%)00:00:00 Iter 1 87.21% Align node1056 MB(6%)00:00:00 Iter 1 88.37% Align node1057 MB(6%)00:00:00 Iter 1 89.53% Align node1057 MB(6%)00:00:00 Iter 1 90.70% Align node1057 MB(6%)00:00:00 Iter 1 91.86% Align node1057 MB(6%)00:00:00 Iter 1 93.02% Align node1058 MB(6%)00:00:00 Iter 1 94.19% Align node1059 MB(6%)00:00:00 Iter 1 95.35% Align node1059 MB(6%)00:00:00 Iter 1 96.51% Align node1059 MB(6%)00:00:00 Iter 1 97.67% Align node1060 MB(6%)00:00:00 Iter 1 98.84% Align node1060 MB(6%)00:00:00 Iter 1 100.00% Align node1060 MB(6%)00:00:00 Iter 1 100.00% Align node
#> 1060 MB(6%)00:00:00 Iter 1 1.15% Root alignment1060 MB(6%)00:00:00 Iter 1 2.30% Root alignment1060 MB(6%)00:00:00 Iter 1 3.45% Root alignment1060 MB(6%)00:00:00 Iter 1 4.60% Root alignment1060 MB(6%)00:00:00 Iter 1 5.75% Root alignment1060 MB(6%)00:00:00 Iter 1 6.90% Root alignment1060 MB(6%)00:00:00 Iter 1 8.05% Root alignment1060 MB(6%)00:00:00 Iter 1 9.20% Root alignment1060 MB(6%)00:00:00 Iter 1 10.34% Root alignment1060 MB(6%)00:00:00 Iter 1 11.49% Root alignment1060 MB(6%)00:00:00 Iter 1 12.64% Root alignment1060 MB(6%)00:00:00 Iter 1 13.79% Root alignment1060 MB(6%)00:00:00 Iter 1 14.94% Root alignment1060 MB(6%)00:00:00 Iter 1 16.09% Root alignment1060 MB(6%)00:00:00 Iter 1 17.24% Root alignment1060 MB(6%)00:00:00 Iter 1 18.39% Root alignment1060 MB(6%)00:00:00 Iter 1 19.54% Root alignment1060 MB(6%)00:00:00 Iter 1 20.69% Root alignment1060 MB(6%)00:00:00 Iter 1 21.84% Root alignment1060 MB(6%)00:00:00 Iter 1 22.99% Root alignment1060 MB(6%)00:00:00 Iter 1 24.14% Root alignment1060 MB(6%)00:00:00 Iter 1 25.29% Root alignment1060 MB(6%)00:00:00 Iter 1 26.44% Root alignment1060 MB(6%)00:00:00 Iter 1 27.59% Root alignment1060 MB(6%)00:00:00 Iter 1 28.74% Root alignment1060 MB(6%)00:00:00 Iter 1 29.89% Root alignment1060 MB(6%)00:00:00 Iter 1 31.03% Root alignment1060 MB(6%)00:00:00 Iter 1 32.18% Root alignment1060 MB(6%)00:00:00 Iter 1 33.33% Root alignment1060 MB(6%)00:00:00 Iter 1 34.48% Root alignment1060 MB(6%)00:00:00 Iter 1 35.63% Root alignment1060 MB(6%)00:00:00 Iter 1 36.78% Root alignment1060 MB(6%)00:00:00 Iter 1 37.93% Root alignment1060 MB(6%)00:00:00 Iter 1 39.08% Root alignment1060 MB(6%)00:00:00 Iter 1 40.23% Root alignment1060 MB(6%)00:00:00 Iter 1 41.38% Root alignment1060 MB(6%)00:00:00 Iter 1 42.53% Root alignment1060 MB(6%)00:00:00 Iter 1 43.68% Root alignment1060 MB(6%)00:00:00 Iter 1 44.83% Root alignment1060 MB(6%)00:00:00 Iter 1 45.98% Root alignment1060 MB(6%)00:00:00 Iter 1 47.13% Root alignment1060 MB(6%)00:00:00 Iter 1 48.28% Root alignment1060 MB(6%)00:00:00 Iter 1 49.43% Root alignment1060 MB(6%)00:00:00 Iter 1 50.57% Root alignment1060 MB(6%)00:00:00 Iter 1 51.72% Root alignment1060 MB(6%)00:00:00 Iter 1 52.87% Root alignment1060 MB(6%)00:00:00 Iter 1 54.02% Root alignment1060 MB(6%)00:00:00 Iter 1 55.17% Root alignment1060 MB(6%)00:00:00 Iter 1 56.32% Root alignment1060 MB(6%)00:00:00 Iter 1 57.47% Root alignment1060 MB(6%)00:00:00 Iter 1 58.62% Root alignment1060 MB(6%)00:00:00 Iter 1 59.77% Root alignment1060 MB(6%)00:00:00 Iter 1 60.92% Root alignment1060 MB(6%)00:00:00 Iter 1 62.07% Root alignment1060 MB(6%)00:00:00 Iter 1 63.22% Root alignment1060 MB(6%)00:00:00 Iter 1 64.37% Root alignment1060 MB(6%)00:00:00 Iter 1 65.52% Root alignment1060 MB(6%)00:00:00 Iter 1 66.67% Root alignment1060 MB(6%)00:00:00 Iter 1 67.82% Root alignment1060 MB(6%)00:00:00 Iter 1 68.97% Root alignment1060 MB(6%)00:00:00 Iter 1 70.11% Root alignment1060 MB(6%)00:00:00 Iter 1 71.26% Root alignment1060 MB(6%)00:00:00 Iter 1 72.41% Root alignment1060 MB(6%)00:00:00 Iter 1 73.56% Root alignment1060 MB(6%)00:00:00 Iter 1 74.71% Root alignment1060 MB(6%)00:00:00 Iter 1 75.86% Root alignment1060 MB(6%)00:00:00 Iter 1 77.01% Root alignment1060 MB(6%)00:00:00 Iter 1 78.16% Root alignment1060 MB(6%)00:00:00 Iter 1 79.31% Root alignment1060 MB(6%)00:00:00 Iter 1 80.46% Root alignment1060 MB(6%)00:00:00 Iter 1 81.61% Root alignment1060 MB(6%)00:00:00 Iter 1 82.76% Root alignment1060 MB(6%)00:00:00 Iter 1 83.91% Root alignment1060 MB(6%)00:00:00 Iter 1 85.06% Root alignment1060 MB(6%)00:00:00 Iter 1 86.21% Root alignment1060 MB(6%)00:00:00 Iter 1 87.36% Root alignment1060 MB(6%)00:00:00 Iter 1 88.51% Root alignment1060 MB(6%)00:00:00 Iter 1 89.66% Root alignment1060 MB(6%)00:00:00 Iter 1 90.80% Root alignment1060 MB(6%)00:00:00 Iter 1 91.95% Root alignment1060 MB(6%)00:00:00 Iter 1 93.10% Root alignment1060 MB(6%)00:00:00 Iter 1 94.25% Root alignment1060 MB(6%)00:00:00 Iter 1 95.40% Root alignment1060 MB(6%)00:00:00 Iter 1 96.55% Root alignment1060 MB(6%)00:00:00 Iter 1 97.70% Root alignment1060 MB(6%)00:00:00 Iter 1 98.85% Root alignment1060 MB(6%)00:00:00 Iter 1 100.00% Root alignment1060 MB(6%)00:00:00 Iter 1 100.00% Root alignment
#> 1060 MB(6%)00:00:00 Iter 2 1.18% Refine tree 1060 MB(6%)00:00:00 Iter 2 2.35% Refine tree1060 MB(6%)00:00:00 Iter 2 3.53% Refine tree1060 MB(6%)00:00:00 Iter 2 4.71% Refine tree1060 MB(6%)00:00:00 Iter 2 5.88% Refine tree1060 MB(6%)00:00:00 Iter 2 7.06% Refine tree1060 MB(6%)00:00:00 Iter 2 8.24% Refine tree1060 MB(6%)00:00:00 Iter 2 9.41% Refine tree1060 MB(6%)00:00:00 Iter 2 10.59% Refine tree1060 MB(6%)00:00:00 Iter 2 11.76% Refine tree1060 MB(6%)00:00:00 Iter 2 12.94% Refine tree1060 MB(6%)00:00:00 Iter 2 14.12% Refine tree1060 MB(6%)00:00:00 Iter 2 15.29% Refine tree1060 MB(6%)00:00:00 Iter 2 16.47% Refine tree1060 MB(6%)00:00:00 Iter 2 17.65% Refine tree1060 MB(6%)00:00:00 Iter 2 18.82% Refine tree1060 MB(6%)00:00:00 Iter 2 20.00% Refine tree1060 MB(6%)00:00:00 Iter 2 21.18% Refine tree1060 MB(6%)00:00:00 Iter 2 22.35% Refine tree1060 MB(6%)00:00:00 Iter 2 23.53% Refine tree1060 MB(6%)00:00:00 Iter 2 24.71% Refine tree1061 MB(6%)00:00:00 Iter 2 25.88% Refine tree1061 MB(6%)00:00:00 Iter 2 27.06% Refine tree1061 MB(6%)00:00:00 Iter 2 28.24% Refine tree1061 MB(6%)00:00:00 Iter 2 29.41% Refine tree1061 MB(6%)00:00:00 Iter 2 30.59% Refine tree1061 MB(6%)00:00:00 Iter 2 100.00% Refine tree
#> 1061 MB(6%)00:00:00 Iter 2 1.15% Root alignment1061 MB(6%)00:00:00 Iter 2 2.30% Root alignment1061 MB(6%)00:00:00 Iter 2 3.45% Root alignment1061 MB(6%)00:00:00 Iter 2 4.60% Root alignment1061 MB(6%)00:00:00 Iter 2 5.75% Root alignment1061 MB(6%)00:00:00 Iter 2 6.90% Root alignment1061 MB(6%)00:00:00 Iter 2 8.05% Root alignment1061 MB(6%)00:00:00 Iter 2 9.20% Root alignment1061 MB(6%)00:00:00 Iter 2 10.34% Root alignment1061 MB(6%)00:00:00 Iter 2 11.49% Root alignment1061 MB(6%)00:00:00 Iter 2 12.64% Root alignment1061 MB(6%)00:00:00 Iter 2 13.79% Root alignment1061 MB(6%)00:00:00 Iter 2 14.94% Root alignment1061 MB(6%)00:00:00 Iter 2 16.09% Root alignment1061 MB(6%)00:00:00 Iter 2 17.24% Root alignment1061 MB(6%)00:00:00 Iter 2 18.39% Root alignment1061 MB(6%)00:00:00 Iter 2 19.54% Root alignment1061 MB(6%)00:00:00 Iter 2 20.69% Root alignment1061 MB(6%)00:00:00 Iter 2 21.84% Root alignment1061 MB(6%)00:00:00 Iter 2 22.99% Root alignment1061 MB(6%)00:00:00 Iter 2 24.14% Root alignment1061 MB(6%)00:00:00 Iter 2 25.29% Root alignment1061 MB(6%)00:00:00 Iter 2 26.44% Root alignment1061 MB(6%)00:00:00 Iter 2 27.59% Root alignment1061 MB(6%)00:00:00 Iter 2 28.74% Root alignment1061 MB(6%)00:00:00 Iter 2 29.89% Root alignment1061 MB(6%)00:00:00 Iter 2 31.03% Root alignment1061 MB(6%)00:00:00 Iter 2 32.18% Root alignment1061 MB(6%)00:00:00 Iter 2 33.33% Root alignment1061 MB(6%)00:00:00 Iter 2 34.48% Root alignment1061 MB(6%)00:00:00 Iter 2 35.63% Root alignment1061 MB(6%)00:00:00 Iter 2 36.78% Root alignment1061 MB(6%)00:00:00 Iter 2 37.93% Root alignment1061 MB(6%)00:00:00 Iter 2 39.08% Root alignment1061 MB(6%)00:00:00 Iter 2 40.23% Root alignment1061 MB(6%)00:00:00 Iter 2 41.38% Root alignment1061 MB(6%)00:00:00 Iter 2 42.53% Root alignment1061 MB(6%)00:00:00 Iter 2 43.68% Root alignment1061 MB(6%)00:00:00 Iter 2 44.83% Root alignment1061 MB(6%)00:00:00 Iter 2 45.98% Root alignment1061 MB(6%)00:00:00 Iter 2 47.13% Root alignment1061 MB(6%)00:00:00 Iter 2 48.28% Root alignment1061 MB(6%)00:00:00 Iter 2 49.43% Root alignment1061 MB(6%)00:00:00 Iter 2 50.57% Root alignment1061 MB(6%)00:00:00 Iter 2 51.72% Root alignment1061 MB(6%)00:00:00 Iter 2 52.87% Root alignment1061 MB(6%)00:00:00 Iter 2 54.02% Root alignment1061 MB(6%)00:00:00 Iter 2 55.17% Root alignment1061 MB(6%)00:00:00 Iter 2 56.32% Root alignment1061 MB(6%)00:00:00 Iter 2 57.47% Root alignment1061 MB(6%)00:00:00 Iter 2 58.62% Root alignment1061 MB(6%)00:00:00 Iter 2 59.77% Root alignment1061 MB(6%)00:00:00 Iter 2 60.92% Root alignment1061 MB(6%)00:00:00 Iter 2 62.07% Root alignment1061 MB(6%)00:00:00 Iter 2 63.22% Root alignment1061 MB(6%)00:00:00 Iter 2 64.37% Root alignment1061 MB(6%)00:00:00 Iter 2 65.52% Root alignment1061 MB(6%)00:00:00 Iter 2 66.67% Root alignment1061 MB(6%)00:00:00 Iter 2 67.82% Root alignment1061 MB(6%)00:00:00 Iter 2 68.97% Root alignment1061 MB(6%)00:00:00 Iter 2 70.11% Root alignment1061 MB(6%)00:00:00 Iter 2 71.26% Root alignment1061 MB(6%)00:00:00 Iter 2 72.41% Root alignment1061 MB(6%)00:00:00 Iter 2 73.56% Root alignment1061 MB(6%)00:00:00 Iter 2 74.71% Root alignment1061 MB(6%)00:00:00 Iter 2 75.86% Root alignment1061 MB(6%)00:00:00 Iter 2 77.01% Root alignment1061 MB(6%)00:00:00 Iter 2 78.16% Root alignment1061 MB(6%)00:00:00 Iter 2 79.31% Root alignment1061 MB(6%)00:00:00 Iter 2 80.46% Root alignment1061 MB(6%)00:00:00 Iter 2 81.61% Root alignment1061 MB(6%)00:00:00 Iter 2 82.76% Root alignment1061 MB(6%)00:00:00 Iter 2 83.91% Root alignment1061 MB(6%)00:00:00 Iter 2 85.06% Root alignment1061 MB(6%)00:00:00 Iter 2 86.21% Root alignment1061 MB(6%)00:00:00 Iter 2 87.36% Root alignment1061 MB(6%)00:00:00 Iter 2 88.51% Root alignment1061 MB(6%)00:00:00 Iter 2 89.66% Root alignment1061 MB(6%)00:00:00 Iter 2 90.80% Root alignment1061 MB(6%)00:00:00 Iter 2 91.95% Root alignment1061 MB(6%)00:00:00 Iter 2 93.10% Root alignment1061 MB(6%)00:00:00 Iter 2 94.25% Root alignment1061 MB(6%)00:00:00 Iter 2 95.40% Root alignment1061 MB(6%)00:00:00 Iter 2 96.55% Root alignment1061 MB(6%)00:00:00 Iter 2 97.70% Root alignment1061 MB(6%)00:00:00 Iter 2 98.85% Root alignment1061 MB(6%)00:00:00 Iter 2 100.00% Root alignment1061 MB(6%)00:00:00 Iter 2 100.00% Root alignment
#> 1061 MB(6%)00:00:00 Iter 2 100.00% Root alignment
#> 1061 MB(6%)00:00:00 Iter 3 1.17% Refine biparts1061 MB(6%)00:00:00 Iter 3 1.75% Refine biparts1061 MB(6%)00:00:00 Iter 3 2.34% Refine biparts1061 MB(6%)00:00:00 Iter 3 2.92% Refine biparts1061 MB(6%)00:00:00 Iter 3 3.51% Refine biparts1061 MB(6%)00:00:00 Iter 3 4.09% Refine biparts1061 MB(6%)00:00:00 Iter 3 4.68% Refine biparts1061 MB(6%)00:00:00 Iter 3 5.26% Refine biparts1061 MB(6%)00:00:00 Iter 3 5.85% Refine biparts1061 MB(6%)00:00:00 Iter 3 6.43% Refine biparts1061 MB(6%)00:00:00 Iter 3 7.02% Refine biparts1061 MB(6%)00:00:00 Iter 3 7.60% Refine biparts1061 MB(6%)00:00:00 Iter 3 8.19% Refine biparts1061 MB(6%)00:00:00 Iter 3 8.77% Refine biparts1061 MB(6%)00:00:00 Iter 3 9.36% Refine biparts1061 MB(6%)00:00:00 Iter 3 9.94% Refine biparts1061 MB(6%)00:00:00 Iter 3 10.53% Refine biparts1061 MB(6%)00:00:00 Iter 3 11.11% Refine biparts1061 MB(6%)00:00:00 Iter 3 11.70% Refine biparts1061 MB(6%)00:00:00 Iter 3 12.28% Refine biparts1061 MB(6%)00:00:00 Iter 3 12.87% Refine biparts1061 MB(6%)00:00:00 Iter 3 13.45% Refine biparts1061 MB(6%)00:00:00 Iter 3 14.04% Refine biparts1061 MB(6%)00:00:00 Iter 3 14.62% Refine biparts1061 MB(6%)00:00:00 Iter 3 15.20% Refine biparts1061 MB(6%)00:00:00 Iter 3 15.79% Refine biparts1061 MB(6%)00:00:00 Iter 3 16.37% Refine biparts1061 MB(6%)00:00:00 Iter 3 16.96% Refine biparts1061 MB(6%)00:00:00 Iter 3 17.54% Refine biparts1061 MB(6%)00:00:00 Iter 3 18.13% Refine biparts1061 MB(6%)00:00:00 Iter 3 18.71% Refine biparts1061 MB(6%)00:00:00 Iter 3 19.30% Refine biparts1061 MB(6%)00:00:00 Iter 3 19.88% Refine biparts1061 MB(6%)00:00:00 Iter 3 20.47% Refine biparts1061 MB(6%)00:00:00 Iter 3 21.05% Refine biparts1061 MB(6%)00:00:00 Iter 3 21.64% Refine biparts1061 MB(6%)00:00:00 Iter 3 22.22% Refine biparts1061 MB(6%)00:00:00 Iter 3 22.81% Refine biparts1061 MB(6%)00:00:00 Iter 3 23.39% Refine biparts1061 MB(6%)00:00:00 Iter 3 23.98% Refine biparts1061 MB(6%)00:00:00 Iter 3 24.56% Refine biparts1061 MB(6%)00:00:00 Iter 3 25.15% Refine biparts1061 MB(6%)00:00:00 Iter 3 25.73% Refine biparts1061 MB(6%)00:00:01 Iter 3 26.32% Refine biparts1061 MB(6%)00:00:01 Iter 3 26.90% Refine biparts1061 MB(6%)00:00:01 Iter 3 27.49% Refine biparts1061 MB(6%)00:00:01 Iter 3 28.07% Refine biparts1061 MB(6%)00:00:01 Iter 3 28.65% Refine biparts1061 MB(6%)00:00:01 Iter 3 29.24% Refine biparts1061 MB(6%)00:00:01 Iter 3 29.82% Refine biparts1061 MB(6%)00:00:01 Iter 3 30.41% Refine biparts1061 MB(6%)00:00:01 Iter 3 30.99% Refine biparts1061 MB(6%)00:00:01 Iter 3 31.58% Refine biparts1061 MB(6%)00:00:01 Iter 3 32.16% Refine biparts1061 MB(6%)00:00:01 Iter 3 32.75% Refine biparts1061 MB(6%)00:00:01 Iter 3 33.33% Refine biparts1061 MB(6%)00:00:01 Iter 3 33.92% Refine biparts1061 MB(6%)00:00:01 Iter 3 34.50% Refine biparts1061 MB(6%)00:00:01 Iter 3 35.09% Refine biparts1061 MB(6%)00:00:01 Iter 3 35.67% Refine biparts1061 MB(6%)00:00:01 Iter 3 36.26% Refine biparts1061 MB(6%)00:00:01 Iter 3 36.84% Refine biparts1061 MB(6%)00:00:01 Iter 3 37.43% Refine biparts1061 MB(6%)00:00:01 Iter 3 38.01% Refine biparts1061 MB(6%)00:00:01 Iter 3 38.60% Refine biparts1061 MB(6%)00:00:01 Iter 3 39.18% Refine biparts1061 MB(6%)00:00:01 Iter 3 39.77% Refine biparts1061 MB(6%)00:00:01 Iter 3 40.35% Refine biparts1061 MB(6%)00:00:01 Iter 3 40.94% Refine biparts1061 MB(6%)00:00:01 Iter 3 41.52% Refine biparts1061 MB(6%)00:00:01 Iter 3 42.11% Refine biparts1061 MB(6%)00:00:01 Iter 3 42.69% Refine biparts1061 MB(6%)00:00:01 Iter 3 43.27% Refine biparts1061 MB(6%)00:00:01 Iter 3 43.86% Refine biparts1061 MB(6%)00:00:01 Iter 3 44.44% Refine biparts1061 MB(6%)00:00:01 Iter 3 45.03% Refine biparts1061 MB(6%)00:00:01 Iter 3 45.61% Refine biparts1061 MB(6%)00:00:01 Iter 3 46.20% Refine biparts1061 MB(6%)00:00:01 Iter 3 46.78% Refine biparts1061 MB(6%)00:00:01 Iter 3 47.37% Refine biparts1061 MB(6%)00:00:01 Iter 3 47.95% Refine biparts1061 MB(6%)00:00:01 Iter 3 48.54% Refine biparts1061 MB(6%)00:00:01 Iter 3 49.12% Refine biparts1061 MB(6%)00:00:01 Iter 3 49.71% Refine biparts1061 MB(6%)00:00:01 Iter 3 50.29% Refine biparts1061 MB(6%)00:00:01 Iter 3 50.88% Refine biparts1061 MB(6%)00:00:01 Iter 3 51.46% Refine biparts1061 MB(6%)00:00:01 Iter 3 52.05% Refine biparts1061 MB(6%)00:00:01 Iter 3 52.63% Refine biparts1061 MB(6%)00:00:01 Iter 3 53.22% Refine biparts1061 MB(6%)00:00:01 Iter 3 53.80% Refine biparts1061 MB(6%)00:00:01 Iter 3 54.39% Refine biparts1061 MB(6%)00:00:01 Iter 3 54.97% Refine biparts1061 MB(6%)00:00:01 Iter 3 55.56% Refine biparts1061 MB(6%)00:00:01 Iter 3 56.14% Refine biparts1061 MB(6%)00:00:01 Iter 3 56.73% Refine biparts1061 MB(6%)00:00:01 Iter 3 57.31% Refine biparts1061 MB(6%)00:00:01 Iter 3 57.89% Refine biparts1061 MB(6%)00:00:01 Iter 3 58.48% Refine biparts1061 MB(6%)00:00:01 Iter 3 59.06% Refine biparts1061 MB(6%)00:00:01 Iter 3 59.65% Refine biparts1061 MB(6%)00:00:01 Iter 3 60.23% Refine biparts1061 MB(6%)00:00:01 Iter 3 60.82% Refine biparts1061 MB(6%)00:00:01 Iter 3 61.40% Refine biparts1061 MB(6%)00:00:01 Iter 3 61.99% Refine biparts1061 MB(6%)00:00:01 Iter 3 62.57% Refine biparts1061 MB(6%)00:00:01 Iter 3 63.16% Refine biparts1061 MB(6%)00:00:01 Iter 3 63.74% Refine biparts1061 MB(6%)00:00:01 Iter 3 64.33% Refine biparts1061 MB(6%)00:00:01 Iter 3 64.91% Refine biparts1061 MB(6%)00:00:01 Iter 3 65.50% Refine biparts1061 MB(6%)00:00:01 Iter 3 66.08% Refine biparts1061 MB(6%)00:00:01 Iter 3 66.67% Refine biparts1061 MB(6%)00:00:01 Iter 3 67.25% Refine biparts1061 MB(6%)00:00:01 Iter 3 67.84% Refine biparts1061 MB(6%)00:00:01 Iter 3 68.42% Refine biparts1061 MB(6%)00:00:01 Iter 3 69.01% Refine biparts1061 MB(6%)00:00:01 Iter 3 69.59% Refine biparts1061 MB(6%)00:00:01 Iter 3 70.18% Refine biparts1061 MB(6%)00:00:01 Iter 3 70.76% Refine biparts1061 MB(6%)00:00:01 Iter 3 71.35% Refine biparts1061 MB(6%)00:00:01 Iter 3 71.93% Refine biparts1061 MB(6%)00:00:01 Iter 3 72.51% Refine biparts1061 MB(6%)00:00:01 Iter 3 73.10% Refine biparts1061 MB(6%)00:00:01 Iter 3 73.68% Refine biparts1061 MB(6%)00:00:01 Iter 3 74.27% Refine biparts1061 MB(6%)00:00:01 Iter 3 74.85% Refine biparts1061 MB(6%)00:00:01 Iter 3 75.44% Refine biparts1061 MB(6%)00:00:01 Iter 3 76.02% Refine biparts1061 MB(6%)00:00:01 Iter 3 76.61% Refine biparts1061 MB(6%)00:00:01 Iter 3 77.19% Refine biparts1061 MB(6%)00:00:01 Iter 3 77.78% Refine biparts1061 MB(6%)00:00:01 Iter 3 78.36% Refine biparts1061 MB(6%)00:00:01 Iter 3 78.95% Refine biparts1061 MB(6%)00:00:01 Iter 3 79.53% Refine biparts1061 MB(6%)00:00:01 Iter 3 80.12% Refine biparts1061 MB(6%)00:00:01 Iter 3 80.70% Refine biparts1061 MB(6%)00:00:01 Iter 3 81.29% Refine biparts1061 MB(6%)00:00:01 Iter 3 81.87% Refine biparts1061 MB(6%)00:00:01 Iter 3 82.46% Refine biparts1061 MB(6%)00:00:01 Iter 3 83.04% Refine biparts1061 MB(6%)00:00:01 Iter 3 83.63% Refine biparts1061 MB(6%)00:00:01 Iter 3 84.21% Refine biparts1061 MB(6%)00:00:01 Iter 3 84.80% Refine biparts1061 MB(6%)00:00:01 Iter 3 85.38% Refine biparts1061 MB(6%)00:00:01 Iter 3 85.96% Refine biparts1061 MB(6%)00:00:01 Iter 3 86.55% Refine biparts1061 MB(6%)00:00:01 Iter 3 87.13% Refine biparts1061 MB(6%)00:00:01 Iter 3 87.72% Refine biparts1061 MB(6%)00:00:01 Iter 3 88.30% Refine biparts1061 MB(6%)00:00:01 Iter 3 88.89% Refine biparts1061 MB(6%)00:00:01 Iter 3 89.47% Refine biparts1061 MB(6%)00:00:01 Iter 3 90.06% Refine biparts1061 MB(6%)00:00:01 Iter 3 90.64% Refine biparts1061 MB(6%)00:00:01 Iter 3 91.23% Refine biparts1061 MB(6%)00:00:01 Iter 3 91.81% Refine biparts1061 MB(6%)00:00:01 Iter 3 92.40% Refine biparts1061 MB(6%)00:00:01 Iter 3 92.98% Refine biparts1061 MB(6%)00:00:01 Iter 3 93.57% Refine biparts1061 MB(6%)00:00:01 Iter 3 94.15% Refine biparts1061 MB(6%)00:00:01 Iter 3 94.74% Refine biparts1061 MB(6%)00:00:01 Iter 3 95.32% Refine biparts1061 MB(6%)00:00:01 Iter 3 95.91% Refine biparts1061 MB(6%)00:00:01 Iter 3 96.49% Refine biparts1061 MB(6%)00:00:01 Iter 3 97.08% Refine biparts1061 MB(6%)00:00:01 Iter 3 97.66% Refine biparts1061 MB(6%)00:00:01 Iter 3 98.25% Refine biparts1061 MB(6%)00:00:01 Iter 3 98.83% Refine biparts1061 MB(6%)00:00:01 Iter 3 99.42% Refine biparts1061 MB(6%)00:00:01 Iter 3 100.00% Refine biparts1061 MB(6%)00:00:01 Iter 3 100.58% Refine biparts1061 MB(6%)00:00:01 Iter 3 100.00% Refine bipartsape and phangorn)library(ape)
library(phangorn)
# Convert the alignment to DNABin
dna_bin <- as.DNAbin(alignment_muscle)
# Distance matrix using K80
dist_matrix <- dist.dna(
dna_bin,
model = "K80"
)
# Neighbor Joining tree
nj_tree <- nj(dist_matrix)
# midpoint rooting
nj_tree <- midpoint(nj_tree)
# Plot tree
plot(
nj_tree,
cex = 0.6,
main = "Neighbor-Joining Tree"
)
A simple NJ tree visualization of the aligned data
sf object for spatial mapping of
occurrences of the genus Monochamus# Filter for genus Monochamus
monochamus_search <- bold_parquet_search(
input.parquet = parquet_file,
taxonomy = "Monochamus",
geography = "Canada",
marker = "COI-5P"
)
# Collect the filtered data
monochamus_data <- bcdm_to_sf(monochamus_search)The sf object can be used directly for mapping
occurrences with external packages like ggplot2.
(Background maps can be created in many ways. The sf
library has been used in this case).
library(sf)
# Creating a background map using the maps package; some map_data country names (ID column) are changed to suit the BCDM country.ocean names
map_data <- st_as_sf(maps::map("world",
plot = FALSE,
fill = TRUE
)) %>%
filter(ID == "Canada")
# Convert the data to WGS84
map_data <- st_transform(
map_data,
4326
)
# Plot
map_plot <- ggplot() +
geom_sf(
data = map_data,
alpha = 0.3,
linewidth = 0.4
) +
geom_point(
data = monochamus_data,
mapping = aes(
x = lon,
y = lat
),
colour = "#011B26",
fill = "#F78E1E",
size = 3,
pch = 21
) +
theme_bw(base_size = 15) +
theme(panel.grid.major = element_line(
colour = "grey50",
size = 0.3,
linetype = 3
)) +
xlab("Longitude") +
ylab("Latitude") +
coord_sf(expand = FALSE) +
ggtitle("Distribution map")
map_plot
Occurrence map of Monochamus
Benchmarking used three representative queries:
Search (bold_parquet_search) and data retrieval
(bold_search_collect) were benchmarked separately using the
rbenchmark package with three replications on the 30th
June 2026 data releaseß(https://doi.org/10.5883/DP-BOLD_Public.30-Jun-2026).
Tests were run on a MacBook Air M2 (8-core CPU, 16 GB RAM).
Across the three datasets, bold_parquet_search completed in
1.1 - 1.3s, while bold_search_collect required
approximately 4 - 12 min, with runtime increasing as dataset size
increased.
BOLDNODE can efficiently search and retrieve very large
datasets; however, the data retrieval (collect) step is
constrained by the user’s available system memory, and collecting very
large query results may exceed the machine’s memory capacity.
BOLDconnectRFunctions from BOLDNODE can be combined with the analysis functions
from the BOLDconnectR package because BOLD data packages
are based on the Barcode Core Data Model (BCDM) (https://github.com/boldsystems-central/BCDM). This
shared data structure enables direct interoperability between packages
without requiring additional data conversion.