Merging Large Data Sets
Overview
Data is typically read from a file or as the result of an HTTP GET request. In each case, the data is first loaded
into the browser as text (string). While the Javascript specification does not mandate a limit to the size a string can take,
in general, most browsers have a limit in the range of 500MB.
This means that to share data that is larger than the 500MB limit, one must split the data into smaller sized chunks, which
are loaded separately, and then merged.
Merging Data
Merging data refers to the situation where you have two array and you wish to create an array that contains
the items from each array concatenated together. This is simple using the
Javascript
spread operator. The spread operator is expressed as
three dots, i.e. "...". When three dots precedes an array, it represents the items of the array,
so that the following sample code represents an array with the elements of two arrays.
let data = [...data1, ...data2]
The following code demonstrates iterating over an array of arrays and concatting to a single array.
let data = [];
for(let set of datasets){
data = [...data, ...set]
}
Try it!
