Current Data Readers will scale because SOP supports "orchestrated", complete parallel Store Readers running independently across machines. But to accommodate
massive, quick data inserts
in the mega scale range
, this B-Tree configuration will be needed to scale up with the challenge.
Implement a "Mega Store" that can scale to "inserting" hundreds of billions, trillions, quadrillions and upwards of records. Mega Store will be a container that is comprised of B-Tree Store members. It will be able to give more opportunities
for parallelism not just on Readers, but even on Writers since parallel Writes among B-Tree members will be allowed.
On Clustered setup, members of the Mega Store can be distributed among member Host Server of the Cluster.
Sample typical configurations:
a.) Basic or typical configuration:
- Accommodate 100 trillion records
- 100,000 B-Tree Store members with each one containing 100 million records
b.) Mega scale configuration:
- Accommodate tens/hundreds of quadrillion records & up
- 50 million B-Tree Store members with each one containing about 500 million records
- Cluster-distributed across 100K+ Servers with independent/multi-way I/O paths to storage centers