1
Vote

Mega Store

description

Current Data Readers will scale because SOP supports "orchestrated", complete parallel Store Readers running independently across machines. But to accommodate massive, quick data inserts in the mega scale range, this B-Tree configuration will be needed to scale up with the challenge.

Implement a "Mega Store" that can scale to "inserting" hundreds of billions, trillions, quadrillions and upwards of records. Mega Store will be a container that is comprised of B-Tree Store members. It will be able to give more opportunities for parallelism not just on Readers, but even on Writers since parallel Writes among B-Tree members will be allowed.

On Clustered setup, members of the Mega Store can be distributed among member Host Server of the Cluster.

Sample typical configurations:
a.) Basic or typical configuration:
  • Accommodate 100 trillion records
  • 100,000 B-Tree Store members with each one containing 100 million records
b.) Mega scale configuration:
  • Accommodate tens/hundreds of quadrillion records & up
  • 50 million B-Tree Store members with each one containing about 500 million records
  • Cluster-distributed across 100K+ Servers with independent/multi-way I/O paths to storage centers

comments