Scalable Object Persistence (SOP) Framework 

version 4.7 released today, 12/26/2014!

NOTE: version 4.7 features threaded Data Reliability - support for multi-threaded Stores & Transactions, large data "smart" file streaming, more efficient memory utilization & addition of VirtualCache, a .Net ObjectCache implementation for ultra high speed, low footprint, scalable caching!

SOP v5: Big Data Store on the Cloud!

SOP Next: Big Data.Net

SOP is a modern, scalable object management engine. Since it is a code library, it can be embedded to your application providing optimal performance due to drastically reduced overhead in data mapping.

It has following key characteristics: scalable, low-footprint & hierarchical B-Tree native* data containers. SOP provides alternative (complementary to RDBMS/NoSQL:Hadoop/MongoDB) avenues for data persistence. Algorithm & Data Structure were purposely designed to fully take advantage of underlying OS' asynchronous, unbuffered overlapping I/O.

SOP contains new, specialized algorithm for optimizing disk I/O. It sits right there in your App (data direct!), 'sports seamless RAM & disk virtualization using combined application and data persistence caching for greatly reduced serialization overheads. These proved to be a great recipe for providing massive scalability to the Application, as we've been seeing.

One of the many actual real world uses of SOP version 4.7x as a caching solution, is its bundled high speed, lightweight & scalable .Net ObjectCache implementation, VirtualCache!. This is a real, fully functional implementation using SOP code library.

*A B-Tree native data container means the data container (a.k.a. - Store) is organized using B-Tree data structures. SOP B-Tree implementations were fine tuned & optimized for use as the key driver of the data storage platform. All Store operations benefitted from this & just by using the Store assures your code to hit the B-Tree index because they are one and the same.

Performance Level

Click on above link to navigate to SOP Performance Level details page.


FREE: Free to use, reuse, extend, no royalty fee on redistribution, no licensing fee, etc...

However, we do accept donations. Money raised for this project will be used to send to college students with big potential to excel, especially in the field of Computer Science. We believe there is strength in numbers, 'together, we can make a difference. Help bring a student closer to his dreams, 'donate now.

Click on this link to donate: 

Release History

This is the 4th installment of the widely popular B-Tree Gold code library originally written in c++ & was released as Open Sourced on late '90s in programmer's haven and similar developer hub internet sites. In version 4.0 (released sometime 2009), the biggest functionalities comprising RAM & Disk virtualization were completed and got used by Customers since then. Today in version 4.5 we've added many key enhancements to further refine, optimize and streamline the code library & its performance, commemorating B-Tree Gold's 12th year anniversary, it was also released in the spirit of Open Source.

Use-Case Scenarios

With open sourcing c# compiler &, there will be lesser opposition in switching over and embracing the MS stack/.Net. In light of this, frameworks such as SOP which is one of the top of the line or top of the line modern data storage solutions, 'can offer big rewards for IT shops.

Real world use-case: create/host your website in, use Windows hosting w/ ASP.Net, use SOP to take advantage of 100GB of file system storage give away for free part of economy hosting. That means one can host a dynamic website w/ almost unlimited data storage capacity, with very affordable fees. :)

I'm actually prototyping it now, creating a sample website with this solution and soon to open source it as well, as a sample real world solution.

Anyway, 'just wanted to add some "beef" of what SOP brings over to the table. As another alternative, 'could be better in some use-case, e.g. - than typical MySQL, Sql Server data storage on popular hosting sites (such as godaddy).

Click on above "Use-Case Scenarios" link to navigate to other use cases' discussion page.


Click on above link to navigate to SOP logical and physical structure details page.

Store/Data Driver Algorithms

Click on above link to navigate to SOP's Store/Data Driver Algorithms details page.

Sample usage


using(var server = new ObjectServer("c:\\SopBin\\OServer.dta"))
    var store = server.StoreNavigator.GetStore<int, string>("SystemFile/MyStore");
    store.Add(1, "Hello World!");
    store.Add(2, "Foo Bar");

This is a very basic, sample usage of SOP framework. It will create (or open if existing) transaction protected SOP server data file containing a single data Store named "MyStore" (key is int, value is string type) in the SystemFile. This Store is (B-Tree) indexed as all Stores in SOP are, :). We are inserting following key/value pairs:

  • key: 1 value: "Hello World!"
  • key: 2 value: "Foo Bar"

At end of the block, we commit the Transaction. No need to define an index as the Store itself is the B-Tree index.

Another example for managing different kinds of objects in the same Store:

using(var server = new ObjectServer("c:\\SopBin\\OServer.dta"))
    var store = server.StoreNavigator.
GetStore<string, object>("SystemFile/MyStore");
new Person{ Firstname = "Joe", Lastname = "Moe" } );
new Address{ Street = "123 Love Lane",
City = "Fremont", State = "Ca" } );
    store.Add("JoeSS#", "603-45-1110");
server.Commit(); }

The code example shows inserting three kinds of objects with different string keys. A Person, an Address and a SS#. Person and Address structure definition was not shown for brevity.

Build Instructions

Click on above link to navigate to Build Instructions page.

Sample Programs

Click on above link to navigate to Sample programs page.

Last edited Dec 26, 2014 at 9:46 PM by grecinto, version 183