Scalable Object Persistence (SOP) Framework 

version 4.7 released today, 12/26/2014!

Code with us on GitHub - https://github.com/SharedCode/Sop; SOP in EF - https://github.com/aspnet/EntityFramework/issues/1879 

NOTE: version 4.7 features threaded Data Reliability - support for multi-threaded Stores & Transactions, large data "smart" file streaming, more efficient memory utilization & addition of VirtualCache, a .Net ObjectCache implementation for ultra high speed, low footprint, scalable caching!

SOP v5: Big Data Store on the Cloud!

SOP Next: Big Data.Net

SOP is a modern, scalable, portable, lightweight & highly adaptive object management system packaged in a code library. What separates SOP from the rest of the pack is its proven adaptability and being written in managed code (c#), it can be adapted quite quickly and easily across diverse use-cases, e.g. - from very thin use as a RAM virtualizer/high performance persisted data cache or on the opposite end, used as a data mgmt platform for providing high performance, enterprise scalable/durable data storage solution that runs on a new env't like the Azure cloud to take advantage of massive distributed processing & geo-redundancy it provides. All these while achieving its primary goal of reducing (and thus, optimizing!) persistence layers involved in application data mgmt to bare minimum.

SOP provides alternative (complementary to RDBMS/NoSQL:Hadoop/MongoDB) avenues for data persistence. Algorithm & Data Structure were purposely designed to fully take advantage of underlying OS' asynchronous, unbuffered overlapping I/O.

SOP contains new, specialized algorithm for optimizing disk I/O. It sits right there in your App (data direct!), 'sports seamless RAM & disk virtualization using combined application and data persistence caching for greatly reduced serialization overheads. These proved to be a great recipe for providing massive scalability to the Application, as we've been seeing.

One of the many actual real world uses of SOP version 4.7x as a caching solution, is its bundled high speed, lightweight & scalable .Net ObjectCache implementation, VirtualCache!. This is a real, fully functional implementation using SOP code library.

SOP data stores are organized using B-Tree data structures, thus, inherently providing high speed data access/mgmt. SOP B-Tree implementations were fine tuned & optimized for use as the key driver of the data storage platform. All Store operations benefitted from this & just by using the Store assures code to hit the B-Tree index because they are one and the same. Also, a Store can contain sub-Store(s) allowing hierarchy or nesting of Stores for more flexibility and structure scalability.

Performance Level

Click on above link to navigate to SOP Performance Level details page.

Cost

FREE: Free to use, reuse, extend, no royalty fee on redistribution, no licensing fee, etc...

However, we do accept donations. Money raised for this project will be used to send to college students with big potential to excel, especially in the field of Computer Science. We believe there is strength in numbers, 'together, we can make a difference. Help bring a student closer to his dreams, 'donate now.

Click on this link to donate: https://www.paypal.com/cgi-bin/webscr?cmd=_s-xclick&hosted_button_id=TQ6ZW4SYBBNF2 

Release History

This is the 4th installment of the widely popular B-Tree Gold code library originally written in c++ & was released as Open Sourced on late '90s in programmer's haven and similar developer hub internet sites. In version 4.0 (released sometime 2009), the biggest functionalities comprising RAM & Disk virtualization were completed and got used by Customers since then. Today in version 4.5 we've added many key enhancements to further refine, optimize and streamline the code library & its performance, commemorating B-Tree Gold's 12th year anniversary, it was also released in the spirit of Open Source.

Use-Case Scenarios

With open sourcing c# compiler & asp.net/.net, there will be lesser opposition in switching over and embracing the MS stack/.Net. In light of this, frameworks such as SOP which is one of the top of the line or top of the line modern data storage solutions, 'can offer big rewards for IT shops.

Real world use-case: create/host your website in godaddy.com, use Windows hosting w/ ASP.Net, use SOP to take advantage of 100GB of file system storage godaddy.com give away for free part of economy hosting. That means one can host a dynamic website w/ almost unlimited data storage capacity, with very affordable fees. :)

I'm actually prototyping it now, creating a sample website with this solution and soon to open source it as well, as a sample real world solution.

Anyway, 'just wanted to add some "beef" of what SOP brings over to the table. As another alternative, 'could be better in some use-case, e.g. - than typical MySQL, Sql Server data storage on popular hosting sites (such as godaddy).

Click on above "Use-Case Scenarios" link to navigate to other use cases' discussion page.

Structure

Click on above link to navigate to SOP logical and physical structure details page.

Store/Data Driver Algorithms

Click on above link to navigate to SOP's Store/Data Driver Algorithms details page.

Sample usage

Code:

using(var server = new ObjectServer("c:\\SopBin\\OServer.dta"))
{
    var store = server.StoreNavigator.GetStore<int, string>("SystemFile/MyStore");
    store.Add(1, "Hello World!");
    store.Add(2, "Foo Bar");
    server.Commit();
}


This is a very basic, sample usage of SOP framework. It will create (or open if existing) transaction protected SOP server data file containing a single data Store named "MyStore" (key is int, value is string type) in the SystemFile. This Store is (B-Tree) indexed as all Stores in SOP are, :). We are inserting following key/value pairs:

  • key: 1 value: "Hello World!"
  • key: 2 value: "Foo Bar"

At end of the block, we commit the Transaction. No need to define an index as the Store itself is the B-Tree index.

Another example for managing different kinds of objects in the same Store:

using(var server = new ObjectServer("c:\\SopBin\\OServer.dta"))
{
    var store = server.StoreNavigator.
GetStore<string, object>("SystemFile/MyStore");
store.Add("PersonJoe",
new Person{ Firstname = "Joe", Lastname = "Moe" } );
store.Add("Address1",
new Address{ Street = "123 Love Lane",
City = "Fremont", State = "Ca" } );
    store.Add("JoeSS#", "603-45-1110");
server.Commit(); }

The code example shows inserting three kinds of objects with different string keys. A Person, an Address and a SS#. Person and Address structure definition was not shown for brevity.

Build Instructions

Click on above link to navigate to Build Instructions page.

Sample Programs

Click on above link to navigate to Sample programs page.

Last edited Apr 8 at 1:13 AM by grecinto, version 197