Haven Fundamentals Part Two
Haven consists of a daemon (referred to as havend) and a library for communicating with that daemon (referred to as libhaven). Individual computers will run a single instance of havend, and any other software (referred to as client software) running on those computers which requires access to the database utilizes the libhaven library to communicate with the local instance of havend.<!--more-->
<h3>havend</h3>
havend instances manage a local database of objects, and connect to (and are connected to by) havend instances on other computers. When a libhaven client, or a remote havend instance, requests an object from the local havend instance, that instance first checks its local database, and returns any match. If it does not find any matches, or if the request was not by UUID but by a key/value query, it forwards the query on to the other havend instances known to it. It determines what havend instances are known to it by introspecting its own local database for objects that specify an appropriate reference to another havend (i.e., IP address and port). havend instances can be externally (that is, external to its introspection) configured to automatically download references from remote havend instances that connect to it, only allow connections from remote havend instances for which it has a reference, and only consider references that are signed by a particular private key. While this does allow the creation of private Haven databases with certain configurations, unconfigured havend instances will contribute to the public database.
When a request is made for an object that does not exist in the local havend instance's database, and a remote havend instance provides it, the local havend instance will copy it into its local database. Depending on external (again, external to introspection) configuration, the havend instance can have such "cached" objects expire to conserve local storage for locally-generated objects, or only expire after an alloted cache size has been exceeded. This way, oft-accessed objects are replicated on many nodes, and provides redundancy against individual havend instance failures and increased access speed for the whole database. When multiple remote havend instances are capable of providing an object, the local havend instance can request pieces from each one for distributed copying. Once copied, the local havend instance becomes a provider for similar requests by remote havend instances. In this, Haven effectively duplicates the function of BitTorrent.
<h3>libhaven</h3>
Internally, havend instances provide the notion of "published" objects to local clients. Published objects are objects that follow the rules, and are immutable upon UUID assignment. Published objects are visible to remote havend instances, and contribute to the database. Unpublished objects do have UUIDs, but they are mutable prior to being published. The unpublished object space is provided as a "testing ground" for data structuring and relationship establishment prior to insertion into the database, and is only accessible by local libhaven clients. Unpublished objects can be long lived, in which case their unpublished status reflects their use as a private data store. Unpublished objects can still reference published objects, and other unpublished objects, and generally appear identical to published objects.
libhaven provides clients with APIs for finding and accessing objects, monitoring havend-to-havend copy progress, and editing unpublished objects. libhaven provides higher-level acknowledgement of pedigree objects, and can be provided with cryptographic information such as public and private keys to automate the creation of signature objects
It is encouraged that libhaven be wrapped in the creation of other libraries that provide higher-level support for specific data storage needs. As an example, I plan to wrap libhaven to provide an interface to standard hierarchical filesystem calls, so that applications designed around the hierarchical filesystem metaphor can work with objects in the Haven database.