Skip to content

nishantgupta91-zz/Scalable-Storage-Service

Repository files navigation

Distributed System with Scalable Storage Service

A storage server architecture with a dynamically controllable, distributed storage service is developed. Data records (i.e., key-value pairs) are distributed over a number of storage servers by exploiting the capabilities of consistent hashing. A single storage server is only responsible for a subset of the whole data space (i.e., a range of successive hash values.) The hash function is used to determine the location of particular tuples (i.e., hash values of the associated keys).

The client library (KVStore) enables access to the storage service by providing the earlier defined KVStorage interface (connect, disconnect, get, put). Furthermore, the library keeps the distribution of data transparent to the client application. The client application only interacts with the storage service as a whole, while the library manages the communication with a single storage server. In order to forward requests to the server that is responsible for a particular tuple, the client library maintains metadata about the current state of the storage service. Metadata at the client side may be stale due to reorganizations within the storage service. Therefore the library has to process requests optimistically. If a client request is forwarded to the wrong storage node, the server will answer with an appropriate error and the most recent version of the metadata. After updating its metadata, the client will eventually retry the request (possibly contacting another storage server.)

Each storage server (KVServer) is responsible for a subset of the data according to its position in the hash space (the ring, as depicted in the below figure.) The position implicitly defines a subrange of the complete hash range.

The storage servers are monitored and controlled by an External Configuration Service (ECS). By means of this configuration service an administrator is able to initialize and control the storage system (i.e., add/remove storage servers and invoke reconciliation of metadata at the affected storage servers).

About

No description, website, or topics provided.

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages