Bucket data resiliency

Last Updated: Feb 12, 2018 08:01AM PST

Data resiliency is the ability of stored objects to recover and continue operating in the case of a failure. In NooBaa, there are two types of data resiliency, replication or erasure coding.
NooBaa objects are broken into parts. These parts are searched for duplications (de-dup) and are encrypted. The data resiliency policy affects how these parts are saved on the bucket resources.

Replication: Full copies of object parts. Identical parts are placed across storage nodes in different resources.

Recommended configuration is set to 3 copies.
This configuration results in:

  • Low rebuild time effort
    In order to rebuild a failed replica, a single healthy replica needs to be read

  • 200% storage overhead
    2 extra replicas are saved for each original object part
     
  • Quicker recovery in failure cases


​Erasure coding: Each object part is broken into fragments and encoded with redundant data pieces.
Recommended configuration is set to 4+2.
This configuration results in:

  • High rebuild time effort
    In order to rebuild a single fragment, the number of fragments which needs to be read is equal to the number of data fragments configured

  • 50% storage overhead
    The overhead is derived from the ratio between the parity and the data fragments​

     
  • Long recovery in failure cases

File parts or fragments distribution changes according to the resource type:

Resource type Replication Erasure Coding
Nodes pools Replicas on nodes pool are distributed between the different nodes in the pool according to the configured policy parameters Fragments resides in nodes pool are distributed between the different nodes in the pool according to the configured policy parameters
Cloud resource NooBaa considers cloud resource as resilient and keeps only one replica on this type of resource. NooBaa considers cloud resource as resilient and keeps only data fragments on this type of resource.
Internal storage NooBaa considers internal storage as resilient and keeps only one replica on this type of resource. NooBaa considers internal storage as resilient and keeps only data fragments on this type of resource.






















 
Did you find this article helpful?
195e69fe2c7a9be6b2b2aa6b87f970548@noobaa.desk-mail.com
https://cdn.desk.com/
false
desk
Loading
seconds ago
a minute ago
minutes ago
an hour ago
hours ago
a day ago
days ago
about
false
Invalid characters found
/customer/en/portal/articles/autocomplete