Previous Next JavaScript must be enabled to correctly display this content

## Steps for Changing the Store's Topology

When you change your topology, you should go through these steps:

1. Make the Topology Candidate

2. Transforming the Topology Candidate

3. View the Topology Candidate

4. Validate the Topology Candidate

5. Preview the Topology Candidate

6. Deploy the Topology Candidate

7. Verify the Store's Current Topology

Creating a new topology is typically an iterative process, trying different options to see what is best before deploying changes. After trying options, examine the topology candidate and decide if it is satisfactory. If not, apply more transformations, or start over with different parameters. You can view and validate topology candidates to determine if they are appropriate.

The possible transformations to expand the store include redistributing data, increasing the replication factor, and rebalancing. These are described in Transforming the Topology Candidate.

You can also decrease the current topology by removing Storage Nodes. See Contracting a Topology.

The following sections walk you through the process of changing your store’s configuration using the Administration CLI.

### Make the Topology Candidate

To create the first topology candidate for an initial deployment, before any Replication Nodes exist, use the `topology create` command. The `topology create` command requires a topology name, a pool name and the number of partitions as arguments.

Note:

Avoid using the dollar sign ($) character in topology candidate names. The CLI displays a warning if you try to create or clone topologies whose names contain the reserved character.

For example:

    kv-> topology create -name firstTopo -pool BostonPool
    -partitions 300

Output:

    Created: firstTopo

Use the `plan deploy-topology` command to deploy this initial topology candidate without further transformations.

After your store is initially deployed, you can create candidate topologies with the `topology clone` command. The source of a clone can be another topology candidate, or the current, deployed topology. The `topology clone` command takes the following arguments:

- `-from <from topology>`

  The name of the source topology candidate.

- `-name <to topology>`

  The name of the clone.

For example:

    kv-> topology clone -from topo -name CloneTopo

Output:

    Created CloneTopo

This variant of the topology clone command that takes the following arguments:

- `-current`

  Specifies using the current deployed topology as a source, so the argument requires no name.

- `-name <to topology>`

  The name of the topology clone.

For example:

    kv-> topology clone -current -name ClonedTopo

Output:

    Created ClonedTopo

### Transforming the Topology Candidate

After you initially deploy your store, you can change it by deploying another topology candidate that differs from the current topology. This target topology is generated by transforming a topology candidate to expand the store by using these commands:

- `topology redistribute`

- `rebalance`

- `change-repfactor`

Alternatively, you can contract the target topology candidate using the `topology contract` command.

Transformations follow the topology rules described in the previous section.

The topology rebalance, redistribute or change-repfactor commands can only make changes to the topology candidate if there are additional, or changed, Storage Nodes available. It uses the new resources to rearrange Replication Nodes and partitions so the topology complies with the topology rules and the store improves on read or write throughput.

The following are scenarios in how you might expand or contract the store.

#### Increase Data Distribution

Use the `topology redistribute` command to increase data distribution to enhance write throughput. The redistribute command works only if new Storage Nodes are added to make creating new replication nodes possible for new shards. With new shards, the system distributes partitions across the new shards, resulting in more Replication Nodes to service write operations.

The following example demonstrates adding a set of Storage Nodes (`node04` — `node07`) and redistributing the data to those nodes. Four Storage Nodes are required to meet the zone's replication factor of four and the new shards require four nodes to satisfy the replication requirements:

    kv-> plan deploy-sn -zn zn1 -host node04 -port 5000 -wait
    Executed plan 7, waiting for completion...
    Plan 7 ended successfully

    kv-> plan deploy-sn -zn zn1 -host node05 -port 5000 -wait
    Executed plan 8, waiting for completion...
    Plan 8 ended successfully

    kv-> plan deploy-sn -zn zn1 -host node06 -port 5000 -wait
    Executed plan 9, waiting for completion...
    Plan 9 ended successfully

    kv-> plan deploy-sn -zn zn1 -host node07 -port 5000 -wait
    Executed plan 10, waiting for completion...
    Plan 10 ended successfully

    kv-> pool join -name BostonPool -sn sn4
    Added Storage Node(s) [sn4] to pool BostonPool
    kv-> pool join -name BostonPool -sn sn5
    Added Storage Node(s) [sn5] to pool BostonPool
    kv-> pool join -name BostonPool -sn sn6
    Added Storage Node(s) [sn6] to pool BostonPool
    kv-> pool join -name BostonPool -sn sn7
    Added Storage Node(s) [sn7] to pool BostonPool

    kv-> topology clone -current -name newTopo
    Created newTopo

    kv-> topology redistribute -name newTopo -pool BostonPool
    Redistributed: newTopo

    kv-> plan deploy-topology -name newTopo -wait
    Executed plan 11, waiting for completion...
    Plan 11 ended successfully

The redistribute command incorporates the new Storage Node capacity that you added to the BostonPool, and creates new shards. The command also migrates partitions to the new shards. If the number of new shards is less than or equal to the current number of shards, the `topology redistribute` command fails.

Note:

Do not execute the `topology redistribute` command against a store with mixed shards. A mixed shard store has shards whose Replication Nodes are operating with different software versions of Oracle NoSQL Database.

The system goes through these steps when it is redistributing a topology candidate:

1. The topology redistribute command creates new Replication Nodes (RNs) for each shard, assigning the nodes to Storage Nodes according to the topology rules. While creating new RNs, the topology command might move existing RNs to different Storage Nodes, to best use available resources while complying with the topology rules.

2. The topology command distributes Partitions evenly among all shards. The partitions in over populated shards are moved to shards with the least number of partitions.

   You cannot specify which partitions the command moves.

#### Increase Replication Factor

You can use the `topology change-repfactor` command to increase the replication factor. Increasing the replication factor creates more copies of the data and improves read throughput and availability. More Replication Nodes are added to each shard so that it has the requisite number of nodes. The new Replication Nodes are populated from existing nodes in the shard. Since every shard in a zone has the same replication factor, and a large number of shards, this command may require a significant number of new Storage Nodes to succeed.

For additional information on how to identify your primary replication factor and to understand the implications of the factor value, see Replication Factor.

The following example increases the replication factor of the store to 4. The administrator deploys a new Storage Node and adds it to the Storage Node pool. The admin then clones the existing topology and transforms it to use a new replication factor of 4.

    kv-> plan deploy-sn -zn zn1 -host node08 -port 5000 -wait
    Executed plan 12, waiting for completion...
    Plan 12 ended successfully

    kv-> pool join -name BostonPool -sn sn8
    Added Storage Node(s) [sn8] to pool BostonPool

    kv-> topology clone -current -name repTopo
    Created repTopo

    kv-> topology change-repfactor -name repTopo -pool BostonPool -rf 4 -zn zn1
    Changed replication factor in repTopo

    kv-> plan deploy-topology -name repTopo -wait
    Executed plan 13, waiting for completion...
    Plan 13 ended successfully

The `change-repfactor` command fails if either of the following occurs:

1. The new replication factor is less than or equal to the current replication factor.

2. The Storage Nodes specified by the storage node pool do not have enough capacity to host the required new Replication Nodes.

#### Balance a Non-Compliant Topology

Topologies must obey the rules described in Determining Your Store's Configuration. Changes to the physical characteristics of the store can cause the current store topology to violate those rules. For example, after performance tuning, you want to decrease the capacity of a Storage Node (SN). If that SN is already hosting the maximum permissible number of Replication Nodes, reducing its capacity will make the store non-compliant with the capacity rules. To decrease the capacity of an SN before using the `topology rebalance` command , use the `change-parameters` command for the storage node capacity. See plan change-parameters.

You can balance a non-compliant configuration using the `topology rebalance` command. This command requires a topology candidate name and a Storage Node pool name.

Before rebalancing your topology, use the `topology validate` command for any violations to the topology rules in your `repTopo` plan:

    kv-> topology validate -name repTopo

Output:

    Validation for topology candidate "repTopo":
    4 warnings.
    sn7 has 0 RepNodes and is under its capacity limit of 1
    sn8 has 0 RepNodes and is under its capacity limit of 1
    sn5 has 0 RepNodes and is under its capacity limit of 1
    sn6 has 0 RepNodes and is under its capacity limit of 1

In this case, there are anticipated warnings, but you do not need improvements to the topology. However, if improvements are needed, then the `topology rebalance` command will move or create Replication Nodes, using the Storage Nodes in the BostonPool pool, to correct any violations. The command does not create additional shards under any circumstances. See Shard Capacity.

    kv-> topology rebalance -name repTopo -pool BostonPool

Output:

    Rebalanced: repTopo

If there are insufficient Storage Nodes, or if an insufficient storage directory size is allocated, the `topology rebalance` command may be unable to correct all violations. In that case, the command makes as much progress as possible, and warns of remaining issues.

#### Contracting a Topology

You can contract a topology by using the `topology contract` command. This command requires a topology candidate name and a Storage Node pool name. This command supports the removal of Storage Nodes and contracts the topology by relocating Replication Nodes, deleting shards, and migrating partitions.

Note:

Decreasing the replication factor is not currently supported. Also, Admin relocation is not supported. If an admin is present on a contracted Storage Node, the contraction operation will fail.

The following example contracts the topology by removing 3 Storage Nodes (sn2, sn5 and sn8). First, you clone the pool using the `pool clone command` and remove the Storage Nodes from the cloned pool using the `pool leave command`. Then, the topology is contracted and deployed using the contracted pool. Finally, the Storage Nodes can be removed using the `plan remove-sn` command. This command automatically stops Storage Nodes before removal.

    # Clone the existing Storage Node pool as to be contractedPool
    kv-> pool clone -name contractedPool -from AllStorageNodes
    Cloned pool contractedPool
    kv-> pool leave -name contractedPool -sn sn2
    Removed Storage Node(s) [sn2] from pool contractedPool
    kv-> pool leave -name contractedPool -sn sn5
    Removed Storage Node(s) [sn5] from pool contractedPool
    kv-> pool leave -name contractedPool -sn sn8
    Removed Storage Node(s) [sn8] from pool contractedPool

    # Generate a contracted candidate topology
    kv-> topology clone -current -name contractedTopology
    Created contractedTopology
    kv-> topology contract -name contractedTopology -pool contractedPool
    Contracted: contractedTopology

    # Deploy the contracted candidate topology as the real topology.
    kv-> plan deploy-topology -name contractedTopology -wait
    Executed plan 16, waiting for completion...
    Plan 16 ended successfully

    # Remove to-be-deleted SNs
    kv-> plan remove-sn -sn sn2 -wait
    Executed plan 17, waiting for completion...
    Plan 17 ended successfully
    kv-> plan remove-sn -sn sn5 -wait
    Executed plan 18, waiting for completion...
    Plan 18 ended successfully
    kv-> plan remove-sn -sn sn8 -wait
    Executed plan 19, waiting for completion...
    Plan 19 ended successfully

### View the Topology Candidate

You can view details of the topology candidate or a deployed topology by using the `topology view` command. The command takes a topology name as an argument. With the topology view command, you can view all at once: the store name, number of partitions, shards, replication factor, host name and capacity in the specified topology.

For example:

    kv-> topology view -name repTopo

Output:

    store=mystore  numPartitions=300 sequence=315
      zn: id=zn1 name=Boston repFactor=4 type=PRIMARY

      sn=[sn1]  zn:[id=zn1 name=Boston] node01:5000 capacity=1
        [rg1-rn1]
      sn=[sn2]  zn:[id=zn1 name=Boston] node02:5000 capacity=1
        [rg1-rn2]
      sn=[sn3]  zn:[id=zn1 name=Boston] node03:5000 capacity=1
        [rg1-rn3]
      sn=[sn4]  zn:[id=zn1 name=Boston] node04:5000 capacity=1
        [rg1-rn4]
      sn=[sn5]  zn:[id=zn1 name=Boston] node05:5000 capacity=1
      sn=[sn6]  zn:[id=zn1 name=Boston] node06:5000 capacity=1
      sn=[sn7]  zn:[id=zn1 name=Boston] node07:5000 capacity=1
      sn=[sn8]  zn:[id=zn1 name=Boston] node08:5000 capacity=1

      shard=[rg1] num partitions=300
        [rg1-rn1] sn=sn1
        [rg1-rn2] sn=sn2
        [rg1-rn3] sn=sn3
        [rg1-rn4] sn=sn4

### Validate the Topology Candidate

You can validate the topology candidate or a deployed topology by using the `topology validate` command. The topology validate command takes a topology name as an argument. If no topology is specified, the current topology is validated. Validation makes sure that the topology candidate obeys the topology rules described in Determining Your Store's Configuration. Validation generates "violations" and "notes".

Violations are issues that can cause problems and should be investigated.

Notes are informational and highlight configuration oddities that may be potential issues, but may be expected.

For example:

    kv-> topology validate -name repTopo
    Validation for topology candidate "repTopo":
    4 warnings.
    sn7 has 0 RepNodes and is under its capacity limit of 1
    sn8 has 0 RepNodes and is under its capacity limit of 1
    sn5 has 0 RepNodes and is under its capacity limit of 1
    sn6 has 0 RepNodes and is under its capacity limit of 1

### Preview the Topology Candidate

You should preview the changes that would be made for the specified topology candidate relative to a starting topology. You use the `topology preview` command to do this. This command takes the following arguments:

- name

  A string to identify the topology.

- start <from topology>

  If -start topology name is not specified, the current topology is used. This command should be used before deploying a new topology.

For example:

    kv-> topology clone -current -name redTopo
    Created redTopo
    kv-> topology redistribute -name redTopo -pool BostonPool
    Redistributed: redTopo
    kv-> topology preview -name redTopo
    Topology transformation from current deployed topology to redTopo:
    Create 1 shard
    Create 4 RNs
    Migrate 150 partitions

    shard rg2
      4 new RNs: rg2-rn1 rg2-rn2 rg2-rn3 rg2-rn4
      150 partition migrations
    kv-> topology validate -name redTopo
    Validation for topology candidate "redTopo":
    No problems

### Deploy the Topology Candidate

When your topology candidate is satisfactory, use the Admin service to generate and execute a plan that migrates the store to the new topology.

Deploy the topology candidate with the `plan deploy-topology` command, which takes a topology name as an argument.

While the plan is executing, you can monitor the plan's progress. You have several options:

- The plan can be interrupted then retried, or canceled.

- Other, limited plans may be executed while a transformation plan is in progress to deal with ongoing problems or failures.

By default, the `plan deploy-topology` command will not deploy a topology candidate if deployment would introduce new violations of the topology rules. You can override this behavior using the optional `-force` plan flag. Do not use the `–force` plan without consideration. Introducing a topology rule violation can have many adverse effects.

The next example shows the topology differences before and after plan deployment. The first `show topology` output lists four Storage Nodes running in Zone 1, with one shard (`rg1`) storing 300 partitions. Storage nodes `sn5 —sn8` are available.

After deploying the plan, the `show topology` output lists storage nodes `sn5 — sn8` as running. Another shard exists (`rg2`), and the partitions are split between the two shards, each with 150 partitions.

    kv-> show topology
    store=mystore  numPartitions=300 sequence=315
      zn: id=zn1 name=Boston repFactor=4 type=PRIMARY

      sn=[sn1]  zn=[id=zn1 name=Boston] node01:5000 capacity=1 RUNNING
        [rg1-rn1] RUNNING
               No performance info available
      sn=[sn2]  zn=[id=zn1 name=Boston] node02:5000 capacity=1 RUNNING
        [rg1-rn2] RUNNING
               No performance info available
      sn=[sn3]  zn=[id=zn1 name=Boston] node03:5000 capacity=1 RUNNING
        [rg1-rn3] RUNNING
               No performance info available
      sn=[sn4]  zn=[id=zn1 name=Boston] node04:5000 capacity=1 RUNNING
        [rg1-rn4] RUNNING
               No performance info available
      sn=[sn5]  zn=[id=zn1 name=Boston] node05:5000 capacity=1
      sn=[sn6]  zn=[id=zn1 name=Boston] node06:5000 capacity=1
      sn=[sn7]  zn=[id=zn1 name=Boston] node07:5000 capacity=1
      sn=[sn8]  zn=[id=zn1 name=Boston] node08:5000 capacity=1

      shard=[rg1] num partitions=300
        [rg1-rn1] sn=sn1
        [rg1-rn2] sn=sn2
        [rg1-rn3] sn=sn3
        [rg1-rn4] sn=sn4

    kv-> plan deploy-topology -name redTopo -wait
    Executed plan 14, waiting for completion...
    Plan 14 ended successfully

    kv-> show topology
    store=mystore  numPartitions=300 sequence=470
      zn: id=zn1 name=Boston repFactor=4 type=PRIMARY

      sn=[sn1]  zn:[id=zn1 name=Boston] node01:5000 capacity=1 RUNNING
        [rg1-rn1] RUNNING
              No performance info available
      sn=[sn2]  zn:[id=zn1 name=Boston] node02:5000 capacity=1 RUNNING
        [rg1-rn2] RUNNING
              No performance info available
      sn=[sn3]  zn:[id=zn1 name=Boston] node03:5000 capacity=1 RUNNING
        [rg1-rn3] RUNNING
              No performance info available
      sn=[sn4]  zn:[id=zn1 name=Boston] node04:5000 capacity=1 RUNNING
        [rg1-rn4] RUNNING
              No performance info available
      sn=[sn5]  zn:[id=zn1 name=Boston] node05:5000 capacity=1 RUNNING
        [rg2-rn1] RUNNING
              No performance info available
      sn=[sn6]  zn:[id=zn1 name=Boston] node06:5000 capacity=1 RUNNING
        [rg2-rn2] RUNNING
              No performance info available
      sn=[sn7]  zn:[id=zn1 name=Boston] node07:5000 capacity=1 RUNNING
        [rg2-rn3] RUNNING
              No performance info available
      sn=[sn8]  zn:[id=zn1 name=Boston] node08:5000 capacity=1 RUNNING
        [rg2-rn4] RUNNING
              No performance info available

      shard=[rg1] num partitions=150
        [rg1-rn1] sn=sn1
        [rg1-rn2] sn=sn2
        [rg1-rn3] sn=sn3
        [rg1-rn4] sn=sn4
      shard=[rg2] num partitions=150
        [rg2-rn1] sn=sn5
        [rg2-rn2] sn=sn6
        [rg2-rn3] sn=sn7
        [rg2-rn4] sn=sn8

### Verify the Store's Current Topology

You can verify the store's current topology by using the `verify` command. The verify command checks the current, deployed topology to make sure it adheres to the topology rules described in Determining Your Store's Configuration.

You should examine the new topology and decide if it is satisfactory. If it is not, you can apply more transformations, or start over with different parameters.

For example:

    kv-> verify configuration

Output:

    Verify: starting verification of store mystore based upon
        topology sequence #470
    300 partitions and 8 storage nodes
    Time: 2024-04-05 06:57:10 UTC   Version: 24.1.11
    See localhost:KVROOT/mystore/log/mystore_{0..N}.log for progress messages
    Verify: Shard Status: healthy:2 writable-degraded:0 read-only:0 offline:0
    Verify: Admin Status: healthy
    Verify: Zone [name=Boston id=zn1 type=PRIMARY allowArbiters=false masterAffinity=false]
        RN Status: online:8 offline:0 maxDelayMillis:0 maxCatchupTimeSecs:0
    Verify: == checking storage node sn1 ==
    Verify: Storage Node [sn1] on node01:5000
        Zone: [name=Boston id=zn1 type=PRIMARY allowArbiters=false masterAffinity=false]
        Status: RUNNING
        Ver: 24.1.11 2024-04-05 09:33:45 UTC  Build id: a72484b8b33c
    Verify:         Admin [admin1]          Status: RUNNING,MASTER
    Verify:         Rep Node [rg1-rn1]      Status: RUNNING,MASTER ...
    Verify: == checking storage node sn2 ==
    Verify: Storage Node [sn2] on node02:5000
        Zone: [name=Boston id=zn1 type=PRIMARY allowArbiters=false masterAffinity=false]
        Status: RUNNING
        Ver: 24.1.11 2024-04-05 09:33:45 UTC  Build id: a72484b8b33c
    Verify:         Rep Node [rg1-rn2]      Status: RUNNING,REPLICA ...
    Verify: == checking storage node sn3 ==
    Verify: Storage Node [sn3] on node03:5000
        Zone: [name=Boston id=zn1 type=PRIMARY allowArbiters=false masterAffinity=false]
        Status: RUNNING
        Ver: 24.1.11 2024-04-05 09:33:45 UTC  Build id: a72484b8b33c
    Verify:         Rep Node [rg1-rn3]      Status: RUNNING,REPLICA ...
    Verify: == checking storage node sn4 ==
    Verify: Storage Node [sn4] on node04:5000
        Zone: [name=Boston id=zn1 type=PRIMARY allowArbiters=false masterAffinity=false]
        Status: RUNNING
        Ver: 24.1.11 2024-04-05 09:33:45 UTC  Build id: a72484b8b33c
    Verify:         Rep Node [rg1-rn4]      Status: RUNNING,REPLICA ...
    Verify: == checking storage node sn5 ==
    Verify: Storage Node [sn5] on node05:5000
        Zone: [name=Boston id=zn1 type=PRIMARY allowArbiters=false masterAffinity=false]
        Status: RUNNING
        Ver: 24.1.11 2024-04-05 09:33:45 UTC  Build id: a72484b8b33c
    Verify:         Rep Node [rg2-rn1]      Status: RUNNING,MASTER ...
    Verify: == checking storage node sn6 ==
    Verify: Storage Node [sn6] on node06:5000
        Zone: [name=Boston id=zn1 type=PRIMARY allowArbiters=false masterAffinity=false]
        Status: RUNNING
        Ver: 24.1.11 2024-04-05 09:33:45 UTC  Build id: a72484b8b33c
    Verify:         Rep Node [rg2-rn2]      Status: RUNNING,REPLICA ...
    Verify: == checking storage node sn7 ==
    Verify: Storage Node [sn7] on node07:5000
        Zone: [name=Boston id=zn1 type=PRIMARY allowArbiters=false masterAffinity=false]
        Status: RUNNING
        Ver: 24.1.11 2024-04-05 09:33:45 UTC  Build id: a72484b8b33c
    Verify:         Rep Node [rg2-rn3]      Status: RUNNING,REPLICA ...
    Verify: == checking storage node sn8 ==
    Verify: Storage Node [sn8] on node08:5000
        Zone: [name=Boston id=zn1 type=PRIMARY allowArbiters=false masterAffinity=false]
        Status: RUNNING
        Ver: 24.1.11 2024-04-05 09:33:45 UTC  Build id: a72484b8b33c
    Verify:         Rep Node [rg2-rn4]      Status: RUNNING,REPLICA ...
    Verification complete, no violations.