Previous Page TOC Next Page



— 15 —


Administering Replication


Replication is the process of updating databases on different servers. As an administrator, you will spend a significant amount of your time monitoring replications and fixing replication problems. A detailed understanding of Notes replication is invaluable when trying to accomplish these tasks. This chapter explains the detailed algorithms behind Notes replication, how to configure replication, and the likely sources of error.

When attempting to understand replication, try to keep in mind the original intent of the Notes developers. Notes considers all replicas to be equal, and by default there exists no "master copy" of a database. When there is no master database, any change to a database will eventually propagate to all existing replicas of that database. The Replicator's task is to ensure that, over time, all replicas of a database are identical. As you venture further from a network of identical databases, the administrative effort required to monitor and correct problems increases. Common variations from a network of identical databases include master design databases from which all design changes originate, and personalized databases containing a subset of the data available in a master database. These examples are covered later in this chapter, following a look inside Notes replication.

Inside Notes Replication


The Notes replicator has a few traps for the unwary. This section describes the algorithms used during server-to-server replication. Client-to-server replication is similar, with the main difference being that a human generally selects specific databases and initiates replication, rather than having these steps performed automatically. If you are running into problems, following the steps outlined here may highlight probable causes of error.

The Replicator task should be loaded when the Notes server is started. The Replicator remains idle until a scheduled replication is due or a replication request is received from the server console or from another server. If the replication request is received from another server, the Refuse Replication Requests flag is checked before beginning a replication. The Replicator builds a list of databases on the two servers, discarding any that have an internal "Do Not Replicate" flag set. In Notes Release 4, this list of databases is kept in memory to speed up this step. The Replicator checks the replication history for each database that the two servers have in common. The replication history contains information about the last time these two databases were replicated.

Next the Replicator scans all the documents in both databases that were modified (changed, added, deleted ) since the last replication. For each modified document, filtering criteria are used to determine if the modification should be replicated to the other database. The filtering criteria involve security settings, including ReaderNames fields, and all the selective replication settings. Table 15.1 lists each type of modification, along with the security and selective replication settings checked by the Replicator. The Replicator looks only in the primary Name and Address Book when determining security access.

Table 15.1. Filtering Criteria Used During Replication.

To Replicate...Security Setting RequiredSelective Replication Settings Required
Form, view, or macro changes Source server has designer or manager access in destination database Form, view, macro replication is always enabled
Access control list changes Source server has manager access in destination database ACL replication is enabled
Deletion of data document Source server must have editor access (or better) in the destination database; if document has a $Readers field, it lists the destination server Source server has "Do not replicate deletions" disabled
Addition of new data document Source server has author access or better to destination database; if note has a $Readers field, it lists the destination server Data in document passes selection formula stored in destination database
Changes to an existing data document Source server has editor access (or better) to destination database; if documen has a $Readers field, it lists the destination server Data in document passes selection formula stored in destination database

If the change passes the filtering criteria, the corresponding document (matching note ID) in the other database is checked.



NOTEDuring replication, documents are identified by their Notes ID. Every Notes document has a world-wide unique ID. All replicas of a document share this unique ID, which contains enough information to uniquely identify each document and determine its last modification date.

After both documents pass the filtering criteria, the modification dates of the two documents are compared. A replication conflict occurs if more than one user, each accessing a different replica of the database, edited the same field in the same document since the last replication. You can configure the Replicator to accept the last change (discarding at least one of the changes) or flag conflicts for resolution by the administrator whenever a replication conflict occurs. If no conflict occurs, the modification is replicated to the other database. If both databases are in Release 4 format, only the modified fields are replicated. If either database is in Release 3.x format, the entire document is exchanged. When replication completes successfully, the replication histories of both databases are updated. A time stamp is entered in each replica using the clock on the other server. If replication fails, no time stamps are issued and an entry is created in the servers’ Notes Log.

This method of replication offers several advantages over systems that rely on e-mail to propagate database changes:

Don't confuse Notes replication with transactions in a relational database management system (RDBMS). If a network connection is lost during replication, updates already performed aren't rolled back. No concept of a rollback exists in Notes. With an RDBMS, partial changes, which may span multiple disk writes, are either completed or rolled back. They are never left partially done.

Now that you understand the basics of Notes replication, let's examine all the ways to configure Notes replication.

Replicating Deletions


You may wonder how the Notes Replicator knows that a note was deleted. Whenever a note is deleted, it is replaced with a replication stub. The replication stub contains only enough information to determine the unique note ID. A deletion stub is just another note with a special class. While the Replicator is scanning all modified notes, the Replicator checks all new deletion stubs. Of course, after the deletion has been replicated to all databases, the deletion stub is just wasted space on your hard disk. Notes rids the database of deletion stubs by purging and compacting a database. Purging marks the deletion stub as free space and compacting reclaims the free space in a database.

One way to get replicas out of sync with each other is to purge a deletion stub before it has replicated to all databases. If Notes replicates with a database containing a modified copy of the note after the deletion stub for that note has been purged, the note is copied back into the database. It's as if the note was never deleted! Care should be taken when setting purge intervals so that all deletions have time to replicate. The default setting of 90 days is appropriate for most databases. Set the interval to 30 days for highly active databases. Even if you are going to use selective replication to disable deletion replication, set the purge interval as if you were replicating deletions. This can save you heartache down the road should you ever want to start replicating deletions.

Creating Replica Databases


Database replicas are identified by their replica IDs, not by file names. Although you don't need to force your replicas to share a common file name, I recommend that you do so. No other way is as convenient to determine when two databases are replicas of each other. Using common file names also prevents you from creating two replicas on the same server, something you should never do. Having more than one replica of a database on a server can lead to confusing and unpredictable results.

Two ways exist to create replicas. The first is to copy the database file, using an operating system command. For example, you could use the COPY command from a command line to create a new database file. Because the new file is an exact duplicate of the old file, it contains the same replica ID.

The second way to create database replicas is to use the Notes client interface. Follow these steps:

  1. Select the database icon from the workspace.
  2. Choose File | Replication | New Replica.
    The New Replica dialog box opens, as shown in Figure 15.1.

    Figure 15.1.
    Creating a replica by using the Notes client interface.

  3. Specify a server, database title, and file name for the new replica. You must change the server or the file name (or both).
  4. If you're creating a replica on another server, you have the option of creating the replica immediately, or at the next scheduled replication. If the other server is connected via a LAN or WAN connection, creating the replica immediately is the best choice. If the server is connected via a modem, select Next Scheduled Replication.
  5. Indicate whether the new replica should use the same ACL, and whether you want to create a full-text index for the new replica.
  6. Choose the desired encryption options for the new replica. If you're creating a replica on a laptop, you should strongly consider encrypting the new replica. Click the Encryption button and specify Strong, Medium, or Simple encryption. Then choose OK. (The advantages and disadvantages of these options are discussed fully in Chapter 18, "Administering Notes Security.") I recommend using strong encryption and seeing whether the performance is acceptable. If you find the performance to be unacceptable, create a second replica, using medium security.
  7. Choose a size limit for the new replica by clicking the Size Limit button. You can specify a 1, 2, 3, or 4 gigabyte limit.
  8. Set the replication options by clicking the Replication Settings button. These options are explained in full in the following section.
  9. Choose OK.

Notes creates your replica with the setting you specified.

Configuring Replication


You have many options to consider when configuring replication. You must choose a replication topology and a replication schedule. Notes also gives you many ways to customize replication. Most organizations don't need to use any of these options, but you may want to test each option to see how you might use it in your organization.

Replication Schedules


You can specify a time range or a specific time for replication to occur. When you specify a range, Notes begins to replicate at the start of the range and repeats replication after an interval that you specify. When you indicate a specific time for replication, Notes initiates replication and, if unsuccessful, retries for up to one hour.

Replication schedules are set using connection documents in the Public Address Book. You can create multiple connection documents for each set of servers, with each connection document specifying a different time range, day of the week, or set of databases. If you use multiple connection documents for a pair of servers, make sure that the time ranges don’t overlap. This error can lead to unpredictable results. I recommend using a single pair of connection documents for each pair of servers. Conflicting priorities and multiple time ranges lead to replication problems.

A few examples may help highlight the different ways to schedule replication. Figure 15.2 shows the most common type of replication (in my personal experience)—a time range with a repeat interval. In Figure 15.2, ServerA will connect to ServerB (the home server) at 8:00 a.m. If the connection is unsuccessful, ServerA will retry periodically until either a connection is made or the time interval ends. After a successful call, ServerA waits four hours before connecting again.

Figure 15.2.
Scheduling a replication time range and repeat interval.

Two ways are available to schedule a daily replication. You can use a time range without a repeat interval, or a specific time without a repeat interval. When you specify a time, Notes retries for one hour after a failed connection. When you specify a time range, Notes retries for the entire time range. Figure 15.3 shows how to use a time range without a repeat interval. Figure 15.3 is identical to Figure 15.2, except that the repeat interval has been deleted. In this case, ServerA connects to ServerB only once a day. If unsuccessful, ServerA repeats the attempt until the time range expires.

Figure 15.3.
When scheduling a daily replication, you can use a time range without a repeat interval.

Figure 15.4 shows how to schedule a daily replication, using a specific time without a repeat interval. Use this technique when you are sure that a one-hour repeat interval is enough to guarantee a successful replication.

Figure 15.4.
You can indicate a specific time when scheduling a daily replication.

You also can specify a list of specific times at which replication will occur. Simply enter a list of times separated by commas in the Call at Times field. This technique is used most often for low- and medium-priority databases.

Forcing Replication


You can initiate replication from the server console with the Replicate, Pull, and Push commands. Replicate forces a two-way replication. Pull replicates changes from the other server, and Push replicates changes from the current server.

Replication Topologies


Chapter 9, "Designing Your Notes Network," discusses the advantages and disadvantages of several replication topologies. The bottom line is that you should always use a hub-and-spoke topology for Notes replication. When you have fewer than five servers, the hub server also can serve as a database server for users. When you have six or more servers, you should use a dedicated hub server. A hub-and-spoke topology reduces the number of connection documents and thus is easier to administer. In a hub-and-spoke topology, the hub server initiates all replication with the spoke servers. The hub server is always listed as the source server and the spokes are always the destination servers.

When you choose a hub-and-spoke replication topology, you don’t have to disregard special cases totally. For instance, you might have a large reference database that exists on only two servers. Instead of taking disk space on the hub server, you can schedule a replication directly between the two servers and specify that only the reference database be replicated. One exception exists to this exception. If you back up databases only from the hub server (as opposed to backing up each server), you need to place a replica of every database on the hub server.

Multiple Replicators


You can propagate data faster by using multiple Replicators. Notes Release 4 enables you to configure up to four Replicator tasks per server. Each Replicator can be updating databases simultaneously with the other Replicators.

Customizing Replication


Notes enables you to specify subsets of data to replicate, and control whether certain database properties are replicated to or from a database. These options are set using the Replication Settings dialog box and the connection document in the Public Address book.

Using the Replication Settings dialog box (choose File | Replication | Settings), you can specify these options:

You need to specify these setting for each replica. They don't replicate from one database to another. You also can customize replication between a pair of servers, using the connection document. You can specify the replication type, the databases that should be replicated, and a time limit for replication.

The Replication Type field controls whether changes should be pushed or pulled from each server. Choose from four options: Pull-Pull, Pull-Push, Pull Only, and Push Only. With Pull-Pull replication, the source server pulls changes from the destination server and then sends a request to the destination server to pull changes from the source server. With Pull-Push replication (the default), the source server pushes changes to the destination server and then pulls changes from the destination server. The replicator on the destination server isn't involved. With Push Only and Pull Only, changes are replicated only in one direction. Push Only replicates changes from the source to the destination. Pull Only replicates changes from the destination to the source. Pull-Push replication is always used for hub-and-spoke topologies and is the default setting.

If you want to replicate a specific set of databases, you can specify the file names in the Files to Replicate field. If this field is blank, all databases in common are replicated.

The Replication Time Limit field limits the amount of time during which two servers can replicate. Simply enter the maximum number of minutes the two servers should spend replicating during one replication session. When the limit is reached, the source server ceases replication. This setting is useful when servers are replicating via modems. You can limit the number of minutes spent replicating during peak hours by setting a time limit. In this case, you should specify another off-peak connection document with no time limit, to guarantee that all changes are replicated within a day's time.

You also can customize replication by using selective replication, which is covered in the next section.

Selective Replication


Selective replication allows you to copy portions of a source database to a destination database. One often unexpected side effect of selective replication is the deletion of documents in the source database. Selective replication deletes any documents that don't pass the selection formula for a database, and it doesn't just filter incoming documents—it deletes documents from the database containing the selection formula. Only documents modified since the selection formula was changed are affected, because the Replicator only processes documents that have been modified since the last replication. Think of selective replication as a method of pruning a database, not as a way of copying subsets of data. Any application using selective replication should ensure that any document created in a database passes the selection formula. Otherwise, your users will surely wonder why their data is being deleted.

Selective replication allows you to

Although selective replication appears to be a good way to create personalized databases, I don't recommend using it this way. Besides the overhead of creating a selection formula for each person, it's just too easy to end up with deleted documents. Use selective replication to disable the replication of views, forms, macros, and ACLs. Use design templates instead to replicate design changes.

To set up selective replication, you use the Replication Settings dialog box. Follow these steps:

  1. Select the database icon from the workspace.
  2. Choose File | Replication | Settings.
    The Replication Settings dialog box appears.
  3. Click the Space Savers icon (it should already be selected). Alternatively, if you want to configure selective replication between a specific pair of servers, click the Advanced icon.
  4. If you are specifying selective replication for a pair of servers, specify the source and destination servers using the When Computer and Receives From drop-down lists.
  5. Select the Replicate a Subset of Documents check box.
    At this point, you need to decide whether you want to replicate documents from certain views, or you want to specify a selection formula. Replicate documents in a view only if you know that you want the documents in that view, and that this situation won't change even if the view selection formula changes. If you replicate based on a view simply to avoid retyping the selection formula, you may need to come back and update the selective replication setting whenever the view selection formula is updated.
  6. If you want to replicate documents that appear in a view, select that view from the list of views. You can select more than one view.
    or
    If you want to replicate based on a selection formula, select the Select by Formula check box and enter the selection formula.
  7. If you want to control whether to replicate design elements (forms, views, and so on), access control lists, agents, replication formulas, and deletions, click the Advanced icon (if you haven't already done so).
    If you want these settings to apply to all servers, select All Servers for both the When Computer and Receives From drop-down lists.
  8. Select the desired options under the Replicate Incoming section.
  9. Close the dialog box.

Replicating Design Changes


There are two methods for distributing design updates: replicating design changes directly from source database to destination database; or indirectly, using design templates. Figure 15.5 compares the data flow with both of these methods.

Figure 15.5.
a)Data flow through a Notes network that isn't using design templates; b)Data flow through a Notes network using design templates.

Introducing a level of indirection by using design templates offers several advantages. The primary advantage for most companies is that using design updates lets you set a more restrictive level of security on your databases. If you replicate design changes directly, all of your servers need at least designer access to all databases; if you replicate design changes with design templates, servers need only editor access to most databases and designer access only to templates. As designer access carries with it several privileges, restricting the number of databases allowing this access is beneficial. Of course, if you need to replicate ACL changes, you must give manager access to all servers and can't use this technique.

Design templates offer other advantages not related to replication, because they can serve as repositories of common forms, macros, and views. Design templates are also key in setting up special administrator views in databases, which relies on Notes' capacity for a single database to inherit designs from several templates.

If your database design guidelines require the use of design templates, an administrator can easily create a design template containing any special administrative views and macros. To customize a database with special administrator views, follow these steps:

  1. Create a new database, giving the database an .NTF extension.
  2. Choose File, Database, Information. Select Design Information and enter a name for this template. Check the option Database Is a Design Template.
  3. Create any special views, macros, or forms in the new template database.
  4. Copy any desired views, forms, or macros to the database you want to customize.
  5. Choose Design, Views; Design, Forms; and Design, Macros; and select the new views, forms, and macros from the list. Select Info and enter the name of the design template. Check the option Inherit from Template.
  6. Choose File, Database, Replication and check the Selective Replication settings. Make sure that replication of views, macros, and forms is disabled.

You can now use your own custom views, forms, and macros without fearing that these items will replicate to and take up space on every server in the network. The method used to create master design databases is the same as the method for adding custom administrative views and macros, with the addition of two extra steps. Make sure that the master design server has designer access to all replicas of the design template. All other servers should have only read access to the templates on the master server. Any changes to the design made on any server, other than the master design server, will eventually be overwritten by the design stored on the master server.

Design templates offer benefits both to an administrator and an application developer. I strongly recommend using design templates for all key databases.

Disabling Replication


You can disable replication of a database with the Notes user interface or with the utility described in the following steps. If you use the Notes interface to disable replication, you can turn it back on at any time. The utility disables replication in a way that can't be undone with the Notes interface, although you can use the utility to turn replication back on. Use the utility when you want to permanently disable the replication of a database. Otherwise, use the Notes user interface.

To disable replication, use this utility:

  1. Open an OS/2 command line.
  2. Type ReplSet database_name DISABLE and press Enter.

To use this utility to re-enable replication, follow these steps:

  1. Open an OS/2 command line.
  2. Type ReplSet database_name ENABLE and press Enter.

Listing 15.1 displays the complete listing of the source for this utility under OS/2. Don't worry if you're not a C programmer. The actual utility is already built and ready to run on the CD included with this book (the executable name is replset).

Listing 15.1. Source Listing for Replication Enable/Disable Utility


/*

 * Purpose: Turn replication ON/OFF for a given database

 *

 * Usage:

 *    Disable replication - ReplSet db_name ENABLE

 */

#include <stdlib.h>

#include <stdio.h>

#include <string.h>

/* Notes API include files */

#include <global.h>

#include <nsfdb.h>

#include <kfm.h>

#include <misc.h>

STATUS far PASCAL NotesMain(int argc, char *argv[])

{

 STATUS        error;

 DBHANDLE      db_handle = NULLHANDLE;

 DBREPLICAINFO replica_info;

 if (argc != 3)

 {

     printf ("Usage: replica database_name ENABLE|DISABLE\n");

     return FALSE;

 }

 /*

  * Open the database.

  */

 if (error = NSFDbOpen (argv[1], &db_handle))

 {

      printf ("Error on Database Open." );

      return FALSE;

 }

 if (error = NSFDbReplicaInfoGet(hdb, &replica_info))

 {

      printf ("Could not get Replica Info");

      NSFDbClose (db_handle);

      return FALSE;

 }

 if (strstr (strupr(argv[2]), "ENABLE") != NULL)

     /* turn replication on  */

     replica_info.Flags &= !(REPLFLG_NEVER_REPLICATE);

 else

     /* else turn it off     */

     replica_info.Flags |= REPLFLG_NEVER_REPLICATE;

 if (error = NSFDbReplicaInfoSet(db_handle, &replica_info))

 {

      printf ("Could not set Replica Info");

      NSFDbClose (db_handle);

      return FALSE;

 }

 NSFDbClose (db_handle);

 return (NOERROR);

}

Monitoring Notes Replication


One of the toughest jobs for an administrator of a distributed system is knowing when a problem is occurring. Ideally, an administrator discovers and corrects any problems before users are aware of or affected by the problem. Products exist today, such as NotesView by Lotus, that can tell you if the Replicator task is alive and if it is reporting any errors. Ensuring that the Replicator task is running isn't enough. Users want access to data, and the Notes administrator must make sure that all databases are receiving updates and that all replication conflicts are resolved.

If you aren't using one of the automated monitors, you should check the Notes Log's Replication Events view for each server at least daily. See Figure 15.6 for a snapshot of the Replication Events view.

Figure 15.6.
The Replication Events view of the Notes log shows all Replicator activity.

The Replication Events view displays all the replication log entries in the log. Figure 15.7 shows a replication log entry, displaying the number of documents deleted, added, and modified, along with the total duration of the replication.

Figure 15.7.
A replication log entry is created every time two databases are replicated.

As an administrator, you will save significant amounts of time if, over time, you develop a history for each database in your network. This history should show the expected number of modifications per day/hour/week along with the expected duration time. Any changes in the replication pattern may signal problems. For example, if the duration of a replication declines dramatically while the number of modifications remains stable, someone may have selected Truncate Attachments in the Selective Replication dialog box.

Troubleshooting Replications


Notes administrators eventually have to resolve inevitable replication problems. The good news is that most problems can be traced to incorrect security settings or incompatible network software. Before attempting to troubleshoot replication problems, be sure you have access to the following information:

You should have direct access to this information or the cooperation of someone who can provide this information quickly.

This is the general procedure you should follow when resolving replication problems:

  1. Determine the recent changes to the server or the affected database, and ask yourself the following questions:


    NOTEEven if a change seems unrelated to the current problem, always consider the most recent change first, as it is most often the cause of any problems.


  2. Determine as specifically as possible the symptoms of the problem.
  3. Isolate the problem. Is only one server or database not replicating correctly (check the ACL)? Is a particular set of servers or users unable to access the server (check your routers or Address Book)? Can you manually initiate either a pull or push replication from the server console (check your connection documents)?
  4. Resolve the problem (correct the relevant ACL, install a software fix, update the Public Address Book).

Use the following list of symptoms and possible causes when trying to resolve replication problems. Try to match your situation with one of the symptoms described here. This list certainly isn't exhaustive, so you may not find an exact match with your current situation. Use this list only as a guide for thinking about replication problems. Also, refer to Chapter 20, "Supporting Dial-Up Users," for supporting dial-up clients.

If none of the situations described here helps solve your problem, visit the Lotus Web site (www.lotus.com) and search their online help reference.

Resolving Replication and Save Conflicts


Notes doesn't enforce any kind of locking on documents. Two situations can arise where an administrator needs to tell Notes what to do. Multiple users can simultaneously edit the same field in the same document in the same database. In this case a save conflict occurs. A replication conflict occurs when multiple users edit the same field in the same document on replica copies of a database. In both of these situations, Notes can't tell which change to accept. Instead, Notes saves both changes and marks them for review by an administrator. Notes creates two documents, a main document and a response document, and places a diamond next to the response document. When a save conflict occurs, Notes stores the first document saved as the main document and all others as responses. When a replication conflict occurs, Notes makes the document with the most changes the main document (ties go to the first document saved) and all others are saved as responses.

The database manager for a database needs to monitor the database regularly for conflicts. Dealing with replication and save conflicts isn't difficult; open both documents, decide which change should be saved, copy the correct information to the main document, and delete the responses.

You also can design databases so that conflicts are minimized or eliminated. For example, you can have all edited documents saved as responses. But you shouldn't turn on this option just to avoid conflicts. If conflicts become a problem on a particular database, have the application developer write a macro that handles conflicts automatically.

Summary


Replication is a great tool. When it works, it helps you propogate changes to remote sites, load-balances servers at headquarters, and enables mobile users to work while disconnected from the network. Notes replication contains several features that make it efficient and easy (relatively speaking) to administer. When you configure replication, use the simplest approach. I always recommend using hub-and-spoke topology, even for the smallest networks. Keep the number of connection documents to a minimum, don't use selective replication unless you really must, and avoid changing the default replication settings without a very good reason. If you follow the guidelines outlined in this chapter, replication won't be a burden on your administrative staff.

Previous Page Page Top TOC Next Page