Database time is too far in the future.

 Hi

Unfortunately FP1 for Domino12.0.1 still contains some bugs.

I know about two different Domino-environments with 12.0.1 FP1 (I mean, two different companies) which constantly expirience issues after moving to Domino 12.0.1 and then to 12.0.1 FP1.

Before FP1 there were issues with DAOS.

Now with FP1 there are issues with other tasks (mainly Replicator) and all cases were accompanied by an error "The caller's SemWait timeout expired." for a specific database.

Here it is a common case:

Environment: two or more Domino-servers 12.0.1 FP1 in a cluster, Windows Server 2016 or higher

The case: you start to see errors like 

[1358:0002-135C] 16.07.2022 22:07:47 Unable to replicate <SERVER> <DATABASE>: The caller's SemWait timeout expired.

It is unclear why it started happening but some of the cases might be connected with a stuck nCompact.exe on this database and high CPU usage by MTA task. It is not clear though if nCompact hung up and caused the error "The caller's SemWait timeout expired." or if the error was the reason of  nCompact hanging.

There was no way to do anything about that - only Domino restart.

Sometimes Domino even refused to shutdown properly, most of the tasks successfully quit but you might not see the final message "server shutdown complete" so in some cases it was required to "kill server".

After server started again you could see a new error for the database, saying "<DATETIME> Database <DATABASE> time is too far in the future."

Check of database icon showed that its modification time was in fufure: 

Again, it was not clear how to fix that new error either:

- nFixup.exe worked but didn't fix anything

- it refused to be copy-style compacted with the same error that database time was in thefuture. 

- desig refresh didn't change anything

The only way was to delete replica and create it again by replicating from another clustermate. It happened several times for each company already.

One more interesting point about all this: several users (for whose maildatabases this issue happened) complained, that emails they deleted or which they moved to their custom folders in their private mailboxes suddenly returned back to Inbox. My guess is that users worked in another replica, then server restart resolved the error "The caller's SemWait timeout expired", and then cluster replicator replicated changes from the inaccessible earlier replica back to normal replicas. Since the inaccessibe earlier replica was more recent (because of time in the future), some or maybe even all changes which user made in a normal replica were overwriten with the acctually old data taken from the replica that was in the future.