I do not recommend Domino 12.0.1 even with FP1 - still many issues

 Hi

Unfortunately FP1 for Domino12.0.1 still contains some important bugs.

I know about two different Domino-environments with 12.0.1 FP1 (I mean, two different companies) which constantly expirience issues after moving to Domino 12.0.1 and then to 12.0.1 FP1.

Before FP1 there were issues with DAOS.

Now with FP1 there are issues with other tasks (mainly Replicator) and all cases were accompanied by an error "The caller's SemWait timeout expired." for a specific database.

Here it is a common case:

Environment: two or more Domino-servers 12.0.1 FP1 in a cluster, Windows Server 2016 or higher

The case: you start to see errors like 

[1358:0002-135C] 16.07.2022 22:07:47 Unable to replicate <SERVER> <DATABASE>: The caller's SemWait timeout expired.

It is unclear why it started happening but some of the cases might be connected with a stuck nCompact.exe on this database and high CPU usage by MTA task. It is not clear though if nCompact hung up and caused the error "The caller's SemWait timeout expired." or if the error was the reason of  nCompact hanging.

There was no way to do anything about that - only Domino restart.

Sometimes Domino even refused to shutdown properly, most of the tasks successfully quit but you might not see the final message "server shutdown complete" so in some cases it was required to "kill server".

After server started again you could see a new error for the database, saying "<DATETIME> Database <DATABASE> time is too far in the future."

Check of database icon showed that its modification time was in fufure: 

Again, it was not clear how to fix that new error either:

- nFixup.exe worked but didn't fix anything

- it refused to be copy-style compacted with the same error that database time was in thefuture. 

- desig refresh didn't change anything

The only way was to delete replica and create it again by replicating from another clustermate. It happened several times for each company already.

One more interesting point about all this: several users (for whose maildatabases this issue happened) complained, that emails they deleted or which they moved to their custom folders in their private mailboxes suddenly returned back to Inbox. My guess is that users worked in another replica, then server restart resolved the error "The caller's SemWait timeout expired", and then cluster replicator replicated changes from the inaccessible earlier replica back to normal replicas. Since the inaccessibe earlier replica was more recent (because of time in the future), some or may be even all changes user made in normal replicas were overwriten with the acctually old data taken from the replica that was in the future.

Failed to create database, can not create multiple databases that differ only in case.

 Hi

A funny thing happened today.

I needed to work while I was on vacation and my Internet was bad: I needed to create a mail-database on the server.

Initially I tried to do it directly from Notes client installed on my laptop.

I started the creation process (everything was standard: Ctrl+N, chose server, mail template etc) but then I quickly realized that it would take ages. So I break the process by clicking Ctrl+Pause but I kept the creation dialog opened in Notes on my laptop to let me copy filename and database title from it later.

I connected to server via RDP, ran Notes client there, issued "dbcache flush" in Domino console and deleted the incomplete database created by my local Notes client. 

No issues so far.

Then I tried to create mail database from the Notes client on the server by using the same filepath and title which I copied from the still opened creation dialog window in my local Notes client. 

However, I got an error saying "Failed to create database, cannot create multiple databases that differ only in case." I quickly googled it and most of the articles I found suggested to issue "dbcache flush" and "sh nlcache reset" which didn't help me at all. 

I tried to create this mail-database on another server in the cluster - it worked. Then I tried to replicate it to the initial server and got the same error. All these things I made from Notes client installed on the server.

I was very confused.

You know what eventually resolved the problem? 

The trick was to close db creation dialog window in my local Notes client - after that the error had gone.

Reply/Forward icon in Mail database

 Hi

Recently we upgraded Notes to V12 and some of our users (which used non-English mail template) complained that they did not see that special icon in Mail views/folders telling which email was replied and/or forwarded.

Such icon is connected with the field $RespondedTo which is managed by Notes client itself, it is not created by some lotusscript or @formula in the mail database 

I noticed that this field was missing for those uses which had the issue with the icon.

After some investigation I discovered that this field is managed by Notes if this checkbox is enabled in user's maildb.

and it was disabled in mailboxes of the users complained about the issue.

The funny thing was that this checkbox was enabled in the mail template so I couldn't see the reasons why it wasn't inherited. To fix this issue I simply disabled and then enabled this checkbox back in the mail template and then ndesign.exe started to handle it correctly.

If you server hangs up during DAOS resync...

 Hi

Recently I worked on the issue with one Domino-server which became unresponsive after running program document with command "tell daosmgr resync force" at night. We noticed that in the morning only and you know how stessful it could be :)

I checked the server via Domino Console and it looked alive, it reacted to commands like "sh ta" or "sh ser", however Notes clients couldn't connect to it. I noticed also some other messages telling that something was wrong, like:

Error connecting to server <SERVERNAME>: Remote system no longer responding

Server task Router on <SERVERNAME> is no longer responding
Server task SMTP Server on <SERVERNAME> is no longer responding

I restarted server and when it went up I noticed following errors:

The DAOS catalog cannot be opened. DAOS cannot operate normally.: The integrity of a database storage container has been lost - the container will be rebuilt.

and also many errors like this:

The database D:\IBM\Domino\data\<database.nsf? was unable to write to file D:\HCL\DAOS\0001\912B6AA17EFDE1C9C12587E9002FDA051912B6AA17EFDE1P.nlo: Release 12.0.1 HF24|January 19, 2022

Without thinking twice I ran "tell daosmgr resync" manually and in 10 minutes the server hanged up again with the same symptoms - reacted to commands via Domino Console but Notes client couldn't connect.

I restarted server again, all the errors from the previous start were shown again, but that time I checked DAOS status and it was REBUILDING.

This time I decided to not run resync while catalog was rebuilding and I decided to wait until the evening and then try to do the resync agin. That time resync worked without issues, the only stupid thing I did - I didn't check the catalog status before the running resync, so I can't say if server did something during the day before I ran the resyn again in the evening.

I communicated this case with HCL and they told me that one of the solutions could be to resync the catalog while server is down by running "ndesign.exe resync" (the similar way you can compact names.nsf or log.nsf).

Both Domino 12.0.1 and IF1 contain bug "The caller's SemWait timeout expired."

Hi

Both Domino 12.0.1 and IF1 (HF11) contain bug "The caller's SemWait timeout expired."

You can see this error in Domino logs or if you configure a corresponding Probe in admin4.nsf (Probe Type: Database, Subtype: Error Monitoring).

There are different consequenses of this problem:

1. sometimes replication does not work properly
2. sometimes router can't send the email
3. ...I think there could be more issues, but the two first I expirienced myself...

The errors in log.nsf / console.log may look like:

20-01-2022 09:01:58 The DAOS catalog cannot be updated.: The caller's SemWait timeout expired.
20-01-2022 09:01:58 Router: Agent message: Notes error: The caller's SemWait timeout expired. 

The errors in ddm.nsf may look like:

Database object write error: E:\Data\mail\sxxxxt.nsf: The caller's SemWait timeout expired.

The only solution is to request another hotfix from IBM HCL (HF24) which includes IF1(HF11) + the fix for this problem (see DCKTCARNVR).

db.createdocumentcollection()

 Hi

This is unbelievable.

After 20 years of working with Notes I found out that there always was a method db.createdocumentcollection() for creating a blank collection.

Why the hell neither IBM nor HCL included it into documentation and made it public? 

Previously I always got an empty collection by either ran search with query which for sure would return nothing or used db.GetProfileDocCollection() - both these approaches worked well but made the code less obvious/self-explaining.

Dim s As New NotesSession
Dim db As notesdatabase
Dim col As notesdocumentcollection
Set db = s.currentdatabase
Set col = db.createdocumentcollection()

TSM failed after installing FP3 for Domino V11

 Hi

We installed FP3 for Domino V11 this summer and when we needed to restore some db from the backup TSM failed to do it (though it previously worked OK). The error was:

ACD0004E An unknown error has been detected.
Warning! Error initializing the connection to the TDP library
Could not connect to the Domino server

I googled this article: https://support.hcltechsw.com/csm?id=kb_article&sysparm_article=KB0090343&sys_kb_id=0ace67511bc03050a67e9759bc4bcb61 which provided an explanation and in my case the solution was to add 

SharedMemoryAllowOnly=0 

into server's notes.ini