Search This Blog

Tuesday, April 12, 2022

Reply/Forward icon in Mail database

 Hi

Recently we upgraded Notes to V12 and some of our users (which used non-English mail template) complained that they did not see that special icon in Mail views/folders telling which email was replied and/or forwarded.

Such icon is connected with the field $RespondedTo which is managed by Notes client itself, it is not created by some lotusscript or @formula in the mail database 


I noticed that this field was missing for those uses which had the issue with the icon.
After some investigation I discovered that this field is managed by Notes if this checkbox is enabled in user's maildb.

and it was disabled in mailboxes of the users complained about the issue.

The funny thing was that this checkbox was enabled in the mail template so I couldn't see the reasons why it wasn't inherited. To fix this issue I simply disabled and then enabled this checkbox back in the mail template and then ndesign.exe started to handle it correctly.




Friday, February 18, 2022

If you server hangs up during DAOS resync...

 Hi

Recently I worked on the issue with one Domino-server which became unresponsive after running program document with command "tell daosmgr resync force" at night. We noticed that in the morning only and you know how stessful it could be :)

I checked the server via Domino Console and it looked alive, it reacted to commands like "sh ta" or "sh ser", however Notes clients couldn't connect to it. I noticed also some other messages telling that something was wrong, like:

Error connecting to server <SERVERNAME>: Remote system no longer responding
Server task Router on <SERVERNAME> is no longer responding
Server task SMTP Server on <SERVERNAME> is no longer responding

I restarted server and when it went up I noticed following errors:

The DAOS catalog cannot be opened. DAOS cannot operate normally.: The integrity of a database storage container has been lost - the container will be rebuilt.

and also many errors like this:

The database D:\IBM\Domino\data\<database.nsf? was unable to write to file D:\HCL\DAOS\0001\912B6AA17EFDE1C9C12587E9002FDA051912B6AA17EFDE1P.nlo: Release 12.0.1 HF24|January 19, 2022

Without thinking twice I ran "tell daosmgr resync" manually and in 10 minutes the server hanged up again with the same symptoms - reacted to commands via Domino Console but Notes client couldn't connect.

I restarted server again, all the errors from the previous start were shown again, but that time I checked DAOS status and it was REBUILDING.


This time I decided to not run resync while catalog was rebuilding and I decided to wait until the evening and then try to do the resync agin. That time resync worked without issues, the only stupid thing I did - I didn't check the catalog status before the running resync, so I can't say if server did something during the day before I ran the resyn again in the evening.

I communicated this case with HCL and they told me that one of the solutions could be to resync the catalog while server is down by running "ndesign.exe resync" (the similar way you can compact names.nsf or log.nsf).


Friday, January 28, 2022

Both Domino 12.0.1 and IF1 contain bug "The caller's SemWait timeout expired."

Hi

Both Domino 12.0.1 and IF1 (HF11) contain bug "The caller's SemWait timeout expired."

You can see this error in Domino logs or if you configure a corresponding Probe in admin4.nsf (Probe Type: Database, Subtype: Error Monitoring).

There are different consequenses of this problem:
  1. sometimes replication does not work properly
  2. sometimes router can't send the email
  3. ...I think there could be more issues, but the two first I expirienced myself...
The errors in log.nsf / console.log may look like:

20-01-2022 09:01:58 The DAOS catalog cannot be updated.: The caller's SemWait timeout expired.
20-01-2022 09:01:58 Router: Agent message: Notes error: The caller's SemWait timeout expired. 

The errors in ddm.nsf may look like:

Database object write error: E:\Data\mail\sxxxxt.nsf: The caller's SemWait timeout expired.

The only solution is to request another hotfix from IBM HCL (HF24) which includes IF1(HF11) + the fix for this problem (see DCKTCARNVR).

Monday, December 6, 2021

db.createdocumentcollection()

 Hi

This is unbelievable.

After 20 years of working with Notes I found out that there always was a method db.createdocumentcollection() for creating a blank collection.

Why the hell neither IBM nor HCL included it into documentation and made it public? 

Previously I always got an empty collection by either ran search with query which for sure would return nothing or used db.GetProfileDocCollection() - both these approaches worked well but made the code less obvious/self-explaining.


Dim s As New NotesSession
Dim db As notesdatabase
Dim col As notesdocumentcollection
Set db = s.currentdatabase
Set col = db.createdocumentcollection()

Wednesday, November 24, 2021

TSM failed after installing FP3 for Domino V11

 Hi

We installed FP3 for Domino V11 this summer and when we needed to restore some db from the backup TSM failed to do it (though it previously worked OK). The error was:


ACD0004E An unknown error has been detected.
Warning! Error initializing the connection to the TDP library
Could not connect to the Domino server

I googled this article: https://support.hcltechsw.com/csm?id=kb_article&sysparm_article=KB0090343&sys_kb_id=0ace67511bc03050a67e9759bc4bcb61 which provided an explanation and in my case the solution was to add 

SharedMemoryAllowOnly=0 

into server's notes.ini

Sunday, August 8, 2021

Do not use NotesRichTextItem.ConvertToHTML() or use it but be ready it will be removed in the future release.

 Hi

Just a suggesstion.

Since R9(?) it was possible to use NotesRichTextItem.ConvertToHTML(), see its description here: https://help.hcltechsw.com/dom_designer/9.0.1/appdev/H_CONVERTOHTML_METHOD_NOTESRICHTEXTITEM.html

It was kind of relief to be able to use a native API instead of making those custom solutions based on MIME/DXL/XmlHttpRequest/...

However, later we noticed that every call of NotesRichTextItem.ConvertToHTML() in server agent showed this message in the Domino console: 

        CSRF Init: iNotes_WA_Security_ReturnUrlCheck> c_CSRFReturnUrlCheck: 1

It is very easy to reproduce it. Create an agent which works in server's context (run it from console or by schedule) and write something like Dim s As New NotesSession Dim db As NotesDatabase Dim doc As NotesDocument Dim rtItem As NotesRichTextItem Dim html As String Set db = s.Currentdatabase Set doc = db.Alldocuments.Getfirstdocument() '<-- This is a doc with richtextitem Body Set rtItem = doc.Getfirstitem("Body") html = rtItem.convertToHTML() Domino console has to show: [0AB8:0025-08E8] 31-05-2021 16:32:21,71 CSRF Init: iNotes_WA_Security_ReturnUrlCheck> c_CSRFReturnUrlCheck: 1 [0AB8:0025-08E8] 31-05-2021 16:32:21 AMgr: Start executing agent 'testAgent' in 'test\test.nsf' [0AB8:0025-08E8] iNotes Init: Credential Store Configuration not enabled, less secure mode. [0AB8:0025-08E8] 31-05-2021 16:32:22 AMgr: Agent 'testAgent' in 'test\test.nsf' completed execution We have a big database with several RichTextItems in every document and for every such item in every document a scheduled agent does some magic and uses rtItem.convertToHTML() - Domino console spams million of such messages for more than 1 hour.

I contacted HCL regarding this and they said that they forgot to hide this method: it is not supported and will be removed in a future release, though it was reproduced even in R12.


DDM probes + Cluster Replicator + intensively used database = Database open error: [filepath.nsf]: Database is currently in use by you or another user

 Hi

Here it is a thing which I spent a few days on.

We had two Domino R11.0.1 FP3 in a cluster. Then we added a 3rd Domino server of the same version into the cluster and after that we started to see many errors like that:

08-08-2021 18:09:53   Database open error: <filepath.nsf>: Database is currently in use by you or another user
08-08-2021 18:09:53   Database open error: <filepath.nsf>: Database is currently in use by you or another user
08-08-2021 18:09:53   Database open error: <filepath.nsf>: Database is currently in use by you or another user
08-08-2021 18:09:53   Database open error: <filepath.nsf>: Database is currently in use by you or another user
08-08-2021 18:09:53   Database open error: <filepath.nsf>: Database is currently in use by you or another user

Despite the errors in logs, all our applications worked OK, we didn't have any consequences of this error except that it was very hard to use log.nsf, because we could get ~5K such messages per day. The errors referred to a few different databases, but not to all databases we had. The errors were logged on all three Domino-servers. 

I googled and found these two articles without a solution:

So, at the end of the day I didn't find anything helpful in the Internet, except the point that the error message was produced by Cluster Replicator. My first attempts to resolve it myself ended up with nothing, so I decided to contact HCL. 

Unfortunatelly, they couldn't help me either, they suggested some basic advices, like try to delete the database and then replicate it again, try to restart Cluster Replicator and so on.

So, I didn't have any choice except trying to resolve it myself. 

After spending two days on that I discovered that the reason of the issues were DDM probes. We always used DDM probes for server monitoring and didn't change anything there for a long time, that's why it was so hard to catch that but after I disabled them - the error messages stopped to appear.

Now, let me tell you my explanation which is a pure theory but looks reasonable to me.

We rolled out 3rd Domino server because we planned to run a new Web-application which had to attract many new users in the Internet and we wanted to be sure that our Domino-environment would handle the increased workload - we were using external load balancer which had to route users to our Domino cluster and 3rd Domino had to help a lot. 

After we added a 3rd Domino, we ran the application as well and we got more Web-users as we expected.  So, basically, what happened: 

  • More users than before worked with just a few databases (kind of an entry point for the new Web-application);
  • Three Cluster Replicators (instead of two) pushed data to replicas on other servers and they had to do it often than before;
  • DDM worked without changes but it seemed the server tasks executing DDM probes needed to have an exclusive access to the database for a short period;
  • Cluster Replicators worked intensively with a few databases and often failed to open the databases while DDM probes were running and therefore Cluster Replicator showed that message and kept the changes in memory until it would be possible to open the database later.
That's basically it.
Maybe there were specific probes which caused this message, I did not check that, I only tested that: 
  • disabling of probes disabled the messages 
  • enabling the same probes back returned the messages back as well
It is hard even to ask HCL to fix anything because it is good to be able to see such messages, - they can help to identify important issues but on another hand when you get log.nsf spammed with 5K messages per day it is not acceptable either. The only idea is - maybe it makes sense to introduce another notes.ini parameter which could suppress the message, not sure.