A Little Quiz: Is this code safe?

sub Initialize

  dim s as new NotesSession
  dim db as NotesDatabase
  dim c as NotesDocumentCollection
  dim doc as NotesDocument
  dim item as NotesItem

  set db = s.CurrentDatabase
  set c = db.UnprocessedDocuments
  set doc = c.getFirstDocument

  if doc.HasItem("Body") then
    set item = doc.GetFirstItem("Body")
    if item.Type = HTML then
      Call doc.PutInFolder("HTML Body Messages",true)
    end if
  end if

end Sub

This LotusScript was converted to HTML using the ls2html routine,
provided by Julian Robichaux at nsftools.com.

You've probably guessed that I wouldn't be asking if this code was safe if the answer was 'Yes', so can you see why the answer is really 'No'?

If I told you that this code can throw an 'Object variable not set' error on the test of item.Type, would you believe me?

It's really true. Even though there's a test of doc.HasItem("Body"), it is possible for doc.GetFirstItem("Body") to fail.

There is more than one way to fix this code to make it safe from that particular error. If you've accepted what I said above, that doc.GetFirstItem("Body") really can fail even though the code appears to be defensive enough with the doc.HasItem("Body") check, then I'm sure you can see that the obvious fix is this:

sub Initialize

  dim s as new NotesSession
  dim db as NotesDatabase
  dim c as NotesDocumentCollection
  dim doc as NotesDocument
  dim item as NotesItem

  set db = s.CurrentDatabase
  set c = db.UnprocessedDocuments
  set doc = c.getFirstDocument

  if doc.HasItem("Body") then
    set item = doc.GetFirstItem("Body")
    if not item is nothing then
      if item.Type = HTML then
        doc.PutInFolder("HTML Body Messages",true)
      end if
    end if
  end if

end Sub

This LotusScript was converted to HTML using the ls2html routine,
provided by Julian Robichaux at nsftools.com.

The seemingly redundant test to make sure that the item object is valid does protect against the 'Object variable not set' error, so this code is safe. This workaround doesn't, however, do anything to shed light on why that error could be thrown.

I have actually run into the situation of HasItem not protecting against 'Object variable not set' previously, but until this week all I had ever done was work around it by testing of the value returned by GetFirstItem before trying to access it. Each time I had to do this, my faith in the HasItem method was somewhat shaken.

A few days ago, I decided that it really wasn't reasonable to believe that HasItem has a bug. I decided to look for the real cause, and when I tracked down the document that threw the error and looked at the MIME source, this is what I saw.

Received: from XXXXXXXXXXXXXX ([XX.XX.XX.XX])
         by XXXXXXXXXXXXXXXX (Lotus Domino Build M10_08082001 Beta 3)
         with ESMTP id 2001082119581368:1105 ;
         Tue, 21 Aug 2001 19:58:13 -0400 
Received: from XXXXXXXXXXXXX (XXXXXXXXXXXXXXXXXXXXX [XX.XX.XX.XX])
                by XXXXXXXXXXXXXXXXXX  (8.11.1/8.11.1) with ESMTP id f7LNwUx03833
                for XXXXXXXXXXXXX; Tue, 21 Aug 2001 19:58:34 -0400 (EDT)
From: XXXXXXXXXXXXXXXXX
To: XXXXXXXXXXXXXXXX
Subject: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
X-Mailer: Lotus Notes Release 5.0.4a  July 24, 2000
Message-ID: 
Date: Tue, 21 Aug 2001 19:58:29 -0400
X-MIMETrack: Serialize by Notes Client on XXXXXXXXXXXXXXXXXXXXX (Release
5.0.4a |July 24, 2000) at 08/21/2001 07:58:29 PM,
                Serialize complete at 08/21/2001 07:58:29 PM,
                Itemize by SMTP Server on XXXXXXXXXXXXXXX (Build M10_08082001 Beta 3|August 08, 2001) at 08/21/2001 07:58:13 PM,
                Serialize by Router on XXXXXXXXXXXXXXXXX (Release 5.0.8 |June 18, 2001) at 08/21/2001 07:58:53 PM,
                Serialize complete at 08/21/2001 07:58:53 PM,
                Serialize by Notes Client on XXXXXXXXXXXXXXXXXXXXXXXXX (Release 8.5|December 05, 2008) at 05/14/2009 11:27:58 PM,
                Serialize complete at 05/14/2009 11:27:58 PM
MIME-Version: 1.0
Content-Type: multipart/alternative; boundary="=_alternative 0083AE3E85256AAF_="

This is a multipart message in MIME format.
--=_alternative 0083AE3E85256AAF_=
Content-Type: text/plain; charset="us-ascii"


--=_alternative 0083AE3E85256AAF_=
Content-Type: text/html; charset="us-ascii"


--=_alternative 0083AE3E85256AAF_=--

Seeing this was enough to bring me to an "Aha! Moment".

Domino converts those headers to four MIME Body items when it stores the message, but there is no content within the headers. Since there is no content, conversion of MIME to rich text cause all four of those items disappear.

sub Initialize

  dim s as new NotesSession
  dim db as NotesDatabase
  dim c as NotesDocumentCollection
  dim doc as NotesDocument
  dim item as NotesItem

  s.ConvertMIME = false

  set db = s.CurrentDatabase
  set c = db.UnprocessedDocuments
  set doc = c.getFirstDocument

  if doc.HasItem("Body") then
     set item = doc.GetFirstItem("Body")
     if item.Type = HTML then
        doc.PutInFolder("HTML Body Messages",true)
     end if
  end if

end Sub

This LotusScript was converted to HTML using the ls2html routine,
provided by Julian Robichaux at nsftools.com.

This code disables automatic conversion of MIME to Notes rich text -- a service that the back-end classes provides for you by default. Doing this makes the code safe. The problem was able to occur becase the HasItem method doesn't force MIME conversion but the GetFirstItem method does, therefore the MIME Body items exist for the HasItem test, but they don't exist for GetFirstItem.. (This may be dependent on the version of Notes or Domino that the code runs on. I've done my testing on 8.5.)

Note that with MIME conversion turned off, the item can't disappear, therefore we don't need to test the value returned by GetFirstItem and I removed that test for clarity in the example code. Of course, an extra bit of defensive coding is almost never a bad idea. If your requirements were to change and force you to turn convertMIME back on, having coded an unnecessary 'if not item is nothing' test might save your butt! In the real code that inspired this post, you can bet that I left that test in the source.

Getting into it a little further, here's a more precise definition of how the problem occurs:

• A MIME Body item exists, so it passes the HasItem("Body") test.

• But if the MIME Body is empty, consisting only of headers with no actual content, then the result of MIME to rich text conversion is not an empty rich text Body item, It is no Body item at all! Hence the failure of GetFirtstItem("Body").

One final note. I was actually investigating an entirely different issue having to do with the Body items in email messages when I wrote code similar to what is shown above and ran into the disappearing Body items. If you read the code closely, there's a dead giveaway of what it was that I was initially interested in: the test of item.Type = HTML. I'm still very much interested in this. If you've written code for processing email messages in Domino, there's a pretty good chance that you've already realized that your code may have to deal with the fact that the Body of the message isn't always rich text. You probably know that it can be MIME, and you may also realize that it can be plain text. It can actually be any valid item type. You can see the constants that define these types in the back-end class documentation for the NotesItem.Type property, and one of those definitions is HTML. I don't know what feaures of Notes and Domino generate this type, when, or why... but I do know that email messages can have a Body item with this type. I also know that since it isn't a NotesRichTextItem, reading the item into a variant and treating it as a NotesRichTextItem is a very bad thing, and that treating it as a regular NotesItem and using the NotesItem.Text property is also a very bad thing because it retuns null -- which is actually clearly documented, but I never had any reason to care about it until this week.

1. Nathan T. Freeman05/15/2009 08:13:46 AM
Homepage: http://nathan.lotus911.com

if not item is null then

should be...

if not item is nothing then

2. Richard Schwartz05/15/2009 10:09:15 AM
Homepage: http://www.poweroftheschwartz.com

@Nathan -- that will teach me to convert Java to LotusScript and post it without actually compiling!
Fixed.

3. Net Toolbox - John07/15/2009 10:08:25 AM
Homepage: http://net-toolbox.com

Nice info about code-security!

4. Mario Varchmin08/03/2009 09:35:54 AM
Homepage: http://www.e-llennium.de

If you use the old Notes integrated web browser, which uses the perweb.nsf for storing web pages, you will find that many of the documents in that database contain a Body item of type HTML.

5. Rick Dickinson09/09/2009 05:48:30 PM
Homepage: http://www.notesguy.com

Of course, the first problem I saw when I looked at this snippet:

set doc = c.getFirstDocument

if doc.HasItem("Body") then

...was that you didn't test to be sure that doc wasn't Nothing.

- Rick "Nothing from Nothing leaves Nothing. You gotta have something, if you gonna leave me" Dickinson

6. Richard Schwartz09/09/2009 08:33:22 PM
Homepage: http://www.poweroftheschwartz.com

Good point, Rick. Unstated assumption that this is a foreground agent running on selected documents, ergo there must be at least one doc in UnprocessedDocuments. But in another context, that would be a problem.