DXL To The Rescue

The other day I came across an issue in a lotus notes application that was kind of weird. The user was getting the ***** error when trying to edit a group in the ACL.

The first thing that came to mind was to check the users access, but they had Manager access to the application. They just wanted to remove a that was put in the name of a group by mistake, so I tried to make the change myself, but I also got the same error. I also tried to remove the entry from the ACL, but also got the same error. The next thing that came to mind was to code an agent to remove the entry by using the back-end lotusscript objects.

So this is what I came up with:

Dim targetDB As NotesDatabase
Dim targetACL As NotesACL
Dim targetEntry As NotesACLEntry

Set targetDB = New NotesDatabase(“”, “”)

If(targetDB.Open(“Server”, “db.nsf”))Then
Set targetACL = targetDB.Acl
Set targetEntry = targetACL.Getentry(“Group With / in its name”)
If(Not(targetEntry Is Nothing))Then
‘targetEntry.Name = “TestEntry”
Call targetEntry.Remove()
Call targetACL.Save()
Else
Print “Entry not found in ACL”
End If
Else
Print “Not able to open db”
End If

But when I tried running the agent I got the same error when trying to execute the line Call targetACL.Save(). I then did some searching on the web and found a few people with the same error, but most of them suggested the ACL was corrupt and that the way they solved the issue was to create a new application and copy the data over, but this would only be an option for me as a last resort. Since it seemed like the cause was the ACL had become corrupt I thought I’d export the ACL to DXL and verify if there was anything strange in the XML.

I created the following agent to export the ACL:

Dim s As New NotesSession
Dim targetDB As NotesDatabase
Dim fileName As String

Set targetDB = New NotesDatabase(“”, “”)

If(targetDB.Open(“Server”, “db.nsf”))Then
REM Open xml file named after target database
Dim stream As NotesStream
Set stream = s.CreateStream()
fileName = “c:temp” & Left(targetDB.FileName, Len(targetDB.FileName) – 3) & “xml”
If Not stream.Open(fileName) Then
Print “Cannot open ” & fileName
Exit Sub
End If
Call stream.Truncate

Dim exporter As NotesDXLExporter
Set exporter = s.CreateDXLExporter

REM Create note collection of actions
Dim nc As NotesNoteCollection
Set nc = targetDB.CreateNoteCollection(False)
nc.Selectacl = true
Call nc.BuildCollection

Set exporter = s.CreateDXLExporter(nc)

Call exporter.SetInput(nc)
Call exporter.SetOutput(stream)

‘ Stops the from being added to output.
exporter.OutputDOCTYPE = False

Call exporter.Process
Else
Print “Could not open db”
End If

After running this agent the following content was generated in the Test.xml:

replicaid=’852574830021CC9F’ path=’CN=Test/OU=Test/OU=Server/O=Test!!db.nsf’
title=’Test DB’ usejavascriptinpages=’false’>

percentused=’97.7727051598594′ numberofdocuments=’39’>
dst=’true’>20101006T162213,08-04
>20101007T094423,47-04

writepublicdocs=’false’/>

deletedocs=’true’/>

level=’editor’ deletedocs=’true’ createpersonalagents=’false’ createpersonalviews=’false’
createsharedviews=’false’ createlsjavaagents=’false’/>
10/06/2010 04:20:38 PM Test User/Test/Test updated Test Admins
10/06/2010 04:20:31 PM Test User/Test/Test added Test Admins

Since I did not see anything strange in the ACL DXL I decided to create an agent that would import the ACL DXL after removing the group with the / in it’s name from the DXL. This is what the agent looked like:

Dim s As New NotesSession
Dim targetDB As NotesDatabase
Dim fileName As String

Set targetDB = New NotesDatabase(“”, “”)

If stream.Bytes = 0 Then
Print “File did not exist or was empty”
Exit Sub
End If

‘Import DXL into new database
Dim importer As NotesDXLImporter
Set importer = s.CreateDXLImporter(stream, targetDB)
importer.ReplicaRequiredForReplaceOrUpdate = False
importer.ACLImportOption = DXLIMPORTOPTION_REPLACE_ELSE_IGNORE

Call importer.Process
Else
Print “Could not open db”
End If

After running this agent I verified the ACL of the application and the group was no longer there. I also tested that new entries could be added and modified without issues.

Has anybody run into similar issues before, where you had to resort to DXL for a fix?

Code That Makes You Say… Huh? #1

Photo By Andyrob

Most of the work I do at my current job is maintenance to existing applications within my team. A while ago I found an interesting line of code within a lotusscript library.

TStamp = Evaluate(“@Text(@Now)”)

It seems the developer that did this doesn’t know that there is a Now function in lotusscript.

I have found this same line in most of the applications that I maintain and I’ve also seen it in applications that I don’t maintain. Worst of all, most of these applications are used globally and this line is used to capture the date and write it to an audit trail. What’s wrong with this you ask? Well you don’t have a consistent audit trail, for example the date format in the UK is not the same as in the US.

Two solutions to fix this would be:

Make the audit trail date field a multi value date field and use the Now function to append the date value to the field. With this approach you can use the field properties to adjust the format for displaying the dates. This allows you to display the dates in the users local format or choose a specific format.
Use the Format function in combination with the Now function to get the same date format everytime. With this approach you ensure a consistent date format within the audit trail, but you loose the ability to display the dates in the users local format.

Have you found code that makes you say Huh in applications you maintain?

P.S. I’ve tried telling the developer that maintains the other applications to modify the code, but that’s another story…

Lotus Notes DB User Activity

Photo By bluelakefreepress

Last week I was asked to provide some stats from an existing application which didn’t have logging built into it. After extensively searching on google and other search engines the only method I found was to use the NotesUserActivity class from the Lotus Sandbox. This class uses the C api to get at the user activity data of a lotus notes database.

I know the user activity is in no way a replacement for building logging into the application, but this is an existing application which didn’t have logging. The issue I ran into was that the user activity in a notes database only holds 1400 records, so this means that if the database is used heavily you might only get a few hours of usage data. Luckily the application I’m reporting on isn’t used that heavily so I got about a month worth of usage data, but I was asked to provide stats since the release of the application. So I was just wondering if there another way to get usage data from a lotus notes application that doesn’t have logging built into it?

Notes Views with Colors

After reading Jake Howlett’s blog entry How-To: Shade Your Views Based on Document’s Age, I wanted to try the technique out on an application that runs on the lotus notes client and I would use the built in feature in Notes for assigning colors to rows.To use this feature you check the “Use value as color” property of the column

Then, assign the column a formula that will evaluate to either a list of three numbers between 0 to 255 or a list of six numbers also between 0 to 255. If you send only three values that will become the RGB color for the text and if you send six values the first three will become the RGB for the background and the other three will be for the text. It is also good to mention that these colors will take effect on all columns right of the color column, so it is not a good idea to create this column at the end of the view.

Example of values for column:
255:0:0 – Red Text
143:255:87:0:0:0 – Green background with black text
255:0:0:0:0:0 – Red background with black text

Here is a screenshot of the resulting view. By adding these visual indicators I could even remove the creation date column and users would be able to tell the age of the documents.

The documents in the application I’m planning to use this technique on are measured in minutes and not on days, so I would schedule the agent to update the view column formula every 5 minutes instead of nightly.

So now I ask, what kind of performance impact would this have on the database if any?

Lotusscript Dir Function Causes Domino Crash

After many debugging and troubleshooting efforts IBM has provided a hotfix for the issue I discussed in Attachment ExtractFile Limitation post. It seems that the lotusscript Dir function causes a server crash in version 7.0.2 and it seems that it may also be the case for version 8 and probably pre 7 versions. When a path longer than 255 characters is passed to the Dir function it causes a crash instead of throwing an error. This can easily be fixed by checking the path length before sending it to the Dir function, but it was a bit troublesome to track down because you don’t expect the server to crash when calling a simple function so most of the time attention is focused on other more complex functions and third-party functions.

Hope this helps anybody that runs into the same situation.

Using POI to Export Lotus Notes Data to Excel

As those who have been reading my previous posts have seen, I have been working on a project that involves exporting meta data from Lotus Notes documents to an excel sheet. I’m using the CreateObject(“Excel.Application”) function to do this. The problem is that the agents that do this export are scheduled and run on a server. This means that excel has to be installed on that server for the process to work and most of the admins that I had to work with did not like the idea of having excel on a server they have to support.

For this reason I started messing around with the apache project POI. This project has pure java ports of file formats based on Microsoft’s OLE 2 Compound
Document Format. So it has classes for creating excel spreadsheets without having excel installed on the server, which would make the admins happy. So I created this simple class to create and write rows to an excel file.

package com.clr.excel;

import java.io.FileOutputStream;
import org.apache.poi.hssf.usermodel.HSSFRichTextString;
import org.apache.poi.hssf.usermodel.HSSFRow;
import org.apache.poi.hssf.usermodel.HSSFSheet;
import org.apache.poi.hssf.usermodel.HSSFWorkbook;

public class ExcelWriter {
    private int curRow;
    private HSSFWorkbook curWB;
    private HSSFSheet curSheet;

    public boolean writeExcelHeader(String[] rowValues) {
        try {
            this.curRow = 0;
            this.curWB = new HSSFWorkbook();
            this.curSheet = this.curWB.createSheet(“Sheet1”);

            //This will write the header row
            return(this.writeToExcel(rowValues));
        } catch(Exception ex) {
            ex.printStackTrace();
            return(false);
        }

    }

public boolean writeToExcel(String[] rowValues) {

try {

HSSFRow row = this.curSheet.createRow(this.curRow);

            for(int i = 0; i < rowValues.length; i++) {
                row.createCell(i).setCellValue(new HSSFRichTextString(rowValues[i]));
            }

this.curRow++;

        } catch(Exception ex) {
            ex.printStackTrace();
            return(false);
        }

return(true);
}

    public boolean writeExcelFile(String fileName) {
        try {
            //Auto fit content
            for(int i = 0; i < 20; i++) {
                this.curSheet.autoSizeColumn((short)i);
            }

            // Write the output to a file
            FileOutputStream fileOut = new FileOutputStream(fileName);
            this.curWB.write(fileOut);
            fileOut.close();
        } catch(Exception ex) {
            ex.printStackTrace();
            return(false);
        }

        return(true);

}

Again I used LS2J to use this class from my lotusScript agent.

These declarations are needed to use LS2J

Uselsx “*javacon”
Use “ExcelWriter”

Agent to create excel sheet.

Sub Initialize
    On Error Goto ErrorHandler
    Dim session As New NotesSession
    Dim db As NotesDatabase
    Dim view As NotesView
    Dim doc As NotesDocument
    Dim js As JAVASESSION
    Dim xlWriterClass As JAVACLASS
    Dim xlWriterObject As JavaObject
    Dim rowValues(1 To 3) As String
    Dim folderName As String

    Print “Started Running CreateExcel”

    folderName = “C:Temp”

    ‘Initialize column headers
    rowValues(1) = “Title”
    rowValues(2) = “Date”
    rowValues(3) = “Name”

    ‘Initialize Excel Writer class
    Set js = New JAVASESSION
    Set xlWriterClass = js.GetClass(“com.clr.excel.ExcelWriter”)
    Set xlWriterObject = xlWriterClass.CreateObject
    If(Not(xlWriterObject.writeExcelHeader(rowValues)))Then
        Print “Error creating excel sheet!”
        Exit Sub
    End If

    Set db = session.CurrentDatabase
    Set view = db.GetView(“MainView”)

    Set doc = view.GetFirstDocument
    Do While(Not(doc Is Nothing))

        rowValues(1) = doc.Title(0)
        rowValues(2) = doc.selectedDate(0)
        rowValues(3) = doc.Name(0)

        ‘Write metadata
        If(Not(xlWriterObject.writeToExcel(rowValues)))Then
            Print “Could not write to excel!”
        End If

        Set doc = view.GetNextDocument(doc)
    Loop

Finished:
    xlWriterObject.writeExcelFile(folderName & “workbook.xls”)

    Exit Sub

ErrorHandler:
    Print “Error ” & Error & ” on line: ” & Erl

    Resume Finished

End Sub

Enjoy the code here.

Attachment ExtractFile Limitation

This might not be new to some of you but today it was the first time I had faced this issue. For the past 2 weeks I have been troubleshooting an agent that is crashing the server. This agent uses third party software to convert documents to pdf, it also uses excel to create a metadata sheet, and it uses code from my last posting to extract attachments. We got IBM and the other vendor involved to help narrow down the problem. I took out the excel part, also the third party code, and today after many many many server crashes we found that the embeddedObject.ExtractFile might be the problem. Well not actually the function but the parameter. Now, I knew that there is a filename limit on windows of 255 characters, but what I didn’t know is that apparently the ExtractFile function has that limit but it takes into account the whole path sent to it. So you might have a filename that is only 100 characters long, but have a deep folder hierarchy and that will cause problems. Anyway we are still waiting on IBM to confirm, but seems pretty clear that this is the problem so we will have to make the folder hierarchy shorter and maybe truncate the filenames.

Anybody else ever face this problem?

[Update]
After a bit more debugging it seems that the problem is happening when the Dir(filename, 0) is called. I use this function to test out if a file already exists before extracting it. I also did more testing on the windows limit and it does seem to be on the whole path, oh well one learns things every day. Anyway we have IBM looking into why the Dir function is not returning an error and just crashes the server. I will have to check the filepath length before sending it to the Dir function.

Zip Extraction using LS2J

The last month I have been working on a project that involves converting notes documents to pdf and importing them within Documentum. This also involved extracting any attachments in the documents and also sending them to Documentum. The problems started when users found some attachments that had been sent to documentum and were zip files. I guess Documentum doesn’t index the content within zip files and the these files would not show up in the users search. So I had to find a way to decompress the zip files and actually send the content of these to Documentum. I went looking around the web and found How to extract file/files from a zip file. This did exactly what I needed, but one little catch all the other logic was already in a lotusScript agent, so I decided to use LS2J. It was my first time using LS2J so I did a search on nsftools.com since I remember that I had read a post about LS2J on Julian’s Blog. After that, I got to work and this is what I put together.

I created the following java script library using the java class from the site above and made minor tweaks for my situation:

package com.clr.zip;

import java.io.File;
import java.io.FileInputStream;
import java.io.FileOutputStream;
import java.util.zip.ZipEntry;
import java.util.zip.ZipInputStream;

public class ZipExtractor {

public String extractZip(String folderName, String fileName) {

try {
byte[] buf = new byte[1024];
ZipInputStream zipinputstream = null;
ZipEntry zipentry;
zipinputstream = new ZipInputStream(new FileInputStream(folderName + fileName));

zipentry = zipinputstream.getNextEntry();
while (zipentry != null) {
// for each entry to be extracted
String entryName = zipentry.getName();

int n;
FileOutputStream fileoutputstream;
File newFile = new File(entryName);
String directory = newFile.getParent();

if (directory == null) {
if (newFile.isDirectory()) {
break;
}
}

fileoutputstream = new FileOutputStream(folderName + entryName);

while ((n = zipinputstream.read(buf, 0, 1024)) > -1) {
fileoutputstream.write(buf, 0, n);
}

fileoutputstream.close();
zipinputstream.closeEntry();
zipentry = zipinputstream.getNextEntry();

}// while

zipinputstream.close();
return(“SUCCESS”);

} catch (Exception ex) {
ex.printStackTrace();
return (“ERROR”);
}

}
}

Then I modified my lotusscript agent to use this class to decompress the zip files.

To be able to use LS2J we have to add these to the Options section of our code

Uselsx “*javacon”
Use “ZipExtractor”

Then declare the following variables to work with the java class:

Dim js As JAVASESSION
Dim zipExtractorClass As JAVACLASS
Dim zipExtractorObject As JavaObject

Now all we have to do is use the following lines to actually decompress a zip file to a certain folder:

Set js = New JAVASESSION

‘Here we get a reference to the ZipExtractor class
Set zipExtractorClass = js.GetClass(“com.clr.zip.ZipExtractor”)

‘Here we get an actual instance of the ZipExtractor class
Set zipExtractorObject = zipExtractorClass.CreateObject

‘This is were the actual java class method is called and parameters are sent
result = zipExtractorObject.ExtractZip(folderName, attachName)

As we can see LS2J can be very handy to provide functionality in lotusscript that normally wouldn’t be possible. Hope others will find this helpful.

Here is a link to a demo db with the java class and the agent so you can mess around with it.

Coding Thoughts

This blog's focus is mainly on WebSphere Portal development, but will also cover things about java in general, VB.NET, etc., really anything interesting that has to do with technology.

lotusscript

DXL To The Rescue

Code That Makes You Say… Huh? #1

Lotus Notes DB User Activity

Notes Views with Colors

Lotusscript Dir Function Causes Domino Crash

Using POI to Export Lotus Notes Data to Excel

Attachment ExtractFile Limitation

Zip Extraction using LS2J