Foreword
This document incorporates by reference the CalConnect Intellectual Property Rights, Appropriate Usage, Trademarks and Disclaimer of Warranty for External (Public) Documents as located at
February 2008 CalConnect Interoperability Test Report
1. Participants
Table 1 — Participants
| Organization | Event Attended | Participants | Products Tested |
|---|---|---|---|
Apple | Regular | Cyrus Daboo (iCal Server) | iCal version 3.0.2 (1236) |
Kerio Technologies | Both | Tomas Hnetila | Kerio MailServer 6.5.0, a special build with CalDAV Realtime S2S Kerio Outlook Connector 6.5.0 (KOFF) |
Marware | Regular | Tony Becker | Project X 1.3.2 (6043) |
Microsoft | Regular | Firdosh Ghyara (Exchange) | Entourage 2008 |
Oracle | Regular | Simon Vaillancourt | |
RPI/Bedework | Regular | Mike Douglass | |
Scalix | Regular | Florian von Kurnatowski | |
Sony Ericsson | Mobile | David Colter | |
Sun Microsystems | Regular | May Ma | |
Zimbra Yahoo | Regular | John Holder | |
CalConnect Reps | |||
Interop Manager | Regular | Pat Egen | Pat Egen “unofficially” tested Lotus Notes as part of the event |
2. Introduction
Participants of the testing event used predetermined test scenarios. Rather than post the full scenarios in this document, they can be found on the CalConnect website at the following URL: http://www.calconnect.org/ioptesting.shtml. The documents used in this testing event were the CalConnect CalDAV Matrix for Draft 08 and the iCalendar Testing Matrix. Summaries and specific findings and issues found are noted in this document.
3. Comments and Findings
3.1. CALDAV testing
3.1.1. Server vendor comments
3.1.1.1. Server 1
The Interop went very well; it was nice to see so many vendors present at the event. As products get more mature the issues found are more interesting and having a chance to discuss design implementation details with other vendors is a great opportunity to improve our products.
3.1.1.2. Server 2
This server vendor was mostly focused on REALTIME Server-to-Sever Scheduling.
Very difficult to do all tests.
One client’s recurrent event exception — deleted instance
EXDATE is without time zone
causes incompatibility with other clients
Another client app — recurrent event exception — an instance moved to different time
time is missing in RECURRENCE-ID ⇒ other clients are not able to recognize the instance
causes incompatibility with other apps
Meeting requests/responses from a client are not recognized as iMIP messages
One client had an issue with meeting requests on delegated accounts
One client shows organizer twice in attendee list for meeting requests
One client doesn’t send update for declined meetings when location is changed
not possible to setup delegation without Open Directory
puts invitation to calendar without asking or noticing user when attendee address doesn’t match
iCalendar and iTIP interoperability problem (upper case MAILTO:)
One CalDAV client requires full permissions to other people calendars
not possible to set permissions for others to create calendars
An iCalendar client has free/busy incompatibility — returns free/busy in XML format by default. Also had an issue with meeting
Realtime Server-to-Server scheduling
found & fixed bug RealTime scheduling
missing DAV:href in <C:recipient>
NOT YET fixed — incorrect Originator: address when sending meeting response
two spaces before HTTP version causes problem with another server (POST /pubcaldav/rtsvc HTTP/1.1) — fixed in URL configuration
saw a minor issue in one server when timezone definition contains comma/spaces (generated by a client in European timezone)
3.1.1.3. Server 3
We started discovering some extra bugs on Wednesday morning, mostly in the freebusy URI stuff. We did quite a bit of testing with another CalDAV client. After determining the client doesn’t support (valid) absolute URLs in DAV responses we changed our server to return relative urls. They tested and ran into a problem which prevented testing scheduling. We didn’t have time to deal with that. Since the interop this has been fixed and now works.
Realtime server-server problems were generally small — problems with invalid certs from one client and handling of authentication by another CalDAV server.
One server had a problem with invalid HTTP which was resolved.
3.1.1.4. Server 4
A CalDAV client tested against our server using the CalDAV Draft 8 scenarios provided the following results. Please refer to the CalDAV document located on the CalConnect server as noted in the Introduction section.
Section 1 — Event Creation
All items passed.
Section 4 — Event Deletion
All items passed except 4.5 which failed.
Section 5 — Access control
N/A
Section 6 — Calendar Management
6.1
passed
6.2
failed
6.3 — 6.6
N/.A
Section 7 — Free Busy Reports
Setup
Passed
7.1
N/A — Free busy sched works
7.1.1
N/A
7.1.2
Passed
7.1.3
N/A
7.1.4
Passed
7.1.5
Passed
Section 8 — Scheduling
Setup
Passed
8.1 through 8.2
Passed
8.3
N/A
8.4
Passed
8.5
N/A
8.6
Passed
EDITORIAL NOTE
Scheduling is new in CalDAV Draft 8. We are starting to see clients doing scheduling with servers. This is the first interop where we see scheduling actually being testing on servers and clients.
3.1.2. CalDAV Client Testing
3.1.2.1. CalDAV Client 1
The following are general things noted with our client testing.
network communication stopped after receiving a 50x errors.
did not treat calendar user addresses as case insensitive (e.g. MAILTO versus mailto).
did not send updates for declined meetings when the location is changed
did not handle authentication with a username and no password specified for a CalDAV account.
did not handle absolute URLs for the calendar home set.
did not handle retrieving events with inline attachments.
did not gracefully handle CalDAV servers that did not support CalDAV scheduling.
did not allow users to setup delegate access without Open Directory support.
sent POSTs for an overridden event using a RECURRENCE-ID with different a time zone then the DTSTART of the master event.
The following is a list of server issues that encountered during testing:
Invitations were not put into the attendee’s inbox.
One CalDAV client tried to set an ACL on a new calendar and received a 501 error.
Deleting calendar returned a 409 not a valid resource.
Free busy worked on the first calendar but not on additional calendars.
Principal URLs don’t work as attendees. Only email addresses worked.
Free busy worked on the first calendar but not on additional calendars.
Principal URLs don’t work as attendees. Only email addresses worked.
did not receive any notifications for invitations or replies because no iTIP messages were placed in the inbox.
Re-inviting an attendee to an event did not change the attendee’s PARTSTAT to NEEDS-ACTION.
Server removed X-WR-iTIPALREADYSENT from the VEVENT when client does a PUT.
not including a METHOD on iTIP messages placed in the INBOX. It appears that when doing a PROPFIND on the inbox, returns ICS data from a calendar instead.
A server failed to delete an overridden instance of a recurring meeting.
When creating a new calendar, server did not maintain the displayname or calendar-color on the calendar.
not including a METHOD on iTIP messages placed in the INBOX.
Deleting the fourth instance of a recurring meeting (Test 2.8) failed because server modified the time of the EXDATE.
Client tried to modify recurring events and overridden events but occasionally received a “304 not modified”.
A server failed to create new calendars.
Client tried to delete events but occasionally received 500 errors.
Free busy didn’t work for recurring events.
client tried to create a new calendar but received a 503 error.
A server modified/removed the organizer property on a PUT.
Modifications to events immediately after creating them often resulted in the server not properly persisting the change. This made it difficult to proceed with the more advanced tests such as overriding recurring events.
Scheduling and FREE/BUSY wasn’t supported on some servers.
3.1.2.2. CalDAV Client 2
error message “Account information not found. Request encountered an unexpected error (domain (null) code 0)”
doesn’t return Free/Busy data in correct format — probably we don’t use correct URL
error message when creating TODO:
Request Error
Request for “New To Do” in “Calendar” in account “xxxxx” failed.
The server responded with “HTTP/1.1 415 UnsupportedMediaType” to operation CalDAVWriteEntityQueueableOperation.
Scheduling not implemented
3.1.2.3. CalDAV Client 3
The following are testing comments.
The client works with several CalDAV servers.
Doesn’t work with several other servers because of one having no MKCALENDAR, one having a bug with del/ mk calendar timing, which is being fixed), and with one that has a Character set issue — my bug.
My things to be fixed:
Character set — from above
Get inbox from properties — server venders have different schemes
Overall things:
Handling “floating” events, all-day, freebusy, etc.
Overall Client things:
Self signed or expired certificates. Most servers are “Beta” at best.
We clients need to be able to deal with “bad” certificate errors.
207 response is “multipart” — must look deeper into body of response
could be all good/bad/mixed if asking for multiple properties.
Overall Server things:
Handling dates — there are a LOT of syntax options. For FreeBusy URL we were unable to find a syntax that all servers were happy with.
3.1.3. iCalendar Testing
During iCalendar, iMIP and iTIP testing the following items were identified.
3.1.3.1. iCalendar Client 1
The results noted are the behaviors that were observed when a message is sent/received.
General — these apply for all vendor testing scenarios
Requesting a refresh of an invitation is not supported
Countering an invitation is not supported
Declining a counter is not supported
Delegate concept is different.
cancels don’t work with one application
Attachments not going through
Accept invitation is not being processed properly. There is some problem converting response messages.
Declining an invitation — same as above
User C declines invitation — Responses fail iCalendar conversion
User C accepts invitation — Responses from one vendor fail iCalendar conversion
3.1.3.1.1. iTIP Testing
All items in the iTIP testing scenarios passed with the following notes:
4.1.1 create single event. Pass — Ran scenario by manually dropping a MIME file in the pickup folder of the transport log.
4.1.5 Create an event using the Value parameter — day events always have a timezone associated with it.
RSVP responses to requests: client allows RSVP per email message and not per attendee. Other mail servers do not have any indication to show how whether a RSVP is required. One server sets the req and opt part as req participants — - this is a bug.
4.2.4 Countering an Event — counter is not supported in one server
4.2.5 Delegating an Event — the concept of delegates is different than that of RFC2445.
4.3 — Publishing busy time — not supported
4.4.6 — Add new instance to a recurring event: adding a single instance is not supported. The complete recurrence rule needs to be updated.
4.4.7 — Add a new series of instances to a recurring event: Adding is not allowed. You can change the recurrence object completely. REFRESH is not supported.
4.4.8 Counter an instance of a recurring event: Not supported
4.7.2 Bad Recurrence-ID: If an invalid iCalender is received server fails the conversion of iCalendar and creates a simple IPM. Note message with the iCalendar body as an attachment.
3.1.3.2. iCalender Client 2
The client went through a number of the iMip tests between their client and another iCalendar client. The following issues were spotted during that and also during client testing originated by other companies.
An original All day event sent in a UK timezone ended up with time shifted to a specific California timezone.
iCalendar sent on behalf of another user did not correctly specify this in the ORGANIZER, although the MIME Sender: and From: headers were correct.
Canceled messages from other server ended up with 2 iCAL “STATUS:CANCELED” entries (not in the original)
Meeting acceptances and cancelled meetings from a server caused problems with incoming internet gateway.
An initial meeting request will often have a sequence number of 1, which another client considers to be an update.
Sequence numbers in Exceptions were not being updated when changes were made to them.
One client discovered an issue where a cache of information for appointments was not being updated when changes were made. This showed up when 2 clients were accessing the same calendar, which may be quite a useful testing methodology as iCalendar. App’s own caching behaviour hides this bug in the originating client.
Thanks for all your work
3.1.3.3. iCalendar Client 3
Using the test scenarios provided, we did the client to client testing with several CalDAV server webmail clients and iCalendar clients to check compatibility. The testing generally covers basic calendar scenarios including:
Invite for Single event, Recurring event, all day event, hourly event, multiday event.
Update single event, series and occurrences in series by changing subject, location, time, date and attendee.
Cancel single event, series, occurrences in series
Meetings with Org and Attendee in different time zones
Calendar responses tracking
3.1.3.4. Testing results
Most tests passed. We are happy to see our application is compatible with most iCalendar formats generated by other applications.
General issues are:
Sequence number:
Some CalDAV server/clients assume sequence number starts with 1 instead of 0. We will show a different banner for such case.
One client doesn’t have this entry in iCalendar. They said using SEQUENCE is optional for organizer according to RFC 2446.
New issues found in our application
Dragging event in Month view will cause a wrong Recurrence-ID generated if the series is not in the same time zone as OS’. It doesn’t repro with other views.
Detailed test results:
One client doesn’t support modified occurrence and negative exceptions well so far. So most testing is done in single event.
Sequence issue mentioned above
We have a known issue: EXDATE doesn’t have time zone information
Another client said they support WebDAV but we couldn’t connect to it. Most likely they don’t support it fully. They will do more investigation.
The exception doesn’t seem to relate to series much except in deletion. Opening or updating such occurrence doesn’t trigger the “for series or for this one” dialog. Important change made to the series(like start/end time or recurring mode) doesn’t overwrite the exceptions. This is a different behavior from other clients.
Another client had Sequence inconsistency — single event or recurring series’ sequence starts from 1 and increases for updates. But exception’s in the series starts at 0 and never increases.
Recurring All day event from our application is not recognized as “All day” in their application. It is shown as a 0 duration event happening each day. Same event synced to another client is displayed correctly and it doesn’t repro with single all day event in the tested application.
One CalDAV server uses RDATE which we do not support. The event can be displayed correctly but the time zone is not recognized.
One client had issues with modifying an occurrence in a recurring series that uses a different time zone than calendar time zone. They generate a wrong Recurrence-ID sometimes. They have the same sequence issue noted above.
Overall this is a very interesting and useful InterOP event! Thanks for organizing it!
4. Summary
This was one of our largest interoperability testing events. Several items were uncovered and generally it was very successful. As usual, it would be nice to have more time. We will be investigating the concept of ongoing, interim testing via the internet to public servers. This will improve the ability to test more applications during our onsite testing events.
Thank you to all the participants and their willingness to take time out of busy schedules to help CalConnect forward the usage of calendaring standards.
Respectfully submitted by Pat Egen, CalConnect Interop Manager.