13. Stores and RecordFactory

13.1. Stores and Substores

When you need to save and keep some information you should use stores. “Store types” shows different types of stores at their hierarchy.

Store types

Store types

13.1.1. MAPI Store

Zarafa MAPi Store is an extension of the Ext JS store which adds support for the open command, which is used by MAPI to request additional data for a record.

An important note here is that Zarafa.core.data.MAPIStore is not a store itself, but is used to collect MAPI Messages.

For example,

Ext.namespace('Zarafa.plugins.facebook.data');

/**
 * @class Zarafa.plugins.facebook.data.FbEventStore
 * @extends Zarafa.core.data.MAPIStore
 *
 * This class extends MAPIStore to configure the
 * proxy and reader in custom way.
 * Instead of defining the records dynamically, reader will
 * create {@link Zarafa.plugins.facebook.data.fbEventRecord} instance.
 *
 */
Zarafa.plugins.facebook.data.FbEventStore = Ext.extend(Zarafa.core.data.MAPIStore,
{
        /**
         * @constructor
         * @param {Object} config Configuration object
         *
         */
        constructor : function(config)
        {
                config = config || {};

                Ext.applyIf(config, {
                        reader : new Zarafa.plugins.facebook.data.FbEventJSONReader({
                        id : 'id',
                        idProperty : 'id',
                        dynamicRecord : false
                }),
                        writer : new Zarafa.core.data.JsonWriter(),
                        proxy  : new Zarafa.core.data.IPMProxy()
                });

                Zarafa.plugins.facebook.data.FbEventStore.superclass.constructor.call(this, config);
        }
});

Ext.reg('facebook.fbeventstore', Zarafa.plugins.facebook.data.FbEventStore);

This is a code example of Facebook plugin. We use the MAPI interface for compatibility with Microsoft Outlook. As far as MAPI provides full control over the messaging system, it is convenient to use it in Kopano WebApp.

A store is a client-side cache of items that exist on the server, and provides a clean interface for loading and CRUD (create, read, update and delete) operations. Many standard Ext JS components use stores to manage the data they display and operate on. In MVC (Model–View–Controller) terms, a store is the default model for many UI components. A common example is the grid panel (Ext.grid.GridPanel).

Displaying a list of tasks in a tasks folder is a matter of constructing a store instance, connecting it to the grid, and calling the load method with a Folder object as a parameter. The grid will automatically issue a generic load command to the store to populate it with data which is then displayed.

Data flow

Data flow

“Data flow” shows how the various components connect to get data from the server to display in a data grid in the browser. A grid panel is connected to a store, which acts as a local cache containing a set of records. The store uses a proxy to talk to the server, which in turn tasks with a server-side list module using the Zarafa communication scheme “Request-Response flow”. The server has different list modules for each type of data (tasks, mail, etc), and there are corresponding stores and proxies on the client.

Stores can do more than just plain data loading. They support pagination and sorting, and it’s very easy to get this to work with the standard Ext JS components. Records can be added, removed, or updated. Changes made to the data in a store can be committed to the server by calling the save method.

A MAPI message consists of properties, but in some cases also a contents table. A spreed meting request could, for example, contain Recipients or Attachments. Distribution lists on the other hand have a list of Members. This data does not fit into the Ext JS model by default. But in the Zarafa.core.data.MAPIRecord support for SubStores has been added.

Each MAPIRecord can contain multiple SubStores which are all serialized/deserialized to/from JSON during the communication with the server. The implementation has been generalized in such a way, that plugins are able to define their own SubStores for records. The contents of a SubStore is serialized/deserialized using the JsonReader/ JsonWriter which have been configured on the SubStore itself. This means that for plugin developers they can easily create their custom table by registering the name and the type of the SubStore on the RecordFactory, and make sure a custom JsonReader and JsonWriter are set on the SubStore.

Example of plugin JSONreader is Facebook event JSON reader:

/*
 * #dependsFile client/zarafa/core/data/RecordCustomObjectType.js
 */
Ext.namespace('Zarafa.plugins.facebook.data');

/**
 * @class Zarafa.plugins.facebook.data.FbEventJSONReader
 * @extends Zarafa.core.data.JsonReader
 */
Zarafa.plugins.facebook.data.FbEventJSONReader = Ext.extend(Zarafa.core.data.JsonReader,
{
        /**
         * @cfg {Zarafa.core.data.RecordCustomObjectType} customObjectType The
         * custom object type which represents the {@link Ext.data.Record
         * records} which should be created using {@link Zarafa.core.data.
         * RecordFactory#createRecordObjectByCustomType}.
         */
        customObjectType : Zarafa.core.data.RecordCustomObjectType.ZARAFA_FACEBOOK_EVENT,

                /**
                 * @constructor
                 * @param {Object} config Configuration options.
                 */
                constructor : function(meta, recordType)
                {
                        meta = Ext.applyIf(meta || {}, {
                                id : 'id',
                                idProperty : 'id',
                                dynamicRecord : false
                        });

                        // If no recordType is provided, force the type to be a recipient
                        if (!Ext.isDefined(recordType)) {
                                recordType = Zarafa.core.data.RecordFactory.getRecordClassByCustomType(meta.customObjectType || this.customObjectType);
                        }

                        Zarafa.plugins.facebook.data.FbEventJSONReader.superclass.constructor.call(this, meta, recordType);
                }
});

Plugin JSON writer can be found, for example, in Spreed attachments JSON writer:

Ext.namespace('Zarafa.plugins.spreed.data');

/**
 * @class Zarafa.plugins.spreed.data.SpreedJsonAttachmentWriter
 * @extends Zarafa.core.data.JsonAttachmentWriter
 */
Zarafa.plugins.spreed.data.SpreedJsonAttachmentWriter = Ext.extend(Zarafa.core.data.JsonAttachmentWriter,
{
        /**
         * Similar to {@link Zarafa.core.data.JsonAttachmentWriter#toHash}.
         * Here we serializing only the data of the records in spreed attachment store.
         * Note that we serialize all the records - not only removed or modified.
         *
         * @param {Ext.data.Record} record The record to hash
         * @return {Object} The hashed object
         * @override
         * @private
         */
        toPropHash : function(record)
        {
                var attachmentStore = record.getAttachmentStore();
                var hash = {};

                if (!Ext.isDefined(attachmentStore))
                        return hash;

                // Overwrite previous definition to something we can work with.
                hash.attachments = {};
                hash.attachments.dialog_attachments = attachmentStore.getId();

                var attachmentRecords = attachmentStore.getRange();
                Ext.each(attachmentRecords, function(attach) {
                        if (!Ext.isDefined(hash.attachments.add)) {
                                hash.attachments.add = [];
                        }
                        var data = attach.data;
                        hash.attachments.add.push(data);
                }, this);

                return hash;
        }
});

The name of the SubStore is used for the Json Data. The contents of the SubStore will be serialized with this name into the Json Object.

Let’s consider spreed substores code for example:

/**
 * #dependsFile client/zarafa/core/data/IPMRecipientStore.js
 */
Ext.namespace('Zarafa.plugins.spreed.data');

/**
* @class Zarafa.plugins.spreed.data.SpreedParticipantStore
* @extends Zarafa.core.data.IPMRecipientStore
*/
Zarafa.plugins.spreed.data.SpreedParticipantStore=Ext.extend(Zarafa.core.data.IPMRecipientStore,
{
        /**
        * @constructor
        * @param config {Object} Configuration object
        */
        constructor : function(config) {
                config = config || {};

                Ext.applyIf(config, {
                        writer : new Zarafa.plugins.spreed.data.SpreedJsonParticipantWriter(),
                        customObjectType : Zarafa.core.data.RecordCustomObjectType.ZARAFA_SPREED_PARTICIPANT,
                        reader: new Zarafa.core.data.JsonRecipientReader({
                                id : 'entryid',
                                idProperty : 'entryid',
                                dynamicRecord : false
                        })
                });

                Zarafa.plugins.spreed.data.SpreedParticipantStore.superclass.constructor.call(this, config)
        }
});

Ext.reg('spreed.spreedparticipantstore', Zarafa.plugins.spreed.data.SpreedParticipantStore);

This is how it is reflected in SpreedRecord class, in constructor:

constructor: function(data)
{
        data = data || {};

        this.initialRecordData.participants = new Ext.util.MixedCollection();
        Zarafa.plugins.spreed.data.SpreedRecord.superclass.constructor.call(this, data);
        this.collectedDataRecords = [];
        this.subStoresTypes =
        {
                'recipients'  : Zarafa.plugins.spreed.data.SpreedParticipantStore
        };
}

Where initialRecordData is an Object of recipients taken from the mailRecords, if they are opened. An important note regarding SubStores is that the SubStore is guaranteed to be available when the Record has been opened (or when it is a phantom record). When the Record has not yet been opened, the SubStore will only have been allocated if the original JsonData contains the data for the SubStore (which is not recommended).

13.1.2. IPM Stores and ShadowStore

The Zarafa implementation of the Ext.data.Store is the IPMStore. Refer to “IPM Store explanation”. This Store contains IPMRecords, and any IPMRecord must always belong to an IPMStore. Each IPMStore is managed by the IPMStoreMgr singleton. The IPMStore has two base classes, the first one is the ListModuleStore which is used in each Context to list all IPMRecords which must be displayed in the Context.

Store UML diagram

Store UML diagram

The second one is ShadowStore contains IPMRecords which are currently being edited within a Dialog, this includes new IPMRecords which must still be created on the server side.

IPM Store explanation

IPM Store explanation

When a Dialog starts editing a IPMRecord it must copy the IPMRecord to the ShadowStore and work on that copy. When a Dialog closes it must remove record from the ShadowStore (after optionally saving the IPMRecord to the server).

Any events from a IPMStore regarding the update for IPMRecords will always be caught by the IPMStoreMgr and raised as separate event from this manager. Any UI Component which contains an IPMRecord must listen to the IPMStoreMgr for events to determine if the IPMRecord has changed and the component has to be updated. Note that listening to the IPMStore to which the Record belongs is not sufficient, because Dialogs place a copy of the IPMRecord into the ShadowStore. In which case the same Message with the same EntryId is represented by two IPMRecords in two different IPMStores.

13.2. RecordFactory

Record definitions are managed by the Zarafa.core.data.RecordFactory. Within the RecordFactory two groups of Record definitions exists, the first group is based on the message class (IPM.Note, IPM.Contact, etc) while the other group is based on the object type (MAPI_MAILUSER, MAPI_DISTLIST, etc). The reason for having two groups comes from MAPI which does not define the PR_MESSAGE_CLASS property for all possible objects (most notably for Address Book items they are missing), while the PR_OBJECT_TYPE is too global to be used in all cases (There is no different value for a Mail and a Contact for instance). Ext JS allows Records to be defined using a list of possible fields, these fields have a name (the property name) and possible serialiation/deserialization information (conversion from text to Integer, Date, Boolean, etc). When a Record is created, this list is used to define which properties will be serialized/deserialized during the communication with the server. As a result, if a Plugin wishes to send an extra property it somehow has to tell Ext JS the property is valid for this Record. This is where the RecordFactory is utilized.

The RecordFactory contains the complete field definitions for all possible Record definitions. During loading, Contexts and Plugins can tell the RecordFactory which fields they wish to use for a particular Message class or object type. For example:

Zarafa.core.data.RecordFactory.addFieldToMessageClass('IPM.Note', [
        {name: 'from'},
]);

This adds the field from to the Record definition for any Record with messageclass IPM.Note. Or, according to Spreed plugin:

Zarafa.core.data.RecordFactory.addFieldToCustomType(Zarafa.core.data.RecordCustomObjectType.ZARAFA_SPREED_ATTACHMENT, Zarafa.plugins.spreed.data.SpreedAttachmentRecordFields);

Where Zarafa.plugins.spreed.data.SpreedAttachmentRecordFields is a set of fields to add to our record:

Zarafa.plugins.spreed.data.SpreedAttachmentRecordFields = [
        {name : 'original_record_entryid', type : 'string', defaultValue : ''},
        {name : 'original_record_store_entryid', type : 'string', defaultValue : ''},
        {name : 'original_attach_num', type : 'int'},
        {name : 'original_attachment_store_id', type : 'string', defaultValue : ''}
];

When adding a new object type it should be added to RecordCustomObjectType enum. Refer to Enums for more information.

Zarafa.core.data.RecordCustomObjectType.addProperty('ZARAFA_SPREED_ATTACHMENT');

When the user creates a new Record, it must call the RecordFactory to request the Record object.

Zarafa.core.data.RecordFactory.createRecordObjectByMessageClass('IPM.Note');

This will create a new phantom record, for the messageclass IPM.Note. This Record now only has a single field which is allowed; namely from.

The plugin variant - as we used in SpreedRecord in conversion function:

convertToSpreedAttachment : function(attachmentRecord)
{
        return Zarafa.core.data.RecordFactory.createRecordObjectByCustomType(Zarafa.core.data.RecordCustomObjectType.ZARAFA_SPREED_ATTACHMENT, attachmentRecord.data);
},

Here we obtain a new Record object, of the custom type ZARAFA_SPREED_ATTACHMENT, with field values taken from the second parameter object. If the second parameter is not specified, values are taken from the field default value will be assigned, e.g. for ‘original_attachment_store_id’ default value will be ‘’.

If we want that a new phantom Records for IPM.Note contain a default value, we can also instruct the RecordFactory:

Zarafa.core.data.RecordFactory.addDefaultValueToMessageClass('IPM.Note', 'from', 'me');

Now when we create a new Record for messageclass IPM.Note the record will automatically have the property from initialized to me.

By default all Record instances which are created by the Record factory use the Ext.data.Record as baseclass. This however is also configurable using the function setBaseClassToMessageClass. To force the usage of Zarafa.core.data.IPMRecord to IPM.Note we can use the statement as follows:

Zarafa.core.data.RecordFactory.setBaseClassToMessageClass('IPM.Note', Zarafa.core.data.IPMRecord);

Now all new Record instances of Zarafa.core.data.IPMRecord for IPM.Note will inherit from Zarafa.core.data.IPMRecord.

The same example with Spreed plugin:

Zarafa.core.data.RecordFactory.setBaseClassToCustomType(Zarafa.core.data.RecordCustomObjectType.ZARAFA_SPREED_ATTACHMENT, Zarafa.core.data.IPMAttachmentRecord);

Inheritence for the messageclass works quite simple. The inheritence tree for Message class is:

'IPM' -> 'IPM.Note' -> 'IPM.Note.test'
      -> 'IPM.Contact'

When a field, default value or base class is assigned to IPM it is automatically propogated to IPM.Note and IPM.Contact. These values can simply be overridden within such a subdefinition. For example:

Zarafa.core.data.RecordFactory.addFieldToMessageClass('IPM', [
        {name: 'body'},
]);

Zarafa.core.data.RecordFactory.addFieldToMessageClass('IPM.Note', [
        {name: 'from'},
]);

Now all Record instances of IPM, IPM.Note and IPM.Contact will contain the field body, but only IPM.Note will have the additional from field. If we now extend our example with default values:

Zarafa.core.data.RecordFactory.addDefaultValueToMessageClass('IPM', 'body', 'test');
Zarafa.core.data.RecordFactory.addDefaultValueToMessageClass('IPM.Contact', 'body', 'contact');

The record instances of IPM and IPM.Note will by default have the value test in the body property. However IPM.Contact will have the value contact in that property.

To add support for a SubStore, the following function can be called:

Zarafa.core.data.RecordFactory.setSubStoreToMessageClass('IPM.Note', 'recipients', Zarafa.core.data.IPMRecipientStore);

This will register the SubStore with the name recipients to all Records which have the MessageClass IPM.Note. The SubStore will be created using the passed constructor (Zarafa.core.data.IPMRecipientStore). The name can be used on a MAPIRecord to detect if it supports this particular SubStore:

var record = Zarafa.core.data.RecordFactory.createRecordObjectByMessageClass('IPM.Note');
record.supportsSubStore('recipients');
// True if the record supports the subStore
record.getSubStore('recipients');
// this returns the Zarafa.core.data.IPMRecipientStore allocated for this record

A final warning, adding fields, default values or base classes can only be done during initial loading. This means that these statements must be done outside any function or class. Calling Zarafa.core.data.RecordFactory.createRecordObjectByMessageClass can only be made safely after the initial loading (thus it can be done safely in functions and classes).