We have recently landed some changes to the raindrop data-model designed to support ‘dynamic dependencies’ in the data-model. A ‘dynamic’ dependency is a dependency that exists due to the content of an item – for example, consider an extension which is processing a message, but needs to query attributes of the message ‘sender’ to determine its output – this message has a dependency on that sender’s attribute; if the sender’s attribute changes, the extension needs to re-run over the message to pick up the new value. As each message has a different sender, each message has different potential dependencies all determined at runtime based on the content of the message; we call these ‘dynamic’ dependencies.
To support this requirement, we have added new meta-data to each raindrop schema – a field called rd_deps. This item is an array of tuples, which each tuple holding an rd_key and a schema ID. In our example above, the rd_deps on the message object would list the rd_key of the sender, and the schema ID which holds the attribute being examined. There can be any number of rd_deps entries for an item. As raindrop processes the work queue, it checks to see if the document being processed is listed in the rd_deps for any items and if so, all these items are re-processed as if they themselves have changed, thereby picking up the new values.
We currently use this dependency model in two places:
Moving a message to “bulk”:
Raindrop now supports the ability to nominate a particular sender of a message (eg, an email address) as being a “bulk sender”. When a message from a bulk sender is received, it is always marked as a “bulk” message, regardless of how the message itself is addressed. This means that if a particular sender is marked as being a bulk sender after messages have been classified, all messages from that sender must be re-processed so the message is correctly classified with the new attribute. This is done by marking the document with this classification as depending on the identity sending the message. Thus, whenever the attribute on an email address is changed, all messages with that dependency are re-run. Note that this works even though the “bulk” attribute on the email address may not yet exist; it may be created in the future.
Conversation summaries:
A conversation summary is a schema which summarizes the messages in a conversation to apply a number of classifications to the conversation as a whole. For example, while each message has an attribute to indicate whether the message is a “personal” message, we also need to classify the conversation itself as being personal or non-personal based on the attributes of the individual messages. Therefore, the conversation summary schema lists as dependencies the schema holding the “personal” attribute of each message. In other words, there will be as many dependencies on the conversation summary schema as there are messages in the conversation. If the ‘personal’ attribute on any message in the conversation changes, the conversation summary extension is re-run.
You may notice some potential “chaining” between these 2 examples; if the sender of an email is marked as a bulk sender, the classification for all messages sent by that user will be re-calculated. This message re-calculation will then independently cause the conversation summary extension to be re-executed as one of its dependencies (the attribute on the message) changed.
Note that these dependencies must be explicitly nominated by the extension author – only they know what was used in the determination of the values. Thus, the extension which classifies the message must explicitly list the key for the sender, while the conversation summary extension must explicitly list the keys of the messages in the conversation. See the rd.ext.core.convo-to-summary and rd.ext.core.msg-email-to-recip-target extensions for examples.
