Package org.frankframework.jdbc

Class JdbcTransactionalStorage<S extends Serializable>

java.lang.Object

org.frankframework.jndi.JndiBase

org.frankframework.jdbc.JdbcFacade

org.frankframework.jdbc.AbstractJdbcMessageBrowser<M>

org.frankframework.jdbc.JdbcTableMessageBrowser<S>

org.frankframework.jdbc.JdbcTransactionalStorage<S>

All Implemented Interfaces:

HasPhysicalDestination, IConfigurable, IConfigurationAware, IMessageBrowser<S>, INamedObject, IScopeProvider, ITransactionalStorage<S>, IXAEnabled, org.springframework.beans.factory.Aware, org.springframework.context.ApplicationContextAware

Direct Known Subclasses:

DummyTransactionalStorage, MessageStoreSender

public class JdbcTransactionalStorage<S extends Serializable>
extends JdbcTableMessageBrowser<S>
implements ITransactionalStorage<S>

Implements a message log (JdbcMessageLog) or error store (JdbcErrorStorage) that uses database table IBISSTORE. A MessageStoreSender and MessageStoreListener pair implicitly includes a message log and an error store. If you have a MessageStoreSender and MessageStoreListener pair it is superfluous to add a JdbcMessageLog or JdbcErrorStorage within the same sender pipe or the same receiver.

Message log: A message log writes messages in persistent storage for logging purposes. When a message log appears in a receiver, it also ensures that the same message is only processed once, even if a related pushing listener receives the same message multiple times.

Error store: Appears in a receiver or sender pipe to store messages that could not be processed. Storing a message in the error store is the last resort of the Frank!Framework. Many types of listeners and senders offer a retry mechanism. Only if several tries have failed, then an optional transaction is not rolled back and the message is stored in the error store. Users can retry messages in an error store using the Frank!Console. When this is done, the message is processed in the same way as messages received from the original source.

How does a message log or error store see duplicate messages? The message log or error store always appears in combination with a sender or listener. This sender or listener determines a key based on the sent or received message. Messages with the same key are considered to be the same.

Storage structure is defined in /IAF_util/IAF_DatabaseChangelog.xml. If these database objects do not exist, the Frank!Framework will try to create them.

N.B. Note on using XA transactions: If transactions are used on Oracle, make sure that the database user can access the table SYS.DBA_PENDING_TRANSACTIONS. If not, transactions present when the server goes down cannot be properly recovered, resulting in exceptions like:

   The error code was XAER_RMERR. The exception stack trace follows: javax.transaction.xa.XAException
        at oracle.jdbc.xa.OracleXAResource.recover(OracleXAResource.java:508)
   

Since:

4.1

Author:

Gerrit van Brakel, Jaco de Groot

Nested Class Summary

Nested classes/interfaces inherited from interface org.frankframework.core.IMessageBrowser

IMessageBrowser.HideMethod, IMessageBrowser.SortOrder, IMessageBrowser.StorageType

Field Summary

Fields

Modifier and Type	Field	Description
protected String	insertQuery
protected static final int	MAXCIDLEN
protected static final int	MAXIDLEN
protected static final int	MAXLABELLEN
protected String	selectKeyForMessageQuery
protected org.springframework.transaction.PlatformTransactionManager	txManager

Fields inherited from class org.frankframework.jdbc.JdbcTableMessageBrowser

useIndexHint

Fields inherited from class org.frankframework.jdbc.AbstractJdbcMessageBrowser

checkCorrelationIdQuery, checkMessageIdQuery, CONTROL_PROPERTY_PREFIX, deleteQuery, getMessageCountQuery, PROPERTY_ASSUME_PRIMARY_KEY_UNIQUE, PROPERTY_USE_PARAMETERS, selectContextQuery, selectDataQuery

Fields inherited from class org.frankframework.jndi.JndiBase

log

Fields inherited from interface org.frankframework.core.ITransactionalStorage

MAXCOMMENTLEN

Constructor Summary

Constructors

Constructor	Description
JdbcTransactionalStorage()

Method Summary

Modifier and Type	Method	Description
void	configure()	Creates a connection, checks if the table is existing and creates it when necessary
protected void	createQueryTexts(IDbmsSupport dbmsSupport)
protected void	createStorage(Connection conn, Statement stmt, IDbmsSupport dbmsSupport)	Acutaly creates storage.
String	getDateFieldType()
String	getKeyFieldType()
protected String	getLogPrefix()
RawMessageWrapper<S>	getMessage(String storageKey)	Retrieves and deletes the message.
String	getMessageFieldType()
int	getRetention()
String	getSchemaOwner4Check()
String	getSequenceName()
protected String	getStorageRefKey()
String	getTextFieldType()
org.springframework.transaction.PlatformTransactionManager	getTxManager()
void	initialize(IDbmsSupport dbmsSupport)	Checks if table exists, and creates when necessary.
boolean	isBlobsCompressed()
boolean	isCheckIndices()
boolean	isCheckTable()
boolean	isCreateTable()
boolean	isOnlyStoreWhenMessageIdUnique()
boolean	isStoreFullMessage()
protected RawMessageWrapper<S>	retrieveObject(String storageKey, ResultSet rs, int columnIndex)
void	setBlobsCompressed(boolean b)	If set to true, the messages are stored compressed
void	setCheckIfTableExists(boolean b)	Deprecated, for removal: This API element is subject to removal in a future version.
void	setCheckTable(boolean b)	If set to true, checks are performed if the table exists and is properly created
void	setCreateTable(boolean b)	Deprecated, for removal: This API element is subject to removal in a future version.
protected void	setDataTypes(IDbmsSupport dbmsSupport)	change datatypes used for specific database vendor.
void	setDateFieldType(String string)	The type of the column the timestamps are stored in
void	setHostField(String hostField)	The name of the column that stores the hostname of the server
void	setKeyFieldType(String string)	The type of the column that contains the primary key of the table
void	setMessageFieldType(String string)	The type of the column message themselves are stored in
void	setOnlyStoreWhenMessageIdUnique(boolean onlyStoreWhenMessageIdUnique)	If set to true, the message is stored only if the MessageId is not present in the store yet.
protected void	setOperationControls()
void	setRetention(int retention)	The time (in days) to keep the record in the database before making it eligible for deletion by a cleanup process. when set to -1, the record will live on forever
void	setSchemaOwner4Check(String string)	Schema owner to be used to check the database
void	setSequenceName(String string)	The name of the sequence used to generate the primary key, for DBMSes that use sequences, like Oracle
void	setSlotId(String string)	Set the slotId, an identifier to keep separate the messages inserted by different JdbcTransactionalStorage instances.
void	setSlotIdField(String string)	The name of the column slotids are stored in
void	setStoreFullMessage(boolean storeFullMessage)	If set to true, the full message is stored with the log.
void	setTextFieldType(String string)	The type of the columns messageId and correlationId, slotId and comments are stored in.
void	setTxManager(org.springframework.transaction.PlatformTransactionManager txManager)
void	setType(String type)	Possible values are E (error store), M (message store), L (message log for Pipe) or A (message log for Receiver). Receiver will always set type to E for errorStorage and always set type to A for messageLog.
void	setTypeField(String typeField)	The name of the column types are stored in
void	start()	Prepares the object for operation.
String	storeMessage(String messageId, String correlationId, Date receivedDate, String comments, String label, S message)	Store the message, returns storageKey.
String	storeMessage(Connection conn, String messageId, String correlationId, Date receivedDate, String comments, String label, S message)	Stores a message in the database and retrieves the value of the primary key for the record just inserted.
protected String	storeMessageInDatabase(Connection conn, String messageId, String correlationId, Timestamp receivedDateTime, String comments, String label, S message)

Methods inherited from class org.frankframework.jdbc.JdbcTableMessageBrowser

createSelector, getFromClause, getIndexName, getPhysicalDestinationName, getSelectListQuery, getTableName, provideFirstRowsHintAfterFirstKeyword, provideIndexHintAfterFirstKeyword, provideTrailingFirstRowsHint, setIndexName, setTableName

Methods inherited from class org.frankframework.jdbc.AbstractJdbcMessageBrowser

applyStandardParameters, applyStandardParameters, browseMessage, containsCorrelationId, containsMessageId, copyFacadeSettings, deleteMessage, getCommentField, getContext, getCorrelationIdField, getDatasource, getDateField, getExpiryDateField, getHideMethod, getHideRegex, getHostField, getIdField, getIterator, getIterator, getKeyField, getLabelField, getMessageCount, getMessageField, getOrder, getPrefix, getSlotId, getSlotIdField, getType, getTypeField, getWhereClause, setCommentField, setCorrelationIdField, setDateField, setExpiryDateField, setHideMethod, setHideRegex, setIdField, setKeyField, setLabelField, setMessageField, setOrder, setPrefix

Methods inherited from class org.frankframework.jdbc.JdbcFacade

getAuthAlias, getConnection, getConnectionWithTimeout, getDataSourceFactory, getDatasourceInfo, getDatasourceName, getDbmsSupport, getDomain, getPassword, getUsername, isConnectionsArePooled, isTransacted, setAuthAlias, setConnectionsArePooled, setDataSourceFactory, setDatasourceName, setDbmsSupportFactory, setJmsRealm, setPassword, setTransacted, setUsername

Methods inherited from class org.frankframework.jndi.JndiBase

getApplicationContext, getAuthentication, getConfigurationClassLoader, getContext, getCredentials, getInitialContextFactoryName, getJmsRealmName, getJndiAuthAlias, getJndiContextPrefix, getJndiEnv, getJndiProperties, getName, getPrincipal, getProviderURL, getSecurityProtocol, getUrlPkgPrefixes, setApplicationContext, setAuthentication, setCredentials, setInitialContextFactoryName, setJndiAuthAlias, setJndiContextPrefix, setJndiProperties, setName, setPrincipal, setProviderURL, setSecurityProtocol, setUrlPkgPrefixes, stop, toString

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Methods inherited from interface org.frankframework.core.IMessageBrowser

browseMessage, containsCorrelationId, containsMessageId, deleteMessage, getContext, getHideMethod, getHideRegex, getIterator, getIterator, getMessageCount, setHideMethod, setHideRegex

Methods inherited from interface org.frankframework.core.INamedObject

getName, setName

Methods inherited from interface org.frankframework.core.ITransactionalStorage

getSlotId, getType, stop

Methods inherited from interface org.frankframework.core.IXAEnabled

isTransacted

Field Details

MAXIDLEN

protected static final int MAXIDLEN

See Also:

• Constant Field Values

MAXCIDLEN

protected static final int MAXCIDLEN

See Also:

• Constant Field Values

MAXLABELLEN

protected static final int MAXLABELLEN

See Also:

• Constant Field Values

insertQuery

protected String insertQuery

selectKeyForMessageQuery

protected String selectKeyForMessageQuery

txManager

protected org.springframework.transaction.PlatformTransactionManager txManager

Constructor Details

JdbcTransactionalStorage

public JdbcTransactionalStorage()

Method Details

getLogPrefix

protected String getLogPrefix()

Overrides:

getLogPrefix in class AbstractJdbcMessageBrowser<S extends Serializable>

setOperationControls

protected void setOperationControls()

Overrides:

setOperationControls in class JdbcTableMessageBrowser<S extends Serializable>

configure

public void configure()
               throws ConfigurationException

Creates a connection, checks if the table is existing and creates it when necessary

Specified by:

configure in interface IConfigurable

Specified by:

configure in interface ITransactionalStorage<S extends Serializable>

Overrides:

configure in class JdbcTableMessageBrowser<S extends Serializable>

Throws:

ConfigurationException

start

public void start()

Description copied from interface: ITransactionalStorage

Prepares the object for operation. After this method is called the storeMessage() and retrieveMessage() methods may be called

Specified by:

start in interface ITransactionalStorage<S extends Serializable>

setDataTypes

protected void setDataTypes(IDbmsSupport dbmsSupport)

change datatypes used for specific database vendor.

createQueryTexts

protected void createQueryTexts(IDbmsSupport dbmsSupport)
                         throws ConfigurationException

Overrides:

createQueryTexts in class JdbcTableMessageBrowser<S extends Serializable>

Throws:

ConfigurationException

initialize

public void initialize(IDbmsSupport dbmsSupport)
                throws JdbcException,
SQLException

Checks if table exists, and creates when necessary.

Throws:

JdbcException

SQLException

createStorage

protected void createStorage(Connection conn,
 Statement stmt,
 IDbmsSupport dbmsSupport)
                      throws JdbcException

Acutaly creates storage. Can be overridden in descender classes

Throws:

JdbcException

storeMessageInDatabase

@Nonnull
protected String storeMessageInDatabase(Connection conn,
 String messageId,
 String correlationId,
 Timestamp receivedDateTime,
 String comments,
 String label,
 S message)
                                 throws IOException,
SQLException,
JdbcException,
SenderException

Throws:

IOException

SQLException

JdbcException

SenderException

storeMessage

@Nonnull
public String storeMessage(String messageId,
 String correlationId,
 Date receivedDate,
 String comments,
 String label,
 S message)
                    throws SenderException

Description copied from interface: ITransactionalStorage

Store the message, returns storageKey. The messageId should be unique.

Specified by:

storeMessage in interface ITransactionalStorage<S extends Serializable>

Throws:

SenderException

storeMessage

@Nonnull
public String storeMessage(@Nonnull
 Connection conn,
 @Nonnull
 String messageId,
 @Nonnull
 String correlationId,
 @Nonnull
 Date receivedDate,
 @Nullable
 String comments,
 @Nullable
 String label,
 @Nonnull
 S message)
                    throws SenderException

Stores a message in the database and retrieves the value of the primary key for the record just inserted.

Parameters:

conn - the database connection

messageId - the ID of the message

correlationId - the correlation ID of the message

receivedDate - the date when the message was received

comments - additional comments for the message (optional)

label - the label for the message (optional)

message - the message object to be stored

Returns:

the value of the primary key for the inserted record

Throws:

SenderException - if there is an error storing the message

retrieveObject

protected RawMessageWrapper<S> retrieveObject(String storageKey,
 ResultSet rs,
 int columnIndex)
                                       throws JdbcException

Overrides:

retrieveObject in class JdbcTableMessageBrowser<S extends Serializable>

Throws:

JdbcException

getMessage

public RawMessageWrapper<S> getMessage(String storageKey)
                                throws ListenerException

Description copied from interface: ITransactionalStorage

Retrieves and deletes the message.

Specified by:

getMessage in interface ITransactionalStorage<S extends Serializable>

Throws:

ListenerException

getStorageRefKey

protected String getStorageRefKey()

setSlotId

public void setSlotId(String string)

Set the slotId, an identifier to keep separate the messages inserted by different JdbcTransactionalStorage instances.
This field is optional, unless the attribute onlyStoreWhenMessageIdUnique is set to true.

Specified by:

setSlotId in interface ITransactionalStorage<S extends Serializable>

Overrides:

setSlotId in class AbstractJdbcMessageBrowser<S extends Serializable>

Parameters:

string - The slotID value for this JdbcTransactionalStorage.

setType

public void setType(String type)

Description copied from interface: ITransactionalStorage

Possible values are E (error store), M (message store), L (message log for Pipe) or A (message log for Receiver).
Receiver will always set type to E for errorStorage and always set type to A for messageLog. SenderPipe will set type to L for messageLog (when type isn't specified).
See MessageStoreSender for type M.

Specified by:

setType in interface ITransactionalStorage<S extends Serializable>

Overrides:

setType in class AbstractJdbcMessageBrowser<S extends Serializable>

setSlotIdField

public void setSlotIdField(String string)

The name of the column slotids are stored in

Overrides:

setSlotIdField in class AbstractJdbcMessageBrowser<S extends Serializable>

Default value

SLOTID

setTypeField

public void setTypeField(String typeField)

The name of the column types are stored in

Overrides:

setTypeField in class AbstractJdbcMessageBrowser<S extends Serializable>

Default value

TYPE

setHostField

public void setHostField(String hostField)

The name of the column that stores the hostname of the server

Overrides:

setHostField in class AbstractJdbcMessageBrowser<S extends Serializable>

Default value

HOST

setSequenceName

public void setSequenceName(String string)

The name of the sequence used to generate the primary key, for DBMSes that use sequences, like Oracle

Default value

seq_ibisstore

setCheckIfTableExists

@Deprecated(forRemoval=true,
            since="7.7.0")
@ConfigurationWarning("Replaced with checkTable")
public void setCheckIfTableExists(boolean b)

Deprecated, for removal: This API element is subject to removal in a future version.

setCheckTable

public void setCheckTable(boolean b)

If set to true, checks are performed if the table exists and is properly created

Default value

false

setCreateTable

@Deprecated(forRemoval=true,
            since="7.9.0")
@ConfigurationWarning("if you want to create and maintain database tables, please enable Liquibase")
public void setCreateTable(boolean b)

Deprecated, for removal: This API element is subject to removal in a future version.

If set to true, the table is created if it does not exist

Default value

false

setMessageFieldType

public void setMessageFieldType(String string)

The type of the column message themselves are stored in

setKeyFieldType

public void setKeyFieldType(String string)

The type of the column that contains the primary key of the table

setDateFieldType

public void setDateFieldType(String string)

The type of the column the timestamps are stored in

setTextFieldType

public void setTextFieldType(String string)

The type of the columns messageId and correlationId, slotId and comments are stored in. N.B. (100) is appended for id's, (1000) is appended for comments.

setBlobsCompressed

public void setBlobsCompressed(boolean b)

If set to true, the messages are stored compressed

Default value

true

setRetention

public void setRetention(int retention)

The time (in days) to keep the record in the database before making it eligible for deletion by a cleanup process. when set to -1, the record will live on forever

Default value

30

setSchemaOwner4Check

public void setSchemaOwner4Check(String string)

Schema owner to be used to check the database

Default value

<current_schema> (only for oracle)

setStoreFullMessage

public void setStoreFullMessage(boolean storeFullMessage)

If set to true, the full message is stored with the log. Can be set to false to reduce table size, by avoiding to store the full message

Default value

true

setOnlyStoreWhenMessageIdUnique

public void setOnlyStoreWhenMessageIdUnique(boolean onlyStoreWhenMessageIdUnique)

If set to true, the message is stored only if the MessageId is not present in the store yet.

Default value

false

isCheckTable

public boolean isCheckTable()

isCheckIndices

public boolean isCheckIndices()

isCreateTable

public boolean isCreateTable()

isBlobsCompressed

public boolean isBlobsCompressed()

isStoreFullMessage

public boolean isStoreFullMessage()

getRetention

public int getRetention()

getSchemaOwner4Check

public String getSchemaOwner4Check()

isOnlyStoreWhenMessageIdUnique

public boolean isOnlyStoreWhenMessageIdUnique()

getKeyFieldType

public String getKeyFieldType()

getDateFieldType

public String getDateFieldType()

getMessageFieldType

public String getMessageFieldType()

getTextFieldType

public String getTextFieldType()

getSequenceName

public String getSequenceName()

getTxManager

public org.springframework.transaction.PlatformTransactionManager getTxManager()

setTxManager

public void setTxManager(org.springframework.transaction.PlatformTransactionManager txManager)