Real time architecture
The architecture provides a framework for designing and deploying event based data processing solutions with the O2MC I/O platform at the heart. The framework provides insight in several components of the O2MC I/O platform as well as the integration with external applications. Below a description of each of the layers in the architecture and its components. The O2MC I/O Platform components can be used from the current cloud platform but they can also be installed on premise. Since the events are generated at the bottom, in the source layer, that layer will be described first before moving up.
The O2MC I/O platform can capture data from any first, second or third party source. The only prerequisite is that the source is connected to the internet. This includes IoT sensors, websites, databases, native apps and many more. Since the architecture is aimed at real time processing of the data, the sources provides the data the moment an interaction takes place.
The integration layer provides overall robustness on the processing the events. The event hub makes sure that all events are (eventually) processed, dealing with spikes in volume and data unavailability. Additionally, event processing is facilitated as well as providing the possibility of including reference data (e.g. IP location look up) in the event on the fly.
Real time processing
The in memory analytics component is the core of the O2MC I/O platform. Tens of thousands of incoming events will be processed in memory in real time. Processing includes but is not limited to ETL, sessionization, cross device user identification, predictive machine learning and other Big Data applications. The results can be distributed to many different end point with different data sets, granularity and intervals.
The O2MC I/O platform has been designed with Privacy by design in mind. As a result data governance is at the core of the O2MC I/O platform and always included in any application. To illustrate this events will not be processed if the owner of the data has not given permission to do so. It is unable for any developer to work around these settings. Additional the platform facilitates an extensive multi-tenant environment with extensive authentication and application isolation.
Apart from real time integrations with third applications, the O2MC I/O platform can provide structured data to databases like MySQL, Oracle and MongoDB as well as provide unstructured data to log files on (s)FTP servers, Hadoop environments and many, many other data stores. The query and connection details can be fully customized and the data stream can be throttled to prevent spikes from causing problems in the data sources. The O2MC I/O platform does not store data so all data is located outside the platform at any location that is desired.
Since the data in the structured and unstructured data stores can be fully customized, using a visualization tool like Tableau, ClickView or PowerBI requires little setup since the data is ready made for reporting. Additionally, the log files can be created in a way such that all the necessary fields and formats are in place for easy integration. The O2MC I/O platform can also integrate real time directly with a third party applications or use any of the source connectors. This can be used to create real time personalization in websites, provide automated remarketing application with real time instructions and deliver relevant information the moment it is available. Repurposing the data in other O2MC I/O platform applications where many best practice libraries are available is also a common use.