D3.5 is the fifth deliverable of WP3 and is entitled 1st Modular architecture and tools for interoperability and virtualisation. It proposes novel methods to contribute with innovative solution for the development of a Data Collection Framework (DCF). This solution is based on two new research challenges namely: resource virtualisation and information interoperability. Interoperability aims to ensure compatible information exchange and autonomous information processing, while resource virtualization fosters system integration, reduce complexity and allow rapid access to resources. Together, these two concepts, when used correctly, will improve system reliability and security.
The proposed DCF can be considered as a comprehensive solution that facilitates collection of raw and complex data provided by different kinds of IoT sources presented in D3.1 entitled 1st Data Collection from real-world resources: IoT devices and legacy systems defined in D3.3 entitled 1st Data collection from resources virtualisation: legacy systems integration such as products location data, supply chain management information, production and delivery plans, etc. In addition to real-time data collection capabilities supported by Publish/Subscribe Message queue, the Complex Event Processing of the DCF (DCF-CEP) is able to provide pattern detection against vast amounts of streamed data, for example: temporal event detection and earlier detection of critical situations. The DCF-CEP uses rules for pattern detection that can be added, modified or deployed to the real-time analysis system seamlessly using DCF resources management. Incorporating a big data storage solution to C2NET offers great power of analysis as well as persistent storage. More specifically, the DCF provides clean and ready-to-process collected data along with the transformation and mapping component.
In the current document, we have defined a comprehensive architecture for data collection. For this, we have detailed the main components and functionalities, and defined the contributors to each one of them. Besides, in order to handle data collection challenges, we specified the different interactions between these components. In the development design process, this architectural solution relies on existing technologies defined in the deliverable D2.10 entitled Selection Technologies and Development Process Guidelines Document. Moreover, it is based on two pillars: Interoperability and Virtualisation in order to ensure the relevance of the proposed DCF architecture.The results from this work are encourging. We have defined a comprehensive architecture for data collection. The components are clearly detailed and assigned to contributors. We also defined how these components will interact and work together to handle data collection challenges. The architectural solution is promising and can benefit from existing technologies presented in D2.10 “Selection Technologies and Development Process Guidelines Document” in the development design process. Moreover, considering interoperability and virtualisation as the pillars of our solution gives garanties for the success of our DCF proposal. This deliverable is organized according to the two following parts:
- Part I provides a synthesis of the deliverable and gives a quick overview of its content;
- Part II presents the deliverable contributions which cover:
- An analysis of data collection requirements coming from WP1 deliverables;
- A literature review on existing methodologies for virtualisation and interoperability;
- A Data Collection Framework solution;
- And finally a conclusion is provided with a summary and hints for future work directions.