Big Data Analytics
Build Knowledge at the Speed and Scale of the Internet of Things
The number of smart things is growing exponentially. By 2020, tens of billions of things will be deployed worldwide, collecting a wealth of diverse data. Traditional computing models collect in-field data and then transmit it to a central data center where analytics are applied to it—but this is no longer a sustainable model. A new approach is needed to transform enormous amounts of collected data into meaningful information. The purpose of analytics is to move from the raw data at the bottom of the pyramid — through structured information, contextual knowledge and integrated wisdom — to well-informed decisions at the top.
In settings such as live events, sensors are generating millions or even billions of data points over the period of a few hours. Decision makers will not have the insights they need to make timely, informed decisions without a massively scalable data collection, integration, and analytics solution.
In response, AGT has developed a big data approach which distributes the analytics and extracts structure from the raw data near the network edge and throughout the entire system – enriching the contextual information in a hierarchical way as shown in the DIKW (Data, Information, Knowledge, Wisdom + Decisions) pyramid. Our approach uses the processing power of smart things to create a distributed intelligence net, and with that, minimize the elapsed time between detection and reaction. It produces smaller, more manageable data sets, creates a more affordable business model using shared services, and ensures the end-to-end security and privacy of the system.
Distributed Analytics Filter Raw Data Close to the Network Edge
The AGT approach reduces the amount of raw data early on for more efficient data collection, transport, storage and analysis. Instead of centralizing all data before beginning analysis, we filter data close to the network edge. Big data analytics can be deployed in cameras, gateways and other edge devices.
Since that filtering must take place without the entire picture, our analytics must incorporate human expertise to guide the decisions about which data is collected and how it is processed. For example, we may multiple readings from sensors of the same or different types to improve the results or extract metadata from raw video.
While sensors and most low-level analytics will be widely distributed, there will still be one or a few central hubs generating a unified view for decision makers. Since these hubs will require more sophisticated tools and complex analytics, they require additional computational and storage resources. Cloud computing enables the sharing of these expensive resources by offering them as services, thus reducing the need for heavy hardware infrastructure in central decision hubs (operations centers). Centralized Hubs Deliver a Unified View