Commodity components and Netezza software combine to extract the utmost throughput
from each MPP node. A dedicated high-speed interconnect from the storage array delivers
data to memory as quickly as each disk can stream. Compressed data is cached in memory
using a smart algorithm, which ensures that the most commonly accessed data is served
right out of memory instead of requiring a disk access. FAST Engines (shown in Figure 2)
running in parallel inside the FPGAs uncompress and filter out 95–98% of table data at
physics speed, keeping only data needed to answer the query. The remaining data in the
stream is processed concurrently by CPU cores, also running in parallel. The process is
repeated on more than a thousand of these parallel Snippet Processors running in the
Netezza appliance.
The FPGA is a critical enabler of the price-performance advantages of the Netezza platform.
Each FPGA contains embedded engines that perform filtering and transformation functions
on the data stream. These FAST engines (shown in Figure 3) are dynamically reconfigurable,
allowing them to be modified or extended through software. They are customized for every snippet through parameters provided during query execution and act on the data stream delivered by a Direct Memory Access (DMA) module at extremely high speed.
FAST engines include:
The Compress engine, a Netezza innovation boosting system performance by a factor of 4
to 8 times. The engine uncompresses data at wire speed, instantly transforming each
block on disk into 4 to 8 blocks in memory. The result is a significant speedup of the
slowest component in any data warehouse, the disk.
The Project and Restrict engines, which further increase performance by filtering out columns and rows respectively, based on the parameters in the SELECT and WHERE clauses in a SQL query.
The Visibility engine, which plays a critical role in maintaining Atomicity, Consistency,
Isolation, and Durability (ACID) compliance at streaming speeds in the Netezza platform.
It filters out rows that should not be seen by a query; for example, rows belonging to a
transaction that is not yet committed.
The Netezza FAST engines provide an extensible framework for innovative future functions to
be added through updates to the Netezza software. These new functions promise further
improvement in system performance, security, and reliability.
from each MPP node. A dedicated high-speed interconnect from the storage array delivers
data to memory as quickly as each disk can stream. Compressed data is cached in memory
using a smart algorithm, which ensures that the most commonly accessed data is served
right out of memory instead of requiring a disk access. FAST Engines (shown in Figure 2)
running in parallel inside the FPGAs uncompress and filter out 95–98% of table data at
physics speed, keeping only data needed to answer the query. The remaining data in the
stream is processed concurrently by CPU cores, also running in parallel. The process is
repeated on more than a thousand of these parallel Snippet Processors running in the
Netezza appliance.
The FPGA is a critical enabler of the price-performance advantages of the Netezza platform.
Each FPGA contains embedded engines that perform filtering and transformation functions
on the data stream. These FAST engines (shown in Figure 3) are dynamically reconfigurable,
allowing them to be modified or extended through software. They are customized for every snippet through parameters provided during query execution and act on the data stream delivered by a Direct Memory Access (DMA) module at extremely high speed.
FAST engines include:
The Compress engine, a Netezza innovation boosting system performance by a factor of 4
to 8 times. The engine uncompresses data at wire speed, instantly transforming each
block on disk into 4 to 8 blocks in memory. The result is a significant speedup of the
slowest component in any data warehouse, the disk.
The Project and Restrict engines, which further increase performance by filtering out columns and rows respectively, based on the parameters in the SELECT and WHERE clauses in a SQL query.
The Visibility engine, which plays a critical role in maintaining Atomicity, Consistency,
Isolation, and Durability (ACID) compliance at streaming speeds in the Netezza platform.
It filters out rows that should not be seen by a query; for example, rows belonging to a
transaction that is not yet committed.
The Netezza FAST engines provide an extensible framework for innovative future functions to
be added through updates to the Netezza software. These new functions promise further
improvement in system performance, security, and reliability.
No comments:
Post a Comment