Floe2 is a distributed, elastic and adaptive framework for large-scale dynamic streaming applications. Floe2 enables reliable processing of continuous high velocity data streams from varied data sources.
A distinguishing feature of Floe2 compared to other stream processing systems such as Storm and Infosphere is it's focus on run-time adaptability to support long running mission-critical applications. This allows Floe2 applications to go beyond traditional data mining and analysis and support complex real-world applications comprising data collection, analytics, control, and finally feedback and adaptations that allows the application to evolve over time.
The table below presents a feature comparison between Floe2 and Storm, a widely used stream processing system.
| Feature | Floe2 | Storm | Notes |
|---|---|---|---|
| distributed stream computations | Y | Y | Both Floe2 and Storm support similar stream processing application topology composed of Processing Elements (Spout/Bolt eqiv. in storm) connected using data flow edges to direct the flow of data streams among them. Both support a variety of Data Stream sources such as RabbitMQ/AMQP, Kafka, JMS, Kestrel and can easily integrate with any existing data base systems. Further, applications built using Storm are easily portable to Floe2. |
| auto-scaling/elastic deployment in response to changing stream data rates | Y | N | Floe2 supports and a *period scheduler* that can provision additional resources to support variable data rates WITHOUT the need to pause the application. |
| Stateful Operators/Bolts | Y | N | NOTE |
| Fault Tolerance - Tuple Processing (atleast once gurantee) | Y | Y | Peer backup, No Replay Required, Peer node takes over. Upstream backup, requires message replay |
| Fault Tolerance - Operator State | Y | N/A | Note |
| Elastic "MapReduce" | Y | N | (Number of mapper/reducer tasks can be increased/decreased at run-time based on the observed data rates without state or message loss). (Number of reduce tasks cannot be changed, "rebalance" command may be used to leverage additional resources by moving tasks to newly available machines) |