NOTE: Brewery was replaced by Bubbles

Flow based data processing with support for variety of data sources, data targets.

Streams

Data are treated as file-like objects where structured data is being passed instead of bytes.

  • Comma separated values (CSV) file/URI resource
  • MS Excel spreadsheet
  • Google Spreadsheet
  • Relational database table
  • MongoDB database collection
  • Directory containing yaml files - one file per record

Read more about stores.

Streams and Higher Order Messaging

Processing network is described as Stream composed of connected processing nodes. Besides traditional network construction there is an option to use higher order messaging to construct a stream:

audit = Stream().fork()

audit.sample(1000)
audit.audit()
audit.formatted_printer()

audit.run()

Metadata

Brewery uses metadata extensively – they describe the structure of processed data flow.

Read more

brewery tool

brewery is a command line tool for running data processing streams, for example:

cat data.csv | brewery pipe audit pretty_printer

Read more.