Skip to content

zhangxin23/spring-cloud-dataflow

 
 

Repository files navigation

Spring Cloud Data Flow Build Status Stories Ready Stores In Progress

The Spring Cloud Data Flow project provides orchestration for data microservices, including long lived stream applications and short lived task applications.

Components

The Core domain module includes the concept of a stream that is a composition of spring-cloud-stream modules in a linear pipeline from a source to a sink, optionally including processor module(s) in between. The domain also includes the concept of a task, which may be any process that does not run indefinitely, including Spring Batch jobs.

The Module Registry maintains the set of available modules, and their mappings to Maven coordinates.

The Module Deployer SPI provides the abstraction layer for deploying the modules of a given stream across a variety of runtime environments. Current SPI implementations support Local deployment, YARN, Cloud Foundry, Mesos and Kubernetes.

The Admin Server is a Spring Boot application that provides a common REST API, UI, and an implementation of the module deployer SPI interface for the runtime environment. For each runtime environment there is a different version of the admin server. The github locations for these admin servers are:

The Shell connects to the Admin’s REST API and supports a DSL that simplifies the process of defining a stream and managing its lifecycle.

Instructions to run the Admin Server for each runtime environments can be found in their respective github repositories.

Contributing

We love contributions. Follow this link for more information on how to contribute.

Building

Clone the repo and type

$ ./mvnw clean install

For more information on building, see this link.

About

Spring Cloud Data Flow provides orchestration for data microservices, including both stream and task processing

Resources

License

Stars

Watchers

Forks

Packages

No packages published

Languages

  • Java 92.0%
  • XSLT 5.0%
  • Shell 2.1%
  • Other 0.9%