Asakusa is a full stack framework for distributed/parallel computing, which provides with a development platform and runtime libraries supporting various distributed/parallel computing environments such as Hadoop, Spark, M³ for Batch Processing, and so on. Users can enjoy the best performance on distributed/parallel computing transparently changing execution engines among MapReduce, SparkRDD, and C++ native based on their data size.

Other than query-based languages, Asakusa helps to develop more complicated data flow programs more easily, efficiently, and comprehensively due to following components.

• Data-flow oriented DSL

  Data-flow based approach is suitable for DAG constructions which is appropriate for distributed/parallel computing. Asakusa offers Domain Specific Language based on Java with data-flow design, which is integrated with compilers.

• Compilers

  A multi-tier compiler is supported. Java based source code is once compiled to inter-mediated representation and then optimized for each execution environments such that Hadoop(MapReduce), Spark(RDD), M³ for Batch Processing(C++ Native), respectively.

• Data-Modeling language

  Data-Model language is supported, which is comprehensive for mapping with relational models, CSVs, or other data formats.

• Test Environment

  JUnit based unit testing and end-to-end testing are supported, which are portable among each execution environments. Source code, test code, and test data are fully compatible across Hadoop, Spark, M³ for Batch Processing and others.

• Runtime execution driver

  A transparent job execution driver is supported.

All these features have been well designed and developed with the expertise from experiences on enterprise-scale system developments over decades and promised to contribute to large scale systems on distributed/parallel environments to be more robust and stable.

./mvnw install -DskipTests

• Install Hadoop with local-mode settings
• Set hadoop command into your PATH variable, or set it to $HADOOP_CMD
• And then run ./mvnw test

• Run ./mvnw install eclipse:eclipse -DskipTests
• And then import projects from Eclipse

If you run tests in Eclipse, please activate Preferences > Java > Debug > 'Only include exported classpath entries when launching'.

• Asakusa Framework SDK
• Asakusa Framework Language Toolset
• Asakusa on Spark
• Asakusa on M³BP
• Asakusa Framework Documentation

• Asakusa Framework Examples
• Asakusa Framework Legacy Modules
• Jinrikisha
• Shafu

• Asakusa Framework Documentation (ja)
• Asakusa Framework Community Site (ja)

• Please report any issues to repository for issue tracking
• Please contribute with patches according to our contribution guide (Japanese only, English version to be added)

• Apache License, Version 2.0