This is the implementation of the GLL-based context-free path querying (CFPQ) algorithm. It is based on a high-performance GLL parsing algorithm implemented in Iguana project. Then, it was modificated to support graph-structured input data. Proposed solution solves both reachability and all paths problems for all pairs and multiple sources cases.

The proposed solution has been evaluated on several real-world graphs for both the all pairs and the multiple sources scenarios. The evaluation shows that the proposed solution is more than 45 times faster than the previous solution for Neo4j and is comparable, in some scenarios and cases, with the linear algebra based solution.

Machine configuration: PC with Ubuntu 20.04, Intel Core i7-6700 3.40GHz CPU, DDR4 64Gb RAM.

Enviroment configuration:

• Java HotSpot(TM) 64-Bit server virtual machine (build 15.0.2+7-27, mixed mode, sharing).
• JVM heap configuration: 55Gb both xms and xmx.
• Neo4j 4.0.3 is used. Almost all configurations are default except two:
  • total off-heap transaction memory (tx_state_max_off_heap_memory parameter) is set to 24Gb
  • pagecache_warmup_enabled is set to true.

The graph data is selected from CFPQ_Data dataset. There are two types of graph:

• Graphs related to RDF analysis problems
• Graphs related to static code analysis problems

A detailed description of the graphs is listed below.

RDF analysis

Static code analysis

Field-sensitive points-to analysis

All queries used in evaluation are variants of same-generation query. The inverse of an x relation and the respective edge is denoted as x_r.

Grammars used for RDF graphs:

G₁

S -> subClassOf_r S subClassOf | subClassOf_r subClassOf 
     | type_r S type | type_r type

G₂

S -> subClassOf_r S subClassOf | subClassOf

Geo

S -> broaderTransitive S broaderTransitive_r
     | broaderTransitive broaderTransitive_r 

Grammar used for static code analysis graphs:

PointsTo

M -> d_r V d
V -> (M? a_r)* M? (a M?)* 

Grammar template used for field-sensitive points-to analysis:

PointsTo -> (assign | load_f Alias store_f )* alloc
Alias -> PointsTo FlowsTo
FlowsTo -> alloc_r (assign_r | store_f_r Alias load_f_r)*

TermiFor all fields f the terminals load_f, store_f are generated

The results of the all pairs reachability queries evaluation on graphs related to RDF analysis are listed below.

The sign ’–’ in cells means that the respective query is not applicable to the graph, so time is not measured.

The evaluation results for single source CFPQ for graphs related to RDF analysis and G₁, G₂, Geo grammars respectively in reachability and all paths scenarious:

The results for graphs related to static code analysis are compared to results of Azimov’s CFPQ algorithm based on matrix operations. The implementation from CFPQ_PyAlgo was taken as the implementation of the matrix CFPQ algorithm. This library contains the implementation for both scenarios, all pairs reachability and single source reachability. To perform matrix operations pygraphblas is used. Pygraphblas is a python wrapper over the SuiteSparse library, which based on the GraphBLAS framework.

The results of the all pairs reachability queries evaluation on graphs related to static code analysis are listed below.

The sign ’–’ in cells means that the respective query and graph require a considerable amount of memory during algorithm execution that leads to unpredictable time to get the result.

The evaluation results for single source CFPQ for graphs related to static code analysis and pointsTo grammar in reachability and all paths scenarious:

The results of the all pairs reachability queries evaluation on graphs related to field-sensitive points-to analysis. The results are compared to results of .NET GLL implementation.

The project is build with Maven.

git clone https://github.com/JetBrains-Research/GLL4Graph
cd GLL4Graph
mvn compile

To replicate experiments:

• make the directory to save results

mkdir results

• run the following command with arguments

mvn exec:java -Dexec.mainClass="benchmark.GraphBenchmark" -Dexec.args="aguments"

• print help

mvn exec:java -Dexec.mainClass="benchmark.GraphBenchmark" -Dexec.args="-h"

usage: GraphBenchmark [-h] -d <dataset name> -gm <path> -gp <path> -gs <storage type> -m <number> -p <problem type> -S <s=value1 a=value2> -w <number>
 -d,--dataset <dataset name>            The name of the dataset, an important component of the file name with the results
 -gm,--grammar <path>                   Path to JSON file contains context-free grammar
 -gp,--graph <path>                     Path to directory contains files nodes.csv and edges.csv
 -gs,--graph_storage <storage type>     Graph storage type, allowed values: NEO4J, IN_MEMORY
 -h,--help                              Print help message
 -m,--measurement_iterations <number>   Number of measurement iterations
 -p,--problem <problem type>            Benchmarking algorithm, allowed values: REACHABILITY, ALL_PATHS
 -S,--scenario <s=value1 a=value2>      Benchmarking scenario and its argument, 's' property allowed values: ALL_PAIRS, MULTIPLE_SOURCES, 'a' property
                                        contains number of nodes if 's' equals ALL_PAIRS or path to file with vertices chunks if 's' equals
                                        MULTIPLE_SOURCES
 -w,--warmup_iterations <number>        Number of warm-up iterations

Here is an example which can be run right after project is downloaded and results directory is created without any additional preparation.

To run all pairs reachability algorithm on Core graph on G₁ grammar use the following command:

mvn exec:java -Dexec.mainClass="benchmark.GraphBenchmark" -Dexec.args="-d Core -gm test/resources/grammars/graph/g1/grammar.json -gp data/core -gs NEO4J -p REACHABILITY -S -s ALL_PAIRS -a $(( $(cat "data/core/nodes.csv" | wc -l)-1 )) -w 1 -m 10"

To get more graph data examples use Python script:

graph_loader.py --graph {graph_name} --relationships {comma_separated_relationships_list}

For example:

graph_loader.py --graph core --relationships subClassOf,type

This project is licensed under OpenBSD License. License text can be found in the license file.