Roadmap
Improved exploration UI (2022)
LynxKite already has a solid UI for exploring graphs. But we have a lot of ideas for improvements. We would like to merge the visualization parameters and the visualization itself into a single rich UI. This would make visualizations in wizards more powerful.
Status: (2021-10-06) In planning stage.
Gremlin support (2022)
We plan to add import and export boxes for Gremlin-based graph databases, such as Cosmos DB and JanusGraph.
Status: (2021-10-06) Early prototyping.
Past items
Upgrade dependencies (2021 Q2)
In the past year our focus was open-sourcing, performance, features, and user experience. A number of our dependencies have made big improvements in the meantime and it’s time to upgrade. We want to upgrade Apache Spark, SBT, Scala, Play Framework, and a number of JavaScript libraries.
Status: (2021-10-06) Released in LynxKite 4.3.0.
More powerful custom visualizations (2021 Q2)
Vegas looked like a good choice for defining custom visualizations at the time. But it had no new releases since 2017, and in fact Scala is not a popular option for data science.
We plan to switch to defining visualizations in the very actively maintained Vega-Lite JSON syntax.
This will be a breaking change. There are almost no users of the Vegas syntax. Rewriting custom visualizations manually will be fairly easy.
Status: (2021-10-06) Released in LynxKite 4.3.0.
Graph database integrations (2020)
LynxKite is not a graph database. As a batch analytical tool it complements graph databases nicely. A graph database can answer interactive queries and handle updates. LynxKite can train models on the whole graph or compute metrics for every vertex.
While integration is already possible via standard data formats, we plan to make this even simpler by adding direct import/export functionality for popular graph databases. We are also considering deeper integrations and partnerships.
Status: (2020-10-08) LynxKite 4.1.0 works well with Neo4j. See our blog post. We have no immediate plans for other graph databases, but are open to suggestions.
NetworKit Integration (2020)
Thanks to Sphynx (see below), it is now possible to access functionality from third party libraries from LynxKite. NetworKit, a performant and rich graph library is a great candidate for integration. With this project we hope to further extend the breadth of graph algorithms available to LynxKite users.
Status: (2021-02-01) LynxKite 4.2.0 includes 42 algorithms from NetworKit. See our Twitter thread about it.
Open-sourcing (2020 Q2)
We are currently in the legal and technical process of open-sourcing LynxKite.
Status: (2020-06-24) LynxKite 4.0 is released open source!
Better progress indicator (2020)
LynxKite has two systems for indicating progress already. But neither gives you a clear indication of how much work has been completed and how much more you have to wait. We want to address this.
Status: (2020-04-27) Much improved in LynxKite 3.2.1. Depending on feedback we may either stop or continue.
Performance improvements (2020 Q1)
Apache Spark is great for distributed computations on large datasets. It is not a great fit for quick computations on small and medium-size datasets. We are addressing that with Sphynx, our next-generation backend. Sphynx is a high-performance graph analytics execution engine written in Go. Starting from version 3.0, LynxKite can choose to execute operations either on Apache Spark or on Sphynx.
The Sphynx implementations have a number of advantages:
- Avoids Spark and HDFS performance overheads.
- Non-distributed implementations of graph algorithms are simpler and faster.
- More efficient in-memory representation.
- We can integrate existing high-performance graph libraries like NetworKit.
Over the quarter we will add Sphynx implementations for more and more operations. We have seen 100× speedups in laboratory benchmarks. We cannot add Sphynx implementations for every operation. But our goal is to cover enough operations that the user experience is vastly improved.
Status: (2020-04-27) Released in LynxKite 3.2.0.
Wizards (2020 Q1)
New users cannot build complex workspaces in LynxKite. But that is where LynxKite’s power shines. We are building “wizards” to put the power of complex workspaces at the fingertips of new users.
Advanced users can turn their workspaces into easy-to-use wizards. LynxKite will also come bundled with a selection of useful wizards.
Status: (2020-04-27) Released in LynxKite 3.2.0.
Automatic attribute prediction from graphs (2020 Q1)
A common data science project is supervised learning: figuring out the attributes of people or other entities based on a smaller number of known examples. This gets more complicated on graphs, because we cannot consider the vertices of the graph individually. We have to work with the graph as a whole to make the most of the connections between vertices.
LynxKite has a number of tools already for solving such node attribute prediction problems. We want to make it easier to use these tools by offering a fully automated workflow. Just take a CSV, choose what you want to predict, and everything happens automatically.
Status: (2020-04-27) Released on try.lynxkite.com.
Steiner tree optimization (2020 Q1)
Graphs often feature in combinatorial optimization problems. The Prize-Collecting Steiner Tree Problem is of practical importance for finding minimum-cost fiber network layouts. We are adding it to LynxKite.
Status: (2020-04-27) Released in LynxKite 3.2.0.
Graph neural networks (2020 Q1)
We have invested in neural networks that work with graphs from the start. We believe they are key to finding patterns in densely connected datasets. Our immediate next step is to add two important GNN applications to LynxKite: node embedding and missing attribute prediction. Expect to see more graph AI capabilities coming down the road.
Status: (2020-04-27) Released in LynxKite 3.2.0.