Welcome to LumoSQL

LumoSQL is a project under active developement. Our goal is to build a reliable and secure database management system that is fully open-source and improves on the performance on classic SQLite.

• 100% downstream and upstream compatibility with SQLite, with same command line interface.

• Modular backends.

• Stability through corruption detection and rollback journaling.

• Reliably tested and benchmarked.

NEWS! - LumoSQL Phase II announcement

Phase II (ongoing)

• Role-based / attribute-based access control
• Implementation of hidden colums and tables
• Row level encryption
• Reseach and design of Lumions
  • Bibliography(download .bib)

Phase I (complete)

LumoSQL started as a combination of two embedded data storage C language libraries: SQLite and LMDB. LumoSQL builds on Howard Chu's 2013 proof of concept sqlightning combining the two codebases. Howard's LMDB library has become a ubiquitous replacement for bdb on the basis of performance, reliability, and license so the 2013 claims of it greatly increasing the performance of SQLite seemed credible. D Richard Hipp's SQLite is used in thousands of software projects, and since three of them are Google's Android, Mozilla's Firefox and Apple's iOS, an improved version of SQLite will benefit billions of people.

• Research

  • SQLite Development Landscape
  • What other software implements useful features?
  • What research has been done on SQLite topics?
  • What is the best way to maintain journals?
  • How database storage systems are scaled?
  • What are savepoints in SQLite?
  • Conclusions prior to development

• Design

  • Feaures that LumoSQL will implement
  • Identifying the API points of SQLite that LumoSQL will intercept
  • Changes to be made to SQLite virtual machine layer

• Implemented Features

  • Build

  LumoSQL build and testing system allows the user to choose any version of SQLite and any available backend version, as well as other options during build in order to build a database best suited for user's needs. The performance of LumoSQL database can be tested and benchmared using the same tool.

  • Not-Forking tool

  In order to make LumoSQL modular and compatible with a range of upstream versions, we have developed a tool that attempts to automate source code tracking. By tracking changes it avoids project level forking and therefore is called a not-forking tool.

  • LMDB and BDB backends

  LMDB provides a fast and reliable way to store key-value data and has been proven by Howard Chu to outperform the native SQLite b-tree in some situations.

  • Row level checksums

  Row level checksums lets us find out if the data has been corrupted and locate the precise row that has been affected, thus making it easier to fix corruption issues.

  • Benchmarking tool

  In order to test the performace of LumoSQL and prove or disprove its effectiveness we want to make sure that our benchmarking results are accurate and reproducible.

LumoSQL is currently under development. Contributions to code and documentation are welcome.

LumoSQL was started in December 2019 by Dan Shearer, who did the original source tree archaeology, patching and test builds. Keith Maxwell joined shortly after and contributed version management to the Makefile and the benchmarking tools.

LumoSQL is supported by the NLnet Foundation.

Published under MIT license.