Open Letter to the SciDB Community
September 24, 2010
The 1st SciDB community meeting will be held October 7th immediately after the XLDB4 conference at Stanford. At the community meeting we will be demonstrating SciDB functionality, using our new array query language AQL - an SQL-like query language extended for SciDB’s array data model. We will also review the roadmap and post it on the SciDB wiki.
In the current system (which we call SciDB 0.5) we can run on a collection of nodes, each running Linux, and connected by TCP/IP. In theory, the number of nodes is unlimited; however, we have only tested SciDB on a few nodes. We believe the system to be quite stable, but users should be cautioned that the system is still a bit immature. We have all basic array operations (including filter, join, project) running on this configuration. The intrepid and curious can download R0.5 source code and read the wiki documentation through the download area on this website.
We are calling this download R0.5 to indicate that it is still very much a ‘work-in-progress’. For example, we are changing the query language between 0.5 and the next release (0.75 expected at the end of the year). Also, uncertainty, provenance and failure recovery are not yet supported. We are also expanding our documentation so that we can engage more volunteers and early adopters.
For those who want to use SciDB on significant projects and need a more robust and stable version, we recommend waiting for R0.75 at year-end.
The rapid growth in SciDB news group subscribers underscores the urgent need for a scalable open-source data management system with integrated analytics to support scientific research on massive data sets. Our community of designers, developers and early adopters is very excited about the progress we have made and the future evolution of the system.