Database Version Management

[Eli Golovinsky]

Hi everyone,

A few months ago I've posted a note here and on other related forums
that I'm thinking about building an open source tool for database
version management, a sort of SCM for RDBMS. I've worked quite a lot on
it since then, and I think it's more or less ready to be tested by
someone other than me.

The concept is simple. Our source control tools all support managing
versions of text files. We want to manage versions of a database. So we
need to represent the database as a text file and store it in the
source control.
We'll also need the ability to generate an SQL script for a given
structure and to compare two structures to generate a script that
changes a database.

That is what Daversy does, or intends to do. There's a command line
client, a provider for both Oracle and SQLite and a both a tutorial and
a wiki to get you started.

Daversy is not production quality yet, but I've tried to test it as
well as I could on my own and it seems to be working.

I'd really love to hear your comments.

www.daversy.org

Eli.

[Marc Girod]

>>>>> "Eli" == egooli@gmail.com

Eli> The concept is simple. Our source control tools all support
Eli> managing versions of text files. We want to manage versions of a
Eli> database. So we need to represent the database as a text file and
Eli> store it in the source control.

Er... no.

The text file is just one imperfect attempt at a generic artefact to
be mapped to the abstract concept of "configuration item".

It is btw pretty much obsolete as the generic interface for SCM
systems. It was used by the very first generation of version control
system, more or less based on diff/patch, but soon, one noticed the
need to support binaries and directories (manage the identity across
name changes and moves).

Even the "universal source file" (file or directory object) is
actually pretty much obsolete now -- in an arbitrary software
configuration, it is pretty much accidental whether an artefact is
'source' or 'derived object' (built, extracted, patched, etc.)

Anyway, one essential criterion for the generic support for the
mapping, is that it should be at a fine enough granularity to be of
some interest.
This is clearly not the case of database files...

So, an SCM system offering support for databases should use some kind
of hook to manage database records, and probably offer this mapping in
*addition* to universal files, used to manage anything outside the
database.

--
Marc Girod P.O. Box 370 Voice: +358-71 80 25581
Nokia BI 00045 NOKIA Group Mobile: +358-50 38 78415
Kara RD B 480.2 Finland Fax: +358-71 80 64474

[Eli Golovinsky]

Have you actually read what I have written on the site? What I wrote
here is a snippet, and not the whole deal. Perhaps it's the wrong
snippet..

I do think that representing a database structure as an XML file (which
is a text file easily handled by all the SCM tools) can provide a
solution to database version management.

When you can generate such an XML from an existing database, generate
creation scripts from such an XML, and compare two of them to generate
'patch' SQL scripts that change the database structure, you can manage
the database's version by storing versions of those XML files in a
source control tool.

All of the above Daversy can already do.

[Jorgen Grahn]

On 25 Oct 2005 11:40:23 -0700, egooli@gmail.com <egooli@gmail.com> wrote: Have you actually read what I have written on the site? What I wrote here is a snippet, and not the whole deal. Perhaps it's the wrong snippet..

Most readers here don't have the patience to follow links, I'm afraid.
(Don't know whether Marc did or not, of course.)
I do think that representing a database structure as an XML file (which is a text file easily handled by all the SCM tools) can provide a solution to database version management.

Text files (XML or not) /may/ be easily handled, or they may not.
It depends on (off the top of my head)
- what they look like
- what the likely "change patterns" are
- how tolerant they are to merge errors; if there is a compiler
or something that catches syntax errors

If a file is columns and lines of numbers, and changes tend to mean adding
"1" to the numbers in colum five, it's useless with all the tools I've
encountered.

I don't know much about databases though, so I cannot say what happens when
one gets "flattened" into a text file.

/Jorgen

--
// Jorgen Grahn <jgrahn@ Ph'nglui mglw'nafh Cthulhu
\X/ algonet.se> R'lyeh wgah'nagl fhtagn!

[Verner Jensen, Værløse]

Hephep - Seems like, based on your description, the tool we've been looking
for. I'll dive into it rigth away ;-)

/Vern.

<egooli@gmail.com> wrote in message
news:1130242501.981430.81490@g49g2000cwa.googlegroups.com... Hi everyone, A few months ago I've posted a note here and on other related forums that I'm thinking about building an open source tool for database version management, a sort of SCM for RDBMS. I've worked quite a lot on it since then, and I think it's more or less ready to be tested by someone other than me. The concept is simple. Our source control tools all support managing versions of text files. We want to manage versions of a database. So we need to represent the database as a text file and store it in the source control. We'll also need the ability to generate an SQL script for a given structure and to compare two structures to generate a script that changes a database. That is what Daversy does, or intends to do. There's a command line client, a provider for both Oracle and SQLite and a both a tutorial and a wiki to get you started. Daversy is not production quality yet, but I've tried to test it as well as I could on my own and it seems to be working. I'd really love to hear your comments. www.daversy.org Eli.