Module migrate.changeset – Schema migration API

This module extends SQLAlchemy and provides additional DDL [1] support.

[1]	SQL Data Definition Language

Module ansisql – Standard SQL implementation

Extensions to SQLAlchemy for altering existing tables.

At the moment, this isn’t so much based off of ANSI as much as things that just happen to work with multiple databases.

class migrate.changeset.ansisql.ANSIColumnDropper(dialect, connection, **kw)

Extends ANSI SQL dropper for column dropping (ALTER TABLE DROP COLUMN).

visit_column(column)

Drop a column from its table.

Parameter:column (sqlalchemy.Column) – the column object

class migrate.changeset.ansisql.ANSIColumnGenerator(dialect, connection, **kw)

Extends ansisql generator for column creation (alter table add col)

visit_column(column)

Create a column (table already exists).

Parameter:column (sqlalchemy.Column instance) – column object

class migrate.changeset.ansisql.ANSIConstraintCommon(dialect, connection, **kw)

Migrate’s constraints require a separate creation function from SA’s: Migrate’s constraints are created independently of a table; SA’s are created at the same time as the table.

get_constraint_name(cons)

Gets a name for the given constraint.

If the name is already set it will be used otherwise the constraint’s autoname method is used.

Parameter:cons – constraint object

class migrate.changeset.ansisql.ANSISchemaChanger(dialect, connection, **kw)

Manages changes to existing schema elements.

Note that columns are schema elements; ALTER TABLE ADD COLUMN is in SchemaGenerator.

All items may be renamed. Columns can also have many of their properties - type, for example - changed.

Each function is passed a tuple, containing (object, name); where object is a type of object you’d expect for that function (ie. table for visit_table) and name is the object’s new name. NONE means the name is unchanged.

start_alter_column(table, col_name)

Starts ALTER COLUMN

visit_column(delta)

Rename/change a column.

visit_index(index)

Rename an index

visit_table(table)

Rename a table. Other ops aren’t supported.

class migrate.changeset.ansisql.AlterTableVisitor(dialect, connection, **kw)

Common operations for ALTER TABLE statements.

append(s)

Append content to the SchemaIterator’s query buffer.

execute()

Execute the contents of the SchemaIterator’s buffer.

start_alter_table(param)

Returns the start of an ALTER TABLE SQL-Statement.

Use the param object to determine the table name and use it for building the SQL statement.

Parameter:param (sqlalchemy.Column, sqlalchemy.Index, sqlalchemy.schema.Constraint, sqlalchemy.Table, or string (table name)) – object to determine the table from

Module constraint – Constraint schema migration API

This module defines standalone schema constraint classes.

class migrate.changeset.constraint.CheckConstraint(sqltext, *args, **kwargs)

Bases: migrate.changeset.constraint.ConstraintChangeset, sqlalchemy.schema.CheckConstraint

Construct CheckConstraint

Migrate’s additional parameters:

Parameters:

• sqltext (string) – Plain SQL text to check condition
• columns (list of Columns instances) – If not name is applied, you must supply this kw to autoname constraint
• table (Table instance) – If columns are passed as strings, this kw is required

create(*a, **kw)

Create the constraint in the database.

Parameters:

• engine (sqlalchemy.engine.base.Engine) – the database engine to use. If this is None the instance’s engine will be used
• connection (sqlalchemy.engine.base.Connection instance) – reuse connection istead of creating new one.

drop(*a, **kw)

Drop the constraint from the database.

Parameters:

• engine (sqlalchemy.engine.base.Engine) – the database engine to use. If this is None the instance’s engine will be used
• cascade (bool) – Issue CASCADE drop if database supports it
• connection (sqlalchemy.engine.base.Connection instance) – reuse connection istead of creating new one.

Returns:

Instance with cleared columns

get_children(**kwargs)

used to allow SchemaVisitor access

class migrate.changeset.constraint.ConstraintChangeset

Bases: object

Base class for Constraint classes.

create(*a, **kw)

Create the constraint in the database.

Parameters:

• engine (sqlalchemy.engine.base.Engine) – the database engine to use. If this is None the instance’s engine will be used
• connection (sqlalchemy.engine.base.Connection instance) – reuse connection istead of creating new one.

drop(*a, **kw)

Drop the constraint from the database.

Parameters:

• engine (sqlalchemy.engine.base.Engine) – the database engine to use. If this is None the instance’s engine will be used
• cascade (bool) – Issue CASCADE drop if database supports it
• connection (sqlalchemy.engine.base.Connection instance) – reuse connection istead of creating new one.

Returns:

Instance with cleared columns

class migrate.changeset.constraint.ForeignKeyConstraint(columns, refcolumns, *args, **kwargs)

Bases: migrate.changeset.constraint.ConstraintChangeset, sqlalchemy.schema.ForeignKeyConstraint

Construct ForeignKeyConstraint

Migrate’s additional parameters:

Parameters:

• columns (list of strings or Column instances) – Columns in constraint
• refcolumns (list of strings or Column instances) – Columns that this FK reffers to in another table.
• table (Table instance) – If columns are passed as strings, this kw is required

autoname()

Mimic the database’s automatic constraint names

create(*a, **kw)

Create the constraint in the database.

Parameters:

• engine (sqlalchemy.engine.base.Engine) – the database engine to use. If this is None the instance’s engine will be used
• connection (sqlalchemy.engine.base.Connection instance) – reuse connection istead of creating new one.

drop(*a, **kw)

Drop the constraint from the database.

Parameters:

• engine (sqlalchemy.engine.base.Engine) – the database engine to use. If this is None the instance’s engine will be used
• cascade (bool) – Issue CASCADE drop if database supports it
• connection (sqlalchemy.engine.base.Connection instance) – reuse connection istead of creating new one.

Returns:

Instance with cleared columns

get_children(**kwargs)

used to allow SchemaVisitor access

class migrate.changeset.constraint.PrimaryKeyConstraint(*cols, **kwargs)

Bases: migrate.changeset.constraint.ConstraintChangeset, sqlalchemy.schema.PrimaryKeyConstraint

Construct PrimaryKeyConstraint

Migrate’s additional parameters:

Parameters:

• cols (strings or Column instances) – Columns in constraint.
• table (Table instance) – If columns are passed as strings, this kw is required

autoname()

Mimic the database’s automatic constraint names

create(*a, **kw)

Create the constraint in the database.

Parameters:

• engine (sqlalchemy.engine.base.Engine) – the database engine to use. If this is None the instance’s engine will be used
• connection (sqlalchemy.engine.base.Connection instance) – reuse connection istead of creating new one.

drop(*a, **kw)

Drop the constraint from the database.

Parameters:

• engine (sqlalchemy.engine.base.Engine) – the database engine to use. If this is None the instance’s engine will be used
• cascade (bool) – Issue CASCADE drop if database supports it
• connection (sqlalchemy.engine.base.Connection instance) – reuse connection istead of creating new one.

Returns:

Instance with cleared columns

get_children(**kwargs)

used to allow SchemaVisitor access

class migrate.changeset.constraint.UniqueConstraint(*cols, **kwargs)

Bases: migrate.changeset.constraint.ConstraintChangeset, sqlalchemy.schema.UniqueConstraint

Construct UniqueConstraint

Migrate’s additional parameters:

Parameters:

• cols (strings or Column instances) – Columns in constraint.
• table (Table instance) – If columns are passed as strings, this kw is required

New in version 0.6.0.

autoname()

Mimic the database’s automatic constraint names

create(*a, **kw)

Create the constraint in the database.

Parameters:

• engine (sqlalchemy.engine.base.Engine) – the database engine to use. If this is None the instance’s engine will be used
• connection (sqlalchemy.engine.base.Connection instance) – reuse connection istead of creating new one.

drop(*a, **kw)

Drop the constraint from the database.

Parameters:

• engine (sqlalchemy.engine.base.Engine) – the database engine to use. If this is None the instance’s engine will be used
• cascade (bool) – Issue CASCADE drop if database supports it
• connection (sqlalchemy.engine.base.Connection instance) – reuse connection istead of creating new one.

Returns:

Instance with cleared columns

get_children(**kwargs)

used to allow SchemaVisitor access

Module databases – Database specific schema migration

This module contains database dialect specific changeset implementations.

Module mysql

MySQL database specific implementations of changeset classes.

Module firebird

Firebird database specific implementations of changeset classes.

class migrate.changeset.databases.firebird.FBColumnDropper(dialect, connection, **kw)

Firebird column dropper implementation.

visit_column(column)

Firebird supports ‘DROP col’ instead of ‘DROP COLUMN col’ syntax

Drop primary key and unique constraints if dropped column is referencing it.

class migrate.changeset.databases.firebird.FBColumnGenerator(dialect, connection, **kw)

Firebird column generator implementation.

class migrate.changeset.databases.firebird.FBConstraintDropper(dialect, connection, **kw)

Firebird constaint dropper implementation.

cascade_constraint(constraint)

Cascading constraints is not supported

class migrate.changeset.databases.firebird.FBConstraintGenerator(dialect, connection, **kw)

Firebird constraint generator implementation.

class migrate.changeset.databases.firebird.FBSchemaChanger(dialect, connection, **kw)

Firebird schema changer implementation.

visit_table(table)

Rename table not supported

Module oracle

Oracle database specific implementations of changeset classes.

Module postgres

PostgreSQL database specific implementations of changeset classes.

class migrate.changeset.databases.postgres.PGColumnDropper(dialect, connection, **kw)

PostgreSQL column dropper implementation.

class migrate.changeset.databases.postgres.PGColumnGenerator(dialect, connection, **kw)

PostgreSQL column generator implementation.

class migrate.changeset.databases.postgres.PGConstraintDropper(dialect, connection, **kw)

PostgreSQL constaint dropper implementation.

class migrate.changeset.databases.postgres.PGConstraintGenerator(dialect, connection, **kw)

PostgreSQL constraint generator implementation.

class migrate.changeset.databases.postgres.PGSchemaChanger(dialect, connection, **kw)

PostgreSQL schema changer implementation.

Module sqlite

SQLite database specific implementations of changeset classes.

class migrate.changeset.databases.sqlite.SQLiteColumnDropper(dialect, connection, **kw)

SQLite ColumnDropper

class migrate.changeset.databases.sqlite.SQLiteColumnGenerator(dialect, connection, **kw)

SQLite ColumnGenerator

add_foreignkey(constraint)

Does not support ALTER TABLE ADD FOREIGN KEY

class migrate.changeset.databases.sqlite.SQLiteSchemaChanger(dialect, connection, **kw)

SQLite SchemaChanger

visit_index(index)

Does not support ALTER INDEX

Module visitor

Module for visitor class mapping.

migrate.changeset.databases.visitor.get_dialect_visitor(sa_dialect, name)

Get the visitor implementation for the given dialect.

Finds the visitor implementation based on the dialect class and returns and instance initialized with the given name.

Binds dialect specific preparer to visitor.

migrate.changeset.databases.visitor.get_engine_visitor(engine, name)

Get the visitor implementation for the given database engine.

Parameters:

• engine (Engine) – SQLAlchemy Engine
• name (string) – Name of the visitor

Returns:

visitor

migrate.changeset.databases.visitor.run_single_visitor(engine, visitorcallable, element, connection=None, **kwargs)

Taken from sqlalchemy.engine.base.Engine._run_single_visitor() with support for migrate visitors.

Module exceptions – Exception definitions

This module provides exception classes.

exception migrate.changeset.exceptions.Error

Changeset error.

exception migrate.changeset.exceptions.InvalidConstraintError

Invalid constraint error.

exception migrate.changeset.exceptions.MigrateDeprecationWarning

Warning for deprecated features in Migrate

exception migrate.changeset.exceptions.NotSupportedError

Not supported error.

Module schema – Additional API to SQLAlchemy for migrations

Schema module providing common schema operations.

migrate.changeset.schema.create_column(column, table=None, *p, **kw)

Create a column, given the table.

API to ChangesetColumn.create().

migrate.changeset.schema.drop_column(column, table=None, *p, **kw)

Drop a column, given the table.

API to ChangesetColumn.drop().

migrate.changeset.schema.alter_column(*p, **k)

Alter a column.

Direct API to ColumnDelta.

Parameters:

• table – Table or table name (will issue reflection).
• engine – Will be used for reflection.
• alter_metadata – Defaults to True. It will alter changes also to objects.

Returns:

Columndelta instance

migrate.changeset.schema.rename_table(table, name, engine=None, **kw)

Rename a table.

If Table instance is given, engine is not used.

API to ChangesetTable.rename().

Parameters:

• table (string or Table instance) – Table to be renamed.
• name (string) – New name for Table.
• engine (obj) – Engine instance.

migrate.changeset.schema.rename_index(index, name, table=None, engine=None, **kw)

Rename an index.

If Index instance is given, table and engine are not used.

API to ChangesetIndex.rename().

Parameters:

• index (string or Index instance) – Index to be renamed.
• name (string) – New name for index.
• table (string or Table instance) – Table to which Index is reffered.
• engine (obj) – Engine instance.

class migrate.changeset.schema.ChangesetTable

Changeset extensions to SQLAlchemy tables.

create_column(column, *p, **kw)

Creates a column.

The column parameter may be a column definition or the name of a column in this table.

API to ChangesetColumn.create()

Parameter:column (Column instance or string) – Column to be created

deregister()

Remove this table from its metadata

drop_column(column, *p, **kw)

Drop a column, given its name or definition.

API to ChangesetColumn.drop()

Parameter:column (Column instance or string) – Column to be droped

rename(name, connection=None, **kwargs)

Rename this table.

Parameters:

• name (string) – New name of the table.
• alter_metadata (bool) – If True, table will be removed from metadata
• connection (sqlalchemy.engine.base.Connection instance) – reuse connection istead of creating new one.

class migrate.changeset.schema.ChangesetColumn

Changeset extensions to SQLAlchemy columns.

alter(*p, **k)

Alter a column’s definition: ALTER TABLE ALTER COLUMN.

Column name, type, server_default, and nullable may be changed here.

Direct API to alter_column()

Example:

col.alter(name='foobar', type=Integer(), server_default=text("a"))

Supported parameters: name, type, primary_key, nullable, server_onupdate, server_default, autoincrement

copy_fixed(**kw)

Create a copy of this Column, with all attributes.

create(table=None, index_name=None, unique_name=None, primary_key_name=None, populate_default=True, connection=None, **kwargs)

Create this column in the database.

Assumes the given table exists. ALTER TABLE ADD COLUMN, for most databases.

Parameters:

• table (Table instance) – Table instance to create on.
• index_name (string) – Creates ChangesetIndex on this column.
• unique_name (string) – Creates UniqueConstraint on this column.
• primary_key_name (string) – Creates PrimaryKeyConstraint on this column.
• alter_metadata (bool) – If True, column will be added to table object.
• populate_default (bool) – If True, created column will be populated with defaults
• connection (sqlalchemy.engine.base.Connection instance) – reuse connection istead of creating new one.

Returns:

self

drop(table=None, connection=None, **kwargs)

Drop this column from the database, leaving its table intact.

ALTER TABLE DROP COLUMN, for most databases.

Parameters:

• alter_metadata (bool) – If True, column will be removed from table object.
• connection (sqlalchemy.engine.base.Connection instance) – reuse connection istead of creating new one.

class migrate.changeset.schema.ChangesetIndex

Changeset extensions to SQLAlchemy Indexes.

rename(name, connection=None, **kwargs)

Change the name of an index.

Parameters:

• name (string) – New name of the Index.
• alter_metadata (bool) – If True, Index object will be altered.
• connection (sqlalchemy.engine.base.Connection instance) – reuse connection istead of creating new one.

class migrate.changeset.schema.ChangesetDefaultClause

Implements comparison between DefaultClause instances

class migrate.changeset.schema.ColumnDelta(*p, **kw)

Extracts the differences between two columns/column-parameters

May receive parameters arranged in several different ways:

• current_column, new_column, *p, **kw

  Additional parameters can be specified to override column differences.

• current_column, *p, **kw

  Additional parameters alter current_column. Table name is extracted from current_column object. Name is changed to current_column.name from current_name, if current_name is specified.

• current_col_name, *p, **kw

  Table kw must specified.

Parameters:

• table (string or Table instance) – Table at which current Column should be bound to. If table name is given, reflection will be used.
• alter_metadata (bool) – If True, it will apply changes to metadata.
• metadata (MetaData instance) – If alter_metadata is true, metadata is used to reflect table names into
• engine (Engine instance) – When reflecting tables, either engine or metadata must be specified to acquire engine object.

Returns:

ColumnDelta instance provides interface for altered attributes to result_column through dict() alike object.

• ColumnDelta.result_column is altered column with new attributes
• ColumnDelta.current_name is current name of column in db

apply_diffs(diffs)

Populate dict and column object with new values

are_column_types_eq(old_type, new_type)

Compares two types to be equal

compare_1_column(col, *p, **k)

Compares one Column object

compare_2_columns(old_col, new_col, *p, **k)

Compares two Column objects

compare_parameters(current_name, *p, **k)

Compares Column objects with reflection

process_column(column)

Processes default values for column

Module migrate.versioning – Database versioning and repository management

This package provides functionality to create and manage repositories of database schema changesets and to apply these changesets to databases.

Module api – Python API commands

Module exceptions – Exception definitions

Provide exception classes for migrate.versioning

exception migrate.versioning.exceptions.ApiError

Base class for API errors.

exception migrate.versioning.exceptions.ControlledSchemaError

Base class for controlled schema errors.

exception migrate.versioning.exceptions.DatabaseAlreadyControlledError

Database shouldn’t be under version control, but it is

exception migrate.versioning.exceptions.DatabaseNotControlledError

Database should be under version control, but it’s not.

exception migrate.versioning.exceptions.Error

Error base class.

exception migrate.versioning.exceptions.InvalidRepositoryError

Invalid repository error.

exception migrate.versioning.exceptions.InvalidScriptError

Invalid script error.

exception migrate.versioning.exceptions.InvalidVersionError

Invalid version error.

exception migrate.versioning.exceptions.KnownError

A known error condition.

exception migrate.versioning.exceptions.NoSuchTableError

The table does not exist.

exception migrate.versioning.exceptions.PathError

Base class for path errors.

exception migrate.versioning.exceptions.PathFoundError

A path with a file was required; found no file.

exception migrate.versioning.exceptions.PathNotFoundError

A path with no file was required; found a file.

exception migrate.versioning.exceptions.RepositoryError

Base class for repository errors.

exception migrate.versioning.exceptions.ScriptError

Base class for script errors.

exception migrate.versioning.exceptions.UsageError

A known error condition where help should be displayed.

exception migrate.versioning.exceptions.WrongRepositoryError

This database is under version control by another repository.

Module genmodel – ORM Model generator

Code to generate a Python model from a database or differences between a model and database.

Some of this is borrowed heavily from the AutoCode project at: http://code.google.com/p/sqlautocode/

Module pathed – Path utilities

A path/directory class.

class migrate.versioning.pathed.Pathed(path)

A class associated with a path/directory tree.

Only one instance of this class may exist for a particular file; __new__ will return an existing instance if possible

classmethod require_found(path)

Ensures a given path already exists

classmethod require_notfound(path)

Ensures a given path does not already exist

Module repository – Repository management

Module schema – Migration upgrade/downgrade

Module schemadiff – ORM Model differencing

Schema differencing support.

class migrate.versioning.schemadiff.SchemaDiff(model, conn, excludeTables=None, oldmodel=None)

Differences of model against database.

compareModelToDatabase()

Do actual comparison.

migrate.versioning.schemadiff.getDiffOfModelAgainstDatabase(model, conn, excludeTables=None)

Return differences of model against database.

Returns:object which will evaluate to True if there are differences else False.

migrate.versioning.schemadiff.getDiffOfModelAgainstModel(oldmodel, model, conn, excludeTables=None)

Return differences of model against another model.

Returns:object which will evaluate to True if there are differences else False.

Module script – Script actions

class migrate.versioning.script.base.BaseScript(path)

Base class for other types of scripts. All scripts have the following properties:

source (script.source())

The source code of the script

version (script.version())

The version number of the script

operations (script.operations())

The operations defined by the script: upgrade(), downgrade() or both. Returns a tuple of operations. Can also check for an operation with ex. script.operation(Script.ops.up)

run(engine)

Core of each BaseScript subclass. This method executes the script.

source()

Returns:source code of the script. Return type:string

classmethod verify(path)

Ensure this is a valid script This version simply ensures the script file’s existence

Raises:InvalidScriptError

class migrate.versioning.script.py.PythonScript(path)

Bases: migrate.versioning.script.base.BaseScript

Base for Python scripts

classmethod create(path, **opts)

Create an empty migration script at specified path

Returns:PythonScript instance

classmethod make_update_script_for_model(engine, oldmodel, model, repository, **opts)

Create a migration script based on difference between two SA models.

Parameters:

• repository (string or Repository instance) – path to migrate repository
• oldmodel (string or Class) – dotted.module.name:SAClass or SAClass object
• model (string or Class) – dotted.module.name:SAClass or SAClass object
• engine (Engine instance) – SQLAlchemy engine

Returns:

Upgrade / Downgrade script

Return type:

string

preview_sql(url, step, **args)

Mocks SQLAlchemy Engine to store all executed calls in a string and runs PythonScript.run

Returns:SQL file

classmethod require_found(path)

Ensures a given path already exists

classmethod require_notfound(path)

Ensures a given path does not already exist

run(engine, step)

Core method of Script file. Exectues update() or downgrade() functions

Parameters:

• engine (string) – SQLAlchemy Engine
• step (int) – Operation to run

source()

Returns:source code of the script. Return type:string

classmethod verify(path)

Ensure this is a valid script This version simply ensures the script file’s existence

Raises:InvalidScriptError

classmethod verify_module(path)

Ensure path is a valid script

Parameter:path (string) – Script location Raises:InvalidScriptError Returns:Python module

module

Calls migrate.versioning.script.py.verify_module() and returns it.

class migrate.versioning.script.sql.SqlScript(path)

Bases: migrate.versioning.script.base.BaseScript

A file containing plain SQL statements.

classmethod create(path, **opts)

Create an empty migration script at specified path

Returns:SqlScript instance

classmethod require_found(path)

Ensures a given path already exists

classmethod require_notfound(path)

Ensures a given path does not already exist

run(engine, step=None, executemany=True)

Runs SQL script through raw dbapi execute call

source()

Returns:source code of the script. Return type:string

classmethod verify(path)

Ensure this is a valid script This version simply ensures the script file’s existence

Raises:InvalidScriptError

Module shell – CLI interface

Module util – Various utility functions

class migrate.versioning.util.Memoize(fn)

Memoize(fn) - an instance which acts like fn but memoizes its arguments Will only work on functions with non-mutable arguments

ActiveState Code 52201

migrate.versioning.util.asbool(obj)

Do everything to use object as bool

migrate.versioning.util.catch_known_errors(f)

Decorator that catches known api errors

migrate.versioning.util.construct_engine(engine, **opts)

New in version 0.5.4.

Constructs and returns SQLAlchemy engine.

Currently, there are 2 ways to pass create_engine options to migrate.versioning.api functions:

Parameters:

• engine (string or Engine instance) – connection string or a existing engine
• engine_dict (dict) – python dictionary of options to pass to create_engine
• engine_arg_* (string) – keyword parameters to pass to create_engine (evaluated with migrate.versioning.util.guess_obj_type())

Returns:

SQLAlchemy Engine

Note

keyword parameters override engine_dict values.

migrate.versioning.util.guess_obj_type(obj)

Do everything to guess object type from string

Tries to convert to int, bool and finally returns if not succeded.

migrate.versioning.util.load_model(dotted_name)

Import module and use module-level variable”.

Parameter:dotted_name – path to model in form of string: some.python.module:Class

Changed in version 0.5.4.

migrate.versioning.util.with_engine(f)

Decorator for migrate.versioning.api functions to safely close resources after function usage.

Passes engine parameters to construct_engine() and resulting parameter is available as kw[‘engine’].

Engine is disposed after wrapped function is executed.

Module version – Versioning management

class migrate.versioning.version.Collection(path)

A collection of versioning scripts in a repository

create_new_python_version(description, **k)

Create Python files for new version

create_new_sql_version(database, **k)

Create SQL files for new version

version(vernum=None)

Returns latest Version if vernum is not given. Otherwise, returns wanted version

latest

Returns:Latest version in Collection

class migrate.versioning.version.Extensions

A namespace for file extensions

class migrate.versioning.version.VerNum(value)

A version number that behaves like a string and int at the same time

class migrate.versioning.version.Version(vernum, path, filelist)

A single version in a collection :param vernum: Version Number :param path: Path to script files :param filelist: List of scripts :type vernum: int, VerNum :type path: string :type filelist: list

add_script(path)

Add script to Collection/Version

script(database=None, operation=None)

Returns SQL or Python Script

migrate.versioning.version.str_to_filename(s)

Replaces spaces, (double and single) quotes and double underscores to underscores