DISQUS

Atamert Ölçgen · 1 year ago

Great post! I don't need sharding for any of my projects. But now I can see how to implement it if needed.

I think it would look much cooler if the database name was a class variable. But I guess it would require patching the models.Manager. Such as;

class MyClass(models.Model):
database = 'secondary'

Jannis Leidel · 1 year ago

Very cool, Eric. One little nitpick with the way the databases are specified in the settings. I know that it's just Python, but it seems cleaner to me to prevent dict() calls. It would be nicer to just use a simple tuple approach and convert them to dictionaries in the backend.

See http://dpaste.de/605/ for an example.

Jijnes Patel · 1 year ago

Nice post. If you're using MySQL, you could use MySQL Proxy, which will accept connections from your Django application server and inspect the queries. Based on rules written in Lua, you could direct queries using the primary / secondary model you have in this post, or split reads to a Slave instance and writes to a Master.

Leah Culver · 1 year ago

That's pretty much how we did things :D Thanks to Malcolm for adding the hooks in Django 1.0 that allow for multiple databases!

Lis · 1 year ago

I'm getting deadlocks when trying to update on mysql/innodb
http://dpaste.com/92408/

enoch · 1 year ago

looks great. any troubleshooting help. i setup the same stuff and keep getting a problem in the admin views for the Link objects that "Something's wrong with your database installation. Make sure the appropriate database tables have been created, and make sure the database is readable by the appropriate user."

what's odd is, that's on listing, but on "add new" link, it brings up a from fine to add a new link....

enoch · 1 year ago

fixed the other problem. type on database name on my part.

looking at a new problem. when i try to access the Link objects i get "type object 'Link" has no attribute 'objects'

l = Link.objects.all()

guille_ · 1 year ago

Not thread-savvy. It would be cool to pass the settings as a dictionary and not changing the global settings object.

jfy3d · 1 year ago

Forenkey has error

how fix it ?

re · 1 year ago

good reading!!

Adrian Holovaty · 10 months ago

As of http://code.djangoproject.com/changeset/10026, this code snippet will no longer work. Fortunately, it's a lot easier/cleaner now. The get_db_wrapper() method can be rewritten something like this (untested):

def get_db_wrapper(self):
database = settings.DATABASES[self.database]
backend = __import__('django.db.backends.' + database['DATABASE_ENGINE']
+ ".base", {}, {}, ['base'])
return backend.DatabaseWrapper(database)

Jordan Keyes · 9 months ago

I have tried this both with Django 1.0.2 and with the latest SVN version (from 3/16/09) using the settings Adrian mentioned. Both ways, the multi_syncdb shown seems to be doing a two-way sync (putting all of my models in both databases).
If I run "python manage.py shell" I can manually create items in the tables, but it doesn't appear to be using the _default_manager to decide where to put the items, just going for the secondary database.
Also, it seems the manager is working, but only partially. Any time I try to access the objects in the tables, I get errors like "type object 'TwebprPress' has no attribute 'objects'" (for TwebprPress.objects.all()) Have I done something wrong?

For reference, I am using Python 2.5, Django from SVN, django-mssql from SVN for SQL 2005 connection.

Jordan Keyes · 9 months ago

I may have solved my own problem. In your example I changed the line
"_default_manager = MultiDBManagr('secondary')" to
"objects = MultiDBManager('secondary')" on each model (changing secondary to the appropriate DB name from the Dict.) and things seem to be working appropriately now. The multi_syncdb is still a little borked, but I can do without that.

Thanks for the great tutorial, Eric!

J · 9 months ago

FYI:

Using _default_manager in a model causes many idle MySQL connections. (Django 1.0)

http://groups.google.com/group/django-users/browse_thread/thread/6ed797f6b00dca90

Praveen · 9 months ago

Hi Nice articles, i have not tried but may be in future will really help.

Richard McMillan · 8 months ago

For those on SVN django: as of 10059 you will need to add :

use_for_related_fields = True

to the subclassed manager for this to work.

ben 10 oyunları · 7 months ago

Based on rules written in Lua, you could direct queries using the primary / secondary model you have in this post, or split reads to a Slave instance and writes to a Master.

Nigel Cohen · 7 months ago

I spent many years trying to create multiple databases with MySQL. I finally got to the simple solution (for the several transactional based database applications we wrote) of preceding the table name with the database name (eg. db1.person inner join db2.organisation).

Could this essential (for large apps) multi-database functionality not be achieved quite simply by setting up a variable in the Model object allowing a database name to be updated - which the MySQL generating code looks at when generating the MySQL code?

Emily · 6 months ago

Thanks for this.

I am having issues with multiple connections not being closed though (has been mentioned above).

Has anyone got a work around for this?

Emily · 5 months ago

If anyone else was stuck with the multiple mysql connections issue, I *think* I have got around it (with help from other people's ideas!):

http://groups.google.com/group/django-users/msg...

ha22109 · 5 months ago

It is throwing an error when i changed my settings.py

the error is:

could not connect to server: No such file or directory
Is the server running locally and accepting
connections on Unix domain socket "/var/run/postgresql/.s.PGSQL.5432"?

DNKorolkov · 5 months ago

Problem with create of MultiDBManager for ManyToManyField.

MultiDBManager.__init__ require 'database' parameter.
Its cause error with call "super(ManyRelatedManager, self).__init__()" in ManyRelatedManager.__init__
(function create_many_related_manager of django/db/models/fields/related.py).

Proposed solution:
Set 'database' as class attribute instead __init__ argument:

MultiDBManager = type('MultiDBManager', (MultiDBManager,), {'database':'db_name'})

P.S. Excuse my English, please ;-).

DNKorolkov · 5 months ago

Of course, different classes must be created for different database:

MultiDBManager_primary = type('MultiDBManager_primary', (MultiDBManager,), {'database':'primary'})

MultiDBManager_secondary = type('MultiDBManager_secondary', (MultiDBManager,), {'database':'secondary'})

sohbet · 5 months ago

that's a great post, thank you!

radyo dinle · 5 months ago

that's a great post, thank you!