2

I have an application that is divided into two main parts:

  1. A web crawler written in Python
  2. A REST API written in Golang

They share a MySQL database, which is mainly populated/updated by the web crawler, and then read from by the REST API.

I wanted to use an ORM on each side to make interacting with the database easier, but now I am not sure if this is a good approach.


On the Python side, I was looking at using peewee, and for Golang I will use gorm.


It doesn't seem like a good idea to maintain two sets of ORM model files, where I would have to change each set whenever I want to change some attributes of the database. Additionally, I am worried that with two ORMs battling for structure of the database, that conflicts might arise, or the models might become out of sync with what the actual db structure is.


Is this a scenario where using an ORM will cause more trouble than its worth? Or maybe I should use a single ORM (for either Python or Golang) and write raw queries for the other side.

4
  • 2
    Just a quick question: can't you write two generators which generate the ORM model files from a shared description? That way you still have 1 representative definition of your model. For bonus points, let it also generate the SQL schema definition (or, if both your libraries support it, let both of them generate the definition, and assert that they generate the same definition) – Sjoerd Job Postmus Oct 13 '15 at 4:28
  • 1
    If you change the DB schema you'll always have to change the clients, ORM or not. – Jeroen Vannevel Oct 13 '15 at 7:19
  • @JeroenVannevel Usually I would change the DB schema via an ORM, but yes you're correct. – TylerAndFriends Oct 13 '15 at 8:48
  • write all your queries using stored procedures - ten you can change your DB schema and keep a fixed API. – gbjbaanb Oct 13 '15 at 10:42
2

I think it is okay to have two sets of ORM if you think it will make your life easier in terms of build the applications. Think of it this way without the ORM as well you will need to change the queries in both applications if you make some database schema changes etc. So I won't count that to be a huge issue. As long you as can gain productivity by using ORM on both applications it should be okay.

0

Use both, and write a Python script that'll import the peewee models and convert them to GORM models.

I've scratched something very basic. I've created models.py with the example in peewee's README.md:

from peewee import *
import datetime

db = SqliteDatabase('my_database.db', threadlocals=True)

class BaseModel(Model):
    class Meta:
        database = db

class User(BaseModel):
    username = CharField(unique=True)

class Tweet(BaseModel):
    user = ForeignKeyField(User, related_name='tweets')
    message = TextField()
    created_date = DateTimeField(default=datetime.datetime.now)
    is_published = BooleanField(default=True)

And here is a simple script that imports it and create Go code:

import peewee

def enumerate_models():
    import models
    for name in dir(models):
        item = getattr(models, name)
        if isinstance(item, type) and issubclass(item, models.BaseModel) and item != models.BaseModel:
            yield item


def model_fields(model):
    for name in dir(model):
        item = getattr(model, name)
        if isinstance(item, peewee.Field):
            yield name, item


def field_go_type(field_spec):
    return {
        peewee.PrimaryKeyField: 'uint',
        peewee.ForeignKeyField: 'uint',
        peewee.TextField: 'string',
        peewee.CharField: 'string',
        peewee.DateTimeField: 'time.Time',
        peewee.BooleanField: 'bool',
    }.get(type(field_spec), '?%s?' % type(field_spec))


for model in enumerate_models():
    print('type' ,model.__name__, 'struct {')
    for field_name, field_spec in model_fields(model):
        field_go_decleration = []
        field_go_decleration.append(field_name)
        field_go_decleration.append(field_go_type(field_spec))
        print('\t' + ' '.join(field_go_decleration))
    print('}')

It's output:

type Tweet struct {
    created_date time.Time
    id uint
    is_published bool
    message string
    user uint
}
type User struct {
    id uint
    username string
}

My script still needs more work to add the annotations, but you get the basic idea.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.