Define a RESTful API for creating/updating other resource definitions?

Question

Let's say I have a users resource, with two properties: name and email as specified by a users JSON Schema document, which right now looks like this:

{
  "$schema": "http://json-schema.org/draft-04/schema#",
  "type": "object",
  "additionalProperties": false,
  "properties": {
    "name": {
      "type": "string"
    },
    "email": {
      "type": "string"
    }
  },
  "required": [
    "name",
    "email"
  ]
}

My requirements state that we need to be able to change the schema, e.g. to add a property such as phoneNumber, and do so via HTTP in a RESTful way. That is, I need to be able to update the JSON Schema definition of the users resource to look like this:

{
  "$schema": "http://json-schema.org/draft-04/schema#",
  "type": "object",
  "additionalProperties": false,
  "properties": {
    "name": {
      "type": "string"
    },
    "email": {
      "type": "string"
    },
    "phoneNumber": {
      "type": "string"
    }
  },
  "required": [
    "name",
    "email",
    "phoneNumber"
  ]
}

Now, clients of the API can create new users that have the additional phoneNumber property (where previously I would have gotten a schema validation error).

I am puzzling over how to do this. One way I can imagine doing it is by creating a "meta-resource" called resources. This resource might have some properties, for example: path and schema. The schema property would be a full JSON Schema object. To update the users resource, then, I could maybe POST to resources with an HTTP request body like:

{
    "path": "users",
    "schema": { ...JSON Schema object goes here... }
}

Is this a reasonable implementation? If not, why not? Alternative ideas? Any pitfalls I should watch out for? Any articles/blogs on this topic that I should read? (I haven't been able to Google successfully for this).

Answer 1

The simplest solution here would seem to but to call PUT on the user resource with the new version of the schema. This is a problematic in general because it's subject to dirty reads. This is also unworkable for your specific scenario due to the versioning requirements that you have noted in the comments below.

As I understand it, a given version of each schema is immutable*. Given that requirement, I think best answer here is that when a change is made to the schema, you would do a POST to the resource root and let it determine the new version number. For example, say you have resource /schema/user/1, if you want to modify it, the schema is retrieved, edited and POSTed to /schema/user which will create resource /schema/user/2. You could try to do some sort of PATCH action against the schema and let it create the new merged version for you but that is far more difficult to implement and I don't think it would be worth it unless there is some specific need for it to happen on the server.

*You may need to make accommodations as necessary to support corrections.

Answer 2

If you really want to modify the data schema from an external source, you'll probably want to ensure that there is a good authorisation system in place to ensure that schema updates are happening in a permitted way. Can you imagine what would happen if someone managed to send malicious schema-altering messages that resulted in all attributes being dropped from everything? I'd also recommend tracking every change that comes in.

In general:

• why update the data schema remotely? Does your domain really change that much that it's necessary to be able to simply add/remove attributes to types in the schema?
• Have you considered how clients are supposed to respond to this changing schema? Do they expect frequent changes?
• What's the underlying datastore? An SQL database? A key-value store? Will existing data need to be changed every time you changes the schema? Will underlying data structures (such as SQL databases) need to be ALTERed every time this happens?
• What about any generated code that matches the schema, will all that need to be regenerated too?

I've never seen this done before. Is there a compelling reason to be able to do this?

---

But if you really want to do this...

This operation is different enough that I think it should have its own "schema-update" endpoint. Maybe something of the form:

http://myserver.com/schemaManagement/(maybe optional schema version number?)/update/$objectType/$operation/$newAttribute

So adding the phoneNumber to person might involve a POST to http://myserver.com/schemaManagement/current/update/person/add/phoneNumber and the content of the message could include details about the datatype being added, whether or not is is optional (and if not, what default value to use), authorisation tokens, etc...

Removing an attribute might look like:

http://myserver.com/schemaManagement/current/update/person/remove/faxNumber

I'm not a REST expert but this seems to me to be a pretty intuitive and easy-to-manage way to handle something like this.

Answer 3

If there is a possibility to change a schema remotely, then there must also be a mechanism for clients to retrieve the current schema.
If you combine that with a REST interface, then it becomes fairly logical to make the schema itself also a REST resource.

This would mean that you would retrieve the current schema for the user resource with

GET http://example.com/api/schemas/user

and you change the schema with

PUT http://example.com/api/schemas/user

and other resources with a schema similarly. The resource targeted by the schema becomes the key into the schemas collection.

Answer 4

How do you handle the updated schema in your Web service?

Using the approach in your question, you probably will have to update your backend code as well. So I do not think that changing/updating resource definitions via RESTful API is a good idea.

If the backend is able to handle dynamic data, I would suggest to use a dynamic schema. A simple example could be a key/value list.

The JSON schema could be something like that:

{
  "$schema": "http://json-schema.org/draft-04/schema#",
  "type": "object",
  "additionalProperties": false,
  "properties": {
    "values": {
       "items": [{
        "properties": {
         "key": {
          "type": "string"
         },
         "value": {
          "type": "string"
         }
        },
        "type": "object"
       }],
       "type": "array"
      }
  },
  "required": [
    "values"
  ]
}

If your backend changes frequently, it could make sense adding an endpoint to GET an updated schema definition from the backend.

Answer 5

Well, you can try to accept JSONPatch documents when you do PATCH HTTP request.