Is it bad practice for an API object definition to contain 3rd party reference Id's as properties?

Question

Like this:

Campaign:
type: object
properties:
  id: 
    type: string
    description: "A GUID identifier"
  referenceId:
    type: string
    description: "A consumers identifier they have used to map their own systems logic to this object."
  name:
    type: string
    description: "'Great Campaign 2017' as an example"

I am concerned about the referenceId.

The system domain is a platform which is integrated with 3rd parties in many ways through data exports and imports of various formats (xml, excel). It is mature enough to allow 3rd parties to integrate with our system through an API and the design of this API is what prompts this question.

We have an object, a Campaign, which has an id that can be used to identify and retrieve the resource. Consumers of our API may have their own reference code to what they consider to be a Campaign within their domain.

There are other objects in our system with 3rd party reference fields like this and it's expected from our existing consumers. However I worry it puts the burden of mapping on us and we do not know what this referenceId is (number, text, json?) and it adds another confusing property to the API for new consumers.

Is it considered bad practice or bad design to allow 3rd Party reference Id fields in the public object definitions for an API?

Answer 1

This is not a problem; it is a necessity. The lack of this field would be problematic for integrating with customer systems.

There are plenty of common use cases for this sort of thing. For example, an API involving billing is likely to allow companies to specify their own invoice numbers, workforce management software needs you to be able to enter your own local employee IDs, etc.

The most straightforward design to avoid any concerns is simply to not take on any responsibility for the field. Just provide it and let customers use it if they choose. Don't validate it or use it in your own logic, as doing so (even with functionality that seems good) could entangle you in the customer's own design problems or bugs, as well as creating vendor specific expectations or feature requests. Certainly don't use this value as an ID internally. The data structure you have shown implies this is the approach you are taking.

In terms of formats, it just needs to be permissive enough to allow anything reasonable, and then you don't have to care what is in it. This is what you have done by making it a string field.

To me, the name referenceID is not so clear. I might call it something like customerLocalID. But then again, maybe your terminology makes sense in your domain. In any case, I don't see a problem for new customers as long as the field is clearly documented (especially highlighting that it is optional).

Answer 2

I don't think that there is a best practice regarding this. Holding an opaque referenceId in your system is required or not depending on your relation with the 3rd party clients.

Strictly speaking, most probably, it is not your system's responsibility to map between your model and the 3rd party model. It's theirs. You just help them to do that mapping by holding that referenceId.

But again, if this is part of your contract between you and them then you have to keep your part of the bargain and provide that opaque property.

Answer 3

3rd party references are a good idea where any particular data is owned by the third party, and you are just a custodian.

It is also extremely helpful to establish a mechanism for idempotency for writes/updates.

So, in the first part, it is important to establish the contract around that reference. If it is unique, then enforce it with the appropriate logic and warning/error codes on write.

For flexibility, it is typical for the references to be arbitrary strings.

Additionally, I recommend to use internal identifiers also, as you have done, so my data model is not reliant on any particular format for keys.

All internal references will then use the internal identifier. This also fits better with the REST world which may do things like apply id's in-line with the URL, see the next point.

On the external API, allow queries using either identifier. You can do it with a separate endpoint, or (in REST world) using a query parameter.

Re idempotency, by using a unique external identifier, it is possible to detect repeated attempts to create a record, avoiding "double write" errors. For me, that is the clear reason to not only support the concept, but make it mandatory, if you can.

Failing that you can use operation transaction id's/message id's, but that is out of scope of the question.