Design Methodology for Developing Interoperable Systems?

Question

A bit of background

The company I work for has been creating database applications since around 1980 and, until relatively recently, most of these systems have been stand-alone "silo" systems.

However, in the last few years, we've seen more of a shift towards gathering "intelligence" across all systems within our customers requirements. For example, a system that might collect data about people's dental records may need to talk to a system that collects data about people's optician records, share information about the same person and provide search capabilities across both systems (makes sense, although the example I've used is fictional).

In the fictional example above, the dental and optician systems may or may not both be written by the company I work for (in other words we are looking at both internally-developed interfaces and interfaces to systems from third-party providers).

What we've done so far

We have a basic web-services API (.asmx) for one of our products, but I'm concerned about the change management, in terms of keeping it up-to-date as we add new features to the main product (in terms of providing the new features through the API without breaking compatibility with older interfaces).

We have also produced one WCF service. The rigid contracts put my mind slightly more at ease than working with the ASMX service, but I think a lack of experience within the company is holding us back, so I'm looking for some guidance on best practice that I can read up on to help move us along (or at least point us in the right direction).

The question

• Is there a particular methodology or programming practice for designing systems to maximise their ability to interface to other systems?

• Are there any good online (or offline) resources for these programming practices (assuming they exist)?

Any other advice you can provide based on the information above would be appreciated, but not essential.

We primarily develop in Microsoft .NET (mainly ASP.NET, with some windows forms applications as well) with a SQL Server/Oracle database back-end, although I expect the programming practices might be language independent. (I've used the .NET tag as an afterthought, but I might take it out later..!)

Thanks for your time.

Answer 1

You can follow the Jeff Bezos method as described here https://plus.google.com/112678702228711889851/posts/eVeouesvaVX

His Big Mandate went something along these lines:

1) All teams will henceforth expose their data and functionality through service interfaces.

2) Teams must communicate with each other through these interfaces.

3) There will be no other form of interprocess communication allowed: no direct linking, no direct reads of another team's data store, no shared-memory model, no back-doors whatsoever. The only communication allowed is via service interface calls over the network.

4) It doesn't matter what technology they use. HTTP, Corba, Pubsub, custom protocols -- doesn't matter. Bezos doesn't care.

5) All service interfaces, without exception, must be designed from the ground up to be externalizable. That is to say, the team must plan and design to be able to expose the interface to developers in the outside world. No exceptions.

6) Anyone who doesn't do this will be fired.

7) Thank you; have a nice day!

Answer 2

I would suggest designing a suite of example programs that interoperate with your system. Use the platforms most likely to be used to interact with you- say maybe a .NET suite, a Java suite, a PHP suite... considering which versions might be in use (i.e. if you provide a Java 5+ suite... are you sure there are not people stuck in 1.4 in your area?).

Maintain those suites, use them as test suites (you can use them to check for breaking changes), provide them freely to whomever wants to interact with your system.

If you give me a jar with classes which encapsulate communication to your system and examples of use, I'll be happy. Moreover, if someone has a problem you'll have less problems like "huh, we are using XFire for Soap, not Axis...".

And above all, communicate!

Answer 3

we've seen more of a shift towards gathering "intelligence" across all systems within our customers requirements.

You clearly need a BI System\Data Warehouse (Business Intelligence). Building one basically entails creating a new central Data store, where all systems push (or have pulled) data into that systems on a regular basis. That data is all married together somehow, then optimized for data retrieval & reporting speed.

Because you are a .Net Shop, I am going to recommend you look at SSAS & SSRS (SQL Server Analysis services and SQL Server Reporting services.)

Obligatory Link : http://en.wikipedia.org/wiki/Business_intelligence

Is there a particular methodology or programming practice for designing systems to maximise

their ability to interface to other systems?

There really is no correlation between methodology and design. I have never heard of a particular methodology claiming to work better for particular programming patters, design or technologies.

Answer 4

My experience with this is very small, but I believe you could look into SOA (service oriented architecture). Its goal is exactly what you want: interoperable software components.

Don't confuse SOA services with the typical .NET services (asmx, wcf). The communication between 2 SOA services can go through asmx or wcf, but can also use Messaging (MSMQ, ...) for example.

Answer 5

I would suggest using something free like Protocol Buffers. It's available for all major languages, comes with generators and is reasonably fast. And it is proven in the field.

From the looks of it, the strength of WCF lies in its ability to consume a huge variety of formats. However in a long term perspective, your interest should be to shape your providers for interoperability, not your consumers.