9
votes
Accepted
How to handle database errors after a successful 3rd party payments API response?
Trying to cover failures like this with business logic and retries is in my view a mistake.
You can always think up a scenario which isn't covered by your logic. In your case, case the DB write has ...
- 79.8k
7
votes
Accepted
How to "state of the art" integrate multiple sub-modules with duplicate dependencies?
Your incompatible Logger implementations cannot be linked because they have the same name. This leads to a simple solution that will solve all your problems:
Incompatible versions should have ...
- 135k
7
votes
Accepted
What is a "streaming app", exactly?
A streaming app is an app that consumes a stream of data.
A stream of data is transmitted data formatted in a way that can be useful even when incomplete. Since partial stream data does not require ...
- 115k
6
votes
Accepted
How to deal with bad third party APIs in a microservices architecture?
I encounter this type of thing all the time, there are some unbelievable commercial SOAP/XML APIs out there. What I normally do is basically two things:
Define a facade for the functionality my ...
- 651
5
votes
Bidirectional sync between two databases with differing schemas
You should not do this.
Doing a unidirectional sync is challenging, adds a pile of constraints to your system, but is a fairly well known problem at this point. When you make things bidirectional, ...
- 110k
4
votes
Accepted
Data integration from heterogeneous sources
The fundamental problem here is that REST is not a standard. It is an architectural style for web APIs. While there are mechanisms to make these APIs self-describing, and there are web-service ...
- 135k
4
votes
Accepted
Good idea to use source control with my Lambda functions?
As a rule you always want to have version control for your deployed code - typically this means a source code repository separate from your application.
In the case of AWS Lambda, there is a built in ...
- 7,813
4
votes
Accepted
How can testability help in identifying and fixing bugs before they are integrated into the system?
How can testability help in identifying and fixing bugs before they are integrated into the system?
I can see two potential reasons for your confusion.
First
Implementing Testability (something which ...
- 254
3
votes
Accepted
Bidirectional sync between two databases with differing schemas
If this is feasible depends heavily on what 'similar' data means, and if the systems are sufficiently different, this can become arbitrary complex, and you may have to resdesign the existing systems ...
- 214k
3
votes
Sending JSON in body of an email
JSON in the email body is problematic. Mostly because it’s between hard and impossible to guarantee that an email body isn’t modified by anyone. JSON in an attachment is just fine, especially if you ...
- 47.5k
3
votes
Accepted
Sending JSON in body of an email
This is a slightly creative architecture, but generally fine. Some caveats:
Email is notoriously unreliable: emails may arrive delayed, out of order, or not at all.
Emails are not necessarily ...
- 135k
3
votes
Accepted
Legacy modernisation - Parallel systems vs Extending Original application?
I'm assuming you have a small team and a limited budget, so my answer assumes the following is true:
Maintaining 2 separate applications would be a big strain on the team
The work to integrate the ...
- 46.3k
3
votes
How can I inform my team when components are ready for integration?
If you don't want to add a new tool then use what you have. I like that task descriptions are editable in most ticketing tools.
Task: 54328 - Make "Number of Dependents" spinner stop allowing ...
- 115k
3
votes
Is there a better way of communicating between systems?
Apache Kafka works in realtime, and is somehow in vogue for solving the subsystem communication problems that used to be solved with centralized databases or queue management systems.
As to the XML, ...
- 2,283
3
votes
What happens after the ETL process?
An ETL is just a procedure. You:
Extract the data from any source. In your case, this would be the CSV files, but it could be anything: data from an external service, data stored in some proprietary ...
- 136k
3
votes
Accepted
Is it reasonable to pass file URLs to microservices instead of the file contents?
I'd say number 2 is perfectly fine in theory but you can improve on it, which would alleviate some of the concerns that are leading you to doubt if it's a valid approach. In simpler terms, your cookie ...
- 56.3k
3
votes
Accepted
Process many types of work in parallel, but sequential for each type of work
We started with experimenting with the dynamic creation of queues. This feels like very dynamically creating/suspending/resuming infrastructure. Is this right way to go?
How to model this in a good ...
- 2,060
2
votes
How do you scale your integration testing?
In my experience in an Agile or DevOps environment where continuous delivery pipelines are common, integration testing should be carried out as each module is completed or adjusted. For example, in ...
- 31
2
votes
Accepted
What is the most appropriate standard for modeling B2B integration
If your role is to choose a standard, choose a standard and don't look for exotic approaches.
BPMN and process modeling
BPMN is an OMG standard since 2006 and the international standard ISO/IEC 19510 ...
- 80.6k
2
votes
How can I inform my team when components are ready for integration?
A board filled with sticky notes (or a Jira board) is as simple as you can get. A glimpse on the board should give enough information about the progress of the team, and if it doesn't, it means that ...
- 136k
2
votes
Process many types of work in parallel, but sequential for each type of work
I'd tackle the problem from different angles.
Is the problem really well-understood?
You didn't explain what the reason for wanting to handle tasks of one kind sequentially. Are there resource ...
1
vote
Accepted
Microservice integration with an external legacy system
(This is not meant to be a comprehensive "cookbook" or "list" answer of all possible design approaches. Simply an answer with one possible design approach.)
Frequently when ...
- 762
1
vote
Options for managing a multi-purpose web service?
Split the single web service into separate web services. Each service would be catered to its specific purpose.
I think this is a good fit. Your 2 use cases is highly likely grow in different ways:
...
- 325
1
vote
Accepted
Options for managing a multi-purpose web service?
It's a common pattern to have one data source as the source of truth, and a secondary data source for retrieval. The secondary data source often trades increased storage requirements for speed of ...
- 2,917
1
vote
How to handle database errors after a successful 3rd party payments API response?
Scenarios like this come up a lot when dealing with services external to your system, especially in microservices architectures. In your case, the only external dependency is PayPal. In my opinion, ...
- 139
1
vote
How to handle 3rd party system API timeout
Consider implementing a Circuit breaker which will find when third party is available to retry request:
Assume that an application connects to a database 100 times per second and the database fails. ...
- 303
1
vote
Accepted
Metadata evolution considerations?
I'd follow these principles:
Keep (meta)data format expandable; do not force expansion.
Allow verification and sanity checks.
Allow and encourage comments.
Always version the data.
Point by point, ...
- 24.3k
1
vote
Accepted
Integration between REST API and database
I would recommend that you create your own interface that your client uses that wraps the vendor interface. Perhaps that's what you mean by 'CRUD' here but be careful with mixing CRUD and REST ...
- 28.9k
1
vote
Integration between REST API and database
As i see it, you have the concept of workspaces for your application, but different clients are interested in more / less information? I think your current approach is unlikely to prove maintainable ...
- 1,123
1
vote
Accepted
CRM and external systems: How to associate data with contacts?
When you have entities in system A and some related data in a second, decoupled system B, and you need to update the data in system B depending on changes to entities in system A, system A needs to ...
- 214k
Only top scored, non community-wiki answers of a minimum length are eligible