Options for managing a multi-purpose web service?

Question

I have a web service that serves two purposes.

1. On a weekly basis, it syncs records from the GIS DB to a workorder management system's database.
2. On a continual basis, it serves up the records to a web map in the work order management system.

---

Here is a summary of purpose #1 and purpose #2:

(There are 40+ spatial views/layers in the web service that are based on spatial tables.)

---

The current multi-purpose web service works fine for #1 (db sync), but it's too slow for #2 (the web map).

Question:

What are my options for managing this multi-purpose web service?

Ideas:

• Split the single web service into separate web services. Each service would be catered to a specific purpose.
• Or precompute the layers in the web services by using materialized views or a python job (to export the views to static tables). This would solve the performance issues in 2A and 2B.
• Or something else?

Answer 1

It's a common pattern to have one data source as the source of truth, and a secondary data source for retrieval. The secondary data source often trades increased storage requirements for speed of access. Factors to consider:

1. Rate of churn at the source
2. Having a good option to determine churn (for example a lastChanged column, or a change data capture (CDC) facility within the database)
3. Computational complexity of building the secondary data source
4. Impact of latency and having outdated information. For example, if your data warehouse is five minutes behind the actual data, and you want to know if you sold more groceries or more sporting goods that day, your are probably OK. If a customer is putting that hot selling TV into the online shopping cart, probably less so.

A data warehouse is better off in another database, because it does not need to be fully ACID, and has other access patterns (Investigate OLTP vs OLAP, even if you don't have a traditional analytics use case)

Answer 2

Split the single web service into separate web services. Each service would be catered to its specific purpose.

I think this is a good fit. Your 2 use cases is highly likely grow in different ways:

1. You might get new data sources to sync, potentially from internal data teams or new external sources.
2. You might get new feature requirements on visualizing the data sources, or simply much higher load.

These look like 2 separate bounded contexts, and highly likely to scale independent of each other which makes it a good fit to break down into 2 separate services.