Episode #125 of the Stack Overflow podcast is here. We talk Tilde Club and mechanical keyboards. Listen now
103

You should never trust the inputs to your software, regardless of source. Not only validating the types is important, but also ranges of input and the business logic as well. Per a comment, this is well described by OWASP Failing to do so will at best leave you with garbage data that you have to later clean up, but at worst you'll leave an opportunity for ...


101

Youtube can't use sequentional IDs for two reasons: Its databases are almost certainly distributed, making sequential numbering complicated. It has a privacy option "Unlisted videos": those that don't show up in the search results, but are available if you know the ID. Therefore, the video IDs should be reasonably random and unpredictable. Whether the ID ...


81

An ideal ReSTful service allows clients (which may not be in-browser) to perform any needed task in one request; because the full state needed to do that is held by the client, not the server. Since the client has full control of the state, it can create the state on its own (if that is legitimate), and only talk to the API to "get 'er done". Requiring ...


77

429 Too Many Requests The user has sent too many requests in a given amount of time. Intended for use with rate limiting schemes. This code has been accepted in RFC 6585 Additional HTTP Status Codes. The 429 status code indicates that the user has sent too many requests in a given amount of time ("rate limiting"). The response representations ...


75

On the form of the IDs: They're using Base64 (using the characters a-z, A-Z, 0-9, -, and _). This allows them to have 6 bits of information per character. YouTube uses 11-character video IDs, which means they can generate 26*11, or more than 7*1019 IDs. As Tom Scott put it, that's "enough for every single human on planet Earth to upload a video every minute ...


68

Databases are not very good at information hiding, which is quite plausible, because their job is to actually expose information. But this makes them a lousy tool when it comes to encapsulation. Why do you want encapsulation? Scenario: you tie a couple of components to an RDBMS directly, and you see one particular component becoming a performance bottle-...


58

Martin Fowler's book "Patterns of Enterprise Architecture" states: The easier question to answer is probably when not to use it. You probably don't need a Service Layer if your application's business logic will only have one kind of client - say, a user interface - and it's use case responses don't involve multiple transactional resources. [...] But ...


58

REST is indeed an architectural style. SOAP is a data protocol. The distinction is important; you cannot compare them directly. The primary purpose of REST is to represent resources on the Internet, and to provide mechanisms for discovering them. In contrast, SOAP is used for communicating structured data between computers, and that's all it really does. ...


54

What is more important and significant about a microservice: its API or its database schema? The API, because that is its contract with the rest of the world. The database schema is simply a convenient way of storing the data managed by the service, hopefully organised in a way that optimises the microservice´s performance. The development team should be ...


40

In my opinion you should mock the webservice calls if this is a unit test, as opposed to an integration test. Your unit test should not test whether the external webservice is working, or whether your integration with it is correct. Without getting too dogmatic about TDD, note that a side effect of turning your unit test into an integration test is that it'...


39

Does the business value of implementing them exceed the cost? If you implement, you need to change not just your server, but all clients (although you can support both formats and only change clients as needed). That will take time and testing, which is a direct cost. And don't underestimate the time taken to really understand protocol buffers (especially ...


38

This, "However, philosophically - the first approach is the only approach.", and this "The proper official RESTful approach is to use Accept: header." are widely perceived to be the case, but are also absolutely incorrect. Here's a brief snippet from Roy Fielding (who defined REST)... "section 6.2.1 does not say that content negotiation should be used all ...


34

Adding a service layer because you have evaluated the idea and concluded its the best approach: good Adding a service layer because that's what all the cool kids are doing: bad If your gut says you don't need one, then don't make one. One of the more disappointing developments in the programming world over the past 10 years or so is that it has become ...


33

Yes, of course. But what makes you think the answer could be different? You surely don't want to let your program behave in some unpredictable manner in case the API does not return what the contract says, don't you? So at least you have to deal with such a behaviour somehow. A minimal form of error handling is always worth the (very minimal!) effort, and ...


31

What you are asking for is Holy water, and you are rejecting the suggestion that you use regular water. The only reason for preferring Holy water over water is religious. There are simple, randomly-seeded PRNGs that cannot be distinguished from true physical randomness by any known process. And these systems are non-deterministic. A real-world computer has ...


31

How about making a cronjob, assuming you have shell access? The cron daemon exists on virtually any UNIX-like system and schedules commands to run based on a description in a file called the crontab. Each line of the file contains a set of fields to indicate the timepoints when a command shall be executed. Your task could be either a standalone program ...


30

No, it won't. The point of unit tests is precisely to test your code in isolation, independent of the external world. Testing your whole system interacting with external parties like web services etc. is integration/system testing. This is also needed in most to all real world projects, but it is a different level than unit tests. Actually it sounds like in ...


29

REST is much more limited than SOAP, which is its strength and the reason for its popularity. In SOAP, the set of operations allowed and the set of data types allowed is essentially limitless. SOAP is a remote procedure protocol, which you use to expose local API's across the network without losing any fidelity. This made SOAP popular in enterprise ...


28

Python sandboxing is hard. Python is inherently introspectable, at multiple levels. This also means that you can find the factory methods for specific types from those types themselves, and construct new low-level objects, which will be run directly by the interpreter without limitation. Here are some examples of finding creative ways to break out of ...


28

Basically, abstraction. SQL requires your clients to know your exact database structure, which is not good. On top of that, analysing the SQL in order to perform special operations based on the value sent as the input is a really difficult thing to do. There are entire softwares which are pretty much responsible only for that. Do you know what those are? If ...


26

HTTP 204 means that something was found, but it's empty. For instance, imagine that you're serving log files through HTTP, with the requests such as http://example.com/logs/[date-goes-here]. On May 18th, 2015: http://example.com/logs/2015-05-19 would return HTTP 404, which means that there are no logs, because, well, it's difficult to log the future. http://...


25

Adding a web service layer gives you an opportunity to make your client more lightweight, both in terms of the required CPU power and the bandwidth used during the processing. Both factors are extremely important to end-users: Using less CPU increases the battery life, Using less bandwidth reduces monthly payments for users with metered plans By ...


25

But is that really important? Consider that the UI has to make a network call to the API; that's pretty big (order of magnitude of milliseconds). Databases are optimized to keep things in memory and execute reads very, very quickly (eg. SQL Server loads and keeps everything in RAM and consumes almost all your free RAM if it can). The Logic In theory, ...


23

i maintain apis and somebody before me added protobuf (because it was "faster"). The only thing faster is RTT because of of smaller payload, and that can be fixed with gzipped JSON. The part that is distasteful to me is the relative work to maintain protobuf (compared to JSON). I use java so we use Jackson object mapping for JSON. Adding to a response ...


22

The normal practice is to structure your database with uniqueidentifier keys (sometimes called UUIDs or GUIDs). You can create them in two places without realistic fear of collision. Next, your Mobile app needs to sync "fact" tables from the server before you can create new rows. When you create new rows, you do it locally, and when you sync again, new ...


22

There are many factors that go into the decision of creating a service layer. I have created service layers in the past for the following reasons. Code that needs to be re-used by multiple clients. Third party libraries that we have limited licenses for. Third parties that need an integration point into our system. Centralizing duplicated business logic. ...


22

This sounds like a case of authentication versus authorization. JWTs are cryptographically signed claims about the originator of a request. A JWT might contain claims like "This request is for user X" and "User X has an administrator roles". Obtaining and providing this proof through passwords, signatures, and TLS is the domain of authentication - proving ...


21

The Robustness Principle--specifically, the "be liberal in what you accept" half of it--is a very bad idea in software. It was originally developed in the context of hardware, where physical constraints make engineering tolerances very important, but in software, when someone sends you malformed or otherwise improper input, you have two choices. You can ...


20

In general, RPC offers far more of a language integration than REST. As you mentioned, this comes with a number of problems in terms of scale, error handling, type safety, etc., especially when a single distributed system involves multiple hosts running code written in multiple languages. However, after having written business systems that use RPC, REST, and ...


18

It depends on how many fields you're talking about, and how they are used. Concrete is preferable for highly structured queries with only a few fields, but if the querying tends to be very free form, then the concrete approach quickly becomes unwieldy with more than three or four fields. On the other hand, it's very difficult to keep a generic API pure. ...


Only top voted, non community-wiki answers of a minimum length are eligible