Backend development philosophy

Question

I feel kind of lost in this backend development process I am attempting right now. Most of the usual development practices I use while developing client-side applications don't apply here... Let me provide some context.

The debugging process

While developing a client side application (iOS, Java desktop app, or whatever), it is easy to quickly setup the project on your favorite IDE, get it running on your machine or a testing device, and debug the hell out of it as much as you like.

On the other hand, it's not as trivial to hook a debugger to your backend code, and especially if it is python code running on Google App Engine (GAE). That's what I am using, and ... yeah. Linters and all help A LOT, but still, semantic issues cannot be resolved that way, obviously.

I am currently going over my recently written backend code and just burying it with logging.debug('msg') statements, asserts and whatnot. This is the only thing I can think of. Is this normal for backend developers? Does logging and digging through logs usually how backend devs iterate on their applications?

Parallelism and request driven

This might be a bit more specific to GAE and other non-blocking backends, honestly. Single threaded servers don't suffer from this problem... Anyways, so when you are dealing with parallelism and everything is driven by socket events, how do backend developers usually test if their backend works?

I did the most naive thing of all time, which is to open python console and using the requests library just send requests and test bit by bit. Then, I went ahead and wrote a kivy app to help me send the requests from a GUI interface and see what is going on, but it's taking more time to maintain the kivy app than develop the backend!! I tried to check for test frameworks for GAE, but they didn't seem easy to get, so I was wondering if they are worth it? Would I be able to simulate 100s of clients using my backend using test frameworks? What do people use these days (For GAE specifically)?

Visualizing the "flow"

Because of my inexperience with backend development, it is surprisingly hard hard for me to keep a clear image of the request/response cycle in my head. I know the basics, I have written a few backend apps, but as soon as it get just a little complex, I have to keep reminding myself where the request goes through by looking at the entry point, and all the steps till the response is made. I am sure if I were able to somehow visualize it, I don't have to keep going back over and over. Instead, I would easily know where a bug would originate, for example, or where is the best place to add a certain feature.

In any case, I was wondering if there is some sort of standard "thing" to design the request flow. I don't know, maybe a UML diagram or something? I tried to sketch it out, but I ended up with a mess. Like, I would sketch the backend design based on the features and requirements, but then the actual logic and model would be left out. Then, I try to include those in the diagram, and it becomes overly complex and cluttered with many weird arrows and boxes. I need something to backend development, like ER diagrams is for relational database design.

Yeah, Sorry. I talk a lot, and I am lost in this world. Help?

Answer 1

Debugging / TESTING!

• Unit test the heck out of your backend code. It's usually easier than unit testing frontend code, because the code isn't waiting for arbitrary clicks or keypresses. It's all "this data comes in, this data goes out". Aggressive unit testing will speed you up phenomenally because you don't have to go all the way back to the start to validate each little bit of code. And then you'll have those tests around as regression testing.

• There is a TDD (Test-driven development) that says you should write your tests first, and then fill in the code that makes them work. But I can only do that halfway, and claim that I'm using "London style". But if that appeals to you, it is a good practice.

• As a high water mark: in Python, I aim for 100% test coverage & it's pretty easy hitting that.

• I'm one of those guys who almost never uses a debugger. Rather, I use unit tests to make gurantees about behavior and sensible logging. But that's just my preference.

Parallelism

• More important to get the thing working at all IMHO

Visualizing

• I find these diagrams essential for my work: sequence diagram, and (less so) ER diagram and state diagram

• Check out http://websequencediagrams.com . It will save your life on a complicated project, and even on simple ones really help visualize the client-server interactions.

• If you draw a sequence diagram, not only will it show the client-server interactions, but every arrow pointing at your service is an API method you must implement. Pretty handy & very direct.

• Yes, I use ER diagrams for both database design and class design when the data reaches a certain complexity.

• Sometimes a state diagram can be very useful.

My opinion is that backend development is easier than frontend devleopment because I see my job as "moving bytes from here to there". But that may be just because I'm really bad at frontend development.