Is it common to prototype in a higher level language? [closed]

Question

I'm currently toying with the idea of embarking on a project that far exceeds my current programming ability in a language I have very little real world experience in (C). Would it be valuable to prototype in a higher level language that I'm more familiar with (like Perl/Python/Ruby/C#) just so I can get the overall design going?

Ultimately, the final product is performance sensitive (it's a database engine), hence the choice of C, but I'm afraid not knowing C well will make me lose the forest for the trees.

While searching for similar questions, I noticed one fellow mention that programmers used to prototype in Prolog, then crank it out in assembler.

Answer 1

Using C doesn't automatically make your application faster. When you have the ability to choose a different programming language for your platform I highly recommend it.

As Bill Harlan stated:

It is easier to optimize correct code than to correct optimized code. Premature optimization actually hinders optimization in the long run. Unnecessary optimization distorts designs, destroys modularity and information-hiding, and makes code much harder to modify. Latent bugs take longer to find. We often discover by profiling, or by changing machines or compilers, that we misjudged the computational effort of our code. Guess what? Now, optimization is much harder than it had to be.

If you can really predict performance issues, consider using C++. cprogramming.com words it very nicely:

You might wonder, however, whether it's worth giving up the reusability of C++ to get the small increase in performance with C, especially when C++ can, where necessary, be written in a C programming style.

---

To better answer your actual question: I would write code in a higher level language, not just prototype it, and only optimize in lower level languages when you encounter performance issues.

Answer 2

It would not be valuable to do that, because a) the parts of those languages that translate directly to C would not be any simpler, and b) the parts that don't translate directly to C would be more difficult to rewrite in C than if you'd written them in C in the first place.

Answer 3

This isn't a question with a categorical yes or no answer. Allow me to weigh in with an anecdote.

Example 1

I was tasked with porting a game written in Java into Flash, AS3. On the surface, this has the potential to go relatively smoothly. After all, you could consider such a job to be more clear-cut than your average client work, because you've already got a completely built-out functional spec in the form of the source game. Java and AS 3 are both high level languages, and AS3 shares many traits in common with Java, such as package structures, single-inheritance/multiple-interface, (opt-in) strong typing, and notions of public/protected/private variable and function declarations. At the time I was still very green at Java, and completely new to the original source codebase. So I just dove in to see what I could find hoping that it would be quick and easy.

As it turned out, the author of the code had attempted to create an abstract engine that could be ported away from Java into some undefined other environment. This gave me hope that it would be straightforward to port. Unfortunately, what I discovered instead was that I was looking at the prospect of re-inventing Flash itself on top of Flash, without the benefit of Threads. A literal port, as it turned out, would simply have been a bad idea... a performance nightmare. On top of that the game implemented its own custom external scripting language, which would have meant creating a parser and lexer for that language if I were hoping to use all the original source data files.

In the end, given the time and budget constraints, the original source code didn't really help very much. The most helpful part about having it was that I knew how to precisely mimic the control flow of the game's logic... but did I really need the original source for that? Probably not.

But that example may not be as relevant because it's sort of the reverse of your situation. I was hoping to use a codebase I did not write, in a language I did not at the time know to speed development in an environment that I was highly familiar with. So here's a different example

Example 2

Noting what the Java developer was doing in trying to create a portable codebase, I set about doing something similar for myself in my Flash work... writing code that didn't rely so much on extending flash.display.* classes for example, and using composition to create views. Not relying so heavily on flash.event.* and instead writing a lightweight message passing system of my own, not tied particularly to a language or platform. I recently finished a game using said framework and wanted to see if it would be easy to port it to C# (A language that I know in as much as it's similar to Java and AS3) as a Unity 3D project. As it turns out, this was much more successful! Because I think more fluently in AS3 than in C#, having the algorithms already written saved a ton of time. All I had to do was simply change the syntax, which isn't difficult when the languages involved are so similar in the first place.

So, just in my own personal experience, I can't say the answer is always going to be yes or no. You should factor in just how dependent upon particular language idioms you happen to be in your high-level language of choice, and whether re-creating those will be easy or difficult in C.

Answer 4

I think it would be valuable to start with pseudocode. Writing a prototype in another language seems like a potential waste of time, since your higher level language isn't likely to translate to 'C' nearly as well as pseudocode will.

Also, by using pseudocode, you'll develop a better understanding of how your system actually works.

During the same time period that you're working on pseudocode, you should be studying C. Then, by the time you're done with your planning, you might be ready to actually implement the thing.

Since your proposed reason for writing it in another language was to help get the design going, you may instead want to use some UML diagrams or something of that nature to get you started.

Answer 5

You can prototype the algorithm - iron out the design errors of the core logic, using definitely "very high level" language (say, Matlab, maybe Ruby). Use it to prove your algorithm works, and works correctly, then implement it from scratch in a "low-level" language.

You won't gain much if you pick C++ or even Java or C# as the "high level" and C as "low level" because the gain in readability will not be significant and "translation" will be still pretty painful and quite bug-prone. The idea is the essence, the engine of your project in high-level implementation should not occupy more than a screen or two, be easy to grasp, easy to read, and all caveats to be painfuly obvious - essentially, a working, runnable block diagram.

Answer 6

A database engine is mostly about handling low level I/O in an optimal way and handling complex structures like b-tree and linked lists efficiently.

So its definitely a C/C++ problem, althought there are some pretty good Java implementations out there.

Developing correct algorithms which perform well is much easier in a higher level language. Its usually a case of trying several variations and comparing the results. You could then translate the "winning" algorithm to C.

A compromise solution could be to write the initial implementation in one of the higher level JVM languages (Jython, Groovy come to mind) and then move class by class to Java when the implementation stabilizes.

Answer 7

I don't think its common, but it is done. One of the sharpest architects that I ever worked with used to do his modeling in Python and then implement that code in C++.

In general to make this worthwhile I think you really have to be performing complex and highly optimizable algorithms that aren't easily expressed in a straightforward manner in the target language. For most "real world/business" situations, its relatively easy to express the high level intent in the same language that we are targeting, and an implementation in said language meets our performance requirements so there is no need/desire to model in a higher level language.

Given your situation, where you have better knowledge of a higher level language, I could see this methodology working out well in the short term. Not only will it provide you with a roadmap to keep things on track, but if you have questions you will be able to ask with greater precision.

Answer 8

Currently I am working on a project that is written in C "because of performance" (this was the original motivation), but indeed if profiled it reveals that it spends most of its time waiting for other systems (a DB, other apps written in Java, "events" on a socket).

If you use the wrong algorithm you obtain bad performance in C too (e.g. if you do a linear search for a key, "since C has not hash tables and we don't want to use other libraries", you go slower than if you do it with a language that has hash tables or similar like C++, Java, C#, Python... and so on).

If you are forced to do it in C for whatever reason, then prototyping in other language you know is to me not so bad idea only if you prototype knowing which "problems" you'll have doing the actual C implementation, that is hard if you are not confident with C. (You will soon discover e.g. that C/C std libs have no containers, just "plain" array; you need non-std libraries). Moreover C is not OO, so if you are prototyping in a OO fashion, it will be harder.

Summarizing, the best thing to do is to do the actual implementation in your "prototyping" language, and then, iff really needed, write CPU-intensive functions in C, but if only C is acceptable, learn it better before doing prototype in other languages and of course before writing the implementation.

Answer 9

There are many performance critical applications written in a higher level language.

I have programmed in Assembler and C in the past, and while it is kinda cool feeling so close to the metal, their use is very limited nowadays.

There are so many things that will hinder performance, that I doubt you will ever reach the part where the language itself is the limiting factor. This is considering it is C vs C#.

Say you get 10%-15% performance increase by the language. This is nothing compared to the orders of magnitude increase in getting the correct algorithm implemented.

When you are programming in C#, you will have much more time to concentrate on architecture and implementation of algorithms/data structures thus leading to better higher-level optimizations.

In the real world you are always time constrained, so spend your time on the right part of the project.

Answer 10

I'm curious as to what your plan is to actually create it in C? Are you going to prototype and then learn C, and then re-code it in C? To me, this seems a bit like the proverbial "eyes being bigger than the stomach" which I think a lot of programmers get caught in while learning new technologies (I know I have). What I mean is that you are trying to design something that is clearly performance sensitive without even yet knowing the ins and outs of the language that you feel it eventually needs to be written in, basically you want to already start designing a C app before you know C when time may be better spent first learning C and then you may gain more insight into how to write the application you want. Perhaps I mistook the question and you intend to hand this off to another programmer to build the program in C, in which case this may be a good idea, at least better than a complete non-programmer trying to design an app :)

Answer 11

Prototyping is done, sometimes, to gain an understanding into the problem you're trying to solve. And sometimes, to get to know the underlying technologies if you are not already familiar with it.

For the case mentioned, you are considering to prototype in a scripting language, say, python and to make the actual code in C.

Evaluating some possibilities:

1. You prototype in python and write the software in C.

Prototyping in a scripting language can help where you quickly want to check output against input. This is useful if you primarily need to test the your logic for solving a problem. Also, useful if you want to quickly put together a demo for other people.

Whatever code you wrote in python will not be used in final software. But, it can aid if you are passing on your prototype to someone who can read python and write in C. Here, prototyping can help communicate an idea.

This method is appropriate for testing logical feasibility of the solution.

2. You prototype in C and write the software in C.

Prototyping in C, which is new to you, has two advantages. One, while you write the prototype, you get to understand the relevant parts of the language, library, API, pitfalls, etc. Two, while you build the final software, you can start from the prototype itself which saves you time and reuses code.

This method fits for testing both, the logical and technological feasibility of the solution.

3. You can consider non-coding ways to prototype depending on the problem at hand.

If it some piece of logic and ideas you want to prototype; pseudo code, flowcharts and block diagrams on paper is good too.

If it is a UI prototype, consider some UI mock-up tool or again, some paper.

Answer 12

I think you should prototype in a language you are familiar with (Pytho/Ruby/C# what not) so that:

1. You take best advantage of the facilities/libraries the language provides.
2. You spend your time deciding on design choices instead of language limitations.

Later, you can use a profiling tool to find areas of bottle neck. Re-Implement in C/C++. Repeat above step a few times, who knows your prototype might be 'fast enough'!

Answer 13

I don't think you're gaining anything by the approach you described, and several people have described why in some detail.

One project that I have been involved with, used this sort of approach: math library development for the Cell/BE and Power7 architectures. The functions were modelled in Haskell (using CoCoNUT), and the output functions were in optimized assembly for a particular target architecture.

In this case the goals were high performance with tuned assembly instructions and the ability to target multiple architectures.

Food for though, I hope you don't starve :)

Answer 14

For a high performance database engine you will presumably need:

• multi-threading,
• explicit memory management,
• asynchronous notifications,
• semaphores or synchronization primitives,
• easy access to low level file system functions.

The algorithms you choose are critical to performance.

The general advice is to start with a high level language, then migrate only the bits that need optimizing into a lower level language.

However, the high level language you choose has to be able to support the algorithms you need to write: and efficient algorithms here may ultimately be dominated by control of threading, efficient use of memory, and the use of the best low level file systems operations available. So if the end goal is performance, you can't prototype in languages that don't support the primitives you need to use.

If you need to test your prototype (or other people need to develop software against its interfaces), you also need to work in a language which supports the intended APIs. You can then allow other people to test their code and carry out your own regression tests while optimizing.

These considerations probably rule out many languages for high level prototyping in this case - and probably all those you mentioned (except possibly C#). But you can, of course, pseudo-code in any language (including English) and, if need be, you can prototype parts of the project (sort functions, for example) in your preferred language.

The close relationship between C++ and C (and negligible performance differences) mean that there are very few reasons not to prefer C++ over C in the final product.

(I'm answering on the assumption that you need a high performance database engine for a particular purpose: if your intentions are more modest then presumably you would be picking up an existing engine off the shelf).

Answer 15

I think C's fame is well deserved, because the magnificent product Unix was written in C. However, compared to people who know C best, I am quite sceptical about why it should be used. It is said that Ken Thompson (after writing a first version of Unix in assembly language) began to write Unix in Fortran, but gave up after a week or a month, and began to use C which was being developed by his colleague Ken Ritchie at the same time.

I was astounded to read recently that Fortran is faster than C and C++.

Richard Mullins