Since you asked why C# did it this way, it's best to ask the C# creators. Anders Hejlsberg, the lead architect for C#, answered why they chose not to go with virtual by default (as in Java) in an [interview][1], pertinent snippets are below.

Keep in mind that Java has virtual by default with the [final][2] keyword to mark a method as non-virtual. Still two concepts to learn, but many folks do not know about the final keyword or don't use proactively. C# forces one to use virtual and new/override to consciously make those decisions.

> There are several reasons. One is **performance**. We can observe that as
> people write code in Java, they forget to mark their methods final.
> Therefore, those methods are virtual. Because they're virtual, they
> don't perform as well. There's just performance overhead associated
> with being a virtual method. That's one issue. 
> 
> A more important issue is **versioning**. There are two schools of thought
> about virtual methods. The academic school of thought says,
> "Everything should be virtual, because I might want to override it
> someday." The pragmatic school of thought, which comes from building
> real applications that run in the real world, says, "We've got to be
> real careful about what we make virtual." 
> 
> When we make something virtual in a platform, we're making an awful
> lot of promises about how it evolves in the future. For a non-virtual
> method, we promise that when you call this method, x and y will
> happen. When we publish a virtual method in an API, we not only
> promise that when you call this method, x and y will happen. We also
> promise that when you override this method, we will call it in this
> particular sequence with regard to these other ones and the state will
> be in this and that invariant. 
> 
> Every time you say virtual in an API, you are creating a call back
> hook. As an OS or API framework designer, you've got to be real
> careful about that. You don't want users overriding and hooking at any
> arbitrary point in an API, because you cannot necessarily make those
> promises. And people may not fully understand the promises they are
> making when they make something virtual.

The interview has more discussion about how developers think about class inheritance design, and how that led to their decision.

Now to the following question:

> I'm not able to understand why in the world I'm going to add a method
> in my DerivedClass with same name and same signature as BaseClass and
> define a new behaviour but at the run-time polymorphism, the BaseClass
> method will be invoked! (which is not overriding but logically it
> should be).

This would be when a derived class wants to declare that it does not abide by the contract of the base class, but has a method with the same name. (For anyone who doesn't know the difference between `new` and `override` in C#, see this [MSDN page][3]).

A very practical scenario is this:

 - You created an API, which has a class called `Vehicle`.
 - I started using your API and derived `Vehicle`.
 - Your `Vehicle` class did not have any method `PerformEngineCheck()`.
 - In my `Car` class, I add a method `PerformEngineCheck()`. 
 - You released a new version of your API and added a `PerformEngineCheck()`.
 - I cannot rename my method because my clients are dependent on my API, and it would break them.
 - So when I recompile against your new API, C# warns me of this issue, e.g.

If the base `PerformEngineCheck` was non-virtual:

    app2.cs(15,17): warning CS0108: 'Car.PerformEngineCheck()' hides inherited member 'Vehicle.PerformEngineCheck()'.
    Use the new keyword if hiding was intended.

And if the base `PerformEngineCheck` was virtual:

    app2.cs(15,17): warning CS0114: 'Car.PerformEngineCheck()' hides inherited member 'Vehicle.PerformEngineCheck()'.
    To make the current member override that implementation, add the override keyword. Otherwise add the new keyword.

 - Now, I must explicitly make a decision whether my class is actually extending the base class' contract, or if it is different functionality but happens to be the same name.
 - By making it `new`, I do not break my clients if the functionality of the base method was different from the derived method. Any code that referenced `Vehicle` will not see `Car.PerformEngineCheck()` called, but code that had a reference to `Car` will continue to see the same functionality that I had offered in `PerformEngineCheck()`.

Of course, the errors that the compiler throws also provide a useful tool for the programmers to not unexpectedly make errors (i.e. either override or provide new functionality without realizing so.)

Like Anders said, real world forces us into such issues which, if we were to start from scratch, we would never want to get into.

EDIT: Added an example of where `new` would have to be used for ensuring interface compatibility.


  [1]: http://www.artima.com/intv/nonvirtual.html
  [2]: http://en.wikipedia.org/wiki/Final_%28Java%29
  [3]: http://msdn.microsoft.com/en-us/library/435f1dw2.aspx