There are several factors to take in account. To illustrate those points, I'll use an example of a field where a user should enter a percentage in a context of a quota defined for a specific task in terms of how much disk space the task could use. 0% means the task wouldn't be able to write anything to disk; 100% means the task could fill all the disk space. Values in between mean what they mean.
As a developer, you're probably considering that the acceptable values are [0, 1, 2, 3, ⋯ 99, 100], and everything else is silly. Let's see why users could still be entering those “silly” values.
The user was entering the value 56, but mistakenly pressed Shift while entering them (for instance because on French keyboard, you have to press Shift to enter digits, and the user was constantly switching between a French keyboard and a QWERTY).
In the same way, you can get a number, with something after or before it, or in between:
Here, the user was probably entering the digits, followed by a tab to move to the next field. Instead of pressing ⇆ , the user pressed the neighbor key.
Misunderstandings and misinterpretations
An empty input is probably the most usual. The user imagined that the field was optional, or didn't know what to put in this field.
The user thought that floating point values were acceptable. Either the user is wrong, and the application should politely explain why only integer values are accepted, or the initial requirements were wrong, and it makes sense to let users enter floating point values.
The user misunderstood that when asked for the space the task could take, the app expected a number. This could indicate a poor user interface. For instance, asking the user “How much disk space should the task take?” invites to this sort of input, while a field with a percent sign following would receive less of that sort of input, because “none %” doesn't make much sense.
The user misunderstood what the percentage means in this case. Maybe the user wanted to tell that the task can take 150% of the currently used space, so if on a disk of 2 TB, 100 GB are used, the task could use 150 GB. Again, a better user interface could help. For instance, instead of having a bare input field with a percent sign appended to it, one could have this:
[____] % of disk space (2 TB)
When the user starts typing, it would change the text on the fly to become this:
[5___] % of disk space (102.4 GB of 2 TB)
Large numbers or numbers with a floating point can be represented differently. For instance, a number 1234.56 could be written like that:
1,234.56. Depending on the culture, the text representation of the same number would differ. In French, the same number will be written like this:
1 234,56. See, a comma where you wouldn't expect one, and a space.
Always expecting a specific format using a specific locale would get you in trouble sooner or later, because users from different countries would have different habits of writing numbers, dates and time, etc.
Humans vs. computers
Ordinary humans don't think the same way as computers. “Twenty-four” is an actual number, independently of what a PC would tell you.
Although (1) most systems don't handle at all this type of input and (2) nearly every user wouldn't imagine entering a number written in full letters, it doesn't mean that such input is silly. In About Face 3, Alan Cooper makes a point that not handling such inputs is indicative of the inability of computers to adapt to humans, and ideally, the interface should be able to handle those inputs correctly.
The only thing I have to add to Alan Cooper's book is that in many cases, numbers are written in digits by mistake. The fact that computers expect their users to make mistakes (and won't tolerate a user who writes correctly) is annoying.
Unicode reserves its own surprises: characters which could look the same are not the same. Not convinced? Copy-paste
"5𝟨" === "56" to the developer tools of your browser, and press Enter.
The reason that those string are not equal is that the Unicode character
𝟨 is not the same as the character
6. This would create a situation where an angry customer would call, telling that your app is not working, providing a screenshot of an input which looks legit, and your app claiming that the input is invalid.
Why would anyone enter a Unicode character which looks like a digit, you would ask? While I wouldn't expect a user entering one unintentionally, a copy-paste from a different source could cause that, and I had a case where the user actually did such copy-paste of a string which contained an Unicode character which wouldn't appear on screen.
Those are the cases you get for an elementary number input field. I would let you imagine what you can have to handle for more complex forms, such as a date, or an address.
My answer is focused on what you called “silly” input. Testing is not about checking the happy paths; it's also about checking that you app doesn't break when a malicious user is intentionally entering weird things, trying to break it. This means that when you're asking for a percentage, you also have to test what happens when the user is responding with a string containing 1,000,000 characters, or a negative number, or a bobby table.