Questions tagged [statistics]
The statistics tag has no usage guidance.
61
questions
-3
votes
2
answers
80
views
Optimal variable-time logging of a real-time data stream
Cross-posted from stats stackexchange
Say I have a logging utility in my application that I use for recording timestamped diagnostic log messages. I want to add tracking of some performance metrics to ...
- 97
1
vote
1
answer
82
views
Ranking results from a Question and Answer game
I have a question and answer trivia game app which randomly picks questions from a database and prompts the user to answer the question correctly. The total number of correct answers, the total number ...
- 121
2
votes
2
answers
128
views
Design a function that indicates significant deviations in response times
I'd need some feedback on how to approach the design of a function that highlights parts of a time series chart. The chart shows the response time of an application, in particular the 90th percentile, ...
- 337
29
votes
5
answers
4k
views
Benchmarking, why discard lowest time?
I've quite frequently seen benchmarks where the tester discarded the highest and the lowest time out of N runs.
Discarding the highest time I understand; it's probably high because of some other ...
- 499
-1
votes
3
answers
559
views
Birthday Paradox, Analytical and Monte Carlo solutions give two systemically slightly different results [closed]
I was doing a Monte Carlo implementation of the Birthday Paradox in Python and I wanted to check if the results where the same as in the analytical implementation of the same problem (As they should ...
0
votes
1
answer
136
views
Weighted correlation coefficient
My problem is to classify input time series signal (128 points, equidistant in time, range 0.0..1.0) in limited amount (say 16) different classes. We have hand-classified 20000 samples and we are ...
- 9
1
vote
0
answers
137
views
What algorithm to count user click/read/turnover for a period of time?
We tried to develop a kinda-real-time statistics module to measure how hot/trend a content is. We know it is hard to achieve high end stuff like youtube/FB/twitter, our goal is simpler :
Count click/...
- 19
2
votes
2
answers
1k
views
How to get a useful measure for latency
I'm writing some web socket (wamp) connectivity tests. basically one end pings, and the other end records it.
let's say I have the latency for each ping. So I'm tracking the min and the max latency ...
- 433
1
vote
1
answer
58
views
Compare two arrays by the number of occurances
I have the following dictionary:
{ "Drama": 8, "Adventure": 8, "Action": 4, "Comedy": 3, "Thriller": 1 }
This dictionary is like a representation of a user's preferences in movies. (in the past he ...
- 121
7
votes
2
answers
853
views
FIFO Min-Max-Heap for Rolling Median
I'm working on a system with hard real-time constraints in c++ and I need a very fast way to calculate the rolling/moving/streaming median of a set of numbers of size N=100 to 300. Normally this size ...
- 81
0
votes
2
answers
304
views
(AI) algorithm to optimize input parameters
Background information
I am building a program that plays checkers as good as possible. It already plays pretty well but the goal is to improve it even more.
This can be done by adding new methods to ...
- 109
3
votes
1
answer
4k
views
How to perform data mining efficiently (in PHP)?
The moment of working on a system that gives you statistics based on some data gathered from the database has arrived in my company.
How do you efficiently gather statistics from a database in such a ...
- 235
1
vote
2
answers
1k
views
Can I use statistical methods to compare benchmark performance figures?
Imagine I am working on a C compiler. I have made some changes to the compiler's code, and am interested to know what effect these changes have on the performance of generated binaries.
I have a ...
- 167
2
votes
1
answer
56
views
How to find most representative event for category?
Description
I've got a series of events (points, objects, letters, what have you) that occur during a labeled time period (category.) Events can happen during multiple categories. I want to find the ...
- 121
5
votes
2
answers
120
views
Estimate if benchmark sampling difference is significative
In order to estimate the fastest function in various cases in a non fully deterministic context, I'm running an experiment calling one or the other at random and recording the duration of the ...
- 1,434
0
votes
0
answers
54
views
Alternative to Wilson Score when I only have the number of ratings and the average rating?
I'm getting a list of items from an external API and each item has a number of ratings and an average rating out of 5.
What is the best way to rank them? Is it statistically sound to just convert an ...
- 285
5
votes
1
answer
301
views
Algorithm for predicting outcome based on parameters
I've never done anything with statistics, so I'm unsure of how to start with this problem that I need to solve for some software that I'm working on.
Basically, there are three parameters
An integer ...
- 153
4
votes
2
answers
162
views
Estimating in groups - How to extract useful statistics / estimates?
As many other projects we have to estimate our jira tasks up front. This is obviously not fun and I've started hacking on a simple webapp to "gameify" this tedious process.
My idea is that an issue ...
- 143
4
votes
1
answer
5k
views
Oscillations detector: how to effectively detect oscillations of a variable?
I monitor a variable in real-time where new value is generated every 1/25 second. Depending on the conditions, this variable may either be stable, i.e. fluctuate a little (+/-1%) around some value (...
- 155
4
votes
2
answers
346
views
Simplifying Some Probabilistic If-Then Spaghetti Code
I've got a fairly substantial code base that I'm trying to simplify. One section in particular deals with probabilistically creating objects. It currently uses hundreds of random number generator ...
0
votes
2
answers
568
views
Advantage of non-deterministic staistical analysis?
I am working with a Java Library used for identifying the language of a given text. It relies on an n-gram analysis of the text to return a set of languages it might be and the "confidence" that it ...
- 131
5
votes
2
answers
149
views
When designing a replacement application, how can I compare its performance to the original, if there are no existing statistics?
I don't believe there are any performance statistics kept for the application I'm being asked to replace, and don't want to ask the individuals doing the work to manually record things like how long ...
3
votes
0
answers
663
views
PHP MySql - Statistical Reporting - Best Approach to query appointment system time utilization
I have built an appointments system for various medical practices. Practice schedule appointments - I am looking to provide analysis on the amount of time that has been utilised by the practice.
I ...
- 1,047
0
votes
1
answer
214
views
Does it make sense to express reliability as a ratio of number of tests and system failures? [closed]
I have seen a reliability of the system expressed like this:
Rel=No of tests executed / Sum of crashes or other critical events
Does it make sense and what actually is the output?
- 523
0
votes
2
answers
297
views
Permutation tests for large sequences
I would like to perform a permutation test on a particularly large data set, i.e. around 4 million entries. Basically, I need to get some number of random permutations of this data set. Usual way to ...
2
votes
3
answers
1k
views
Count unique visitors by group of visited places
I'm facing the problem of counting the unique visitors of groups of places.
Here is the situation:
I have visitors that can visit places. For example, that can be internet users visiting web pages, ...
- 980
0
votes
2
answers
4k
views
Word recognition in a string without spaces or punctuation marks
I have a small C# project that reads a file and gives me an output: a string that does not contain spaces nor any types of punctuation marks. It may also contain a few misspellings.
Ex.
Output: ...
- 111
1
vote
1
answer
607
views
Is it possible to get probabilities from a support vector machine?
From what I understand, SVM's take a discrete number of x and y values from which to learn from, then when given new x values map it to one y value (category). Is it possible to use SVM's or something ...
- 173
0
votes
1
answer
809
views
scipy.stats.percentileofscore() wrong?
i'm a data scientist and python noob, but I seem to have found a bug with scipy.stats.percentileofscore. it may be "by design" but please bear with me.
if you have an array, you will get an upper ...
- 193
0
votes
1
answer
1k
views
How do I mutate a value in a genetic algorithm using Gaussian distribution?
I've been reading 'Introduction to Evolutionary Algorithms'. This method is stated, but not described, and I can't find anything more specific online. p44/45 of 2nd Ed for reference.
adding to the ...
- 113
3
votes
6
answers
829
views
Unit Testing with an Optimization Problem
Suppose I'm making an algorithm that identifies the subject of a picture. It could be anything that a computer doesn't do that well, but I'm not expecting to get the right answer every time - 80% is ...
- 133
1
vote
1
answer
178
views
Even distribution through a chain of resources
I'm working on an algorithm which routes tasks through a chain of distributed resources based on a hash (or random number).
For example, say you have 10 gateways into a service which distribute tasks ...
- 321
3
votes
1
answer
726
views
How to do inference in a card game?
In the game there are 5 players, each with 8 cards in hand. No other card is present.
Using Monte Carlo (MCTS) I run many simulations of the game from the current state to the end of the match. In ...
- 153
0
votes
0
answers
359
views
Algorithm for generating hyperexponential distribution
I need to generate a hyperexponential distribution for my project. I have already implemented a poisson generating algorithm given by Donald Knuth, but I couldn't find an algorithm for generating a ...
4
votes
1
answer
599
views
Algorithm, AI, or intelligent agent suggestions for ingesting poorly-formatted & variable data from different document types
Thanks for looking!
Background
I have been tasked with writing a program to normalize and ingest data from various sources to a common database.
For the sake of simplicity, let's say that the program ...
- 3,315
1
vote
2
answers
2k
views
Statistics collection engine for C++ systems
We have a research project with idea->prototype->statistics development cycle.
Anyway, our final product is a prototype, so the statistics collection suite is
not used persistently. Supposing I have ...
- 113
3
votes
2
answers
166
views
Randomness of wrapped RNG
There are plenty of questions around regarding random number generation, but I couldn't find anything that exactly matched my question. Apologies if I missed one.
Most random number generators in ...
- 2,275
4
votes
4
answers
388
views
Is code that terminates on a random condition guaranteed to terminate?
If I had a code which terminated based on if a random number generator returned a result (as follows), would it be 100% certain that the code would terminate if it was allowed to run forever.
while (...
- 173
4
votes
1
answer
120
views
Statistical Software Quality Control References
I'm looking for references about hypothesis testing in software management. For example, we might wonder whether "crunch time" leads to an increase in defect rate - this is a surprisingly difficult ...
- 227
4
votes
2
answers
2k
views
statistics for checking imported data?
I'm working on a data migration of several hundred nodes from a Drupal 6 to a Drupal 7 site. I've got the data exported to the new site and I want to check it. Harkening back to my statistics classes, ...
- 662
0
votes
3
answers
2k
views
Get system info from C program?
I'm writing a little program in C that I want to use to output some system stats to my HD44780 16x2 character display. The system I'll be working with is a Debian ARM system and, although irrelevant, ...
- 145
11
votes
1
answer
15k
views
Programming language trends
Are there some statistics out there that show the approx. number of users that popular programming languages have and the rate this number increases over time?
- 255
0
votes
2
answers
372
views
why is a naturally ordered set generated at 9 but not 10 digits
code found at: https://softwareengineering.stackexchange.com/q/158606/44933
Presuming that the problem indicated above is due to the test app and not due specifically to method generateRandomDigits()....
- 183
7
votes
2
answers
102
views
Testing applications with statistical features [duplicate]
Possible Duplicate:
How should I test randomness?
Are their some techniques described for testing applications that have as a requirement some statistical or probabilistic features? They may for ...
- 485
0
votes
1
answer
701
views
Impact of Website Redesign on Google Analytics
I just finished a redesign for a website that is currently using Google Analytics. I want to continue to use Google Analytics, but I'm not sure if I should create a new profile for the new site or ...
- 103
13
votes
4
answers
19k
views
Google Analytics with multiple environments
We are planning to use Google Analytics in our organization and I am in charge for setting it up. I was wondering how to deal with multiple environments. We sure do not want to collect data during ...
- 8,731
2
votes
1
answer
539
views
Calculate statistics for web application
I'm planning to add a couple of usage statistics to my web application. Some common things like: registered users, amount of comments/posts, etc. over time (last 24 hours / last week / last month / ...
- 123
13
votes
4
answers
22k
views
Where can I get a diverse set of sample text? [closed]
I am trying to gather statistics on character or word sequences used in the English language for use in a software project.
Where can I can get a large amount (several GB would be nice) of English ...
- 601
3
votes
2
answers
237
views
Google Analytics and customers privacy
I work in a company that sells SaaS web applications for HR and financial issues. Our customers are key accounts to which we guarantee the privacy of their data, for obvious reasons.
We need to make ...
- 131
5
votes
2
answers
969
views
What kind of metrics should I collect for a website/web app
I have a website that allows users to register for accounts, login, and renew an annual membership fee. They can also update their personal profile, their personal profile, look at a list of their ...
- 5,873