Questions tagged [hashing]

A hash function is any algorithm that maps data of arbitrary length to data of a fixed length. The values returned by a hash function are called hash values, hash codes, hash sums, checksums or simply hashes. From Wikipedia: http://en.wikipedia.org/wiki/Hash_function

Filter by
Sorted by
Tagged with
-2
votes
0answers
20 views

Concept Check: Data Server Nodes, Fetch Nodes

I had this idea for a way to make it possible to run a MUCH MUCH smaller node but still mine on a chain. I don't know if there is anything useful in it, but decided I would post it here for feedback ...
2
votes
2answers
87 views

Modulo Division of Hash Output

In a design book I was reading they describe a method to determine a database sharding scheme by taking the hash (MD5, SHA1, whatever) of a userid (integers or uuids) and then (whether encoded or not) ...
0
votes
0answers
94 views

Designing a unique id reliably based on a String date?

I am creating an Android application that sets alarms based on the String date of a list of Objects (PendingIntents, with each taking in an id, that is an integer). In order to cancel the ...
2
votes
2answers
129 views

Is there a reliable way to get get the fingerprint of a file hosted online, without fully downloading it?

Background Tertiary to this question, I have been building my own imageboard that prevents [for example] duplicate images from being downloaded again and again on behalf of the client. How I do this, ...
3
votes
1answer
576 views

Is consistent hashing required for sharding?

I am reading about scaling of database and came to know about sharding technique. But I also read about consistent hashing technique. So how practically sharding is implemented? Do we arrange nodes in ...
1
vote
1answer
828 views

Is it ok to use redis scan extensively?

In redis docs, it is stated that keys command should not be used in production, since it blocks other processes while executing, it is better to use scan iteration over all keys with some batch size. ...
5
votes
1answer
678 views

Why use strong checksums to detect random errors in a filesystem like btrfs?

Btrfs supports crc32c, xxhash, sha256 and blake2b as checksums when storing and reading files. crc32c and xxhash are designed to detect random errors while sha256 and blake2 are considered ...
-4
votes
4answers
896 views

Time efficient way to count pairs in an array whose sum is divisible by a specific number?

Given an array of integers, we want to find how many explicit pairs can be made such that their sum is divisible by 60. The pairs are not necessarily non-unique. For example, let's say the input into ...
-1
votes
1answer
228 views

Is this no-collision-type hash algorithm safe?

I've designed a hash algorithm based on mathematical exponentiation, exponentiating each number by its following number. To be able to exponentiate indefinitely, numbers are first normalized to a 0-1 ...
-5
votes
6answers
269 views

(closed) Why do we bother hashing data?

I don't understand why we hash data like passwords. I know it's harder to steal hashed data rather than just plain text, and it takes longer, but once one of those public hashing methods gets cracked, ...
-3
votes
2answers
576 views

Which hash algorithm is best for speed and identical hashes. (a lot of collisions)

I'm searching an existing hash function or trying to make a hash function that has a lot of collisions. Regularly a hash is used for it's ability to create unique hashes for hash tables or security ...
0
votes
1answer
37 views

Combine Multiple Minimal Perfect Hashing

I'm building some Minimal Perfect Hash from multiple datasets using the CHD algorithm I would like to know if there exists a "smart" way to combine all the different MPH obtained from ...
1
vote
1answer
115 views

Does the SHA256 hashing algorithm change based on the content encoding?

I am starting to look into how to implement SHA256 in JavaScript, and found this for example. It requires UTF-8 encoding it sounds like. Another one I saw required/supported only ASCII encoding and ...
1
vote
3answers
321 views

hash-like algorithm to identify passwords which are "too similar" to previous ones from history

One common issue with secure passwords is that users tend to "cheat", one common cheating pattern we meet recently is the "password swap" antipattern where the user basically keeps using the same two ...
-2
votes
1answer
57 views

Find circular references of first order (bi-directional referencing)

I have million of objects, each with an array smaller than 10 elements, which are the names of other objects in the dataset. Basically { a:[b,c,d,], b:[c,d,e], c:[a,e,f], ... e:[a,b,c] } will ...
-1
votes
1answer
129 views

How to implement FNV-1(a) in SQLite?

Moved I originally posted this on SoftwareEngineering because that's where this related question was; but having looked into the Help in detail, I think my question is more on-topic for stackoverflow,...
5
votes
3answers
2k views

How to perform consistent hashing on any Python object that works with hash()?

I have a Python library that performs a kind of calculation given a parameter-object. A requirement of the parameter object is that it be both hashable and serializable. It's a long calculation, so it ...
5
votes
1answer
1k views

Using PostgreSQL MD5 hash to calculate a 64 bit hash value for advisory lock functions?

I have a particular problem addressing the PostgreSQL advisory locking functions using the bigint variants. Basically I want to create a 64 bit bigint value from a text type obtained with the ...
140
votes
3answers
22k views

Where do "magic" hashing constants like 0x9e3779b9 and 0x9e3779b1 come from?

In code dealing with hash tables, I often find the constant 0x9e3779b9 or sometimes 0x9e3779b1. For example hash = n * 0x9e3779b1 >>> 24 Why is this particular value used?
1
vote
1answer
267 views

Hashing from within the database or outside?

I am in the process of evolving architecture for an 'equity analysis system'. I will be using SQLServer as the database, but I am not going the .NET route and won't be using the built-in 'membership' ...
0
votes
1answer
179 views

Hashset vs Treeset

I've always loved trees, that nice O(n*log(n)) and the tidiness of them. However, every software engineer I've ever known has asked me pointedly why I would use a TreeSet. From a CS background, I don'...
0
votes
0answers
98 views

Searching/Hashing Wikipedia - at each position for keys 1..31 long, the fastest way?

A C coder here. My wish is to find the best way to search words and phrases within the ever-growing English Wikipedia XML dump. As a first layer/step, hashing is the answer, yes? The problem is that ...
1
vote
1answer
497 views

Client Side Hashing + Server Side Hashing

Suppose one wishes to implement an account system for their service/website. There have been cases with Instagram for example where passwords were accidentally stored in plain text due to ...
-3
votes
2answers
788 views

What are some disadvantages for a hashmap with timestamp as key? [closed]

"What are some problems that can arise from using a HashMap with timestamp as keys"? I, for one, do not see any major issues with this kind of "setup". Is this "setup" a no-go/an anti-pattern/...
0
votes
1answer
42 views

portable hashable string representation of data object

I have a class of (flat) objects that are going to be passed around between three different parties, each possibly running a different software stack. my_object: item: "string_a" money: ...
15
votes
6answers
5k views

How to implement float hashing with approximate equality

Let's say we have the following Python class (the problem exists in Java just the same with equals and hashCode) class Temperature: def __init__(self, degrees): self.degrees = degrees ...
0
votes
1answer
605 views

Distribute jobs evenly among a set of servers

I have the need to distribute a set of long running jobs across a cluster of containers in ECS. These jobs essentially would need to open a socket connection with a remote server and begin streaming ...
2
votes
1answer
709 views

Explanation of how to resolve Hash conflicts in HAMT or hashtables in general

I am working on trying to understand HAMT and now am uncertain generally what to do when you run into a conflict in a hash. All I understand so far is you create a list to append the keys to, but I ...
0
votes
3answers
182 views

Is there a secure way to check previous passwords purely on the client-side?

We have a requirement for a security audit that our password policy must disallow the re-use of a previous password from the last 4 used passwords. We can accomplish this fairly easily by making a ...
10
votes
5answers
1k views

If passwords are stored hashed, how would a computer know that your password is similar to the last one if you try resetting your password?

If passwords are stored hashed, how would a computer know that your password is similar to the last one if you try resetting your password? Wouldn't the two passwords be totally different since one is ...
3
votes
0answers
395 views

LSH for a wordlist

I am trying to write a spellchecker, I have a huge wordlist (at least 500K, because of the nature of the language). The performance would suffer a lot if I got the lavenshtein distance of all of the ...
2
votes
1answer
1k views

Where and how to handle user password hashing in Clean Architecture?

I'm currently building a new application and trying to apply some of the principles of Clean Architecture. One of my first roadblocks is implementing my own Identity system (to avoid being tightly ...
0
votes
1answer
344 views

A collision-free hash-like function for use in hash tables and other data structures?

A short introduction to the problem: I'm working with a small database where I have a table of strings (web URLs, to be precise) as pairs: hash|string. Another table references these strings by hash ...
1
vote
1answer
152 views

Hashing tag list to perform faster search

I have a list of items. Each item have various properties. Each item can be tagged, like tagging a post here. Right now, tags are represented using a Tags property, that is a string and it looks like ...
3
votes
0answers
215 views

Hashing algorithm without the avalanche effect

I know that one of the most important features of hashing algorithms, is that minor changes in the input should reflect great change in the hash itself (the avalanche effect), but I'm currently ...
0
votes
1answer
445 views

Logic Behind a Basic Hash Table Hashing Algorithm

I am trying to write a Hash Table in Java based on some Princeton article about it. The most commonly used method for hashing integers is called modular hashing: we choose the array size M to be ...
5
votes
3answers
6k views

What is an example for a one-way hash function?

I am doing a bit of research on hash functions. I understand the concept that it is an equation that is easy to do one way (you take the number 00011010 for example and do reasonably simple math with ...
-1
votes
1answer
403 views

What is the SLOWEST 32 hashing function without excessive collisions

I am currently using a CRC32 hash to generate a locality sensitive hash for some of my data. For much the same reasons as PKBDF2 makes a better choice than, say, SHA26, for hashing passwords I ...
-1
votes
4answers
1k views

Why can't Dictionary(TKey, TValue) update it's hashcodes?

I've read that an object's hashcode should not change while it is part of a Dictionary. Because if it changed, there would be no way to find it in the Dictionary. Why doesn't the Dictionary class ...
1
vote
1answer
1k views

Reverse engineering a checksum or CRC

I am trying to reverse engineer a checksum or CRC wherein an 8 bit number* gets converted to a 5 bit number for error checking. I have an incomplete list of data values and checksums, and need to ...
0
votes
2answers
79 views

Hashable classes: Should I only keep the id's for results or entire class

Given 2 objects / classes: A class for a cargo ship. It has a small set of attributes, say current location, size, number of crew and some other things. Most important, it has an attribute for the id ...
-3
votes
1answer
305 views

How does Steam resume downloads without computing a partial hash?

The Steam Client application allows one to download games purchased through the associated Steam account. A typical modern game uses quite a bit of disk space (in the order of 50-70 GB), so it is not ...
0
votes
3answers
304 views

What can I infer from Hilbert's map?

How do I understand Hilbert's map in this answer? The author of the answer has mapped the hash value (integer) of some strings to a 2D image. I understand that Hilbert's curve is a technique for 1D &...
1
vote
2answers
864 views

Map/store run time ID for objects

What I want to achieve: I need to obtain a run time (unique) id for "any object" (for classes either in java SE, classes I have created or classes provided by other third parties... any instance of a ...
9
votes
6answers
4k views

Is there an "anti-hash" or "similarity hash" / similarity measure algorithm?

The idea of hashes is they get drastically different results for even the smallest change of data. What I am asking for is the opposite of that. An algorithm that will produce proximity hash values ...
1
vote
0answers
202 views

Data structure for fast bi-directional SHA-1 to SHA-3 mapping

What data structure would be best choice for fast and performant bidirectional SHA-1 to SHA-3 mapping? Or in other words fixed-length string to fixed-length string mapping? To be more precise: ...
0
votes
3answers
696 views

algorithm for perfect substring temporarily replacement

I have an interesting question about string and delimiters. There is a random string str. There are some symbols (OR SEQUENCES OF SYMBOLS) that are predefined and should be temporarily replaced ...
1
vote
2answers
496 views

Why do we need datastructures other than HashMap

Map (or HashMap) takes Constant time for Insert, Remove and Retrieve. While all other data structures which I know so far, do not take constant time and their time for above operations depends on the ...
3
votes
1answer
291 views

When should a general purpose hash table assume hash equality implies logical equality?

For a general purpose hash table that aims for both high performance and correctness, when, if ever, does it make sense to assume that hash equality implies logical equality? To lay some ground rules ...
2
votes
1answer
2k views

If I am using just HashMap can I override only hashCode method?

If I need to use only HashMap, why do I need to override equals() method along with hashCode()? How will not implementing equals() affect the working of HashMap?