Well, you could look it up in Wikipedia... But since you want an explanation, I'll do my best here:
They provide a mapping between an arbitrary length input, and a (usually) fixed length (or smaller length) output. It can be anything from a simple crc32, to a full blown cryptographic hash function such as MD5 or SHA1/2/256/512. The point is that there's a one-way mapping going on. It's always a many:1 mapping (meaning there will always be collisions) since every function produces a smaller output than it's capable of inputting (If you feed every possible 1mb file into MD5, you'll get a ton of collisions).
The reason they are hard (or impossible in practicality) to reverse is because of how they work internally. Most cryptographic hash functions iterate over the input set many times to produce the output. So if we look at each fixed length chunk of input (which is algorithm dependent), the hash function will call that the current state. It will then iterate over the state and change it to a new one and use that as feedback into itself (MD5 does this 64 times for each 512bit chunk of data). It then somehow combines the resultant states from all these iterations back together to form the resultant hash.
Now, if you wanted to decode the hash, you'd first need to figure out how to split the given hash into its iterated states (1 possibility for inputs smaller than the size of a chunk of data, many for larger inputs). Then you'd need to reverse the iteration for each state. Now, to explain why this is VERY hard, imagine trying to deduce a and b from the following formula: 10 = a + b. There are 10 positive combinations of a and b that can work. Now loop over that a bunch of times: tmp = a + b; a = b; b = tmp. For 64 iterations, you'd have over 10^64 possibilities to try. And that's just a simple addition where some state is preserved from iteration to iteration. Real hash functions do a lot more than 1 operation (MD5 does about 15 operations on 4 state variables). And since the next iteration depends on the state of the previous and the previous is destroyed in creating the current state, it's all but impossible to determine the input state that led to a given output state (for each iteration no less). Combine that, with the large number of possibilities involved, and decoding even an MD5 will take a near infinite (but not infinite) amount of resources. So many resources that it's actually significantly cheaper to brute-force the hash if you have an idea of the size of the input (for smaller inputs) than it is to even try to decode the hash.
They provide a 1:1 mapping between an arbitrary length input and output. And they are always reversible. The important thing to note is that it's reversible using some method. And it's always 1:1 for a given key. Now, there are multiple input:key pairs that might generate the same output (in fact there usually are, depending on the encryption function). Good encrypted data is indistinguishable from random noise. This is different from a good hash output which is always of a consistent format.
Use a hash function when you want to compare a value but can't store the plain representation (for any number of reasons). Passwords should fit this use-case very well since you don't want to store them plain-text for security reasons (and shouldn't). But what if you wanted to check a filesystem for pirated music files? It would be impractical to store 3 mb per music file. So instead, take the hash of the file, and store that (md5 would store 16 bytes instead of 3mb). That way, you just hash each file and compare to the stored database of hashes (This doesn't work as well in practice because of re-encoding, changing file headers, etc, but it's an example use-case).
Use a hash function when you're checking validity of input data. That's what they are designed for. If you have 2 pieces of input, and want to check to see if they are the same, run both through a hash function. The probability of a collision is astronomically low for small input sizes (assuming a good hash function). That's why it's recommended for passwords. For passwords up to 32 characters, md5 has 4 times the output space. SHA1 has 6 times the output space (approximately). SHA512 has about 16 times the output space. You don't really care what the password was, you care if it's the same as the one that was stored. That's why you should use hashes for passwords.
Use encryption whenever you need to get the input data back out. Notice the word need. If you're storing credit card numbers, you need to get them back out at some point, but don't want to store them plain text. So instead, store the encrypted version and keep the key as safe as possible.
Hash functions are also great for signing data. For example, if you're using HMAC, you sign a piece of data by taking a hash of the data concatenated with a known but not transmitted value (a secret value). So, you send the plain-text and the HMAC hash. Then, the receiver simply hashes the submitted data with the known value and checks to see if it matches the transmitted HMAC. If it's the same, you know it wasn't tampered with by a party without the secret value. This is commonly used in secure cookie systems by HTTP frameworks, as well as in message transmission of data over HTTP where you want some assurance of integrity in the data.
A key feature of cryptographic hash functions is that they should be very fast to create, and very difficult/slow to reverse (so much so that it's practically impossible). This poses a problem with passwords. If you store sha512(password), you're not doing a thing to guard against rainbow tables or brute force attacks. Remember, the hash function was designed for speed. So it's trivial for an attacker to just run a dictionary through the hash function and test each result.
Adding a salt helps matters since it adds a bit of unknown data to the hash. So instead of finding anything that matches md5(foo), they need to find something that when added to the known salt produces md5(foo.salt) (which is very much harder to do). But it still doesn't solve the speed problem since if they know the salt it's just a matter of running the dictionary through.
So, there are ways of dealing with this. One popular method is called key strengthening (or key stretching). Basically, you iterate over a hash many times (thousands usually). This does two things. First, it slows down the runtime of the hashing algorithm significantly. Second, if implemented right (passing the input and salt back in on each iteration) actually increases the entropy (available space) for the output, reducing the chances of collisions. A trivial implementation is:
There are other, more standard implementations such as PBKDF2, BCrypt. But this technique is used by quite a few security related systems (such as PGP, WPA, Apache and OpenSSL).
The bottom line, hash(password) is not good enough. hash(password + salt) is better, but still not good enough... Use a stretched hash mechanism to produce your password hashes...
Do not under any circumstances feed the output of one hash directly back into the hash function:
The reason for this has to do with collisions. Remember that all hash functions have collisions because the possible output space (the number of possible outputs) is smaller than then input space. To see why, let's look at what happens. To preface this, let's make the assumption that there's a 0.001% chance of collision from sha1() (it's much lower in reality, but for demonstration purposes).
Now, hash1 has a probability of collision of 0.001%. But when we do the next hash2 = sha1(hash1);, all collisions of hash1 automatically become collisions of hash2. So now, we have hash1's rate at 0.001%, and the 2nd sha1() call adds to that. So now, hash2 has a probability of collision of 0.002%. That's twice as many chances! Each iteration will add another 0.001% chance of collision to the result. So, with 1000 iterations, the chance of collision jumped from a trivial 0.001% to 1%. Now, the degradation is linear, and the real probabilities are far smaller, but the effect is the same (an estimation of the chance of a single collision with md5 is about 1/(2128) or 1/(3x1038). While that seems small, thanks to the birthday attack it's not really as small as it seems).
Instead, by re-appending the salt and password each time, you're re-introducing data back into the hash function. So any collisions of any particular round are no longer collisions of the next round. So:
Has the same chance of collision as the native sha512 function. Which is what you want. Use that instead.
- Hong Kong is number one in Asia for enterprise encryption, with customer personal information the top data protection priority, reports nCipher... - May 27th, 2020
- Are social giants morally obligated to break encryption? - ACS - May 27th, 2020
- Facebook plot to encrypt ALL chats will help child abusers to hide, former police chief warns - The Sun - May 27th, 2020
- Encryption Software Market To Expand At A Robust 14.27% Cagr Of 2020 | Sophos,McAfee,Check Point Software Technologies,Proofpoint,Trend Micro - 3rd... - May 27th, 2020
- Encryption Software Market Forecast Revised in a New Market Expertz Report as COVID-19 Projected to Hold a Massive Impact on Sales in 2020 | Long-term... - May 27th, 2020
- Global Homomorphic Encryption Market Analysis 2020-2025: by Key Players with Countries, Type, Application and Forecast Till 2025 - Cole of Duty - May 27th, 2020
- COVID-19 Impact ON AES Encryption Software Market: Size, Market Analysis, Application, Growth Drivers, Trends, status and Research Report by 2025 -... - May 27th, 2020
- Cloud Encryption Software Market 2020: Potential growth, attractive valuation make it is a long-term investment | Know the COVID19 Impact | Top... - May 27th, 2020
- Global Encryption Key Management Market 2020 Insights, Key Player's Competition, Trends, Sales, Revenue, Supply, Demand, Growth Analysis and Forecast... - May 27th, 2020
- Starting to look at email security. Looking for guidance - Encryption Methods and Programs - BleepingComputer - May 25th, 2020
- Global Cloud Encryption Technology Market Projected to Reach USD XX.XX billion by 2025- Gemalto, Sophos, Symantec, SkyHigh Networks, Netskope etc. -... - May 25th, 2020
- Impact of Covid-19 on Cloud Encryption Technology Market is Expected to Grow at an active CAGR by Forecast to 2025 | Top Players Gemalto, Sophos,... - May 25th, 2020
- Zoom will seek public feedback on plan for stronger encryption - The Indian Express - May 16th, 2020
- Encryption Software Market Research Report 2020 By Size, Share, Trends, Analysis and Forecast to 2026 - Cole of Duty - May 16th, 2020
- Almost half of organisations have been reported to the ICO for a potential data breach - ResponseSource - May 16th, 2020
- VPN Tunnels explained: what are they and how can they keep your internet data secure - TechRadar - May 16th, 2020
- The Week in Ransomware - May 15th 2020 - REvil targets Trump - BleepingComputer - May 16th, 2020
- WhatsApp Video Calls Will Soon Support 50: This Is Why 8s The Limit For Your Security - Forbes - May 16th, 2020
- How to Use Encryption for Defense in Depth in Native and Browser Apps - InfoQ.com - May 14th, 2020
- Analyzing Encrypted RDP Connections - Security Boulevard - May 14th, 2020
- Analysis on Impact of COVID-19-Global Cloud Encryption Software Market 2020-2024| Increasing Use of In-built Cloud Encryption Solutions to Boost... - May 14th, 2020
- Vcrypt ransomware brings along a buddy to do the encryption - Naked Security - May 14th, 2020
- Move over Zoom, this encryption company just released the first fully end to end encrypted conferencing solution - Yahoo Finance - May 14th, 2020
- GovCon Expert Chuck Brooks: Three Steps for Protecting Data in the Public and Private Sectors - GovConWire - May 14th, 2020
- What is the difference between Symmetric and Asymmetric Encryption? - TWCN Tech News - May 14th, 2020
- Encryption Key Management Software Market Growth by Top Companies, Trends by Types and Application, Forecast to 2026 - Cole of Duty - May 14th, 2020
- IoT Security Solution For Encryption Market Growth by Top Companies, Trends by Types and Application, Forecast to 2026 - Cole of Duty - May 14th, 2020
- Mobile Encryption Technology Market Growth by Top Companies, Trends by Types and Application, Forecast to 2026 - Cole of Duty - May 14th, 2020
- Data Encryption Service Market Growth by Top Companies, Trends by Types and Application, Forecast to 2026 - Cole of Duty - May 14th, 2020
- Congress May Hand Bill Barr the Keys to Your Online Life - The New Republic - May 14th, 2020
- DataLocker Sentry K300 8GB Encrypted Thumb Drive Review - TweakTown - May 14th, 2020
- Hardware Encryption Technology Market Growth by Top Companies, Trends by Types and Application, Forecast to 2026 - Cole of Duty - May 14th, 2020
- Global Cloud Encryption Software Market SHARE, SIZE 2020| EMERGING RAPIDLY WITH LATEST TRENDS, GROWTH, REVENUE, DEMAND AND FORECAST TO 2026 -... - May 14th, 2020
- Mobile Encryption Market Growth by Top Companies, Trends by Types and Application, Forecast to 2026 - Cole of Duty - May 14th, 2020
- Hardware Based Encryption Market Growth by Top Companies, Trends by Types and Application, Forecast to 2026 - Cole of Duty - May 14th, 2020
- Email Encryption Software Market Incredible Possibilities, Growth With Industry Study, Detailed Analysis And Forecast To 2025 - Bulletin Line - May 14th, 2020
- Google Duo is coming to the web via Chrome; features Family mode, end-to-end encryption - Moneycontrol - May 14th, 2020
- Global trade impact of the Coronavirus Commercial Encryption Software Market Applications and Company's Active in the Industry Science Market Reports... - May 2nd, 2020
- Email Encryption Market Growth Opportunities, Challenges, Key Companies, Drivers and Forecast to 2026 Cole Reports - Cole of Duty - May 2nd, 2020
- U.S. Hardware Encryption Market (2019 to 2026) - by Algorithm & Standard, Architecture and Field-Programmable Gate Array, Product, Application,... - May 2nd, 2020
- Innovative Encryption Algorithm Developed in South Korea - BusinessKorea - May 2nd, 2020
- Online course trains students in the bizarre world of quantum computing - Livescience.com - May 2nd, 2020
- Encryption Software Market Growth Opportunities, Challenges, Key Companies, Drivers and Forecast to 2026 Cole Reports - Cole of Duty - May 2nd, 2020
- COVID19 impact: Global Cloud Encryption Software Market Trends (Constraints, Drivers, Opportunities, Threats, Challenges, recommendations and... - May 2nd, 2020
- Review of the iStorage datAshur Pro2, an encrypted thumbdrive for home and work - Neowin - May 2nd, 2020
- Kanguru expands encrypted flash drive range with new 256GB options - Geeky Gadgets - May 2nd, 2020
- Global Encryption Management Solutions Market Size |Incredible Possibilities and Growth Analysis and Forecast To 2026 | Check Point Software... - May 2nd, 2020
- The COVIDSafe app was just one contact tracing option. These alternatives guarantee more privacy - The Conversation AU - May 2nd, 2020
- Data Encryption Service Market Detailed Analysis of Current Industry Figures With Forecasts Growth by 2026| Microsoft, IBM, OneNeck - News Log Book - May 2nd, 2020
- ACLU, EFF still trying to get documents unsealed in Facebook encryption case - CyberScoop - April 29th, 2020
- Advanced Encryption Standard (AES): What It Is and How It Works - Security Boulevard - April 29th, 2020
- How Let's Encrypt changed the web with free, easy encryption - Fast Company - April 29th, 2020
- Group video calls of up to 100 participants, with encryption and noise cancellation - Explica - April 29th, 2020
- Analysis of COVID-19-Encryption Management Solutions Market 2019-2023 | Rising Demand For Digitalization to Boost Growth | Technavio - Yahoo Finance - April 17th, 2020
- Protecting consumers personal data becomes top reason for encryption, global study involving nCipher Security finds - Cambridge Independent - April 17th, 2020
- Signal: Well be eaten alive by EARN IT Acts anti-encryption wolves - Naked Security - April 17th, 2020
- Coronavirus tracing tech policy 'more significant' than the war on encryption - ZDNet - April 17th, 2020
- How a former NSA scientist grasped the Holy Grail of encryption and changed the paradigm for safely sharing data - SiliconANGLE - April 17th, 2020
- Zoom will let paying customers pick which data center their calls are routed from - The Verge - April 17th, 2020
- Encryption will be broken in the next four to five years - - Enterprise Times - April 17th, 2020
- Global Hardware-based Full Disk Encryption Market 2020 Comprehensive Research, SWOT Analysis, Key Players and Forecast by 2025 - Galus Australis - April 17th, 2020
- Signal Threatens to Leave the US If EARN IT Act Passes - WIRED - April 17th, 2020
- What is homomorphic encryption and how can it help in elections? | Microsoft On The Issues - Microsoft - April 17th, 2020
- Encryption Software Market Booming by Size, Trends, Top Key players and Forecast to 2026 - Science In Me - April 17th, 2020
- Hardware Encryption Technology Market SWOT Analysis by Key Outlook to 2026 | Illumina, Thermo Fisher Scientific, Roche Diagnostics - Cole of Duty - April 17th, 2020
- Bill to protect children online ensnared in encryption fight | TheHill - The Hill - March 13th, 2020
- Child exploitation bill earns strong opposition from encryption advocates - Washington Examiner - March 13th, 2020
- Senators Pretend That EARN IT Act Wouldn't Be Used To Undermine Encryption; They're Wrong - Techdirt - March 13th, 2020
- Patent hints that encrypted displays could appear on future Apple devices - TechSpot - March 13th, 2020
- Senators dispute industry claims that a bill targeting tech's legal shield would prohibit encryption - CNBC - March 11th, 2020
- The EARN IT Act Is a Sneak Attack on Encryption - WIRED - March 11th, 2020
- Krk WiFi vulnerability affected WiFi encryption on over a billion devices - Privacy News Online - March 11th, 2020
- The Benefits of Encryption and the Implications of Creating Backdoors - American Action Forum - March 11th, 2020
- Big Boom in Encryption Key Management Software Market that is Significantly Growing with Top Key Players Netlib Security, Fortanix, Avery Oden, AWS -... - March 11th, 2020
- Mobile Encryption Market to Witness Robust Expansion throughout the Forecast 2020-2026: McAfee(Intel Corporation), Blackberry, T-Systems... - March 11th, 2020
- Email Encryption Market Rising Trends, Technology and Business Outlook 2020 to 2026 - Best Research Reports - March 11th, 2020
- Crypto, Encryption, and the Quest for a Secure Messaging App - Bitcoin News - March 8th, 2020
- Encryption Flaws Leave Millions of Toyota, Kia, and Hyundai Cars Vulnerable to Key Cloning - Gizmodo - March 8th, 2020
- IoT Security Solution for Encryption Market to Boom In Near Future by 2026 Industry Key Players: Cisco Systems, Intel Corporation, IBM Corporation -... - March 8th, 2020
- What are the top-rated encrypted texting apps? - Fox Business - March 8th, 2020