So. The reason (almost) all files have approximately the same number of zeroes is because most unique states found within a set of bits belong to the ratios that are the closest to being equal.
It is interesting to note that in the separation of an area by fence, the largest area is attainable only when all four sides of the fence have the same length. In other words if the fence is rectangular, given 4 meters of fencing, the largest area within the fence is achieved if all sides have 1 meters of fencing. All other distribution of fencing have inferior area.
Air molecules always want to be as far away from each other as possible, distributing themselves at equal distance.
Heat as well distributes itself whereby all matter contains the same amount of heat.
These three are interesting parallels with the ratio distribution in part 1. Naturally, everything is heading toward equal distribution.
So. Information theory.
To describe information theory we must talk about File compression and decompression algorithms.
File compression algorithm is to change the value of the bits within the file/bit-set, and to reduce the number of bits that are used to represent it. Therefore the File compression algorithm alters the value of bits and reduces the number of bits required to store the file.
Now, a software, graphic, word document requires specific structure within the file for it to interpret the data correctly. Compression results in that structure being destroyed, therefore the software application cannot be started in a compressed form, the graphics cannot be displayed in the compressed form and the word document cannot be displayed in the compressed form.
A decompression algorithm recovers the original value of the bits and also the original size of the file/bit-set, (which is another way of saying, it recovers the file completely). The file be that a software, graphics, word document, can now be interpreted by its respective application, but the size has grown back to its original size.
This is a short summary of file compression/decompression algorithms.
As you may guess, the file decompression algorithm is the mirror opposite of the file compression algorithm, it simply undoes what the file compression algorithm has done.
Now. Each File compression algorithm has a File decompression algorithm associated with it. So, I will keep it simple and unless it is explicitly necessary I will be talking about both under the term: File compression algorithm.
So, whats the concept behind file compression algorithm?
The concept requires two different definitions of space.
The first simply, is the number of bits we are talking about (and therefore the total number of unique states within that bit set).
The second is a specific percentage of the states within that bit-set.
#1. A bit-set of 8 bits.
Now then, the number of unique states within a bit-set is given by: 2^S where 'S' is the number of bits.
Given the size of our bit-set, 8 the total number of unique states within that bit-set is given by 2^8.
2^8 = 256.
Fun fact, go back to part one, count the total number of unique states across 0-8 ones in the bit-set of 8 bits. The number you will end up with is 256.
So, back to file compression.
Definition #1 given as 8 bits. From this we can deduce that the space in question has 256 unique states.
Definition #2 given as 50%. In combination with the above definition, from this we can deduce that the space in question has 128 unique states.
128 is half of 256, it is also the total number of unique states within a bit-set of 7 bits. Simply put, each extra bit added to a bit-set doubles the total unique states within that bit-set.
From this it follows that each bit removed from a bit-set halves the total number of unique states within the bit-set.
So. We have two definition of spaces, the combination of these two definition shall be called a space complex.
We will need two space complexes to describe File compression.
They are represented in function form.
SC(8,50) and SC(7,100).
So what can we do, given these parameters. We can take all the possibilities, addressed by the space on the right and place them into the space on the left. The compression algorithm has to keep a record as to which states are placed inside the smaller space.
Mundane, isn't it?
But, we saved a single bit from being written onto the hard drive and we took a 50% chance of crashing our file compression software. The thing is, if you write this file compression algorithm and just throw any random 8 bit file at it, there is a 50% chance that the file will contain one of the 128 unique states that are represented by the compression algorithm and therefore can be compressed, but there is also a 128 unique states that are not represented by the compression algorithm.
It is a law of file compression that, for it to successfully compress a file, it can only do so given that the file is in one of the small percentage of unique states, the file compression algorithm was designed to represent and therefore compress. What happens when this isn't the case varies from file compression algorithm to file compression algorithm. Some immediately terminate themselves, some try and run the risk of termination simply because they broke the fundamental rules which they must function in. Some display a notification,saying the file cannot be compressed, from our perspective these three outcomes are identical. Other file compression algorithm lose the actual value of the bit and others again provide a compressed file, larger than the original file, whereby the decompression algorithm, as its description above suggests, recovers the original state of the bits and the original size of the bit-set. Given a failed compression, where the compressed file is larger than the original, the decompressing fulfills the role of the compressor. The file will never be smaller than in its original state, but during decompression it will at least shrink from the bloated, compressed file to its original state.
There are two categories a file compression algorithm is measured by. Infinite/finite and lossy/lossless.
Can it compress infinitely and is it guaranteed to recover the original file, both in length and value of its bits (zero or one). It is impossible to write an algorithm that fulfills both conditions. J-PEG or JPEG is a file format, also a compression algorithm that damages the file in the process of compressing it, this is unrecoverable damage. This damage has (very) minimal effect on picture quality relative to the amount of bits saved by it. JPEG can also be scaled as to how much it should compress an image, in other words, how much damage to the quality of the image is tolerable and therefore how small can the file size get. In this case damage to the file, that is a lack of integrity in recovering the bits is a feature, not a bug.
JPEG is an example of an infinite and lossy file compression algorithm.
Next to JPEG is PNG file format. Also used to represent image data (images). Also supports compression. Unlike JPEG, PNG's file compression in not-lossy. PNG also supports the alpha channel, which does not deal with what the color of a pixel is, instead to what degree is this pixel transparent.
A file compression algorithm, based on the image above is easy to design, but that is not why it is used here, it is used here to describe the fundamental concept of file compression. Almost all file compression algorithms you will find are more complex in implementation and explanation then the example above. Nevertheless, all of them come back to the concept described above. Select a subset of a "large" space, find a smaller space in which all of the states within the selected subset, will fit into. Write a file compression algorithm, which for one, maps all states selected from the large space, to the "small" space. You are done.