1

Topic: To relieve the person from BigData by means of 5 lines of the code

Subject a bit cheerful, on it not in algorithms. Particulars: https://habrahabr.ru/post/324772/it is short. It is necessary for the person   ID-shki in format XX000000, where XX - two letters of the Latin alphabet, 0 - number from 0 to 9. A principal condition - that damned competitors could not ID calculate under number how many all clients in basis. Competitors can do  records on purpose to receive n ID, going successively. For the person the task seemed mega-difficult, what even it was necessary to attract this terrible BigData which neither the friend nor the enemy in the person does not know. I.e. sorted out all possible variants, mixed, saved in one big = on 20  and then long optimized basis quickly to select these ID. How by means of the genius of engineering thought and 5-10 lines on the pseudocode to relieve the person from big and  BigData?

2

Re: To relieve the person from BigData by means of 5 lines of the code

S> As by means of the genius of engineering thought and 5-10 lines on the pseudocode to relieve the person from big and  BigData? Serial numbers + the strong enciphering, not?

3

Re: To relieve the person from BigData by means of 5 lines of the code

Hello, ononim, you wrote: O> serial numbers + the strong enciphering, not? It is necessary to check up still that did not coincide with one of the previous. And if coincided -  and again to check. If almost all logins are occupied are can be long. Hardly faster - to store in an array all already  logins in the arranged type, after enciphering to take a remainder of division on an amount of the free logins, binary search to find, to what of intervals new login got, and to calculate, to that it is equal now. After that it is necessary to interpose it into this arranged array, but it costs as much, how many it is necessary to check up once on coincidence by search. If O (m) it is inadmissible to store  logins in the binary depthbalanced tree with ordinal statistics (an amount of nodes with values less leaking) then all manages in O (log (m)), with a constant of the order of six 32-bit elements on record. It is necessary to be ready that competitors hammer almost in all numbers, that is to hold a file on approximately 16 . Besides, if they hammer at once in all numbers except last they will precisely know to that this is equal last Here only what sense? Then it does not turn out to write to the abstract that worked with BigData.

4

Re: To relieve the person from BigData by means of 5 lines of the code

O>> serial numbers + the strong enciphering, not? The C> Should be checked up still that did not coincide with one of the previous. And if coincided -  and again to check. Enciphering! = hash coding. To check it is necessary nothing. Enough serial increment of the initial data. It is finite if to consider that never comes overfillings and izymanija/rejuzanija previous id-shnikov, in that case it is necessary  to roll down to a DB.

5

Re: To relieve the person from BigData by means of 5 lines of the code

Hello, Shmj, you wrote: S> As by means of the genius of engineering thought and 5-10 lines on the pseudocode to relieve the person from big and  BigData? Literally one of these days read something on a subject, somewhere here. Pseudorandom sequence of Galois, the local user  wrote about it.

6

Re: To relieve the person from BigData by means of 5 lines of the code

Hello, ononim, you wrote: O>>> serial numbers + the strong enciphering, not? The C>> Should be checked up still that did not coincide with one of the previous. And if coincided -  and again to check. O> enciphering! = hash coding. To check it is necessary nothing. Enough serial increment of the initial data. It is finite if to consider that never comes overfillings and izymanija/rejuzanija previous id-shnikov, in that case it is necessary  to roll down to a DB. Well what for a DB. XX000000 26*26*1000000 = 676000000 keys 676000000 / 8 = 84 500 000 byte  . Here at us the counter , good , that an output in a range [0. The counter-1], and one linear pass on  which of an output  receives a real new key. And with enciphering would not communicate, because that you will do, if the key flowed away.

7

Re: To relieve the person from BigData by means of 5 lines of the code

AG> XX000000 AG> 26*26*1000000 = 676000000 keys AG> 676000000 / 8 = 84 500 000 byte  . Well  from my belltower - a DB special case. AG> and with enciphering would not communicate, because that you will do, if the key flowed away. Well probability  a key in this case I would estimate approximately just as probability  convergence which with its help it is possible to filch - that is an amount of clients. So it not a relevant problem. AG> here at us the counter , good , that an output in a range [0. The counter-1], and one linear pass on  which of an output  receives a real new key. And concerning protection against megahackers which it is direct so it would be desirable to know how many at you clients and all on it, I think that firmness of the given system at usage of any  rests in  this most .

8

Re: To relieve the person from BigData by means of 5 lines of the code

9

Re: To relieve the person from BigData by means of 5 lines of the code

Hello, Shmj, you wrote: S> the Subject a bit cheerful, on it not in algorithms. S> particulars: https://habrahabr.ru/post/324772/and generally how this task dares? Why it is impossible simply  this sequentially on the fly, at user registration?

10

Re: To relieve the person from BigData by means of 5 lines of the code

Hello, Shmj, you wrote: to Take GIUD (if who has not enough - two), to cipher casual key (third GIUD th), to take averages n byte, to transform in necessary format, to check up on doubles. All.

11

Re: To relieve the person from BigData by means of 5 lines of the code

Hello, ononim, you wrote: O> Enciphering! = hash coding. To check it is necessary nothing. Enough serial increment of the initial data. It is finite if to consider that never comes overfillings and izymanija/rejuzanija previous id-shnikov, in that case it is necessary  to roll down to a DB. Not absolutely. For example, we have a cipher with 64-bit units. Well, for example, DES. If to feed with its counter, we receive pseudorandom sequence of numbers in a range from 0 to 18446744073709551615 without repetitions. And, for example, it is necessary to us from 0 to 9999999. Hence, DES not very much approaches, too many improper answers it will produce. We note that if to cut off from DES' superfluous bits the sequence remains pseudorandom, but the warranty  is lost. I would take the cipher with long unit N such that 2^N-1 there is less range upper bound to which it is necessary to get, and 2 ^ (N-1) - 1 already it is less. Would feed with its counter and discarded improper values - them no more half because we so selected the size of the unit. A question in where to take the cipher with necessary (arbitrary) long the unit. I think, this question has the answer, but these should be engaged.

12

Re: To relieve the person from BigData by means of 5 lines of the code

Hello, ononim, you wrote: O> Well it is necessary   which of them ciphers with very small unit and thus stands to known plain text attack. It is possible to begin with reading of it. Basically, in this article it is told how to generate a piece of pseudorandom swap from N numbers, needlessly to store in storage all N numbers, and thus for reasonable time. They do from this the block cipher, but basically, article contains that the decision of the task delivered in the beginning of this discussion. Attentively to read article laziness.

13

Re: To relieve the person from BigData by means of 5 lines of the code

Hello, wildwind, you wrote: W> to Take GIUD (if who has not enough - two), to cipher casual key (third GIUD th), to take averages n byte, to transform in necessary format, to check up on doubles. All. At first, this construction with three GUID is better nothing, than simply  the necessary amount of bits from the cryptography generator of random numbers. Actually, even is worse, because it can and not produce all possible variants of output value. Secondly, check on doubles can appear expensive pleasure. Thirdly, when the most part of values is already used, everyone  with very high probability will be a double, and will find  unique value all more difficult and more difficult.

14

Re: To relieve the person from BigData by means of 5 lines of the code

Hello, Shmj, you wrote: S> As by means of the genius of engineering thought and 5-10 lines on the pseudocode to relieve the person from big and  BigData? , and from what pores pity for 700 million became ? I thought that the nursery  begins with 700 billion, data volumes - petabytes, and it still well and optimally packed data. And here, if especially not . Stupidly  from a range and a result addition in an array of Boolean variables not ? Under an array from 700 million Boolean 87 mbytes, volume children's are necessary. If n time  (we admit N = 100) did not turn out plus an increment from  (let still n time) always we get on already generated number, the range not bad enough means we hammered. It happens likely years through 100. In 100 years if suddenly it is required further, we transverse on all array linearly and there where remained empty - we shift indexes already in an array from the whole indexes in the first array. Linearly running of these pity 87 mbytes will occupy fractions of a second, and it is 1 time only it will be necessary to do. Further the array on 97 mbytes is released, it is not so necessary, and simply sequentially we select elements from the remained array.

15

Re: To relieve the person from BigData by means of 5 lines of the code

Hello, Pzz, you wrote: Pzz> At first, this construction with three GUID is better nothing, than simply  the necessary amount of bits from the cryptography generator of random numbers. Actually, even is worse, because it can and not produce all possible variants of output value. At us the task to receive all possible variants is not necessary. Pzz> Secondly, check on doubles can appear expensive pleasure. From what? Pzz> Thirdly when the most part of values is already used, everyone  with very high probability will be a double, and will find  unique value all more difficult and more difficult. If the project lives to this stage, ID it is possible to expand length. It is possible in advance.

16

Re: To relieve the person from BigData by means of 5 lines of the code

Hello, ononim, you wrote: O> Well it is necessary   which of them ciphers with very small unit and thus stands to known plain text attack. It is possible to begin with reading of it. Well and if to try without exotic algorithms, on the simple. I offer such algorithm. We have: 1. Secret key for HMAC SHA1 (without a difference, it is possible and MD5). 2. Numbers of users in basis in sequence, upward we give . We do: 1. Number of the user is represented in a type 16+6=22 bit. It with a store. 2. Conditionally we divide 22 bits into 2 groups on 11 bits. For enciphering: 1. We do HMAC the right part (from the right 11 bits) with secret key. We receive the HASH 1. 1 user cannot receive a hash, as it does not have secret key. 2. We superimpose XOR th turned out HASH 1 on the left 11 bits. 3. We do HMAC the left 11 bits (which already XOR-nutye) with secret key. We receive the HASH 2. Since The user does not have secret key it cannot make reverse conversion. 4. We superimpose XOR th HASH 2 on the right 11 bits. A reverse fresh-water sponge: 1. We do the HASH 2 of the left 11 bits (HMAC with a key). 2. Again XOR th it is superimposed on the right part. 3. We do HMAC the right part, it is a hash 1. 4. We superimpose XOR th on the left part. There will be a probability that after XOR number hardly more than it is required (but less than on 1 bit). With it it is possible to make something (type on the unit value to take), but it would be desirable  another: Where I was pierced?

17

Re: To relieve the person from BigData by means of 5 lines of the code

Hello, Shmj, you wrote: S> it is short. It is necessary for the person   ID-shki in format XX000000, where XX - two letters of the Latin alphabet, 0 - number from 0 to 9. A principal condition - that damned competitors could not ID calculate under number how many all clients in basis. Competitors can do  records on purpose to receive n ID, going successively. The most stupid - to take the list of all ID, to mix them and to save in a file. It will be the order of pair gigabyte (all 676 million variants). More interesting decision - to use "Format Preserving Encryption". Standards on it while are not present, but manually is easily enough implemented "cycle-walking". We do the counter in 32 bits, then we take RC5 with the size of the unit in 32 bits, we cipher value and it is done cycle-walking while it is not located in 676 million (on the average 8 steps, nonsense). A telemarket. As a bonus, reversed - it is possible to take algorithm ciphered ID and to receive original value of the counter.

18

Re: To relieve the person from BigData by means of 5 lines of the code

S> Where I was pierced? To tell the truth to understand laziness, but I know precisely that if you not with a bias in cryptography that better the  not to compose mathematics, and to use that that mathematicians composed.

19

Re: To relieve the person from BigData by means of 5 lines of the code

S>> it is short. It is necessary for the person   ID-shki in format XX000000, where XX - two letters of the Latin alphabet, 0 - number from 0 to 9. A principal condition - that damned competitors could not ID calculate under number how many all clients in basis. Competitors can do  records on purpose to receive n ID, going successively. A C> the Most stupid - to take the list of all ID, to mix them and to save in a file. It will be the order of pair gigabyte (all 676 million variants). If that it too as a matter of fact enciphering, such to itself "an onetime pad" variety.

20

Re: To relieve the person from BigData by means of 5 lines of the code

21

Re: To relieve the person from BigData by means of 5 lines of the code

Hello, ononim, you wrote: S>> Where I was pierced? O> to tell the truth to understand laziness, but know precisely that if you not with a bias in cryptography that better the  not to compose mathematics, and to use that that mathematicians composed. Is not present, etudes let's understand

22

Re: To relieve the person from BigData by means of 5 lines of the code

23

Re: To relieve the person from BigData by means of 5 lines of the code

Hello, Cyberax, you wrote: a C> More interesting decision - to use "Format Preserving Encryption". Standards on it while are not present, but manually is easily enough implemented "cycle-walking". We do the counter in 32 bits, then we take RC5 with the size of the unit in 32 bits, we cipher value and it is done cycle-walking while it is not located in 676 million (on the average 8 steps, nonsense). And as to you my algorithm: http://rsdn.org/forum/etude/6740933.1 the Author: Shmj Date: 29.03 23:34?

24

Re: To relieve the person from BigData by means of 5 lines of the code

Hello, Shmj, you wrote: a C>> More interesting decision - to use "Format Preserving Encryption". Standards on it while are not present, but manually is easily enough implemented "cycle-walking". We do the counter in 32 bits, then we take RC5 with the size of the unit in 32 bits, we cipher value and it is done cycle-walking while it is not located in 676 million (on the average 8 steps, nonsense). S> And as to you my algorithm: http://rsdn.org/forum/etude/6740933.1 the Author: Shmj Date: 29.03 23:34 S>? I congratulate, you invented a network of Fejstelja with HMAC in quality  functions It is quite reasonable approach: https://en.wikipedia.org/wiki/Format-pr … el_network About the firmness, two rounds can be insufficiently so I stupidly would add some more rounds there (simply to repeat process some times). Well and 22 bits are a little not on a statement of the problem as does not cover all key space (29 bits - the closest value).

25

Re: To relieve the person from BigData by means of 5 lines of the code

S>>> Where I was pierced? O>> to tell the truth to understand laziness, but know precisely that if you not with a bias in cryptography that better the  not to compose mathematics, and to use that that mathematicians composed. S> is not present, etudes let's understand Well as kind people identified your method the Author: Cyberax Date: 30.03 00:29, here to you new reading matter, on a subject known plain text attacks to it