MinekPo1

MinekPo1 , 6 days ago

if I'm not mistaken , a example of a problem where O(n!²) is the optimal complexity is :

There are n traveling salespeople and n towns . find the path for each salesperson with each salesperson starting out in a unique town , such that the sum d₁ + 2 d₂ + ... + n dₙ is minimised, where n is a positive natural number , dᵢ is the distance traveled by salesperson i and i is any natural number in the range 1 to n inclusive .

pre post edit, I realized you can implement a solution in 2(n!) :(

MinekPo1 , 6 days ago

this would assume that finding the next prime is a linear operation , which is false

MinekPo1 , 6 days ago

unless the problem space includes all possible functions f , function f must itself have a complexity of at least n to use every number from both lists , else we can ignore some elements of either of the lists , therby lowering the complexity below O(n!²)

if the problem space does include all possible functions f , I feel like it will still be faster complexity wise to find what elements the function is dependant on than to assume it depends on every element , therefore either the problem cannot be solved in O(n!²) or it can be solved quicker

MinekPo1 , 6 days ago

honestly I was very suspicious that you could get away with only calling the hash function once per permutation , but I couldn't think how to prove one way or another.

so I implemented it, first in python for prototyping then in c++ for longer runs... well only half of it, ie iterating over permutations and computing the hash, but not doing anything with it. unfortunately my implementation is O(n²) anyway, unsure if there is a way to optimize it, whatever. code

as of writing I have results for lists of n ∈ 1 .. 13 (13 took 18 minutes, 12 took about 1 minute, cant be bothered to run it for longer) and the number of hashes does not follow n! as your reasoning suggests, but closer to n! ⋅ n.

desmos graph showing three graphs, labeled #hashes, n factorial and n factorial times n

link for the desmos page

anyway with your proposed function it doesn't seem to be possible to achieve O(n!²) complexity

also dont be so negative about your own creation. you could write an entire paper about this problem imho and have a problem with your name on it. though I would rather not have to title a paper "complexity of the magic lobster party problem" so yeah

MinekPo1 , 6 days ago (edited 6 days ago)

actually all of my effort was wasted since calculating the hamming distance between two lists of n hashes has a complexity of O(n) not O(1) agh

I realized this right after walking away from my devices from this to eat something :(

edit : you can calculate the hamming distance one element at a time just after rehashing that element so nevermind

MinekPo1 , 5 days ago

you forgot about updating the hashes of items after items which were modified , so while it could be slightly faster than O((n!×n)²) , not by much as my data shows .

in other words , every time you update the first hash you also need to update all the hashes after it , etcetera

so the complexity is O(n×n + n×(n-1)×(n-1)+...+n!×1) , though I dont know how to simplify that

MinekPo1 , 4 days ago

Agh I made a mistake in my code:

if (recalc || numbers[i] != (hashstate[i] & 0xffffffff)) {
	hashstate[i] = hasher.hash(((uint64_t)p << 32) | numbers[i]);
}

Since I decided to pack the hashes and previous number values into a single array and then forgot to actually properly format the values, the hash counts generated by my code were nonsense. Not sure why I did that honestly.

Also, my data analysis was trash, since even with the correct data, which as you noted is in a lineal correlation with n!, my reasoning suggests that its growing faster than it is.

Here is a plot of the incorrect ratios compared to the correct ones, which is the proper analysis and also clearly shows something is wrong.

Desmos graph showing two data sets, one growing linearly labeled incorrect and one converging to e labeled #hashes

Anyway, and this is totally unrelated to me losing an internet argument and not coping well with that, I optimized my solution a lot and turns out its actually faster to only preform the check you are doing once or twice and narrow it down from there. The checks I'm doing are for the last two elements and the midpoint (though I tried moving that about with seemingly no effect ???) with the end check going to a branch without a loop. I'm not exactly sure why, despite the hour or two I spent profiling, though my guess is that it has something to do with caching?

Also FYI I compared performance with -O3 and after modifying your implementation to use sdbm and to actually use the previous hash instead of the previous value (plus misc changes, see patch).

MinekPo1 , 1 month ago

autistic complaining about units

ok so like I don't know if I've ever seen a more confusing use of units . at least you haven't used the p infix instead of the / in bandwith units .

like you used both upper case and lowercase in units but like I can't say if it was intentional or not ? especially as the letter that is uppercased should be uppercased ?

anyway

1Mb

is theoretically correct but you likely ment either one megabyte (1 MB) or one megibyte (MiB) rather than one megabit (1 Mb)

~325mb/s

95mb/s

and

9mb/s

I will presume you did not intend to write ~325 milibits per second , but ~325 megabits per seconds , though if you have used the 333 333 request count as in the segment you quoted , though to be fair op also made a mistake I think , the number they gave should be 3 exabits per second (3 Eb/s) or 380 terabytes per seconds (TB/s) , but that's because they calculated the number of requests you can make from a 1 gigabit (which is what I assume they ment by gbit) wrong , forgetting to account that a byte is 8 bits , you can only make 416 666 of 4 kB (sorry I'm not checking what would happen if they ment kibibytes sorry I underestimated how demanding this would be but I'm to deep in it now so I'm gonna take that cop-out) requests a second , giving 380 terabits per second (380 Tb/s) or 3.04 terabytes per second (3.04 TB/s) , assuming the entire packet is exactly 114 megabytes (114 MB) which is about 108.7 megibytes (108.7 MiB) . so anyway

packet size		theoretical bandwidth
1 Mb	416.7 Gb/s	52.1 GB/s
1 MB	3.3 Tb/s	416.7 GB/s
1 MiB	3.3 Tb/s	416.7 GB/s
300 kb	125.0 Gb/s	15.6 GB/s
300 kB	1000.0 Gb/s	125.0 GB/s
300 kiB	1000.0 Gb/s	125.0 GB/s
30 kb	12.5 Gb/s	1.6 GB/s
30 kB	100.0 Gb/s	12.5 GB/s
30 kiB	100.0 Gb/s	12.5 GB/s

hope that table is ok and all cause im in a rush yeah bye