If there are integer lists like these:
a_list = [2501, 2783, 3088, 3980, 465, 1001, 39392911, 39394382, 488955,489087, ......]
b_list = [474, 498, 47478821, 47479800, 3774, 8970, 484000, 486000......]
every 2 numbers indicate a range of natural numbers, for example, the ranges of a_list would be:
2501 2783 # 2501,2502,2503,2504,2505,2506,......,2783
3088 3980
465 1001
39392911 39394382
488955 489087
......
For a given number, search for the range where it belongs to, and with priority of a_list > b_list i.e. if a range is found in a_list, stop searching and move on to the next number for searching.
I had test run for searching around 50 numbers which took about 7 minutes. I have a big dataset which could be 20 million numbers need to be searched in his way.
How to code this to do it faster?
============= more conditions and information =============
could be more than 10 thousand numbers in each list.
could be up to 30 million numbers for searching.
the size of list is always n * 2
a_list: [1st < 2nd, 3rd < 4th, ......]
the numbers in the lists might occur more than once.
the priority: a_list > b_list.
I have code as following:
hasFound = 0
if hasFound == 0:
for x, y in izip(*[iter(a_list)]*2): # gives every 2 numbers
if aNumber in range(x,y):
a_list_counter +=1
hasFound = 1
break
if hasFound == 0:
for x, y in izip(*[iter(b_list)]*2):
if aNumber in range(x,y):
b_list_counter += 1
hasFound = 1
break
Many thanks in advance.
解决方案
Toss them all in one big dictionary:
a_list = [2501, 2783, 3088, 3980, 465, 1001, 39392911, 39394382, 488955,489087, ......]
b_list = [474, 498, 47478821, 47479800, 3774, 8970, 484000, 486000......]
# into
ranges = {'a': [2501, 2783, 3088, 3980, 465, 1001, 39392911, 39394382, 488955,489087, ......],
'b': [474, 498, 47478821, 47479800, 3774, 8970, 484000, 486000......]}
Then go through each list in order, mostly the way you were doing it before:
numbers = [list of your target numbers]
scores = {} # dict to store results in
for number in numbers:
for range_name in sorted(ranges):
range_list = ranges[range_name]
groups = zip(*[iter(range_list)] * 2)
if any(start <= number < end for start,end in groups):
scores.setdefault(range_name, 0) += 1
Alternatively (and I'm not sure if this is faster or not) you could do:
for number in numbers:
for range_name in sorted(ranges):
range = ranges[range_name]
if sorted(range + [number]).index(number) % 2:
scores.setdefault(range, 0) += 1
In this case you're throwing a new number into a sorted list, re-sorting it (which is fast using TimSort), and seeing if it falls between two existing numbers.