Hello all,
My program uses many millions of integers, and Python is allocating
way too much memory for these. I can''t have the performance hit by
using disk, so I figured I''d write a C extension to define a new type.
Problem is, my C knowledge is years old and regardless of my
attempts distutils will not recognise my installation of the MS
compiler.
I am thinking, is there any easier way to use a 5 or 6 bit integer
in python? Even a regular 8-bit would be fine. I only need to
represent either 32 or 64 distinct numbers.
thanks,
Glen
解决方案On Fri, 19 Dec 2003 11:42:49 +1100, Glen Wheeler wrote:
Hello all,
My program uses many millions of integers, and Python is allocating
way too much memory for these. I can''t have the performance hit by
using disk, so I figured I''d write a C extension to define a new type.
Problem is, my C knowledge is years old and regardless of my
attempts distutils will not recognise my installation of the MS
compiler.
I am thinking, is there any easier way to use a 5 or 6 bit integer
in python? Even a regular 8-bit would be fine. I only need to
represent either 32 or 64 distinct numbers.
You can store them efficiently in an array, e.g., for unsigned bytes
(or ''b'' in place of ''B'' for signed):
import array
bytes = array.array(''B'', range(10))
bytes
array(''B'', [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]) bytes[3]
3
We can only speculate on further help, until you tell us what you are doing ;-)
Regards,
Bengt Richter
Glen Wheeler writes:My program uses many millions of integers, and Python is allocating
way too much memory for these. I can''t have the performance hit by
using disk, so I figured I''d write a C extension to define a new type.
Problem is, my C knowledge is years old and regardless of my
attempts distutils will not recognise my installation of the MS
compiler.
I am thinking, is there any easier way to use a 5 or 6 bit integer
in python? Even a regular 8-bit would be fine. I only need to
represent either 32 or 64 distinct numbers.
Look at the docs for the array module.
Glen Wheeler wrote:Hello all,
My program uses many millions of integers, and Python is allocating
way too much memory for these. I can''t have the performance hit by
using disk, so I figured I''d write a C extension to define a new type.
Problem is, my C knowledge is years old and regardless of my
attempts distutils will not recognise my installation of the MS
compiler.
I am thinking, is there any easier way to use a 5 or 6 bit integer
in python? Even a regular 8-bit would be fine. I only need to
represent either 32 or 64 distinct numbers.
You''re trying to solve the wrong problem. Python caches small integers, so
you''ve only got a very small number of distinct integer objects. This is a
good thing, because each Python object has a eight byte overhead.
Introducing a new type wouldn''t result in any memory savings.
The real overhead comes from the references to those objects. Each
reference is four bytes (on 32 bit computers), and there is no way to
decrease this. Your best option is to use high level objects that contain
many integers internally but don''t store them as individual Python
references. The array module is one choice, Numarray is another. If
immutability isn''t an issue, you could even use strings (with each character
treated as an eight bit integer).
--
Rainer Deyke - ra*****@eldwood.com - http://eldwood.com