Getting values with the right type in Redis-CSDN博客

Question:

I'm using redis in my python application to store simple values like counters and time stamp lists, but trying to get a counter and comparing it with a number I came across a problem.

If I do:

import redis
...
myserver = redis.Redis("localhost")
myserver.set('counter', 5)

and then try to get that value like this:

if myserver.get('counter') < 10:
     myserver.incr('counter')

then I get a type error in the if statement because I'm comparing '5' < 10, which means I'm storing an integer value and getting a string one (which can be considered as a different value).

My question is: is this supposed to work like that? I mean its a very basic type, I understand if I have to parse objects but an int? Seems that I'm doing something wrong.

Is there any configuration I'm missing?

Is there any way to make redis return the right type and not always a string? I say this because its the same for lists and datetimes or even floating point values.

Could this be a problem with the redis-py client I'm using and not redis itself?

Answers:

1 . Looks like that is how redis stores its data:

redis 127.0.0.1:6379> set counter 5
OK
redis 127.0.0.1:6379> type counter
string
redis 127.0.0.1:6379> incr counter
(integer) 6
redis 127.0.0.1:6379> type counter
string

If you really want to, you could probably monkeypatch the redis-py client to infer data types.

2. Response callbacks

Technically speaking you need to take care of that on your own.

However, have a look at this link, especially at the part of their README that refers to parsers and response callbacks, maybe that's something you can use. Question would be whether this is an overkill for you or not.

Response callbacks will do the trick. It's not complicate at all, just one line and all will be taken care of.

In [2]: import redis
In [3]: r = redis.Redis()
In [10]: r.set_response_callback('HGET', float)
In [11]: r.hget('myhash', 'field0')
Out[11]: 4.6

for hmget, it returns a list of strings, not one single string, so you need to construct a little more comprehensive callback function:

In [12]: r.set_response_callback('HMGET', lambda l: [float(i) for i in l])

In [13]: r.hmget('myhash', 'field0')
Out[13]: [4.6]

same for hgetall.

3. Depending on needs, we have 3 ways to store data into Redis

String

All data are stored and read back as bytes.
This is the native way method that used by Redis.
We could use redis-cli to inspect and manipulate the data
However, data may not be stored efficiently. For example 12345 is stored as string using at least 5 bytes even it could be represented by a 2-byte unsigned short.
You could limited to use various data structures offered by Redis.
The data stored within Redis is portable across different versions of Python and programming languages.

Pickle

All Python data are converted into binary data using pickle library before stored into Redis.
Data must be converted back to Python data types using pickle library before could be used.
It could model complex and dynamic data structures not offered by Redis.
Once the data is converted back to Python data types, We could use powerful Python to manipulate the data.
It uses more memory within Redis as pickle includes meta data, references in its output.
It is not possible to use redis-cli to manipulate the data, such as INCR.
The data stored within Redis is not portable across different versions of Python and programming languages.

Struct

All Python data are converted into binary data using struct library before stored into Redis.
Data must be converted back to Python data types using struct library before could be used.
It could model complex and static data structures not offered by Redis.
Once the data is converted back to Python data types, We could use powerful Python to manipulate the data.
It uses the least memory out of these 3 methods
It is not possible to use redis-cli to manipulate the data, such as INCR.
The data stored within Redis is portable across different versions of Python and programming languages.

from redis import Redis
import sys
import pickle
import struct

redis = Redis()

# Data
d = [1234, 3.1415, b'Good', True]
k = ['int', 'float', 'byte', 'bool']

## Using String
print(f'String\n{"_"*50}')
# Need to convert True to string or int or float
d2 = [1234, 3.1415, b'Good', int(True)]
redis.hset('k0', mapping=dict(zip(k,d2)))
print(redis.hmget('k0', *k))
print()


## Using Pickle
print(f'Pickle\n{"_"*50}')
packed = pickle.dumps(d)
print(f'Pickled data occupies {sys.getsizeof(packed)} bytes')
print(packed)
redis.set('k1', packed)
print(f'{pickle.loads(redis.get("k1"))}')
print()


## Using Struct
print(f'Struct\n{"_"*50}')
s = struct.Struct('H f 10s ?')
print(f'The structure occupies {s.size} bytes.')
packed = s.pack(*d)
print(f'{packed}')
redis.set('k2', s.pack(*d))
print(f'{s.unpack(redis.get("k2"))}')

Expected output:

String
__________________________________________________
[b'1234', b'3.1415', b'Good', b'1']

Pickle
__________________________________________________
Pickled data occupies 69 bytes
b'\x80\x04\x95\x19\x00\x00\x00\x00\x00\x00\x00]\x94(M\xd2\x04G@\t!\xca\xc0\x83\x12oC\x04Good\x94\x88e.'
[1234, 3.1415, b'Good', True]

Struct
__________________________________________________
The structure occupies 19 bytes.
b'\xd2\x04\x00\x00V\x0eI@Good\x00\x00\x00\x00\x00\x00\x01'
(1234, 3.1414999961853027, b'Good\x00\x00\x00\x00\x00\x00', True)