Docs

reem

class reem.RedisInterface(host='localhost', marshallers=[<reem.marshalling.NumpyMarshaller object>])

initialize()

Call before doing anything with this connection :return:

class reem.KeyValueStore(interface='localhost')

Dictionary Like object used for get/set paradigm

The KeyValueStore object is one that users will frequently use. It keeps Reader and Writer objects for each key that the user has read or written to. It does not actually handle the getting and setting of data but produces KeyAccessor objects that assist with path construction and call the reader and writer’s write and read methods.

interface

Defines the connection to Redis this

Type

str or RedisInterface

reader will use. If a str, then a RedisInterface will be created and

connected to automatically.

get(path)

Reads a path in the form ‘key.element1.element2’ from the redis server. If the path doesn’t exist, an error is raised.

Marginally faster than nested accessors.

set(path, value)

Sets a value in a path in the form ‘key.element1.element2’ from the redis server. If any elements along the path, except for the last key, do not exist, an error is raised.

Marginally faster than nested accessors.

__ensure_key_existence(key)

Ensure that the requested key has a reader and writer associated with it.

Returns: None

track_schema_changes(set_value, keys=None)

Performance optimization for skipping schema update checks

Stop checking for schema updates when setting data. Use ONLY if your data’s schema is static and you are really trying to eek out every bit of optimization.

Parameters

• set_value (according to) – True/False indicating if the keys’ schema should be tracked

• keys (List[str]) – List of keys to track. If None, all present and future keys are tracked

• set_value –

Returns: None

class reem.PublishSpace(interface='localhost')

Convenience class for publishing

This class keeps track of PublishWriter objects for each key the user has published on.

__ensure_key_existence(key)

Identical to KeyValueStore method of same name but does not instantiate a Reader object

get(path)

Reads a path in the form ‘key.element1.element2’ from the redis server. If the path doesn’t exist, an error is raised.

Marginally faster than nested accessors.

set(path, value)

Sets a value in a path in the form ‘key.element1.element2’ from the redis server. If any elements along the path, except for the last key, do not exist, an error is raised.

Marginally faster than nested accessors.

track_schema_changes(set_value, keys=None)

Performance optimization for skipping schema update checks

Stop checking for schema updates when setting data. Use ONLY if your data’s schema is static and you are really trying to eek out every bit of optimization.

Parameters

• set_value (according to) – True/False indicating if the keys’ schema should be tracked

• keys (List[str]) – List of keys to track. If None, all present and future keys are tracked

• set_value –

Returns: None

class reem.CallbackSubscriber(channel, interface, callback_function, kwargs)

Callback Subscriber Implementation

call_user_function(channel, message)

Wrapper callback function (wrapping user function) for this class to work with a RawSubscriber object Fits required interface for a ChannelSubscriber callback function :param channel: channel published to :param message: message that was published :return: None :rtype: None

listen()

Makes this subscriber start listening

Returns: None

update_local_copy(channel, message)

Update the local copy of the data stored under this channel name in redis.

Parameters

• channel – the name of the channel that was published.

• message – message published on that channel

Returns: None

value()

Get data stored at root

Where as the reader can do server["foo"].read() with if server is a KeyValueStore, accessing the root value of a subscriber is not as easy. This method retrieves all data stored underneath a top level key.

Returns: all data stored underneath a top level Redis key.

class reem.SilentSubscriber(channel, interface)

Silent Subscriber Implementation

update_local_copy(channel, message)

Update the local copy of the data stored under this channel name in redis.

Parameters

• channel – the name of the channel that was published.

• message – message published on that channel

Returns: None

listen()

Makes this subscriber start listening

Returns: None

value()

Get data stored at root

Where as the reader can do server["foo"].read() with if server is a KeyValueStore, accessing the root value of a subscriber is not as easy. This method retrieves all data stored underneath a top level key.

Returns: all data stored underneath a top level Redis key.

reem.connection

class reem.connection.RedisInterface(host='localhost', marshallers=[<reem.marshalling.NumpyMarshaller object>])

initialize()

Call before doing anything with this connection :return:

class reem.connection.KeyValueStore(interface='localhost')

Dictionary Like object used for get/set paradigm

The KeyValueStore object is one that users will frequently use. It keeps Reader and Writer objects for each key that the user has read or written to. It does not actually handle the getting and setting of data but produces KeyAccessor objects that assist with path construction and call the reader and writer’s write and read methods.

interface

Defines the connection to Redis this

Type

str or RedisInterface

reader will use. If a str, then a RedisInterface will be created and

connected to automatically.

get(path)

Reads a path in the form ‘key.element1.element2’ from the redis server. If the path doesn’t exist, an error is raised.

Marginally faster than nested accessors.

set(path, value)

Sets a value in a path in the form ‘key.element1.element2’ from the redis server. If any elements along the path, except for the last key, do not exist, an error is raised.

Marginally faster than nested accessors.

__ensure_key_existence(key)

Ensure that the requested key has a reader and writer associated with it.

Returns: None

track_schema_changes(set_value, keys=None)

Performance optimization for skipping schema update checks

Stop checking for schema updates when setting data. Use ONLY if your data’s schema is static and you are really trying to eek out every bit of optimization.

Parameters

• set_value (according to) – True/False indicating if the keys’ schema should be tracked

• keys (List[str]) – List of keys to track. If None, all present and future keys are tracked

• set_value –

Returns: None

class reem.connection.PublishSpace(interface='localhost')

Convenience class for publishing

This class keeps track of PublishWriter objects for each key the user has published on.

__ensure_key_existence(key)

Identical to KeyValueStore method of same name but does not instantiate a Reader object

get(path)

Reads a path in the form ‘key.element1.element2’ from the redis server. If the path doesn’t exist, an error is raised.

Marginally faster than nested accessors.

set(path, value)

Sets a value in a path in the form ‘key.element1.element2’ from the redis server. If any elements along the path, except for the last key, do not exist, an error is raised.

Marginally faster than nested accessors.

track_schema_changes(set_value, keys=None)

Performance optimization for skipping schema update checks

Stop checking for schema updates when setting data. Use ONLY if your data’s schema is static and you are really trying to eek out every bit of optimization.

Parameters

• set_value (according to) – True/False indicating if the keys’ schema should be tracked

• keys (List[str]) – List of keys to track. If None, all present and future keys are tracked

• set_value –

Returns: None

class reem.connection.SilentSubscriber(channel, interface)

Silent Subscriber Implementation

update_local_copy(channel, message)

Update the local copy of the data stored under this channel name in redis.

Parameters

• channel – the name of the channel that was published.

• message – message published on that channel

Returns: None

listen()

Makes this subscriber start listening

Returns: None

value()

Get data stored at root

Where as the reader can do server["foo"].read() with if server is a KeyValueStore, accessing the root value of a subscriber is not as easy. This method retrieves all data stored underneath a top level key.

Returns: all data stored underneath a top level Redis key.

class reem.connection.CallbackSubscriber(channel, interface, callback_function, kwargs)

Callback Subscriber Implementation

call_user_function(channel, message)

Wrapper callback function (wrapping user function) for this class to work with a RawSubscriber object Fits required interface for a ChannelSubscriber callback function :param channel: channel published to :param message: message that was published :return: None :rtype: None

listen()

Makes this subscriber start listening

Returns: None

update_local_copy(channel, message)

Update the local copy of the data stored under this channel name in redis.

Parameters

• channel – the name of the channel that was published.

• message – message published on that channel

Returns: None

value()

Get data stored at root

Where as the reader can do server["foo"].read() with if server is a KeyValueStore, accessing the root value of a subscriber is not as easy. This method retrieves all data stored underneath a top level key.

Returns: all data stored underneath a top level Redis key.

class reem.connection.Writer(top_key_name, interface)

Responsible for setting data inside Redis

The Writer class is an internal class that is used for all data sent to Redis (not including pub/sub messages). Each key that will have nested data below requires a new instantiation of Writer

top_key_name

The name of the Redis key under which JSON data will be stored. To Redis, this will become a ReJSON key name. It is also used to generate the Redis key name that marshallers use to store non JSON data.

Type

str

interface

Defines the connection to Redis this writer will use

Type

RedisInterface

__initialize_metadata()

Pull metadata for this key from Redis or set a default

Returns

None

send_to_redis(set_path, set_value)

Execute equivalent of JSON.SET self.top_key_name <set_path> <set_value>

From the user’s perspective, it executes JSON.SET self.top_key_name <set_path> <set_value> except that set_value can be json-incompatible. In such a case, it determines what non-serializable types are inside set_value, stores the serializable data as a JSON, and stores the non-serializable data using self.interface’s marshallers.

Parameters

• set_path (str) – path underneath JSON key to set

• set_value – value to set

Returns

None

delete_from_redis(del_path)

Execute equivalent of JSON.DEL self.top_key_name <set_path>

Parameters

del_path (str) – path underneath JSON key to delete

Returns

None

__process_metadata(set_path, set_value)

Handle metadata updates

Given the path and value the user would like to set, check if there are non-serializable data types and update metadata locally and in Redis. Happens without pipeline

Parameters

• set_path (str) – path underneath JSON key to set

• set_value – value to set

Returns

None

__publish_non_serializables(set_path, set_value)

Publish JSON incompatible data to Redis

Given a set, publish the non-serializable components to redis, given that metadata has been updated already

Parameters

• set_path (str) – path underneath JSON key to set

• set_value – value to set

Returns

None

__publish_serializables(set_path, set_value)

Publish the serializable portion of set_value

Take out the non-serializable part of set_value and publish it at set_path

Parameters

• set_path (str) – path underneath JSON key to set

• set_value – value to set

Returns: None

class reem.connection.Reader(top_key_name, interface)

Responsible for getting data from Redis

The Reader class is an internal class that is used for all read from Redis (not including pub/sub messages). Each key that will have nested data below requires a new instantiation of Reader

top_key_name

The name of the Redis key under which JSON data is stored

Type

str

interface

Defines the connection to Redis this reader will use

Type

RedisInterface

read_from_redis(read_path)

Read specified path from Redis

This is the only public method of the Reader class. It will retrieve the data stored at a specified path from Redis. At a high level, it reads data stored with ReJSON and inserts non-JSON compatible data at appropriate paths using the metadata associated with this key.

Parameters

read_path (str) – path the user wants to read

Returns: data stored at value in Redis

update_metadata()

Update the local copy of metadata if a relevant path has been updated.

The metadata listener is a redis client subscribed to key-space notifications. If a relevant path is updated, this Reader’s pull_metadata flag will be turned on. If pull_metadata is True, then the reader will fetch metadata from the Redis server.

Returns: None

queue_reads(read_path)

Queue reads in a pipeline

Queue all redis queries necessary to read data at path into the appropriate redis pipeline. First, queue decoded pipeline with the ReJSON query Next, queue all the special path reads with the non-decoded pipeline and marshallers

Parameters

read_path – path user wants to read

Returns: None

build_dictionary(read_path)

Execute pipelines and consolidate data into a dictionary

Parameters

read_path – path user wants to read

Returns: The data stored at read_path in Redis

pull_special_path(read_path)

Directly pull a non-JSON path

If the user specified path is not in JSON, this will retrieve the data directly without going through ReJSON.

Parameters

read_path – path user wants to read

Returns:

class reem.connection.PublishWriter(top_key_name, interface)

Defines PublishWriter behavior

The PublishWriter is identical to the writer but publishes a message when it writes a value.

send_to_redis(set_path, set_value)

PublishWriter equivalent of Writer send_to_redis

This is an equivalent function to Writer’s send_to_redis method but also publishes a message indicating what channel has been updated.

Parameters

• set_path (str) – path underneath JSON key to set

• set_value – value to set

Returns: None

delete_from_redis(del_path)

Execute equivalent of JSON.DEL self.top_key_name <set_path>

Parameters

del_path (str) – path underneath JSON key to delete

Returns

None

reem.marshalling